Ideally yes but that would not work with the model latency. But it is already partly covered when a ghost enters its corridor it is asked again. Here's how the decisions work in detail, also below a copy https://github.com/opper-ai/jevman-benchmark/blob/main/READM...
How decisions work
When a character commits to a corridor its next junction is known, so the game asks jev about it straight away (one System One request per frame, one choice question per character, options = legal directions described with computed facts: distances to pellets, power pellets, fruit and ghosts, whether the nearest ghost is coming closer, and whether a ghost can reach the end of the corridor before Pac-Man). Up to three requests are in flight at once. If the character reaches the junction before the answer, it waits there (its panel card says "thinking…"). After 2 s, or on an error, it uses a greedy rule.
Pac-Man can also get a second question mid-corridor: when a dangerous ghost is in the corridor ahead, or can reach the junction at its end before he does, jev is asked whether to keep going or turn back right now (pacman_escape). Pac-Man keeps moving while it is open, and each situation is asked once.
Ideally yes but that would not work with the model latency. But it is already partly covered when a ghost enters pacmans corridor he is asked again. Here's how the decisions work in detail https://github.com/opper-ai/jevman-benchmark/blob/main/READM...
This is neat. At least I'm still better than the models at pac-man.
Also love the idea of a shared pool for users to try things out. I was considering more of a crowdfunded approach for one of my toy projects, something like... Giving it $10 in credits to begin and somehow allowing users to feed a buck in if they wanted.
Great to hear, if it's about inference credits we actually built a thing called an AI wallet where users can just easily pay for their own inference on apps. if that's interesting lmk, id be happy to support this https://opper.ai/ai-wallet
Quick update, we doubled the models tested based on community contributions, and a new model took the #1 spot. also well over 1000 games have been played against decision model ghosts.
https://opper.ai/jevman-benchmark/#leaderboard
Someone actually ran the benchmark on Qwen 3.8 Flash Next NVFP4, which is a chat model and submitted it via PR. We merged and it's live on the leader board in the top 3.
How decisions work
When a character commits to a corridor its next junction is known, so the game asks jev about it straight away (one System One request per frame, one choice question per character, options = legal directions described with computed facts: distances to pellets, power pellets, fruit and ghosts, whether the nearest ghost is coming closer, and whether a ghost can reach the end of the corridor before Pac-Man). Up to three requests are in flight at once. If the character reaches the junction before the answer, it waits there (its panel card says "thinking…"). After 2 s, or on an error, it uses a greedy rule.
Pac-Man can also get a second question mid-corridor: when a dangerous ghost is in the corridor ahead, or can reach the junction at its end before he does, jev is asked whether to keep going or turn back right now (pacman_escape). Pac-Man keeps moving while it is open, and each situation is asked once.
Also love the idea of a shared pool for users to try things out. I was considering more of a crowdfunded approach for one of my toy projects, something like... Giving it $10 in credits to begin and somehow allowing users to feed a buck in if they wanted.
I trained some to do some interesting things, including playing doom: https://github.com/nicobrenner/jeffy
I’ll try training one for this benchmark, seems like fun
I wish the controls were a little easier on mobile.
What even is this reality.