Agents / October 2026
I built an agent that plays Cuphead, then found out why it kept losing
A search-based agent that fights Cuphead bosses from the game's exact internal state, and what it took to learn why it lost.
- Repository
- Wkamfar/cuphead-agent
- Stack
- Python 3.11, C# BepInEx plugin
- Size
- 45k lines Python, 13k C#
- Tests
- 1,956 passing
- Built with
- Claude Code, 7 subagents
- License
- MIT
- 1,020/1,020same-seed replays identical across game processes
- 0.33%of candidate moves called safe would have been hit (held out, 30 frames)
- 52/646whole fights won, every loss in the final phase
- 6 to 39final-phase wins of 100 with two position terms
How it works
A BepInEx plugin runs Cuphead in lockstep with Python and exports every entity's exact position and hitbox. Every two frames the agent simulates 16 short input scripts against a forward model of the hazards, drops the ones predicted to be hit and lets a judge pick from the rest. A slow loop diagnoses each hit afterwards, asks an LLM reviewer for playbook edits, and keeps or reverts them through replay tests and paired statistics.
your copy of Cuphead 1.3.4 cupagent/ (Python; Linux, WSL or Windows)
+---------------------------+ TCP, lockstep +------------------------------------------------+
| mod/CupheadGym | protocol v4 | fast loop, every tick (cupagent/sim, judge, |
| BepInEx 5 plugin: inputs | <---------------> | control): state -> forward model -> maneuver |
| in; exact state, entities,| (docs/SPEC.md) | library -> safety filter -> judge -> executor |
| frames, events, hashes out| +------------------------+-----------------------+
+---------------------------+ | every tick
cupagent/mock: the same protocol, no game v
episode logs and labels (cupagent/data, DATASET.md)
| per batch of attempts
v
slow loop (cupagent/review): hindsight diagnosis ->
clusters -> reviewer model -> playbook edit ->
replay tests + paired gate (cupagent/eval)
|
judges, compared on identical candidate sets:
heuristic | cup_s1 | fine-tuned Laya (cupagent/learn)
|
v
vision-only student (planned; VISION_STUDENT.md)Why it was losing
The infrastructure held up: replays were deterministic and the forward model rarely called a hit safe. The agent still won about 8% of fights, and four rounds of mechanical fixes didn't change that. The logs pointed at position. In the final Carrot phase the agent dealt nearly all its damage from under the boss and took nearly all its hits at the walls. Every time it left the middle, the judge had chosen to, by a small score gap.
Testing the fix
I compared the current judge against six weightings of two position terms, firing_position (fp) and center_bias (cb), each playing the same 100 seeds from the start of the Carrot phase. Every weighting won more often. The strongest won 39 times against 6: 36 seeds were won only with it and 3 only without it (exact McNemar p = 3.6 × 10−8). It is still the best of six on one block of seeds, at the edge of the grid, so it needs confirming on fresh seeds.
What didn't work
The LLM review loop's first run on the real game accepted nothing. Its replays branched after the agent had already left the firing zone, so they tested escapes from positions that were already lost.
- 118hits diagnosed from 40 fights
- 41sent to the reviewer in 3 clusters
- 3edits proposed, all about position
- 3passed the schema check
- 0passed the replay check (60% bar)
Learned judges, trained only on the mock game, learned to copy the hand-written heuristic, because the mock's labels disagreed with it on 0.35% of ticks.
What went well
- Reading ground truth from the game instead of pixels made every test exact and repeatable.
- Versioned data with provenance, and small verified commits.
- Reading the mechanics off the logs before writing code.
- An independent evaluator, which caught a gain that didn't replicate and test seeds that had been worn out.
What didn't
- I followed the milestone checklist instead of asking why it was losing. The position analysis was one script over logs that existed from the first hour.
- Strategy was left to the slowest part of the system, the review loop.
- Testing on the mock game first gave false confidence.
- The milestones measured infrastructure, so they read as passed while the agent won 7% to 8%.
- The orchestration was expensive: one session ran about 24 hours with about 30 subagent runs.
Status and next steps
| Milestone | On the real game | Evidence |
|---|---|---|
| M1 lockstep mod | All eight exit criteria met | successful |
| M2 fast loop | False-safe 0.33% against a 5% criterion | successful as worded |
| M2b win the Carrot phase | Proposed, not met | none |
| M3 dataset tooling | 355 episodes and a one-hour pilot | validated |
| M4 slow loop | One iteration, nothing accepted | implemented |
| M5 learned judges | Mock game only | mock validated |
| M6 vision-only student | Not started | none |
Next: confirm the best weighting on fresh seeds 60400 to 60599, then spend the never-played test seeds once, on whole fights. After that, the goal is a vision-only student that learns from this agent.
View the repositoryFindings reportPost-mortem and plan
Not affiliated with or endorsed by StudioMDHR; Cuphead is their trademark. The repository contains no game files. Project data from the repository at commit 7ebf4e0.