Finicky Sawfish

The round-6 regime (gamma 0.997, lambda 0.97, lr 2e-4, entropy 0.01, snapshot 0.5 with pool 20, def_opening drill fading by 50%) has gained Elo five rounds running (+48, +55, +60, +74, +36). The club is now 2nd at 1730. Attacker results improved sharply vs Caiman (30→65) and Classy Cat (18→48). But attacker vs bots-greedy collapsed from 31% to 10%, which met round 10's trigger. The analyst found these are timeouts where our attacker stalls or hovers while greedy parks in a corner. The mirror rarely shows a parked defender, so the attacker never learns to dig one out. I keep hparams, opponents and the def_opening term, so the defender's gains carry over. I add a small telescoping attacker closing term: each step's reduction in distance to the defender, divided by max_dist. Because it telescopes, a game's total is bounded by the starting distance over max_dist, so it cannot saturate the cap. It rewards actually closing, not speed or direction changes, so it should not bring back the jitter of round 2's attacker_stall term. It fades by 50%, leaving match prep on outcome only.

The round-6 regime (gamma 0.997, lambda 0.97, lr 2e-4, entropy 0.01, snapshot 0.5 with pool 20, and a defender opening drill fading by 50%) has now gained Elo four rounds running: +48, +55, +60, +74. The club is at 1694, 2nd place, with no rollback. Last round's warning triggers were not reached. Entropy held at about 0.95 and vf_explained_var recovered to 0.95. The mirror rebalanced toward the defender, with attacker_win_rate falling from 0.79 to 0.33 and episode_len rising to 385. A defender-strong mirror should now give the attacker harder, more evasive practice opponents. That targets the club's main remaining weakness: attacker timeouts against greedy (31%), Caiman (30%) and Classy Cat (18%). The club is still improving and the rollback rule caps the downside, so the plan is to keep every section and just play this round, rather than add a confound.

The round-6 regime has now gained Elo three rounds running: +48, +55, +60, with no rollback, reaching 1619, a club high, 4th place. The club is still improving under these settings, so the plan is to play another round with every section kept. The warning signs from round 8 are real but have not yet hurt results: entropy fell from 1.37 to 0.98, vf_explained_var from 0.88 to 0.74, and snapshot defender win rate from 0.88 to 0.40. Round 3 showed that changing a working setup can collapse it, and the rollback rule caps the downside of keeping. The remaining weaknesses are defender evasion against fast chasers (Finch 6%, Classy Cat 21%, Pegasus 37%) and attacker timeouts vs Cat (13%), greedy (26%) and Caiman (34%). A directed-evasion defender drill is the prepared next step if this round plateaus or the policy keeps narrowing.

The round-6 regime (gamma 0.997, lambda 0.97, lr 2e-4, entropy 0.01, snapshot 0.5 with pool 20, and a defender opening drill fading by 50%) has now gained Elo two rounds running: +48, then +55, with no rollback, reaching 1559, a club high. The club is still improving under these settings, so the best plan is to keep everything for another round rather than add a confound. There are warning signs to watch, not yet to act on: mirror attacker_win_rate drifted from 0.70 to 0.79, entropy fell from 1.41 to 1.30, and vf_explained_var slipped from 0.93 to 0.91. Round 3 showed that changing a working setup can collapse it. The remaining weaknesses are defender evasion against fast chasers (Classy Cat 1%, Finch 5%, Pegasus 21%, Caiman 30%) and attacker timeouts vs greedy, Cat and Finch. If this round plateaus, round 8's data will justify a targeted drill: defender velocity directed away from the attacker, or an attacker closing term.

Round 6's coordinated change (gamma 0.997, lambda 0.97, lr 2e-4, snapshot 0.5 with pool 20, and a defender opening drill) was the first change to fix the structural mirror collapse. In training, attacker_win_rate fell from 1.0 to 0.6, episode_len_mean rose from 124 to 285 and entropy recovered from 1.18 to 1.46. Elo rose +48 with no rollback. The new long-horizon regime has had only one round, and the value function is still adapting (vf_loss 0.019, explained_var 0.92). The best plan is to consolidate: keep every lever the same so the policy can keep learning under balanced mirror and snapshot training. Changing anything now would also make round 6's gain uninterpretable. Remaining weaknesses: defender 0/100 vs Classy Cat and Caiman, and attacker 11/100 vs bots-greedy. Both should improve as the defender keeps learning in a non-collapsed mirror, which gives the attacker harder, less parked practice opponents.

The plateau comes from a structural failure in mirror training: every round the attacker side wins about 100% of mirror games and the defender stops learning. The defender's win arrives only at the 480-step timeout. At gamma 0.99 (horizon about 100 steps) that win is worth almost nothing, so the defender learns only to delay the tag locally and then parks. In the tournament it loses to bots-greedy and Pronghorn (4/100 each). This round makes a bigger, coordinated change aimed at defender credit assignment and opponent diversity. (1) gamma goes from 0.99 to 0.997 and lambda_ from 0.95 to 0.97, so survival to the timeout is visible in the value targets. lr goes down slightly to 2e-4 to keep the longer-horizon targets stable. (2) snapshot_ratio goes to its 0.5 maximum with pool_size 20, so more games are played against varied frozen ancestors instead of a co-collapsing mirror. (3) A defender-only drill rewards each step's gain in distance from the attacker. It telescopes, so it cannot saturate, and round 2's closing term had card_auc 0.99. The attacker gets no drill, since its pursuit already works. The drill fades by 50% of iterations, leaving a long outcome-only phase.

Round 4 (entropy 0.01, snapshot 0.35, pool 10) gained +60 with no rollback, so that baseline stays. The remaining weakness is the defender role. Training ended with mirror attacker_win_rate 1.0, snapshot defender win 0 and episode_len 54, so the defender side stopped learning. In the tournament the defender parks: speed 28 vs league 58, corner_time 0.44 vs 0.18. It loses almost every game to Pegasus (0/100), Roadrunner (1/100) and Magpie (7/100), and wins only 29/100 vs bots-greedy. A defender-only practice drill should teach the defender to run: keep distance, keep moving, stay out of corners and off walls. The drill fades by 60% of iterations, before match prep. The attacker gets zero drill, since round 2's attacker stall term hurt. Hparams and opponents are unchanged so the drill is the only change.

Round 3's rollback shows that pure-outcome mirror training collapsed. Entropy fell from 1.83 to 1.32 and the mirror attacker_win_rate hit 0.97, so the defender side stopped learning and the policy overfit to beating its own collapsing defender. That version lost to almost every real defender and attacker in the league. We restart from the round-2 version. The fix targets the collapse directly. First, double entropy_coeff from 0.005 to 0.01 to keep exploration and stop the policy narrowing. Second, raise snapshot_ratio from 0.2 to 0.35 with a smaller pool (pool_size 10), so the learner faces recent, frozen, stronger versions of itself rather than a co-collapsing mirror. Both changes aim at the same failure mode: loss of diversity and robustness. No drill: past drills either hurt (round 2) or proved irrelevant.

The round-2 Elo drop (1518 to 1488) came mainly from the drill. The attacker_stall term rewarded nothing useful and pushed the attacker toward jittery, slow play: mean_speed fell to 42.7 against a league average of 59, with 5.9 direction changes per second. Round 1 had no drill and the same hparams and opponents, and it gained Elo while training stably. Removing the drill and training on outcome only for all 48 iterations should let PPO recover full-speed straight pursuit, since quick tags in the samples come from fast, straight chases. Hparams and opponents stay unchanged so that this round isolates the drill's effect.

The club's main weakness is the attacker role against stronger, evasive or corner-parking defenders: 10% against Benign Roadrunner, 34% against Flexible Caiman, 43% against Classy Cat. Many of these losses are timeouts where the attacker drifts or stalls. A potential-style drill helps credit assignment for pursuit without changing the win objective. It rewards the attacker for each step's reduction in distance and the defender for each step's gain. A small stall penalty is added for the attacker. Hparams and opponents stay as in round 1, which gained Elo and trained stably, so the drill is the only change.