Flexible Caiman

Six rounds of dense attacker shaping (pursue, hurry, far, no_stall, approach, close_log) have moved training numbers but never tournament attacking: still 0/100 vs greedy, 13 vs Classy Cat, with jitter at 3.98/s against a league 2.79. The drill itself is now the likely bottleneck. Its defender part (alive about 0.08 per episode weighted, plus pinned and no_freeze) rewards passive wall-sitting and teaches the shared network to avoid walls. That spills into the attacker, which won't follow runners onto walls (attacker wall_time 0.34 vs 0.40). Its attacker part rewards speed-toward-target proxies that the outcome alone does not value. Meanwhile entropy collapsed from 1.17 to 0.86 last round, which locks in the hovering optimum. The bigger change is to remove the drill entirely and train on win/loss only for all 48 iterations, about 1.3x the outcome-only time of a normal round. Three changes go with it. Entropy is restored to 0.012 to re-open exploration of committed chases. lr drops slightly to 0.00025 for stability without the drill. The snapshot share rises to 0.5 with a larger pool of 30, so the attacker faces a wider variety of runner versions, including older wall-hugging ones like greedy's style. Mirror attacker win rate is already about 0.45, so the outcome signal for the attacker is dense enough to learn from.

Five rounds of dense attacker terms (pursue, hurry, far, no_stall, approach, steady) were all earned in training but never changed tournament attacking. Our attacker still hovers at 80-120 units, jitters at 4.4 direction changes per second, and scores 0 against greedy and 3 against Classy Cat. None of those terms rewards the decisive last 100 units more than any other stretch: the telescoping close term was worth only about 0.045 per episode. A log-distance potential, close_log = log(prev_d/d) with d floored at 10, has three properties that fit this. It telescopes, so hovering earns nothing and retreating costs. It is heavily concentrated near the target: going from 100 to 10 units is worth as much as going from 300 to 30. It is potential-based, so it does not change the optimal policy, only how fast the policy finds the finish. I drop commit and steady. Commit's anti-brake penalty (about -9 per episode) may discourage the sharp turns needed to cut off wall-huggers. I raise no_stall to break the standoffs. Lower entropy (0.007) should reduce greedy-play jitter. A longer fade (end_frac 0.65) gives the attacker drill more time before match prep. The defender drill is unchanged.

The attacker still hovers: speed 40.5 against a league 62.8, 5.0 direction changes per second, 0/100 against greedy and 2/100 against Classy Cat. Round 8's no_stall term cut stalling in training (about -48 per episode) but did not carry over to the tournament. The pursue term is signed-squared, so at the attacker's typical speed of about 40 it pays almost nothing (p≈0.14, pursue≈0.04). There is no gradient leading from slow creeping to fast committed chasing: only the already-fast regime earns reward, and the policy never reaches it. Replacing it with a linear approach term gives every bit of extra speed toward the defender a proportional reward, starting from zero speed. A new 'steady' term penalises reversing the velocity direction between steps while moving, which targets the jitter that keeps the ball from accelerating. Diagonal commitment toward the defender is the main way to reach top speed. Commit (anti-brake) stays, close stays, and no_stall is kept at a lower weight because the linear approach term now carries that gradient. Defender terms, hparams and opponents are unchanged, so the result can be attributed to the attacker drill.

The attacker is the club's hole: 10/100 vs greedy, 11 vs Classy Cat and Magpie, 14 vs Finch. Our version was rolled back to round 6, so it still has round 6's slow, jittery, hovering attacker. Round 7's per-step 'far' penalty (about -137 per episode before weighting) is the likely culprit. It is dense and does not depend on the action, so it mostly shifted value, and the attacker learned to stand still in standoffs. That fits the analyst's games, where both balls sit fixed for over 1000 ticks. Round 6's flat hurry term had the same flaw and gained only +28. The fix is to make every attacker term depend on the action and target the stall directly. Pursue (signed-squared velocity toward the defender) is kept and raised to 0.003. Close stays as the telescoping finish term, 3x inside 70 units. Commit stays as the anti-braking term. A new no_stall term gives -1 per step when the attacker is slower than 40 with the defender within 150 units. That is the exact behaviour in our losses, and unlike hurry or far it can be escaped by moving. Hurry, far and intercept are all dropped, so the drill stays a small, interpretable change from round 6's structure. Defender terms, hparams and opponents are unchanged, as the analyst advised, to isolate the attacker change.

The attacker is the club's hole: 9/100 against greedy and 14/100 against Classy Cat. It is slow (49 vs a league 62.5), jittery (4.36 vs 2.8 direction changes per second) and parks at a distance while defenders sit on walls. Round 6's flat hurry penalty dominated the attacker drill (about -398 per episode before weighting). Because it was constant, it carried no information about how to close in, so it likely just shifted value and fed the hovering optimum. Pursue also aimed at where the defender is, not where it is going. Against runners like greedy and Classy Cat, chasing the current position makes the attacker re-aim and jitter every step. The new attacker drill drops hurry and has four terms. Intercept rewards fast velocity toward a lead point (the defender's position plus its velocity times a short look-ahead), signed-squared so only committed motion pays. Far is a per-step penalty proportional to distance, so hovering 60-120 units away costs something and creeping in is rewarded. Close stays as the telescoping finish term, 3x inside 70 units. Commit stays as the anti-braking term. The defender terms are unchanged, and hparams and opponents are kept to isolate the attacker change.

Our defender is now strong (88-96 vs most clubs), but the attacker is the club's hole: 0/100 vs greedy, 4/100 vs Magpie and 37 vs Sawfish. It hovers 40-60 units from a defender pinned on a wall or corner, at low speed (46.8 vs league 60.8) and with heavy jitter (5.72 vs 3.07 direction changes per second). Round 5's pursue term was too weak, about 0.05 per episode weighted, against a defender drill worth more. Its normalisation also let slow jitter earn small positive values. Attacker wall_time is low (0.24 vs 0.40): it seems to have learned that being near walls is bad, a carry-over from the shared network's defender pinned penalty, so it won't follow a defender into a wall or corner. The new attacker drill has four parts. Pursue is doubled to 0.0025 and squared-signed, so only fast, committed motion toward the target pays well. The telescoping close term is boosted 3x inside 70 units to reward finishing. A small per-step hurry penalty makes long hovering games cost something. A commit term penalises braking, measured as acceleration opposed to the current velocity, while the target is within 150 units. The defender terms are unchanged, as the analyst advised, so the robust runner is preserved. Hparams and opponents are kept.

Round 4's defender drill made our runner robust: Elo +61, defender wins vs greedy rose from 4 to 73. Two weaknesses remain. First, the attacker does not commit. Against greedy (6/100) and Sawfish (19/100) it jitters at 4.35 direction changes/s against a league 2.16 and moves slowly (49.8 vs 66.5), hovering instead of driving at the defender. The training attacker win rate also collapsed to 0.38. Second, our defender still freezes in corners when the attacker is near, and the alive term (weighted total about 0.13 per episode) dominated the defender drill. The new drill adds an attacker 'pursue' term: each step, our velocity projected onto the direction to the defender, normalised to [-1,1]. This directly rewards fast, committed chasing over jitter. The telescoping close term stays. For the defender, alive is cut to 0.0002 and escape is kept but widened to 160 units. A no_freeze penalty applies when the defender is slow with the attacker within 120 units, and the corner/wall pinned penalty stays. Entropy drops slightly to 0.01 to reduce dithering. Opponents stay at 0.4 snapshots, which helped the defender.

Our weakness against the top clubs is split across both roles. As defender we get pinned in corners and walls by fast, direct chasers (4/100 vs greedy, 1/100 vs Classy Cat). As attacker we hover at 60-90 units without finishing evasive defenders. Round 3's defender terms were too weak in practice: per-episode weighted totals were only about 0.05, 0.02 and -0.035. Two of them also paid even when the chaser was far away, so they never targeted the moment of danger. The new drill concentrates the defender signal on close-range danger. Escape is a telescoping reward for opening distance only when the attacker is within 120 units. Pinned is a per-step penalty for sitting in a corner (-1) or on a wall (-0.5) while the attacker is within 120. Alive is a small per-step survival bonus that directly tracks the defender's winning outcome. The attacker's closing term is doubled inside 120 units to reward finishing over hovering. More snapshot games (0.4) give the defender more varied attackers. Lambda 0.97 with gamma 0.997 should improve long-horizon credit for survival.

The defender freezes because it barely sees its own win: with gamma 0.99, a timeout 480 steps away is discounted to about 0.008, so in mirror play only the attacker's short-horizon tag reward is learnable. That explains the mirror attacker win rate of 1.0, episodes of 67 steps and a frozen defender. Raising gamma to 0.997 (about 0.24 at 480 steps) makes survival worth learning. The new drill targets the defender with per-step terms: keep the opponent far away, stay in open space away from walls and corners, and avoid standing still when the attacker is near. The attacker keeps a small telescoping closing term so its chasing stays sharp against a harder practice partner. A little more entropy also fights the freeze.

Self-play collapsed into attacker dominance (mirror attacker win rate 0.9) with defenders freezing in corners, so our attacker never learned to finish a moving or well-placed defender (2/100 vs Sawfish and Pegasus) and our defender stalls. A distance drill helps both roles: attackers are rewarded for closing distance, defenders for opening it, plus a small mobility bonus for defenders. The distance term telescopes over a game, so its total is bounded. With slightly more entropy and more snapshot games, the defender should toughen up and give the attacker a harder practice partner.