Live universal bot · recurrent PPO / native C inference
ULTRA
A compact recurrent policy trained through exact-engine league self-play, then stripped down to a deterministic C actor that can pilot every SSH Fighter character through one semantic control interface.
WIRE STATE · 30 HZ28-float actor-visible observation25 populated features + 3 reserved zeros
RECURRENT COREGRU · 384 state unitstactical memory within the current round
POLICY READOUT16 logits9 locomotion + 7 combat
FACTORISED ACTION9 × 7 = 63 semantic combinationsindependent argmax heads · facing-relative movement
NATIVE COMPILERsemantic commitment → SSH Fighter inputdeterministic C inference and motion compilation
00 / ACTUAL SYSTEM
The deployed model, separated from the roadmap
One recurrent actor. Seventeen bodies.
Ultra is deliberately smaller and more conventional than TISSUE-0. Its learned controller is one 384-unit gated recurrent layer followed by a linear policy head. Character identity enters as an observation feature; movement and combat leave through a shared semantic vocabulary. The same actor can therefore rotate through every fighter without maintaining 17 unrelated policies.
The current live artifact is ULM2 despite retaining the historical filename ultra.ulm1. It is 1,932,364 bytes and has SHA-256 d34e7eab…02c3d8. The deployed actor has 483,088 parameters. Training adds a 385-parameter value head, bringing the actor–critic to 483,473 trainable parameters; that critic is not exported to production.
Ultra’s design documents describe richer projectile encoders, slower opponent memory, action masks and offline search distillation. Those remain research directions. Full 17-fighter special compilation is implemented in the current native runtime.
01 / OBSERVATION
A narrow actor-visible contract
What the recurrent core can actually see
Facing-canonical relative position and velocity, horizontal distance and vertical separation.
Self health, opponent health and signed health advantage, each normalized to the game scale.
Stun, airborne, active-attack, casting and crouching indicators for the two fighters.
A four-way attack category plus both fighters’ normalized attack-frame counters.
Facing direction and Ultra’s current fighter index across the 17-character roster.
Present in the ULM2 tensor contract but currently written as zero by the production encoder.
The production encoder does not currently read projectiles, acknowledgement history, round timers, hit-stop, previous actions or opponent character identity. It receives only public fighter state; no hidden simulator fields cross the deployment boundary.
02 / RECURRENCE
PyTorch semantics, reproduced by hand in C
The state transition is the policy’s memory
For every observation xₜ, Ultra evaluates reset, update and candidate gates in the same order as a PyTorch GRU. The hidden vector is both the temporal memory and the sole input to the policy readout.
Single-layer GRU
rₜ = σ(Wᵢᵣxₜ + bᵢᵣ + Wₕᵣhₜ₋₁ + bₕᵣ)zₜ = σ(Wᵢzxₜ + bᵢz + Wₕzhₜ₋₁ + bₕz)nₜ = tanh(Wᵢₙxₜ + bᵢₙ + rₜ ⊙ (Wₕₙhₜ₋₁ + bₕₙ))hₜ = (1 − zₜ) ⊙ nₜ + zₜ ⊙ hₜ₋₁Three gates × 384 units × 28 observation features.
Three dense 384 × 384 hidden-state transforms.
2,304 GRU biases and a 384 → 16 linear action head.
The state resets at match start and when a large health restoration indicates a new round. It provides short-horizon tactical memory inside a round, not persistent opponent modelling across rounds.
03 / CONTROL
Factorised semantic action space
Sixteen logits express sixty-three commitments
The first nine logits choose the Cartesian product of horizontal intent—neutral, toward or away—and vertical posture—standing, crouching or jumping. The remaining seven choose no attack, punch, kick, throw or one of three character-relative special slots.
Production takes an independent argmax from each head. A deterministic compiler converts toward and away into absolute left or right from current facing, combines posture and combat, and spells supported special motions. There is no stochastic sampling once weights and recurrent state are fixed.
28 features→GRU 384→9 + 7 logits→two argmax choices→SSH input
The native table now compiles three canonical specials for each of all 17 fighters, including multi-step FDF and FDB motions. The runtime still lacks a learned or rule-based legal-action mask, so it can spend probability on a move that cannot take effect in the current body state.
04 / LEAGUE
Exact-engine recurrent PPO
Training against the policy’s own history
Roll out
Six exact-engine workers collect 70 matches each, producing 420 recurrent trajectories per training iteration. Self and opponent fighters are sampled across the full roster.
Choose rivals
Prioritized fictitious self-play samples recent historical snapshots, the latest snapshot and a pinned champion. Approximate opponent weight is (1 − win rate)² + 0.03.
Estimate credit
The trainer predicts values, computes generalized advantage estimates with γ = 0.997 and λ = 0.95, then normalizes advantages over the batch.
Update
Five recurrent PPO epochs use 200-step truncated sequences, a 0.2 policy clip, clipped value loss, entropy bonus, Adam and a global gradient-norm cap of 1.0.
Snapshot
Every third global iteration becomes a historical opponent. The league retains recent versions so improvement against one checkpoint cannot erase all older pressure.
Challenge live
A candidate must beat the deployed checkpoint and clear a separate fundamentals suite before the service swaps its weight artifact. The outgoing model is archived.
05 / OBJECTIVE
Dense combat signal, sparse outcome signal
Damage shapes tactics; winning closes the loop
Each fight frame rewards dealt damage more strongly than it penalizes received damage and charges a small time cost. A terminal match win contributes the outcome bonus. Timeout-capped trajectories receive a smaller terminal-scale contribution so truncated data can still participate without being treated as a fully observed match.
Rollout reward
rₜ = 0.012 · damage_dealt − 0.008 · damage_received − 0.001r_terminal += won · (1.0 if naturally complete, otherwise 0.4 at the 750-step cap)The learner samples both heads with temperature during data collection. Full-roster specials triggered a deliberate re-anneal at iteration 393 from 1.10 toward 0.85.
The special-expansion curriculum similarly restarted entropy near 0.040 and decays toward 0.005, forcing the policy to revisit newly effective action slots.
Adam starts at approximately 3 × 10⁻⁴ and linearly anneals to a floor of 1 × 10⁻⁴.
Historical opponents act greedily while the candidate samples, keeping the sampled policy likelihood attributable to the learner alone.
06 / PROMOTION
Automated challenger-to-incumbent release
A checkpoint must defeat what is already live
The release loop now evaluates a candidate over 100 paired-seat head-to-head games and separately runs a 40-game fundamentals suite. Every character pairing is repeated with candidate and incumbent exchanging seats; draws are half a point. Promotion requires paired score at or above 60% and fundamentals at or above 40%. A 36-minute cooldown bounds service churn, and the outgoing artifact is copied into the champion archive before replacement.
Current operational gate
- candidate paired H2H ≥ 60% over 100 games
- fundamentals score ≥ 40% over 40 games
- native artifact loads before restart
- outgoing live model retained as champion
- minimum 36 minutes between promotions
- persistent service reconnects to the public queue
The earlier stream of 100% results is retired: candidate always occupied seat A, and self-versus-self incorrectly scored 100%. The repaired invariant is self-versus-self = 50%. Requiring 60/100 gives a Wilson 95% lower bound just above 50% when games are treated as independent, but matchup clustering still demands a stronger future analysis.
07 / EVIDENCE
Claims separated by strength
What Ultra demonstrates—and what it does not
The production path is a single-layer GRU actor trained in PyTorch and evaluated by a handwritten C implementation of the same gate equations. The live artifact contains no Python or Torch runtime.
ajax-bot-ultra is an active resident service. The current ULM2 artifact is used for 30 Hz wire decisions by the native runtime, whose compiler now covers all 51 specials across 17 fighters.
A release-audit invariant exposed seat bias in the legacy evaluator: an identical checkpoint scored 100% against itself. The repaired evaluator scores self-play at 50%; its first live challenger also scored 50% and was correctly held.
Legacy promotion percentages are therefore withdrawn as strength evidence. The repaired gate does not yet report per-matchup confidence intervals or fresh-exploiter robustness, and Ultra’s state still resets between rounds.
The next scientific upgrade is evaluation, not another headline mechanism.
Run at least 10,000 matched games across fighter, seat and opponent slices; publish match-clustered intervals; hold out new exploiters; measure recurrent-state interventions; and require worst-slice floors. That would separate genuine general-purpose control from repeatedly overfitting the latest league checkpoint.
Open the research reports →08 / RUNTIME
A deliberately tiny production boundary
Training is Python. Fighting is C.
receive SSH JSON state at the public actor boundaryencode 28 normalized float32 observation valuesupdate one 384-unit GRU step with ULM2 weightsselect argmax locomotion and combat decisionscompile facing-relative semantics into wire inputsend one JSON action and retain recurrent stateThe binary stores no training graph, optimizer, replay buffer or value head. It reads packed float arrays from ULM2 and performs dense loops, sigmoid and tanh directly. At the latest inspection, the resident process used roughly 5 MB of memory. If model loading fails, the binary can fall back to a heuristic controller; the current service log confirms the ULM2 neural path loaded successfully.
09 / FAILURE MODES
Where the current version is weakest
The dossier is a specification for falsification
A universal fighter cannot be called complete while the live encoder ignores the projectile array. Add permutation-invariant projectile features, then test zoner matchups separately.
All 51 specials compile, but logits are not masked by the current body state. Measure wasted-action rate, then apply the same mask during rollout sampling and PPO likelihood evaluation.
Resetting hidden state protects stability but prevents opponent habits learned in one round from influencing the next. Evaluate a second slow state before adopting it.
Even 100 paired-seat games can be matchup-clustered or schedule-specific. Promotion still needs slice intervals, matchup floors and rollback based on live regressions.
Ultra is currently the pragmatic bot: small enough to train continuously, simple enough to export exactly, and strong enough to serve as a moving baseline. Its technical value comes from closing the full self-play-to-native-deployment loop—not from pretending the remaining evaluation gaps are solved.