Capstone: Build the Lab

01A contest, day by day

Back Track King, CodinGame's Summer Challenge 2026, ran for two weeks, from September 7 to 21.^[1] In that window you write a first bot, port the rules, write a search, measure it, perhaps train a network, and submit, while the leagues above you open on dates set in advance and every other player improves at the same time. The lab is the repository and the routine that keep those two weeks from turning into a pile of half-remembered experiments.

Most of the lab gets built in the first days of a contest, and most of it carries over to the next one. The table maps a two-week contest onto the sixteen steps in section 03. It's one schedule that fits an event like Back Track King; a shorter contest squeezes the middle rows.

Machine learning has no row in that table. A network needs a verified fast engine, a trainer and an exporter before its first useful training run, and then hours to days of compute (section 07), so in a first contest the search path is the one that fits. The lab's own Back Track King results show that order.

02One repository, one job per folder

A week into a contest, the questions that cost the most time are about where things came from. Which file was live on Tuesday? Which maps produced the 58% you wrote down yesterday? Was the engine you trained on the version you verified? A layout that gives every artifact one home answers each of those with a path.

The tree below mirrors the author's lab, with a light-cycle game as the example.^[3] The middle column says what each folder holds, and the right column links the page that teaches it. The Rust side is a Cargo workspace: one top-level file that builds every Rust crate in the repository together.

my-lab/
├── Cargo.toml              Rust workspace: every crate below              Trainer
├── lab, train.sh           one command line, one training entry point     Trainer
├── crates/
│   ├── cg-env/             Game and Agent contract, batched simulator     Referees
│   └── cg-nn/              network inference the exported bot includes    Shipping
├── framework/rl/           trainer, cloning, export, verify, live tools   Trainer
├── games/
│   ├── registry.json       one entry per game: puzzle id, run, engine     Section 10
│   └── tron/               one folder per game
│       ├── rules/          statement per league, protocol, time limits    First bot
│       ├── engine/         your port of the referee's rules               Referees
│       ├── bots/baseline/  the contest bot, frozen byte for byte          Section 04
│       ├── bots/search/    search bots and variants, each a new name      Search
│       └── rl.py           what the trainer needs to know about the game  Section 07
├── third_party/
│   ├── sources.lock.json   pinned referee and tool commits                Section 04
│   ├── checkouts/          ignored: referee and match-runner clones
│   └── artifacts/          ignored: JDK, built referees and bots
├── experiments/
│   ├── seed-ledger.json    seed ranges: train, validation, final          Section 05
│   ├── <name>-<date>.json  one tracked summary per experiment             Measuring
│   ├── live-submissions/   submitted.jsonl, battles-<id>.jsonl            Section 06
│   └── runs/               ignored: checkpoints, logs, exports
├── docs/provenance.md      every borrowed idea and where it came from     Leaderboard
└── tests/                  engine, parity and harness tests               Referees

Three rules keep the layout honest.

• Track sources and small records; ignore what you can regenerate. Checkpoints, logs, built binaries and referee checkouts live under ignored folders. Experiment summaries, the seed ledger and the submission records are a few kilobytes each, and they go into Git.
• Keep secrets out of the tree. The API token lives in your home directory (section 06), not in an ignored file inside the repository, where one git add -f would publish it.
• Never edit a frozen file. The contest bot under bots/baseline/ is the yardstick for every later comparison and the teacher for behavior cloning. A changed version gets a new name.

# .gitignore: build output and Python environments
target/
.venv/
__pycache__/
# large experiment data you can regenerate: checkpoints, logs, exports
experiments/runs/
experiments/datasets/
# third-party clones and built tools, pinned in third_party/sources.lock.json
third_party/checkouts/
third_party/artifacts/
# local environment files
.env
.env.*

03Sixteen steps from statement to Legend

Each step below builds one thing and ends with a check you can run. Don't start a step until the one before it passes. A search tuned on an unverified engine, or a network measured by a broken harness, produces numbers that look like progress and mean nothing.

What do you actually need to do this by hand? For a search bot, three things: a simulator proved against the official referee, a measurement harness that plays on maps you didn't tune on, and the search itself. That's steps 1 to 9, the path that took five of the lab's games to Legend. For a network on top, you add the network, a trainer and an exporter with its checks, which is steps 10 to 14. The rest of this page covers the records and the routine that let you trust what those pieces report.

1. Repository. A Cargo workspace with the engine contract and network crates, the training framework, one folder per game and a registry listing them, pinned third-party sources, tracked experiment summaries, tests, one command-line entry point, and a .gitignore for everything large. Done whengit status is clean and nothing large or secret is tracked.WhereSection 02
2. Freeze your bot. Copy the contest submission byte for byte to games/<id>/bots/baseline/original.<ext>, record its SHA-256, and never edit the copy. Done whenthe hash matches the code CodinGame holds for that submission.WhereSection 04
3. Rules on disk. The statement of each league you reach, saved as its own file, with notes on the protocol and the time limit per turn. Done whenevery input and output line is described.WhereYour First Bot
4. The official referee, locally. Clone the referee, pin its commit, build it, and play your bot against itself through a match runner in both seats on fixed seeds. Done when100 seeded games finish with no crash and their logs are saved.WhereReferees and Simulators
5. Engine port. Your Game (full state, the lines each seat receives, step(commands), scores) and Agent (read one seat's input, observe, print a command), a fuzz command generator, and the referee's random generator and map generation from a seed. Done whenit compiles, and the same seed and commands always give the same frames.WhereReferees and Simulators
6. Prove the engine. Record referee games between fuzz agents, replay their commands in your engine and compare every frame. Done whenhundreds of games replay with zero mismatches, or each mismatch is explained as the referee's own nondeterminism.WhereReferees and Simulators
7. Measurement harness. Native bots running as processes inside your simulator, matches on validation seeds in both seats, Wilson intervals, a seed ledger, and a panel of distinct opponents. Done whenyour bot against a copy of itself measures .5 with draws counted as half, and you know whether your game mirrors to 0–0.WhereMeasuring Bot Strength
8. Search bot. The search that fits the game, in your bot's language, tuned on validation seeds, tested once on fresh ones, then played against the boss in live test games. Done whenit beats your frozen bot on fresh seeds outside the interval, and beats the boss live.WhereMinimax, MCTS, Evolution, Fast Bots
9. Submit and record. Submit, append the source hash and submission id to the game's submitted.jsonl, and save the ranking battles once they reach 100%. Done whenthe league, rank and score are recorded with the battle list.WhereHow Bot Contests Work, Section 06
10. Decide on learning. Count the choices per turn, read what the top players did, and estimate the cost in days and compute. Done whenyou have a written decision with reasons.WhereWhen to Use Machine Learning
11. Network and export smoke test. Observation buffers and action decoding in your Agent, a network family sized for the 100,000-character limit, and an export of an untrained network played on the referee. Done whenthe untrained export fits, compiles, decides like the Python model turn for turn and answers inside the time limit.WhereNetworks for Game Bots, Shipping a Network
12. Behavior cloning. Demonstrations recorded from your native bots through the protocol, a cross-entropy training run, and agreement measured on held-out games. Done whenthe clone scores close to its teacher on validation seeds.WhereBehavior Cloning
13. Self-play. Batched rollouts, GAE and PPO, an opponent pool with a share of native bots, a champion gate, weight averaging, quantization-aware training, checkpoints and a status file, run overnight. Done whenpromotions happen and the scores against your frozen bots rise.WhereSelf-Play, Building the Trainer
14. Export and gate. Quantize, compress and pack the weights, assemble one file, compile it as the platform would, and run the parity, smoke-game, size and timing checks. Done whenevery gate passes and the report is saved with hashes.WhereShipping a Network, Section 07
15. The live loop. A live panel written down before you look, a submission only if it passes, the battles recorded, losses replayed through your engine, then the next hypothesis. Done wheneach decision has a record, and a worse submission gets reverted.WhereMeasuring, Learning from the Leaderboard, Section 08
16. Next contest. A new games/<id>/ folder with its engine, game spec and registry entry, then steps 2 to 15 again, reusing everything else. Done whenthe new game's engine passes step 6.WhereSection 10

04Freeze, pin and prove

Steps 2 to 6 are the ones that get skipped when a contest is moving fast, and every later number depends on them. If the frozen bot drifts, every comparison against it drifts too. If your engine disagrees with the referee on some turns, your search and your trainer both get good at a game nobody else is playing.

Freezing means a byte-for-byte copy plus a hash. A SHA-256 hash is a 64-character fingerprint computed from a file's bytes: change one character and the fingerprint changes completely, so a recorded hash proves later that a file is the one you submitted. Section 06 shows how to fetch the code CodinGame holds for a submission and compare the two hashes.

mkdir -p games/tron/bots/baseline
cp ~/contest/bot.rs games/tron/bots/baseline/original.rs   # the exact file you submitted
sha256sum games/tron/bots/baseline/original.rs             # write this hash into provenance.json
chmod a-w games/tron/bots/baseline/original.rs             # changed versions get new names

The lab records each frozen bot in a small provenance.json beside it. Its Back Track King entry, trimmed:^[3]

{
  "original_file": "original.rs",
  "original_bytes": 65978,
  "original_sha256": "4a4231b1b34ed97fb476181cba8aac6e37d22c5c20aaf1178b0a9d387a3c78af",
  "strategy_changes": false
}

Rules on disk. Save the statement of every league you reach in rules/, one file per league. A promotion can bring new inputs and actions, and CodinGame's league documentation tells players to read the statement again after each one.^[4] The API in section 06 returns your current league's statement as Markdown, so saving it is one command.

The referee, pinned. When a game's referee is published, clone it and write down the exact commit you built, because the repository can change after you clone it. The lab pins every third-party source in third_party/sources.lock.json. This is its entry for the official Back Track King referee:^[3]

{
  "name": "official",
  "repository": "https://github.com/CGjupoulton/SummerChallenge2026",
  "commit": "23b2a02781d02751b723432865889fbc5dd75e35",
  "checked_on": "2026-10-10"
}

A match runner plays many games between bots through the referee and tallies the results. cg-brutaltester, a community tool, takes the referee's command, up to four players, a thread count, a game count and a log folder; -s replays each map with the seats swapped, and -i fixes the starting seed of the run so it can be repeated.^[5] Referees built on CodinGame's current SDK need a small command-line class added before cg-brutaltester can drive them, and the tool's README still lists timeout handling among its incoming features, so don't count on it to catch a bot that answers too slowly. Time your bot separately. For SDK referees, the SDK's own MultiplayerGameRunner can also play headless games on a fixed seed through its simulate() method.^[6]

java -jar third_party/artifacts/cg-brutaltester.jar \
  -r "java -jar third_party/artifacts/referee.jar" \
  -p1 ./target/release/original -p2 ./target/release/original \
  -t 3 -n 100 -s -i 20261030 -l logs/self-play/   # 3 threads, 100 games, swap seats, fixed seed

The engine contract. Your engine is your own port of the referee's rules, and the lab splits it into two halves.^[3] Game is the referee side: it holds the full state, produces the lines each seat receives and applies both players' commands at once. Agent is the bot side: it reads one seat's input and turns a decision into a command line. The same Agent code runs inside the simulator and inside the submitted bot, so an observation can only use what that seat was actually sent, which rules out leaks of hidden state. The signatures, trimmed from the lab's crates/cg-env/src/lib.rs:

/// One seat's view of the game: parsing, observation and command formatting.
pub trait Agent: Sized {
    /// Per-thread scratch memory for `observe` (path searches and the like).
    type Scratch: Default + Send;
    fn spec() -> Spec;
    /// Reads the initialization input.
    fn read_init(input: &mut dyn LineSource) -> Self;
    /// Reads one turn's input and updates any memory.
    fn read_frame(&mut self, input: &mut dyn LineSource);
    /// Writes this turn's observation. Buffers arrive zeroed.
    fn observe(&mut self, scratch: &mut Self::Scratch, out: &mut Out);
    /// The command line for an action vector of length `spec().action_len`.
    fn command(&mut self, action: &[i32]) -> String;
    /// A random but well-formed command, used to verify the engine against the referee.
    fn fuzz(&mut self, rng: &mut SplitMix64) -> String;
}

/// The referee: full game state and simultaneous turn resolution.
pub trait Game: Sized + Send + Clone {
    /// What a new game is built from: usually a seed for the official map generator.
    type Scenario: Send + Sync;
    fn scenario_from_seed(seed: u64) -> Self::Scenario;
    fn new(scenario: &Self::Scenario) -> Self;
    fn init_lines(&self, seat: usize) -> Vec<String>;
    fn frame_lines(&self, seat: usize) -> Vec<String>;
    /// Applies both commands. Returns the number of rejected command parts per seat.
    fn step(&mut self, commands: [&str; 2]) -> [u32; 2];
    fn done(&self) -> bool;
    /// The scores CodinGame ranks by (after any tie-break the referee applies).
    fn scores(&self) -> [i32; 2];
    /// Turns played so far.
    fn turn(&self) -> u32;
}

Proving the engine. A fuzz agent prints random but well-formed commands, which reach corners of the rules your own bot never visits. Record games between two fuzz agents on the official referee, keeping what each seat received and printed every turn. Then start your engine from the same seed, feed it the same commands and compare every frame. The first frame that differs names the rule you got wrong; Referees and Simulators walks through one. A mismatch can also come from the referee itself, through nondeterminism: when wind pushes several monsters out of a base in one turn, Spring Challenge 2022's referee gives each a random new direction in an order set by Java identity hash codes, which change from run to run.^[3] Explain each such case in writing, or keep looking.

05Records that keep you honest

You tune a search on 200 maps, the new version wins 58% of them, and the next morning you test it again on the same maps to be sure. Two deterministic bots, ones that always make the same move in the same position, replay the same games on the same seeds, so the new version wins 58% again and you've learned nothing. The fix is bookkeeping: every result carries the seeds, seats, opponents, code and engine that produced it, and the seeds come from ranges reserved in advance.

The reservation file is the seed ledger. It splits the seeds into three kinds: training seeds the trainer may use, validation seeds for screens and tuning, and final-test seeds that are used once and retired. Each experiment takes a fresh range and writes down what took it and when. An example ledger:

{
  "train": {"first": 10000000, "count": 10000000},
  "validation": [
    {"first": 3500000, "count": 64,
     "used_by": "search-r3-screen", "date": "2026-10-29"}
  ],
  "final": [
    {"first": 2454096, "count": 256,
     "used_by": "net-3350-final", "date": "2026-10-14"}
  ]
}

Not every game's map comes from a single seed. The map is whatever a game log's options line names: Tron Battle's holds the players' start positions, and Mad Pod Racing's holds a seed plus the checkpoint layout.^[7] Whatever that line holds is what the ledger reserves.

The same rule applies inside a trainer. JOiy, who took a self-play network to Legend in Back Track King, wrote that his promotion gate first ran on "the same 20 000-map database training drew from. A checkpoint can clear that gate by memorising boards." His fix held out the last 2,000 maps for every gate and benchmark.^[8]

Each experiment then gets one small JSON summary under experiments/, named for what it tested and its date. It holds the repository commit and the SHA-256 of every bot that played; the seed range from the ledger, with both seats on every map; the opponents, the number of games and the result with its interval; and the decision you took, with the reason.

Live results get the same treatment as lines in JSONL files that only ever grow. CodinGame's API keeps no score history per submission, and its battle list has positions but no scores or seeds,^[9] so a record you don't write yourself is gone. The lab keeps a submitted.jsonl and a rankings.jsonl per game, plus one battles-<submission>.jsonl per submission. A trimmed line from its Tron Battle record:^[2]

{"submission_id": 12345678, "submitted": "2026-10-29T00:00:00", "puzzle": "tron-battle",
 "file": "games/tronbattle/bots/search/<bot source>", "bytes": 27361,
 "sha256": "<SHA-256 of the file>",
 "language": "<CodinGame language name>", "upload_verified": true}

The full line also carries a note with the local and live results that justified the submission. upload_verified means the hash of the code CodinGame returned for the submission matched the file's.

06Drive CodinGame from scripts

A live panel of 40 pairs of test games, a submission with its upload checked, 150 ranking battles saved before you forget which bot played them: clicking through the website for each of those is slow, and it leaves no record. CodinGame has a public API for test games, submissions, rankings and battle logs, documented in plain Markdown that you can read without an account.^[10]

Every call except a few public listings needs an access token. You get one through the OAuth device flow: your script asks the server for a device code, you approve a short user code in your own browser while signed in to CodinGame, and the script keeps asking until the server hands over the token.^[11] The approval page lets you pick the token's lifetime (two years by default), and you can revoke it later from your account's token settings.^[9] A token carries scopes; by default it can read your results, start test games and submit.

mkdir -p ~/.config/codingame && chmod 700 ~/.config/codingame
# client_name is what the approval page shows you.
curl -s -X POST https://www.codingame.com/api/v1/auth/device \
  -d client_name='my-bot-lab' > ~/.config/codingame/device.json
# Show only the code to check and the page to open in your browser.
python3 -c 'import json, sys; d = json.load(open(sys.argv[1])); print(d["user_code"], d["verification_uri_complete"])' \
  ~/.config/codingame/device.json

Open that page, sign in, check that the code matches and accept. The device code expires after 15 minutes.^[9] Meanwhile, the next block polls every five seconds; the response that finally carries the token goes straight into the file and is never echoed. Both blocks follow the device-flow commands in CodinGame's skill documentation, with Python reading the JSON.^[9]

device_code=$(python3 -c 'import json, sys; print(json.load(open(sys.argv[1]))["device_code"])' \
  ~/.config/codingame/device.json)
interval=5
while true; do
  sleep "$interval"
  reply=$(curl -s -X POST https://www.codingame.com/api/v1/auth/token \
    -d grant_type=urn:ietf:params:oauth:grant-type:device_code -d device_code="$device_code")
  case "$reply" in
    *'"access_token"'*)   # into a file only you can read; never echoed
      (umask 077; printf '%s' "$reply" \
        | python3 -c 'import json, sys; print(json.load(sys.stdin)["access_token"], end="")' \
        > ~/.config/codingame/token)
      rm ~/.config/codingame/device.json; echo 'Token saved.'; break ;;
    *authorization_pending*) ;;                  # not approved yet: ask again
    *slow_down*) interval=$((interval + 5)) ;;   # asking too often: back off
    *) printf '%s\n' "$reply"; break ;;          # refused or expired: the reply says which
  esac
done

cat > .git/hooks/pre-commit <<'HOOK'
#!/bin/sh
# Refuse any commit that adds a CodinGame API token. Only added lines count,
# so a commit that deletes a leaked token still goes through.
if git diff --cached -U0 | grep '^+' | grep -Eq 'cgp_[A-Za-z0-9]{42}'; then
  echo 'pre-commit: a CodinGame API token is staged; remove it before committing' >&2
  exit 1
fi
HOOK
chmod +x .git/hooks/pre-commit

The rest of the commands share a small helper that reads the token from its file on every call, plus two variables. A permanent puzzle lives under puzzles/<id>; during a live contest the same calls go under challenges/<id>.^[10] A test game takes the map from an earlier log's options line, and your code always plays as agent 0, with the opponents after it in order.

cg() {   # every call reads the token from its file; nothing here prints it
  curl -s -H "Authorization: Bearer $(cat ~/.config/codingame/token)" \
       -H 'Content-Type: application/json' "$@"
}
API=https://www.codingame.com/api/v1
GAME=puzzles/tron-battle             # during a contest: challenges/<public_id>
OPTIONS='(16,10)(24,9)'              # copied from a log's options line: Tron start positions

python3 - "$OPTIONS" > /tmp/test-game.json <<'PY'
import json, sys
code = open('games/tron/bots/search/search.rs', encoding='utf-8').read()
print(json.dumps({'language': 'Rust', 'code': code, 'opponents': ['boss'], 'game_options': sys.argv[1]}))
PY
id=$(cg -X POST "$API/$GAME/test-games" --data @/tmp/test-game.json \
     | python3 -c 'import json, sys; print(json.load(sys.stdin)["test_game_id"])')
mkdir -p logs
while true; do   # each poll waits up to 30 s for the game to end, for one request
  cg "$API/test-games/$id?wait=30" > "logs/test-$id.txt"
  grep -q '^status: running' "logs/test-$id.txt" || break
done
head -n 3 "logs/test-$id.txt"   # "status: done" and the log, or "status: failed" with the reason

Test games and submissions are rationed per user over sliding time windows. The API documentation gives typical values and says the windows depend on the game:^[10]

Over a limit, the API answers with status 429 and a Retry-After header. Wait that long: an earlier retry is rejected again.

Submitting follows the same pattern. Check the upload by asking for the code CodinGame now holds and comparing its hash with your file's, then append the record.

file=games/tron/bots/search/search.rs
python3 -c 'import json, sys; print(json.dumps({"language": "Rust", "code": open(sys.argv[1], encoding="utf-8").read()}))' \
  "$file" > /tmp/submit.json
submission=$(cg -X POST "$API/$GAME/submissions" --data @/tmp/submit.json \
  | python3 -c 'import json, sys; print(json.load(sys.stdin)["submission_id"])')
# The two hashes must match: the code CodinGame holds, and your file.
cg "$API/$GAME/submissions/$submission" \
  | python3 -c 'import hashlib, json, sys; print(hashlib.sha256(json.load(sys.stdin)["code"].encode()).hexdigest())'
sha256sum "$file"
# One line per submission, appended, never edited.
mkdir -p experiments/live-submissions/tron-battle
python3 - "$submission" "$file" 'search r3: endgame fill; passed live panel 29-11' \
  >> experiments/live-submissions/tron-battle/submitted.jsonl <<'PY'
import datetime, hashlib, json, subprocess, sys
submission, path, note = sys.argv[1:4]
code = open(path, encoding='utf-8').read()
print(json.dumps({
    'submission_id': int(submission),
    'submitted': datetime.datetime.now().isoformat(timespec='seconds'),
    'file': path,
    'sha256': hashlib.sha256(code.encode()).hexdigest(),
    'characters': len(code),
    'commit': subprocess.run(['git', 'rev-parse', 'HEAD'], capture_output=True, text=True).stdout.strip(),
    'note': note,
}))
PY

Ranking takes a while after a submission. The progress call reports the share of ranking games played, the score and rank, and the league boss's score; the numbers are final at 100%. Then save the battle list, 100 games per page, and fetch the log of any game you want to replay. A log holds the options line and every bot's printed commands, which is what your engine needs to rebuild the game (Learning from the Leaderboard). Ranking logs carry no stderr, only your own test games do.^[10] Fetching 150 logs is 150 requests, more than one minute's window, so pace them.

cg "$API/$GAME/progress"   # percentage, score, league_rank, league_boss; final at 100

# At 100%: every ranking game of the submission, newest first, one JSON line each.
out=experiments/live-submissions/tron-battle/battles-$submission.jsonl
before=''
while true; do
  cg "$API/$GAME/battles?submission=$submission&limit=100${before:+&before=$before}" > /tmp/page.json
  python3 -c 'import json, sys; [print(json.dumps(b)) for b in json.load(open(sys.argv[1]))["battles"]]' \
    /tmp/page.json >> "$out"
  before=$(python3 -c 'import json, sys; print(json.load(open(sys.argv[1])).get("next_before", ""))' /tmp/page.json)
  [ -n "$before" ] || break
done

cg "$API/games/913337323?format=json" > logs/battle-913337323.json   # options, agents, every bot's output

07Add a network without breaking the lab

Steps 10 to 14 add a training pipeline to a lab that already works, and done in that order they change nothing you built for the search path. The engine you proved becomes the trainer's simulator. Your search and hand-written bots become the network's teachers and its yardsticks, and the harness judges the network by the same rules as every other bot.

Step 10 comes first for a reason: When to Use Machine Learning shows how to tell from a game's structure whether a network is likely to beat a search, and the lab has a clear case of the answer being no. In Ultimate Tic-Tac-Toe its trained network scored 0 of 64 against its Monte Carlo tree search bot.^[12]

One file per game for the trainer. Everything the shared trainer needs to know about a game fits in games/<id>/rl.py: where the engine library is, which seeds are for training and which for validation, the network, the reward, the native opponents, the cloning setup, and what the exported bot compiles. A trimmed copy of the lab's Ultimate Tic-Tac-Toe spec, with comments added:^[3]

SPEC = GameSpec(
    id='tictactoe', title='Ultimate Tic-Tac-Toe',
    crate='tictactoe-engine', library='libtictactoe_engine.so',  # the Rust engine behind the C ABI
    train_scenarios={'first': 10_000_000, 'count': 10_000_000},  # seeds the trainer may use
    valid_scenarios={'first': 3_500_000, 'count': 100_000},      # seeds for gates; never trained on
    build_model=nets.build, model_config=families.grid_config, decode=families.decode,
    reward_modes=('boards', 'terminal'), defaults=dict(reward='terminal', horizon=32),
    native_opponents=native_opponents,                           # protocol bots for benchmarks
    bc=BCSpec(pairings=[('original', 'original')], games=256),   # cloning demonstrations
    bot_agent='crate::agent::TicTacToeAgent', time_limit_ms=100, # what the exported bot compiles
)

One file you can run until you stop it. train.sh builds what it needs, resumes the run if one exists, and trains until you press Ctrl+C. Ctrl+C asks it to stop instead of killing it: it finishes the iteration in progress, saves the latest and the champion checkpoints, then exports the champion and checks the export before it exits. Saving takes a moment; the checks take minutes, because they replay thousands of turns and play games on the official referee. The header of the lab's script, trimmed:^[3]

#!/usr/bin/env bash
# Default: GPU self-play PPO on the bit-exact Rust simulator (builds, prepares and resumes).
#   ./train.sh                 run/resume overnight training (Ctrl+C saves and exports)
#   ./train.sh --hours 9       same, stopping on its own after nine hours
#   ./train.sh <game> [...]    another registered game (games/registry.json), e.g. trollfarm;
#                              same options (--hours, trainer flags); see docs/multi-game.md

Gates on the export. An export is a new program, so it gets its own gates before anything submits it: the source must fit the character limit, compile as the platform would compile it, decide like the trained model turn by turn (the parity check), finish a few smoke games on the official referee, and answer inside the time limit. Shipping a Network covers each one. The same export report shows what passing looks like: 56,947 characters with Unicode packing, 95,021 in plain ASCII, both under the 100,000 limit; zero parity mismatches with a tie tolerance of 0.0001 logits; and a slowest turn after the first of 17.7 ms against the 50 ms limit (the first turn gets 1,000 ms).^[7]

Submitting while training runs. You don't have to stop training to submit. The trainer keeps the current champion in its own checkpoint file, so you copy that file, export from the copy while training continues, run the gates, submit the file that passed and record it. The checks compete with the trainer for CPU time, so expect them to run slower while it's busy. The lab's Spring Challenge 2022 network went to the ladder this way: champion 99,250 was exported from a copy at 17:37 on October 30, passed its export gates and was submitted about two minutes later, while its run kept training.^[12]

08From a lost game to the next submission

Once a bot is live, the work turns into a loop. A lost battle suggests a weakness, the weakness suggests a change, and the change has to prove itself twice, once on your machine and once on the ladder, before it replaces what's live. Every lap ends with something written down, whether or not anything was submitted.

1. Diagnose. Download the battles you lost, rebuild each one in your engine from its options line and both players' printed commands, and find the turn where it turned. Learning from the Leaderboard shows how to read them.
2. Hypothesis. One change you can test, written down before you test it. "The endgame fill misjudges regions with a single exit" is a hypothesis; "make it smarter" isn't.
3. Local panel. The candidate against the live bot and a panel of distinct opponents, on fresh validation seeds from the ledger, in both seats. It moves on only if it beats the live bot outside the interval without losing ground against the rest of the panel.
4. Live panel. Test games on CodinGame against the players you need to beat: the candidate and the live bot play the same opponents on the same seeds, and the pass rule is written down before the first game. That is a live panel, and writing the rule first is what pre-registered means.
5. Submit. Submit a candidate only if it passed both panels.
6. Record. The submission line, the battles once ranking reaches 100%, the final rank and score. If the new bot finishes worse than the one it replaced by more than the noise of one submission, resubmit the old file; the record says which one it was.

The live panel exists because local stand-ins for other players' bots can be wrong in ways your own panel can't see. The lab's records have one case of each outcome.

The loop is the end of a chain of gates that starts at the commit. Each gate looks for one kind of defect, and each kind is caught by different gates. A size check catches an oversized file every time; only games against other bots can show that a candidate is weaker, and games are noisy. The widget is a model of that chain, with defect and catch rates chosen for illustration; none of them was measured.

How often should you make a new baseline? Freeze a bot as the new baseline when a live result confirms it beats the current one, and keep the older baselines in your panel, so a candidate that beats only the newest shows up as a regression against the others. The lab's Back Track King run replaced its frozen baseline six times over 74,335 iterations. Its later networks show why the live check comes first: the 71,100 network scored .53 against the 47,250 baseline locally, then finished live about a point below the lowest of six submissions of 47,250.^[2]

09When to stop

Near the top of a ladder, the improvements you can still find get smaller than the noise in one submission's score. You can keep running panels for days to tell two bots apart, and each day buys a smaller answer. Deciding to stop, or to change approach, is part of the procedure, and your records can make the decision.

Start from how much a submission's score moves on its own. Submit identical code several times, and the spread of the final scores is your noise band. A change smaller than the band can't be seen in one submission, and a panel that could see it costs many test games; Measuring Bot Strength works out how many.

Two numbers from your records settle it. The first is the noise band. The second is the gap to the next result you care about, a promotion or a rank. When candidate after candidate lands inside the band and the gap is several bands wide, another round of tuning won't close it; the next gain needs a different idea, such as a new input for the network, a search on top of it, or a different mix of training opponents. If you have no such idea, stop and spend the days on the next game, where the same lab starts again at step 2. After Back Track King stopped, the lab took five more games to Legend with search and hand-written bots within four days (Case Study: Five Legends by Search).

10Reuse it for the next contest

A new contest brings a new game, and the temptation is a new repository. Most of the lab doesn't know which game it's playing, though. It talks to engines through the Game and Agent contract and to bots through the text protocol, so only a handful of files are specific to one game.

What can you reuse for the next contest? The engine contract and the batched simulator built on it; the trainer with its opponent pool, champion gate and guards against self-play drift; behavior cloning from native bots; quantization-aware training and the weight codec; the exporter with its parity checks; the measurement harness and the seed ledger; and the live tools for test games, submissions and battle records. For each new game you write the rules and the map generator, bit-exact against the referee; how a position becomes network input and how a network output becomes a command; the network's shape; and the reward.^[3]

Adding a game to the lab follows the same five moves every time:

1. Engine. games/<id>/engine/: a Game with the referee's rules, protocol text and map generation (with the same random generator the referee seeds), a fuzz policy, and an Agent.
2. Register. An entry in games/registry.json and the crate in the workspace Cargo.toml.
3. Verify. Replay referee games, or live test games when no referee is published, until every game replays exactly.
4. Game spec. games/<id>/rl.py, with network defaults sized for the 100,000-character limit.
5. Train. ./train.sh <id>, measured by the same harness as everything else.

11Where labs go wrong

Editing the frozen bot. A quick fix to original.rs silently moves the yardstick for every comparison you made before it. Copy it under a new name and change the copy.

Retesting on the tuning seeds. Deterministic bots replay the same games on the same seeds, so the second test repeats the first one's luck. Take a fresh range from the ledger for every confirmation.

Benchmarking against an identical copy. In a game where two identical deterministic bots mirror each other into a draw every time, any near-copy "wins" by breaking the mirror. Measuring Bot Strength shows the trap; keep different opponents in every panel.

Trusting one submission's score. Six submissions of identical code finished up to 1.6 points apart in the lab's records. Compare a result with your noise band before calling it a change.

Tracing a script that holds the token. Bash's set -x prints every command after expansion, and that includes the Authorization header. Don't trace API scripts, and don't paste their output anywhere.

Burning the test-game budget early. One live panel of 80 games takes the whole hourly allowance. Decide in the morning which panels the day is for.

Trusting a local runner on timing. cg-brutaltester's README lists timeout handling as a feature still to come, so a bot that wins locally can still time out live. Time every turn yourself and keep a safety margin.

12What's next

Networks Inside Search picks up where step 14 leaves off. A policy network decides at a glance and a search checks a few moves deeply, and the strongest bots in several CodinGame games do both. It covers PUCT, the one-turn search the lab wrapped around its Back Track King network, and why testing a search only against its own network overstates the gain.

13Sources

Numbers marked as the author's lab come from its records, named in each note. CodinGame's limits and endpoints are as documented on 2026-10-30. Widget readouts are computed in the page from the controls, and both widgets are models: nothing in them was measured.

1. CodinGame, 2026. Back Track King, Summer Challenge 2026. Contest dates, September 7 to 21, 2026, from the contest page's own data (retrieved 2026-10-30).
2. Author's lab records, 2026 (private repository). Contest and live results: AGENTS.md (October 15 API check, October 24 and 26 entries), docs/provenance.md, experiments/nn-final-20261014.json, experiments/h2h-20261019.json, and experiments/live-submissions/ (summer-challenge-2026-back-track-king/submitted.jsonl, evaluations-20261024.json, comparison-20261024.json; tron-battle/submitted.jsonl); training timeline from experiments/runs/nn-main/metrics.jsonl and evaluations.jsonl.
3. Author's lab records, 2026 (private repository). Layout and code: README.md, .gitignore, crates/cg-env/src/lib.rs, framework/rl/spec.py, games/tictactoe/rl.py, games/registry.json, games/backtrackking/bots/baseline/provenance.json, third_party/sources.lock.json, train.sh and docs/multi-game.md (what carries over, and adding a contest).
4. CodinGame, 2026. Leagues and promotion, API documentation (retrieved 2026-10-30). New rules after a promotion, and reading the statement again.
5. dreignier, 2026. cg-brutaltester README, commit d6bbeb3. Command-line flags, seat swaps and starting seeds, the command-line class for SDK referees, and timeout handling still listed as a future feature.
6. CodinGame, 2026. Game Runner, CodinGame SDK documentation source, commit e78a9a4. Headless games with a fixed seed through simulate().
7. Author's lab records, 2026 (private repository). Engine verification in experiments/verify/*/summary.json and README.md (Back Track King's 2,540 games), games/spring2022/README.md (Spring Challenge 2022's 1,000 games) and docs/multi-game.md (Fantastic Bits' two differences); map options in experiments/verify/tronbattle-live and madpodracing-legend-live; the last Back Track King stop in experiments/runs/nn-main/continuous-training.log, the file times of its two exports, and their report.json files.
8. JOiy, 2026. JOiy's bot: Summer challenge 2026, GitHub gist posted by tcordel. A promotion gate that drew on the training maps, and the 2,000 held-out maps that replaced it.
9. CodinGame, 2026. Agent Skill: CodinGame, API documentation (retrieved 2026-10-30). The device flow's token lifetime and revocation, and keeping your own records because the API keeps no score history.
10. CodinGame, 2026. CodinGame API reference (retrieved 2026-10-30). Endpoints for test games, submissions, progress, battles and game logs; the token format; the rate limits and the 429 rule.
11. William Denniss, John Bradley, Michael B. Jones, Hannes Tschofenig, 2019. RFC 8628: OAuth 2.0 Device Authorization Grant. The device flow: a device code and a user code, approval in a browser on another device, and polling for the token.
12. Author's lab records, 2026 (private repository). AGENTS.md (October 30 entries), docs/postmortem-lessons.md (Ultimate Tic-Tac-Toe network against MCTS; Spring Challenge 2022 round 3; Game of Drones rounds 4 and 5), docs/multi-game.md (dates the games were added), experiments/live-submissions/*/rankings.jsonl and submitted.jsonl (standings on October 30), and experiments/runs/spring2022/training.log with the export folder of champion 99,250 (exported and submitted while that run kept training).