When to Use Machine Learning

01Machine learning sets the numbers for you

In light cycles, the game behind CodinGame's Tron Battle, each player steers a bike that leaves a wall behind it, and the first to crash loses. The evaluation you wrote in Evaluation Functions scores a position by adding up features, numbers measured from the board, each times a weight: so much for each cell of territory you reach first, so much for each free square next to your head. You picked those weights yourself and tuned them with small tournaments. Machine learning is the other way to get them. You write the program with those numbers left open, collect data, and let a second program adjust the numbers until the first one does better on that data. The adjustable numbers are the program's parameters, and adjusting them is training.

Training changes those numbers and nothing else. The code that reads the board, the features it computes and the rule that turns scores into a move all stay as you wrote them. So the decisions that make or break a learned bot are still yours: what the program sees, what counts as doing better, where the data comes from, and how you check the result.

A classic early example is a game program. In 1959 Arthur Samuel of IBM described a checkers program that adjusted the coefficients of its own scoring formula as it played. His abstract reports that it learned "to play a better game of checkers than can be played by the person who wrote the program," in "8 or 10 hours of machine-playing time", starting from the rules and "a redundant and incomplete list of parameters" whose signs and weights nobody had set.^[1] It also played itself. One copy, called Alpha, adjusted its weights after every move; the other, Beta, kept the same formula for a whole game; and when Alpha won, Beta took over Alpha's weights. Keeping new weights only when they win is the idea behind the promotion gates in Measuring Bot Strength.

Every page in this series answers part of one question: how much of a bot to write by hand, how much to leave to search, and how much to train. This page is about the third part, and about when it pays.

02Train three weights on recorded games

Start with a question your evaluation already half answers: from this position, who goes on to win? Record a few thousand light-cycle positions and note, for each one, whether the player who just moved won the game. That's a dataset: examples whose right answers, the labels, you already know. Learning from labelled examples is supervised learning.

The model is your evaluation with one change. It still adds up weights times the three features (territory margin, free cells next to your head, distance between the two heads) plus a constant called the bias. Then it squashes that total into a chance between 0 and 1 with the sigmoid function. A large positive total means "almost surely wins", a large negative one "almost surely loses", and zero means even odds. This model is logistic regression.

Training needs a number that says how wrong the current weights are. The usual one is the log loss: when the player won, it's −ln of the chance the model gave a win, and when the player lost, −ln of the chance it gave a loss. A confident right answer costs almost nothing. A confident wrong answer costs a lot.

As formulas, the prediction and its penalty are:

In words: multiply each feature by its weight, add them up with the bias, and squash the total into a number between 0 and 1. That's the predicted chance of a win. The loss charges −ln of the chance the model gave to what really happened, and training lowers its average over the whole dataset.

To lower that average, training nudges each weight in the direction that reduces it. The direction comes from the slope of the loss with respect to the weight, its gradient. For this model the slope works out to something short: (chance − label) × feature, averaged over the examples, with the label 1 for a win and 0 for a loss. Gradient descent repeats one step: compute that slope for every weight and move each weight a little against it. The size of the move is set by the learning rate, and one pass over the whole dataset is an epoch.

Written as an update rule:

In words: each weight moves against its average slope, scaled by the learning rate η (eta). Positions the model under-called pull the weights up along their features, and positions it over-called push them down.

The widget runs exactly this on 10,603 positions from 150 games on a 15 × 10 board, between two flood-fill bots (each moves where the most cells stay reachable) that play a random legal move on one turn in five so the games vary. Before training it rescales each feature to an average of 0 and a typical spread of 1, because territory margins in these games run from −130 to +135 cells while free neighbours stay between 0 and 3, and one learning rate can't suit both scales. So its weights are per typical spread of each feature. It also keeps 4,198 positions from 60 other games out of training. That held-out set shows whether the model learned something general or memorized its examples.

Why didn't a model that predicts the winner play better? Predicting who wins compares positions that differ a lot, and a big territory lead beats a small one. Choosing a move compares the three positions one move away, which often have nearly the same territory, so the other features decide. The loss scored the first job and never looked at the second. Training makes a model better at exactly what its loss measures, which is why choosing what to measure is part of the work.

The panels show two more things. The held-out loss (0.633) ends a little above the training loss (0.601): the model does slightly worse on games it never saw. With three weights the gap can't grow much, but a network with thousands of weights can fit its training games far better than new ones, which is overfitting. And the learning rate is a real choice: too small wastes epochs, and too large overshoots the bottom and bounces. Linear Models covers logistic regression and the learning rate in more depth.

03Three ways a bot learns

Bot programmers use learning in three ways. They differ in the data they need and in what limits the result.

Copy a teacher. Record games of a bot you already have and train a network to predict the move it chose in each position. That's supervised learning again, with the teacher's move as the label, and for bots it's called behavior cloning. The data is cheap, since your simulator can record a million decisions overnight, but the teacher caps the result: a clone usually plays a little below the bot it copied. RamzaBeoulve, 4th in Back Track King (Summer Challenge 2026), used it as a head start: "My network didn't start from scratch. I first trained it with behavioral cloning on my best heuristic bot to give it a strong starting point and avoid spending days before getting decent PPO results."^[3]

Learn from results. Let the bot play, usually against copies of itself, which is self-play, and make the moves that led to wins more likely. That's reinforcement learning. Instead of labels it gets a reward, a score that arrives after it acts, often only when the game ends. No teacher limits it, but it needs a great many games: the first network to reach Legend in the author's lab took 917,630 of them.^[4]

Learn an evaluation, keep the search. A network scores positions, or suggests which moves to look at, and a search still makes the decision. The weights can come from either of the first two ways. The win predictor in the widget above was a tiny learned evaluation, trained on game results.

Some strong bots combine all three. RamzaBeoulve cloned his heuristic bot, trained the clone by self-play with PPO, a reinforcement learning method, and played it inside a search. The author's lab did the same in that game: a clone first, then self-play, and later a one-turn search around the network.^[4]

04What learning costs before it wins a game

A night of training sounds cheap. Most of the bill arrives before the first useful game, because a learned bot needs five things in place before training can start.

1. A simulator that matches the referee exactly and runs fast. Training plays its games in your copy of the rules, hundreds of thousands of them. A rule that's off by one unit teaches the network a different game, and a slow simulator turns one night into a month. Referees and Simulators builds and checks one.
2. A teacher bot. Your best hand-written bot, to clone as a starting point and to measure every network against. Self-play from random moves spends days relearning what your bot already knows.
3. Compute. A GPU for the network updates, and enough CPU cores to run thousands of games at once.
4. A measurement discipline. Fixed seeds, both seats, a panel of opponents and an interval on every result, as in Measuring Bot Strength. Scores against the network's own earlier versions say little about how it does against anyone else.
5. A way to ship the weights. The submission is one source file of at most 100,000 characters that runs on CodinGame's CPU inside the turn limit, so the network's arithmetic has to be code in that file. The weights have to be packed into it as text, which Shipping a Network covers.

In that game, then, learning cost a few days of setup and failed attempts, about half a day of training on one consumer GPU for the first Legend network, and about a week more for the best one. Training past that point didn't help live. In a game where search already does well, the same days usually buy more when spent on the search.

05Read the game before you choose

Two numbers settle most of the question: how many choices one turn offers, and how many positions your search can look at before the turn's time runs out.

In light cycles you choose among at most 3 moves: straight, left or right. In Ultimate Tic-Tac-Toe you play in the small board you were sent to, so 9 cells at most, and up to 81 on the first move or when that board is closed. Mad Pod Racing accepts any target point and thrust, but several top bots searched only a handful of moves per pod; Owl, 2nd in the 2016 Coders Strike Back contest, used 7.^[5] The widget uses 12 per pod (turn −18°, 0° or +18°, with thrust 0, full, BOOST or SHIELD), so 144 for your two pods. Back Track King gives you 3 paint points a turn, enough to lay track on 3 plain cells, and 1 disruption point to spend on one region.^[6] Picking 3 cells out of k free ones and 1 region out of r gives C(k, 3) × r options, where C(k, 3) counts the ways to choose 3 things from k.

The general rule says how deep a search that tries every move gets in one turn:

In words: trying every move at every step means looking at b^d positions to see d moves ahead, so the search gets as deep as the turn's budget of positions allows. Real searches prune and sample, so they reach further, but the comparison between games holds.

From those counts come the signals. Search is the better first bet when a turn offers few choices, when the game turns on forcing tactics where one move loses outright, and when you can write an evaluation that is cheap and roughly right. Learning is the better bet when a turn combines several decisions into millions of options, and when the judgment is hard to write as a formula (which of 324 cells will matter in ten turns?). Moving at the same time doesn't settle it either way: three of the contests in the table below were won by searches built for games where everyone moves at once.

Ultimate Tic-Tac-Toe shows that a missing evaluation isn't enough, on its own, to make learning win. The game has "no good, simple heuristic evaluation function for a board position", in the words of Tom Alard, 2nd in its arena when he wrote that.^[8] Yet a small network trained on about 2,000 replays and played without search finished last in Gold in 2018, by its author JFB's account.^[9] Near the top of the arena in 2026, bots search with a network inside: Alard runs a Monte Carlo tree search variant over a network of about 215,000 parameters, and he names MrSubZero's alpha-beta search with a similar network among the top three.

Back Track King sits at the other end. With 253 million options a turn, a search that tries every move can't finish even one turn. The three top-four finishers who wrote up their bots all trained networks by reinforcement learning: Indrill (1st), Pduhard (3rd) and RamzaBeoulve (4th). Pduhard and RamzaBeoulve also searched, checking a few candidate plans and letting the network judge them, which is the subject of Networks Inside Search.

06What won CodinGame contests, 2022 to 2026

"The winner used reinforcement learning" is true of some contests and not of others. The table lists every CodinGame bot contest from 2022 to 2026, with the method its winner described.^[10]

Networks trained by reinforcement learning won 4 of the 12: reCurse in Fall Challenge 2023 and Winter Challenge 2024, delineate in Spring Challenge 2026 and Indrill in Summer Challenge 2026. Search or hand-written rules won the other 8. The recent contests lean toward learning, with 3 of the last 5, and the places just below the winners show the mix. In Summer Challenge 2024 reCurse finished 2nd. His network chose moves directly, which makes it a policy network, and he wrote that "even a very good policy network could not be as competitive on its own anymore (unlike the previous challenge), so I started looking into search."^[11]

The table records what won and what the winners wrote. It doesn't measure which method is stronger. A few players win again and again (bowwowforeach and MSz three times each, reCurse and delineate twice), so a winner's method says as much about the player as about the method. delineate won one contest with hand-written rules and another with a trained network. And each contest is a different game, with its own count of choices per turn.

07If the computer learns, where is the skill?

A common question: if the computer trains the bot, where is the skill, and shouldn't everyone's bots come out the same?

Two players can run the same training algorithm and still end up with very different bots, because training only sets the numbers. You still decide what the network sees, how its outputs encode a move, what it's rewarded for, how exact and fast the simulator is, who it plays against, how you measure progress, how much compute you spend, whether a search sits on top, and how the weights fit into 100,000 characters. Each of those changes the bot, and you can't tell whether a change helped without a measurement setup you also built. The skill moves from tuning weights by hand to designing the system that tunes them and checking what it produces.

The top Back Track King write-ups show how far one idea spreads:

• Indrill, 1st: deep Q-learning, a different family of reinforcement learning from PPO, and no search at play time: "my selection is a simple max". His reward adds each turn's change in a projected final score to the game result.^[7]
• Pduhard, 3rd: PPO self-play with half the games against older versions of the network, a reward equal to the turn's income gap, plus 30 to the winner and minus 30 to the loser at the end, and a small search on top that scored 0.839 against the same network without it.^[12]
• RamzaBeoulve, 4th: a graph network started by cloning his heuristic bot, trained with PPO, then searched with Smitsimax using the network's move probabilities as guides.^[3]
• JOiy, 13th (Legend): PPO with a reward only at the end of the game, which he reports "produced every good bot", and a penalty that keeps the network near the policy it started from.^[13]

All four used reinforcement learning on the same game, and their inputs, rewards and searches all differ. Indrill and Pduhard rewarded progress every turn, each in a different way, while JOiy rewarded only the result.

08Decide, then start the project

Back to the series' question: how much to write by hand, how much to search, how much to train. For any one game the answer starts from the bot you already have. A common way to ask it is which training model to use.

Start from your heuristic or search bot, with its strength measured. If the game is tactical with few choices per turn, improve the search first (Minimax and Alpha-Beta, Monte Carlo Tree Search, Evolution and Beam Search); a learned evaluation can come later, inside that search. If one turn combines decisions into millions of options and no formula you can write scores a position well, clone your bot into a network and then train the network by self-play with PPO. "Model" here means the method. The network's shape comes from the game, and it's the next page's subject.

Before you train anything, check that you can tick every line of this list:

1. Your current bot is measured: a ladder score, and a local panel of opponents with intervals.
2. You counted the choices per turn and how deep a search gets in the time limit.
3. You read the game's post-mortems, if it has any, for what the top bots did (Learning from the Leaderboard).
4. You have a bit-exact simulator, or a plan for one, fast enough to play a million games.
5. You sized the network to the file: parameters times bits per weight, against the characters left after your bot's code.
6. You wrote the decision down, with the result that would change it. Capstone: Build the Lab makes this one step of the routine.

09Logistic regression in code

These complete programs train the model from section 02 by full-batch gradient descent and check it on a case with a known answer: given examples drawn from known weights, training from zero finds those weights again. The test in main plants weights 1.5, −2 and 0.5 with bias 0.25, draws 20,000 examples whose three features each lie between −2 and 2, and labels each one a win with the planted model's chance. It trains for 300 epochs at learning rate 1, asserts that the loss never rises (allowing for rounding), and asserts that every recovered weight is within 0.1 of its planted value.

All three versions draw the same examples from the same seeded generator and print the same result: the loss falls from 0.6931 to 0.3481, and the weights come out at 1.505, −2.005 and 0.498 with bias 0.245. The PyTorch version doesn't write the slope formula at all. It asks autograd for the gradient of the loss, which is how the trainers in the rest of this series work.

// Logistic regression trained by gradient descent, with a harness that plants
// known weights, draws labelled examples from them, and checks that training
// from zero finds those weights again.
#include <array>
#include <cassert>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <vector>

constexpr int kFeatures = 3;
using Features = std::array<double, kFeatures>;

// SplitMix64: a small seeded generator, so every run draws the same examples
// (the Rust and Python versions use the same one and get the same data).
struct SplitMix64 {
    std::uint64_t state;
    std::uint64_t next() {
        state += 0x9E3779B97F4A7C15ull;
        std::uint64_t mixed = state;
        mixed = (mixed ^ (mixed >> 30)) * 0xBF58476D1CE4E5B9ull;
        mixed = (mixed ^ (mixed >> 27)) * 0x94D049BB133111EBull;
        return mixed ^ (mixed >> 31);
    }
    // Uniform in [0, 1): the top 53 bits, which a double holds exactly.
    double uniform() { return static_cast<double>(next() >> 11) * 0x1.0p-53; }
};

struct Model {
    Features weights{};  // one weight per feature; training starts them at zero
    double bias = 0.0;
};

double sigmoid(double score) { return 1.0 / (1.0 + std::exp(-score)); }

// The model's chance that the label is 1: a weighted sum, squashed into (0, 1).
double predict(const Model& model, const Features& features) {
    double score = model.bias;
    for (int i = 0; i < kFeatures; ++i) score += model.weights[i] * features[i];
    return sigmoid(score);
}

// Mean log loss: how surprised the model is by the real labels, on average.
double meanLoss(const Model& model, const std::vector<Features>& rows,
                const std::vector<int>& labels) {
    double total = 0.0;
    for (std::size_t row = 0; row < rows.size(); ++row) {
        const double chance = predict(model, rows[row]);
        total -= labels[row] == 1 ? std::log(chance) : std::log(1.0 - chance);
    }
    return total / static_cast<double>(rows.size());
}

// One full-batch step downhill. For this model and loss, the slope of the
// mean loss along each weight is the average of (prediction - label) * feature.
void gradientStep(Model& model, const std::vector<Features>& rows,
                  const std::vector<int>& labels, double learningRate) {
    Features gradient{};
    double biasGradient = 0.0;
    for (std::size_t row = 0; row < rows.size(); ++row) {
        const double error = predict(model, rows[row]) - labels[row];
        for (int i = 0; i < kFeatures; ++i) gradient[i] += error * rows[row][i];
        biasGradient += error;
    }
    const double count = static_cast<double>(rows.size());
    for (int i = 0; i < kFeatures; ++i) model.weights[i] -= learningRate * gradient[i] / count;
    model.bias -= learningRate * biasGradient / count;
}

int main() {
    // The weights the examples are drawn from: the answer training should find.
    const Model planted{{1.5, -2.0, 0.5}, 0.25};
    SplitMix64 random{2026};
    std::vector<Features> rows;
    std::vector<int> labels;
    for (int example = 0; example < 20000; ++example) {
        Features features;
        for (double& value : features) value = 4.0 * random.uniform() - 2.0;  // in [-2, 2)
        // Label 1 with the planted model's chance, the way a position the
        // model rates at 0.7 goes on to win 7 games in 10.
        const int label = random.uniform() < predict(planted, features) ? 1 : 0;
        rows.push_back(features);
        labels.push_back(label);
    }

    Model model;  // all zeros, so every first prediction is 0.5
    const double startLoss = meanLoss(model, rows, labels);
    double previousLoss = startLoss;
    for (int epoch = 1; epoch <= 300; ++epoch) {
        gradientStep(model, rows, labels, 1.0);
        const double loss = meanLoss(model, rows, labels);
        assert(loss <= previousLoss + 1e-12);  // downhill every epoch, up to rounding
        previousLoss = loss;
    }

    for (int i = 0; i < kFeatures; ++i)
        assert(std::abs(model.weights[i] - planted.weights[i]) < 0.1);
    assert(std::abs(model.bias - planted.bias) < 0.1);
    std::printf("loss %.4f -> %.4f\n", startLoss, previousLoss);
    std::printf("weights %.3f %.3f %.3f, bias %.3f\n", model.weights[0], model.weights[1],
                model.weights[2], model.bias);
}

// Logistic regression trained by gradient descent, with a harness that plants
// known weights, draws labelled examples from them, and checks that training
// from zero finds those weights again.
const FEATURES: usize = 3;
type Features = [f64; FEATURES];

// SplitMix64: a small seeded generator, so every run draws the same examples
// (the C++ and Python versions use the same one and get the same data).
struct SplitMix64 {
    state: u64,
}

impl SplitMix64 {
    fn next(&mut self) -> u64 {
        self.state = self.state.wrapping_add(0x9E37_79B9_7F4A_7C15);
        let mut mixed = self.state;
        mixed = (mixed ^ (mixed >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9);
        mixed = (mixed ^ (mixed >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB);
        mixed ^ (mixed >> 31)
    }
    // Uniform in [0, 1): the top 53 bits, which an f64 holds exactly.
    fn uniform(&mut self) -> f64 {
        (self.next() >> 11) as f64 / (1u64 << 53) as f64
    }
}

struct Model {
    weights: Features, // one weight per feature; training starts them at zero
    bias: f64,
}

fn sigmoid(score: f64) -> f64 {
    1.0 / (1.0 + (-score).exp())
}

// The model's chance that the label is 1: a weighted sum, squashed into (0, 1).
fn predict(model: &Model, features: &Features) -> f64 {
    let mut score = model.bias;
    for i in 0..FEATURES {
        score += model.weights[i] * features[i];
    }
    sigmoid(score)
}

// Mean log loss: how surprised the model is by the real labels, on average.
fn mean_loss(model: &Model, rows: &[Features], labels: &[u8]) -> f64 {
    let mut total = 0.0;
    for (row, &label) in rows.iter().zip(labels) {
        let chance = predict(model, row);
        total -= if label == 1 { chance.ln() } else { (1.0 - chance).ln() };
    }
    total / rows.len() as f64
}

// One full-batch step downhill. For this model and loss, the slope of the
// mean loss along each weight is the average of (prediction - label) * feature.
fn gradient_step(model: &mut Model, rows: &[Features], labels: &[u8], learning_rate: f64) {
    let mut gradient = [0.0; FEATURES];
    let mut bias_gradient = 0.0;
    for (row, &label) in rows.iter().zip(labels) {
        let error = predict(model, row) - f64::from(label);
        for i in 0..FEATURES {
            gradient[i] += error * row[i];
        }
        bias_gradient += error;
    }
    let count = rows.len() as f64;
    for i in 0..FEATURES {
        model.weights[i] -= learning_rate * gradient[i] / count;
    }
    model.bias -= learning_rate * bias_gradient / count;
}

fn main() {
    // The weights the examples are drawn from: the answer training should find.
    let planted = Model { weights: [1.5, -2.0, 0.5], bias: 0.25 };
    let mut random = SplitMix64 { state: 2026 };
    let mut rows: Vec<Features> = Vec::new();
    let mut labels: Vec<u8> = Vec::new();
    for _example in 0..20_000 {
        let mut features = [0.0; FEATURES];
        for value in features.iter_mut() {
            *value = 4.0 * random.uniform() - 2.0; // in [-2, 2)
        }
        // Label 1 with the planted model's chance, the way a position the
        // model rates at 0.7 goes on to win 7 games in 10.
        let label = if random.uniform() < predict(&planted, &features) { 1 } else { 0 };
        rows.push(features);
        labels.push(label);
    }

    let mut model = Model { weights: [0.0; FEATURES], bias: 0.0 }; // every first prediction is 0.5
    let start_loss = mean_loss(&model, &rows, &labels);
    let mut previous_loss = start_loss;
    for _epoch in 1..=300 {
        gradient_step(&mut model, &rows, &labels, 1.0);
        let loss = mean_loss(&model, &rows, &labels);
        assert!(loss <= previous_loss + 1e-12); // downhill every epoch, up to rounding
        previous_loss = loss;
    }

    for i in 0..FEATURES {
        assert!((model.weights[i] - planted.weights[i]).abs() < 0.1);
    }
    assert!((model.bias - planted.bias).abs() < 0.1);
    println!("loss {:.4} -> {:.4}", start_loss, previous_loss);
    println!(
        "weights {:.3} {:.3} {:.3}, bias {:.3}",
        model.weights[0], model.weights[1], model.weights[2], model.bias
    );
}

# Logistic regression trained by gradient descent in PyTorch, with the same
# harness: plant weights, draw labelled examples, train from zero, check.
import math

import torch

MASK = (1 << 64) - 1


class SplitMix64:
    """The same seeded generator as the C++ and Rust versions, so the same data."""

    def __init__(self, seed):
        self.state = seed

    def next(self):
        self.state = (self.state + 0x9E3779B97F4A7C15) & MASK
        mixed = self.state
        mixed = ((mixed ^ (mixed >> 30)) * 0xBF58476D1CE4E5B9) & MASK
        mixed = ((mixed ^ (mixed >> 27)) * 0x94D049BB133111EB) & MASK
        return mixed ^ (mixed >> 31)

    def uniform(self):
        return (self.next() >> 11) / float(1 << 53)  # top 53 bits, in [0, 1)


planted_weights = [1.5, -2.0, 0.5]
planted_bias = 0.25
random = SplitMix64(2026)
rows, labels = [], []
for example in range(20_000):
    features = [4.0 * random.uniform() - 2.0 for _ in range(3)]  # in [-2, 2)
    score = planted_bias
    for weight, value in zip(planted_weights, features):
        score += weight * value
    chance = 1.0 / (1.0 + math.exp(-score))
    rows.append(features)
    labels.append(1.0 if random.uniform() < chance else 0.0)

rows = torch.tensor(rows, dtype=torch.float64)
labels = torch.tensor(labels, dtype=torch.float64)

# requires_grad asks PyTorch to work out the gradient of anything computed
# from these tensors, so the slope formula never has to be written by hand.
weights = torch.zeros(3, dtype=torch.float64, requires_grad=True)
bias = torch.zeros((), dtype=torch.float64, requires_grad=True)
learning_rate = 1.0
start_loss = previous_loss = None
for epoch in range(1, 301):
    scores = rows @ weights + bias
    # Mean log loss, computed from the raw scores (it applies the sigmoid itself).
    loss = torch.nn.functional.binary_cross_entropy_with_logits(scores, labels)
    if start_loss is None:
        start_loss = loss.item()
    assert previous_loss is None or loss.item() <= previous_loss + 1e-12
    previous_loss = loss.item()
    loss.backward()  # fills weights.grad and bias.grad
    with torch.no_grad():  # the update itself is not part of the model
        weights -= learning_rate * weights.grad
        bias -= learning_rate * bias.grad
        weights.grad.zero_()
        bias.grad.zero_()

final_loss = torch.nn.functional.binary_cross_entropy_with_logits(rows @ weights + bias, labels).item()
assert final_loss <= previous_loss + 1e-12  # the 300th step went downhill too
for found, planted in zip(weights.tolist(), planted_weights):
    assert abs(found - planted) < 0.1
assert abs(bias.item() - planted_bias) < 0.1
print(f"loss {start_loss:.4f} -> {final_loss:.4f}")
print("weights {:.3f} {:.3f} {:.3f}, bias {:.3f}".format(*weights.tolist(), bias.item()))

10Where the decision goes wrong

Training on the wrong question. The widget's evaluation predicted the winner 68% of the time and still didn't play better. Judge a trained bot by playing it against your yardsticks, never by its loss alone.

Copying the winner's method instead of reading the game. A method that won one contest is evidence about that game. Count the choices, read the post-mortems, then decide.

Training before the simulator is exact. Every game played in a wrong simulator teaches the wrong rules, and a network can learn to exploit a bug in it. Prove the engine against the referee first.

Throwing away the hand-written bot. It's the teacher, the first yardstick and the fallback submission. As of October 30, 2026, five of the author's lab's eight Legend promotions came from search or hand-written bots.

Forgetting the file limit until the end. Work out how many weights fit in 100,000 characters before you choose the network's size, so the file limit shapes the design from the start.

11What's next

Neural Networks for Game Bots starts the project. It turns a light-cycle board into stacked grids of numbers, builds a small network that outputs a probability for each move and a predicted result, gives illegal moves zero probability, and runs the same network in PyTorch and in Rust with a check that both choose the same move.

12Sources

Contest results and quotes are from each player's own write-up. Numbers marked as the author's lab are first-hand records from one lab and aren't general results. Widget readouts are computed in the page from seeded games.

1. Arthur L. Samuel, 1959. Some Studies in Machine Learning Using the Game of Checkers, IBM Journal of Research and Development 3(3), doi:10.1147/rd.33.0210. The abstract's report that the program learned to play better checkers than its author, and the Alpha and Beta self-play rule.
2. Tom M. Mitchell, 1997. Machine Learning, McGraw-Hill, section 1.1. The experience, task and performance definition and its checkers example.
3. RamzaBeoulve, 2026. 4th place solution, CodinGame forum, Summer Challenge 2026 feedback and strategies, post 10. Behavior cloning of his heuristic bot before PPO, his graph network and his Smitsimax search.
4. Author's lab records, 2026 (private repository). Back Track King timeline and engine check on 2,540 referee games: AGENTS.md, docs/provenance.md; 917,630 games and 11.7 hours to the first Legend network, 21.8 million games in all: experiments/runs/nn-main/metrics.jsonl, experiments/nn-final-20261014.json; the network's 0–0–64 against the Ultimate Tic-Tac-Toe MCTS bot: games/tictactoe/README.md; standings of nine games on October 30, 2026: docs/postmortem-lessons.md and experiments/live-submissions/*/rankings.jsonl (Spring Challenge 2022 promoted to Legend that evening); 615 live-test maps: experiments/datasets/btk-maps-live-test.btkm.
5. Owl, 2016. Coders Strike Back, "Your ideas", CodinGame forum, post 55. Seven moves per pod in his 2nd-place contest bot.
6. CodinGame game statements: Tron Battle (30 × 20 grid, 2 to 4 cycles per battle, under 100 ms a turn), Ultimate Tic-Tac-Toe (100 ms a turn), Mad Pod Racing (75 ms a turn in the starting league), Back Track King (3 paint points and 1 disruption point a turn, 50 ms a turn).
7. Indrill, 2026. Ranked 1st, CodinGame forum, Summer Challenge 2026 feedback and strategies, post 5. Deep Q-learning with no search, a projected-score reward, and 1 to 8 million valid track placements a turn.
8. Tom Alard, 2026. Fighting for #1 in the Ultimate Tic-Tac-Toe Arena (archived copy). Random-playout MCTS reaches about the top 50; the top bots search with a small network inside, his with about 215,000 parameters.
9. JFB, 2018. Ultimate Tic-Tac-Toe puzzle discussion, CodinGame forum, post 26. A small network trained on about 2,000 replays and played without search finished last in Gold.
10. CodinGame, contest listing and final leaderboards, read on 2026-10-30: the 12 bot contests of 2022 to 2026 and their winners. Each winner's own write-up is linked from the table.
11. reCurse, 2024. Summer Challenge 2024 post-mortem (2nd place). Why a policy network alone stopped being enough and he added a search.
12. Pduhard, 2026. Summer Challenge 2026, Back Track King: 3rd, README. PPO self-play against a pool of past versions, the income-gap reward, and a small search that scored 0.839 against the same network without it.
13. JOiy, 2026. JOiy's bot: Summer challenge 2026 (gist, last updated 2026-10-10). PPO with a reward only at the end of the game, and a penalty that keeps the network near its starting policy.