Save Games & Serialization
A player has sixty hours in your game. You ship a patch that adds a stamina stat, and a week later the power cuts out halfway through an autosave. Their save has to survive both, and this page builds a save system that does. A save is the opposite of the read-only data the asset pipeline cooks: it's the player's one-of-a-kind world state, with no source to rebuild it from. Three things make it hard: you can't write a pointer to disk, the format will change, and a crash mid-write must not eat the save.
- Save references between objects as ids or generational handles, and reconnect them with a two-pass load.
- Decide what to save and what to recompute, and store world changes as a delta over the cooked level.
- Version a save format, migrate old saves forward, and avoid the traps (reused field numbers, changed defaults) that break them.
- Write a save atomically with a temp file, fsync and rename, keep the previous save as a backup, and catch damaged files with a CRC-32 before trusting a byte.
- Take a consistent autosave mid-play without hitching the frame, by copying at a tick boundary and writing on a worker.
01A save is mutable state, not assets
Lose a cooked texture and the build machine re-cooks it from the artist's file in minutes. Lose a save and sixty hours of play are gone. , turning objects into bytes and back, is one toolbox, but a save and a cooked asset are opposite jobs for it. A cooked asset is authored once, immutable at runtime, shared by every player, and regenerable from the source art. A save is written at runtime, unique to one player, and has no source: lose it and it's gone. Unreal makes the split explicit with a base class for exactly this, USaveGame, "a base class for a save game Object that can be used to save state about the game"[1].
A cooked asset can change format freely: the build just re-cooks it from the source DCC file. A save has no source to re-cook from, so when your data layout changes in a patch, old saves on players' disks must still load. That single constraint, covered in §5, is why save code is harder than asset code even though they share the same serializer. Everything else here (the object graph, delta saves, atomic writes) is in service of getting mutable state to and from disk safely, patch after patch.
02The object-graph problem
A guard is chasing the player when the game saves, and after a reload it should still be chasing the player. Game objects point at each other: an AI's target, a projectile's owner, a node's parent. Those references are pointers in memory, and a pointer is just the address where an object happens to live this run: "pointer values usually change from run to run"[2]. Write a raw pointer to disk and on the next load it points at garbage. The fix is to serialize a reference as something position-independent and re-resolve it on load.
Three ways to encode a reference, from worst to best for a save[2]:
- Raw pointer. Unusable directly: it needs a fix-up table converting saved offsets back to addresses after load, which is what this section avoids.
- Array index. Safe to save, but breaks when items move or are removed, and forces one big homogeneous array.
- Generational handle. A small value, typically
{index, generation}, that uniquely names an object and stays valid across a save/load. This is the encoding to prefer.
Loading an object graph is two passes[3]: pass 1 instantiates every object so all targets exist, recording id → new-object; pass 2 walks the saved references and re-resolves each id to the freshly-created object. This re-resolution is called (or swizzling). The generation counter in a handle is what makes a stale reference safe: if the slot was freed and reused, the generations no longer match and the handle resolves to invalid, not to whatever now occupies the slot. This is the same handle from the Gameplay Layer, now crossing a save boundary. Note the two distinct failure modes: a missing target (the referenced object was not saved) needs a null-and-skip; a reused slot is what the generation check catches. They are not the same bug.
Worked example
In the widget below, the Player lives at 0x4a10 this run, and the Guard's target holds 0x4a10. Save that as id #1 instead. On reload the Player comes back at 0x9c80. Pass 1 creates all three objects and records #1 → 0x9c80; pass 2 rewrites the Guard's target and the Turret's owner from #1 to 0x9c80. A saved raw 0x4a10 would point at whatever happens to live there now. For the generation check: a handle saved as {index 7, generation 3} whose slot was freed and reused now finds generation 4 there, so it resolves to invalid instead of the new occupant.
Save the graph, quit, reload. With ids, the references re-resolve through the handle table and reconnect. Flip on "store raw pointers" and the saved addresses point at nothing after reload: the dangling reference this section exists to prevent.
struct Handle { uint32_t index; uint32_t generation; }; // position-independent reference
// SAVE: write the {index, generation}, never the raw pointer.
void writeRef(Writer& out, Handle handle) { out.u32(handle.index); out.u32(handle.generation); }
// LOAD is TWO passes:
// pass 1: instantiate every saved object, recording id -> new slot (so all targets exist)
// pass 2: re-resolve each saved handle through the table (the fix-up)
Entity* resolveRef(const HandleTable& table, Handle handle) {
if (handle.index >= table.slots.size()) return nullptr; // damaged or foreign save: never index out of range
const Slot& slot = table.slots[handle.index];
if (slot.generation != handle.generation) return nullptr; // stale: slot reused -> invalid, NOT the wrong object
return slot.entity; // the live pointer, valid this run
}
#[derive(Clone, Copy, Serialize, Deserialize)]
struct Handle { index: u32, generation: u32 } // position-independent reference (serde derives the bytes)
// LOAD pass 2: re-resolve a saved handle to a live entity through the table.
fn resolve_ref(table: &HandleTable, handle: Handle) -> Option<&Entity> {
let slot = table.slots.get(handle.index as usize)?; // out of range -> None
if slot.generation != handle.generation { return None; } // stale handle -> None, never the wrong entity
slot.entity.as_ref() // pass 1 already re-instantiated every object
}
Serialization libraries enforce this rule for you. cereal refuses raw pointers outright and dedups shared references: data behind a shared_ptr "is serialized only once, even if several shared_ptr... that point to the same data are serialized," and is re-linked on load[4]. Unity's default inline serialization is the cautionary tale: it can't represent null, it turns one object referenced from two fields into two separate copies, and it can't represent cycles[5]. Unity's manual shows the workaround: an ISerializationCallbackReceiver that flattens a tree into an array of nodes, each storing its child count and the index of its first child, the id-reference pattern applied by hand. (Fields marked [SerializeReference] are the exception: they store null, shared and cyclic references.)
03Save vs recompute
A save that includes the navmesh and every cached path is big, and it can disagree with itself after a patch. Persist authoritative state, the facts nothing else can reproduce; recompute derived state, anything you can calculate from those facts, on load. Saving derived state bloats the file and creates a second source of truth that can drift out of sync with the authoritative fields it was computed from.
- Persist: positions, health, inventory, quest flags, the RNG seed and stream position, and the world-mutation deltas from §4.
- Recompute: the navmesh and spatial acceleration structures, render and physics proxies, cached paths, animation runtime state, and derived UI. They reload with the level or rebuild from the authoritative data.
An object spawned during play (a dropped item, a summoned creature) is saved as the fact of its existence plus its authoritative fields, then re-spawned on load, not serialized as a live engine object with all its transient machinery. SPUD, an open-source Unreal persistence library, does exactly this: runtime-spawned actors "will be automatically re-spawned on load"[6]. Unreal guidance also warns against keeping persistent data in framework actors that are rebuilt with each level (GameMode, GameState, GameSession)[1]. The rule of thumb: if you can recompute it from what you saved, don't save it.
Worked example
A pack holds 12 items, and the save also stores their cached total weight, 42.5 kg. Patch 1.1 changes a sword from 3 kg to 3.5 kg. A player carrying two swords loads a save that still says 42.5 kg while the items now add up to 43.5 kg, and the encumbrance check uses whichever number it reads. Recompute the total from the item list on load and there's nothing to drift.
04Full vs delta saves
A level has thousands of placed objects, and by the time the player saves they've changed a few dozen of them. The cooked level is already on disk, read-only and shared by every player. Re-storing all of it in each save is wasteful, so most level-based games persist only a delta over that baseline: the objects whose state changed from their authored default. Unreal's serializer has this built in, in its default mode "a property's value is not altered from its default... the property will not be serialized"[7].
Bethesda's Creation Engine is the canonical shipped example. A Skyrim save stores change forms, the deltas over the baked plugin data, and writes references as packed three-byte RefIDs whose type bits say whether the form comes from a plugin or was created at runtime (plugin index 0xFF) and exists only in that save[8]. The save also records the plugin list it was written against, so saved references can be mapped onto the current load order. Removing a mod mid-playthrough is what breaks saves: change forms and script state still reference records that no longer exist, a real-world dangling reference.
Worked example
Say a level has 5,000 placed objects with 48 bytes of state each: 240,000 bytes for a full snapshot. The player has opened 40 doors and looted 25 chests, so 65 objects differ from the cooked level. With an 8-byte id plus 48 bytes each, the delta is 65 × 56 = 3,640 bytes: about 1.5% of the snapshot. It grows only with what the player changes.
The practical delta is "changed from the class default," cheap to detect by comparing each object to its baseline. It is not a graph diff between two whole saves, which is expensive and rarely worth it; a single-scene or roguelike-run game legitimately snapshots its entire state, and an MMO keeps the authoritative state server-side. Match the strategy to the game, but know that for an open world the delta-over-baseline is what keeps each save small.
A delta only works if every object in the cooked level has an id that stays the same from build to build, because the save says "object 114: smashed", not "the barrel at row 2, column 3"; the Skyrim RefIDs above play that role. The id also exposes the removed-mod failure described above: when a patch or a removed mod deletes an object, the save still holds a change for it. The loader has to skip that record (and usually log it) instead of failing the whole load.
05Versioning & migration
Patch 1.1 adds a stamina stat, and the first player to load an old save gets a crash, or worse, a stamina value read from whatever bytes happen to follow the health field. Versioning is what breaks in production. You will ship, then patch the save format, and players still have saves written by the old code. They must keep loading. The mechanism is a version tag plus : store a version number in the save, and keep code that upgrades each older version to the next, the same pattern as database schema migrations. Unreal's package format works this way, with an engine version number plus per-feature custom versions[9].
Pick a save version and load it into the current (v3) game. Older saves migrate up: a missing field gets a default, a renamed field is mapped. New code reads every old save:
constexpr uint32_t SAVE_VERSION = 3; // bump on every format change
std::optional<PlayerState> loadPlayer(Reader& in) {
uint32_t version = in.u32(); // the version tag is the FIRST thing in the blob
if (version > SAVE_VERSION) return std::nullopt; // written by a NEWER build: refuse, don't misread
PlayerState player;
player.health = in.f32(); // every version (stored as "hp" in v1/v2; same bytes)
player.stamina = version >= 2 ? in.f32() : 100.0f; // added in v2 -> default for older saves
player.perks = version >= 3 ? in.readPerks() : Perks{}; // added in v3 -> default for older saves
return player; // new code reads ANY old save by defaulting missing fields
}
const SAVE_VERSION: u32 = 3; // bump on every format change
fn load_player(input: &mut Reader) -> Option<PlayerState> {
let version = input.u32(); // version tag first
if version > SAVE_VERSION { return None; } // written by a NEWER build: refuse, don't misread
// Struct fields evaluate top to bottom, so the reads happen in on-disk order.
Some(PlayerState {
health: input.f32(), // every version
stamina: if version >= 2 { input.f32() } else { 100.0 }, // added in v2
perks: if version >= 3 { input.read_perks() } else { Vec::new() }, // added in v3
})
}
Worked example
Using the reader above and the widget's values: a v1 save starts with the version, 1, then hp = 80. The v3 reader takes 80 as health from the same bytes, skips the stamina read because 1 < 2 and sets 100, and skips perks because 1 < 3 and sets an empty list. A v2 save carries stamina = 55 after hp, so the reader keeps 55 and defaults only the perks. A save written by a future v4 build fails the 4 > 3 check and is refused rather than misread. Now change the stamina default from 100 to 120 in patch 1.2: every v1 save quietly loads with 120 stamina, a value its player never had.
Positional binary needs an explicit version branch like the code above. A tagged format (Protocol Buffers, FlatBuffers) buys both compatibility directions cheaply: old code ignores unknown fields, new code defaults missing ones. But the rules are strict, and breaking them is where saves actually die: protobuf says never reuse a field number and reserve deleted ones so they cannot be recycled into a type-mismatched field[10]; FlatBuffers adds new fields only at the end and never deletes (only deprecates)[11]. The subtlest trap: editing a default value after ship silently rewrites the meaning of every old save that relied on the old default. Unreal's GUID-based custom versions exist precisely so two engine branches can evolve the format on independent tracks without colliding on one integer[9].
06Atomic writes
The power cuts out while the game is writing the save. A save is overwritten in place at your peril. A crash, a power loss, or a killed process partway through the write leaves the file half old data and half new, and unreadable. That is the worst outcome, because the previous good save is gone too. The fix is an : write a temp file in the same directory, force it to disk, then rename it over the real save.
Pick a strategy and crash mid-write. In-place corrupts the only save; temp-then-rename leaves the previous save fully intact, because the real file is never touched until the rename:
Worked example
The widget writes a save as 24 blocks over about 1.8 s. Crash in place at block 10 and save.dat holds 10 new blocks and 14 old ones, a file no version of your code ever wrote. With temp + rename, the crash leaves 10 new blocks in save.tmp, which is discarded, and save.dat still holds all 24 blocks of the previous save. The player loses only the progress since the last save.
Temp-then-rename guarantees you never get a half-written file, a torn write. It does not protect against bytes that go bad after the write (a failing drive, a truncated copy) or against a logically-invalid save, a migration bug that writes complete-but-wrong data. A checksum validated on load catches the damaged bytes. It can't catch the migration bug: the checksum was computed over the wrong data, so it matches. A rolling backup slot (keep the previous save.bak) covers both, because when the newest save fails a check or turns out to be wrong, there's an older one to load. And fsync before the rename is not optional. flush() only moves data to the OS buffer; without the sync, a power loss can leave the save replaced by a file whose bytes never reached the disk[12].
The whole save path
The program below puts §2 through §6 into one file format and one save routine. The file starts with a 16-byte header: a magic number, so the loader can reject a file that isn't a save at all; the format version; the payload length; and a CRC-32 of the payload, the same checksum a gzip file stores for its data[13]. Change any single bit of the payload and the CRC changes, so a damaged file fails the comparison on load. It catches accidents, not cheating: anyone who edits a save on purpose can recompute it.
Saving writes save.tmp, flushes it to the device, renames the old save.dat to save.bak, renames save.tmp to save.dat, and on POSIX flushes the directory too. POSIX requires a rename over an existing file to be atomic: anyone looking sees the old file or the new one, never neither[14]. Linux's fsync documentation warns that flushing a file doesn't guarantee its directory entry reached the disk, which takes a separate fsync on the directory[15].
On Windows, MoveFileEx with MOVEFILE_WRITE_THROUGH doesn't return until the move is on disk[16]. On Apple platforms, fsync hands the data to the drive but doesn't make the drive write it out of its own cache; fcntl(F_FULLFSYNC) does, so the C++ listing uses that there[17]. Rust's File::sync_all already makes the same choices: fsync on Linux, F_FULLFSYNC on Apple, FlushFileBuffers on Windows. Its fs::rename calls rename on Unix and MoveFileExW on Windows, but without the write-through flag, so on Windows the Rust version's rename may not be on disk yet when save returns[18].
Loading checks the header and checksum before trusting a byte, checks the entity count against the payload size before allocating anything, migrates a v1 file by defaulting stamina, re-resolves references by id in a second pass, and falls back to save.bak when save.dat fails.
// A save file you can trust: a versioned binary format with a CRC-32 checksum,
// written atomically (temp file, flush to disk, rename) with the previous save
// kept as a backup, then loaded with validation, migration and reference fix-up.
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <fstream>
#include <iterator>
#include <optional>
#include <string>
#include <unordered_map>
#include <vector>
#ifdef _WIN32
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#else
#include <fcntl.h>
#include <unistd.h>
#endif
using Bytes = std::vector<std::uint8_t>;
constexpr std::uint32_t kMagic = 0x45564153; // the bytes "SAVE" read as a little-endian u32
constexpr std::uint32_t kVersion = 2; // v2 added stamina
constexpr std::uint32_t kHeaderSize = 16; // magic, version, payload length, CRC-32
constexpr std::uint32_t kNoTarget = 0;
struct Entity {
std::uint32_t id = 0;
float x = 0, y = 0, health = 0, stamina = 0;
std::uint32_t targetId = kNoTarget; // what the file stores: an id, never a pointer
int target = -1; // what the game uses after fix-up: an index, -1 for none
};
using World = std::vector<Entity>;
// CRC-32 as gzip defines it (RFC 1952: reflected polynomial 0xEDB88320), bit by bit.
std::uint32_t crc32(const std::uint8_t* data, std::size_t size) {
std::uint32_t crc = 0xFFFFFFFFu;
for (std::size_t i = 0; i < size; ++i) {
crc ^= data[i];
for (int bit = 0; bit < 8; ++bit) crc = (crc >> 1) ^ (0xEDB88320u & (0u - (crc & 1u)));
}
return ~crc;
}
// Little-endian writes and bounds-checked reads, so the format is the same on every machine.
void putU32(Bytes& out, std::uint32_t value) {
for (int i = 0; i < 4; ++i) out.push_back(std::uint8_t(value >> (8 * i)));
}
void putF32(Bytes& out, float value) { std::uint32_t bits; std::memcpy(&bits, &value, 4); putU32(out, bits); }
struct Reader {
const Bytes& bytes; std::size_t at = 0; bool ok = true;
std::uint32_t u32() {
if (at + 4 > bytes.size()) { ok = false; return 0; } // never read past the end
std::uint32_t value = 0;
for (int i = 0; i < 4; ++i) value |= std::uint32_t(bytes[at + i]) << (8 * i);
at += 4;
return value;
}
float f32() { std::uint32_t bits = u32(); float value; std::memcpy(&value, &bits, 4); return value; }
};
Bytes encode(const World& world, std::uint32_t version) {
Bytes payload;
putU32(payload, std::uint32_t(world.size()));
for (const Entity& entity : world) {
putU32(payload, entity.id);
putF32(payload, entity.x); putF32(payload, entity.y); putF32(payload, entity.health);
if (version >= 2) putF32(payload, entity.stamina);
putU32(payload, entity.targetId);
}
Bytes file;
putU32(file, kMagic); putU32(file, version); putU32(file, std::uint32_t(payload.size()));
putU32(file, crc32(payload.data(), payload.size()));
file.insert(file.end(), payload.begin(), payload.end());
return file;
}
// Validate everything before trusting anything; on failure, say which check failed.
std::optional<World> decode(const Bytes& file, std::string& error) {
Reader header{file};
std::uint32_t magic = header.u32(), version = header.u32(), length = header.u32(), storedCrc = header.u32();
if (!header.ok) { error = "shorter than a header"; return std::nullopt; }
if (magic != kMagic) { error = "not a save file"; return std::nullopt; }
if (version == 0 || version > kVersion) { // a newer build wrote it: refuse rather than misread
error = "unknown version " + std::to_string(version);
return std::nullopt;
}
if (length != file.size() - kHeaderSize) { error = "length mismatch (truncated?)"; return std::nullopt; }
if (crc32(file.data() + kHeaderSize, length) != storedCrc) {
error = "checksum mismatch";
return std::nullopt;
}
Reader in{file, kHeaderSize};
std::uint32_t count = in.u32();
std::size_t bytesPerEntity = version >= 2 ? 24 : 20;
if (std::uint64_t(count) * bytesPerEntity + 4 != length) { // before allocating; 64-bit math can't wrap
error = "payload doesn't match its own counts";
return std::nullopt;
}
World world(count);
for (Entity& entity : world) { // pass 1: create every object
entity.id = in.u32();
entity.x = in.f32(); entity.y = in.f32(); entity.health = in.f32();
entity.stamina = version >= 2 ? in.f32() : 100.0f; // migrate: v1 had no stamina
entity.targetId = in.u32();
}
std::unordered_map<std::uint32_t, int> indexOf;
for (int i = 0; i < int(world.size()); ++i) indexOf[world[i].id] = i;
for (Entity& entity : world) { // pass 2: re-resolve references by id
auto found = indexOf.find(entity.targetId);
entity.target = found == indexOf.end() ? -1 : found->second; // missing target: drop the reference
}
return world;
}
#ifndef _WIN32
// fsync, except on Apple platforms, where fsync can stop at the drive's own write cache.
bool flushToDevice(int fd) {
#ifdef __APPLE__
return ::fcntl(fd, F_FULLFSYNC) != -1;
#else
return ::fsync(fd) == 0;
#endif
}
#endif
// Write bytes to a file and force them to the storage device before returning.
bool writeDurably(const std::string& path, const Bytes& bytes) {
#ifdef _WIN32
HANDLE file = CreateFileA(path.c_str(), GENERIC_WRITE, 0, nullptr, CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL, nullptr);
if (file == INVALID_HANDLE_VALUE) return false;
DWORD written = 0;
bool ok = WriteFile(file, bytes.data(), DWORD(bytes.size()), &written, nullptr) && written == bytes.size()
&& FlushFileBuffers(file);
CloseHandle(file);
return ok;
#else
int fd = ::open(path.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd < 0) return false;
std::size_t done = 0;
while (done < bytes.size()) { // write() may write less than asked
ssize_t chunk = ::write(fd, bytes.data() + done, bytes.size() - done);
if (chunk <= 0) { ::close(fd); return false; }
done += std::size_t(chunk);
}
bool ok = flushToDevice(fd); // to the device, not just the OS cache
::close(fd);
return ok;
#endif
}
// Replace `to` with `from` by renaming, never by copying bytes. On POSIX, readers see the old
// file or the new one, never a mix.
bool replaceFile(const std::string& from, const std::string& to) {
#ifdef _WIN32
return MoveFileExA(from.c_str(), to.c_str(), MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH);
#else
return std::rename(from.c_str(), to.c_str()) == 0;
#endif
}
void syncDirectory() { // POSIX: make the renames themselves durable
#ifndef _WIN32
int fd = ::open(".", O_RDONLY);
if (fd >= 0) { flushToDevice(fd); ::close(fd); }
#endif
}
bool save(const World& world) {
if (!writeDurably("save.tmp", encode(world, kVersion))) return false;
replaceFile("save.dat", "save.bak"); // keep the previous save; fails harmlessly the first time
bool ok = replaceFile("save.tmp", "save.dat");
syncDirectory();
return ok;
}
// Test helper: overwrite a file with no durability, to damage it on purpose.
void overwrite(const std::string& path, const Bytes& bytes) {
std::ofstream(path, std::ios::binary).write(reinterpret_cast<const char*>(bytes.data()), std::streamsize(bytes.size()));
}
Bytes readFile(const std::string& path) {
std::ifstream file(path, std::ios::binary);
return Bytes(std::istreambuf_iterator<char>(file), {});
}
// Try the newest save, then the backup.
std::optional<World> load() {
for (const char* path : {"save.dat", "save.bak"}) {
std::string error;
if (auto world = decode(readFile(path), error)) {
if (world->size() < 2) return world; // valid, but no player and guard to print
const Entity& player = (*world)[0];
int guardTarget = (*world)[1].target;
std::printf(" loaded %s: player health %.0f, stamina %.0f; guard's target: %s\n", path,
player.health, player.stamina, guardTarget < 0 ? "none" : "the player");
return world;
}
std::printf(" %s rejected: %s\n", path, error.c_str());
}
return std::nullopt;
}
int main() {
std::printf("crc32(\"123456789\") = %08X (expected CBF43926)\n",
crc32(reinterpret_cast<const std::uint8_t*>("123456789"), 9));
World world = {{1, 0, 0, 100, 55, kNoTarget}, {2, 5, 3, 80, 60, 1}, {3, -4, 8, 250, 0, 1}};
save(world); // first save
world[0].health = 70; // the player takes a hit
save(world); // second save; the first becomes save.bak
std::printf("clean load:\n"); load();
// Damage the file on purpose (the game itself only ever writes through save()).
Bytes damaged = readFile("save.dat");
damaged[30] ^= 0x01; // one flipped bit on disk
overwrite("save.dat", damaged);
std::printf("one bit flipped:\n"); load();
Bytes truncated = readFile("save.bak");
truncated.resize(40); // a torn write from an in-place save
overwrite("save.dat", truncated);
std::printf("truncated:\n"); load();
world[1].targetId = 9; // a reference to an object that was never saved
Bytes oldFormat = encode(world, 1); // a v1 save: no stamina field
overwrite("save.dat", oldFormat);
std::printf("v1 file with a dangling reference:\n"); load();
}
// A save file you can trust: a versioned binary format with a CRC-32 checksum,
// written atomically (temp file, flush to disk, rename) with the previous save
// kept as a backup, then loaded with validation, migration and reference fix-up.
use std::collections::HashMap;
use std::fs::{self, File};
use std::io::{self, Write};
const MAGIC: u32 = 0x4556_4153; // the bytes "SAVE" read as a little-endian u32
const VERSION: u32 = 2; // v2 added stamina
const HEADER_SIZE: usize = 16; // magic, version, payload length, CRC-32
const NO_TARGET: u32 = 0;
#[derive(Clone)]
struct Entity {
id: u32,
x: f32, y: f32, health: f32, stamina: f32,
target_id: u32, // what the file stores: an id, never a pointer
target: Option<usize>, // what the game uses after fix-up: an index
}
type World = Vec<Entity>;
fn entity(id: u32, x: f32, y: f32, health: f32, stamina: f32, target_id: u32) -> Entity {
Entity { id, x, y, health, stamina, target_id, target: None }
}
// CRC-32 as gzip defines it (RFC 1952: reflected polynomial 0xEDB88320), bit by bit.
fn crc32(data: &[u8]) -> u32 {
let mut crc = 0xFFFF_FFFFu32;
for &byte in data {
crc ^= byte as u32;
for _ in 0..8 { crc = (crc >> 1) ^ (0xEDB8_8320 & 0u32.wrapping_sub(crc & 1)); }
}
!crc
}
// Little-endian writes and bounds-checked reads, so the format is the same on every machine.
fn put_u32(out: &mut Vec<u8>, value: u32) { out.extend_from_slice(&value.to_le_bytes()); }
fn put_f32(out: &mut Vec<u8>, value: f32) { out.extend_from_slice(&value.to_le_bytes()); }
struct Reader<'a> { bytes: &'a [u8], at: usize, ok: bool }
impl Reader<'_> {
fn u32(&mut self) -> u32 {
match self.bytes.get(self.at..self.at + 4) {
Some(four) => { self.at += 4; u32::from_le_bytes(four.try_into().unwrap()) }
None => { self.ok = false; 0 } // never read past the end
}
}
fn f32(&mut self) -> f32 { f32::from_bits(self.u32()) }
}
fn encode(world: &World, version: u32) -> Vec<u8> {
let mut payload = Vec::new();
put_u32(&mut payload, world.len() as u32);
for entity in world {
put_u32(&mut payload, entity.id);
put_f32(&mut payload, entity.x); put_f32(&mut payload, entity.y); put_f32(&mut payload, entity.health);
if version >= 2 { put_f32(&mut payload, entity.stamina); }
put_u32(&mut payload, entity.target_id);
}
let mut file = Vec::new();
put_u32(&mut file, MAGIC); put_u32(&mut file, version); put_u32(&mut file, payload.len() as u32);
put_u32(&mut file, crc32(&payload));
file.extend_from_slice(&payload);
file
}
// Validate everything before trusting anything; on failure, say which check failed.
fn decode(file: &[u8]) -> Result<World, String> {
let mut header = Reader { bytes: file, at: 0, ok: true };
let (magic, version, length, stored_crc) = (header.u32(), header.u32(), header.u32(), header.u32());
if !header.ok { return Err("shorter than a header".into()); }
if magic != MAGIC { return Err("not a save file".into()); }
if version == 0 || version > VERSION {
return Err(format!("unknown version {version}")); // a newer build wrote it: refuse rather than misread
}
if length as usize != file.len() - HEADER_SIZE { return Err("length mismatch (truncated?)".into()); }
if crc32(&file[HEADER_SIZE..]) != stored_crc { return Err("checksum mismatch".into()); }
let mut input = Reader { bytes: file, at: HEADER_SIZE, ok: true };
let count = input.u32() as usize;
let bytes_per_entity = if version >= 2 { 24 } else { 20 };
if count.checked_mul(bytes_per_entity).map(|bytes| bytes + 4) != Some(length as usize) { // before allocating
return Err("payload doesn't match its own counts".into());
}
let mut world = World::with_capacity(count);
for _ in 0..count { // pass 1: create every object
let id = input.u32();
let (x, y, health) = (input.f32(), input.f32(), input.f32());
let stamina = if version >= 2 { input.f32() } else { 100.0 }; // migrate: v1 had no stamina
let target_id = input.u32();
world.push(entity(id, x, y, health, stamina, target_id));
}
let index_of: HashMap<u32, usize> = world.iter().enumerate().map(|(i, entity)| (entity.id, i)).collect();
for entity in &mut world { // pass 2: re-resolve references by id
entity.target = index_of.get(&entity.target_id).copied(); // missing target: drop the reference
}
Ok(world)
}
// Write bytes to a file and force them to the storage device before returning.
fn write_durably(path: &str, bytes: &[u8]) -> io::Result<()> {
let mut file = File::create(path)?;
file.write_all(bytes)?;
file.sync_all() // fsync on Linux, F_FULLFSYNC on Apple, FlushFileBuffers on Windows
}
// fs::rename replaces `to` by renaming: rename(2) on Unix, MoveFileExW on Windows (without
// MOVEFILE_WRITE_THROUGH, so there the rename may reach the disk after save() returns).
fn sync_directory() {
#[cfg(unix)] // POSIX: make the renames themselves durable
if let Ok(dir) = File::open(".") { let _ = dir.sync_all(); }
}
fn save(world: &World) -> io::Result<()> {
write_durably("save.tmp", &encode(world, VERSION))?;
let _ = fs::rename("save.dat", "save.bak"); // keep the previous save; fails harmlessly the first time
let result = fs::rename("save.tmp", "save.dat");
sync_directory();
result
}
// Test helper: overwrite a file with no durability, to damage it on purpose.
fn overwrite(path: &str, bytes: &[u8]) { fs::write(path, bytes).expect("write test file"); }
// Try the newest save, then the backup.
fn load() -> Option<World> {
for path in ["save.dat", "save.bak"] {
match decode(&fs::read(path).unwrap_or_default()) {
Ok(world) if world.len() < 2 => return Some(world), // valid, but no player and guard to print
Ok(world) => {
let guard_target = if world[1].target.is_some() { "the player" } else { "none" };
println!(" loaded {path}: player health {:.0}, stamina {:.0}; guard's target: {guard_target}",
world[0].health, world[0].stamina);
return Some(world);
}
Err(error) => println!(" {path} rejected: {error}"),
}
}
None
}
fn main() -> io::Result<()> {
println!("crc32(\"123456789\") = {:08X} (expected CBF43926)", crc32(b"123456789"));
let mut world = vec![entity(1, 0.0, 0.0, 100.0, 55.0, NO_TARGET), entity(2, 5.0, 3.0, 80.0, 60.0, 1),
entity(3, -4.0, 8.0, 250.0, 0.0, 1)];
save(&world)?; // first save
world[0].health = 70.0; // the player takes a hit
save(&world)?; // second save; the first becomes save.bak
println!("clean load:"); load();
// Damage the file on purpose (the game itself only ever writes through save()).
let mut damaged = fs::read("save.dat")?;
damaged[30] ^= 0x01; // one flipped bit on disk
overwrite("save.dat", &damaged);
println!("one bit flipped:"); load();
let mut truncated = fs::read("save.bak")?;
truncated.truncate(40); // a torn write from an in-place save
overwrite("save.dat", &truncated);
println!("truncated:"); load();
world[1].target_id = 9; // a reference to an object that was never saved
overwrite("save.dat", &encode(&world, 1)); // a v1 save: no stamina field
println!("v1 file with a dangling reference:"); load();
Ok(())
}
The output walks through four loads. The clean load finds health 70 and stamina 55. One flipped bit fails the checksum, so the loader falls back to save.bak, the previous save, with health 100. A file cut off at 40 bytes fails the length check and falls back the same way. A v1 file passes every check and loads with stamina defaulted to 100, and the guard's reference to object 9, which was never saved, comes back as no target. A crash between the two renames leaves no save.dat at all; the loader then falls back to save.bak, and the complete save.tmp is left behind for a smarter loader to try.
The widget is that file, byte for byte. Damage it one way, then press load:
- Error reporting to the player. The loader prints why a file failed. A game tells the player the newest save was damaged and that it loaded the previous one, and keeps the damaged file for a bug report.
- More than one backup. One
save.bakcovers one bad save. Rolling several, or keeping one per save slot, covers a migration bug that writes several bad saves in a row. This program also rotatessave.datintosave.bakwithout checking it, so after a fallback load the next save replaces the good backup with the damaged file; a game validatessave.datbefore rotating it, or keeps the damaged file under another name. - Per-section checksums and a schema. One CRC over the whole payload says the file is damaged but not where. Large saves checksum each section and tag each field, as §5's tagged formats do.
- Compression and encryption. Put both after encoding and before the checksum, so the CRC covers what's actually on disk.
- Platform save APIs. Consoles and some stores provide their own save storage with their own commit rules; the temp-and-rename dance is for plain files.
07Saving live
An autosave fires mid-fight. If it captures the player's health after this frame's damage but the enemy's attack timer before it, the save describes a moment that never happened. The save must be a consistent snapshot taken at a tick boundary, not mid-update with half the systems stepped. Capture the state at the end of a frame, then write it off-thread so the I/O does not hitch the game. The hard case is streaming: some cells are streamed out, their live objects do not exist in memory, so the snapshot merges the live objects' current state with the last-known persisted delta for the streamed-out regions.
Worked example
Say the authoritative state is 2 MB. Copying it at the end of a frame at roughly 10 GB/s takes about 0.2 ms, which the frame can absorb. Writing those 2 MB to a drive at roughly 100 MB/s takes about 20 ms, more than a whole 60 fps frame. So the copy happens on the game thread at the tick boundary, and the write happens on another thread from the copy.
This is the clean contrast with the netcode tutorial. Deterministic lockstep ships only inputs, so every machine must compute bitwise-identical results to stay in sync[19]. A save serializes state directly, so loading restores the world wholesale, no replay, no determinism requirement. The one exception is a replay-style save that stores an input log instead of state: that inherits lockstep's exact determinism constraint. State snapshots and command logs are different designs with different failure modes; do not conflate them.
The production save system also needs: the streamed-out-region delta merge in full, save-thread synchronization, encryption and anti-tamper for competitive titles, cloud-save conflict resolution, per-chunk checksums, endianness byte-swapping for cross-platform saves (a PC save loaded on a console), and compaction of dead change-forms so a long playthrough's save does not grow without bound. The Asset Pipeline tutorial owns the byte-level format and endianness machinery this shares.
Wrong answers, and why: addresses cannot be pinned across runs and a bare index is fragile, so references are saved as generational handles and fixed up on load; an appended field needs a version branch or a tagged format, and refusing old saves discards progress; a checksum detects torn-write corruption but only an atomic temp-rename prevents it; fsync is about durability, not atomicity; and a change record for a deleted object is skipped, not fatal.
08Pitfalls
09What's next
That completes the persistence pair with Open-World Streaming: the world stays resident around the player, and the changes they make to it survive a crash, a quit, and a patch that changed the format. The full Build a Game Engine path, foundations through the 3D capstone, is on the series hub.
- WizardCell. "Unreal Engine Persistent Data Compendium." wizardcell.com. Quotes the
USaveGameclass description, and warns against persisting runtime data in GameMode, GameState and GameSession, which are rebuilt with each level. - Noel Llopis. "Managing Data Relationships" (2010), Games from Within. gamesfromwithin.com. Pointer values change run to run; the raw-pointer / array-index / handle encoding taxonomy and why handles win for serialization.
- "Automatic Serialization in C++ for Game Engines" (2022), IndieGameDev. indiegamedev.net. The two-pass load: instantiate all objects, then fix up references.
- cereal documentation, "Pointers." uscilab.github.io/cereal. Raw pointers are refused; shared references behind a
shared_ptrare serialized once and re-linked on load. - Unity Technologies. Unity Manual, "Serialization rules" and "Custom serialization." docs.unity3d.com, docs.unity3d.com. Inline serialization can't represent null, shared references or cycles;
[SerializeReference]can. The custom-serialization example flattens a tree into nodes with a child count and first-child index (the id-reference pattern by hand). - sinbad/SPUD (Steve's Persistent Unreal Data). github.com/sinbad/SPUD. Runtime-spawned actors are re-spawned on load rather than serialized as live objects.
- Christian Haase. "An Unreal Engine saving & loading system." medium.com. Delta serialization against the class default, only changed-from-default properties are written.
- UESP. "Skyrim Mod:Save File Format." en.uesp.net. Change forms (deltas over the baked plugin data), the plugin list stored in each save, and the packed three-byte RefID whose type bits distinguish plugin forms from forms created at runtime (plugin index
0xFF). - Epic Games. "Versioning of Assets and Packages in Unreal Engine." dev.epicgames.com. Engine and GUID-based custom version numbers for independent format tracks; a removed field is ignored and a new field holds its default.
- Protocol Buffers. "Language Guide (proto3)." protobuf.dev. Unknown fields are ignored and missing fields defaulted; never reuse a field number and reserve deleted ones.
- FlatBuffers. "Schema Evolution." flatbuffers.dev. New fields only at the end of a table; fields are deprecated, never deleted; changing a default rewrites the meaning of prior data.
- Elijah Lopez. "Avoid Data Corruption by Syncing to the Disk." blog.elijahlopez.ca. The write-temp, fsync, atomic-rename sequence, and why
flush()is not enough withoutfsync. - P. Deutsch. RFC 1952, "GZIP file format specification version 4.3." 1996. rfc-editor.org. The CRC-32 the listing computes (reflected polynomial
0xEDB88320, pre- and post-inverted), with sample code. - The Open Group. POSIX.1-2024, "rename." pubs.opengroup.org. Renaming over an existing file is atomic: the new name refers to either the old or the new file throughout.
- Linux man-pages. "fsync(2)." manpages.debian.org. fsync flushes a file's data and metadata, but the directory entry needs an fsync on the directory.
- Microsoft. "MoveFileExW function." learn.microsoft.com.
MOVEFILE_REPLACE_EXISTING, andMOVEFILE_WRITE_THROUGH: the call returns once the move is on disk. - Apple. "fsync(2)," Mac OS X man pages. developer.apple.com.
fsyncmoves data to the drive, but the drive may not write it out for some time;F_FULLFSYNCasks the drive to flush its buffered data to permanent storage. - The Rust standard library.
std::fs::renameandFile::sync_alldocs, and the std source at 1.97.0. doc.rust-lang.org; sync_all; unix.rs; windows.rs.fs::renameisrenameon Unix andMoveFileExWwith onlyMOVEFILE_REPLACE_EXISTINGon Windows;sync_allisfsync,fcntl(F_FULLFSYNC)on Apple targets, andFlushFileBufferson Windows. - SnapNet. "Netcode Architectures Part 1: Lockstep." snapnet.dev. Lockstep ships only inputs and requires bitwise-identical simulation; a state-snapshot save restores directly and needs no determinism.