Build a Game Engine · Resources

Asset Pipeline & Serialization

Your game decodes PNGs every time a level loads. Renaming one texture breaks every material that pointed at it, last month's files stop loading after you add a field, and a header that loads fine on PC crashes on a phone. All four are asset-pipeline problems. The asset pipeline is the offline machine that turns authoring files like a .png or an .fbx into blobs the engine can load fast, often in place. This tutorial builds that machine in C++ and Rust: the (the offline conversion from source file to runtime data), a versioned binary format, stable references, a dependency graph, hot reload, and the alignment rules that make , using the file's bytes in place with no parse, actually work.

Time~55 min LevelMid PrereqsHelpful, not required: File Streaming (how bytes get off disk) and Bit Shifting (for the binary layout). StackC++ & Rust

What you'll be able to do by the end

Explain why engines cook source assets offline, and what a cooked texture looks like next to its PNG.
Lay out a binary header with fixed-width fields, explicit padding and a documented byte order, and read it without an alignment crash.
Version a format and migrate old files on load, or choose schema evolution instead.
Reference assets by GUID on disk and by generational handle at runtime, and re-cook only what an edit touches.
Package assets with an index, hot-reload them safely, and know the conditions zero-copy loading needs.

01Why a pipeline

Every time a level loads, your engine decodes its PNGs on the CPU, builds their mip chains and converts them for the GPU, and does exactly the same work again on the next load. Source formats are built for authoring and interchange, not for runtime. A PNG must be decoded before a GPU can sample it; a WAV must be parsed; an FBX file, Autodesk's 3D interchange format, carries a whole editor scene that has to be converted. Even glTF, Khronos's open 3D format, whose binary buffers are designed to reach the GPU with little processing, stores textures as PNG or JPEG and uses its own vertex layout rather than the engine's. Unreal puts it plainly: it stores content "in particular formats which it uses internally, such as PNG for texture data or WAV for audio. However, this content needs to be converted to different formats for the various platforms"^[2].

So an engine ships a second representation: cooked data, laid out for fast loading and the target hardware. The conversion also generates mip chains (each texture's progressively half-size copies), transcodes textures to GPU block formats (fixed-rate compressed formats the GPU samples directly), and lays the data out for the platform.

Worked example

Decoded, a 2048 × 2048 albedo PNG is 2048 × 2048 × 4 bytes = 16 MiB of RGBA pixels, plus another third for its mip chain, and the CPU has to produce all of it before the GPU sees a pixel. The cooked BC7 version is 1 byte per pixel: 4 MiB, or about 5.3 MiB with mips. The GPU samples it directly, so at load the engine reads 5.3 MiB and hands it over, with no decode and no mip generation.

There is no single right format

The design space splits along axes: text vs binary, parse vs zero-copy, a version tag vs schema evolution, GUIDs vs paths. The right choice depends on your platforms, your iteration speed, and whether data crosses a version boundary at runtime. This tutorial builds one coherent point in that space and names the tradeoffs at each axis.

02Bake vs runtime load

Iterating on a level means loading it hundreds of times, so any work the load repeats is paid hundreds of times. The cook (or bake) is an offline transform from source to a runtime-ready, often platform-specific artifact: import → process → cook → package, which Gregory calls the asset conditioning pipeline^[1]. Heavy work happens once, offline; the runtime does a light load and fixup, a small pass that turns stored offsets into real pointers^[11]. The Bitsquid engine compiled its JSON source data into runtime data in a separate folder, and once running, the engine "never sees the generic data folder so it can't 'cheat' and read anything there"^[3].

Cooked is not the same as compressed

Cooking is transform plus layout; compression is a separate, optional packaging step (its own tutorial). A cooked blob can be uncompressed. And "cook once, run anywhere" fails for platform-specific cooks: texture block format, endianness, and alignment can differ per target, which is why Unreal and Unity both build per target platform^[2]^[14]. Keep import (source → engine intermediate) distinct from cook (intermediate → shipped package).

Worked example

Follow a 1024 × 1024 hero.png through the stages. Import decodes it to 4 MiB of RGBA. Process adds the mip chain (5.3 MiB) and compresses it to BC7, 1.3 MiB with mips. Cook writes that as a runtime blob with a small header, and package appends it to assets.pak at some offset, recorded in the index. At runtime, load is one 1.3 MiB read from that offset. Everything before it ran once, on the build machine.

One asset flows through the pipeline below; hit "edit the source" to trigger a hot re-cook of just that asset:

03Binary serialization

The tools write a cooked header on a PC, and the same bytes load fine there but crash on an ARM phone or read as nonsense on another machine. A cooked format is binary. Three fundamentals decide whether it loads correctly and quickly: byte order, alignment, and what you're allowed to put in it.

Endianness. Endianness is the order a multi-byte number's bytes are stored in. Pick and document one. Most targets a game cares about are little-endian, which is why formats standardize on it: glTF mandates that "all buffer data... MUST use little endian byte order"^[4]. "Little-endian everywhere" is still an over-claim (network byte order is big-endian, and some hardware is too), so the format must declare its order rather than assume.
Alignment. A type with alignment N must sit at an address that's a multiple of N; the compiler pads between fields to satisfy it. Use fixed-width types (<cstdint>) and lock the layout with static_assert, a compile-time check, on size and alignment.
POD only. To write a struct straight to disk it must be plain data, no virtual functions, because a vtable pointer (the hidden pointer to a class's table of virtual functions) is a runtime address that means nothing in a file^[11].

Worked example

The header in §4 has a 4-byte magic at offset 0, a 4-byte version at 4 and a 2-byte flags at 8. The next field, the 4-byte payloadBytes, needs a multiple of 4, so the compiler inserts 2 bytes of padding and places it at 12. The 8-byte contentHash lands at 16, and the struct is 24 bytes with 8-byte alignment. Byte order matters just as much: version 3 is stored little-endian as the bytes 03 00 00 00, and a big-endian machine that reads them without swapping sees 50,331,648.

A misaligned cast is undefined behavior, not just slow

Casting a byte buffer to a struct only works if the buffer satisfies the struct's alignment. On x86-64 an unaligned scalar load works, typically at little or no cost unless it splits a cache line (the 64-byte unit memory moves in on those CPUs) or a page. On ARM it depends on the core and the instruction: older 32-bit cores and some instructions (multi-register loads, atomics) raise an alignment fault, which Linux and Android deliver as a bus error (SIGBUS). And the language rules forbid it regardless of the CPU: in C, even forming the misaligned pointer is undefined behavior, which the compiler may assume never happens, and in C++ reading through it is^[8]. Copy with memcpy (or std::bit_cast from an aligned byte array) when alignment isn't guaranteed. In Rust, use #[repr(C)] for a defined field order you can serialize; the default #[repr(Rust)] layout is unspecified.

The inspector lays out an asset header byte by byte, with padding highlighted. Pick an older file version to see its on-disk layout next to the current struct it migrates into:

Each colored block is a field on a byte ruler numbered from 0, with padding marked "pad". Click v1, v2 and v3, then tick "force pack(1)". The 16-bit flags field leaves a 2-byte gap so the next 32-bit field stays 4-aligned, and the v3 struct is 24 bytes. A v1 file has no flags, so its payloadBytes sits at offset 8 and 4 pad bytes keep the hash 8-aligned; the loader reads that layout and fills flags with its default. A v2 file already has the v3 layout, so only its version number changes. Toggle "force pack(1)" and the padding vanishes: the struct shrinks to 22 bytes with payloadBytes at offset 10 and the hash at 14, both misaligned: undefined behavior in C++ everywhere, tolerated by x86, a fault on some ARM cores and instructions.

04Versioning & migration

You add a flags field to the header, and every file the artists saved last month fails to load. Data outlives the code that wrote it, so a format must evolve. Two valid strategies:

Version tag + migration. The header carries a version; the loader switches on it and migrates old data up to the current in-memory form. Total control, but you write every migration.
Schema evolution. Compatibility is built into the format's structure instead. Protocol Buffers key fields by number, not name: the number "cannot be changed once your message type is in use," old binaries skip fields they don't recognize (and keep them when re-serializing), and a deleted field's number must be reserved so it's never reused^[7]. FlatBuffers uses per-table vtables so adding fields at the end and deprecating (never deleting or reordering) keeps forward and backward compatibility^[6].

Bumping the version doesn't make old data readable

A version tag only detects which layout you have; you still need a loader for each old layout and a default-fill rule for fields it lacked, or old files fail to load. The version field has to sit at the same offset in every version, so it can be read before the rest of the layout is known. And these are different mechanisms: a version tag plus migration converts old to new; schema evolution makes one layout tolerate added and removed fields without a migration step. Forward compatibility (old code reads new data) and backward compatibility (new code reads old data) are also distinct; state which you're claiming.

Worked example

A v1 file arrives. The loader reads 4 bytes at offset 4, the one place every version keeps its version number, and gets 1. In v1 there was no flags field, so payloadBytes sits at offset 8, then 4 bytes of padding, then the hash at 16: still 24 bytes. The loader copies those bytes into the old struct, then builds the current one field by field, with flags set to its default of 0 and the version rewritten to 3. A v2 file already has the v3 layout, so only its version number changes.

A versioned header, loaded and migrated from any supported version

#include <cstddef>
#include <cstdint>
#include <cstring>

// The current (v3) header, identical on disk and in memory.
struct AssetHeader {              // fixed-width, no virtuals, safe to write as bytes
    uint32_t magic;               // sentinel: catch a wrong/corrupt file early
    uint32_t version;             // drives the migration switch
    uint16_t flags;               // added in v2; a 2-byte pad follows so the next field is 4-aligned
    uint32_t payloadBytes;
    uint64_t contentHash;         // 8-byte aligned at offset 16
};
static_assert(sizeof(AssetHeader) == 24, "layout drifted");
static_assert(alignof(AssetHeader) == 8, "alignment drifted");

// v1 had no flags field, so payloadBytes sat at offset 8.
struct AssetHeaderV1 {
    uint32_t magic;
    uint32_t version;
    uint32_t payloadBytes;
    uint64_t contentHash;         // 4 pad bytes before it keep it 8-aligned
};
static_assert(sizeof(AssetHeaderV1) == 24, "v1 layout drifted");

// Read whichever header version the file holds into the current struct.
// memcpy rather than a pointer cast: the file buffer need not be aligned.
// The file is little-endian and so is every current PC and console CPU, so
// the bytes are copied as-is; a big-endian host would have to swap them.
bool loadHeader(const std::byte* bytes, size_t size, AssetHeader& out) {
    if (size < 8) return false;
    uint32_t version;              // same offset (4) in every version
    std::memcpy(&version, bytes + offsetof(AssetHeader, version), sizeof version);
    switch (version) {
        case 1: {
            if (size < sizeof(AssetHeaderV1)) return false;
            AssetHeaderV1 old;
            std::memcpy(&old, bytes, sizeof old);
            out = AssetHeader{old.magic, 3, 0, old.payloadBytes, old.contentHash};  // flags: default 0
            return true;
        }
        case 2:                    // same layout as v3; only the number changes
        case 3:
            if (size < sizeof(AssetHeader)) return false;
            std::memcpy(&out, bytes, sizeof out);
            out.version = 3;
            return true;
        default:
            return false;          // unknown or newer: refuse
    }
}

use bytemuck::{Pod, Zeroable};

// The current (v3) header, identical on disk and in memory.
#[repr(C)]                          // defined field order; #[repr(Rust)] is unspecified
#[derive(Clone, Copy, Pod, Zeroable)]
struct AssetHeader {
    magic: u32,
    version: u32,
    flags: u16,                     // added in v2
    _pad: u16,                      // explicit: derive(Pod) rejects implicit padding
    payload_bytes: u32,
    content_hash: u64,              // 8-byte aligned at offset 16
}
const _: () = assert!(core::mem::size_of::<AssetHeader>() == 24);

// v1 had no flags field, so payload_bytes sat at offset 8.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable)]
struct AssetHeaderV1 {
    magic: u32,
    version: u32,
    payload_bytes: u32,
    _pad: u32,                      // keeps content_hash 8-aligned, as the C++ compiler's padding does
    content_hash: u64,
}
const _: () = assert!(core::mem::size_of::<AssetHeaderV1>() == 24);

// Read whichever header version the file holds into the current struct.
// pod_read_unaligned copies, so the file buffer need not be aligned. Like the
// C++ version it reads the little-endian file in native byte order.
fn load_header(bytes: &[u8]) -> Option<AssetHeader> {
    let version = u32::from_ne_bytes(bytes.get(4..8)?.try_into().ok()?);  // same offset in every version
    match version {
        1 => {
            let old: AssetHeaderV1 = bytemuck::pod_read_unaligned(bytes.get(..24)?);
            Some(AssetHeader {
                magic: old.magic, version: 3, flags: 0, _pad: 0,   // flags: default 0
                payload_bytes: old.payload_bytes, content_hash: old.content_hash,
            })
        }
        2 | 3 => {                    // v2 has the v3 layout; only the number changes
            let mut header: AssetHeader = bytemuck::pod_read_unaligned(bytes.get(..24)?);
            header.version = 3;
            Some(header)
        }
        _ => None,                      // unknown or newer: refuse
    }
}

05References & dependencies

An artist renames granite.png to granite_01.png, and every material that used it renders with the missing-texture placeholder. Assets point at other assets, and how you encode that reference decides whether moving a file breaks the game.

GUID = stable identity. A global ID that survives move and rename. Unity stores one per asset in a sidecar .meta so it "can move or rename the asset without breaking anything"; lose the .meta and "any reference to that asset is broken"^[9]. A path, by contrast, breaks the moment a file moves.
Runtime handle ≠ GUID. At load, GUIDs resolve to compact runtime handles. A common shape is a generational handle (index + generation), so a freed slot reused by a new asset doesn't alias an old handle^[10]. This is the same pattern as entity IDs in the ECS (entity component system) and the handles in Memory Allocators.
The cooker records which assets each asset references; in the well-behaved case that forms a DAG, a directed acyclic graph whose arrows never loop back. Touch a source asset and it, plus every asset that depends on it, must re-cook.

Worked example

Slot 7 of the mesh table holds rock.mesh at generation 3, so the handle is (7, 3). The level unloads the rock; slot 7 is freed and reused for tree.mesh, and its generation becomes 4. A system still holding (7, 3) looks up slot 7, sees generation 4, and gets "stale handle" instead of silently drawing a tree where the rock was. The GUID in the level file never changed; only the runtime handle did.

In the widget, arrows point from an asset to the assets that use it, the direction a change spreads. Click an asset (or focus the graph and use the arrow keys and Enter) to "edit" it; the dirty set, the edited asset plus everything that depends on it, propagates downstream:

Green assets are clean and orange ones must re-cook; the line above the graph counts them. Editing shader.inc reads "dirties 4 dependents": the shader, the material, the mesh, and the prefab, everything downstream, while rock.png stays clean. Editing rock.png dirties 3. That's incremental cooking: touch one file, re-cook only what depends on it. The traversal keeps a visited set, so a reference cycle, which real projects do end up with, can't loop forever.

Wrong answers, and why: GUID and handle are opposite roles (persisted identity vs runtime index), not synonyms; and a cast crash that's platform-specific is an alignment/layout problem, not memory or mmap availability.

06Packaging

Your build has a file per asset, and startup spends its time opening them. Thousands of small files mean thousands of open() calls and seeks. A package concatenates cooked assets into a few archives plus an index (offset, size, and ID per entry), which cuts I/O overhead and lets the streamer pull contiguous data.

Unreal's .pak is "an archive file format... to store cooked content" with an index for its file system, and UE5's IoStore splits content (.ucas) from metadata (.utoc)^[13]; Unity's AssetBundle is a platform-specific archive grouping assets^[14]. Compression layers on here (per-entry or whole-archive) and is its own tutorial; the index plus contiguous layout is what makes the prioritized streaming from the File Streaming tutorial feasible.

Worked example

A package holds 10,000 assets. Give each index entry a 64-bit ID, a 64-bit offset and a 64-bit size: 24 bytes, so the whole index is 240,000 bytes, about 234 KiB, read once at startup. Loading an asset is then a lookup in that table and one read at a known offset in a file that's already open, instead of 10,000 separate opens.

07Hot reload

You nudge a material's roughness and want to see the change in the running game right away, without a restart and a level reload. The pipeline watches the source, re-cooks the changed asset and its dependents, and swaps the live copy without restarting. Bitsquid's tool sent a network message telling the engine to reload the changed file "nearly instantaneously"^[3].

The two things that make hot reload hard

Dangling references: if anything holds a raw pointer into the old asset's memory, the swap invalidates it. This is exactly why generational handles matter; a stale handle is caught on next access instead of dereferencing freed memory. In-flight uses: an asset can be mid-render or mid-job when the reload fires, so a naive free-and-replace is a data race. Production systems typically refcount (count the holders and free at zero) or double-buffer (publish the new copy with an atomic pointer swap and keep the old one alive), and defer the free until no reader is active. And dependents must re-cook, not just the touched file: editing a shader include must reprocess every material that uses it.

Worked example

A texture is reloaded during frame 120, while two draw calls recorded earlier in that frame still reference the old copy, so its count is 2. The reload publishes the new texture for everything that asks from now on. When the GPU finishes frame 120, both draw calls release the old copy, the count reaches 0, and only then is its memory freed. Frame 121 draws with the new texture.

08Zero-copy loading

Parsing a level file means reading every byte, building every object and allocating each one separately, all before the first frame. Zero-copy loading skips that: it lays the cooked file out so the bytes are the in-memory image, and the engine memory-maps it (asks the OS to map the file into its address space and read pages on first touch) and uses it in place, with no parse and no per-object allocation. Cap'n Proto, a serialization format whose wire format is its in-memory format, is built on that premise is that "the encoding is appropriate both as a data interchange format and an in-memory representation," so you can mmap a file and "the OS won't even read in the parts that you don't access"^[5]. FlatBuffers gives "access to serialized data without parsing/unpacking"^[6]; Rust's rkyv crate lays out archived versions of your types exactly as they sit in memory, with relative pointers (offsets from the pointer's own position) in place of heap pointers, so reading is a cast rather than a parse^[15].

Worked example

A mesh record at file offset 0x100 starts with its vertex-array reference, stored as 0x40: an offset from the reference's own position. The vertices therefore sit 0x40 bytes past that field, at file offset 0x140, wherever the OS maps the file, and the reference works without a fixup. A raw pointer saved by the tools would hold an address from the tools' own process, which means nothing in the game's.

Zero-copy is not free, and not unconditional

It requires the cooked layout to match the target's struct layout and padding, its alignment, and its endianness. Violate any one and the cast is wrong or undefined. The buffer must be aligned: mmap returns page-aligned memory, starting on a page boundary (fine), but a plain read into a Vec<u8> or malloc region is not aligned for an arbitrary type, so rkyv ships an AlignedVec for exactly this. Because the layout is the target's layout, a blob cooked for one platform is not portable to another, the same per-platform-cook point from §2. The layout work moves into the cook, and the price is layout rigidity. References become offsets or relative pointers (a raw pointer's value changes every run^[10]), patched once at load or resolved on access^[12]. C++ adds an object-lifetime rule on top: the bytes have to become objects before they're read as structs. C++20's implicit object creation covers a byte array used as storage for implicit-lifetime types (plain structs like this header qualify), and C++23's std::start_lifetime_as states it explicitly.

09Pitfalls

"Cooked means compressed"Cooking is transform + layout; compression is a separate packaging step.

Crash on ARM, fine on x86A misaligned cast of a cooked blob: undefined behavior everywhere, and a fault on the ARM cores and instructions that require alignment.

Old files won't loadBumped the version but wrote no loader for the old layout or default-fill rule.

Reference breaks on renameReferencing by path instead of a stable GUID.

Stale asset after reloadHolding a raw index/pointer instead of a generational handle.

Cooker re-cooks everythingNo dependency graph, or no visited-set so a reference cycle loops.

Rust struct serializes wrong#[repr(Rust)] field order is unspecified; use #[repr(C)].

zero-copy garbage on another platformThe layout is the target's; re-cook per platform (endian/align/format).

10What's next

Packages are the natural home for Data Compression, the next module: how LZ and entropy coding shrink them, and why decompression speed, not just ratio, decides load times. After that comes the renderer, which consumes the cooked meshes and textures this pipeline produces. The full path is on the series hub.

Jason Gregory. Game Engine Architecture, 3rd ed. (2018), "Resources and the File System." gameenginebook.com. Resource managers and resource GUIDs, the asset conditioning pipeline, the offline-tools-vs-runtime split.
Epic Games. "Cooking Content in Unreal Engine." dev.epicgames.com. Internal PNG/WAV converted to platform formats; per-target cooked output.
Niklas Frykholm (Bitsquid). "Our Tool Architecture" (2010). bitsquid.blogspot.com. JSON source compiled to runtime data in a separate folder (the "can't cheat" line is from the author's reply in the comments); network-message hot reload.
The Khronos Group. glTF 2.0 Specification. registry.khronos.org. Buffer data must be little-endian; the GLB binary container and its chunk alignment.
Kenton Varda. Cap'n Proto. capnproto.org. The encoding is also the in-memory representation; mmap and access in place with no parse step.
Google. FlatBuffers documentation. flatbuffers.dev. Access serialized data without parsing; vtables for forward/backward schema compatibility.
Google. Protocol Buffers, proto3 language guide. protobuf.dev. Field numbers are the wire identity and can't change; reserved on delete; unknown fields skipped and preserved.
Adrien Guinet (Quarkslab). "Unaligned accesses in C/C++: what, why and solutions to do it properly" (2018). blog.quarkslab.com. Misaligned pointer conversion is UB in the C and C++ standards; x86 tolerates it; an alignment fault on an ARM Android target; the memcpy fix.
Unity Technologies. "Asset Metadata." docs.unity3d.com. The .meta sidecar GUID; move/rename safety; losing the meta breaks references.
Noel Llopis. "Managing Data Relationships" (2010). gamesfromwithin.com. Pointers are unsafe to serialize; indices are safe; handles as (index, counter, type).
Noel Llopis. "Delicious Data Baking" (2009). Game Developer. gamedeveloper.com. The offline bake, load-in-place blobs, offset references, and the POD-only constraint.
Tom Hulton. "Load in Place data structures and Pointer Fixups" (2011). tomhulton.blogspot.com. Storing references as offsets and patching them at load in a single read.
Epic Games. "Packaging Projects" / IoStore. dev.epicgames.com. The .pak archive and index; UE5 IoStore .ucas/.utoc split.
Unity Technologies. "AssetBundles." docs.unity3d.com. A platform-specific archive grouping assets, built per target.
David Koloski. rkyv. rkyv.org. Total zero-copy: archived types laid out exactly as in memory, read in place; aligned buffers (AlignedVec).

01Why a pipeline

02Bake vs runtime load

03Binary serialization

04Versioning & migration

05References & dependencies

06Packaging

07Hot reload

08Zero-copy loading

09Pitfalls

10What's next

See also