SIMD
from Scratch

01What SIMD buys you, and what it costs

A modern desktop CPU core can issue two 256-bit floating-point multiply-adds per cycle. At 4 GHz, one core can produce 64 billion single-precision FMAs per second from a single hardware thread^[1]. The same core, running the same kernel one scalar element at a time, peaks at one-eighth of that: the two FMA ports issue at the same rate either way, but each scalar FMA produces one lane instead of eight.

In a shipping engine the gap usually shows up as headroom rather than as a peak throughput number. A scalar particle tick can fall to a fraction of its cost once the data is in SoA form and the integration is one vfmadd231ps per axis per eight particles; the same holds for skinning, instance-transform passes, frustum and occlusion culling, and the math-heavy parts of physics narrow-phase. The realized speedup varies with the kernel and the layout. The vector form processes 8 or 16 records per instruction while the loop overhead stays roughly fixed, and the time saved goes to other systems or more entities.

SIMD only pays off when:

• Your data is laid out for it. A vector load picks up sixteen contiguous bytes (SSE) or thirty-two (AVX); it does not pick up four scattered x components from four scattered Particle records. Whether the inputs are in SoA or AoS form usually decides whether a SIMD rewrite is a win. §4 is the long answer.
• The work has lane-level parallelism. A floating-point sum where each iteration adds into the previous result is one serial chain, bound by the add's latency. Vectorizing it means regrouping the sum into independent partial sums, which the compiler won't do under strict IEEE-754 rules. §7.
• Branches in the inner loop are rare or vectorizable. A vector can't take a branch in some lanes and not others, so the SIMD form computes both sides of the branch and blends the results. AVX-512's mask registers are the first x86 SIMD with first-class predication. §8.
• The compiler did not already do it for you. Modern autovectorizers handle clean array maps on contiguous data, and floating-point reductions when allowed to reorder them. Hand-rolled intrinsics are the right answer when the compiler bailed out on the pattern, not when it didn't. §13.

The ceiling on the speedup is the lane count: for float data, 8× on AVX/AVX2 and 16× on AVX-512. Real kernels land below it, often well below, because layout conversion, tail handling, shuffles, and memory bandwidth each take a share. The rest of this tutorial makes those costs visible.

02A short history of x86 vector extensions

x86 SIMD is nearly thirty years of extensions, each layered on the previous one. Reading what your compiler emits in 2026 means knowing which generation a given mnemonic comes from and what hardware it requires.

The practical state in 2026: a portable x86-64 binary can assume SSE2. -march=x86-64-v3 (the x86-64 psABI level that adds AVX2, FMA, BMI1/2, and a few others) matches the PS5 / Xbox Series feature set and is the natural PC target for code shared with those builds. AVX-512 stays opt-in on x86 client and is standard on current server parts. Arm NEON is the parallel story for Switch, Switch 2, every modern Android phone, Apple Silicon, and Windows on Arm; §15 is the bridge.

03The lane model: how a SIMD register actually works

A SIMD register holds several values of the same type, in fixed positions called lanes. One instruction operates on every lane at the same time. The instruction's mnemonic encodes the lane element format: ps for packed single-precision floats, pd for packed doubles, b/w/d/q for packed 8 / 16 / 32 / 64-bit integers. The scalar variants are ss (scalar single) and sd (scalar double); they touch only the low lane and leave the rest unchanged^[9].

The same physical register file appears at three widths, and each width is a strict prefix of the next:

The lower 128 bits of ymm0 is xmm0. The lower 256 bits of zmm0 is ymm0. AVX-512-capable cores extend the architectural register count to thirty-two (xmm0–xmm31, etc., reachable only through AVX-512's EVEX encoding); pre-AVX-512 cores see sixteen^[2].

The widget below shows one instruction at the three widths. vaddps on a YMM register adds eight pairs of single-precision floats in one μop; on a ZMM register it adds sixteen. Every lane pulses at once because the hardware computes all of them in the same operation:

Two facts about the lane model that beginners get wrong, and that the rest of this tutorial depends on:

1. A scalar op costs the same as a packed one. vaddss xmm0, xmm1, xmm2 adds the low single-precision floats of xmm1 and xmm2; the upper three lanes of the result are copied from xmm1 (the VEX form copies them from the first source; legacy SSE addss leaves the destination's upper lanes in place). On Skylake a scalar addss has identical latency and throughput to a packed addps: both are one μop on the same two ports^[1]. Scalar SSE math is not cheaper than packed SSE math; it does one lane of work for the same cost.
2. Lane index is positional, not by name. A 256-bit YMM register is not a struct with named fields. Whether lane 0 holds x, lane 1 holds y, etc. is your convention. Most SoA code dedicates a whole register to one component (a register full of x values from many particles), not to one record with four components.

The intrinsic-level vocabulary, since the compiler hides the register names behind type-tagged values^[9]:

#include <immintrin.h>   // pulls in every Intel intrinsic header

// SSE / SSE2: 128-bit registers. Type tag carries the element format.
__m128   vec4Floats;   // 4 × float
__m128d  vec2Doubles;  // 2 × double
__m128i  vec16Bytes;   // 16 × int8, or 8 × int16, or 4 × int32, or 2 × int64

// AVX / AVX2: 256-bit registers.
__m256   vec8Floats;
__m256d  vec4Doubles;
__m256i  vec32Bytes;

// AVX-512: 512-bit registers.
__m512   vec16Floats;
__m512d  vec8Doubles;
__m512i  vec64Bytes;

// Mask register (AVX-512). One bit per lane; eight for __m512d, sixteen for __m512.
__mmask8  doubleMask;
__mmask16 floatMask;

// Load from memory. _ps = packed single, _pd = packed double, _si* = integer.
float* sourcePointer = ...;
__m256 loaded = _mm256_loadu_ps(sourcePointer);   // unaligned load: any address
__m256 loadedAligned = _mm256_load_ps(sourcePointer);  // requires 32-byte aligned source
_mm256_storeu_ps(destinationPointer, loaded);     // unaligned store

Aligned and unaligned loads have identical performance on every Intel core from Nehalem (2008) onward when the address actually is aligned; the cost only appears when an unaligned load crosses a cache-line boundary, which adds a few cycles^[10]. Modern code uses _mm256_loadu_ps by default and accepts the rare cache-line crossing as the cost of not having to manually align every buffer.

04AoS, SoA, and AoSoA: the layout decision that dominates everything

A scalar Particle in an engine is often laid out as:

struct Particle {     // 32 bytes per particle
  float positionX, positionY, positionZ;
  float velocityX, velocityY, velocityZ;
  float lifetime;
  float sizePixels;
};

Particle particles[100000];

// Update step. Scalar, one particle at a time.
void tickScalar(float deltaTime) {
  for (int i = 0; i < 100000; ++i) {
    particles[i].positionX += particles[i].velocityX * deltaTime;
    particles[i].positionY += particles[i].velocityY * deltaTime;
    particles[i].positionZ += particles[i].velocityZ * deltaTime;
    particles[i].lifetime  -= deltaTime;
  }
}

This is Array of Structures (AoS): one record at a time, every field of one particle contiguous. It is what a C++ programmer reaches for by default, and it is awkward to vectorize. To produce one SIMD register of positionX values, the CPU has to read positionX from particle 0, skip 28 bytes, read positionX from particle 1, skip 28 bytes, and so on. A 32-byte vector load at particle 0 picks up exactly one particle: one positionX and seven other fields. Filling a register with eight positionX values takes either eight separate loads or eight vector loads plus a transpose of roughly two dozen shuffles.

The cure is to transpose:

struct ParticleSoa {
  float* positionX;   // N floats, contiguous
  float* positionY;
  float* positionZ;
  float* velocityX;
  float* velocityY;
  float* velocityZ;
  float* lifetime;
  float* sizePixels;
  int    particleCount;
};

// Update step, AVX2. Processes 8 particles per loop iteration.
// Assumes particleCount is a multiple of 8 (pad the arrays; §5 covers tails).
void tickVector(ParticleSoa& particles, float deltaTime) {
  __m256 deltaTimeVector = _mm256_set1_ps(deltaTime);  // broadcast scalar to all 8 lanes
  for (int i = 0; i < particles.particleCount; i += 8) {
    // One load per stream, one FMA per stream. Eight particles updated per iteration.
    __m256 posX = _mm256_loadu_ps(particles.positionX + i);
    __m256 velX = _mm256_loadu_ps(particles.velocityX + i);
    posX = _mm256_fmadd_ps(velX, deltaTimeVector, posX);          // posX += velX * dt
    _mm256_storeu_ps(particles.positionX + i, posX);

    __m256 posY = _mm256_loadu_ps(particles.positionY + i);
    __m256 velY = _mm256_loadu_ps(particles.velocityY + i);
    posY = _mm256_fmadd_ps(velY, deltaTimeVector, posY);
    _mm256_storeu_ps(particles.positionY + i, posY);

    __m256 posZ = _mm256_loadu_ps(particles.positionZ + i);
    __m256 velZ = _mm256_loadu_ps(particles.velocityZ + i);
    posZ = _mm256_fmadd_ps(velZ, deltaTimeVector, posZ);
    _mm256_storeu_ps(particles.positionZ + i, posZ);

    __m256 lifeRemaining = _mm256_loadu_ps(particles.lifetime + i);
    lifeRemaining = _mm256_sub_ps(lifeRemaining, deltaTimeVector);
    _mm256_storeu_ps(particles.lifetime + i, lifeRemaining);
  }
}

Same arithmetic, eight particles per iteration. Every load pulls in eight useful floats, half a cache line. Every FMA does eight multiply-adds, and no shuffles are needed to get the data into lanes.

The widget below shows what one load picks up for a pass that reads only positionX, such as a sort key or a coarse cull. In AoS the 32-byte load fetches one particle's worth of mixed fields, of which one is wanted; in SoA the same load fetches eight consecutive positionX values. The array scrolls past a fixed box; the cells inside the box are the bytes a single vmovups brings in:

The third option is AoSoA: an array of blocks, each holding a SIMD width's worth of records in SoA form. A block of 8 particles' xs, then 8 ys, then 8 zs, then on to the next 8 particles. This keeps the lane-friendly access pattern inside a block while keeping one record's fields within a few cache lines of each other. Archetype-based ECS frameworks use the same idea at a larger grain: Unity's Entities package stores each component type as its own array inside fixed 16 KiB chunks^[11], which is SoA inside a chunk and AoSoA across chunks. Mike Acton's data-oriented design talk makes the general case for choosing layout by access pattern^[13].

Three rules to remember when picking a layout:

• SoA pays where a kernel touches few fields of many records. The particle tick uses position and velocity; the cull pass uses just position and boundsRadius. SoA wins decisively here because the unused fields don't get pulled into the cache at all.
• AoS pays where a kernel touches many fields of few records. Inserting an item into a sorted container, dereferencing a single picked entity for inspection, serializing one record to disk. Don't fight the layout for these.
• The decision is per-system, not per-codebase. A renderer that has one hot pass over many entities and occasional access to a few selected entities reasonably stores both: a SoA "transform pool" feeding the renderer, with a thin AoS handle for game logic. The translation lives at the system boundary and runs once per frame.

05Alignment, loads, and the cache-line crossing

Three load instructions cover almost all SIMD memory access^[9]:

• _mm256_loadu_ps (vmovups in the disassembly): load eight floats from any address. No alignment requirement.
• _mm256_load_ps (vmovaps): load eight floats from a 32-byte-aligned address. Faults if the address isn't aligned, which makes it a cheap runtime check that the layout invariant holds; when the address is aligned, the two forms perform identically on AVX-era hardware^[10].
• _mm256_stream_ps (vmovntps): a non-temporal store that bypasses the cache and requires a 32-byte-aligned address. Used when writing a large block that you know won't be read again before it's evicted (frame buffer composition, large texture uploads, particle render-output buffers). Wrong by default; right occasionally.

The performance cost that does show up in practice is the cache-line crossing. An x86 L1 cache line is 64 bytes. A 32-byte load that starts at offset 48 within a line straddles the boundary and needs a second L1 access. On Skylake-class cores a split-line load uses a load port twice, halving the load throughput for that μop and adding a few cycles of latency^[10]. In a hot loop that issues two loads per cycle, a stream of split-line loads can cut inner-loop throughput nearly in half. The cost comes from the line crossing itself, and vmovups pays it wherever the address happens to fall.

Tools to control alignment in C++^[14]:

// 1) Type-level alignment: every variable of this type gets the wider alignment.
struct alignas(32) ParticleBlock {
  float positionX[8];
  float positionY[8];
  float positionZ[8];
};

// 2) Variable-level: a single declaration gets the wider alignment.
alignas(64) float buffer[1024];

// 3) Heap allocation: plain malloc and new guarantee 16 bytes on x86-64 with glibc
//    and the Windows CRT. For wider alignment, ask for it explicitly.
//    std::aligned_alloc (C++17; the size must be a multiple of the alignment):
float* page = static_cast<float*>(std::aligned_alloc(32, 1024 * sizeof(float)));
std::free(page);
//    MSVC doesn't provide std::aligned_alloc; use _aligned_malloc / _aligned_free there.

//    new of an over-aligned type picks the aligned operator new automatically (C++17):
ParticleBlock* blocks = new ParticleBlock[128];   // 32-byte aligned; delete[] matches
delete[] blocks;

//    For a plain float buffer, call the aligned allocation function directly
//    and free it with the matching aligned delete:
float* page2 = static_cast<float*>(::operator new[](1024 * sizeof(float), std::align_val_t{32}));
::operator delete[](page2, std::align_val_t{32});

// 4) Telling the compiler an existing pointer is aligned. C++20 has a portable form;
//    older compilers use __builtin_assume_aligned (GCC/Clang) or __assume (MSVC).
float* alignedPointer = std::assume_aligned<32>(rawPointer);
// The compiler may now use aligned loads and skip a peel loop that reaches alignment.

Three practical alignment rules:

1. Align your big arrays to the SIMD width you target. 32 bytes for AVX/AVX2, 64 for AVX-512. Use alignas on the type or an aligned allocation. Don't hand-pad with extra bytes; the padding stops being right the moment someone changes the struct.
2. Use loadu by default. On aligned data it costs nothing extra (both instructions decode to the same μop). On misaligned data the cache-line-crossing penalty is paid only by the loads that actually cross a line, not by every loadu. Reach for load when alignment is an invariant you want checked: a misaligned pointer faults at the load instead of quietly running slower.
3. Pad your arrays to a multiple of the SIMD width. A loop over a 503-element array running 8 floats at a time has to handle the last 7 elements specially: either a scalar tail, an AVX-512 masked load, or a deliberate over-read into a padded buffer. Padding the array to 504 (a multiple of 8) and zeroing the tail is the cheapest of the three, at the cost of a few bytes of extra memory.

06Your first kernel: dot product of two arrays

A reduction that turns up in renderer, physics, and ML code: given two float arrays of length N, compute their dot product. The scalar form is one multiply-add per element:

float dotScalar(const float* aArray, const float* bArray, int elementCount) {
  float accumulator = 0.0f;
  for (int i = 0; i < elementCount; ++i) {
    accumulator += aArray[i] * bArray[i];
  }
  return accumulator;
}

The natural AVX2 form processes eight elements per iteration, accumulating into a single 256-bit vector and reducing to a scalar at the end:

float dotAvx2Naive(const float* aArray, const float* bArray, int elementCount) {
  __m256 accumulator = _mm256_setzero_ps();
  int i = 0;
  for (; i + 8 <= elementCount; i += 8) {
    __m256 aVector = _mm256_loadu_ps(aArray + i);
    __m256 bVector = _mm256_loadu_ps(bArray + i);
    // Fused multiply-add: accumulator += aVector * bVector, one rounding step.
    accumulator = _mm256_fmadd_ps(aVector, bVector, accumulator);
  }
  // Horizontal sum of the 8 lanes in `accumulator`. Reduce-and-shuffle tree.
  __m128 lowHalf  = _mm256_castps256_ps128(accumulator);            // lanes 0..3
  __m128 highHalf = _mm256_extractf128_ps(accumulator, 1);          // lanes 4..7
  __m128 sum128 = _mm_add_ps(lowHalf, highHalf);                    // 4 sums
  sum128 = _mm_hadd_ps(sum128, sum128);                             // 2 sums in low two lanes
  sum128 = _mm_hadd_ps(sum128, sum128);                             // 1 sum in low lane
  float totalSum = _mm_cvtss_f32(sum128);

  // Scalar tail for the last 0..7 elements that didn't fit a vector iteration.
  for (; i < elementCount; ++i) totalSum += aArray[i] * bArray[i];
  return totalSum;
}

Three working parts here, each of which is a SIMD pattern you'll reuse:

1. The main loop. One FMA per iteration. An FMA can take only one of its inputs straight from memory, so the compiler emits one plain load and folds the other into the FMA's memory operand: two loads plus one FMA per eight elements^[1].
2. The horizontal sum. Reducing an 8-lane vector to one scalar: add the two 128-bit halves, then fold the remaining four lanes. vhaddps is the SSE3 horizontal-add instruction; it's used twice to fold four lanes into one.
3. The scalar tail. Loop iterations that don't fit a full vector. For elements past the last i + 8 <= elementCount boundary, fall back to scalar. AVX-512 has an alternative using a masked load (§8) that handles the tail without a separate loop.

On Skylake this kernel runs roughly one iteration every 4 cycles in steady state, capped by the loop-carried dependency on accumulator. vfmadd231ps has a 4-cycle latency on Skylake; each iteration's FMA depends on the previous, so the loop is latency-bound, not throughput-bound^[1]. That is the serial-chain problem from §1: the FMA ports sit idle most of the time while the single dependency chain serializes.

The fix is in §7: use multiple accumulators to break the chain.

07Breaking the dependency chain

The bottleneck in the §6 kernel is that every iteration's FMA depends on the previous iteration's accumulator. The fix is to keep several independent accumulators and sum them at the end. How many you need is set by Skylake's FMA pipeline: 4-cycle latency, two-per-cycle throughput. Little's law gives the number of independent FMAs that have to be in flight at once to keep both ports busy: latency × throughput = 4 × 2 = 8. With eight independent accumulator chains, both FMA ports can start a new FMA every cycle, as long as nothing else runs out first^[1].

float dotAvx2Unrolled(const float* aArray, const float* bArray, int elementCount) {
  // Eight independent partial sums. Each is its own dependency chain;
  // with FMA latency 4 and throughput 2, eight in-flight FMAs saturate the ports.
  __m256 sum0 = _mm256_setzero_ps(), sum1 = _mm256_setzero_ps();
  __m256 sum2 = _mm256_setzero_ps(), sum3 = _mm256_setzero_ps();
  __m256 sum4 = _mm256_setzero_ps(), sum5 = _mm256_setzero_ps();
  __m256 sum6 = _mm256_setzero_ps(), sum7 = _mm256_setzero_ps();

  int i = 0;
  for (; i + 64 <= elementCount; i += 64) {
    // 8 independent FMAs per iteration; the CPU schedules across both FMA ports.
    sum0 = _mm256_fmadd_ps(_mm256_loadu_ps(aArray + i +  0), _mm256_loadu_ps(bArray + i +  0), sum0);
    sum1 = _mm256_fmadd_ps(_mm256_loadu_ps(aArray + i +  8), _mm256_loadu_ps(bArray + i +  8), sum1);
    sum2 = _mm256_fmadd_ps(_mm256_loadu_ps(aArray + i + 16), _mm256_loadu_ps(bArray + i + 16), sum2);
    sum3 = _mm256_fmadd_ps(_mm256_loadu_ps(aArray + i + 24), _mm256_loadu_ps(bArray + i + 24), sum3);
    sum4 = _mm256_fmadd_ps(_mm256_loadu_ps(aArray + i + 32), _mm256_loadu_ps(bArray + i + 32), sum4);
    sum5 = _mm256_fmadd_ps(_mm256_loadu_ps(aArray + i + 40), _mm256_loadu_ps(bArray + i + 40), sum5);
    sum6 = _mm256_fmadd_ps(_mm256_loadu_ps(aArray + i + 48), _mm256_loadu_ps(bArray + i + 48), sum6);
    sum7 = _mm256_fmadd_ps(_mm256_loadu_ps(aArray + i + 56), _mm256_loadu_ps(bArray + i + 56), sum7);
  }
  // Fold the eight accumulators together in pairs, then a horizontal sum.
  __m256 fold01 = _mm256_add_ps(sum0, sum1);
  __m256 fold23 = _mm256_add_ps(sum2, sum3);
  __m256 fold45 = _mm256_add_ps(sum4, sum5);
  __m256 fold67 = _mm256_add_ps(sum6, sum7);
  __m256 fold0123 = _mm256_add_ps(fold01, fold23);
  __m256 fold4567 = _mm256_add_ps(fold45, fold67);
  __m256 totalVector = _mm256_add_ps(fold0123, fold4567);

  __m128 sum128 = _mm_add_ps(_mm256_castps256_ps128(totalVector),
                             _mm256_extractf128_ps(totalVector, 1));
  sum128 = _mm_hadd_ps(sum128, sum128);
  sum128 = _mm_hadd_ps(sum128, sum128);
  float totalSum = _mm_cvtss_f32(sum128);
  for (; i < elementCount; ++i) totalSum += aArray[i] * bArray[i];
  return totalSum;
}

Each iteration now issues eight independent FMAs, and the latency chain no longer limits the loop. The FMA ports alone could now sustain two FMAs per cycle, 8× the single-accumulator loop's 0.25. This dot product hits a different wall first: every FMA consumes two fresh 32-byte loads, and Skylake has two load ports, so the loop tops out at one FMA per cycle, 8 elements per cycle, or 4× the single-accumulator version^[1]. Four accumulators already reach that limit (4 chains ÷ 4-cycle latency = one FMA per cycle), which is one reason many production dot products stop at four.

Eight accumulators pay off when an FMA needs at most one load, so the FMA ports become the limit: a sum of squares, a polynomial evaluated across many inputs, the matrix transform in §11 where one operand is a broadcast held in a register. They also help on cores with more load bandwidth, such as Golden Cove with three load ports. Sixteen accumulators rarely help on AVX2 hardware because there are only sixteen YMM registers and the extra accumulators start to spill. Ice Lake / Sunny Cove and AMD Zen 3 and Zen 4 have the same 4-cycle FMA latency and two-per-cycle throughput, so the same arithmetic applies^[1][6].

The widget models just the two FMA ports for a 32-FMA trace (256 elements) under both accumulator schemes, so it shows the latency effect in isolation:

-O3 -ffast-math (or specifically -fassociative-math, which GCC honors only together with -fno-signed-zeros and -fno-trapping-math) is what lets the autovectorizer perform this transformation for you. Without it, floating-point addition's non-associativity prevents the compiler from regrouping the sum^[15]. The compiler's autovectorization report (-fopt-info-vec on GCC, -Rpass=loop-vectorize on Clang) tells you when it succeeded.

A caveat: the reordered sum produces a slightly different result than the strict left-to-right sum. The two usually differ in the last few ULPs, more when large terms cancel^[15]; neither is more correct in general, but they're not bit-identical. If your engine needs deterministic, cross-platform reproducible results (replays, multiplayer lockstep, deterministic physics), the reordered sum can be a problem. The fix is to fix the reduction tree shape (always the same tree, regardless of array length) and never let the compiler re-associate, which means leaving -fassociative-math off and writing the SIMD form explicitly.

08Masking and predication: conditionals without branches

A pre-AVX-512 vector ALU has no conditional execution. vaddps ymm0, ymm1, ymm2 operates on every lane; there is no "operate on lanes where some condition holds." When the source has a branch whose outcome differs per lane, the SIMD form does the work for both sides and blends the result, with the blend driven by a comparison-produced mask. The simplest case needs no blend at all:

// Clamp every element of `inputArray` to [0, 1]. Written with if-statements, the scalar
// version is two compares per element; vminps / vmaxps do both tests without a branch.
// Assumes elementCount is a multiple of 8.
void clampZeroOne(float* inputArray, int elementCount) {
  __m256 zeroVector = _mm256_setzero_ps();
  __m256 oneVector  = _mm256_set1_ps(1.0f);
  for (int i = 0; i < elementCount; i += 8) {
    __m256 inputVector  = _mm256_loadu_ps(inputArray + i);
    // min(input, 1) then max(result, 0). Two SIMD min/max ops, no branches.
    __m256 belowOne  = _mm256_min_ps(inputVector, oneVector);
    __m256 inRange   = _mm256_max_ps(belowOne, zeroVector);
    _mm256_storeu_ps(inputArray + i, inRange);
  }
}

vminps and vmaxps are the cleanest case of vectorized branching: there's a hardware instruction for the exact lane-wise min and max, so no comparison-and-blend is needed. The general case (a predicate that isn't a comparison the SIMD ISA has a direct op for) uses a comparison followed by a blend:

// Apply an alpha-blended override to every element where `maskArray[i]` is non-zero,
// keep the original value otherwise. In scalar, this is one branch per element.
// Assumes elementCount is a multiple of 8.
void conditionalLerp(float* destination, const float* overrideValue,
                     const int* maskArray, int elementCount, float alpha) {
  __m256 alphaVector   = _mm256_set1_ps(alpha);
  __m256 oneMinusAlpha = _mm256_set1_ps(1.0f - alpha);
  for (int i = 0; i < elementCount; i += 8) {
    __m256i predicateBits = _mm256_loadu_si256((const __m256i*)(maskArray + i));
    // All-ones in the lanes where maskArray[i] == 0. AVX2 has no integer "not equal"
    // compare, so test for zero and swap the blend operands below instead.
    __m256 maskIsZero = _mm256_castsi256_ps(
      _mm256_cmpeq_epi32(predicateBits, _mm256_setzero_si256()));

    __m256 originalLanes = _mm256_loadu_ps(destination + i);
    __m256 overrideLanes = _mm256_loadu_ps(overrideValue + i);
    __m256 blendedLanes  = _mm256_fmadd_ps(originalLanes, oneMinusAlpha,
                                           _mm256_mul_ps(overrideLanes, alphaVector));
    // blendv takes its second operand where the mask lane's sign bit is set:
    // original where maskArray[i] == 0, blended everywhere else.
    __m256 selectedLanes = _mm256_blendv_ps(blendedLanes, originalLanes, maskIsZero);
    _mm256_storeu_ps(destination + i, selectedLanes);
  }
}

Two SIMD primitives here:

• Compares produce a mask of all-ones in lanes where the predicate holds and all-zeros where it doesn't: vpcmpeqd / vpcmpgtd for integers (AVX2 has only equal and greater-than) and vcmpps with a predicate immediate for floats^[9]. In SSE/AVX the mask occupies a regular vector register; in AVX-512 it lands in a separate K mask register (more on that below).
• vblendvps selects between two vectors lane by lane based on the sign bit of a third vector: a floating-point "if (predicate) take B else take A." Two μops on Haswell and Skylake, three on Alder Lake's P-cores, one on Zen 2 and later^[1].

This does strictly more work than the scalar version: every lane computes the blended result even when its mask is zero, and the blend throws that result away. The win comes from processing eight elements per iteration and never paying a branch mispredict. The tradeoff resembles the branchy-vs-branchless story from the Assembly tutorial, with one difference: the eight-wide processing usually pays for the wasted lanes even when the predicate is predictable. The exception is a heavy body behind a predicate that is almost always false, where skipping the work outright (a scalar branch, or a "does any lane need this?" test on the whole vector) wins.

AVX-512: first-class lane masking

The story changes substantially on AVX-512. Every arithmetic instruction has a masked variant that operates only on lanes where a mask register is set; lanes where the mask is clear either keep their old value (merge-masking) or get zeroed (zero-masking)^[7]. There are eight mask registers, K0 through K7. The EVEX encoding has a 3-bit mask-selector field; the value 0 in that field is reserved to mean "no mask" (every lane active), which makes K0 unusable as an actual writemask. K0 still exists as a general mask register for KMOV, KAND, and the other mask-manipulation instructions; you just can't name it as the writemask on a masked vector op^[35].

void clampZeroOneAvx512(float* inputArray, int elementCount) {
  __m512 zeroVector = _mm512_setzero_ps();
  __m512 oneVector  = _mm512_set1_ps(1.0f);

  int i = 0;
  for (; i + 16 <= elementCount; i += 16) {
    __m512 inputVector = _mm512_loadu_ps(inputArray + i);
    __m512 inRange = _mm512_min_ps(_mm512_max_ps(inputVector, zeroVector), oneVector);
    _mm512_storeu_ps(inputArray + i, inRange);
  }

  // Tail: no scalar loop needed. A mask of (1 << remaining) - 1 enables only the live lanes.
  if (i < elementCount) {
    __mmask16 tailMask = (__mmask16)((1u << (elementCount - i)) - 1u);
    __m512 inputVector = _mm512_maskz_loadu_ps(tailMask, inputArray + i);   // inactive lanes read as 0
    __m512 inRange     = _mm512_min_ps(_mm512_max_ps(inputVector, zeroVector), oneVector);
    _mm512_mask_storeu_ps(inputArray + i, tailMask, inRange);                // store only active lanes
  }
}

Three things AVX-512's mask model makes cheap that pre-AVX-512 code paid for explicitly:

• Tail handling. The masked load and store make the scalar tail loop go away. Element counts that aren't multiples of the SIMD width stop being a special case. (AVX already had vmaskmovps for masked loads and stores, but the mask lives in a vector register and the store form is microcoded on Zen 2 through Zen 4, about 40 μops each^[1]; AVX-512 makes masking part of every instruction.)
• Sparse updates. "Update only the entities where isActive" becomes a single masked store; lanes whose isActive bit is zero don't write to memory, so there is no load-blend-store round trip.
• Conditional FMA. vfmadd231ps zmm0 {k1}, zmm1, zmm2 updates only the lanes where k1 is set; the other lanes keep their old value. This is a true predicated SIMD instruction, the way ARM SVE works natively.

The widget below runs vfmadd231ps zmm0 {k1}, zmm1, zmm2 across sixteen lanes. The mask register is the row of 0/1 bits at the top; lanes where the bit is 1 compute zmm0 + zmm1 × zmm2, lanes where it's 0 keep their old value. The pattern buttons change the mask for the same inputs; Step feeds new inputs and runs the FMA again, accumulating into zmm0:

09Shuffles, blends, broadcasts: rearranging the lanes

Shuffles rearrange lanes within a register or across two registers. They get data into the lane order a kernel needs, and when a kernel runs slower than its arithmetic suggests, a shuffle bottleneck is a common cause. The main members of the family:

For non-arithmetic work, the most useful of these is pshufb^[4]. It performs a parallel byte-granularity lookup: for each output byte position i, the output is table[indices[i] & 0x0F], or zero if the index byte's top bit is set, where table is the first operand (a 16-byte table) and indices is the second. One instruction, sixteen 16-entry lookups in parallel. Shipping code uses it for UTF-8 validation (Keiser & Lemire 2021^[16]), base64 encode/decode (Muła & Lemire 2018^[17]), hex conversion, and per-nibble population counts, among many other workloads.

The widget shows one pshufb. Pick a table; the arrows trace each output byte back to the table entry its index selects:

Two reliable rules about shuffles:

1. In-lane shuffles are cheap; cross-lane shuffles cost more latency. AVX's 256-bit register is two 128-bit "lanes" stitched together. vshufps, vpshufb, vpshufd all operate on each 128-bit lane independently. vpermps and vperm2f128 cross the lane boundary; on Skylake they have 3-cycle latency against 1 cycle for the in-lane shuffles^[1]. On Skylake every shuffle of either kind issues on port 5 alone, one per cycle, so a shuffle-heavy loop is port-5-bound regardless. The fastest 8-element transpose uses as few cross-lane shuffles as it can.
2. Movmask is the bridge from vector to scalar. When you need to branch on whether any (or every) lane satisfied a predicate, vmovmskps extracts the sign bits into a 32-bit integer; tzcnt or popcnt on that integer answers "which lane was the first match" or "how many matched." This is the pattern for early-exit SIMD search loops: SIMD-compare a 256-bit chunk, extract the mask, branch on it^[4].

10Gather and scatter: when you have to load from N pointers

A vector load assumes contiguous addresses. A gather instruction (AVX2 and later) takes a base pointer and a vector of indices, and loads one element per lane from base[indices[lane]]. The hardware does N independent loads internally and assembles the result:

// 8 vertices, each with a bone-index. Look up the 8 bone matrices' first entries.
float bonePositionX[MAX_BONES];        // SoA bone data, one float per bone
int boneIndices[8] = {3, 17, 3, 8, 42, 8, 17, 5};

__m256i indicesVector = _mm256_loadu_si256((__m256i*)boneIndices);
// Gather: each lane loads bonePositionX[indicesVector[lane]], 4-byte stride.
__m256 boneXVector = _mm256_i32gather_ps(bonePositionX, indicesVector, 4);

Gather is the direct way to vectorize random-access patterns: skinning with per-vertex bone indices, lookup-table-driven shading, ECS systems where the components for a job aren't laid out in the order the job processes entities. The cost picture is the part to understand.

Gather has both a latency and a throughput number, and they are very different. On Haswell (2013, the first AVX2 part), a vpgatherdd ymm (8 × int32) takes around 20 cycles to produce its result and decodes to 34 μops, with a reciprocal throughput of 11 cycles^[1]: a back-to-back stream of gathers starts a new one every 11 cycles. On Haswell, eight scalar loads plus inserts often matched or beat it. Skylake cut the reciprocal throughput to 5 cycles, Ice Lake stayed at 5, and Alder Lake's P-cores reach about 3. AMD's gathers were slower for longer: 16 cycles on Zen 2, 8 on Zen 3 and Zen 4, under 6 on Zen 5^[1]. The numbers can also move after a chip ships: the August 2023 microcode mitigation for the Downfall vulnerability slowed gathers on Skylake through Ice Lake / Tiger Lake client and server parts. Intel cited penalties of up to 50% in extreme cases; Phoronix's application benchmarks found smaller but still significant losses in gather-heavy code^[39].

A useful rule of thumb: a gather costs about one load-port slot per lane, so its throughput ceiling is close to that of the same number of scalar loads; what it saves is the instructions to insert each value into a vector. It wins when the data is cache-resident and the alternative is eight scalar loads and seven inserts. It loses badly when the indices turn out to be sequential, where one ordinary vector load does the same job.

Scatter (the reverse: write each lane to base[indices[lane]]) is AVX-512 only. When two lanes name the same destination, the element writes are architecturally ordered from the lowest lane to the highest, so the highest lane's value is the one that sticks; the hardware may skip the earlier, fully-overlapped writes^[2]. Scatter is also slow: a 16-lane vpscatterdd has a reciprocal throughput of about 17 cycles on Skylake-X, roughly one element per cycle, and about 11 on Ice Lake and Alder Lake^[1].

The pragmatic answer in most engine code is to avoid gather and scatter by restructuring the data:

• Sort by bone index before skinning; vertices with the same bone now load contiguously.
• Sort by component-id in an ECS so the inner loop touches one chunk's worth of components per iteration.
• If indices are stable across many frames, transpose once into a denser layout and reuse it.

When gather is unavoidable, the typical pattern is to gather a small amount of data (a single float per lane), then do many arithmetic ops in vector form before the next gather, so the gather's cost is spread over a lot of compute.

11A worked engine kernel: batched 4×4 matrix-vector transform

Renderers, physics, and animation code all apply 4×4 transforms to large batches of points. The scalar form is sixteen multiplies and twelve adds per point. The SIMD form depends on where the parallelism comes from:

• One vertex at a time, 128 bits. Each vec4 is one __m128; the 4×4 matrix-vector product is four broadcasts, a multiply, and three FMAs (the form from the Assembly tutorial's §13). One vertex per kernel call; useful when points arrive one at a time. Throughput-poor on AVX2 because half of each 256-bit operation goes unused.
• Eight vertices at a time, AoSoA, 256 bits. The data layout the throughput-tuned form needs: 8 vertices laid out as (x[0..7], y[0..7], z[0..7], w[0..7]). One YMM register per component. The transform broadcasts each matrix entry to all eight lanes and FMAs across all eight vertices in parallel.

The 8-at-once form suits any stage that pushes many points through one matrix, such as instance transforms or particle output. The kernel is short:

// AoSoA block: 8 vertices laid out (x[0..7], y[0..7], z[0..7], w[0..7]).
struct alignas(32) VertexBlock8 {
  float xCoordinates[8];
  float yCoordinates[8];
  float zCoordinates[8];
  float wCoordinates[8];   // usually 1.0f for positions, 0.0f for direction vectors
};

struct Matrix4x4 { float m[16]; };   // row-major: m[row * 4 + column]

void transformVertexBlock(const Matrix4x4& transform, VertexBlock8& block) {
  // Load each input component (8 vertices' x in one ymm register, etc.). One vmovaps each.
  __m256 xLanes = _mm256_load_ps(block.xCoordinates);
  __m256 yLanes = _mm256_load_ps(block.yCoordinates);
  __m256 zLanes = _mm256_load_ps(block.zCoordinates);
  __m256 wLanes = _mm256_load_ps(block.wCoordinates);

  // Output x = m00*x + m01*y + m02*z + m03*w, applied to all 8 vertices in parallel.
  // Each broadcast (vbroadcastss) replicates one matrix entry across all 8 lanes.
  __m256 outputX = _mm256_mul_ps  (_mm256_set1_ps(transform.m[ 0]), xLanes);
  outputX        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[ 1]), yLanes, outputX);
  outputX        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[ 2]), zLanes, outputX);
  outputX        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[ 3]), wLanes, outputX);

  __m256 outputY = _mm256_mul_ps  (_mm256_set1_ps(transform.m[ 4]), xLanes);
  outputY        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[ 5]), yLanes, outputY);
  outputY        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[ 6]), zLanes, outputY);
  outputY        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[ 7]), wLanes, outputY);

  __m256 outputZ = _mm256_mul_ps  (_mm256_set1_ps(transform.m[ 8]), xLanes);
  outputZ        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[ 9]), yLanes, outputZ);
  outputZ        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[10]), zLanes, outputZ);
  outputZ        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[11]), wLanes, outputZ);

  __m256 outputW = _mm256_mul_ps  (_mm256_set1_ps(transform.m[12]), xLanes);
  outputW        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[13]), yLanes, outputW);
  outputW        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[14]), zLanes, outputW);
  outputW        = _mm256_fmadd_ps(_mm256_set1_ps(transform.m[15]), wLanes, outputW);

  _mm256_store_ps(block.xCoordinates, outputX);
  _mm256_store_ps(block.yCoordinates, outputY);
  _mm256_store_ps(block.zCoordinates, outputZ);
  _mm256_store_ps(block.wCoordinates, outputW);
}

The work per vertex is exactly the same as the scalar form: four output components, each a four-term dot product against a matrix row. The difference is that every FMA does eight in parallel. The kernel has four independent output chains (one per output component), each four ops deep (one multiply + three FMAs), so its critical-path latency is roughly 4 ops × 4-cycle FMA latency = 16 cycles. In a loop over many blocks the out-of-order core overlaps consecutive blocks and the throughput limit takes over: 16 multiply/FMA ops per block at 2 per cycle is 8 cycles per block. With the broadcasts inside the loop as written, the 16 broadcast loads plus 4 vertex loads (20 loads at 2 per cycle) push that to about 10 cycles. The realistic range is 8 to 20 cycles per block of 8, or roughly 1 to 2.5 cycles per vertex, depending on whether the broadcasts are hoisted and how well the surrounding code lets blocks overlap.

What this code intentionally skips. The matrix broadcasts are reloaded every call: a real engine hoists them out of the inner loop (broadcast the matrix into 16 registers once, transform many vertex blocks against it). The batching choice depends on whether you're skinning (many matrices, few vertices each) or instancing (one matrix, many vertices each); the form above is the instancing form. Skinning's per-vertex bone lookup needs the gather pattern from §10. DirectXMath's XMVector3TransformStream shows both approaches in one function: its SSE path runs the one-vertex broadcast form four vertices per iteration, while its NEON path deinterleaves four vertices into SoA registers with vld3q_f32 and runs the batch form^[19].

12A worked engine kernel: 8 spheres against the frustum per iteration

Most renderers cull bounding volumes against the camera frustum before submitting draw calls, on the CPU or, in GPU-driven renderers, in a compute pass. The scalar form is one sphere-vs-six-planes test per object; the SIMD form does eight at once and lets the predicate pass through the lane-mask machinery from §8.

A frustum plane is the equation nx · x + ny · y + nz · z + d = 0 with n the unit outward normal and d the plane's offset from the origin. A point lies outside the plane when nx · px + ny · py + nz · pz + d > 0. For a bounding sphere centered at (px, py, pz) with radius r, the sphere is fully outside the plane when that same dot product is greater than r; it's fully inside when the dot product is less than -r; it straddles otherwise. The standard cull keeps any object that isn't fully outside any of the six planes^[20].

The SoA form has all eight spheres' xs in one register, all eight ys in the next, eight zs, eight radii. One plane test is three FMAs (the plane offset rides in as the first addend), a compare, and an and-not; six planes is eighteen FMAs, six compares, and six and-nots, the same instruction count the scalar form spends on a single sphere.

struct alignas(32) SphereBatch8 {
  float centerX[8], centerY[8], centerZ[8], radius[8];
};

struct FrustumPlane { float normalX, normalY, normalZ, distance; };

// Returns an 8-bit mask: bit i is 1 if sphere i passed (was not culled), 0 if outside.
int cullBatchAgainstFrustum(const SphereBatch8& spheres, const FrustumPlane planes[6]) {
  __m256 centerXVector = _mm256_load_ps(spheres.centerX);
  __m256 centerYVector = _mm256_load_ps(spheres.centerY);
  __m256 centerZVector = _mm256_load_ps(spheres.centerZ);
  __m256 radiusVector  = _mm256_load_ps(spheres.radius);

  // Start with all 8 visible; lanes that fail any plane get masked off.
  __m256 visibleMask = _mm256_castsi256_ps(_mm256_set1_epi32(-1));

  for (int p = 0; p < 6; ++p) {
    // Broadcast one plane's coefficients to all 8 lanes.
    __m256 planeNormalX = _mm256_set1_ps(planes[p].normalX);
    __m256 planeNormalY = _mm256_set1_ps(planes[p].normalY);
    __m256 planeNormalZ = _mm256_set1_ps(planes[p].normalZ);
    __m256 planeOffset  = _mm256_set1_ps(planes[p].distance);

    // signedDistance[i] = n·center[i] + d, computed in parallel for 8 spheres.
    __m256 signedDistance = _mm256_fmadd_ps(planeNormalX, centerXVector, planeOffset);
    signedDistance        = _mm256_fmadd_ps(planeNormalY, centerYVector, signedDistance);
    signedDistance        = _mm256_fmadd_ps(planeNormalZ, centerZVector, signedDistance);

    // A sphere is fully outside this plane if signedDistance > radius. Mask off those lanes.
    __m256 outsidePlane = _mm256_cmp_ps(signedDistance, radiusVector, _CMP_GT_OQ);
    visibleMask = _mm256_andnot_ps(outsidePlane, visibleMask);    // clear lanes where outsidePlane
  }
  // Extract the 8 sign bits as a single int; bit i is 1 if sphere i is still visible.
  return _mm256_movemask_ps(visibleMask);
}

Six planes × (three FMAs + one compare + one and-not) is thirty μops per batch of eight spheres, plus the broadcasts. The 24 FMAs and compares share Skylake's two FMA-capable ports (the and-nots can go to a third), so a batch takes on the order of 12 to 15 cycles, about 2 cycles per sphere^[1]. A branch-free scalar version runs about the same thirty μops for each sphere, so the vector form's advantage approaches the lane count, 8×. Scalar code that exits after the first failing plane narrows the gap on scenes where most objects are culled early, and gives some of it back in branch mispredicts. The widget runs both:

13Autovectorization, intrinsics, and when each wins

Four ways to get vector code out of a C++ compiler, in roughly decreasing order of how often you should use them:

1. Plain scalar loops, autovectorized. Write the obvious scalar code, compile with -O3 -march=native, look at the autovectorization report (-fopt-info-vec on GCC, -Rpass=loop-vectorize on Clang). The autovectorizer handles clean maps and reductions on contiguous arrays of trivially-copyable types. When it works, this is the right answer.
2. Pragmas and attributes that nudge the autovectorizer. #pragma omp simd (enabled by -fopenmp-simd) or #pragma GCC ivdep declare that an inner loop has no loop-carried dependencies and the compiler may vectorize it^[21]. __restrict declares that pointers don't alias. std::assume_aligned or __builtin_assume_aligned gives the compiler an alignment guarantee. The source stays scalar and readable while the output is vector code.
3. Portable SIMD libraries. Google's Highway^[22], std::simd in C++26^[23], xsimd, SIMDe (which header-translates <immintrin.h> calls to NEON or WebAssembly SIMD^[24]). The right answer when you want the explicit style of intrinsics but need it to run on x86, Arm, and the web. Engines ship their own versions of the idea: Unreal's VectorRegister4Float wrapper lowers to SSE or NEON per platform^[12], and Unity's Mathematics package and Burst compiler^[11] are a closely related design.
4. Architecture-specific intrinsics. <immintrin.h> on x86, <arm_neon.h> on ARM. The form used throughout this tutorial. The right answer when the autovectorizer bailed out on the pattern you need, when you have to use an instruction the compiler can't pick from source (a specific pshufb table, an AVX-512 mask predicate), or when you need bit-identical results across architectures and the compiler's autovec choices vary by version.

The mental model: scalar loops are for the parts of the codebase that don't profile hot. Pragmas and portable wrappers are for the parts that do profile hot but aren't the absolute bottleneck. Intrinsics are for the kernels that sit near the top of the frame profile and have stayed there after every other optimization. Hand-written assembly sits below all four and is rare in engine code outside platform glue such as context switches.

A practical heuristic from Mike Acton, restated for SIMD specifically: look at the data and the access pattern first, write the inner loop second, switch to intrinsics third^[13]. Most "the compiler won't autovectorize this" complaints are downstream of a layout problem the compiler isn't allowed to fix.

14AVX-512 in 2026: the downclocking story

Game code rarely uses AVX-512, first because most Intel client chips of the last few years don't have it, and second because of its reputation for lowering clock speed. The reputation comes from Skylake-SP: "AVX-512 instructions drop the CPU's clock, the clock drop eats the speedup, so don't use them." That was a fair summary of the first generation and has been less true with each one since.

What actually happened, generation by generation:

• Skylake-SP (Xeon Scalable 1st gen, 2017) and its Xeon W siblings. Frequency licenses: L0 for scalar and light vector code, L1 for heavy AVX2 or light AVX-512, L2 for heavy (floating-point and multiply) AVX-512. Travis Downs's test chip, a Xeon W-2104, runs at 3.2, 2.8, and 2.4 GHz in L0, L1, and L2; he measured a stall of roughly 8 to 20 µs at each voltage transition during which the core dispatches at a quarter of its normal rate, and about 680 µs after the last wide instruction before the core returns to L0^[26]. Because the clock applies to the whole core, a few AVX-512 kernels scattered through scalar code slow the scalar code down too.
• Cascade Lake (2019). The same license scheme.
• Ice Lake (client 2019, server 2021). Much smaller penalties. On an Ice Lake laptop chip, Downs measured heavy AVX-512 at 3.6 GHz against 3.7 GHz for scalar code with one core active, and no license-based reduction at all with more cores active^[26]. Phoronix measured the Ice Lake Xeon Platinum 8380's peak clock dropping by about 175 MHz under AVX-512-heavy benchmarks^[36].
• Sapphire Rapids (2023). In the same Phoronix tests, peak frequency was similar with and without AVX-512^[36].
• Intel client (Alder Lake / Raptor Lake / Meteor Lake / Arrow Lake, 2021–2024). AVX-512 is disabled or absent because the E-cores lack the unit; this is the source of "modern Intel desktops don't have AVX-512" in 2026.
• AMD Zen 4 (2022). Executes each 512-bit operation as two halves on its 256-bit datapaths, so 512-bit throughput per instruction is half of a two-FMA-unit Intel server core's. There is no AVX-512-specific clock offset; Phoronix saw no AVX-512 downclocking on Genoa^[36][37].
• AMD Zen 5 (2024). Full 512-bit datapaths on desktop and server parts; the Strix Point laptop chips keep 256-bit datapaths^[37]. There are no fixed AVX-512 frequency offsets, though code that combines 512-bit loads with FMAs makes the core back off its clock gradually over a few milliseconds, as any power-hungry code does^[38].

The practical advice in 2026:

• On current servers (Xeon Ice Lake or later, EPYC Zen 4 or later), use AVX-512 where it measures faster.
• On consumer Intel from Alder Lake through Arrow Lake, AVX-512 isn't available; -march=x86-64-v3 (AVX2 / FMA / BMI2) is the target for code that has to run on those machines.
• On consumer AMD from Zen 4 onward, AVX-512 is available with no AVX-512-specific clock penalty.
• For shipping game binaries, the usual pattern is multi-versioning: compile two or three versions of the hot kernels for different instruction sets and pick one at startup. GCC and Clang can generate the variants and the dispatcher from target / target_clones attributes (function multiversioning)^[27]; MSVC has no equivalent, so Windows builds typically compile separate files per instruction set and fill a function-pointer table after checking CPUID.

The Skylake-SP-era advice to avoid AVX-512 for frequency reasons describes hardware few players own. The current advice: use AVX-512 where it's available and faster, fall back to AVX2 where it isn't, and dispatch at startup.

15ARM: NEON, SVE2, and porting the math

Arm's AArch64 (Armv8-A and later) makes NEON its mandatory SIMD: thirty-two 128-bit registers, V0–V31. NEON is roughly comparable to SSE4 (128-bit, no native masking, no gather). For game programmers it's the Nintendo Switch, Switch 2, Apple Silicon Macs, current iPhones and Android phones, and the AWS Graviton server line^[28].

The translation from x86 SSE/AVX2 intrinsics to NEON is mostly mechanical. _mm_add_ps ↔ vaddq_f32. _mm_mul_ps ↔ vmulq_f32. _mm_fmadd_ps(a, b, c) ↔ vfmaq_f32(c, a, b): note that NEON takes the accumulator first. _mm_loadu_ps ↔ vld1q_f32. The dot-product kernel from §6 with NEON intrinsics:

#include <arm_neon.h>

float dotNeon(const float* aArray, const float* bArray, int elementCount) {
  float32x4_t accumulatorA = vdupq_n_f32(0.0f);    // 4-lane register of zeros
  float32x4_t accumulatorB = vdupq_n_f32(0.0f);
  int i = 0;
  for (; i + 8 <= elementCount; i += 8) {
    float32x4_t aLow  = vld1q_f32(aArray + i);
    float32x4_t bLow  = vld1q_f32(bArray + i);
    float32x4_t aHigh = vld1q_f32(aArray + i + 4);
    float32x4_t bHigh = vld1q_f32(bArray + i + 4);
    accumulatorA = vfmaq_f32(accumulatorA, aLow,  bLow);    // fma: dst + a*b, one rounding step
    accumulatorB = vfmaq_f32(accumulatorB, aHigh, bHigh);
  }
  float32x4_t combined = vaddq_f32(accumulatorA, accumulatorB);
  // Horizontal sum: AArch64 has vaddvq_f32, one intrinsic that reduce-adds all 4 lanes.
  float totalSum = vaddvq_f32(combined);
  for (; i < elementCount; ++i) totalSum += aArray[i] * bArray[i];
  return totalSum;
}

Two NEON conveniences SSE doesn't have. vaddvq_f32 is a single intrinsic for the horizontal sum (it compiles to two faddp pairwise adds; integer reductions such as addv are single instructions). And NEON arithmetic has always used three-operand forms, so there's no legacy-SSE-style "destination is one of the sources" constraint. Apple's CPU optimization guide for the M-series and Arm's per-core software optimization guides are the practical references^[29][30].

SVE and SVE2: vector-length-agnostic

The longer-term ARM story is SVE (Scalable Vector Extension), which adds vector-length-agnostic instructions: the same binary runs on hardware whose vectors are any multiple of 128 bits, up to 2048. SVE2 is an optional feature in the ARMv9-A profile (not mandatory), implemented in practice by the Neoverse server line and a slice of mobile silicon^[31]. The shipping examples worth knowing about:

• Fujitsu A64FX in the Fugaku supercomputer. SVE (not SVE2) at 512 bits per vector. First major SVE deployment, 2020.
• AWS Graviton 3 (Neoverse V1). SVE at 256 bits, implemented as two 256-bit pipes. The first cloud-server SVE rollout, 2022.
• AWS Graviton 4 (Neoverse V2). SVE2 at 128 bits, four 128-bit pipes. Arm narrowed the vector width from V1 to V2; peak FP throughput per core is unchanged because V2's four 128-bit pipes match V1's two 256-bit ones.
• Armv9 client parts. Phone SoCs based on Cortex-X4 / X925 / A720 implement SVE2 at 128 bits. Apple's M4 (Armv9.2-A, 2024) implements SME (Scalable Matrix Extension) but does not expose non-streaming SVE2 to user code; SVE-style instructions on M4 are only available inside an SME streaming-mode region^[34]. Code that wants to run on M4 has to use NEON or stay inside SME.

SVE has first-class predication: every instruction takes a predicate register, the way AVX-512 does, so the mask-and-blend dance from §8 disappears. Code written for SVE looks closer to AVX-512 with mask registers than to NEON. The price is that you can't reason about lane count at compile time; every loop has to be written in a "while there's still work" form (the SVE WHILELT instruction generates the predicate for "lanes within bounds").

For game code in 2026 the practical situation is: NEON is what every shipping Arm target supports, including Switch 2 and every consumer Apple Silicon Mac. SVE2 is a server-side story (Graviton, other Neoverse-based instances) plus a slice of Android phones. The Highway library^[22] has SVE and NEON backends, so kernels written against it can target both without a rewrite. Writing pure NEON intrinsics in 2026 is much like writing pure SSE intrinsics: it runs on everything in its family today but leaves the wider vector units on newer hardware unused.

16Pitfalls

• Mixing legacy SSE encodings with VEX-encoded AVX. Running non-VEX SSE instructions (movaps xmm0, ...) while the upper halves of the YMM registers hold data from VEX AVX code (vmulps ymm0, ...) costs extra unless a vzeroupper intervenes. On Sandy Bridge through Broadwell each switch triggers a state save or restore of dozens of cycles; on Skylake and later each legacy SSE instruction instead carries an extra blend μop and a dependency on the full register^[32]. Compilers insert vzeroupper before calls and returns in AVX code, and intrinsic code compiled with AVX enabled is VEX-encoded throughout. Hand-written assembly that mixes the two has to insert vzeroupper itself.
• Forgetting the scalar tail. A loop processing 8 floats at a time over an array of 503 elements leaves 7 elements undone. Common bug patterns: a vector loop bound that reads past the end (i < count where i + 8 <= count was meant), a tail loop that starts at the wrong index and redoes or skips elements. AVX-512's masked load and store make the tail a non-issue; pre-AVX-512 code should pad arrays to a multiple of the SIMD width and zero the padding, or write the scalar tail with a clear loop bound.
• Treating scalar SSE math as cheaper than packed. addss and addps have identical latency and throughput on mainstream Intel and AMD cores of the last fifteen years^[1]. Three lanes of an XMM register sitting unused doesn't make the instruction faster. The win is in using those lanes.
• Cross-lane shuffles on the critical path. vperm2f128, vpermps, and vextractf128 have 3-cycle latency on Skylake, against 1 cycle for in-lane shuffles^[1]. A horizontal sum at the end of a long reduction amortizes this; a cross-lane shuffle inside a loop-carried dependency chain adds two cycles to every iteration.
• Gather on cold data. A gather's result isn't ready until its slowest lane arrives, so one cache miss among eight lanes stalls everything that depends on the whole register. On a large working set with random indices, a gather buys little over scalar loads. Restructure the data instead.
• Compiling for the dev machine's instruction set. A binary compiled with -march=skylake-avx512 can crash with SIGILL on a chip without AVX-512. Shipping game binaries either pick a conservative baseline (x86-64-v2, or x86-64-v3 for titles that require AVX2) or dispatch at startup. Test the dispatch path on hardware that lacks the higher-tier instructions; the failure mode is "works on the dev machine, crashes at boot on a player's machine"^[27].
• Floating-point determinism. A reduction reordered for SIMD doesn't produce the same bit pattern as a scalar left-to-right reduction. For replays, lockstep multiplayer, and deterministic physics, write the SIMD reduction tree explicitly (the same tree every time, regardless of length) and don't enable -fassociative-math^[15].
• Denormals. A particle that has decayed to a near-zero velocity can produce subnormal numbers, and on many Intel cores an operation that produces or consumes one can take a microcode assist costing on the order of a hundred cycles^[32]. The fix is to set Flush-to-Zero (FTZ) and Denormals-Are-Zero (DAZ) in MXCSR; physics and audio codebases routinely do. MXCSR is per thread, so set it on every worker thread, not just the main one.

17What's next

Where to go from here:

• Pick a hot kernel in your engine and rewrite it twice. Once as plain scalar with __restrict and #pragma omp simd, once with intrinsics. Compare the disassembly and the measured frame cost. The gap is the size of the autovec-vs-intrinsics decision in your codebase.
• Read the Intel Intrinsics Guide^[9] for the instructions you're using. Each entry gives the instruction it maps to and a pseudo-code description of the lane semantics, and many list latency and throughput for recent Intel cores. The reference is searchable by mnemonic, by intrinsic name, or by feature flag.
• Set up llvm-mca on a small slice of your inner loop. A few minutes of llvm-mca -mcpu=skylake on a 30-instruction kernel shows which port is saturated and where a dependency chain pins the throughput. uica.uops.info is a similar tool with a more detailed front-end model^[33].
• Skim Wojciech Muła's blog and Daniel Lemire's blog, at 0x80.pl and lemire.me/blog. Between them they cover hundreds of small SIMD techniques with microbenchmarks on x86 and Arm.
• Pair with the Assembly tutorial and the Memory Model tutorial. The first covers the ports, encodings, and calling conventions under the intrinsics; the second covers atomics and ordering for the multithreaded code that runs these kernels.

18Sources & further reading

Numbered citations refer to the superscripts above. Most are freely available; a few are vendor manuals that require a no-cost registration.

1. Abel, A., & Reineke, J. (2019). uops.info: Characterizing Latency, Throughput, and Port Usage of Instructions on Intel Microarchitectures. ASPLOS. Project home: uops.info. Machine-measured latency, throughput, and port-usage tables for most x86-64 instructions on Intel cores from Conroe onward and AMD cores from Zen+ onward, including the AVX-512 mask-register forms.
2. Intel Corporation. Intel® 64 and IA-32 Architectures Software Developer's Manual. Order Numbers 253665–253669. intel.com. Volume 1 has the SSE, AVX, and AVX-512 programming chapters; Volume 2 has the per-instruction encoding tables and pseudo-code semantics.
3. Advanced Micro Devices. AMD64 Architecture Programmer's Manual, Volume 1: Application Programming. Publication #24592. amd.com (archived copy). The AMD64 architecture reference, including the mandatory SSE2 baseline that every x86-64 compiler may assume.
4. Muła, W. Practical SIMD and bit-twiddling notes. 0x80.pl. The reference body of work on pshufb-driven tricks, including byte-shuffle parallel lookups, UTF-8 validation, base64 codec routines, and hundreds of related microbenchmarks across SSE, AVX2, AVX-512, and NEON.
5. Muller, J.-M., Brisebarre, N., de Dinechin, F., Jeannerod, C.-P., Lefèvre, V., Melquiond, G., Revol, N., Stehlé, D., & Torres, S. (2018). Handbook of Floating-Point Arithmetic (2nd ed.). Birkhäuser. The reference for FMA rounding semantics: a single rounding step after the multiply-then-add, instead of two.
6. Advanced Micro Devices. Software Optimization Guide for the AMD Zen4 Microarchitecture (publication #57647) and Software Optimization Guide for the AMD Zen5 Microarchitecture (#58455). Zen 4 guide (archived), Zen 5 guide. AMD's counterparts to the Intel optimization manual: pipeline structure, the Zen 4 AVX-512 implementation on 256-bit datapaths, and latency/throughput tables for SIMD instructions.
7. Intel Corporation. Intel® Architecture Instruction Set Extensions Programming Reference. intel.com. The normative document for AVX-512 mask registers, the EVEX prefix, AVX-512F/BW/DQ/VL feature splits, and the masked memory operations used in §8.
8. Intel Corporation. (2023–2025). Intel® Advanced Vector Extensions 10.2 (Intel® AVX10.2) Architecture Specification. intel.com. The converged-ISA spec unifying the AVX-512 feature set. The original 2023 proposal allowed a 256-bit-max implementation; the 2025 revisions removed that option and made 512-bit support mandatory for AVX10.2 on all cores, as reported at the time by Phoronix.
9. Intel Corporation. Intel® Intrinsics Guide. intel.com/intrinsics-guide. The searchable reference for the intrinsics in the <immintrin.h> family: the instruction each one maps to, its CPUID feature flag, pseudo-code for the operation, and latency/throughput for many of them.
10. Fog, A. The microarchitecture of Intel, AMD and VIA CPUs and Instruction tables. agner.org/optimize. Manuals 3 and 4 of the agner.org/optimize series, periodically updated since 1996; the microarchitecture manual was last revised in 2026 and the instruction tables in 2025.
11. Unity Technologies. The Burst compiler, the Mathematics package, and the Entities package. Burst manual; Entities: archetypes and chunks. Burst compiles a C# subset to native SIMD code such as SSE, AVX2, and NEON; Unity.Mathematics provides the float4 / float4x4 types it vectorizes. The Entities page documents the 16 KiB chunks with one array per component type.
12. Epic Games. VectorRegister4Float, Unreal Engine API reference. dev.epicgames.com. Unreal's portable SIMD register type (the full source, Math/VectorRegister.h, needs a linked Epic account on GitHub): it wraps __m128 on x86 and float32x4_t on Arm, and Unreal's math library is built on top of it.
13. Acton, M. (2014). Data-Oriented Design and C++. CppCon. YouTube. The widely cited talk arguing that data layout should follow the access pattern of the code that transforms it, with SoA versus AoS as the running example.
14. ISO/IEC. C++ standard, [basic.align] and [expr.alignof]. The normative spec for alignas, alignof, std::aligned_alloc, and the C++17 over-aligned operator new. cppreference's overview at en.cppreference.com/w/cpp/language/alignas is the practical entry point.
15. Muller, J.-M., et al. (2018). Handbook of Floating-Point Arithmetic (2nd ed.). Birkhäuser. Chapter 3 covers the non-associativity of FP addition and the rounding bounds for reordered reductions. The reference for "is the SIMD-reordered sum still numerically correct" questions.
16. Keiser, J., & Lemire, D. (2021). Validating UTF-8 In Less Than One Instruction Per Byte. Software: Practice and Experience 51(5). arXiv:2010.03090. A branchless UTF-8 validator built on pshufb lookups; the approach is used in simdjson and in the simdutf library, which ships in Node.js and Chromium.
17. Muła, W., & Lemire, D. (2018). Faster Base64 Encoding and Decoding using AVX2 Instructions. ACM Transactions on the Web. arXiv:1704.00605. A pshufb-driven base64 codec; the paper reports roughly 10× faster encoding and 7× faster decoding than the state-of-the-art implementations it compared against.
18. Drepper, U. (2007). What Every Programmer Should Know About Memory. PDF. The long-form treatment of the memory hierarchy, cache lines, prefetch, and bandwidth limits. Older than this tutorial's other references but still a clear single document on the memory side of SIMD performance.
19. Microsoft. DirectXMath. learn.microsoft.com and github.com/microsoft/DirectXMath. Microsoft's header-only SIMD math library, shipped with the Windows SDK; the source has SSE2, SSE4, AVX, AVX2/FMA3, and NEON code paths, including the XMVector3TransformStream batch transform.
20. Akenine-Möller, T., Haines, E., Hoffman, N., Pesce, A., Iwanicki, M., & Hillaire, S. (2018). Real-Time Rendering (4th ed.). CRC Press. Chapter 19 covers view-frustum and hierarchical culling; Chapter 22 has the sphere-plane intersection test used in §12.
21. OpenMP Architecture Review Board. OpenMP Application Programming Interface, Version 5.2. openmp.org. The normative spec for #pragma omp simd, the standardized way to declare "this loop has no loop-carried dependencies and may be vectorized."
22. Wassenberg, J., et al. Highway: A C++ library for SIMD. github.com/google/highway. Portable SIMD library with backends for SSE/AVX/AVX-512/NEON/SVE/RVV and runtime dispatch; used in Chromium, Firefox, and the libjxl JPEG XL codec, among others.
23. Kretz, M. (2018). P0214R9 — Data-parallel vector library. WG21. open-std.org. The proposal that became std::experimental::simd in the Parallelism TS 2 (ISO/IEC TS 19570:2018). Its successor paper, P1928, merged std::simd into the C++26 working draft in 2024.
24. SIMDe Contributors. SIMDe: SIMD Everywhere. github.com/simd-everywhere/simde. Header-only translation of x86 SIMD intrinsics to NEON, WebAssembly SIMD, AltiVec, and a fallback portable C. Used by codebases that want to keep <immintrin.h> source but ship on ARM and the web.
25. LLVM Project. The LLVM Loop Vectorizer. llvm.org/docs/Vectorizers.html. The reference for what the autovectorizer can and can't do, the diagnostics it emits, and the pragmas (#pragma clang loop vectorize_width(N)) that override its decisions.
26. Downs, T. (2020). Gathering Intel on Intel AVX-512 Transitions (travisdowns.github.io) and Ice Lake AVX-512 Downclocking (travisdowns.github.io). Measurements of license transitions, transition stalls, and relaxation time on a Skylake-SP-derived Xeon W, and of AVX-512 frequencies on an Ice Lake laptop chip, with reproducible benchmarks.
27. Free Software Foundation. Function Multiversioning. gcc.gnu.org. GCC's mechanism for emitting several versions of a function (one per __attribute__((target("avx2"))), one per ("avx512f"), etc.) and dispatching at load time through an IFUNC resolver. Clang supports the same attributes.
28. ARM Limited. Arm® Architecture Reference Manual for A-profile architecture. Document DDI 0487. developer.arm.com. The normative reference for AArch64 Advanced SIMD (NEON), SVE, and SVE2: the register file, the instructions, and which features are mandatory or optional in each architecture version.
29. ARM Limited. Arm Cortex-A optimization guides. developer.arm.com. Per-core software optimization guides with latency and throughput tables (Cortex-A78, A715, Neoverse N1/N2/V1/V2); the Arm-side counterpart to Agner Fog's tables.
30. Apple Inc. Apple Silicon CPU Optimization Guide. developer.apple.com. Apple's official guide for the M-series cores; covers the wide front end, the cluster topology (P-cores and E-cores), and the NEON throughput characteristics that differ from a typical ARM Cortex part.
31. ARM Limited. Arm SVE2 Architecture. developer.arm.com. The scalable-vector ISA implemented on the A64FX (Fugaku) and across the Neoverse line (V1/V2/N2). SVE2 is an optional feature in the ARMv9-A profile; the spec does not mandate it for V9-A conformance. Covers vector-length-agnostic loop forms, predicate registers, and gather/scatter.
32. Intel Corporation. Intel® 64 and IA-32 Architectures Optimization Reference Manual. Order Number 248966. intel.com. Intel's microarchitecture guide: the SSE-AVX transition behavior on each generation and the vzeroupper recommendation, Skylake's page-split load improvement, denormal assists, and the MXCSR flush-to-zero / denormals-are-zero flags.
33. Abel, A., & Reineke, J. (2022). uiCA: Accurate Throughput Prediction of Basic Blocks on Recent Intel Microarchitectures. ICS. Project home: uica.uops.info. A simulator with a richer model of the front end, μop cache, and back-end ports than llvm-mca; useful when you need a more precise prediction of a kernel's steady-state IPC.
34. Apple Developer Forums (2024). Does Apple M4 support ARM SVE instructions? developer.apple.com/forums. A developer report of SVE instructions faulting with EXC_BAD_INSTRUCTION on M4, answered with the LLVM source comment that describes M4 as Armv9.2-A without SVE, which the Arm architecture allows because SVE is optional. The M4's SME implementation includes a streaming SVE mode, which is the only place SVE-style instructions run on it.
35. Downs, T. (2019). A Note on Mask Registers. travisdowns.github.io/blog/2019/12/05/kreg-facts.html. The reference for the K0 / EVEX-encoding-0 distinction and the practical implications for AVX-512 mask register allocation.
36. Larabel, M. (2023). AVX-512 Performance Comparison: AMD Genoa vs. Intel Sapphire Rapids & Ice Lake. Phoronix. phoronix.com. Benchmarks with AVX-512 on and off on dual-socket Ice Lake, Sapphire Rapids, and Genoa servers; the final page reports a ~175 MHz peak-clock drop on Ice Lake, similar clocks with and without AVX-512 on Sapphire Rapids, and no AVX-512 downclocking on Genoa.
37. Yee, A. J. (2024, updated 2025). Zen5's AVX512 Teardown + More... numberworld.org. Measurements of Zen 5's AVX-512 implementation by the author of y-cruncher, including the comparison with Zen 4's 256-bit datapaths and the finding that the Strix Point laptop chips keep 4 × 256-bit execution.
38. Lam, C. (2025). Zen 5's AVX-512 Frequency Behavior. Chips and Cheese. chipsandcheese.com. Clock-speed traces for Zen 5 and Skylake-X when switching from scalar to AVX-512 code: no fixed offset on Zen 5, a gradual back-off under heavy 512-bit load plus FMA mixes, and a fixed drop plus transition period on Skylake-X.
39. Larabel, M. (2023). Initial Benchmarks Of The Intel Downfall Mitigation Performance Impact. Phoronix. phoronix.com. Application benchmarks before and after the Gather Data Sampling microcode update on affected Skylake-through-Ice-Lake / Tiger Lake systems, with Intel's own "up to 50%" worst-case figure for context.

Further reading

Not cited above, but worth the time:

• Jeffers, J., Reinders, J., & Sodani, A. (2016). Intel Xeon Phi Processor High Performance Programming: Knights Landing Edition. Morgan Kaufmann. The programming guide for the first AVX-512 hardware (Knights Landing); its chapters on masked operations and gather/scatter explain the mask model in depth.
• Valient, M. (2009). The Rendering Technology of Killzone 2. GDC. guerrilla-games.com. PS3-era renderer internals, including work moved onto the SPU vector processors.
• Giesen, F. (ryg). The ryg blog. fgiesen.wordpress.com. Practitioner writeups on SIMD, codec implementation, the graphics pipeline, and dependency-chain analysis of real kernels.
• Lemire, D. Daniel Lemire's blog. lemire.me/blog. Microbenchmarks and SIMD analyses on modern x86 and Arm, usually with published code.
• Hennessy, J. L., & Patterson, D. A. (2017). Computer Architecture: A Quantitative Approach (6th ed.). Morgan Kaufmann. The textbook on pipelining, out-of-order execution, and cache hierarchies. Chapter 4 (data-level parallelism) is the long-form version of §3 and §7 of this tutorial.
• Intel Corporation. Intel® VTune™ Profiler User Guide. intel.com. Intel's profiler; its microarchitecture analyses attribute stalls to ports, memory levels, and front-end limits on a running workload.