The Opcode Master: setupHeader, the L3 Wire-Opcode Word & the Per-Op Reconciliation Table

All symbols and addresses on this page apply to neuronx_cc 2.24.5133.0+58f8de22 (cp310 wheel). The encoder bodies and setupHeader live in neuronxcc/starfish/lib/libwalrus.so (.text/.rodata base 0x62d660 / 0x1c72000, VA == file offset; .plt thunks 0x5e9020–0x62d650 — cite the body, not a thunk). The defining bir::InstructionType enum is in libBIR.so (md5 12bb979f7ca41248252abb0f16b2da98). cp311/cp312 share the .text logic; the opcode integers and the L1/L2/L3 arithmetic are ABI-stable across cp3xx, only the VAs drift. Other wheels differ — treat every address as version-pinned. See versions.

Abstract

A BIR op carries three distinct integer identities, and conflating them is the most common cross-layer error when reading Walrus codegen. This page is the authoritative reconciliation of all three into one per-op master mapping:

• L1 — the bir::InstructionType ordinal (0..109). The IR class id: what the node is. Defined in libBIR.so, recovered byte-exact from InstructionType2string.
• L2 — the NEURON_ISA_TPB_OPCODE enum value the COLLECT_OPCODES arm inserts into a std::set<u32>. The silicon micro-op id: which engine micro-op the op compiles to. Engine-relative; written as a clean dword in the COLLECT arm and as a byte in the GENERATE arm.
• L3 — the 16-bit wire header word setupHeader stamps at bundle[0:2]: (0x10 << 8) | L2byte. The bytes that hit the 64-byte bundle on the wire.

The header word is produced by one function — setupHeader — and the headline structural fact is that it is byte-for-byte identical in CoreV2, CoreV3, and CoreV4: six instructions, 15 bytes, no data dependence on anything but the opcode byte. There is exactly one variation point in the whole 110-op set: the opcode byte differs per generation for a handful of ops (most prominently Activation: 0x21 on V2/V3 → 0x25 on V4), and QuantizeMx OR-s bit 0 of byte[1] (0x10 → 0x11) on SATURATE. Everything else is constant.

The deliverable is the 110-row × {V2,V3,V4} opcode-word table in §3, each row carrying its L1 ordinal, its L2 collect-int, its three per-arch L3 words, its bundle count, and the reconciliation flags. It is the source the per-engine encoding pages (PE matmul, Pool, Activation, …, DMA) and the planned opcode appendix cite for which 16-bit word lands at bundle[0:2].

1. setupHeader — byte-identical across all three generations

setupHeader(void* this, void* bundle, void* opcodeByte) writes the 4-byte header word. At all three nm-resolved addresses the bodies are identical to the byte (0f b6 02 c6 46 01 10 88 06 31 c0 66 89 46 02 c3):

// CoreV2GenImpl::setupHeader @0x1172120 — byte-identical to
// CoreV3 @0x1369280 and CoreV4 @0x143f440; only the entry RVA differs.
// System V: rdi=this, rsi=&bundle (64-B buffer), rdx=&opcodeByte.
void setupHeader(void* this_, uint8_t* bundle, const uint8_t* opcodeByte) {
    uint8_t op = *opcodeByte;        // 1172120  movzbl (%rdx),%eax
    bundle[1]  = 0x10;               // 1172123  movb   $0x10,0x1(%rsi)  -- inst_word_len = 16 dwords
    bundle[0]  = op;                 // 1172127  mov    %al,(%rsi)       -- the L3 opcode byte
    *(uint16_t*)(bundle + 2) = 0;    // 117212b  mov    %ax,0x2(%rsi)    (eax already 0 via xor)
    // 1172129  xor %eax,%eax  ;  117212f  ret
}

The 4-byte header it produces:

Read little-endian, *(u16*)bundle = 0x10NN where NN is the opcode byte. Are there ANY non-16 inst_word_len values? No. byte[1] is the literal 0x10 in all three bodies with no data dependence; every CoreV{2,3,4} bundle is exactly 64 bytes (emplace_back<std::array<u8,64>> + fwrite(…, 0x40, …)). The MX opcodes 0x1009 / 0x100A / 0x10E3 are full words: their high byte 0x10 is the length, the low byte (0x09 / 0x0A / 0xE3) is the opcode. The only op that perturbs the high byte is QuantizeMx, which OR-s bit 0 (0x10 → 0x11) when its SATURATE flag is set.

Dispatch. setupHeader is a virtual reached through the Generator vtable slot at +0x48 (slot 9). In the canonical encoder body (the CoreV2 Rng visitor visitInstRng @0x12376a0), the call site is call *0x48(%rax) @0x1237802, immediately after the opcode byte is materialized on the stack. The few MX ops write the word directly (mov $0x10NN,%esi ; mov %si,(bundle)) instead of calling setupHeader, but the wire bytes are byte-identical to the {opcode, 0x10, 0, 0} skeleton.

2. The emission skeleton — how a visitInst reaches setupHeader

Every CoreV* encoder shares one skeleton; the per-op deltas are body-field writes only. The codegen mode is read from *((int*)this + 156) ([this+0x270]):

Each opcode therefore leaves a dual witness in its encoder body: movl $op,-X(%rbp) (mode 0) and movb $op,-Y(%rbp) (mode 1, immediately before mov %rax,…+0x48 / call *0x48(%rax)). Both witnesses are present and agree for Pool 0x45, Activation(V2) 0x21, Activation(V4) 0x25, QuantizeMx 0xE3, Rng 0x4D, CollectiveSend/Recv 0xCB, MaxIndex 0x6D+0x6E, RangeSelect 0xBC, NonzeroWithCount 0xF2, and the MX word stores; the opcodes in §3 are read from that pair.

The engine is NOT a wire field. The physical engine rides the BIR Instruction at +0x90 (EngineType), pinned by I-strand codegen. The encoder only reads it for the census DenseMap key. The Eng column below is metadata, not a bundle field. Wire engine ordinals: {PE=0, ACT=1, POOL=2, DVE=3, SP=4}.

Class hierarchy & the per-arch override rule

Generator → CoreV2Gen/CoreV2GenImpl → CoreV3Gen/CoreV3GenImpl → CoreV4Gen/CoreV4GenImpl. Target is selected by std::variant<monostate, CoreV2Gen, CoreV3Gen, CoreV4Gen>. The opcode for an op is the opcode in the most-derived override that exists for the running target; absent an override the base (CoreV2) opcode is inherited. The override rosters (nm-counted):

• CoreV2GenImpl::visitInst* — 68 distinct bodies (the full base set).
• CoreV3GenImpl::visitInst* — 13 overrides: TensorTensor, Matmult, MatmultSparse, CoreBarrier, GPSIMDSB2SB, GetSequenceBounds, RangeSelect, MaxIndexAndMatchReplace, NonzeroWithCount, GetRandState, RandGetState, RandSetState, SetRandState.
• CoreV4GenImpl::visitInst* — 5 overrides: Activation, Exponential, MatmultMx, QuantizeMx, Rand2.

Critically, most overrides keep the same opcode byte — the override only extends the int32 / extended-engine field bands (e.g. the CoreV3 DVE overrides TensorTensor, RangeSelect, MaxIndexAndMatchReplace, NonzeroWithCount). The genuine per-arch opcode renumber is rare; see §4(b).

3. The master 110-row opcode-word table (L1 | L2 | L3 per arch)

The headline deliverable. Opcode words shown as 0x10NN (NN = L2 low byte). V2word = CoreV2 wire word; V3word/V4word: = inherits the prior arch verbatim, — = not encoded on that arch (gen-gated), (lowered) = no encoder. #B = 64-byte bundles this one BIR op emits. Flags: [≠] L1≠L2, [COLn] N ITs collide on one L2, [ΔVn] per-arch opcode renumber, [NEWvn] gen-new op, [MB:n] multi-bundle, [SPLIT] one IT forks opcodes by a runtime flag, [LOW] lowered / no encoder, [BASE] abstract base, [+1=cast] low bit = dtype-cast flag.

NOTE-A (IndirectLoad/Save L2 byte) — a split, not a single value. generateIndirectLoadSave emits two different opcodes, chosen at runtime by the bound-check flag at Inst+0x1ED. With the flag set the op encodes as 0xC4 (range/bound-checked); that arm is guarded by the diagnostic "Indirect DMA bound-check can only be on Pool engine", so bound-checked indirect DMA is Pool-only. With the flag clear it encodes as 0xD6 (plain). Both values reach the per-instruction std::set<u32> through ordinary COLLECT-arm _M_insert_unique calls in the same function, which is why the op carries [SPLIT] rather than one L2 byte.

Anchors: generateIndirectLoadSave @ 0x1268c00; flag test cmp byte ptr [rax+0x1ED],0 @ 0x1268d0c/0x1268d13; movl $0xC4,-0xB0(%rbp) @ 0x1268f89 + _M_insert_unique @ 0x1268f96; movl $0xD6,-0xB0(%rbp) @ 0x1269576 + _M_insert_unique @ 0x1269583.

NOTE-B (RangeSelect second word). The V3 RangeSelect encoder @ 0x135f8c0 may append a 0x9B (DveReadAccumulator) drain word, so the practical emission is 2 words: primary 0xBC (188) + accumulator-drain 0x9B. The op opcode itself is 0xBC; the appended drain word is a HIGH-confidence read.

NOTE-C (NonzeroWithCount). L1 IT = 104, agreed by both InstructionType2string and the sameInst arm; L2/L3 opcode = 0xF2 (242).

GOTCHA — Do not read NonzeroWithCount as "bir 231". 231 = 0xE7 is IndirectCopy's L2 opcode, not this op's instruction-type ordinal; there is no 231-vs-242 IT split.

IT 5 ReadActivationAccumulator and IT 101 ActivationReadAccumulator are two distinct read-accumulator bodies (visitInstReadActivationAccumulator @0x125b830, visitInstActivationReadAccumulator @0x1234f70) that both stamp 0x24.

4. Cross-cutting flag rosters

(a) L2 non-injective collisions [COLn] — distinct ITs → ONE L2 opcode

The wire opcode is never a unique per-IT key; these collisions are real and intentional, disambiguated by bundle fields / direction / engine:

NET: 9 collision groups. A reimplementation must never assume opcode↔IT is 1:1.

(b) Per-arch L3 differences [ΔV4] / [ΔV3] / [NEWvn]

• Genuine opcode renumber (different byte, same op, across arch): Activation IT4 — V2=0x21 / V3=0x21(inh) / V4=0x25. This is the single clearest V2→V4 opcode-byte divergence; both bytes are byte-verified in the binary (§5). The V4 body opens with an act-accumulator-mode [rax+0x128] test (the fused-activation slot).
• GEN3-NEW (no CoreV2 sibling → abs→opcode at V3): MatmultSparse(9) 0x06+0x07, GPSIMDSB2SB(33) 0xBF, RangeSelect(63) 0xBC, GetSequenceBounds(64) 0xBE, CoreBarrier(87) 0xD8, MaxIndexAndMatchReplace(91) 0x6D+0x6E, RandSetState(98) 0x78, RandGetState(99) 0x77, NonzeroWithCount(104) 0xF2, SetRandState(59) Pool-arm 0x78.
• GEN4-NEW (abs→opcode at V4): MatmultMx(95) 0x09+0x0A, QuantizeMx(96) 0xE3, Rand2(97) 0xE2, Exponential(103) 0x30, plus the Activation(4) re-emit 0x25.
• Field-extension ONLY (opcode UNCHANGED — V3/V4 widen the int32 / extended bands; do not mistake for renumbers): TensorTensor(31), TensorScalar/Ptr(28/29), TensorScalarCache(30), RangeSelect/NonzeroWithCount field bands.

(c) Multi-bundle ops [MB:n] — one IT → n wire words

NET: 11 multi-bundle ops. A reimplementation MUST iterate the op's COLLECT set (a std::set<u32> legally holding >1 opcode), not assume one opcode per instruction.

(d) Lowered-away / no-encoder ITs [LOW]

• Abstract / lowered (LowerAC etc.): Generic(0), GenericRelu(2), AbstractCopy(3), GenericCopy(1), TensorCopyDynamicSrc(24), TensorCopyDynamicDst(25), GenericIndirectLoad(42), GenericIndirectSave(44), Select(51→GenericCopy+0xEA), Loop(105), DynamicForLoop(106), DoWhile(108), DMABlock(107), TongaReduceMacroSymbolic(109).
• Kernel / container (no ISA bundle): BIRKernel(54), NKIKernel(55), NKIKLIRKernel(56).
• Debug / control (no setupHeader): DevicePrint(57, protobuf only), Break(83).
• No CoreV* encoder (other op ships): Rand(60 → Rand2/IT97).
• Abstract bases (leaves carry opcodes): MatmultBase(7), DMA(18), Collective(47), Terminator(77), DMADescriptor(68) + family (69–72).
• User-defined opcode: InlineASMBytes(93) — opcode = user byte0.

NET: ~25 ITs with no own fixed opcode. Of the 110 ITs, ~85 reach a concrete L2/L3 opcode (counting field-split forks once).

5. Evidence anchors and limits

For every opcode below the COLLECT dword store (mode 0) and the GENERATE byte store (mode 1) carry the same low value, and the L3 word equals 0x10<<8 | byte.

Further byte-exact anchors: Pool movl $0x45 @0x1239f51; Rng movl $0x4d @0x1237788; MaxIndex two arms in one body movl $0x6d @0x12546d3 + movl $0x6e @0x1254732; MX words mov $0x1009,%esi @0x143e4cc + mov $0x100a,%esi @0x143ed5a; the +0x48 vtable dispatch call *0x48(%rax) @0x1237802.

Limits. Directly pinned in the binary: the three setupHeader bodies, the +0x48 dispatch, and 11 opcode stores across V2/V3/V4 (rows 1–5 above plus Pool/Rng/MaxIndex/MX). The remaining ~70 opcode integers come from per-op reads of each _M_insert_unique site — HIGH confidence, consistent with the L2≡L3-low invariant, but not individually re-disassembled here. The L2-name column (enum_variant_string_opcode core_v4 decoder, reported at 0x143fd80) is HIGH: the helper is an internal symbol absent from the nm table, so its address could not be independently pinned. The [LOW] classification of ITs 105–109 is [INFERRED] — no visitInst body was located, consistent with the absence of a sameInst arm — and encoder presence for IT107 DMABlock / IT108 DoWhile / IT109 TongaReduceMacroSymbolic remains [SPECULATIVE].

See also

• The 64-Byte Instruction Bundle & Header Skeleton — the bundle this header word opens; the slot families that fill the other 60 bytes.
• InstructionType — the L1 enum (110 ordinals) this table maps from; sameInst and the family predicates.
• The Walrus codegen driver page (Part 8) — the driver that selects the CoreV{2,3,4} target and calls setupHeader (in-flight at time of writing).
• The per-engine encoding pages (PE matmul, Pool, Activation, DVE, DMA, …) fill in the body fields for each opcode word above.
• The planned opcode appendix (Part 14) cites this page as the authoritative IT→opcode reference.