The Opcode Master: setupHeader, the L3 Wire-Opcode Word & the Per-Op Reconciliation Table

All symbols and addresses on this page apply to neuronx_cc 2.24.5133.0+58f8de22 (cp310 wheel). The encoder bodies and setupHeader live in neuronxcc/starfish/lib/libwalrus.so (.text/.rodata base 0x62d660 / 0x1c72000, VA == file offset; .plt thunks 0x5e9020–0x62d650 — cite the body, not a thunk). The defining bir::InstructionType enum is in libBIR.so (md5 12bb979f7ca41248252abb0f16b2da98). cp311/cp312 share the .text logic; the opcode integers and the L1/L2/L3 arithmetic are ABI-stable across cp3xx, only the VAs drift. Other wheels differ — treat every address as version-pinned. See versions.

Abstract

A BIR op carries three distinct integer identities, and conflating them is the single most common cross-layer error in earlier opcode maps. This page is the authoritative reconciliation of all three into one per-op master mapping:

• L1 — the bir::InstructionType ordinal (0..109). The IR class id: what the node is. Defined in libBIR.so, recovered byte-exact from InstructionType2string. CONFIRMED.
• L2 — the NEURON_ISA_TPB_OPCODE enum value the COLLECT_OPCODES arm inserts into a std::set<u32>. The silicon micro-op id: which engine micro-op the op compiles to. Engine-relative; written as a clean dword in the COLLECT arm and as a byte in the GENERATE arm.
• L3 — the 16-bit wire header word setupHeader stamps at bundle[0:2]: (0x10 << 8) | L2byte. The bytes that hit the 64-byte bundle on the wire.

The header word is produced by one function — setupHeader — and the headline structural fact is that it is byte-for-byte identical in CoreV2, CoreV3, and CoreV4: six instructions, 15 bytes, no data dependence on anything but the opcode byte. There is exactly one variation point in the whole 110-op set: the opcode byte differs per generation for a handful of ops (most prominently Activation: 0x21 on V2/V3 → 0x25 on V4), and QuantizeMx OR-s bit 0 of byte[1] (0x10 → 0x11) on SATURATE. Everything else is constant.

The deliverable is the 110-row × {V2,V3,V4} opcode-word table in §3, each row carrying its L1 ordinal, its L2 collect-int, its three per-arch L3 words, its bundle count, and the reconciliation flags. It is the source the per-engine encoding pages (PE matmul, Pool, Activation, …, DMA) and the planned opcode appendix cite for which 16-bit word lands at bundle[0:2].

1. setupHeader — byte-identical across all three generations

setupHeader(void* this, void* bundle, void* opcodeByte) writes the 4-byte header word. Disassembled directly from the binary at all three nm-confirmed addresses, the bodies are identical to the byte (0f b6 02 c6 46 01 10 88 06 31 c0 66 89 46 02 c3):

// CoreV2GenImpl::setupHeader @0x1172120  (CONFIRMED — byte-identical to
// CoreV3 @0x1369280 and CoreV4 @0x143f440; only the entry RVA differs).
// System V: rdi=this, rsi=&bundle (64-B buffer), rdx=&opcodeByte.
void setupHeader(void* this_, uint8_t* bundle, const uint8_t* opcodeByte) {
    uint8_t op = *opcodeByte;        // 1172120  movzbl (%rdx),%eax
    bundle[1]  = 0x10;               // 1172123  movb   $0x10,0x1(%rsi)  -- inst_word_len = 16 dwords
    bundle[0]  = op;                 // 1172127  mov    %al,(%rsi)       -- the L3 opcode byte
    *(uint16_t*)(bundle + 2) = 0;    // 117212b  mov    %ax,0x2(%rsi)    (eax already 0 via xor)
    // 1172129  xor %eax,%eax  ;  117212f  ret
}

The 4-byte header it produces:

Read little-endian, *(u16*)bundle = 0x10NN where NN is the opcode byte (CONFIRMED). Are there ANY non-16 inst_word_len values? No. byte[1] is the literal 0x10 in all three bodies with no data dependence; every CoreV{2,3,4} bundle is exactly 64 bytes (emplace_back<std::array<u8,64>> + fwrite(…, 0x40, …)). The MX opcodes 0x1009 / 0x100A / 0x10E3 are full words: their high byte 0x10 is the length, the low byte (0x09 / 0x0A / 0xE3) is the opcode. The only op that perturbs the high byte is QuantizeMx, which OR-s bit 0 (0x10 → 0x11) when its SATURATE flag is set.

Dispatch. setupHeader is a virtual reached through the Generator vtable slot at +0x48 (slot 9). In the canonical encoder body (the CoreV2 Rng visitor visitInstRng @0x12376a0), the call site is call *0x48(%rax) @0x1237802, immediately after the opcode byte is materialized on the stack (CONFIRMED — disassembled this pass). The few MX ops write the word directly (mov $0x10NN,%esi ; mov %si,(bundle)) instead of calling setupHeader, but the wire bytes are byte-identical to the {opcode, 0x10, 0, 0} skeleton.

2. The emission skeleton — how a visitInst reaches setupHeader

Every CoreV* encoder shares one skeleton; the per-op deltas are body-field writes only. The codegen mode is read from *((int*)this + 156) ([this+0x270]):

The dual witness — movl $op,-X(%rbp) (mode 0) and movb $op,-Y(%rbp) (mode 1, immediately before mov %rax,…+0x48 / call *0x48(%rax)) — is how every opcode in §3 was re-pinned. This page verified the dual witness live in the binary for Pool 0x45, Activation(V2) 0x21, Activation(V4) 0x25, QuantizeMx 0xE3, Rng 0x4D, CollectiveSend/Recv 0xCB, MaxIndex 0x6D+0x6E, RangeSelect 0xBC, NonzeroWithCount 0xF2, and the MX word stores.

The engine is NOT a wire field. The physical engine rides the BIR Instruction at +0x90 (EngineType), pinned by I-strand codegen. The encoder only reads it for the census DenseMap key. The Eng column below is metadata, not a bundle field. Wire engine ordinals: {PE=0, ACT=1, POOL=2, DVE=3, SP=4}.

Class hierarchy & the per-arch override rule

Generator → CoreV2Gen/CoreV2GenImpl → CoreV3Gen/CoreV3GenImpl → CoreV4Gen/CoreV4GenImpl. Target is selected by std::variant<monostate, CoreV2Gen, CoreV3Gen, CoreV4Gen>. The opcode for an op is the opcode in the most-derived override that exists for the running target; absent an override the base (CoreV2) opcode is inherited. The override rosters (nm-counted):

• CoreV2GenImpl::visitInst* — 68 distinct bodies (the full base set).
• CoreV3GenImpl::visitInst* — 13 overrides: TensorTensor, Matmult, MatmultSparse, CoreBarrier, GPSIMDSB2SB, GetSequenceBounds, RangeSelect, MaxIndexAndMatchReplace, NonzeroWithCount, GetRandState, RandGetState, RandSetState, SetRandState.
• CoreV4GenImpl::visitInst* — 5 overrides: Activation, Exponential, MatmultMx, QuantizeMx, Rand2.

Critically, most overrides keep the same opcode byte — the override only extends the int32 / extended-engine field bands (e.g. the CoreV3 DVE overrides TensorTensor, RangeSelect, MaxIndexAndMatchReplace, NonzeroWithCount). The genuine per-arch opcode renumber is rare; see §4(b).

3. The master 110-row opcode-word table (L1 | L2 | L3 per arch)

The headline deliverable. Opcode words shown as 0x10NN (NN = L2 low byte). V2word = CoreV2 wire word; V3word/V4word: = inherits the prior arch verbatim, — = not encoded on that arch (gen-gated), (lowered) = no encoder. #B = 64-byte bundles this one BIR op emits. Flags: [≠] L1≠L2, [COLn] N ITs collide on one L2, [ΔVn] per-arch opcode renumber, [NEWvn] gen-new op, [MB:n] multi-bundle, [SPLIT] one IT forks opcodes by a runtime flag, [LOW] lowered / no encoder, [BASE] abstract base, [+1=cast] low bit = dtype-cast flag.

*NOTE-A (IndirectLoad/Save L2 byte) — SPLIT, not a single value. CORRECTION (#827 audit). An earlier draft resolved this op to a single L2 0xD6 and dismissed the competing 0xC4 reading as "the indirect-AP sub-word or an older constant." That is wrong: generateIndirectLoadSave @0x1268c00 inserts both opcodes into the per-instruction std::set<u32> via genuine COLLECT-arm _M_insert_unique calls, selected at runtime by the bound-check flag Inst+0x1ED. The body branches on cmp byte ptr [rax+0x1ED],0 ; jz loc_1268FD0 @0x1268d0c/0x1268d13: bound-check set → movl $0xC4,-0xB0(%rbp) @0x1268f89 → _M_insert_unique @0x1268f96 (the path guarded by "Indirect DMA bound-check can only be on Pool engine"); bound-check clear (the jz target) → movl $0xD6,-0xB0(%rbp) @0x1269576 → _M_insert_unique @0x1269583. Both have identical set-insert status in the same function, so the op is [SPLIT] — 0xC4 (range/bound-checked) / 0xD6 (plain). CONFIRMED — both movl + _M_insert_unique sites and the Inst+0x1ED guard byte-verified.

★NOTE-B (RangeSelect second word). The V3 RangeSelect encoder (0x135f8c0) may append a 0x9B (DveReadAccumulator) drain word, so the practical emission is 2 words: primary 0xBC (188) + accumulator-drain 0x9B. The op opcode is 0xBC. STRONG.

★NOTE-C (NonzeroWithCount). L1 IT = 104 (CERTAIN, from both InstructionType2string and the sameInst arm); L2/L3 opcode = 0xF2 (242). An earlier "bir 231" annotation conflates 231 = 0xE7 = IndirectCopy's L2 with this op's IT — there is no 231-vs-242 IT split. CONFIRMED.

IT 5 ReadActivationAccumulator and IT 101 ActivationReadAccumulator are two distinct read-accumulator bodies (visitInstReadActivationAccumulator @0x125b830, visitInstActivationReadAccumulator @0x1234f70) that both stamp 0x24.

4. Cross-cutting flag rosters

(a) L2 non-injective collisions [COLn] — distinct ITs → ONE L2 opcode

The wire opcode is never a unique per-IT key; these collisions are real and intentional, disambiguated by bundle fields / direction / engine:

NET: 9 collision groups. A reimplementation must never assume opcode↔IT is 1:1.

(b) Per-arch L3 differences [ΔV4] / [ΔV3] / [NEWvn]

• Genuine opcode renumber (different byte, same op, across arch): Activation IT4 — V2=0x21 / V3=0x21(inh) / V4=0x25. This is the single clearest V2→V4 opcode-byte divergence; both bytes are byte-verified in the binary (§5). The V4 body opens with an act-accumulator-mode [rax+0x128] test (the fused-activation slot).
• GEN3-NEW (no CoreV2 sibling → abs→opcode at V3): MatmultSparse(9) 0x06+0x07, GPSIMDSB2SB(33) 0xBF, RangeSelect(63) 0xBC, GetSequenceBounds(64) 0xBE, CoreBarrier(87) 0xD8, MaxIndexAndMatchReplace(91) 0x6D+0x6E, RandSetState(98) 0x78, RandGetState(99) 0x77, NonzeroWithCount(104) 0xF2, SetRandState(59) Pool-arm 0x78.
• GEN4-NEW (abs→opcode at V4): MatmultMx(95) 0x09+0x0A, QuantizeMx(96) 0xE3, Rand2(97) 0xE2, Exponential(103) 0x30, plus the Activation(4) re-emit 0x25.
• Field-extension ONLY (opcode UNCHANGED — V3/V4 widen the int32 / extended bands; do not mistake for renumbers): TensorTensor(31), TensorScalar/Ptr(28/29), TensorScalarCache(30), RangeSelect/NonzeroWithCount field bands.

(c) Multi-bundle ops [MB:n] — one IT → n wire words

NET: 11 multi-bundle ops. A reimplementation MUST iterate the op's COLLECT set (a std::set<u32> legally holding >1 opcode), not assume one opcode per instruction.

(d) Lowered-away / no-encoder ITs [LOW]

• Abstract / lowered (LowerAC etc.): Generic(0), GenericRelu(2), AbstractCopy(3), GenericCopy(1), TensorCopyDynamicSrc(24), TensorCopyDynamicDst(25), GenericIndirectLoad(42), GenericIndirectSave(44), Select(51→GenericCopy+0xEA), Loop(105), DynamicForLoop(106), DoWhile(108), DMABlock(107), TongaReduceMacroSymbolic(109).
• Kernel / container (no ISA bundle): BIRKernel(54), NKIKernel(55), NKIKLIRKernel(56).
• Debug / control (no setupHeader): DevicePrint(57, protobuf only), Break(83).
• No CoreV* encoder (other op ships): Rand(60 → Rand2/IT97).
• Abstract bases (leaves carry opcodes): MatmultBase(7), DMA(18), Collective(47), Terminator(77), DMADescriptor(68) + family (69–72).
• User-defined opcode: InlineASMBytes(93) — opcode = user byte0.

NET: ~25 ITs with no own fixed opcode. Of the 110 ITs, ~85 reach a concrete L2/L3 opcode (counting field-split forks once).

5. Adversarial self-verification — strongest claims re-checked vs the binary

Each opcode below was re-disassembled this pass at the cited address. The COLLECT dword store (mode 0) and the GENERATE byte store (mode 1) carry the same low value, and the L3 word equals 0x10<<8 | byte. All are byte-exact.

Additional spot-checks this pass (all byte-exact): Pool movl $0x45 @0x1239f51; Rng movl $0x4d @0x1237788; MaxIndex two arms in one body movl $0x6d @0x12546d3 + movl $0x6e @0x1254732; MX words mov $0x1009,%esi @0x143e4cc + mov $0x100a,%esi @0x143ed5a; the +0x48 vtable dispatch call *0x48(%rax) @0x1237802.

Re-verify ceiling (honest). Independently re-pinned this pass: the three setupHeader bodies, the +0x48 dispatch, and 11 opcode stores across V2/V3/V4 (rows 1–5 above plus Pool/Rng/MaxIndex/MX). The remaining ~70 opcode integers are transcribed from prior per-op reads of each _M_insert_unique site — graded STRONG (consistent with the L2≡L3-low invariant, but not each re-disassembled here). The L2-name column (enum_variant_string_opcode core_v4 decoder, said to live @0x143fd80) is STRONG: the helper is an internal symbol not exported in the nm table, so its address could not be independently re-pinned on this page. The [LOW] classification of ITs 105–109 is INFERRED (no visitInst body located; consistent with no sameInst arm); IT107 DMABlock / IT108 DoWhile / IT109 TongaReduceMacroSymbolic encoder presence is SPECULATIVE.

See also

• The 64-Byte Instruction Bundle & Header Skeleton — the bundle this header word opens; the slot families that fill the other 60 bytes.
• InstructionType — the L1 enum (110 ordinals) this table maps from; sameInst and the family predicates.
• The Walrus codegen driver page (Part 8) — the driver that selects the CoreV{2,3,4} target and calls setupHeader (in-flight at time of writing).
• The per-engine encoding pages (PE matmul, Pool, Activation, DVE, DMA, …) fill in the body fields for each opcode word above.
• The planned opcode appendix (Part 14) cites this page as the authoritative IT→opcode reference.