TriggerCollective2 + Ext (0xD9 / 0xDA)

The v2 / SBUF-tensor collective trigger. Where the base TriggerCollective (0xC8) addresses its operands as flat DRAM tensor handles (input_tensor_id / output_tensor_id u32 + num_elements u64), TriggerCollective2 lets the collective operate directly on SBUF (on-chip SRAM) tensors, which need a full 2-D access pattern (start address + 2 strides + 2 shape dims) per operand. That metadata does not fit in one 64-byte TPB instruction word, so v2 is encoded as two back-to-back 64-byte pseudo-instructions:

1. 0xD9 PSEUDO_TRIGGER_COLLECTIVE2 — op/dtype/ctype/cc_dim + the two 24-byte SBUF 2-D source descriptors src0, src1.
2. 0xDA PSEUDO_EXTENSION (the _ext struct) — instr_chaining, group_id/channel_id/stream_id, the 24-byte dst descriptor, an explicit collective-scratch (cc) buffer, DMA priority, A2Av/permute knobs.

Both pseudo-ops are translated host-side by the NRT runtime (they never reach hardware as-is); the host encoder picks an NCFW ring/kangaring, mesh, or hierarchical algorithm, and the only thing the GPSIMD device ever decodes is the lowered on-chip leg SB2SB_Collective (0xBF) plus its DMA descriptors.

Scope tags. Every fact is anchored to a header file:line, an ISA opcode/enum value, an IDA-sidecar addr/symbol, or a libnrt DWARF DIE. v2–v4 (sunda/cayman/mariana) are byte-grounded; v5 (maverick) struct interiors are INFERRED from identical bytes (the structs are proven byte-identical, see §1).

1. Arch coverage & byte-identity

The two headers ship under four neuron_<arch>_arch_isa/tpb/ trees. Hashing each header with only the one-line * ISA header for NC-vN. comment removed yields a single digest per file across all four arches:

[HIGH · OBSERVED] Both structs are byte-identical across all four arch variants — only the NC-vN doc comment differs. CAYMAN = NC-v3 (carried from the SM/arch map; line 3 of the cayman header reads * ISA header for NC-v3.). All citations below use the cayman copy.

2. Opcode encoding

From aws_neuron_isa_tpb_common.h NEURON_ISA_TPB_OPCODE enum and instruction_mapping.json (struct2opcode):

Pseudo-op class rule. 0xD9/0xDA lie in the pseudo range 0xC0..0xDF (upper three bits 0b110). The verbatim comment at common.h:263:

"NRT relies on the fact that all pseudo instructions have upper three bits of the opcode equal to 0b110. Pseudo instructions are generated by compiler and translated into non-pseudo HW instructions by NRT."

0xD9 = 0b1101_1001 and 0xDA = 0b1101_1010 both satisfy this — neither opcode ever executes as-is on hardware; the runtime rewrites them at model load. [HIGH · OBSERVED]

GOTCHA — 0xDA is a shared extension opcode. instruction_mapping.json binds PSEUDO_EXTENSION (0xDA) to two structs: NEURON_ISA_TPB_PSEUDO_TRIGGER_COLLECTIVE2_EXT_STRUCT and NEURON_ISA_TPB_PSEUDO_DMA_EXT_STRUCT. Disambiguation is positional — the ext is read as a collective2-ext iff it immediately follows a 0xD9. The ext header (…collective2_ext.h:13-22) mandates the pair be contiguous: "This instruction must follow a PseudoTriggerCollective2 instruction … immediately in the instruction stream … we expect these two instructions to always be back-to-back." [HIGH · OBSERVED]

3. Struct #1 — PSEUDO_TRIGGER_COLLECTIVE2 (0xD9), 64 B

Source: aws_neuron_isa_tpb_pseudo_trigger_collective2.h:32-41. Offsets/sizes are compile-verified (gcc, sizeof == 64):

DELTA vs base 0xC8. There is no input_tensor_id/output_tensor_id u32 handle, no top-level num_elements u64, no src/dst_offset_elems. Operands are full SBUF 2-D access patterns; element count is implicit in each descriptor's num_elem[2]. ctype moved from offset 32 (base) to 14, and a new cc_dim field occupies offset 15. [HIGH · OBSERVED]

3a. src1_bitmap — A2Av metadata

For ALL_TO_ALL_V, src1 (from struct #1) is reinterpreted as a tensor whose rows carry per-rank send/recv counts and displacements; the src1_bitmap field in the ext (§4) selects which rows are present. Verbatim from …collective2_ext.h:70-89:

row0: send counts        bit0: send count
row1: send displacement  bit1: send displacement
row2: recv count         bit2: recv count
row3: recv displacement  bit3: recv displacement

We expect bits 0 and 1 to always be set for A2Av. Bits 2 and 3 are optional.
When bit 2 is clear, the recv count row is still present in the src1 tensor
but is treated as an OUTPUT location populated with the actual recv count
during execution of the A2Av operation.

The libnrt encoder reads exactly these bits — observed assertions src1_dynamic_bitmap & send_count_bitmask, … & send_displ_bitmask, the size predicate enc_op != ENC_ALLTOALL_V || src1_size_bytes / src1_dtype >= rank_n * 4, and the validation src1 tensor has wrong num_elem and/or step_elem. So v2/ext natively supports variable-size all-to-all. **[HIGH · OBSERVED — header doc

• libnrt strings]**

4. Struct #2 — PSEUDO_TRIGGER_COLLECTIVE2_EXT (0xDA), 64 B

Source: aws_neuron_isa_tpb_pseudo_trigger_collective2_ext.h:57-92. Compile- verified sizeof == 64, every offset confirmed:

NOTE. All "NEW" fields are capabilities the base 0xC8 left to runtime defaults: explicit channel_id/stream_id concurrency, an explicit cc scratch buffer, a DMA priority class, A2Av metadata, cross-SBUF replication, and instruction-bundle chaining. [HIGH · OBSERVED]

Supporting type layouts

All compile-verified against common.h (cayman copy):

// common.h:625 — the SBUF 2-D access pattern (24 B). sizeof == 24.
typedef struct NEURON_ISA_TPB_PSEUDO_ADDR8TENSOR2D {
    NEURON_ISA_TPB_PSEUDO_ADDR8 start_addr;   // 8  union (common.h:616)
    int32_t                     step_elem[2]; // 8  2-D signed strides, in elements
    uint32_t                    num_elem[2];  // 8  2-D shape, element counts
} NEURON_ISA_TPB_PSEUDO_ADDR8TENSOR2D;

// common.h:616 — start_addr marker byte selects the resolution mode:
//   addr_immediate | addr_reg | addr_table | addr_tbl_offs | addr_var | addr_unknown
//   (addr_var = PseudoAddrVar8 = neff var id + imm offset, resolved at model load)

// common.h:499 — ADDR4 (4 B union) used by cc_buf_start_addr
typedef union NEURON_ISA_TPB_ADDR4 {
    NEURON_ISA_TPB_PARTITION_OFFSET addr_immediate; // u32 partition offset
    NEURON_ISA_TPB_ADDR_REG4        addr_reg;       // {regnum,…,marker}
} NEURON_ISA_TPB_ADDR4;

// common.h:713 — DMA QoS (1 B, packed)
typedef struct NEURON_ISA_TPB_DMA_CONFIGS {
    uint8_t priority_class    : 3;  // DMA QoS class
    uint8_t reserved_bitfield : 5;  // must be zero
} NEURON_ISA_PACKED NEURON_ISA_TPB_DMA_CONFIGS;

QUIRK — priority_class is range-checked to 0..4, not 0..7. Although the field is 3 bits wide (0..7), the ISA validation pseudocode at common.h:2070-2073 reads is_valid_dma_configs(d) = (d.priority_class <= 4) && (d.reserved_bitfield == 0). Only classes 0–4 are legal. [HIGH · OBSERVED] The device side honours this map: libnrtucode_internal.so exports nrtucode_core_dge_{set,get}_priority_class_map (string priority_classes) — the DGE (DMA Generation Engine) priority-class table this 3-bit field indexes.

NEURON_ISA_TPB_HEADER (4 B) and NEURON_ISA_TPB_EVENTS (8 B) are identical to the base op (compile-verified sizeof 4 / 8).

5. ISA enum values

NEURON_ISA_TPB_COLLECTIVE_TYPE — ctype, off 14 (common.h:792-803)

Identical enum to base 0xC8 — only the field offset moved 32 → 14. Cross-reference collective-enums.md. [HIGH · OBSERVED]

NEURON_ISA_TPB_COLLECTIVE_DIMENSION — cc_dim, off 15 (NEW; …collective2.h:27-30)

DIM_0 = 0x0, DIM_1 = 0x1. The host validates cc_dim against the op; libnrt strings (with the vendor's original typos preserved) [OBSERVED]:

%d rank cc_dim 0 sb2sb expectes input is distributed to ranks in free dimension.
    But got input num_elem (%ld,%ld) and ouptut num_elem (%ld,%ld).
%d rank cc_dim 0 sb2sb expectes inputs to be concatenated in free dimension. …
Collective instruction: does not support cc_dim 0 for this op %d
Collective instruction: does not support cc_dim 1 for this op %d
Invalid cc_dim: %d

cc_dim selects whether the collective shards/concatenates along the SBUF free dimension (DIM_0) or partition dimension (DIM_1); not all ctypes accept both. Field + values are [HIGH · OBSERVED]; the exact DIM_0 = free / DIM_1 = partition reading is [MED · INFERRED] from the "free dimension" wording on the cc_dim 0 path.

NEURON_ISA_TPB_DTYPE — dtype, off 13 (common.h:722-739)

BFLOAT16=0x6 FP16=0x7 FP32=0xA FP32R=0xB FP8_EXP3=0xD FP8_EXP4=0xE FP8_EXP5=0xF UINT8=0x3 UINT16=0x5 UINT32=0x9 UINT64=0x1 INT8=0x2 INT16=0x4 INT32=0x8 INT64=0xC.

dtype passes through verbatim to the DMA descriptor. The libnrt DWARF enum SDMA_DTYPE uses the same encodings (BFLOAT16=6, FP16=7, FP32=10, FP32R=11, FP8_E3=13, FP8_E4=14, FP8_E5=15, INT8=2, INT16=4, INT32=8, INT64=12, UINT8=3, UINT16=5, UINT32=9, UINT64=1). [HIGH · OBSERVED — DWARF enum]

NEURON_ISA_TPB_ALU_OP — op, off 12 (common.h:940-982)

BYPASS=0x00 ADD=0x04 SUBTRACT=0x05 MULT=0x06 DIVIDE=0x07 MAX=0x08 MIN=0x09; int-engine forms ADD_INT=0xC4 MAX_INT=0xCF MIN_INT=0xD0. Practically a reducing collective uses ADD / MAX / MIN. This ISA op lowers to the DMA-side reduce enum SDMA_CCETYPE (§7).

Ext-only enums

6. Two-instruction decode & chaining

The NRT collectives decoder (libnrt.so, DWARF source path /opt/workspace/KaenaRuntime/nrt/nrt_collectives.cpp) carries an encoder-context struct whose members are exactly the parse-state for the back-to-back pair. From the DWARF DW_AT_data_member_location of the context DIE [HIGH · OBSERVED]:

// encoder context (member byte offsets from DWARF):
bool    collective2_in_progress;      // @ +297  (type: bool)
uint8_t collective2_instr[64];        // @ +298  (array_type, DW_AT_upper_bound 63)
uint8_t collective2_first_instr[64];  // @ +363  (array_type, DW_AT_upper_bound 63)
int     chained_cc_op_n;              // @ +427  (chaining counter)

Both buffers are exactly one 64 B instruction word (upper_bound 63 ⇒ 64 elements), which is the hard proof of the 2×64 B model. On seeing 0xD9 the decoder copies the word into collective2_first_instr and sets collective2_in_progress = true; the next instruction must be the 0xDA ext, combined with the buffered first word into one logical cc-op.

Chaining. instr_chaining (NONE/INITIAL/BODY/FINAL) together with chained_cc_op_n (and the related __chained_cc_op_n member) lets several collective2 pairs be fused into one bundle — INITIAL opens, BODY continues, FINAL closes, NONE = standalone.

The struct members and array bounds are [HIGH · OBSERVED]; the exact FSM transition order is [MED · INFERRED] from the enum + member names — no traced state machine. permute_chain (ext off 43) is a separate "protected run of permutes" marker set via encd_set_ctx_permute_chain (libnrt @ 0x238b30), distinct from instr_chaining. [OBSERVED]

7. Lowering → firmware → NCFW dispatch

Stage A — compiler

The compiler emits the 0xD9 + 0xDA pair into the NEFF instruction stream for the TOP_SP engine (see top-sp-lowering.md). Pseudo class 0b110 ⇒ not HW-native. [HIGH — opcode-class rule + header "generated by compiler"]

Stage B — host NRT: decode + lower

libnrt.so decodes the pair (Stage B above) and builds an enc_operation / cc-op record. The compiler's own collective vocabulary lives in DWARF/IDA enums [HIGH · OBSERVED] — note the encoder enum is richer than the ISA ctype:

// IDA enums.json — enc_op_type (cc_op KIND; maps from ISA ctype)
ENC_ALLGATHER=0 ENC_ALLREDUCE=1 ENC_BROADCAST=2 ENC_REDUCE=3
ENC_REDUCE_SCATTER=4 ENC_SEND=5 ENC_RECV=6 ENC_ALLTOALL=7 ENC_PERMUTE=8
ENC_PERMUTE_REDUCE=9 ENC_PERMUTE_IMPLICIT=10 ENC_PERMUTE_REDUCE_IMPLICIT=11
ENC_ALLTOALL_V=12 ENC_OP_INVALID=13

// enc_comm_type (topology scope)
H_COMM_INTRA_ID=0  H_COMM_INTER_ID=1  H_COMM_MAX_ID=2

// SDMA_CCETYPE (the DMA reduce op — ISA ALU_OP `op` lowers to this)
SDMA_CCETYPE_ADD=0 FMA=1 MAX=2 MIN=3 EXT=4 GCE=5

CORRECTION (vs. SX-CCL-02 backing report). The report's enc_op_type table stopped at ENC_PERMUTE_IMPLICIT=10. The actual libnrt enum continues: ENC_PERMUTE_REDUCE_IMPLICIT=11, ENC_ALLTOALL_V=12, ENC_OP_INVALID=13. The presence of ENC_ALLTOALL_V is consistent with the A2Av path (§3a) and the enc_mesh_primitive::__compose_alltoallv_full_mesh() symbol.

Algorithm selection. enc_hier_primitive::__select_algorithms() (libnrt @ 0x14c620, [0x14c620, 0x14cb47], DWARF enc/enc_primitive.cc:13668) probes capability predicates and picks one enc_alg_type per direction. Its callees (from the callgraph sidecar) are exactly the predicates [HIGH · OBSERVED]:

It emits a per-direction selection logged verbatim as:

[nec_dev %d] Hier algorithm selections - alg_intra_allg %d alg_intra_redsct %d
             alg_inter_allr %d alg_inter_allg %d alg_inter_redsct %d

The chosen enc_alg_type is the algorithm selector — full value table [HIGH · OBSERVED]:

// IDA enums.json — enc_alg_type
ENC_ALG_RING=0  ENC_ALG_HIER=1  ENC_ALG_MESH=2  ENC_ALG_KANGARING=3
ENC_ALG_SINGLE_CYCLE_RING=4  ENC_ALG_INTRA_RDH=5  ENC_ALG_SINGLE_STEP_MESH=6
ENC_ALG_INTER_RDH=7  ENC_ALG_TWO_STEP_POD_MESH=8  ENC_ALG_LATENCY_OPT_MESH=9
ENC_ALG_BW_OPT_MESH=10  ENC_ALG_INVALID=11

// mesh sub-type (enc_alg_mesh.mesh_type, picked by enc_mesh_primitive::
//   __select_mesh_subtype @ 0x14b520) — enum enc_alg_mesh_type
ENC_ALG_FULL_MESH=0 ENC_ALG_GROUPED_MESH=1 ENC_ALG_MESH_TRN2=2
ENC_ALG_MESH_SWITCH=3 ENC_ALG_MESH_INVALID=4

Per-op algorithm legality. The encoder asserts the selected algorithm is in the legal set for each (direction, op), verbatim from libnrt assertion strings [HIGH · OBSERVED]:

(MESH is additionally selectable for 2-D SBUF inputs — see the log Two-dimensional input is only supported by mesh algorithm.)

Composing the operation. The hier/mesh/metaring primitives then build the concrete step program:

• enc_hier_primitive::compose_operation / __compose_{allreduce,allgather,redsct} — hierarchical.
• enc_mesh_primitive::compose_operation / __compose_*_full_mesh, __compose_alltoallv_full_mesh — mesh.
• enc_metaring_primitive::__compose_{allreduce,allgather,redsct}_channel and the …_kangaring variants — ring / kangaring.

These emit the ring/mesh step primitives enc_primitive::recv_reduce_{send,copy,copy_send,write}(SDMA_CCETYPE, …) and direct_reduce_send_{kangaring,permute}(SDMA_CCETYPE) — the SDMA descriptors that carry the reduce op + dtype + SBUF/DRAM addresses originally encoded in src0/src1/dst. The per-step reduce is performed in-transfer by the SDMA CCE engine (cce-in-transfer.md). [HIGH · OBSERVED]

channel_id / stream_id / group_id resolution. stream_id resolves via enc_context::stream_map_get_stream_id(uint32_t) (libnrt @ 0xf6900), bounded by stream_id < NEC_MAX_STREAM_N && ring_channel_id < NEC_MAX_CHANNELS and channel_id < metaring->channel_n. The runtime enforces a one-to-one stream_id ↔ group_id mapping (string "stream_id <-> group_id mapping violation. There should be a one-to-one mapping"). [HIGH · OBSERVED]

Stage C — NCFW firmware

The host serializes the per-collective record into the model profile / NCFW context; NCFW packs a cc-op spad-control entry whose byte0 holds a 4-bit algo_type (bits[0:3]) selecting the ring/mesh/hier union, plus a 3-bit algo_sub_type (bits[4:6]).

RECONCILIATION (the contribution this lane adds). NCFW analysis (firmware side) could not recover the algo_type → name mapping (no enum table in libncfw). It is recoverable from the host side: the 4-bit firmware algo_type is the low nibble of NRT's enc_alg_type (the values 0..10 fit in 4 bits), and algo_sub_type is the 3-bit enc_alg_mesh_type (FULL/GROUPED/TRN2/SWITCH). So firmware algo_type 0 = ENC_ALG_RING, 1 = ENC_ALG_HIER, 2 = ENC_ALG_MESH, 3 = ENC_ALG_KANGARING, 4 = ENC_ALG_SINGLE_CYCLE_RING, …

The enc_alg_type value table is [HIGH · OBSERVED]; the 1:1 identity with the firmware's 4-bit field is [MED · INFERRED] — matching width + shared ring/mesh/hier/kangaring vocabulary across both binaries. No byte-level host→device write of enc_alg_type into the spad entry was traced.

On-chip leg (what the device actually decodes)

The GPSIMD/POOL device ucode (libnrtucode_internal.so) decodes SB2SB_Collective (0xBF) via decode_sb2sb_collective, logging:

P%i: SB2SB_Collective : total_src_nelem = %zu, total_dst_nelem = %zu,
     dtype=%d, dst_nc=%zu, src_nc=%zu, mask=0x%x
P%i: SB2SB_Collective: num_chans=%0d, tpb_idx=%u

and driving Q7 iDMA / rdma descriptor generation (Q7: rdma_desc_gen …).

The pseudo TRIGGER_COLLECTIVE2/_ext is NOT in the device ucode's decode set. The complete Decode : log set is {ExtendedInstCopy, ExtendedInstCptcDecode, ExtendedInstRand{Get,Set}State, ExtendedInstTensorTensorArith, GetSequenceBounds, SB2SB_Collective, Sbuf2Sbuf} — no TriggerCollective. The device only ever sees the lowered HW legs; the pseudo-ops are resolved host-side. [HIGH · OBSERVED — strings in libnrtucode_internal.so]

Not routed through POOL 0xf0 dispatch. The POOL extended-opcode 0xf0 path dispatches custom-op kernels (cptc/copy/rand/tensor-tensor), not the collective trigger. [HIGH]

8. Base (0xC8) vs v2 (0xD9) vs ext (0xDA) — field diff

All rows grounded in the two v2 headers + the base header (…pseudo_trigger_collective.h:21-35):

Summary. "2" = operate on SBUF tensors with full 2-D access patterns (needs two instructions); "_ext" = the carrier for the destination plus every v2-only knob (channel/stream concurrency, explicit cc scratch, DMA priority, A2Av metadata, cross-SBUF replication, instruction-bundle chaining).

9. Confidence summary

HIGH · OBSERVED

• Both structs' full 64 B layout, every field type/width/offset (gcc-verified, sizeof == 64 each), byte-identical across sunda/cayman/mariana/maverick.
• Opcodes 0xD9/0xDA; instruction_mapping.json bindings; 0xDA shared with PSEUDO_DMA_EXT (positional disambiguation); pseudo-op class rule.
• All ISA enum values (COLLECTIVE_TYPE, COLLECTIVE_DIMENSION, ALU_OP, DTYPE, INSTR_CHAINING, UNIQUE_TENSORS, PERMUTE_CHAIN, CROSS_SBOUTPUT_TRANSFER), priority_class <= 4 validation.
• src1_bitmap A2Av row/bit semantics (verbatim header); libnrt A2Av asserts.
• Two-instruction decode-state members in libnrt DWARF (collective2_in_progress@297, collective2_instr[64]@298, collective2_first_instr[64]@363, chained_cc_op_n@427).
• Host enums enc_op_type (0..13), enc_comm_type (0..2), enc_alg_type (0..11), enc_alg_mesh_type (0..4), SDMA_CCETYPE (0..5), SDMA_DTYPE (= ISA DTYPE).
• __select_algorithms @ 0x14c620 → enc_can_post_* call graph; per-op algorithm-legality asserts; per-direction selection log.
• Device ucode decodes only the lowered SB2SB_Collective/DMA legs, not the pseudo trigger; not routed through POOL 0xf0.
• dtype passes through verbatim to the SDMA descriptor.

MED · INFERRED

• cc_dim DIM_0 = free / DIM_1 = partition exact reading.
• 1:1 identity between NRT enc_alg_type and the firmware 4-bit algo_type (matching width + shared vocabulary; no byte-level host→device write traced).
• instr_chaining FSM transition order (enum + member names).
• ISA ALU_OP → SDMA_CCETYPE exact full map (ADD/MAX/MIN align clearly).

LOW

• Concrete TOP_SP CSR register numbers / DMA tail-pointer offsets (absent from all of these artifacts; same gap as the base op).
• Which numeric algo_type a given (ctype, topology, world-size) resolves to — depends on enc_can_post_* thresholds not fully enumerated here.

CARRIED (not re-grounded in this lane)

• The cc_op_info{cc_op,op_type,alg,alg_name} protobuf accessor names (mutable_/set_allocated_cc_op_info) cited by the backing report were not found as symbols/strings in this libnrt.so; the host→firmware cc-op record is real (the enc_alg_type selection is observed) but the specific protobuf accessor naming is unverified here.

See also

• TriggerCollective (0xC8) — the base op this extends.
• Collective-Type + cc_op Enum Reference.
• ALL_REDUCE — the most common ctype lowered through this path.
• Current-Processing-Rank-ID (0xDB) — the rank-loop pseudo-op.
• NCFW algorithms: ring / kangaring · mesh · hierarchical.
• SDMA CCE in-transfer reduce — the per-step reduce.