Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Collective-Type + cc_op Enum Reference

This is the enum-consolidation page for the GPSIMD collective-communication subsystem — the single authoritative cross-reference that every collective op page links back to. It pins, byte-exact, the complete enum set spread across two layers of the stack:

  • the on-instruction ISA enums baked into the TPB instruction word and read verbatim from the shipped Cadence Vision-Q7 (ncore2gp) ISA headers (aws-neuronx-gpsimd-customop-lib 0.21.2.0, the four per-arch copies neuron_{sunda(v2),cayman(v3),mariana(v4),maverick(v5)}_arch_isa/tpb/), and
  • the host-runtime encoder enums carried in libnrt.so (NRT, the host process that lowers the pseudo-ops), re-extracted directly from that binary's DWARF (.debug_info, BuildID 8bb57aba0fb2e0035f1d88e9fc4fb3e7387c102e).

The collective subsystem is unusual in that the same logical operation set is described by four different "kind" vocabularies at four layers (ISA COLLECTIVE_TYPE, host enc_op_type, host/public nrt_op_type, firmware cc_op), and the reduce arithmetic is described by three separate packed enums at three fold scopes (REDUCE_OP, CCE_OP/SDMA_CCETYPE, DGE_COMPUTE_OP). The bulk of this page is reconciling those parallel vocabularies, because conflating their ordinals is the single most common re-implementation error — they agree on operation names but disagree on values.

Every value below is anchored to its defining symbol and file:line (ISA headers) or DWARF DIE offset (libnrt.so). Confidence is tagged HIGH/MED/LOW × OBSERVED/INFERRED/CARRIED.

Source-of-truth contract. The committed op pages (trigger-collective, trigger-collective2-ext, all-reduce, s3d3-collective, sendrecv, …) cite the enum values from here. Where a committed page and the binary disagree, this page carries a CORRECTION callout and the binary wins.


1. The collective opcode space

The enums on this page ride a fixed family of TPB opcodes. Two are real hardware instructions (the cross-lane compute-reduce and the SB2SB iDMA leg); the rest are pseudo-ops that NRT lowers host-side. The pseudo-op class rule is verbatim in common.h: an opcode is a pseudo-op iff its upper three bits are 0b110 (0xC0..0xDF) — "generated by compiler and translated into non-pseudo HW instructions by NRT". None of the 0xC*/0xD* collective ops execute on hardware; NRT decomposes them into SB2SB_COLLECTIVE (0xBF) + DMA + EVENT_SEMAPHORE (0xA0) legs.

opcodemnemonic (NEURON_ISA_TPB_OPCODE_*)classrole / structline
0x7CCROSS_LANE_REDUCE_ARITHHWS4D4_CR — COMPUTE reduce, fp32 arithcommon.h:215
0x7DCROSS_LANE_REDUCE_BITVECHWS4D4_CR — COMPUTE reduce, bitwisecommon.h:216
0xBFSB2SB_COLLECTIVEHWS3D3_COLLECTIVE — the intra-node iDMA leg (s3d3)common.h:262
0xC3PSEUDO_DMABARRIERpseudoDMA-ordering fence (dma-barrier)common.h:265
0xC7PSEUDO_TRIGGER_ALL_REDUCEpseudofixed-form all-reduce (all-reduce)common.h:269
0xC8PSEUDO_TRIGGER_COLLECTIVEpseudogeneric collective; carries ctype (trigger-collective)common.h:270
0xCBPSEUDO_SEND_RECVpseudoNCCL P2P; peer_id+is_send (sendrecv)common.h:273
0xD5PSEUDO_SYNC_BARRIERpseudoall-engine within-core barrier (sync-barrier)common.h:283
0xD8PSEUDO_CORE_BARRIERpseudocross-pcore intra-VNC barrier (core-barrier)common.h:286
0xD9PSEUDO_TRIGGER_COLLECTIVE2pseudov2 (SBUF tensors); 1st of pair (trigger-collective2-ext)common.h:287
0xDAPSEUDO_EXTENSIONpseudov2 _ext; 2nd of paircommon.h:288
0xDBPSEUDO_CUR_PROCESSING_RANK_IDpseudoper-iteration rank-id inject (rank-id)common.h:289
0xDCPSEUDO_GID_LOADpseudoload this rank's GID into a registercommon.h:290

[HIGH/OBSERVED — every ordinal read from the cayman common.h OPCODE enum + instruction_mapping.json.] The RDMA gather variants (rdma-gather-pseudo-ops) reuse 0xC8/0xD9 with their own ctype/alg field values.


2. ISA collective enums (carried in the instruction word)

All bodies are verbatim from the cayman (v3) headers. The underlying type is the 1-byte NEURON_ISA_PACKED enum unless noted. Cross-arch identity is in §10.

2.1 NEURON_ISA_TPB_COLLECTIVE_TYPE (ctype) — common.h:792, 1 B

The "which collective" kind carried by 0xC8 (offset 32) and 0xD9 (offset 14); fixed to ALL_REDUCE for the dedicated 0xC7. 10 enumerators, 0x0..0x9.

valueenumerator (…_COLLECTIVE_TYPE_*)note
0x0INVALID
0x1ALL_REDUCE
0x2REDUCE_SCATTER
0x3ALL_GATHER
0x4ALL_TO_ALL
0x5PERMUTE
0x6PERMUTE_REDUCEheader: "PERMUTE_REDUCE is not found in XLA, leaving it for future extension"
0x7PERMUTE_IMPLICIT
0x8PERMUTE_REDUCE_IMPLICIT
0x9ALL_TO_ALL_V

[HIGH/OBSERVED — full body @common.h:792-803.]

2.2 NEURON_ISA_TPB_COLLECTIVE_DIMENSION (cc_dim) — trigger_collective2.h:27, 1 B

New in v2 (0xD9 offset 15): which SBUF dimension the collective acts along. DIM_0 = 0x0, DIM_1 = 0x1. The host validator strings (libnrt .rodata, OBSERVED) read "%d rank cc_dim 0 sb2sb expectes input is distributed to ranks in free dimension" and "Collective instruction: does not support cc_dim 0/1 for this op %d"DIM_0 selects the FREE dimension, DIM_1 the PARTITION dimension; not all ctypes accept both. [field+values HIGH/OBSERVED; DIM_0=free reading MED from the "free dimension" wording.]

2.3 NEURON_ISA_TPB_LNC_SIZE_FMT (lnc_size_fmt) — common.h:815, 1 B

Header comment: "LncSize setting for sb2sb collective using q7 iDMA". The peer/die grouping for SB2SB (0xBF offset 32) and the participant model for CORE_BARRIER (a VNC = an LNC-grouped pcore set).

valueenumerator (…_LNC_SIZE_FMT_*)grouping (verbatim header)
0LNC1"NC copies to itself. Used for self-test only."
1LNC2"NC0-NC1, NC2-NC3, NC4-NC5, NC6-NC7"
2LNC4"NC0-NC3, NC4-NC7"
3LNC8"NC0-7"

GOTCHA — SB2SB rejects LNC4/LNC8. SB2SB is an intra-node, ≤2-NC operation (LNC1 self / LNC2 pair). The host string "We do not support a SB2SB for multi-node workloads" (OBSERVED) confirms LNC4/LNC8 are rejected on the SB2SB leg even though they are legal enum members. [enum HIGH/OBSERVED @common.h:815-820.]

The committed pages cite the grouping as LNC1=0 / LNC2=1 / LNC4=2 / LNC8=3 — this matches the header exactly.

2.4 The collective2-_ext enums — trigger_collective2_ext.h

These four enums ride the 0xDA PSEUDO_EXTENSION struct. Note the headers double-prefix the enumerator names (e.g. NEURON_ISA_TPB_INSTR_CHAINING_INSTR_CHAINING_NONE).

enum (NEURON_ISA_TPB_*)ext offvaluesline
INSTR_CHAINING1NONE=0x0 INITIAL=0x1 BODY=0x2 FINAL=0x3 ("only 2 bits used")ext.h:27
UNIQUE_TENSORS42UNKNOWN=0x0 NO=0x1 YES=0x2 (replica-group uniqueness hint)ext.h:35
PERMUTE_CHAIN43OFF=0x0 ON=0x1 (protected-permute run)ext.h:43
CROSS_SBOUTPUT_TRANSFER45OFF=0x0 FREE_DIM=0x1 PART_DIM=0x2 (replicate output in free/partition dim)ext.h:51

INSTR_CHAINING ties to the firmware cc_op.trigger_next bit (§7). [all HIGH/OBSERVED.]

2.5 NEURON_ISA_TPB_DMA_CONFIGS — common.h:713, 1 B packed struct (ext off 41)

Not an enum — the DMA QoS priority carrier on the _ext struct:

typedef struct NEURON_ISA_TPB_DMA_CONFIGS {        // common.h:713
    uint8_t priority_class    : 3;   // DMA QoS priority class 0..7
    uint8_t reserved_bitfield : 5;   // must be zero
} NEURON_ISA_PACKED NEURON_ISA_TPB_DMA_CONFIGS;

[HIGH/OBSERVED.]

2.6 NEURON_ISA_TPB_DTYPE (op/leg element dtype) — common.h:722, 4-bit packed

The Rosetta dtype that passes verbatim to the SDMA descriptor (== SDMA_DTYPE == nrt_dtype, §5.3). Values (header is in name order; tabulated here by ordinal):

0x0 INVALID0x1 UINT640x2 INT80x3 UINT80x4 INT160x5 UINT160x6 BFLOAT160x7 FP16
0x8 INT320x9 UINT320xA FP320xB FP32R0xC INT640xD FP8_EXP30xE FP8_EXP40xF FP8_EXP5

Header notes: INVALID "used in RTL for bitvec"; FP32 "RTL will use 0xB for FP22 partial fp32 type" (this 0xB overlap surfaces as the RESERVED alias in SDMA_DTYPE, §5.3). [HIGH/OBSERVED @common.h:722-739.]


3. Host runtime collective enums (libnrt.so DWARF)

All bodies below were re-extracted directly from libnrt.so's DWARF this pass (objdump --dwarf=info; DW_TAG_enumeration_typeDW_TAG_enumeratorDW_AT_const_value). Underlying type is a 4-byte enum. The DIE offset is given for each so the extraction is re-runnable.

3.1 enc_op_type — the cc_op KIND in the host encoder (DIE <0x3bbd0>)

The host re-encoding of ctype plus the host-only decomposition kinds. 13 real op kinds (0..12) with two value-13 sentinels.

valueenumeratorsource
0ENC_ALLGATHERISA ALL_GATHER=0x3
1ENC_ALLREDUCEISA ALL_REDUCE=0x1
2ENC_BROADCASThost-only (decomposition leg)
3ENC_REDUCEhost-only (decomposition leg)
4ENC_REDUCE_SCATTERISA REDUCE_SCATTER=0x2
5ENC_SEND0xCB is_send=1
6ENC_RECV0xCB is_send=0
7ENC_ALLTOALLISA ALL_TO_ALL=0x4
8ENC_PERMUTEISA PERMUTE=0x5
9ENC_PERMUTE_REDUCEISA PERMUTE_REDUCE=0x6
10ENC_PERMUTE_IMPLICITISA PERMUTE_IMPLICIT=0x7
11ENC_PERMUTE_REDUCE_IMPLICITISA PERMUTE_REDUCE_IMPLICIT=0x8
12ENC_ALLTOALL_VISA ALL_TO_ALL_V=0x9
13ENC_OP_INVALID== ENC_OP_N
13ENC_OP_Ncount sentinel (DIE <0x3bc37>, const_value 13)

NOTE. ENC_OP_INVALID and ENC_OP_N are both ordinal 13 (two DW_AT_const_value : 13 enumerators in the same DIE), confirming exactly 13 real op kinds (0..12). This matches sendrecv.md (ENC_SEND=5, ENC_RECV=6) and trigger-collective2-ext.md (ENC_PERMUTE_IMPLICIT=10 … ENC_ALLTOALL_V=12, ENC_OP_INVALID=13) byte-for-byte. [HIGH/OBSERVED.]

3.2 enc_comm_type — topology SCOPE (DIE <0x34a15>)

H_COMM_INTRA_ID = 0, H_COMM_INTER_ID = 1, H_COMM_MAX_ID = 2 (count). INTRA = within a node/die-group; INTER = across nodes/dies. A hierarchical all-reduce runs both an INTRA leg and an INTER leg (§8 legality). [HIGH/OBSERVED.]

3.3 enc_alg_type — the algorithm selector (DIE <0x61eb3>)

11 real algorithms (0..10) + INVALID=11. All 0..10 fit the firmware's 4-bit algo_type nibble (§7).

valueenumeratorroutes to NCFW
0ENC_ALG_RINGring channel tape
1ENC_ALG_HIERhierarchical intra+inter
2ENC_ALG_MESHmesh event tape
3ENC_ALG_KANGARINGring channel tape
4ENC_ALG_SINGLE_CYCLE_RINGring channel tape (all-reduce only)
5ENC_ALG_INTRA_RDHrecursive-doubling/halving
6ENC_ALG_SINGLE_STEP_MESHmesh event tape
7ENC_ALG_INTER_RDHrecursive-doubling/halving
8ENC_ALG_TWO_STEP_POD_MESHmesh event tape
9ENC_ALG_LATENCY_OPT_MESHmesh event tape
10ENC_ALG_BW_OPT_MESHmesh event tape
11ENC_ALG_INVALID

This matches trigger-collective2-ext.md's …ENC_ALG_BW_OPT_MESH=10 ENC_ALG_INVALID=11 exactly. [HIGH/OBSERVED.]

3.4 enc_alg_mesh_type — mesh sub-type / firmware algo_sub_type (DIE <0x125d7c>)

ENC_ALG_FULL_MESH=0, ENC_ALG_GROUPED_MESH=1, ENC_ALG_MESH_TRN2=2, ENC_ALG_MESH_SWITCH=3, ENC_ALG_MESH_INVALID=4. 4 real sub-kinds (0..3), all fit the firmware's 3-bit algo_sub_type. [HIGH/OBSERVED.]

3.5 reduction_type_t — the ring read/write fold PATTERN (typedef → DIE <0x60c99b>)

This is a ring access pattern, not the reduce op. It is the 3rd argument of enc_primitive::recv_reduce_*(enc_half_chunk_index, SDMA_CCETYPE, reduction_type_t, …) and direct_reduce_send_*.

valueenumeratormeaning
0RING_2R1Wring fold: 2 reads / 1 write
1RING_2R2Wring fold: 2 reads / 2 writes
2KANGARING_NR1Wkangaring: N reads / 1 write

NOTE — do not confuse with the reduce op. reduction_type_t selects the descriptor read/write rendezvous shape of the ring step; the actual add/max/min is the separate SDMA_CCETYPE argument that sits beside it in the same signature. This matches the all-reduce.md callout exactly. [HIGH/OBSERVED — DW_AT_const_value 0/1/2 read from DIEs <0x60c99c>/<0x60c9a2>/<0x60c9a8>.]

CORRECTION — argument position. SX-CCL-11 §3.5 described reduction_type_t as the "2nd argument" of recv_reduce_*. The demangled libnrt signature is recv_reduce_send(enc_half_chunk_index, SDMA_CCETYPE, reduction_type_t, bool, bool) (@0x16ad70), so it is the 3rd argument (SDMA_CCETYPE is 2nd). all-reduce.md already states "3rd argument" — that page is correct; the sibling report's "2nd" is off by one. [HIGH/OBSERVED.]

3.6 enc_mesh_event_type — the mesh event-tape event kinds (DIE <0x34a50>)

The mesh-variant "event tape" event kinds. SX-CCL-11 pinned only the first 7; the full 61-enumerator body was recovered this pass. The enum is partitioned by algorithm phase via named sentinels (ENC_*_NUM_EVENT_START / ENC_*_NUM_EVENT_TYPE), which alias the first/last ordinal of each phase block — this is the per-algorithm event-window layout the mesh sema overlay (§7) indexes into.

rangephase block (delimited by sentinel aliases)key events
0..7COMMON (ENC_COMMON_NUM_EVENT_TYPE=8)EVT_SYNC=0 EVT_GLOBAL_HNDSHK=1 EVT_LOCAL_HNDSHK=2 EVT_INTER_GRP_BRDCST=3 EVT_FUNCTION_BARRIER_FIRST_COLL=4 EVT_FUNCTION_BARRIER_LAST_COLL=5 EVT_REDUCE_LOCAL_HNDSHK=6 EVT_INTRA_GRP_BRDCST=7
8..13MESH (ENC_MESH_NUM_EVENT_START=8, ENC_MESH_NUM_EVENT_TYPE=14)EVT_REDUCE_COPY=8 EVT_REDUCE_COPY_2=9 EVT_REDUCE_WRITE=10 EVT_INTER_GRP_BRDCST_2=11 EVT_LOCAL_AND_POD_GRP_BRDCST=12 (…_2=13)
14..22A2A (ENC_A2A_NUM_EVENT_START=14, ENC_A2A_NUM_EVENT_TYPE=23)EVT_LOCAL_HNDSHK_1=14 …_2=15 EVT_GLOBAL_HNDSHK_1=16 EVT_INTER_GRP_BRDCST_1=17 (…_RMV_ROUTING_1=18) EVT_INTRA_GRP_BRDCST_1=19 EVT_2DEV_BRDCST=20 EVT_2DEV_HNDSHK=21 EVT_COPY_FROM_HOST=22
23..52RDH (ENC_RDH_NUM_EVENT_START=23, ENC_RDH_NUM_EVENT_TYPE=53)EVT_RH_STEP_0..9=23..32 (EVT_RDH_LOCAL_HANDSHAKE=32 EVT_RDH_AXES_HANDSHAKE=33), EVT_RD_STEP_0..6=34..40, EVT_RDH_AXES_HANDSHAKE_2=41, EVT_1DEV_RDH/RD/RH_STEP_*=42..46, EVT_2DEV_RD_STEP_0..4=47..51, EVT_RDH_LOCAL_PEER_HANDSHAKE=52

The EVT_FUNCTION_BARRIER_FIRST_COLL / _LAST_COLL events tie to the NCFW function-barrier. [HIGH/OBSERVED — all 61 DW_AT_const_value read this pass; the phase-start aliases (=8, =14, =23, =53) confirm the 4-phase partitioning.]

3.7 encd_neigh — the SB2SB peer selector (DIE <0x3a703>)

The host analogue of the device lnc_size_fmt peer grouping.

valueenumeratormeaning
0ENCD_NEIGH_LOCALself-copy (LNC1 self-test)
1ENCD_NEIGH_NEXTring next neighbor
2ENCD_NEIGH_PREVring prev neighbor
3ENCD_NEIGH_GATEWAY
4ENCD_NEIGH_PEER_RMTVremote-V peer (cross-die)
5ENCD_NEIGH_PEER_RMTV2
6ENCD_NEIGH_PEER_LOCALLNC2-partner peer
7ENCD_NEIGH_NEXT_PEER_RMTV
8ENCD_NEIGH_INVALID== ENCD_NEIGH_NUM (count)

[HIGH/OBSERVED — ordinals read this pass; INVALID/NUM both const_value 8.]


4. The op-kind cross-map (ctypeenc_op_type)

The on-instruction COLLECTIVE_TYPE (ctype) is RE-ENCODED by the host into enc_op_type; the ordinals DIFFER — do not conflate. The host nrt_collectives encoder builds enc_op_type from ctype plus the host-only decomposition kinds.

ISA ctype→ host enc_op_type
ALL_REDUCE = 0x1ENC_ALLREDUCE = 1
REDUCE_SCATTER = 0x2ENC_REDUCE_SCATTER = 4
ALL_GATHER = 0x3ENC_ALLGATHER = 0
ALL_TO_ALL = 0x4ENC_ALLTOALL = 7
PERMUTE = 0x5ENC_PERMUTE = 8
PERMUTE_REDUCE = 0x6ENC_PERMUTE_REDUCE = 9
PERMUTE_IMPLICIT = 0x7ENC_PERMUTE_IMPLICIT = 10
PERMUTE_REDUCE_IMPLICIT = 0x8ENC_PERMUTE_REDUCE_IMPLICIT = 11
ALL_TO_ALL_V = 0x9ENC_ALLTOALL_V = 12
(no ISA ctype)ENC_BROADCAST = 2 — host-only leg
(no ISA ctype)ENC_REDUCE = 3 — host-only leg
PSEUDO_SEND_RECV (0xCB)ENC_SEND = 5 / ENC_RECV = 6 (the is_send bit)
INVALID = 0x0ENC_OP_INVALID = 13

SEND/RECV (enc 5/6) are carried by the dedicated 0xCB (sendrecv: peer_id u64 @16, is_send u8 @24); BROADCAST/REDUCE (enc 2/3) are internal decomposition legs of the larger collectives. [name-correspondence HIGH; the ctype→enc_op_type numeric remap is MED — ordinals OBSERVED on both sides, no byte-level remap table dumped.]


5. The reduce-op layers and the dtype pass-through

5.1 The three reduce-op layers (the core deliverable)

There are three distinct reduce-op enums (plus a fourth DMA-generation compute enum), at three different fold scopes. They are separate packed types that do NOT share an encoding — they agree only on operation names; the per-element add/max/min math is the same ground-truth primitive everywhere.

layer / scopeenum (full body, defining symbol)where used
(1) COMPUTE intra-vector cross-laneNEURON_ISA_TPB_REDUCE_OP (s4d4_cr.h:62, 1 B): ADD=0 AVERAGE=1 MAX=2 BITWISE_OR=3 BITWISE_AND=4 BITWISE_XOR=5CrossLaneReduce (0x7C/0x7D): reduce the SIMD lanes of one vector reg → scalar
(2a) DMA collective cross-buffer/cross-core (ISA)NEURON_ISA_TPB_CCE_OP (common.h:1002, 1 B): ADD=0x00 MAX=0x02 MIN=0x03 (header: "Same encoding as SDMA CCE op encoding"; // Multiply = 0x01 is a reserved comment slot, not an enumerator)the SDMA CCE reduce-on-transfer descriptor baked into the ring/mesh step
(2b) DMA collective (host carrier, superset)SDMA_CCETYPE (libnrt DWARF <0x337bd>, 4 B): ADD=0 FMA=1 MAX=2 MIN=3 EXT=4 GCE=5the argument type of every recv_reduce_* / direct_reduce_* ring/mesh step primitive
(+) DGE copy-compute DMA-generation engineNEURON_ISA_TPB_DGE_COMPUTE_OP (common.h:837, 1 B): NONE=0x00 ADD=0x01 MULTIPLY=0x02 MAX=0x03 MIN=0x04 (header: B=A; B+=A; B*=A; B=MAX(A,B); B=MIN(A,B))DGE compute-on-copy (reduction-add DMA mode) — distinct from the CCE collective reduce

COMPUTE semantics (verbatim header): ADD/AVERAGE/MAX are fp32 math; OR/AND/XOR are raw u8/u16/u32 with no fp32 conversion; AVERAGE multiplies each element by the scale f32 field then adds. The compute reduce also carries NEURON_ISA_TPB_REDUCE_AXIS (s4d4_cr.h:71): ALL=0 (reduce all dims → single value), PARTITIONS=1 (reduce only across SBUF/PSUM partitions).

The host-side public reduce op is nrt_op_type (libnrt DWARF <0xd8fa>): NRT_OP_ADD=0 NRT_OP_FMA=1 NRT_OP_MAX=2 NRT_OP_MIN=3 NRT_OP_INVALID=15 — taken by nrt_cc_prepare and translated to SDMA_CCETYPE value-preservingly for {ADD,FMA,MAX,MIN} (translate_op_type is identity; only the invalid op hits the __assert_fail cold tail @0x7f5b0).

5.2 Membership logic — why three+one enums, where each is used

The four enums sit at fold scopes that compose innermost → outermost, plus one side path. Reproduced as annotated pseudocode naming the real symbols:

// Innermost: fold the SIMD lanes of ONE vector register to a scalar.
//   opcode 0x7C/0x7D, struct S4D4_CR, on the POOL engine.
scalar = CrossLaneReduce(vector, NEURON_ISA_TPB_REDUCE_OP op,   // ADD/AVG/MAX/OR/AND/XOR
                                 NEURON_ISA_TPB_REDUCE_AXIS ax); // ALL | PARTITIONS

// Outermost (collective): fold across BUFFERS / CORES on the SDMA engine.
//   the ring/mesh/hier reduce-scatter & all-reduce legs.
//   The host carries SDMA_CCETYPE; only {ADD,MAX,MIN} reach the HW descriptor,
//   whose field is the ISA CCE_OP {0,2,3} byte-subset.
status = enc_primitive::recv_reduce_send(enc_half_chunk_index chunk,
                                         SDMA_CCETYPE   cce,   // ADD0/FMA1/MAX2/MIN3/EXT4/GCE5
                                         reduction_type_t pat, // §3.5 ring shape, NOT the op
                                         bool, bool);
//   CCE descriptor field <- (uint8_t) cce, but only ADD0/MAX2/MIN3 (== CCE_OP) ever
//   reach hardware; FMA1/EXT4/GCE5 are host-runtime-only carrier modes.

// Side path: DMA-generation engine compute-on-copy (reduction-add DMA).
//   B = op(A, B) during the copy; NONE == pure memcpy. DISTINCT from CCE_OP.
dge_copy(dst /*B*/, src /*A*/, NEURON_ISA_TPB_DGE_COMPUTE_OP op); // NONE0/ADD1/MUL2/MAX3/MIN4

The collective op KIND (enc_op_type, §3.1) is a third, orthogonal axis — it is the operation (all-reduce vs reduce-scatter) that parameterizes the legs with one of these reduce ops; it is not itself a reduce op.

GOTCHA — the alignment islands. REDUCE_OP and CCE_OP/SDMA_CCETYPE agree on ADD=0 and MAX=2 but disagree on slot 1 (REDUCE_OP AVERAGE=1 vs SDMA FMA=1) and slot 3 (REDUCE_OP BITWISE_OR=3 vs SDMA MIN=3). They are not interchangeable. DGE_COMPUTE_OP is NONE-based, so its ADD=1/MAX=3/MIN=4 align with nothing. A reimplementation that copies a REDUCE_OP value into an SDMA CCE descriptor will silently compute MIN where it meant BITWISE_OR. [reconciliation HIGH — every body OBSERVED.]

The ISA ALU_OP (common.h:939) is the full ALU enum; the collective in-DMA reduce path uses only its ADD=0x04 / MAX=0x08 / MIN=0x09 members as the op field that trigger-collective then maps to CCE_OP/DGE_COMPUTE_OP.

Why two DMA reduce vocabularies (CCE_OP vs SDMA_CCETYPE). CCE_OP is the hardware reduce-primitive encoding baked into the SDMA descriptor (3 documented ops). SDMA_CCETYPE is the host runtime's superset carrier that also names FMA (the scale-operand reduce), EXT and GCE (extended/generic CCE forms). They share the {0,2,3} encoding, so the host value drops straight into the HW descriptor. The byte-level proof of the subset relationship is the kbin_cce_op_to_sdma_cce_op remap (CSWTCH.21 = {1, 0, 3, 2} @0x9d6e00), which renumbers a kbin op into the SDMA op space. [HIGH/OBSERVED.]

5.3 The dtype pass-through chain

The element dtype passes through with no re-encoding across ISA → SDMA → public:

ISA DTYPE (common.h:722)SDMA_DTYPE (DWARF <0x33738>)nrt_dtype (DWARF <0xd937>)
BFLOAT16=0x6 FP16=0x7 FP32=0xA FP32R=0xB INT8=0x2 INT64=0xC FP8_EXP3=0xD …byte-identical (FP32R=11, INT64=12, FP8_E3=13 …)byte-identical (FLOAT16=7 BFLOAT16=6 FLOAT32=10 FP32R=11 …)

NOTE — the 0xB/11 aliasing. SDMA_DTYPE carries both SDMA_DTYPE_FP32R=11 and SDMA_DTYPE_RESERVED=11 (two const_value 11 enumerators), surfacing the FP32 RTL note "0xB reserved for FP22 partial fp32". nrt_dtype likewise aliases UNKNOWN=0 with INVALID=0. [HIGH/OBSERVED.]

5.4 cce_dtypes — the reducible-dtype seed table (.rodata @0x9b9f40)

The dtypes the SDMA CCE reduce path seeds descriptors for. 6 entries of {u64 dtype, u64 group}, dumped byte-exact:

dtypenamegroup (CCE element-packing factor)
0x06BFLOAT162
0x07FP162
0x0BFP32R4
0x0DFP8_E31
0x0EFP8_E41
0x0FFP8_E51

Plain FP32 (0x0A) and the integer dtypes are absent — the CCE all-reduce is a {BF16, FP16, FP32R, FP8} float reduce. [table HIGH/OBSERVED; "exhaustive reducible set" MED — it is the descriptor-seed set.]


6. The WAIT/UPDATE sync-mode families

Every TPB instruction (including the barriers and collectives) can hold an inline EVENTS block with one wait and one update primitive. The two enums split into an EVENT family (1-bit flag) and a SEMAPHORE family (32-bit counter). SYNC_BARRIER (0xD5) uses the EVENT family; CORE_BARRIER (0xD8) uses the SEMAPHORE family. The encoding regularity: +0x10 = COMPLETE variant (fires after the instruction fully completes, vs READ = after reads-done), +0x80 = REG-operand variant.

6.1 cayman / mariana / sunda (NC-v3/v4/v2) — the common form

WAIT_MODE (common.h:313, 1 B):

valueenumeratorfamily
0x0NONE
0x1..0x5WAIT_FOR_SEM_{EQ,LT,LE,GT,GE}_IMMSEM (imm) — GE_IMM=0x5 is the counted-barrier wait
0x6WAIT_FOR_EVT_SETEVT — the event-barrier wait (SYNC_BARRIER)
0x7WAIT_FOR_EVT_SET_THEN_CLEAREVT
0xEWAIT_FOR_EVT_CLEAREVT
0xFWAIT_FOR_EVT_CLEAR_THEN_SETEVT
0x81..0x85WAIT_FOR_SEM_{EQ,LT,LE,GT,GE}_REGSEM (reg)
0xFFINVALID

UPDATE_MODE (common.h:347, 1 B) — the arrive side:

familyenumerators (value)
EVENTEVT_SET_READ=0x1 EVT_CLR_READ=0x2 EVT_SET_COMPLETE=0x11 EVT_CLR_COMPLETE=0x12
SEMAPHORE (READ)SEM_INC_READ=0x3 SEM_DEC_READ=0x4 SEM_ADD_IMM_READ=0x5 SEM_ADD_REG_READ=0x85 SEM_SUB_IMM_READ=0x7 SEM_SUB_REG_READ=0x87 SEM_WR_IMM_READ=0x9 SEM_WR_REG_READ=0x89
SEMAPHORE (COMPLETE)SEM_INC_COMPLETE=0x13 SEM_DEC_COMPLETE=0x14 SEM_ADD_IMM_COMPLETE=0x15 SEM_ADD_REG_COMPLETE=0x95 SEM_SUB_IMM_COMPLETE=0x17 SEM_SUB_REG_COMPLETE=0x97 SEM_WR_IMM_COMPLETE=0x19 SEM_WR_REG_COMPLETE=0x99
NONE=0x0 INVALID=0xFF

[HIGH/OBSERVED @common.h:313-330 / 347-370.]

6.2 maverick (NC-v5) — DIVERGES

CORRECTION / GENERATIONAL DELTA. On NC-v5 (maverick), WAIT_MODE and UPDATE_MODE are a different enum from the other three gens. The barrier op pages diffed the barrier structs (byte-identical across all four arches) but not the mode enums, so this delta was not previously flagged. Confirmed this pass by a 4-arch md5 diff: cayman == mariana == sunda; maverick differs.

WAIT_MODE (maverick, common.h:331):

  • DROPS the entire EVENT familyWAIT_FOR_EVT_{SET, SET_THEN_CLEAR, CLEAR, CLEAR_THEN_SET} (0x6/0x7/0xE/0xF) are gone.
  • KEEPS SEM_*_IMM (0x1..0x5) and SEM_*_REG (0x81..0x85).
  • ADDS WAIT_FOR_SEM_{EQ,LT,LE,GT,GE}_REG_OFFSET = 0x91..0x95 (register-offset operand wait form).
  • ADDS UNORDERED = 0xFE (verbatim: "the instruction will not wait on any wait condition … faster than 'None' as it avoids hardware synchronization through the semaphore block").
  • INVALID = 0xFF kept.

UPDATE_MODE (maverick, common.h:367): DROPS the EVENT family (EVT_SET/CLR_READ 0x1/0x2, EVT_*_COMPLETE 0x11/0x12) — only the SEMAPHORE INC/DEC/ADD/SUB/WR (READ + COMPLETE) forms remain.

Plus a new maverick-only enum NEURON_ISA_TPB_EVT_SEM_NONBLOCKING_CMD (common.h:388): NOP=0x40 SEM_INC=0x43 SEM_DEC=0x44 SEM_ADD=0x45 SEM_SUB=0x47 SEM_WRITE=0x49 READ_SEMAPHORE=0x60 READ_EVENT=0x61 — a separate non-blocking event/semaphore command space.

Re-implementation impact. A SYNC_BARRIER (event-based on v2/v3/v4) cannot be lowered the same way on v5: WAIT_FOR_EVT_SET does not exist there, so the runtime must express the same handshake through the SEMAPHORE forms or EVT_SEM_NONBLOCKING_CMD. [enum divergence HIGH/OBSERVED; the "v5 folded events into the semaphore/nonblocking space" reading is INFERRED MED.]


7. The cc_op firmware command word

The NCFW spad-control collective-op entry is the firmware's per-step program word: {1-byte header @+0} + {cc_op command word @+1}. NRT asserts spad_ctrl->header.cc_op == 1 (header bit0 = "this entry is an active collective op"). The command-word layout (see the NCFW config schema ring-protocol-config-command and spad-ccop-tsync):

// cc_op command word (entry + 1)
struct cc_op {
    // byte 0
    uint8_t algo_type     : 4;  // = low nibble of enc_alg_type (§3.3): RING0..BW_OPT_MESH10
    uint8_t algo_sub_type : 3;  // = enc_alg_mesh_type (§3.4): FULL0/GROUPED1/TRN2_2/SWITCH3
    uint8_t trigger_next  : 1;  // ring/chain step flag (ties to INSTR_CHAINING, §2.4)
    // byte 1
    uint8_t reporter            : 1;  // this channel is the leader/reporter
    uint8_t ring_wait_complete  : 1;
    uint8_t ring_send_complete  : 1;
    // +0x3 union, selected by algo_type:
    union {
        uint32_t channel_list;       // RING family: bitmask of the 32 ring channels
        struct {                     // MESH family:
            uint16_t sema_shift_offset;  // +0x3
            uint16_t sema_mask;          // +0x5  (indexes the enc_mesh_event_type window, §3.6)
        };
    };
};

The single algo_type switch binds a spad entry to an algorithm: RING/KANGARING/SINGLE_CYCLE_RING → NCFW ring channel tape; MESH/*_MESH → NCFW mesh event tape (with the sema_shift_offset/sema_mask overlay); HIER/*_RDH → NCFW hierarchical intra+inter legs.

NOTE — the reduce op is NOT in the cc_op word. The NCFW disasm reads no SDMA_CCETYPE-width field in the command word. The reduce arithmetic rides the SDMA CCE descriptor (SDMA_CCETYPE, §5.1) plus the host cc_op_info{op_type}. So spad.algo_type = "how to move/route"; SDMA_CCETYPE = "what to compute". [OBSERVED-absence HIGH; the algo_typeenc_alg_typenibble identity is MED — width(4b) + shared vocabulary +cc_op_info{alg} carrier; the host→device write crosses into the Xtensa-LX NCFW core, for which no disassembler ships.]


8. The (op × scope × alg) legality table

The host __select_algorithms assigns an enc_alg_type per LEG; each leg's legal set is one of the .rodata assertion strings below (dumped byte-exact this pass). A hierarchical collective carries FIVE per-leg choices — the runtime log string (@0x7ed4b0, OBSERVED) is: "Hier algorithm selections - alg_intra_allg %d alg_intra_redsct %d alg_inter_allr %d alg_inter_allg %d alg_inter_redsct %d".

LEG (op × scope)legal enc_alg_type setassertion string @.rodata
INTRA all-gatherRING || KANGARING0x7f4b48
INTRA reduce-scatterRING || KANGARING0x7f4db8
INTER all-gatherRING || INTER_RDH ( || MESH || inter_metaring )0x7f4a48 / 0x7f4a10 / 0x7f4bd0
INTER all-reduceRING || INTER_RDH || SINGLE_CYCLE_RING ( || MESH || inter_metaring )0x7f4f28 / 0x7f4ef0 / 0x7f5010
INTER reduce-scatterRING || INTER_RDH ( || MESH || inter_metaring )0x7f51a0 / 0x7f5168

As a matrix (✓ = legal):

leg ↓ \ alg →RINGKANGARINGINTER_RDHSINGLE_CYCLE_RINGMESH (/metaring)
intra all-gather
intra reduce-scatter
inter all-gather
inter all-reduce
inter reduce-scatter

The membership logic, as the runtime expresses it (each row is a literal C assertion, reproduced naming the real symbol):

// per-leg legality asserts (the .rodata strings above; __select_algorithms output)
assert(alg_intra_allg   == ENC_ALG_RING || alg_intra_allg   == ENC_ALG_KANGARING);
assert(alg_intra_redsct == ENC_ALG_RING || alg_intra_redsct == ENC_ALG_KANGARING);
assert(alg_inter_allg   == ENC_ALG_RING || alg_inter_allg   == ENC_ALG_INTER_RDH
       || (alg_inter_allg == ENC_ALG_MESH) || inter_metaring);
assert(alg_inter_allr   == ENC_ALG_RING || alg_inter_allr   == ENC_ALG_INTER_RDH
       || alg_inter_allr == ENC_ALG_SINGLE_CYCLE_RING
       || (alg_inter_allr == ENC_ALG_MESH) || inter_metaring);
assert(alg_inter_redsct == ENC_ALG_RING || alg_inter_redsct == ENC_ALG_INTER_RDH
       || (alg_inter_redsct == ENC_ALG_MESH) || inter_metaring);

Key gating facts (OBSERVED):

  • SINGLE_CYCLE_RING is all-reduce-only. enc_can_post_single_cycle_ring (@0xfaa80) guards op_type == 1 (ENC_ALLREDUCE): the 4th arg (enc_op_typeebp) is tested cmp $0x1,%ebp ; jne reject (@0xfaadd). It appears in the inter all-reduce row only. Strings "Single Cycle ALLR", "NEURON_RT_DBG_SINGLE_CYCLE_RING_ALLR_CC" confirm.
  • KANGARING is intra-only (intra all-gather + intra reduce-scatter); never inter.
  • MESH (and inter_metaring) is an inter alternative for all three reducing legs.
  • A hierarchical all-reduce decomposes as intra reduce-scatter → inter all-reduce → intra all-gather, so it composes the rows above.
  • SB2SB op gating: "We do not support SB2SB for op type %d" — only a subset of op types use the on-chip SB2SB (0xBF) leg.

[legality strings HIGH/OBSERVED — re-dumped this pass; the matrix is the direct transcription. Which numeric alg a given (world-size, dtype, topology) resolves to depends on the enc_can_post_* thresholds — MED, not enumerated.]


9. Ring / mesh sub-selectors (summary)

  • enc_alg_mesh_type (firmware algo_sub_type, §3.4): FULL0/GROUPED1/TRN2_2/SWITCH3/INVALID4. Picked by algo_sub_type bits[4:6] of the cc_op word (§7).
  • reduction_type_t (ring/kangaring read/write fold pattern, §3.5): RING_2R1W0/RING_2R2W1/KANGARING_NR1W2.
  • enc_mesh_event_type (mesh event-tape kinds, §3.6): the 61-event tape, phase-partitioned COMMON/MESH/A2A/RDH.
  • encd_neigh (SB2SB peer selector, §3.7): LOCAL0/NEXT1/PREV2/GATEWAY3/PEER_RMTV4/PEER_RMTV2_5/PEER_LOCAL6/NEXT_PEER_RMTV7/INVALID8.

10. Cross-arch stability

4-arch md5 diff this pass (sunda v2 / cayman v3 / mariana v4 / maverick v5):

statusenums
STABLE (byte-identical all 4)COLLECTIVE_TYPE, COLLECTIVE_DIMENSION, LNC_SIZE_FMT, CCE_OP, DGE_COMPUTE_OP, REDUCE_OP, REDUCE_AXIS, DTYPE, ALU_OP, the four collective2-_ext enums, DMA_CONFIGS, the collective-family OPCODE values
DIVERGENTWAIT_MODE / UPDATE_MODE: cayman == mariana == sunda; maverick (v5) differs — drops the EVENT family, adds _REG_OFFSET (0x91..0x95) + UNORDERED (0xFE) + the maverick-only EVT_SEM_NONBLOCKING_CMD enum (§6.2)

The host enc_*/SDMA_*/nrt_* enums live in a single libnrt.so that serves all gens (one binary, no per-arch divergence). CAYMAN = NC-v3. See the master capability matrix for the full cross-gen support view.

VERIFICATION NOTE. v2–v4 enum bodies are byte-grounded across all four shipped header copies. The maverick (v5) header is present and its WAIT_MODE/UPDATE_MODE/EVT_SEM_NONBLOCKING_CMD divergence is OBSERVED; where a v5-only interior is not separately corroborated by the binary it is flagged INFERRED inline.


Cross-references