The aws_hal_q7_* HAL

This page reconstructs the host-side, per-architecture register programmer for the Vision-Q7 "Pool" engine — the GPSIMD/custom-op DSP — as it ships inside libnrt.so. The aws_hal_q7_* family is the slice of the vendored KaenaHal (version 2.31.0.0) that lays out the BAR0 register map for the Q7 cluster, validates and installs the Q7 IRAM/DRAM ucode into every core over programmed-IO, programs the per-engine NX/Q7 local registers, and drives the full reset → start bring-up (run-stall release). It is the host counterpart of the device-side bring-up documented in Boot / Reset Sequence: the HAL is what BAR0-writes the image and clears the run-stall that the device reset vector waits behind.

Everything here is byte-pinned to a shipped artifact this session: the host x86-64 ELF libnrt.so.2.31.24.0 (122,956,336 B, BuildID 8bb57aba…387c102e, 2.31.24.0 git 0b044f4ce, with debug_info, not stripped), disassembled with stock objdump at the addresses recovered from the IDA function sidecar. For .text/.rodata the rule VMA == file offset holds (confirmed via readelf -SW: .text VMA 0x3dbc0 / off 0x3dbc0; .rodata VMA 0x7cf000 / off 0x7cf000), so every address below is both a VA and a file offset. The KaenaHal source-path __FILE__ strings (…/KaenaHal-2.31.0.0/…/src/{common,sunda, cayman,mariana}/…) are read out of .rodata and are the per-arch navigation key — the Q7 HAL lives in common/q7/aws_hal_q7.c, the Pool bring-up in common/tpb/aws_hal_stpb_pooling.c, and the per-arch register offsets in {sunda,cayman,mariana}/arch/aws_hal_arch_offsets_*.c. Where prompt-level folklore or a sibling note disagrees with the binary, the binary wins — and an in-place CORRECTION callout says so.

Confidence tags follow the Confidence & Walls Model: OBSERVED = a byte/string/instruction read from a shipped artifact this session; INFERRED = reasoned over OBSERVED facts; CARRIED = consolidated from a cited cross-page anchor; crossed with HIGH/MED/LOW. Callouts: QUIRK (counter-intuitive but real), GOTCHA (a reimplementation trap), CORRECTION (overturns a naive reading), NOTE (orientation).

WALL — v5 (Maverick / NC-v5) is not in this binary. This libnrt.so registers Q7 HAL implementations for exactly three architectures: Sunda (NC-v2), Cayman (NC-v3), Mariana (NC-v4). Maverick (NC-v5) is header-OBSERVED only elsewhere in the corpus; any v5 claim on this page is INFERRED by extrapolation from the Cayman/Mariana pair, and the arch_id 36 binding for v5 is itself INFERRED. The byte-grounded walls below are Sunda/Cayman/Mariana.

0. One-screen orientation — what this HAL is

aws_hal_q7_* is ~16 Q7-named symbols plus their supporting per-arch NX/Q7 local-register writers, all inside KaenaHal. It does four things:

1. Parameter lookup — aws_hal_get_q7_params{,_<arch>} returns the Q7 engine geometry tuple {base_offset, iram_size, dram_size, iram_rsvd, dram_rsvd, num_q7}, keyed on an al_hal_q7_owner value (POOLING vs COMPUTE_CLUSTER). §3
2. Ucode install — aws_hal_q7_ucode_eng_init{,_<arch>} validates the {iram,dram} image sizes against the params, then for each of num_q7 cores streams the DRAM then IRAM image into the engine over BAR0 via write_padded → al_mem_write_buf → axi_write → ndl_bar_write. §4
3. Swap / DKL — aws_hal_q7_swap_table / _swap_table_get_register_offsets / _swap_file_io_table: the runtime dynamic-kernel-load hook. REAL only on Sunda (pokes per-core local regs); NO-OP STUBS on Cayman + Mariana (their DKL is firmware-resident). §5
4. Engine control — the surrounding aws_hal_stpb_<arch>_pooling_init: the full Pool bring-up (regs_init → ucode_seq_init → q7_ucode_eng_init → [hw_decode] → dma_init → release_seq_run_stall → release_eng_run_stall), i.e. reset/start/halt control plus the SEQ/Q7 local-register programming. §6, §7

The single dispatch substrate is a global struct kaena_khal @0xcaeb80 whose per-arch function pointers are filled at init. al_hal_tpb_get_arch_type() (reads hal_target @0xcaeb60, asserts the value is in {1..4}) is checked first, then the generic trampoline tail-jumps through the populated slot. The writes never leave the BAR0 MMIO aperture (axi_write / ndl_bar_write) or the per-CSR aperture (csr_write). No DMA installs the ucode — it is a programmed-IO copy.

1. The Q7 HAL symbol set

Recovered with the IDA function sidecar; each address below is independently re-disassembled in the sections that follow. [HIGH/OBSERVED]

Generic arch-dispatching trampolines (.text):

Per-arch implementations:

Per-arch Pool bring-up callers (aws_hal_stpb): 0x46e8d0 stpb_sunda_pooling_init, 0x4737b0 stpb_cayman_pooling_init, 0x467d30 stpb_mariana_pooling_init, 0x458350 aws_hal_stpb_init (top), 0x45c170 aws_hal_stpb_pooling_init (dispatch via khal+0x368).

Provenance .rodata strings (binary-derived, citeable): common/q7/aws_hal_q7.c, common/tpb/aws_hal_stpb_pooling.c, common/arch/aws_hal_arch_offsets.c, {sunda,cayman,mariana}/arch/aws_hal_arch_offsets_*.c.

2. The generic dispatch model (kaena_khal slot table)

Every generic aws_hal_q7_* function follows one pattern. Disassembling aws_hal_get_q7_params @0x44bfb0: [HIGH/OBSERVED]

// aws_hal_get_q7_params @0x44bfb0 — the canonical trampoline shape.
// Args are forwarded UNCHANGED; the per-arch impl has the same signature.
int aws_hal_get_q7_params(int owner, u64 *base, u64 *iram_sz, u64 *dram_sz,
                          u64 *iram_rsvd, u64 *dram_rsvd, u32 *num_q7) {
    if (al_hal_tpb_get_arch_type() == 0 /* AL_HAL_TPB_ARCH_TYPE_INVALID */)
        __assert_fail("al_hal_tpb_get_arch_type() != AL_HAL_TPB_ARCH_TYPE_INVALID");
    void *fn = *(void **)(&kaena_khal + 0xb8);     // khal+0xb8 = get_q7_params slot
    if (fn == NULL)
        __assert_fail("kaena_khal.khal_arch.get_q7_params");
    return ((q7_params_fn)fn)(owner, base, iram_sz, dram_sz,
                              iram_rsvd, dram_rsvd, num_q7);   // jmp *%rax (tail call)
}

The disassembly shows it literally: call al_hal_tpb_get_arch_type → test %eax,%eax; je <assert> → lea 0x862b95(%rip),%rax # caeb80 <kaena_khal> → mov 0xb8(%rax),%rax → test %rax,%rax; je <assert> → restore the saved args (%r14→0x50(%rsp), %r15→%r8, %r13→%rcx, …) → jmp *%rax. The two __assert_fail arms carry the exact __PRETTY_FUNCTION__ and the assert text.

Confirmed kaena_khal slot offsets (the per-Q7 dispatch table fields): [HIGH/OBSERVED; 0x3a0 MED]

al_hal_tpb_get_arch_type @0x44bca0 reads hal_target @0xcaeb60; valid {1..4}; 0 asserts. The enum (carried from the arch-type model): 0=INVALID, 2=Sunda (NC-v2), 3=Cayman (NC-v3), 4=Mariana (NC-v4). Value 1 passes the range check but no Q7 impl is registered for it — a legacy/Inf1 slot. [HIGH/OBSERVED; enum binding CARRIED]

2.1 The two physical write primitives

The HAL ultimately reaches BAR0 through exactly two primitives. [HIGH/OBSERVED]

(a) Bulk image write — al_mem_write_buf @0x265990. Asserts dst and len are 4-byte aligned, then len >>= 2 to a word count and calls axi_write(dst, src, words):

// al_mem_write_buf @0x265990
//   26599a: test $0x3,%al  ; jne <assert>     -- dst addr 4B-aligned
//   26599e: test $0x3,%dl  ; jne <assert>     -- len     4B-aligned
//   2659a3: shr  $0x2,%edx                    -- len >>= 2  => word count
//   2659a6: call axi_write                    -- 315850
//   2659ab: test %eax,%eax ; jne <assert>     -- axi_write must succeed
int al_mem_write_buf(u64 dst, const void *src, u32 len) {
    assert((dst & 3) == 0 && (len & 3) == 0);
    return axi_write(dst, src, len >> 2);
}

axi_write @0x315850 resolves the BAR-region table via tdrv_arch_ops+0x1b0 (a callback returning an array of 0x30-byte records {valid@0, base@8, limit@0x10, bar-handle@0x28}), finds the record whose [base, limit-4] spans dst, and calls ndl_bar_write(handle, 0, src, words, &ctx) — the BAR0 MMIO write down to the ndl driver portal.

(b) Single-CSR write — al_reg_write32 @0x265c50. csr_write @0x315820 → csr_write_array(addr,&val,1) → same BAR-region resolve → ndl_bar_write of one 32-bit word. al_reg_read32 @0x2658a0 is the read peer. Every per-register HAL helper (run-stall, dma_ctrl, engine_base, …) uses (b).

NOTE. Image payload goes through (a), word-blasted; control registers through (b). Both land in BAR0 via ndl_bar_write. No DMA is used to install the ucode — it is a programmed-IO copy. [HIGH/OBSERVED]

3. aws_hal_get_q7_params — the Q7 engine parameter map

Recovered signature (x86-64 SysV ABI): [HIGH/OBSERVED]

// owner = al_hal_q7_owner: 2 = AL_HAL_POOLING_Q7 (the GPSIMD/custom-op engine)
//                          5 = AL_HAL_COMPUTE_CLUSTER_Q7 (collectives CC engine)
void aws_hal_get_q7_params(int owner /*edi*/, u64 *base /*rsi*/, u64 *iram_size /*rdx*/,
                           u64 *dram_size /*rcx*/, u64 *iram_rsvd /*r8*/, u64 *dram_rsvd /*r9*/,
                           u32 *num_q7 /*[rsp+0x10]*/);

3.1 Sunda — pooling only

aws_hal_get_q7_params_sunda @0x478f90 accepts owner==2 only; any other value falls through to __assert_fail("aws_hal_get_q7_params_sunda") (note: no CC branch). The constants are stored as immediate movqs, read directly from the disasm: [HIGH/OBSERVED]

// aws_hal_get_q7_params_sunda @0x478f90
//   478f94: cmp $0x2,%edi ; jne <assert>          -- POOLING only
//   478f9e: movq $0x2980000,(%rsi)   base_offset
//   478fa5: movq $0x10000, (%rdx)    iram_size  = 64 KiB
//   478fac: movq $0x10000, (%rcx)    dram_size  = 64 KiB
//   478fb3: movq $0x0,     (%r8)     iram_rsvd  = 0
//   478fba: movq $0x0,     (%r9)     dram_rsvd  = 0
//   478fc1: movl $0x8,     (%rax)    num_q7     = 8   (%rax = num_q7 ptr from [rsp+0x10])

→ Sunda Q7 = 8 cores, 64 KiB IRAM + 64 KiB DRAM each, no reserved tail.

3.2 Cayman / Mariana — dual owner, byte-identical to each other

aws_hal_get_q7_params_cayman @0x47ae70 and aws_hal_get_q7_params_mariana @0x477610 return the same geometry; only num_q7 depends on the owner. Disasm of the Cayman variant: [HIGH/OBSERVED]

// aws_hal_get_q7_params_cayman @0x47ae70
//   47ae74: cmp $0x2,%edi ; je 0x47aeb1 (POOLING => num_q7=8)
//   47ae7e: cmp $0x5,%edi ; jne <assert>     (else __assert_fail aws_hal_get_q7_params_cayman)
//   ---- owner==5 path (CC), num_q7=4 ----   ---- owner==2 path (POOL), num_q7=8 ----
//   movq $0x3100000,(%rsi)   base_offset
//   movq $0x20000, (%rdx)    iram_size  = 128 KiB
//   movq $0x40000, (%rcx)    dram_size  = 256 KiB
//   movq $0x60000, (%r8)     iram_rsvd  = 0x60000
//   movq $0x40000, (%r9)     dram_rsvd  = 0x40000
//   movl $0x4 / $0x8, (%rax) num_q7     = 4 (CC) / 8 (POOLING)

→ Cayman/Mariana Q7 = 128 KiB IRAM + 256 KiB DRAM per core (double Sunda's IRAM, 4× DRAM), with a reserved IRAM tail 0x60000 and reserved DRAM tail 0x40000 — the DKL/ext-ISA staging window (see §5).

CROSS-CHECK. The 256 KiB per-core Q7 DRAM matches POOL_Q7_CORE{n}_DRAM (256 KiB @ SoC 0x2803180000 + n*0x100000) on the device-side SoC map. base_offset is the BAR0-relative offset of the Q7 IRAM aperture; the nrtucode layer adds aws_get_tpb_addr(pcore) to turn it into the absolute SoC address. [HIGH structure; base→SoC composition CARRIED]

3.3 Per-arch geometry summary

4. aws_hal_q7_ucode_eng_init — the ucode install

Recovered signature: [HIGH/OBSERVED]

// imgs points at TWO nrt_ucode_img records:
//   imgs[0] = IRAM { bin@+0x00, size@+0x08 }
//   imgs[1] = DRAM { bin@+0x10, size@+0x18 }
int aws_hal_q7_ucode_eng_init(void *tpb_handle /*rdi*/, u64 image_base /*rsi*/,
                              nrt_ucode_pair *imgs /*rdx*/, void *swap_cb /*rcx*/,
                              void *swap_ctx /*r8*/, int owner_idx /*r9d*/);

4a. Owner mapping — the v2-vs-v3/v4 split

[HIGH/OBSERVED]

• Sunda (@0x46b9a0): r9d must be 0. The disasm opens test %r9d,%r9d; jne <reject>; the reject arm logs "Invalid q7 owner %u for v2 (only pooling supported)" and returns -1. Internally it calls aws_hal_get_q7_params_sunda(owner=2).
• Cayman (@0x4714c0) / Mariana (@0x464b00): r9d==0 → get_q7_params(owner=2, POOLING); r9d==1 → get_q7_params(owner=5, COMPUTE_CLUSTER); else → log "Invalid q7 owner %u" and return -1.

→ The HAL's owner_idx ∈ {0,1} maps to the al_hal_q7_owner ∈ {2,5}. Sunda is pooling-only; Cayman/Mariana are dual-owner.

4b. Size validation — the host-side admission gate

After resolving params, the loader compares each image size against its aperture. Both error strings are present verbatim in .rodata. [HIGH/OBSERVED]

if (imgs[0].size /* IRAM, +0x08 */ > iram_size)
    { al_hal_log("IRAM ucode too big! doesn't fit, size=%lu", imgs[0].size); return -1; }
if (imgs[1].size /* DRAM, +0x18 */ > dram_size)
    { al_hal_log("DRAM ucode too big! doesn't fit, size=%lu", imgs[1].size); return -1; }

This is the host-side per-image bounds gate complementing the device prelink segment-bounds check and the device NUM_POOL_CORES assert (security posture; CARRIED).

4c. The per-core install loop

[HIGH/OBSERVED]

// Per-core IRAM/DRAM destination addresses are computed from base + iram_rsvd/dram_rsvd and
// image_base (the iram_total/dram_total stride). The loop bound is the params' num_q7 word,
// read back from [rsp+0x10] into the loop counter (0x24(%rsp)).
for (u32 core = 0; core < num_q7; core++) {
    // (i) DRAM FIRST — the data image lands before the code image:
    if (write_padded(dst_dram[core], imgs[1].bin, imgs[1].size, dram_alloc_size,
                     swap_cb, swap_ctx, "POOL eng dram") != 0)
        { al_hal_log("Failed to load pooling q7 dram core %u", core); return -1; }
    // (ii) IRAM SECOND:
    if (write_padded(dst_iram[core], imgs[0].bin, imgs[0].size, iram_alloc_size,
                     swap_cb, swap_ctx, "POOL eng iram") != 0)
        { al_hal_log("Failed to load pooling q7 iram core %u", core); return -1; }
}

QUIRK — broadcast-by-replication. With num_q7 == 8 for POOLING, the same image is written to all 8 cores — the engine is an 8-core SPMD cluster running one identical program (consistent with the device-side "all-POOL broadcast", total_cpus == 8). [HIGH/OBSERVED]

GOTCHA — order is DRAM-then-IRAM, per core. A reimplementation that writes IRAM first (or all DRAM then all IRAM) diverges from the shipped order. The data image is installed before the code image on each core. [HIGH/OBSERVED]

Cayman and Mariana loop bodies are byte-identical to each other and structurally identical to Sunda; only the owner-branch and the params differ. [HIGH/OBSERVED]

4d. write_padded @0x473eb0 — the pad+copy+write primitive

[HIGH/OBSERVED]

// write_padded @0x473eb0
//   473ec3: cmp %r8,%rcx          assert(padded_size >= size)
//   473ee3: call al_zalloc        buf = al_zalloc(padded_size)   -- zero-fills the pad tail
//   473ee8: test %rax,%rax ; je <"calloc failed! size=%lu">
//   473efd: call memcpy           memcpy(buf, src, size)
//   473f02: test %r12,%r12 ; je <else>     -- swap_cb present?
//   473f1a: call *%r12            rc = swap_cb(ctx_lo, ctx_hi, buf, padded_size, owner)  (DKL hook)
//   473f26: call al_free          al_free(buf)
int write_padded(u64 dst, const void *src, u64 size, u64 padded_size,
                 swap_fn swap_cb, void *swap_ctx, const char *tag) {
    assert(padded_size >= size);
    void *buf = al_zalloc(padded_size);        // zero pad tail
    if (!buf) { al_hal_log("calloc failed! size=%lu", padded_size); return -1; }
    memcpy(buf, src, size);
    int rc = swap_cb ? swap_cb(swap_ctx_lo, swap_ctx_hi, buf, padded_size, owner)  // DKL seam
                     : al_mem_write_buf(dst, buf, padded_size);                    // BAR0 PIO
    al_free(buf);
    return rc;   // on swap_cb error: "memcpy_fn(%p) failed with %d"
}

NOTE — the zero-pad matters. The image is zero-padded to the per-core aperture size before the word-blast, so the engine IRAM/DRAM is fully initialized with no stale tail. The optional swap_cb is the seam the nrtucode layer can interpose to stream/swap images instead of a straight MMIO copy (see The nrtucode Subsystem); in the stpb bring-up path it is passed through from the stpb config. [HIGH/OBSERVED]

5. The swap / dynamic-kernel-load (DKL) mechanism

aws_hal_q7_swap_table / _swap_table_get_register_offsets / _swap_file_io_table are the runtime image-swap (DKL) control surface. The critical finding is per-arch.

5a. Sunda — REAL (pokes per-core NX local registers)

aws_hal_q7_swap_table_sunda @0x46b7d0 computes base = aws_hal_arch_get_xt_local_reg_offset(...) then al_reg_write32s three registers — the swap-table descriptor triple. The disasm shows the three lea 0x83c/0x840/0x844(%rbx),%rdi; call al_reg_write32 sequences (each preceded by an al_hal_log): [HIGH/OBSERVED]

// aws_hal_q7_swap_table_sunda @0x46b7d0
//   46b7f4: call aws_hal_arch_get_xt_local_reg_offset -> base in %rbx
//   46b819: lea 0x83c(%rbx),%rdi ; call al_reg_write32   nx_rsvd_space0  (value 0 = base/lo)
//   46b845: lea 0x840(%rbx),%rdi ; call al_reg_write32   nx_rsvd_space1  (value 1 = len/hi)
//   46b870: lea 0x844(%rbx),%rdi ; call al_reg_write32   nx_rsvd_space2  (value 2 = index/ctrl)

The nx_rsvd_space0..2 registers are repurposed reserved local regs carrying the DKL swap-table pointer triple. aws_hal_q7_swap_table_get_register_offsets_sunda @0x46b890 returns those three addresses (base+0x83c/0x840/0x844) so the firmware/nrtucode layer can DMA a swap descriptor to them directly. aws_hal_q7_swap_file_io_table_sunda @0x46b8e0 al_reg_write32s the host↔device file/stdio I/O table: base+0x70 dma_tx_ring_base, base+0x74 dma_tx_ring_length, base+0x838 q7_intr_info — wiring the pool_stdio channel into the SEQ DMA TX ring + the Q7 interrupt-info register. [HIGH/OBSERVED]

5b/5c. Cayman + Mariana — NO-OP stubs

[HIGH/OBSERVED]

// aws_hal_q7_swap_table_cayman @0x471470  (Mariana @0x464ab0 byte-identical):
//   471470: xor %eax,%eax ; ret
// aws_hal_q7_swap_table_get_register_offsets_cayman @0x471480  (Mariana @0x464ac0):
//   writes 0 to each of the 3 out-pointers (null-guarded), then repz ret
// aws_hal_q7_swap_file_io_table_cayman @0x4714b0  (Mariana @0x464af0):
//   4714b0: xor %eax,%eax ; ret

CORRECTION — only Sunda uses the HAL swap registers. An earlier framing treated the swap_table family as the generic DKL mechanism for all arches. The binary disagrees: on Sunda the DKL swap + the stdio/file-IO table are programmed by the host poking per-core NX local registers; on Cayman and Mariana these HAL hooks are empty — DKL is handled entirely on the device firmware side (the dynamic-kernel-load path) over the prelinked ext-ISA image streamed into the reserved IRAM/DRAM tail (iram_rsvd 0x60000 / dram_rsvd 0x40000 of §3). This is precisely why Cayman/Mariana carry reserved IRAM/DRAM tails that Sunda does not — those tails are the DKL staging region, managed by firmware, not by the host HAL. [HIGH for the stub/real split; firmware-resident DKL CARRIED]

6. The per-engine local-register map

The HAL addresses every engine's control registers as `tpb_base + per-engine-type local-reg base

• register-offset. The base computer is aws_hal_arch__get_xt_local_reg_offset(u64 tpb_base /rdi/, int eng_type /esi/, int owner /dl/) → tpb_base + table[eng_type], with the Q7 type folding owner` into the base.

6a. Sunda — 5 engine types

aws_hal_arch_sunda_get_xt_local_reg_offset @0x479070 is a jump table (cmp $0x4,%esi; ja <assert>; movslq (%rcx,%rsi,4),%rax; add %rcx,%rax; jmp *%rax with table @0x9f2700). The five arms read as immediate mov $imm,%eax; add %rdi,%rax; ret: [HIGH/OBSERVED]

esi>4 → __assert_fail. The Q7 arm (@0x4790ab) is the cmp $0x1,%dl; sbb %rax,%rax; and $0xfffffffffd800000,%rax; add $0x2860000,%rax idiom — i.e. owner==1 ? 0x2860000 : 0x60000.

6b. Cayman — 6 engine types (incl. the bit-35 secure-aperture fold)

aws_hal_arch_cayman_get_xt_local_reg_offset @0x47afe0 (table @0x9f41e0): [HIGH/OBSERVED]

esi>5 → __assert_fail. The Q7 owner!=0 base is computed as 0x802860000 minus a 0xfffffff800000000 add (i.e. subtract 0x800000000) → 0x2860000. That 0x8_00000000 is bit 35 — the PEB_APB_IO secure/privileged aperture — masked off in the host view. (The neighboring aws_hal_get_eng_hw_decode_table_params_cayman @0x47af40 shows the same movabs $0xfffffff800000000 fold for its 0x802b88000 base, independently confirming the idiom.)

NOTE — Mariana eng-type table is treated as Cayman-like (MED). Mariana's get_xt_local_reg_offset was not separately tabled here, but its swap stubs and ucode_eng_init mirror Cayman, and its run-stall path uses the Cayman-style real eng-release (see §7). [MED/INFERRED]

6c. The HAL-flattened NX/Q7 local-register offsets

Relative to the §6a/§6b base, recovered from the per-register writer functions (each is a one-line add $off,%rdi; jmp al_reg_write32 thunk, or names the register in its al_hal_log): [HIGH/OBSERVED]

6d. The offset-vs-spec reconciliation

This is the single most important reimplementation trap on the page. [HIGH/OBSERVED that the offsets differ; MED on the cause]

The device-side register spec (the tpb_xt_local_reg CSR block, Part 13 — plain text here, not yet authored) lays the NX bundle out as release_run_stall@0x0000, start_ctrl@0x0004, run_state@0x0008, dma_rx_base@0x000C, dma_tx_base@0x0010, instr_halt_ctrl@0x0014, intr_ctrl@0x0018, intr_info@0x001C; the Q7 bundle @0x3000 with release_run_stall@0x3000 [7:0] reset 0xFF, start_ctrl@0x3004, run_state_0..7@0x3008..0x3024, intr_ctrl@0x3028, intr_info_0..7@0x302C..0x3048; a window bundle @0x2000 (stride 0x1C); hw_decode@0x4000; tensor_replace@0x5000; notific@0x6000.

The host HAL uses a different, flattened offset table (the §6c numbers) whose register names match the device spec's logical registers but whose byte offsets do not coincide with the spec's bundle offsets. Concretely:

• The HAL writes "release_run_stall" at +0x04 — which is start_ctrl's offset in the device spec.
• The HAL reads "run_state" at +0x14 — which is instr_halt_ctrl's device offset.

KaenaHal was generated from a separate (Sunda-physical / register-spec) layout, not the device-side per-arch register spec. The two are two views of the same registers and must not be conflated.

GOTCHA. For a reimplementation: trust the device CSR spec (tpb_xt_local_reg, Part 13) for the device register semantics, and the §6c table for the host HAL's actual write offsets. The functional intent is unchanged — write 0 → release run-stall, read low bit → running — but the byte offsets differ. The +N shift is OBSERVED; the "separate spec" cause is MED (inferred from the consistent shift and the sunda_* string prefix). [HIGH offsets / MED cause]

7. The full Pool bring-up sequence (reset / start / halt control)

aws_hal_stpb_init @0x458350 (top) runs aws_hal_notific_init (per-TPB notification ring) → stpb_eng_notif_enable / evsem_notif_enable / error_notif_enable (arm the 3 notification classes — ERROR/EVT_SEM/INST) → aws_hal_stpb_regs_init → aws_hal_stpb_pooling_init @0x45c170 (dispatch via khal+0x368 to the per-arch impl). The per-arch pooling_init is the actual Q7 bring-up. The Sunda variant aws_hal_stpb_sunda_pooling_init @0x46e8d0 recovers this config struct (rdi=cfg): [HIGH/OBSERVED]

The steps, with addresses re-disassembled: [HIGH/OBSERVED]

STEP 1  regs_init @0x46e5d0
        - instr_debug_level_set @0x46e4b0: arch_stpb_to_instr_debug_level();
          write seq instr-debug ctrl @ tpb_regs+0x104; write NX-local instr_dbg_ctrl @+0x804;
          read NX-local run_state @+0x14 (bit0).
        - read tpb_regs+0xa00 (notific_sw_queue_num0); splice engine-id nibbles from
          cfg+0xe4 (<<0x14, mask 0xF00000) and cfg+0xe8 (<<0x18 & <<0x10); write back
          tpb_regs+0xa00.  => programs the notification SW-queue identity for the Pool eng.

STEP 2  ucode_seq_init @0x46e6a0
        get_seq_params_sunda(); validate seq iram/dram sizes;
        write_padded(seq IRAM "iram") + write_padded(seq DRAM "dram");
        init_nx_registers @0x46cf10; write NX-local engine_base_address_lo@+0x28 /
        _hi@+0x2c (the SEQ I-fetch base from cfg+0x10).

STEP 3  q7_ucode_eng_init(handle, image_base, &cfg[0x48], swap_cb, swap_ctx, owner_idx=0)
        -> §4: streams the (possibly overridden) Q7 image into all 8 cores' IRAM/DRAM over BAR0.

STEP 3b (CAYMAN/MARIANA only) hw_decode_table_init(&cfg[0x68])
        programs the HW-decode CAM/table (see HW-Decode CAM-Table Programming), interposed
        between Q7 install and dma_init.  (Sunda HAS aws_hal_..._hw_decode_table_init @0x46e8c0
        but its pooling_init does not call it in-line; MED whether Sunda programs it elsewhere.)

STEP 4  dma_init @0x46e840
        - dma_ctrl(base, 0)  : write NX-local dma_ctrl @+0x40 = 0  (DMA disabled/reset).
        - if cfg+0x88[0]!=0  : dma_regs(rx: cfg+0x18/0x20/0x28, sel=0) then
                               dma_regs(tx: cfg+0x40/0x48/0x50, sel=1) — programs
                               mem_window2_lo/hi @+0x210/+0x214 and the dma_rx ring
                               head/tail/tail_inc @+0x58/+0x5c/+next (the SEQ iDMA
                               instruction-cache-fill ring).
        - dma_ctrl(base, 1)  : write dma_ctrl @+0x40 = 1  (DMA enabled/started).

STEP 5  release_seq_run_stall @0x46e7e0  => RELEASES THE NX/SEQ CORE FROM RESET RUN-STALL.
STEP 6  release_eng_run_stall  @0x46e810  => RELEASES (or not) the 8 Q7 cores — see §7.1.

7.1 The run-stall release — the Sunda-vs-Cayman+ split

This is the heart of reset/start control, and the disasm is unambiguous. Both STEP 5 and STEP 6 resolve the same base via get_xt_local_reg_offset(eng_type=2 POOL, owner=1), then tail-jump a per-arch writer. [HIGH/OBSERVED]

// aws_hal_stpb_sunda_pooling_release_seq_run_stall @0x46e7e0
//   46e7e4: mov $0x2,%esi ; mov $0x1,%edx          eng_type=2 (POOL), owner=1
//   46e7ee: call aws_hal_arch_sunda_get_xt_local_reg_offset
//   46e7f3: xor %esi,%esi                           value = 0 (release)
//   46e7fc: jmp aws_hal_arch_sunda_write_tpb_nx_local_reg_release_run_stall  // @0x4796a0
//
// aws_hal_arch_sunda_write_tpb_nx_local_reg_release_run_stall @0x4796a0
//   4796cb: lea 0x4(%rbx),%rdi                      base + 0x04
//   ...     call al_reg_write32                     write 0 to NX release_run_stall
//   => NX/SEQ core begins executing.

// aws_hal_stpb_sunda_pooling_release_eng_run_stall @0x46e810
//   46e814: mov $0x2,%esi ; mov $0x1,%edx ; call get_xt_local_reg_offset
//   46e82c: jmp aws_hal_arch_sunda_write_tpb_xt_local_reg_q7_release_run_stall  // @0x4796e0
//
// aws_hal_arch_sunda_write_tpb_xt_local_reg_q7_release_run_stall @0x4796e0
//   4796e0: repz ret                                 <<< NO-OP STUB >>>

CORRECTION / QUIRK — on Sunda the host does NOT release the Q7 cores. STEP 6's Sunda Q7 writer @0x4796e0 is a bare repz ret: the 8 Q7 compute cores are not released by this register on Sunda — they are driven by the SEQ via EVT_SEM/run_state. On Cayman/Mariana, release_eng_run_stall (e.g. Cayman @0x4735a0) is a real function that computes the Q7 base and clears its per-core release_run_stall (the 0xFF → 0x00 of the device Q7 bundle release_run_stall@0x3000). i.e. Cayman/Mariana release the 8 Q7 cores from the host; Sunda releases them from the SEQ. The matching device-side detail (Sunda SEQ EVT_SEM vs Cayman/Mariana host 0xFF→0x00 stall-clear) is the sibling page HW-Decode CAM-Table Programming. [HIGH/OBSERVED for the stub-vs-real split]

7.2 Reset / start / halt — the control-register summary

[HIGH/OBSERVED for the start path; halt-being-device-driven CARRIED]

• RESET state. NX release_run_stall reset = 1 (held stalled); Q7 release_run_stall reset = 0xFF (all 8 stalled) — device CSR spec.
• START. Write 0 to NX release_run_stall (HAL +0x04) → NX runs. On Cayman/Mariana, write 0x00 to the Q7 per-core release_run_stall bits → Q7 cores run. The start_ctrl.ctrl bit (device NX@0x04 / Q7@0x3004) latches start_addr valid and exits Halt. dma_ctrl@+0x40 enable+start gates the SEQ I-fetch DMA.
• HALT. The HAL has no explicit "halt" writer in the Pool bring-up. Halt is a device-driven (firmware) state — the device fault path writes a halt CSR and spins; the host observes it via the notification ring + run_state @+0x14 bit0.

8. SBUF / PSUM / state-buffer relationship

The aws_hal_q7_* HAL does not program SBUF/PSUM banks directly — those are on-chip SRAMs the engines address through the datapath, not host CSRs. What this HAL does program that touches the buffer model: [HIGH structure; bank geometry CARRIED]

• (a) The NX address-translation windows (mem_window2_lo/hi @+0x210/+0x214, written in dma_init STEP 4) remap a requester (NX, or a selected Q7) address into a SoC address — they pin the Q7's view of SBUF / HBM-scratch / dynamic HBM windows.
• (b) The per-core Q7 DRAM (§3): iram_size/dram_size define the per-core apertures the ucode is installed into; the DKL reserved tails (Cayman/Mariana) carve the ext-ISA staging region within DRAM.
• (c) The notification SW-queue base (notific_sw_queue_num0 @+0xa00, STEP 1) + the file-IO/stdio TX ring (§5a) bind the Pool engine's completion + host↔device console channels into SBUF/dataram.

The actual SBUF (32 MiB, 16 ECC banks) and PSUM (4 MiB, 32 banks = 4 clusters × 8) banking is device-side datapath geometry driven by the PE/Pool sequencers, not by aws_hal_q7_*.

9. Per-arch difference matrix

[HIGH/OBSERVED except MED rows flagged]

NOTE. Cayman & Mariana get_q7_params + ucode_eng_init are byte-identical to each other; the (NC-v3 vs NC-v4) divergence is in the firmware DKL flavor + the device rev, not the host HAL. Sunda is the structural outlier (smaller apertures, no CC, host-resident swap, SEQ-released Q7). [HIGH/OBSERVED]

10. The end-to-end programming trace (one picture)

[steps OBSERVED; the full chain is INFERRED from the per-step disasm]

[register] nrt_set_pool_eng_ucode(iram,dram) -> pool_eng_*_bin globals
[init]     nrt_init -> tpb_eng_init_hals_v2 builds the 5-engine stpb cfg; sets
           cfg[Q7].{iram,dram} = stock q7 ucode bins, then OVERRIDES with the
           registered globals if non-NULL ->
           aws_hal_stpb_init @0x458350 -> notific_init + 3x notif_enable + regs_init
           -> aws_hal_stpb_pooling_init (khal+0x368) -> stpb_<arch>_pooling_init:
             regs_init        (notific SW-queue id + instr-dbg)
             ucode_seq_init   (seq IRAM/DRAM write_padded + engine_base_addr_lo/hi)
             q7_ucode_eng_init(owner=0) -> get_q7_params(owner=2) -> validate sizes
               -> per-core loop: write_padded(DRAM) then write_padded(IRAM)
               -> al_mem_write_buf -> axi_write -> ndl_bar_write (BAR0 PIO)
             [cayman/mariana] hw_decode_table_init
             dma_init         (dma_ctrl=0; rx/tx ring regs + mem_window2; dma_ctrl=1)
             release_seq_run_stall (NX +0x04 <- 0)                 => SEQ runs
             release_eng_run_stall (Q7: sunda=noop / cayman+=clear) => Q7 runs
[run]      Q7 Pool engine executes the (custom) kernel; completion via the notific
           SW-queue + file-IO/stdio ring (Sunda local regs / firmware-side Cayman+);
           faults via the 16-B notification record.

11. Self-verification — the top-5 claims, re-checked this session

Each was re-disassembled at the named address with bounded objdump against this libnrt.so:

1. Dispatch shape + khal+0xb8 slot. aws_hal_get_q7_params @0x44bfb0: call al_hal_tpb_get_arch_type → je <assert> → lea …#caeb80 <kaena_khal>; mov 0xb8(%rax),%rax → je <assert> → jmp *%rax. Confirmed — kaena_khal @0xcaeb80, slot +0xb8, tail-jump. [HIGH/OBSERVED]
2. Sunda geometry. get_q7_params_sunda @0x478f90: cmp $0x2,%edi; jne <assert>, then movq $0x2980000 / $0x10000 / $0x10000 / $0x0 / $0x0 and movl $0x8. Confirmed — base 0x2980000, IRAM/DRAM 64 K, no rsvd, 8 cores, pooling-only. [HIGH/OBSERVED]
3. Cayman dual-owner geometry + secure fold. get_q7_params_cayman @0x47ae70: je for owner==2 (num_q7=8), cmp $0x5 for CC (num_q7=4); both base 0x3100000, IRAM 0x20000, DRAM 0x40000, rsvd 0x60000/0x40000. The sibling …hw_decode_table_params_cayman independently shows the movabs $0xfffffff800000000 bit-35 fold. Confirmed. [HIGH/OBSERVED]
4. Sunda swap REAL vs Cayman STUB. swap_table_sunda @0x46b7d0 lea 0x83c/0x840/0x844(%rbx); call al_reg_write32 ×3; swap_table_cayman @0x471470 = xor %eax,%eax; ret; …get_register_offsets_cayman @0x471480 zeroes its 3 out-pointers. Confirmed. [HIGH/OBSERVED]
5. Run-stall split. release_seq_run_stall @0x46e7e0 → writes NX +0x04 = 0 (via @0x4796a0, lea 0x4(%rbx),%rdi); release_eng_run_stall @0x46e810 tail-jumps the Sunda Q7 writer @0x4796e0 which is repz ret (no-op). Confirmed — host releases NX, Sunda's Q7 release is a stub. [HIGH/OBSERVED]

Supporting confirmations: al_mem_write_buf @0x265990 (test $0x3 align asserts, shr $0x2 word-count, call axi_write), ucode_eng_init_sunda @0x46b9a0 (test %r9d,%r9d; jne <reject>, then call get_q7_params_sunda, size cmps), write_padded @0x473eb0 (al_zalloc → memcpy → call *%r12 (swap_cb) | al_mem_write_buf → al_free). All five error strings ("IRAM ucode too big! …", "DRAM ucode too big! …", "Invalid q7 owner %u for v2 …", "Failed to load pooling q7 {dram,iram} core %u") read verbatim from .rodata.

12. Confidence, gaps, corrections

HIGH/OBSERVED: the kaena_khal dispatch slots (0xb8/0xf0/0x368/0x390/0x398/0x3a8); the three get_q7_params value tuples; the ucode_eng_init owner mapping (0→2 pooling, 1→5 CC; Sunda pooling-only), size-validation gates, and per-core DRAM-then-IRAM write_padded loop; write_padded (zalloc+pad / memcpy / swap_cb-or-al_mem_write_buf); the al_mem_write_buf word-blast → axi_write → ndl_bar_write BAR0 path; the al_reg_write32 single-CSR path; the swap real-vs-stub per-arch split; the per-engine local-reg base tables (Sunda 5-type, Cayman 6-type incl. bit-35 fold); the §6c HAL-flattened offsets; the full bring-up call sequence and the Sunda-noop / Cayman-real Q7 release split.

MED: the khal+0x3a0 slot = swap_file_io_table (adjacency-inferred); the Mariana xt_local_reg eng-type table + hw_decode in-line step (mirror-inferred from Cayman); the §6d "separate register spec" cause (the functional shift is OBSERVED, the cause MED); the Sunda hw_decode_table_init programming site.

LOW / not covered: the exact write_padded swap_cb body when interposed by nrtucode (the seam is OBSERVED; the callback target is a runtime pointer); the device-side firmware DKL bind for Cayman/Mariana; the notification/EVT_SEM completion-ring byte format.

CORRECTIONS made on this page:

1. Swap family is NOT a generic DKL mechanism. Only Sunda uses the HAL swap registers; Cayman/Mariana are NO-OP stubs with firmware-resident DKL (§5).
2. HAL register offsets differ from the device CSR spec offsets. Two views of the same registers — trust the device spec for semantics, §6c for host write offsets (§6d).
3. Cayman/Mariana get_q7_params + ucode_eng_init are byte-identical; the NC-v3/v4 split is firmware/rev, not host HAL (§9).

Cross-references

• The nrtucode Subsystem + Device Bring-Up — the layer above this HAL (the swap_cb interposer, the platform_rw/memhandle impls, the ext-ISA DKL libs).
• HW-Decode CAM-Table Programming — STEP 3b and the device-side Q7 run-stall-release detail (Sunda SEQ EVT_SEM vs Cayman/Mariana host 0xFF→0x00).
• Boot / Reset Sequence + Startup Config — the device-side reset vector that waits behind the run-stall this HAL clears.
• tpb_xt_local_reg CSR (Part 13, control/csr/tpb-xt-local-reg.md — plain text, not yet authored) — the authoritative device-side register-bundle layout this HAL drives, reconciled in §6d.