Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Subsystem ↔ Binary ↔ Source-TU Matrix

Binaries (version pin): libnrt.solibnrt.so.1libnrt.so.2.31.24.0 · 2.31.24.0-0b044f4ce · BuildID[sha1] 8bb57aba0fb2e0035f1d88e9fc4fb3e7387c102e · ELF64 DYN x86-64, NOT stripped, full DWARF v4 · aws-neuronx-dkms 2.27.4.0 (GPL-2.0 .ko, kernel module) · libncfw.so · BuildID a98f8e1c… · firmware carrier, NO DWARF · libnrtucode_extisa.so · BuildID 7bb03bc4… · GPSIMD ext-ISA provider, NO DWARF · libnccom.so · BuildID 9c00176c… + libnccom-net.so · BuildID 3415f096… · NCCL-fork collectives (dlopen'd, NOT DT_NEEDED) · libnds.a · datastore static lib (6 neuron_ds*.c.o members, compiled once and linked into libnrt.so).

Part XVI — Appendices / REFERENCE · Evidence grade: every source-TU cell is a verbatim DWARF DW_AT_name path from libnrt.so's .debug_info (331 compile units, comp_dir = /opt/workspace/KaenaRuntime/build/private/develop/almalinux/RelWithDebInfo); every binary boundary is from readelf -d/-V and the dlopen string table.

Abstract

This page is the master jump table of the book. Every other page documents one subsystem in depth; this matrix answers the orthogonal question — "I need subsystem X: which binary is it in, which translation unit implements it, and which page covers it?" — in a single lookup. The columns are deliberately the four coordinates a reimplementer needs before opening any deep page: the Part (where the book files it), the subsystem, the binary it lives in, the source TU(s) recovered from DWARF, and the exact wiki page path. Each row carries a Confidence tag because the binary→TU→page mapping is itself reverse-engineered: a row is HIGH when a DWARF compile-unit path is cited verbatim by a coverage cell, MEDIUM when the TU is name-matched only by basename or folded into a neighbour CU by -O2 inlining, and LOW when the binary is present but carries no DWARF (libncfw.so, libnrtucode_extisa.so) so the TU column is structural, not address-grounded.

The wiki is dual-organized, and this matrix is the index across both organizations. One axis is the subsystem Part (Parts I–XV group code by what it does — silicon model, model loading, collectives, profiling), and the row order below follows it. The other axis is the binary lane (Part II is the libnrt.so forensics lane; Part III is the kernel-driver .ko lane; Part X/XI/XII are the firmware/microcode/NCCL-fork binary lanes). A subsystem can appear on both axes — the datastore is a one-page map in the Part III kernel lane (kernel/datastore.md) and a four-page deep treatment in its own Part XIV (datastore/*), because the counter plane straddles the .ko and libnds.a. The ## Dual-Organization note at the foot of the page enumerates those straddles so a reader who lands on one face can find the other.

Binary legend

BinarySONAME / nameBuildIDDWARF?Role
libnrt.so.2.31.24.0libnrt.so.18bb57aba… (sha1)full DWARF v4Userspace runtime: public C API, TDRV device driver, NEFF/KELF load, exec, DMA, on-device collectives engine, profiling. 203 first-party CUs / 4,407 fns.
aws-neuronx-dkms 2.27.4.0neuron.ko(GPL .ko)kernel DWARF (GPL source)Kernel driver: PCI probe, char-dev/IOCTL plane, DMA op layer, UDMA, DHAL vtables, NQ engine, reset, kernel datastore slabs, sysfs/metrics.
libncfw.solibncfw.soa98f8e1c…noneCollectives-firmware carrier: 8 embedded Xtensa CC-engine blobs + per-arch serializers; dlopen'd by the runtime for firmware upload.
libnrtucode_extisa.solibnrtucode_extisa.so7bb03bc4…noneGPSIMD/Q7 ext-ISA provider (nrtucode_* API), 13 Q7 microcode blobs, IVP vector-ISA tables; dlopen'd by ucode.c.
libnccom.so + libnccom-net.solibnccom.so / libnccom-net.so9c00176c… / 3415f096…(separate pkg)NCCL-fork multi-node collectives transport + aws-ofi-nccl/libfabric net plugin. dlopen'd, not DT_NEEDED; the bidirectional ABI is 37 neuron* imports ← 16 nec_* reverse-callbacks.
libnds.a(static archive)folded into libnrt.so DWARFNeuron DataStore static lib: 6 neuron_ds*.c.o members == the nds/ first-party CUs; compiled once, statically linked into libnrt.so.

NOTE — the on-device collectives algorithm engine (enc_*/alg_ring/kangaring/mesh/rdh) is statically embedded inside libnrt.so (the enc/ and tdrv/encd/ CUs), NOT in libncfw.so or libnccom.so. Only the multi-node transport crosses the dlopen boundary into libnccom.so. Do not assume "collectives" == "the dlopen'd lib"; most of the collective compute is in libnrt.so itself — see Part IX vs Part XII below.


How to read a row

binary is where the code executes. source TU(s) is the DWARF DW_AT_name path under /opt/workspace/KaenaRuntime/ (first-party) or the vendored/.ko/firmware origin. wiki page is the exact SUMMARY.md path — every cell below is a real, shipped page. A + between TUs lists co-implementers of one subsystem; a TU listed under two Parts means the book splits one TU's surface across two pages (e.g. nrt_profile.cpp → inspect-profile API in Part XIII).


Part 0 — Reference Apparatus

SubsystemBinarySource TU(s)Wiki pageConf
The five-binary map (this index's prose companion)all five— (cross-binary census)front/five-binaries.mdHIGH
First-party source-tree reconstructionlibnrt.soall 203 first-party CUs (DWARF comp_dir)front/source-tree.mdHIGH
Binary section/segment layout censuslibnrt.so(ELF structure, no TU)reference/binary-layout.mdHIGH
Extraction provenance and package setall five(package manifests)reference/extraction-status.mdHIGH

Part I — Silicon & Architecture Model

SubsystemBinarySource TU(s)Wiki pageConf
Generation enum (V2/V3/V4) + cloud naminglibnrt.sotdrv/tdrv_arch_type.carch/generations-enum.mdHIGH
PCI device-ID → arch maplibnrt.so + .kotdrv/tdrv_arch_type.c (+ kernel pci-probe)arch/pci-device-ids.mdHIGH
Per-generation HW geometrylibnrt.sotdrv/encd/archs/{cayman,mariana,sunda}.carch/hw-geometry.mdHIGH
Coretype numberinglibnrt.sotdrv/tdrv_arch_type.carch/coretype-numbering.mdHIGH
Memory hierarchy / BAR / state bufferlibnrt.sotdrv/tdrv_arch_type.c + tdrv/dma_memory.carch/memory-hierarchy.mdMEDIUM

NOTE — the silicon model is recovered from libnrt.so's per-arch fan-out (cayman/mariana/sunda), not from the .ko. The three *ArchHeaders Brazil packages (CaymanArchHeaders-1.0.826.0, MarianaArchHeaders-1.0.1133.0, SundaArchHeaders-1.0.1851.0) version-stamp the register layouts these TUs compile against but carry no functions.


Part II — Binary Anatomy & Forensics (libnrt.so)

SubsystemBinarySource TU(s)Wiki pageConf
Heavy-frame census (first-party vs vendored)libnrt.soall 331 CUs (DWARF partition)forensics/overview.mdHIGH
ELF anatomy (DT_NEEDED, sections, DWARF)libnrt.so(ELF structure)forensics/elf-anatomy.mdHIGH
Vendored SBOM (Abseil/protobuf/Rust/libarchive/zlib/simdjson)libnrt.so122 vendored CUs (see SBOM page)forensics/vendored-sbom.mdHIGH
Static-init pipelinelibnrt.so(.init_array, cross-TU)forensics/static-init.mdMEDIUM
Globals / singletons atlaslibnrt.sokmgr/xu/comp_handle_pool.c + cross-TU .bssforensics/globals-atlas.mdMEDIUM
Dispatch-table taxonomylibnrt.sotdrv/tdrv_arch_type.c + tdrv/encd/archs/arch.cforensics/dispatch-tables.mdMEDIUM
C++ class hierarchy / RTTIlibnrt.soenc/*.cc + Abseil/protobuf vtablesforensics/rtti-class-hierarchy.mdMEDIUM
String domainlibnrt.so.rodata + nrt/nrt_interned_string_db.cppforensics/string-domain.mdHIGH
CRT / PLT / loader surfacelibnrt.so(CRT glue, .plt)forensics/crt-plt.mdHIGH

Part III — Kernel Driver (DKMS, GPL Source)

Every row in this Part lives in aws-neuronx-dkms 2.27.4.0 (neuron.ko); the source TUs are kernel .c files from the GPL DKMS tree, not the libnrt.so DWARF set. Listed by page; TUs given where the GPL filename is known.

SubsystemBinarySource TU(s)Wiki pageConf
Driver overview.ko(DKMS root)kernel/overview.mdHIGH
Module layout / build.koMakefile + module initkernel/module-layout.mdHIGH
PCI probe / device detection.koneuron_pci.ckernel/pci-probe.mdHIGH
Char device, fops, mmap.koneuron_cdev.ckernel/cdev-mmap.mdHIGH
IOCTL dispatch + privilege gate.koneuron_ioctl.ckernel/ioctl-dispatch.mdHIGH
IOCTL catalog.koneuron_ioctl.ckernel/ioctl-catalog.mdHIGH
Memory IOCTL handlers.koneuron_ioctl.c + neuron_mempool.ckernel/ioctl-mem.mdHIGH
DMA IOCTL handlers.koneuron_ioctl.c + neuron_dma.ckernel/ioctl-dma.mdHIGH
NQ / semaphore IOCTL handlers.koneuron_ioctl.c + neuron_nq.ckernel/ioctl-nq.mdHIGH
Pod / misc IOCTL handlers.koneuron_ioctl.ckernel/ioctl-pod.mdHIGH
Memory pool / MC handle table.koneuron_mempool.ckernel/mempool-handles.mdHIGH
DMA op layer / completion marker.koneuron_dma.ckernel/dma-op-layer.mdHIGH
DMA rings / H2T queues.koneuron_ring.ckernel/dma-rings.mdHIGH
UDMA core (Annapurna/Alpine fork).koudma/udma.ckernel/udma-main.mdHIGH
UDMA M2M builder.koudma/udma_m2m.ckernel/udma-m2m.mdMEDIUM
UDMA IOFIC interrupt controller.koudma/udma_iofic.ckernel/udma-iofic.mdMEDIUM
DHAL core (ndhal vtable-of-vtables).kondhal/neuron_dhal.ckernel/dhal-core.mdHIGH
DHAL V2 (Trn1/Inf2).kondhal/ndhal_v2.ckernel/dhal-v2.mdHIGH
DHAL V3 (Trn2).kondhal/ndhal_v3.ckernel/dhal-v3.mdHIGH
DHAL V4 (Trn3).kondhal/ndhal_v4.ckernel/dhal-v4.mdHIGH
Notification-queue engine.koneuron_nq.ckernel/notification-queues.mdHIGH
TopSP notification path.koneuron_topsp.ckernel/topsp.mdMEDIUM
Reset state machine.koneuron_reset.ckernel/reset.mdHIGH
Cooperative RW lock (CRWL).koneuron_crwl.ckernel/crwl.mdMEDIUM
Pod election (UltraServer).koneuron_pod.ckernel/pod-election.mdMEDIUM
FW-IO MiscRAM mailbox.kofw_io.ckernel/fw-io.mdMEDIUM
DMA-buf export / P2P.koneuron_dmabuf.ckernel/dmabuf-p2p.mdMEDIUM
Kernel datastore (per-process slabs).koneuron_ds.c (kernel)kernel/datastore.mdHIGH
Metrics aggregation.koneuron_metrics.ckernel/metrics.mdMEDIUM
Sysfs metrics tree.koneuron_sysfs_metrics.ckernel/sysfs.mdMEDIUM
Power telemetry.koneuron_power.ckernel/power.mdMEDIUM
Log ring / PID / trace / reg / core.koneuron_log.c + misckernel/misc.mdMEDIUM

NOTE — the kernel TU filenames are reconstructed from the GPL DKMS source tree, not from libnrt.so DWARF (the .ko is a separate binary). The KaenaHal-2.31.0.0 al_hal_*/aws_hal_* DHAL has NO compile-unit in libnrt.so — it is kernel/driver-side; its strings reach libnrt.so only through inlined assert headers. That negative DWARF evidence is what assigns DHAL to the Part III .ko lane, not the Part IV userspace lane.


Part IV — Userspace Runtime Core (libnrt.so)

SubsystemBinarySource TU(s)Wiki pageConf
Runtime overviewlibnrt.sonrt/nrt_init.cpp + cross-TUruntime/overview.mdHIGH
Public C API: lifecycle / init / teardownlibnrt.sonrt/nrt_init.cpp + nrt/nrt_infodump.cppruntime/api-lifecycle.mdHIGH
Public C API: device / config / env parselibnrt.sonrt/nrt_config.cpp + utils/instance_helpers.cruntime/api-device-config.mdHIGH
Public C API: tensor surface / I/Olibnrt.sonrt/nrt_tensor.cpp + tdrv/tensor.cruntime/api-tensors.mdHIGH
Public C API: async / sendrecv / collectives thunkslibnrt.sonrt/nrt_async.cpp + nrt/nrt_async_sendrecv.cpp + nrt/nrt_collectives.cppruntime/api-async-collectives.mdHIGH
Config structs (nrt_config/nrt_global_config)libnrt.sonrt/nrt_config.cppruntime/config-structs.mdHIGH
Env-var catalog (NEURON_RT_*)libnrt.sonrt/nrt_config.cpp (51 parse fns)runtime/env-vars.mdHIGH
Error / status codes (NRT_STATUS)libnrt.sonrt/nrt_status.cpp + nrt/nrt_status_priority.cppruntime/error-codes.mdMEDIUM
Interned string databaselibnrt.sonrt/nrt_interned_string_db.cppruntime/interned-strings.mdHIGH
TDRV bring-up / lifecyclelibnrt.sotdrv/tdrv.c + tdrv/init.cruntime/tdrv-lifecycle.mdHIGH
TDRV device-memory allocator (dmem)libnrt.sotdrv/dma_memory.c + tdrv/mem_ref.cruntime/tdrv-dmem.mdHIGH
TDRV DMA descriptor rings / swap-queuelibnrt.sotdrv/dma_ring.c + tdrv/dma_dynamic_ring_set.c + tdrv/sw_dma_queue.cruntime/tdrv-dma-rings.mdHIGH
TDRV HBM scratchpad / sync-pointlibnrt.sotdrv/scratchpad.c + tdrv/sync_point.cruntime/tdrv-scratchpad.mdHIGH
TDRV tensor object layerlibnrt.sotdrv/tensor.cruntime/tdrv-tensor.mdHIGH
TDRV arch-ops dispatch / sync-event accessorslibnrt.sotdrv/tdrv_arch_type.c + tdrv/helper.cruntime/tdrv-arch-ops.mdHIGH
Per-arch geometry / address maplibnrt.sotdrv/{cayman,sunda,mariana}/tdrv_arch_*.cruntime/arch-geometry.mdHIGH
Per-arch CSR / reg-offset accessorslibnrt.sotdrv/csr.c + tdrv/dma_queue_reg_offset.c + tdrv/tpb_reg_offset.cruntime/arch-csr-offsets.mdHIGH
Per-arch notification / INTC HALlibnrt.sotdrv/notification.c + tdrv/hal_platform.cruntime/arch-notification.mdMEDIUM
Per-arch SDMA / UDMA M2M enginelibnrt.sotdrv/dma_ring.c + tdrv/{cayman}/dma_desc_util.cruntime/arch-sdma.mdMEDIUM
Per-arch STPB engine-init / DGE / poolinglibnrt.sotdrv/instr_dge.c + tdrv/vtpb.cruntime/arch-stpb.mdMEDIUM
KaenaHal adapter / platform serviceslibnrt.sotdrv/hal_platform.cruntime/hal-adapter.mdMEDIUM
KaenaHal TPB/STPB arch-dispatch shimslibnrt.sotdrv/instr_tpb_functions.c + tdrv/xt_cc.cruntime/hal-tpb-shims.mdMEDIUM
KaenaHal register / reg-offset accessorslibnrt.sotdrv/csr.c + tdrv/tpb_reg_offset.cruntime/hal-registers.mdMEDIUM
KaenaHal UDMA/SDMA descriptor build + IOFIClibnrt.sotdrv/dma_ring.c + tdrv/vring.cruntime/hal-udma-iofic.mdMEDIUM
Logging (nlog) / ntrace / Rust logger bridgelibnrt.sonlog/nlog.c + tdrv/ntrace.c (+ Rust log crate)runtime/logging.mdHIGH
Device book (db / dbtc)libnrt.sotdrv/db.c + tdrv/dbtc.cruntime/device-book.mdHIGH
NDL: neuron driver layer (IOCTL/mmap wrappers)libnrt.so (vendored KaenaDriverLib)KaenaDriverLib-2.27.4.0/src/ndl.cruntime/ndl.mdHIGH

QUIRK — runtime/ndl.md documents code that is in libnrt.so but is not first-party: ndl.c (123 fns) is the vendored KaenaDriverLib-2.27.4.0 userspace driver shim, statically linked in. The same package's kernel-side counterpart is the .ko IOCTL plane (Part III). The nec_* reverse-callbacks from libnccom.so tail-call into this NDL layer (e.g. nec_get_device_countjmp ndl_available_devices).


Part V — Model Format & Loading (NEFF / KBIN)

SubsystemBinarySource TU(s)Wiki pageConf
Model-file journey overviewlibnrt.sokelf/neff.cpp + kelf/kelf.cppneff/overview.mdHIGH
NEFF container (gzip-tar)libnrt.so (+ vendored libarchive/zlib)kelf/neff.cpp + libarchive + zlibneff/container.mdHIGH
NEFF metadata schema (TVM-Relay JSON)libnrt.so (+ vendored simdjson)kmgr/neff_json_parser.cpp + simdjson.cppneff/metadata-schema.mdHIGH
NEFF section taxonomylibnrt.sokelf/kelf.cppneff/section-taxonomy.mdMEDIUM
dtype systemlibnrt.sokelf/load_numpy.c + kelf/kelf.cppneff/dtype-system.mdMEDIUM
kelf2kbin: JSON → KBIN loweringlibnrt.sokelf/kelf2kbin.cppneff/kelf2kbin.mdHIGH
KBIN structureslibnrt.sotdrv/kbin.c + tdrv/kbin_patch.cneff/kbin-structs.mdHIGH
Static memory planning (mem_ref)libnrt.sotdrv/mem_ref.cneff/memory-planning.mdHIGH
Compute-resource build (KBL)libnrt.sotdrv/tdrv.c + kmgr/dlr.cppneff/compute-resource-build.mdMEDIUM
Load pipeline (parse → build → stage → relocate)libnrt.sokmgr/dlr.cpp + kmgr/dlr_kelf.cppneff/load-pipeline.mdHIGH

Part VI — TPB Instruction Set: Encoding & Validation

SubsystemBinarySource TU(s)Wiki pageConf
TPB engine instruction model overviewlibnrt.sotdrv/instruction_block.c + tdrv/instr_common.cisa/overview.mdHIGH
Pseudo-instruction lowering (SP/TopSP)libnrt.sotdrv/instr_pseudo_branching.c + tdrv/instr_sunda_pseudo_range_check.cisa/pseudo-instruction-lowering.mdMEDIUM
64-byte instruction record formatlibnrt.sotdrv/instruction_block_common.cisa/instruction-record.mdHIGH
FP8 / dtype encodinglibnrt.sotdrv/instr_tpb_functions.cisa/fp8-dtype-encoding.mdMEDIUM
ISA validator architecture / entry treelibnrt.sotdrv/instruction_block_mariana.c (297 fns)isa/validator-architecture.mdHIGH
Per-arch validators (SUNDA/CAYMAN/MARIANA deltas)libnrt.sotdrv/instruction_block_{cayman,sunda,mariana}.cisa/validators-per-arch.mdHIGH

Part VII — Execution Engine

SubsystemBinarySource TU(s)Wiki pageConf
Hot inference path overviewlibnrt.sotdrv/exec.c + kmgr/kmgr_async_exec.ccexec/overview.mdHIGH
Exec manager (KMGR) facade / model DBlibnrt.sokmgr/dlr.cpp + kmgr/kmgr_metrics.cppexec/kmgr-facade.mdHIGH
XU work-queue / worker threadslibnrt.sokmgr/xu/queue.c + kmgr/xu/xu_worker.cc + kmgr/xu/tpb_xu.ccexec/xu-workers.mdHIGH
Submit path (bind → stage → doorbell)libnrt.sotdrv/hw_exec_queue.c + tdrv/exec.cexec/submit-path.mdHIGH
Completion engine (NQ harvest / error decode)libnrt.sotdrv/notification.c + tdrv/infer_error_subtype.cexec/completion-engine.mdMEDIUM
xu_queue / device_exec_request ABIlibnrt.sokmgr/xu/queue.c + kmgr/xu/comp_handle_pool.cexec/xu-queue-abi.mdMEDIUM

Part VIII — DMA & Descriptor Engine

SubsystemBinarySource TU(s)Wiki pageConf
DMA overviewlibnrt.sotdrv/dma_ring.c + tdrv/dma_memory.cdma/overview.mdHIGH
16-byte descriptor wire formatlibnrt.sotdrv/dma_desc_util.cdma/descriptor-format.mdHIGH
SDMA / CCE / TDG meta-control overlayslibnrt.sotdrv/dma_ring_act_tbl.c + tdrv/dma_ring_imcpy.cdma/meta-ctrl-overlays.mdMEDIUM
Ring / trigger / doorbell / completion cyclelibnrt.sotdrv/dma_ring.c + tdrv/dma_ring_setup.cdma/ring-cycle.mdHIGH
Virtual rings (vring) / packet builderslibnrt.sotdrv/vring.c + tdrv/sw_dma_queue.cdma/virtual-rings.mdHIGH
IOFIC interrupt model (userspace view)libnrt.sotdrv/hal_platform.cdma/iofic.mdMEDIUM

Part IX — On-Device Collectives (enc / encd / cc_op)

The collective compute engine is statically embedded in libnrt.so (enc/ = 52 CUs / 1,062 fns; tdrv/encd/ builders). The multi-node transport (libnccom.so) is Part XII.

SubsystemBinarySource TU(s)Wiki pageConf
Collective-compute architecture overviewlibnrt.soenc/enc.cc (352 fns)collectives/overview.mdHIGH
Engine core (enc_context / accessors)libnrt.soenc/enc.cccollectives/engine-core.mdHIGH
Comm context / bootstraplibnrt.soenc/neuron_nccl.cc + enc/async_sr/kv_store.cccollectives/comm-context.mdMEDIUM
libnrt ↔ libnccom boundary (nec_/nccl)libnrt.solibnccom.soenc/neuron_nccl.cc + tdrv/encd/nec_vil.ccollectives/nccl-boundary.mdHIGH
Algorithm taxonomy (ring/mesh/hier/kangaring/RDH)libnrt.soenc/enc_primitive.cc (200 fns) + enc/rdh.cccollectives/algorithm-taxonomy.mdHIGH
Mesh composer (alg_mesh_initializer)libnrt.soenc/pod_mesh_primitive.cccollectives/mesh-composer.mdMEDIUM
Ring scheduling mathlibnrt.soenc/enc_primitive.cccollectives/ring-scheduling.mdMEDIUM
Hierarchical / RDH compositionlibnrt.soenc/rdh.cc + enc/inter_rdh.cccollectives/hierarchical-rdh.mdHIGH
enc primitives (send/recv leaves)libnrt.soenc/enc_primitive.cc + enc/memory_segment.cccollectives/enc-primitives.mdHIGH
Switch-platform broadcast / barrier tableslibnrt.soenc/switch_platform/enc_switch_platform.cc + enc/barrier.cccollectives/switch-broadcast-barrier.mdMEDIUM
Topology partitioning (union-find)libnrt.soenc/replica_groups.cccollectives/topology-partition.mdMEDIUM
Bananaphone IPC / proxy driverlibnrt.soenc/bananaphone.c + enc/proxy_queue.cccollectives/proxy-driver.mdHIGH
Async send/recv (point-to-point)libnrt.soenc/async_sr/async_sr.cc + async_sr/async_sr_ofi.cccollectives/send-recv.mdHIGH
cc_op_entry on-device ISAlibnrt.sotdrv/instr_collectives.c + tdrv/encd.ccollectives/cc-op-isa.mdMEDIUM
148-byte ring channel descriptorlibnrt.sotdrv/encd.ccollectives/channel-descriptor.mdMEDIUM
encd: device-side descriptor emitterlibnrt.sotdrv/encd.c (229 fns)collectives/encd-overview.mdHIGH
encd: DMA descriptor / devmem floorlibnrt.sotdrv/encd.c + tdrv/encd/arch_device_mapping.ccollectives/encd-dma-devmem.mdMEDIUM
encd: semaphore / TopSP bring-uplibnrt.sotdrv/encd.c + tdrv/instr_collectives.ccollectives/encd-sema-topsp.mdMEDIUM
encd: per-arch ops dispatchlibnrt.sotdrv/encd/archs/{arch,cayman,mariana,sunda}.ccollectives/encd-arch-ops.mdHIGH

GOTCHA — the per-event switch-platform collective TUs (enc/switch_platform/events/* and ops/*, 31 small .cc files: 20 events + 11 ops) are template-instantiated and -O2-inlined: addr2line on their low_pcs resolves to STL header frames, so the ENC-ALG coverage cells folded them into enc.cc/enc_primitive.cc by address without naming the event TU. collectives/switch-broadcast-barrier.md and collectives/algorithm-taxonomy.md are the owning pages, but the per-event source filenames (e.g. intra_chip_broadcast_event.cc, proxy_buff_supported_local_reduce_event.cc) are a known source-map gap — see the CORRECTION below.


Part X — Collectives Firmware (libncfw)

libncfw.so has NO DWARF; TU column is structural (embedded-blob layout, recovered by binary analysis), hence LOW/MEDIUM confidence.

SubsystemBinarySource TU(s)Wiki pageConf
Firmware overviewlibncfw.so(no DWARF)firmware/overview.mdMEDIUM
Carrier librarylibncfw.so(carrier glue, no DWARF)firmware/carrier-library.mdMEDIUM
Embedded payloads (8 Xtensa blobs)libncfw.so(8 .rodata blobs)firmware/embedded-payloads.mdMEDIUM
Serializer families (per-arch CC dumpers)libncfw.so(per-arch serializers, no DWARF)firmware/serializer-families.mdLOW
NCFW sequencer (Xtensa LX disasm)libncfw.so(Xtensa firmware image)firmware/ncfw-sequencer.mdLOW
Firmware upload path (DKMS → device DRAM)libncfw.so + .kotdrv/ucode_lib.c (carrier side) + kernel fw_io.cfirmware/upload-path.mdMEDIUM

Part XI — GPSIMD / Q7 Microcode & ISA

libnrtucode_extisa.so has NO DWARF; the libnrt.so-side facade (ucode.c) does.

SubsystemBinarySource TU(s)Wiki pageConf
Q7 GPSIMD engine overviewlibnrtucode_extisa.so(no DWARF)gpsimd/overview.mdMEDIUM
Xtensa / Vision-Q7 identificationlibnrtucode_extisa.so(ISA image)gpsimd/xtensa-vision-q7.mdMEDIUM
Xtensa toolchain / TIE configlibnrtucode_extisa.so(TIE config)gpsimd/xtensa-toolchain.mdLOW
ext-ISA provider (nrtucode_* API)libnrtucode_extisa.sonrtucode_* exports (no DWARF)gpsimd/extisa-provider.mdMEDIUM
ext-ISA dispatch tables / arch-id schemelibnrtucode_extisa.so(dispatch tables)gpsimd/dispatch-tables.mdMEDIUM
Microcode loader (RELA + FLIX, UCPL)libnrt.so + ext-ISAtdrv/ucode_lib.c + ucode/ucode.cgpsimd/microcode-loader.mdMEDIUM
13 Q7 microcode blobslibnrtucode_extisa.so(13 .rodata blobs)gpsimd/q7-blobs.mdMEDIUM
IVP vector-ISA catalog (1065 ops)libnrtucode_extisa.so(IVP op tables)gpsimd/ivp-isa-catalog.mdMEDIUM
ucode.c dlopen facadelibnrt.soucode/ucode.c + nrt/nrt_ucode.cppgpsimd/ucode-facade.mdHIGH

Part XII — Multi-Node Collectives: libnccom (NCCL Fork)

The transport lib + its libnrt.so-side bridge. libnccom.so is dlopen'd; the bridge trampolines (ncclXneuronX dlsym pointers) live in libnrt.so.

SubsystemBinarySource TU(s)Wiki pageConf
NCCL-fork overview (2.31.24+nrt2.0)libnccom.so(separate pkg)nccom/overview.mdMEDIUM
Communicator init / bootstraplibnccom.so(separate pkg)nccom/comm-init.mdMEDIUM
Topology detection / graph buildlibnccom.so(separate pkg)nccom/topology.mdMEDIUM
Ring algorithm construction (host-side)libnccom.so(separate pkg)nccom/algorithm-ring.mdMEDIUM
Tree / CollNet (dead code)libnccom.so(separate pkg)nccom/algorithm-tree.mdLOW
Send/recv primitives / protocolslibnccom.so(separate pkg)nccom/send-recv-prims.mdMEDIUM
Proxy / progress engine (+ libnrt bridge)libnccom.solibnrt.soenc/proxy_queue.cc (bridge side)nccom/proxy-engine.mdMEDIUM
Tuning model (dead cost model)libnccom.so(separate pkg)nccom/tuning.mdLOW
Intra-node transport (P2P)libnccom.so(separate pkg)nccom/transport-intra.mdMEDIUM
Inter-node transport (EFA / libfabric)libnccom-net.so(aws-ofi-nccl plugin)nccom/transport-efa.mdMEDIUM
Net plugin (ncclNet_v6)libnccom-net.so(separate pkg)nccom/net-plugin.mdMEDIUM
libnrt ↔ libnccom ABIlibnrt.solibnccom.soenc/neuron_nccl.cc (37 neuron* imports / 16 nec_* exports)nccom/abi.mdHIGH

NOTE — every libnccom.so/libnccom-net.so TU cell is (separate pkg): those binaries ship in aws-neuronx-collectives, absent from the runtime-lib extract, so their internal TUs are unverified (Confidence ≤ MEDIUM). The one HIGH row is nccom/abi.md — the boundary is fully observable from libnrt.so's import/dlsym table and the 16 exported nec_* reverse-callback thunks.


Part XIII — Profiling, Trace & Telemetry

SubsystemBinarySource TU(s)Wiki pageConf
Three trace producers overviewlibnrt.sonrt/nrt_profile.cpp (94 fns)trace/overview.mdHIGH
Inspect / profile APIlibnrt.sonrt/nrt_inspect.cpp + nrt/nrt_inspect_config.cpptrace/inspect-profile-api.mdHIGH
System monitor / debug streamlibnrt.sonrt/nrt_system_monitor.cpp + nrt/nrt_debug_stream.cpp + ndebug/ndebug_stream.ctrace/system-monitor.mdHIGH
NTFF trace file format (ntff.proto)libnrt.so (vendored protobuf)ntff.pb.cc (535 fns, KaenaProfilerFormat)trace/ntff-format.mdHIGH
NTFF wire tables (TcParseTable decode)libnrt.sontff.pb.cc + neuron_trace.pb.cctrace/ntff-wire-tables.mdMEDIUM
SysTraceEventType taxonomy (46 variants)libnrt.sonrt/nrt_sys_trace.cpp + nrt/nrt_sys_trace_api.cpptrace/event-taxonomy.mdHIGH
neuron_rustime: sys_trace capture enginelibnrt.so (first-party Rust)neuron_rustime cgu (Rust std/core)trace/rust-capture.mdMEDIUM
neuron_rustime: serde_json serializerlibnrt.so (Rust)neuron_rustime + serde_json-1.0.145trace/rust-serde.mdMEDIUM
neuron_rustime: cbindgen FFI boundarylibnrt.so (Rust)nrt/nrt_sys_trace_capture.cpp (C side) + Rust cgutrace/rust-ffi.mdMEDIUM
Telemetry / metrics / error reportinglibnrt.sonrt/nrt_profile.cpp + kmgr/kmgr_metrics.cpptrace/telemetry-errors.mdMEDIUM

NOTE — the trace producers split across origins inside one binary: the C API + protobuf serializer is first-party C++/KaenaProfilerFormat (nrt/nrt_*, ntff.pb.cc), while the capture/serialize engine is the first-party AWS neuron_rustime Rust crate (NOT vendored — it has no index.crates.io path). The Rust rows are MEDIUM because the cgu-merged Rust CUs do not expose per-function source TUs the way the C++ CUs do.


Part XIV — The Neuron DataStore (NDS)

The datastore straddles two binaries: per-process slabs in the .ko, the userspace accessor lib libnds.a (== the nds/ first-party CUs, statically linked into libnrt.so).

SubsystemBinarySource TU(s)Wiki pageConf
Shared-memory counter-plane overviewlibnds.a (in libnrt.so) + .konds/neuron_ds.cdatastore/overview.mdHIGH
Kernel side (per-process slabs).kokernel neuron_ds.cdatastore/kernel-side.mdHIGH
Userspace side (libnds.a)libnds.a (in libnrt.so)nds/neuron_ds.c + nds/neuron_ds_counters.c + nds/neuron_ds_object.c + nds/neuron_ds_models.c + nds/neuron_ds_process.c + nds/neuron_ds_helpers.cdatastore/userspace-libnds.mdHIGH
NDS wire formatlibnds.a / .konds/neuron_ds_object.c + nds/neuron_ds_models.cdatastore/wire-format.mdMEDIUM

QUIRK — libnds.a is the only one of the six listed binaries that is not an independent runtime artifact: its 6 neuron_ds*.c.o members are compiled once and folded into libnrt.so, so its functions appear in libnrt.so's DWARF under nds/. It is catalogued separately because the same source is what the build links, and because the counter plane it implements is shared with the .ko's per-process slabs.


Part XV — Security & Attack Surface (White-Hat)

These pages are cross-cutting analyses, not new code; the binary/TU columns point at the subsystems they audit.

SubsystemBinarySource TU(s)Wiki pageConf
Privilege-gate model overview.koneuron_ioctl.c (audit target)security/overview.mdMEDIUM
IOCTL attack surface (14 findings).koneuron_ioctl.c + handler TUssecurity/ioctl-attack-surface.mdMEDIUM
FW-IO trust boundary.ko + libncfw.sofw_io.c (audit target)security/fw-io-trust.mdMEDIUM
Hardening recommendationsall(cross-cutting)security/hardening.mdMEDIUM

Part XVI — Appendices

SubsystemBinarySource TU(s)Wiki pageConf
This matrixall five + libnds.aall CUs / boundariesappendix/subsystem-matrix.mdHIGH
Symbol-version manifest (.gnu.version_d)libnrt.so(NRT_2.0.0 / NRT_3.0.0 nodes)appendix/symbol-versions.mdHIGH
Global / singleton address indexlibnrt.socross-TU .bss/.dataappendix/globals-index.mdHIGH
Known-bugs / anomalies catalogall(cross-cutting)appendix/known-bugs.mdMEDIUM
Phase-3 deep-dive backlogall(open items)appendix/deep-dive-backlog.mdHIGH

Dual-Organization — subsystem Part vs binary lane

The book files the same physical code under two complementary schemes; a subsystem reached via one will often have a deeper or shallower face under the other. The straddles a reader should know:

SubsystemSubsystem-Part faceBinary-lane / other faceWhy it splits
DataStorePart XIV datastore/* (deep: wire format, slabs, libnds)Part III kernel/datastore.md (map: the .ko's per-process slabs)The counter plane is shared kernel↔userspace; libnds.a is in libnrt.so, slabs are in the .ko.
On-device collectivesPart IX collectives/* (the enc/+encd/ compute engine, in libnrt.so)Part XII nccom/* (the libnccom.so transport)Compute is statically embedded in libnrt.so; transport is the dlopen'd libnccom.so. The boundary is collectives/nccl-boundary.mdnccom/abi.md.
KaenaHal / DHALPart IV runtime/hal-*.md (the libnrt.so adapter shims)Part III kernel/dhal-*.md (the .ko ndhal vtable-of-vtables)KaenaHal has no CU in libnrt.so — the real DHAL is kernel-side; userspace only carries adapter shims.
DMA enginePart VIII dma/* (userspace ring/descriptor view, libnrt.so)Part III kernel/dma-*.md, kernel/udma-*.md (the .ko op layer + UDMA core)The descriptor wire format is shared; the userspace tdrv/dma_* builds it, the kernel neuron_dma.c/udma.c executes it.
NDL ↔ IOCTLPart IV runtime/ndl.md (vendored KaenaDriverLib userspace shim, in libnrt.so)Part III kernel/ioctl-*.md (the .ko IOCTL plane it calls)ndl.c issues the IOCTLs that neuron_ioctl.c services — same package (KaenaDriverLib/KaenaDriver 2.27.4.0), two sides.
MicrocodePart XI gpsimd/microcode-loader.md, ucode-facade.md (loader, in libnrt.so)Part XI gpsimd/q7-blobs.md, ivp-isa-catalog.md (the libnrtucode_extisa.so payload)The loader ucode.c is in libnrt.so; the ext-ISA provider and blobs are the no-DWARF libnrtucode_extisa.so.
FirmwarePart X firmware/* (the libncfw.so carrier + blobs)Part X firmware/upload-path.md (the .ko fw_io.c upload side)The carrier holds the images; the .ko DMAs them to device DRAM.

CORRECTION (F-SRCMAP §4b) — the per-event switch-platform collective TUs (enc/switch_platform/events/*.cc, e.g. intra_chip_broadcast_event.cc, proxy_buff_supported_local_reduce_event.cc, intra_rank_dimr1w_event.cc, and ~13 peers) are a genuine source-map gap: they have real DWARF compile units but, being template-instantiated and -O2-inlined, no coverage cell names them — they are folded by address into enc.cc/enc_primitive.cc. Their owning pages are collectives/switch-broadcast-barrier.md and collectives/algorithm-taxonomy.md, but the source filename → page link for these specific TUs is asserted by directory membership, not by an address-grounded cite. Treat those event-TU rows (covered collectively under the Part IX switch-platform entries) as the lowest-confidence first-party mappings in this matrix.

Cross-References

  • Symbol-Version Manifest — the NRT_2.0.0 / NRT_3.0.0 .gnu.version_d nodes and the dlopen'd libnccom boundary that the collectives rows reference
  • Source-Tree Reconstruction (KaenaRuntime) — the full first-party TU inventory (203 CUs / 4,407 fns) that the source-TU column draws from
  • ELF Anatomy — the DT_NEEDED list, DWARF presence, and section table that ground the binary legend
  • Vendored-Library SBOM — the version pins for the 122 vendored CUs (Abseil, protobuf, Rust, libarchive, zlib, simdjson) that appear as (+ vendored …) in the source-TU cells