Overview and Heavy-Frame Census

Binary: extracted/aws-neuronx-runtime-lib_2.31.24.0-0b044f4ce_amd64/opt/aws/neuron/lib/libnrt.so → libnrt.so.1 → libnrt.so.2.31.24.0 · Version 2.31.24.0-0b044f4ce (.nrt_brazil_version @0xad41f0 = "2.31.24.0") · BuildID[sha1] 8bb57aba0fb2e0035f1d88e9fc4fb3e7387c102e · ELF64 LSB DYN x86-64, SONAME libnrt.so.1 · 122,956,336 bytes on disk (~117 MiB) · NOT stripped — .symtab 25,623 entries (readelf -SW: Size 0x96228 / entsize 0x18; nm --defined-only = 24,500 defined) + full DWARF v4 present. All four PT_LOAD segments are identity-mapped (p_offset == p_vaddr), so every PROGBITS section — .text, .rodata, and .data — has VMA == file offset; .bss is SHT_NOBITS and has no file content.

CORRECTION — earlier revisions of this page (and runtime/* siblings) claimed .data/.bss differ from their VMA by a 0x400000 delta. That is wrong for libnrt.so. readelf -lW libnrt.so.2.31.24.0 shows the RW LOAD segment at Offset 0xbeeaa0 == VirtAddr 0xbeeaa0 (delta zero), and readelf -SW shows .data at Address 0xc07e00 / Off 0xc07e00 — identical. The 0x400000 delta is a fact about a different binary (the libtpu / Kaena-profiler image), not libnrt. Read .data-resident globals at their VMA directly.

Part II — Binary Anatomy & Forensics / Section map · Evidence grade: byte-anchored (counts cross-checked nm/readelf/objdump vs IDA *_functions.json/*_frames.json/*_callgraph.json sidecars) · back to index

Abstract

This page is the orientation map for reading libnrt.so as an artifact. The runtime ships as a single 117 MiB shared object that statically vendors every heavy dependency it touches: objdump -p lists DT_NEEDED only for libgcc_s/libutil/librt/libpthread/libdl/libstdc++/libm/libc/ld-linux — there is no NEEDED entry for protobuf, Abseil, simdjson, zlib, or the Rust runtime, so all of those live inside this .text. Reading it is the familiar exercise of opening a large stripped-but-here-unstripped C++/Rust monolith: most of the byte mass is mechanical (vendored OSS object code and compiler-generated glue you must recognize and skip), and a minority is the first-party KaenaRuntime logic that the rest of this Part dissects.

The job of this page is to draw that line quantitatively. DWARF exposes 331 compile units: 203 first-party KaenaRuntime TUs (4,407 defined functions) built from the single source root /opt/workspace/KaenaRuntime/, plus 122 vendored/third-party TUs (4,384 functions) — Abseil, the Rust std/core/alloc crates, the KaenaProfilerFormat protobuf, Libarchive, simdjson, KaenaDriverLib, zlib. Those 8,791 DWARF subprograms-with-low_pc are the named code; IDA recovers 17,372 functions total, the balance being PLT thunks, .cold split fragments, and outlined trampolines that carry no DWARF subprogram of their own. By byte budget the heavy mass splits roughly 41.3% vendored / 58.7% first-party (3,148,991 B / 4,469,317 B over the 7,618,308-byte summed function extent from IDA functions.json — which is 96.0% of the 0x790b19 = 7,932,697-byte ELF .text section; the ~4% balance is inter-function padding and .text-resident data the function records do not span).

The page then quantifies the four "heavy" axes — byte size, stack-frame size, in-degree, fan-out — to point a reimplementer at where the work concentrates. The runtime's weight is not in its small public C API (nrt-api proper is ~0.20 MB / 341 fns); it is in ISA instruction validation (~1.42 MB / 369 fns, the largest single first-party region), collective-algorithm composition (~1.06 MB / 1,129 fns), and NEFF parsing (few but enormous per-function). Each deep region is a sibling page in this Part or in the subsystem Parts; this overview links them and does not duplicate their derivations.

For reimplementation, the contract this page establishes is:

• The provenance split — which functions are first-party KaenaRuntime (worth reimplementing), which are vendored OSS (link the real library instead), and which are compiler-generated (reproduce structurally, never byte-for-byte). All three are quantified below with real counts.
• The heavy-frame census — the byte-size, stack-frame, in-degree, and fan-out outliers, each anchored to symbol + address, so a port knows which functions are line-count sinks, which are stack-overflow hazards, and which leaves are reached by a third of the call graph.
• The reading order — a pointer table to every Part-II sub-page and to the subsystem Parts where each densest region is reimplemented in full.

NOTE — the "117 MB" and "122,956,336 bytes" figures are the same object measured two ways (117.27 MiB). The disk size is dominated by DWARF (.debug_*) and the embedded ucode/dve/hw_decode data blobs; executable code is only 7.6 MB. A reimplementer sizing the logic should reason about the .text budget, not the on-disk size.

1. Provenance split — the pie, as a table

Every function in libnrt.so falls into one of three provenance classes. The split is the single most useful fact for orienting a reimplementation: it says what to rebuild, what to link, and what to regenerate. Counts below are from the DWARF CU partition (readelf --debug-dump=info, DW_AT_name/DW_AT_comp_dir) for the function/CU columns and from the IDA *_functions.json .size aggregate for the byte budget; the two methods are reconciled in §1.3.

1.1 By DWARF compile unit (the authoritative function partition)

fn = DW_TAG_subprogram with a DW_AT_low_pc inside the CU, deduped by low_pc. This is the count of defined bodies, and it is HIGH confidence — every path is a verbatim DW_AT_name string.

CORRECTION (F-3PVER) — the source-map cell originally noted "no protobuf-runtime CU is in libnrt … Abseil is the C++ support runtime instead", reading the absence of a protobuf-runtime DWARF CU as absence of the runtime. The runtime is statically linked: nm shows 3,003 defined google::protobuf text symbols including SerialArena/ThreadSafeArena/ArenaStringPtr and VerifyVersion @0x6fb690. The DWARF gap is a -g-less build of the runtime TUs (no line tables), not missing code. Abseil is also present — both are; they are not alternatives. See Vendored-Library SBOM.

1.2 By byte budget (where the line-count actually lives)

The CU table counts functions; this table counts bytes, which is the better proxy for reimplementation effort. Attribution is by demangled name-prefix over all 17,372 IDA records (MED–LOW on the edges, since .constprop/.isra/.part thunks can mis-bucket by a few percent; the totals are HIGH from objdump -p + .size sums).

QUIRK — the public C API (nrt_*) that consumers link against is one of the smallest first-party regions (0.20 MB, 341 fns). The runtime's mass is behind that thin façade: NEFF parsing, ISA validation, and collective-algorithm composition. A reimplementer who scopes effort from the export list (145 NRT_* GLOBAL symbols — 137 NRT_2.0.0 + 8 NRT_3.0.0; see ELF Anatomy §2) will badly under-budget. The exports are thunks into a 4.5 MB first-party body.

NOTE — AL-HAL (al_*, AnnapurnaLabs/Annapurna Labs HAL) is marked FIRST-PARTY* because it ships only inside AWS Neuron but is a distinct upstream component vendored from the driver tree. The HAL functions installed into kaena_khal are owned by Part IV's HAL pages; this Part owns the struct/singleton layout in Globals Atlas.

1.3 Reconciling the two counts (DWARF 8,791 vs IDA 17,372)

The DWARF subprogram count (8,791) and the IDA function count (17,372) differ by 8,581. That gap is not error; it is the mechanical residue:

17,372  IDA-recovered functions (functions.json)
 8,791  DWARF subprograms-with-low_pc  (= 4,407 first-party + 4,384 vendored)
 ──────
 8,581  PLT stubs + .plt.sec thunks
        + .cold split fragments (e.g. nrt_config_parse_init_config.cold @0x3f18e, 0xf82)
        + compiler-outlined .constprop/.isra/.part clones
        + static-data trampolines  — none carry their own DWARF subprogram

A reimplementer should treat the DWARF set as the logical function inventory (what to reproduce) and the IDA delta as toolchain artifacts (regenerated automatically by any optimizing compiler). The .cold split in particular means a single source function appears as two IDA records: the hot half keeps the nm t symbol; the cold half is a separate chunk. Byte sizes in §3 are the full extent including .cold.

2. First-party subsystem tree

The 4,407 first-party functions are built from one source root, /opt/workspace/KaenaRuntime/, in 203 TUs. The fn-weighted tree below is the second-level map a reader uses to decide which subsystem Part to open; each leaf is reimplemented in the named Part, not here.

KaenaRuntime/                                              CUs    fns   → owning Part
 ├─ tdrv/    device driver + instruction-block builders     84   2413   Part IV / Part VI
 │    ├─ cayman/ mariana/ sunda/   (per-silicon arch+isa+dve)
 │    └─ encd/ (+ encd/archs/)     execution-engine descriptor compiler  → Part IX
 ├─ enc/     host collectives engine                         52   1062   Part IX
 │    ├─ async_sr/                 async P2P sendrecv (OFI/rendezvous)
 │    └─ switch_platform/          switch-fabric collective composer
 ├─ nrt/     public C API + config/profile/sys_trace         27    377   Part IV / Part XIII
 ├─ kelf/    KELF/KBIN/NEFF container parse + numpy/HLO stats  5    318   Part V
 ├─ kmgr/    model/kernel manager (dlr, xu worker queue)      10    117   Part VII
 ├─ nds/     Neuron DataStore (folded from libnds.a)           6     36   Part XIV
 ├─ ucode/   microcode loader glue                             1     26   Part XI
 ├─ utils/   md5/sha256/instance/time helpers                  5     23   —
 ├─ nlog/    logging + backtrace + symbol resolution           3     15   Part IV
 ├─ ndebug/  ndebug_stream                                     1     10   Part XIII
 ├─ dve_config/  DVE config + per-arch                         4      7   Part IV
 ├─ dx/      dx/ring.c (notification ring)                     1      2   Part VIII
 ├─ hw_decode/  hw_decode table glue                           1      1   Part VI
 └─ build/private/.../*_bins.c  GENERATED blob stubs           3      0   (data, no code)
                                                            ────  ─────
                                                             203   4407

The densest single TUs (by DWARF fn-count) are the per-arch instruction-block builders and the collectives core, which is exactly where §3's byte-size outliers land:

GOTCHA — the per-arch fan-out is triplicated, not parameterized. cayman/mariana/sunda each get their own instruction_block_*.c (248/297/238 fns), tdrv_arch_*.c, encd/archs/*.c, and dve_dynamic_config_*.c. A reimplementation that factors the three arches into one templated builder will not match the binary's call graph or its per-arch tdrv_arch_ops (488 B = 61 pointer slots, nm: tdrv_arch_ops @0xc97180 → tdrv_arch_ops_initialized @0xc97368, span 0x1e8) and kaena_khal (~200-slot @0xcaeb80) dispatch structs — those are installed by separate register_funcs_v{2,3,4} paths. See Dispatch-Table Taxonomy.

3. Heavy-frame census

Four independent "heavy" axes matter to a port, and all four are present in the data. Each is anchored to symbol + address; byte sizes are HIGH (spot-checked nm: 0x8a1d0=0x5a30=23,088 ✓, 0x2f2350=0x4620=17,952 ✓, 0x45fd50=0x73e=1,854 ✓).

3.1 Axis A — byte-size outliers (line-count sinks)

These are where reimplementation line-count concentrates. The top of the distribution is one config parser, then a wall of collectives composers/schedulers and ISA validators. SIZE is full extent including .cold; BLK/INSN from functions.json; NCLR = distinct callers.

Across the top-50 by size: 28 are collectives (enc_*/rdh_*/inter_rdh_*), 6 NEFF-parse, 4 ISA-validate, 2 nrt-config, and 6 vendored (protobuf ×2, abseil ×2, simdjson ×4). The collectives composer/scheduler family is the dominant heavy region.

NOTE — simdjson appears twice in the heavy list — simdjson::westmere::dom_parser_implementation::stage2 (0x4fa790, 10,451 B) and simdjson::haswell::…stage2 (0x4f5820, 10,171 B). Both are present because simdjson 0.9.0 ships per-microarchitecture copies and selects at runtime by CPU dispatch. A reimplementation that links one simdjson build will not reproduce the dual copy; that is correct, not a discrepancy.

3.2 Axis B — stack-frame outliers (clean-room hazards)

These are the functions a naïve port will get wrong at runtime, not at compile time: they reserve enormous stack frames that overflow a default thread stack. frames.json vars[] enumerate the exact large locals.

GOTCHA — enc_find_cc_context_by_signature (0x12a0c0) reserves a 6.75 MB stack frame for one enc_ctx struct local. On a default 8 MB pthread stack this is one nested call from overflow; on the smaller stacks used by worker pools it overflows immediately. Any reimplementation must heap-allocate enc_ctx (and the act_local_storage_tbls_setup staging buffers) rather than reproduce the on-stack layout. The binary gets away with it because these run on the main thread early in bring-up; a port that calls them from a kmgr/xu worker will crash. Census: 8 functions exceed a 64 KB frame; 341 exceed 8 KB. Full list → TDRV Bring-Up.

3.3 Axis C — in-degree outliers (the leaves to port first)

The most-called functions are the leaves a reimplementation must stub or port first, because a third of the call graph reaches them. Runtime intrinsics are filtered out to surface the first-party hot leaves. Two metrics: distinct-caller count (NCLR) and total call-site count.

QUIRK — nlog_write and the al_* HAL primitives are tiny (119–163 B) but pervasive — they are the first leaves any port must stub. The neuron_isa_tpb_*_dbg_bool/merge_bool leaves come in _0/_1/base triplets (three arch variants of the same 119/392-B validation primitive); the 369-function ISA-validate tree is built entirely from these. Reference intrinsic toppers (filtered above): __assert_fail 2,308 distinct callers, _Unwind_Resume 1,751, operator delete 1,382, operator new 1,185, memcpy 738.

3.4 Axis D — fan-out outliers (the deepest orchestration chains)

The widest functions — most distinct callees — are the collective schedulers and the config/NEFF parsers. These are the deepest orchestration roots; a port must reproduce their call structure, not just their bodies.

Three named hot-path roots tie the axes together:

NEFF LOAD       nrt_load → parse_neff_json (0xe1d10) → {parse_one_dma_block 0x4bc620,
                  parse_one_engine_instr 0x4b7e30, parse_one_variable 0x4b36b0}
                  → kelf_load_from_neff (0x4c0870) → encd_load_executable (0x246ae0)
                  ↳ bottoms out in simdjson westmere/haswell stage2 (0x4fa790 / 0x4f5820)

COLLECTIVES     enc_post_operation (0x11f790) → enc_init_comm (0x135d60)
                  → init_hierarchical_groups (0x12c820) → alg_mesh_build_full_mesh (0x125fb0)
                  → enc_*_primitive::__compose_* (28 composers)
                  → rdh_scheduler::schedule_* (deepest fan-out, 538–583 callees)

ISA VALIDATE    is_valid_neuron_instruction (0x2f2350) / is_valid_neuron_isa_instruction (0x3a60b0)
                  / ib_is_valid_engine_instruction_v4 (0x43f810)
                  → ~50 dbg_is_valid_<op>_{,_0,_1} per-opcode validators (31–41 KB frame each)
                  → neuron_isa_tpb_*_dbg_bool/merge_bool leaves (§3.3)

DEVICE BRING-UP tdrv_init (0x26a310) — 244 callees; installs tdrv_arch_ops + kaena_khal

4. Mechanical vs first-party-interesting

The governing distinction for reading this binary: most bytes are mechanical and should be recognized-and-skipped; the minority that is first-party KaenaRuntime is what the rest of this book reimplements. Three classes of mechanical mass:

The first-party-interesting mass — the 4.47 MB / 8,873 fns worth deep pages — is the ISA-validation DSL, the collective composers/schedulers, the NEFF parse pipeline, the tdrv device layer, the per-arch instruction-block builders, the nrt_config parser, and the neuron_rustime sys_trace bridge. The RTTI surface confirms the partition independently: of 201 vtables, 97 are google::protobuf, 16 Abseil, 13 std, 11 simdjson (all vendored), while only 40 ntff:: (AWS-authored schema) and 24 first-party global-namespace classes (enc_*/mem_ref*/dma_desc*/alg_mesh_*) carry domain logic — and there is no multiple/virtual inheritance anywhere (244 _ZTI = 53 __class_type_info root + 191 __si_class_type_info, 0 __vmi_class_type_info). See RTTI Class Hierarchy.

NOTE — the ntff:: 40-message set sits on the mechanical/interesting boundary: the accessors are generated (mechanical, regenerate), but the schema (40 message classes, the .ntff trace wire format) is AWS-authored and reimplementation-relevant. Treat the .pb.cc bodies as protoc output and the message field layout as a first-party artifact — covered in NTFF Trace Format.

5. Reading order — the Part-II sub-pages

This overview is the index for Part II. Each sibling page takes one forensic axis to byte-level depth; open them in roughly this order.

Related Components

Cross-References

• ELF Anatomy — section/segment table, DWARF presence, and proof that all PT_LOAD segments are identity-mapped (.data is VMA==offset — no 0x400000 delta)
• Vendored-Library SBOM — version pins for the 41.3% vendored byte mass; protobuf/Rust conflict resolutions
• Static-Init Pipeline — the 77-ctor .init_array order and lazy-vs-eager global init
• Globals and Singletons Atlas — the first-party singleton control plane (kaena_khal, nrt_config, …)
• Dispatch-Table Taxonomy — the per-arch vtable pair behind the triplicated cayman/mariana/sunda code
• C++ Class Hierarchy and RTTI — the 24 first-party + 40 ntff:: classes vs the 137 vendored vtables
• String Domain — the KaenaRuntime __FILE__ provenance markers that ground the source map
• CRT / PLT / Loader Surface — DT_NEEDED, PLT/GOT, and the libncfw/libnrtucode dlopen edges
• Part IV — Runtime Core — where the first-party-interesting tdrv/nrt/HAL mass is reimplemented
• Part VI — TPB ISA — where the largest single first-party region (ISA validation) is reimplemented
• Part IX — Collectives — where the heaviest algorithmic mass (composers/schedulers) is reimplemented
• back to index