The neff_header POD, the In-Memory BOM, and the NeffPackager Writer

All symbols and addresses on this page apply to neuronx_cc 2.24.5133.0+58f8de22 (cp310 wheel). The writer lives in libwalrus.so (.text/.rodata base 0x62d660/0x1c72000, VA == file offset). The NEFF emission path is byte-identical in the cp311/cp312 wheels, but their addresses shift by a small per-wheel delta (e.g. addToBom is 0x153fb80 here, 0x153fae0 in cp312); treat every address as version-pinned. All claims were re-verified against the cp312 libwalrus.so via nm -DC, strings, and grep -aob; the in-function byte offsets come from the cp310 IDA decompile. See versions.

Abstract

A NEFF is not a binary file with a header struct at offset 0. It is a libarchive PAX tar stream (gzip-wrapped by default; the container itself is the subject of Part 12.1) whose "sections" are tar members — named files copied verbatim from a per-core build directory. The struct that everyone calls "the NEFF header" — neuronxcc::backend::neff_header, the flat POD carrying the format version (2), a 0x400-byte region size, an RFC-4122-v4 UUID, the output filename, a per-engine present bitmap, and the 64-bit feature mask — is built on the stack in-process by NeffFileWriter::initializeNeffHeader (0x1540a00) and its authoritative on-disk copy is serialized inside info.json (itself one tar member). There is no leading on-disk header record to parse. This is the single most important fact on the page, and the one a reimplementer is most likely to get wrong.

The second hazard is the word "BOM." It denotes three unrelated objects in this codebase, and prior analysis conflated them. (1) The NEFF BOM — Bill-Of-Materials — is the writer's in-memory file manifest: a std::map<boost::filesystem::path, std::string> at NeffFileWriter+0x90 mapping each artifact's on-disk path to its in-tar member name, populated one entry at a time by addToBom (0x153fb80) and drained in key order by writeArchiveFile (0x153e030). It has no byte marker and is never serialized as a manifest record — the tar member list is the manifest. (2) The UTF-8 BOM (0xEF 0xBB 0xBF) is the byte-order-mark the nlohmann JSON lexer tolerates at the head of any JSON sidecar (skip_bom, 0x853a50); it has nothing to do with the NEFF container. (3) The source-named object bom is bir::ModuleArtifactInfo, the per-module record that owns the engine→file DenseMaps. The page pins each sense separately.

For reimplementation, the contract is:

• The neff_header POD field layout — every offset, the override (file.neff) fast path vs. the info.json-driven normal path, and the proof that the header lives inside info.json rather than as a leading record.
• The BOM std::map<path,member> entry format, the addToBom upsert algorithm, and the IR-signature side-set that decides which members feed the whole-archive MD5.
• The writeArchiveFile libarchive PAX write loop — member ordering, per-member stat/fopen/fread, and the MD5 "IR signature."

The neff_header POD

Purpose

neff_header is the metadata block describing one compiled NEFF: which format version it is, which of the five TPB engines carry instructions, a unique build identity (UUID), the output filename, and the 64-bit feature bitmask the runtime uses to gate loader behavior. It is the closest thing a NEFF has to an "ELF header" — but unlike an ELF header it is never written as a leading binary record. It is materialized on the stack, then its fields are emitted as JSON keys into info.json.

QUIRK — there is no on-disk header struct. A reimplementer who seeks e_ident/magic at offset 0 of a .neff finds either the gzip magic 0x1F 0x8B (default) or the tar ustar magic at 0x101 of the first 512-byte member header — never neff_header. The POD is built by initializeNeffHeader purely so writeArchiveFile can carry it as metadata and so the packager can serialize its fields into the info.json member. To read a NEFF's "header," you extract info.json and parse its JSON. This corrects the long-standing "NEFF ELF container" premise: nm -DC libwalrus.so shows the complete libarchive tar API including archive_write_set_format_pax and zero ELF-writer symbols (ELFObjectWriter/elf_begin/elf_newscn/Elf64_Ehdr/writeELF = no hits).

Field Layout

The POD is built on the stack in initializeNeffHeader (0x1540a00). The first two quadwords are a single 16-byte rodata constant (0x1DD7900, read directly: 02 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00); the rest are written field-by-field from info.json (normal path) or hardcoded (override path).

NOTE — two different "versions" live in one NEFF. neff_header.version == 2 is the binary container-format version (the rodata OWORD). It is not the same as the info.json version major/minor (a 1.x pair, default 1.1 in the override path), nor the def.json schema string "0.6" / neff.json "0.5". A reimplementer who reads info.json["version"] gets the 1.x document version, not the 2. The 2 is structural and lives in neff_header+0.

The UUID — ChaCha20 CSPRNG

The 16-byte UUID at +204 is a standard RFC-4122 version-4 (random) UUID, but the randomness source is a ChaCha20-12 stream cipher, not /dev/urandom or boost::random. The rodata constant immediately after the version OWORD — 0x1DD7910, the ASCII "expand 32-byte k" (xxd-confirmed, the canonical ChaCha/Salsa sigma constant) — is the cipher's block-init string. After 16 random bytes are drawn, the version and variant nibbles are forced in the usual way:

// initializeNeffHeader @0x1540a00 — UUID field (header+204), normal path
chacha20_fill(uuid, 16);              // ChaCha20-12 keystream; sigma "expand 32-byte k" @0x1DD7910
uuid[6] = (uuid[6] & 0x0F) | 0x40;    // RFC-4122 version 4 (random) nibble
uuid[8] = (uuid[8] & 0x3F) | 0x80;    // RFC-4122 variant (10xxxxxx)

NOTE — "version 4" here is the UUID version (RFC-4122 random), an entirely separate 4 from the neff_header.version == 2 container version. Both numbers are real and both matter; do not collapse them.

Override vs. Normal Path

initializeNeffHeader branches on NeffFileWriter+0x108 (offset 264), the internalOverride bool set when the writer is in the single-output file.neff fast path:

function initializeNeffHeader(neff_header &h):      // 0x1540a00
    h.qword[0] = 2;                                  // rodata OWORD @0x1DD7900 (version)
    h.qword[1] = 0x400;                              // region size
    h.qword[16] = 0;
    h.qword[24] = *(this + 0xC8);                    // section/flag count (=1)
    fill_uuid_v4(&h.uuid[204]);                       // ChaCha20-12, sigma @0x1DD7910

    if (*(this + 0x108)) {                            // internalOverride (writer+264)
        strcpy(&h.filename[220], "file.neff");        // hardcoded name (string @rodata)
        h.version_major[168] = 1;
        h.version_minor[476] = 1;
        memset(&h.engine_present[480], 0, 64);        // zero the engine bitmap
        // skips info.json entirely
    } else {
        path ij = findInfoJson();                     // 0x153ca20 -> <baseDir>/info.json
        json j  = loadJsonFile(ij);
        h.version_major[168] = j["version"].major;    // info.json uint
        h.version_minor[476] = j["version"].minor;
        copy_filename(&h.filename[220], j /* name */);
        copy_engine_bitmap(&h.engine_present[480], j);// per-engine bool array
    }
    h.feature_flags[192] = compute_features();        // writeNEFFFeatures 0x15294b0
    // h.cc_field[544], h.arch_mode[552] from writer+192 / writer+88

GOTCHA — the override path zeroes the engine bitmap and skips info.json. In the file.neff single-output fast path, the engine-present array at +480 is memset to zero and info.json is never read. A reimplementer who assumes the bitmap always reflects the real engine set will be wrong for override-mode NEFFs. The bitmap is only meaningful on the normal path, where it is copied from info.json's per-engine bool array.

The In-Memory BOM Manifest

Purpose

The BOM (Bill-Of-Materials) is the writer's manifest of every file that will become a tar member. It is the NEFF's "section index" — except it is an in-process std::map, never shipped in the NEFF. What the runtime sees is the result: the tar member list, whose names are exactly the BOM map's values, in BOM key order (path-sorted, deterministic).

QUIRK — the BOM is a std::map, not a serialized table. It has no byte marker, no per-entry offset/size, and no per-entry checksum. Offsets and sizes come from stat() at write time; integrity is a single whole-archive MD5 over a subset of members (below), not a per-entry hash. Anyone hunting for a 0xEF 0xBB 0xBF "BOM marker" is hunting a phantom — that byte sequence is the nlohmann JSON UTF-8 byte-order-mark, a property of the JSON sidecars' encoding, checked by skip_bom (0x853a50), not the NEFF manifest.

The Three "BOM" Senses

GOTCHA — 0xEF 0xBB 0xBF is not a NEFF manifest marker. The name collision makes it tempting to read those bytes as the BOM's on-disk marker. They are the plain UTF-8 byte-order-mark that the nlohmann lexer tolerates at the head of any JSON sidecar; they are never stored in the BOM and are not per-entry. The only code in libwalrus touching 0xEF/0xBB/0xBF is skip_bom (string "invalid BOM; must be 0xEF 0xBB 0xBF if given", present twice, once per lexer instantiation). The NEFF BOM in sense 1 has no byte marker at all.

Entry Format

A BOM entry is a std::_Rb_tree node in the NeffFileWriter+0x90 map:

struct bom_entry {            // std::_Rb_tree node in map @writer+0x90
    boost::filesystem::path on_disk_path;   // KEY   — node+0x20; post-computeOutputPath
    std::string             tar_member_name;// VALUE  — node+0x40; the in-tar member name
};                             // NO offset/size/hash/type field — those are derived at write time

There is no offset, size, checksum, or type field. File typing is expressed three other ways (member-name convention; the def.json var['type'] tag; and a boolean MD5-sig membership), none of which lives in the BOM node.

Algorithm — addToBom

function addToBom(path file, optional<string> entryNameOverride):  // 0x153fb80
    out = computeOutputPath(file);            // 0x153c7a0 — the MAP KEY
                                              //   rehomes per-module files under nc<core>/
                                              //   if file in the per-module dir map (writer+0x68),
                                              //   else joins to the writer base dir
    entry = entryNameOverride.has_value()     // disasm: cmp byte [a3+0x20],0 @0x153fbcb
              ? *entryNameOverride
              : out;                          // else the computeOutputPath result -> the MAP VALUE

    // --- IR-signature classification (writer+0xD8, a std::set<string>) ---
    ext = file.extension();                   // 0x175D1C0 = filename() then rfind('.') = path::extension()
    if (ext.compare(".npy") == 0)             // @0x153fc6f; ".npy" @rodata 0x1c8b929 — every *.npy matches
        irsig_set.insert(entry);              // extension equals ".npy"
    if (ext.find(".dbg", 0, 4) != npos)       // @0x153fc9b; ".dbg" @rodata 0x1c86c5b (n=4,pos=0)
        irsig_set.insert(entry);

    // --- BOM map UPSERT (last-write-wins on the path key) ---
    node = rb_tree_insert_or_locate(map@+0x90, file);   // sub_153FA40; path-compares walk tree
    node->value._M_assign(entry);             // @0x153fd53 -> node+0x40; re-adding overwrites

GOTCHA — last-write-wins on the path key. Re-adding the same on-disk path overwrites its member name. The .dbg debug-info files are registered twice — once by writeDefJson under "<eng>_dbg"/"<eng>_asm_dbg", and again by writePackageFile (0x15200e0:551) under the literal override "debug_info". A .dbg file's final tar member name is whichever registration ran last. Either way its basename contains .dbg, so it enters the MD5 IR-sig set regardless.

Which members are MD5-signed

The addToBom predicate admits a member into the IR-signature set iff its file extension equals ".npy" or contains ".dbg". Adding the "info.json" pre-seed the NeffFileWriter constructor contributes, the signed set is {info.json} ∪ {*.dbg debug-info} ∪ {*.npy const/weight}. Because the test is on the extension, every *.npy weight is signed. The per-engine .bin instruction streams are not signed (extension .bin).

Anchors: the predicate sits at 0x153fc6f/0x153fc9b, testing the extension string that sub_175D1C0 (0x153fc28) returns — filename() (sub_175B780, rfind '/') then rfind('.') (0x175d252: mov esi,0x2e) = path::extension() — against targets 0x1c8b929=".npy" and 0x1c86c5b=".dbg", both byte-resolved. The info.json seed is well-corroborated but [INFERRED]: 0x1543eb0 is the NeffFileWriter constructor and it does load "info.json" at 0x15440cf (via aKernelDebugInfoJson+0Dh), but the entry itself is the ctor's __cxa_demangle→strlen→strstr preamble, not the insert into the IR-sig set, and the path to _M_get_insert_hint_unique_pos was not single-stepped.

GOTCHA — compare(".npy")==0 runs on the EXTENSION, so every *.npy weight is signed. The compared string is sub_175D1C0's return — file.filename() then rfind('.'), i.e. path::extension() — not the whole basename. A real weight mlp_0_bias.npy has extension() == ".npy", so compare() returns 0 and it is signed. Reading compare(".npy")==0 as "the basename must literally be .npy" is the trap: compare operates on the extension substring, which equals .npy for the whole weight family. The .npy branch is live, and the .npy literal it uses is itself the .npy tail of a weight-name string in .rodata.

The NeffPackager Writer Path

Entry Point

NeffPackager::run(vector<unique_ptr<Module>>&)        0x15307e0  — pass "neff_packager" (last)
  └─ writePackageFile(NeffFileWriter&, modules)        0x15200e0  — BOM assembler
       ├─ writeNeffJson(...)        0x152c740  -> neff.json  (v0.5 subgraph table)
       ├─ writeDefJson(...)         0x152a0e0  -> def.json   (v0.6 per-core, + per-engine addToBom)
       ├─ writeNEFFFeatures(...)    0x15294b0  -> neff_header+192 feature mask
       ├─ kelf- prefix scan         0x15200e0:31  — collect kelf-<N>.json into BOM
       └─ debug_info dual-register  0x15200e0:551 — addToBom .dbg under "debug_info"
  └─ NeffFileWriter::writeFile(outPath, uncompressed)  0x1541040  — top-level driver
       ├─ initializeNeffHeader(neff_header&)            0x1540a00  — POD build
       ├─ writeArchiveFile(&hdr, md5, compress)         0x153e030  — libarchive PAX write
       └─ MD5::final -> hex -> log "IR signature (MD5): <hex>" -> dump BOM

Per-Engine Registration — writeDefJson

writeDefJson (0x152a0e0) walks the bir::ModuleArtifactInfo (sense-3 bom) DenseMaps — getEngInstrFile (instruction files) and getEngDMADescFile (DMA-descriptor files), bucket stride 40 bytes — and for each engine emits a def.json token and registers the file in the BOM. The engine name has two spellings at two layers, the single most error-prone point:

• On-disk .bin/.json basenames use bir::EngineInfo2string (libBIR) = TitleCase: PE, Pool, Activation, SP, DVE. Confirmed directly by the shipped consumer analyze_neff_artifacts.py line 74: for engine in ['Pool','SP','DVE','PE','Activation'].
• def.json "definition" key tokens use the NEFF-local formatter sub_15248C0 (neff_packager.cpp:49) = lowercase, a byte-confirmed strcpy switch: 1→"pool" 2→"act" 3→"pe" 4→"dma" 5→"dve" 6→"sp" (default 0/7 throws NeuronAssertion 1222).

// writeDefJson instr loop @0x152a0e0:275-351 — stride 40 over getEngInstrFile DenseMap
for (eng, basename) in module.bom.getEngInstrFile():     // TitleCase basename, e.g. "PE.bin"
    token = sub_15248C0(eng.type) + "_instr";            // lowercase, e.g. "pe_instr"  @296-299
    if (*((int*)module + 43) <= 49)                       // arch-level gate @284
        defjson["definition"][tok]["_instr"] = basename;  // @314
    addToBom(artifactDir / basename, /*override=*/token); // @340  -> path "../PE.bin" => member "pe_instr"

// dma-desc loop @392-549: addToBom(artifactDir/<dmaBasename>, override=<eng token>)   @544
// dbg loop      @682-727: token + "_dbg",      addToBom @727
// asm-dbg loop  @794-839: token + "_asm_dbg",  addToBom @839

So def.json["definition"]["pe"]["_instr"] == "PE.bin"; the BOM then maps the on-disk .../PE.bin path → the in-tar member name "pe_instr". The five .bin-bearing engines are {PE,Pool,Activation,SP,DVE}; DMA (token 4) has no .bin — DMA descriptors fold into the issuing compute engine's .bin stream, and the per-engine .json is the descriptor-table sidecar. See .bin emission for the per-engine stream contents.

Algorithm — writeArchiveFile

function writeArchiveFile(void *hdr, md5 &sig, bool compress):   // 0x153e030
    a = archive_write_new();
    archive_write_set_format_pax(a);                  // POSIX-extended (PAX) tar
    if (compress) {                                   // compress = uncompressed_flag ^ 1 (gzip default ON)
        archive_write_add_filter_gzip(a);
        archive_write_set_filter_option(a, ...) x2;   // gzip level/options (not decoded, Gap G3)
    }
    archive_write_open_filename(a, outFile);

    for (path, member) in BOM map@+0x90:               // KEY ORDER (path-sorted, deterministic)
        if (stat(path, &st) != 0)                      // else NeuronAssertion "statResult==0" (cpp:310)
            fail();
        ae = archive_entry_new();
        archive_entry_copy_stat(ae, &st);
        archive_entry_set_pathname(ae, member);        // member name = the BOM VALUE
        archive_entry_set_uname/_gname/_mtime/_ctime/_atime(ae, ...);
        fp = fopen(path, "rb");                        // else NeuronAssertion "fp" (cpp:322) +
                                                       //   log "writeKelp missing file <path>"
        archive_write_header(a, ae);
        while ((n = fread(buf, 1, 0x2000, fp))) {       // 8 KiB streaming chunks
            if (irsig_set@+0xD8 contains member)        // _Rb_tree-search the sig set on member name
                md5_update(&sig, buf, n);               // log "Adding <n> bytes of <file> to the IR Signature."
            archive_write_data(a, buf, n);
        }
        fclose(fp); archive_entry_free(ae);
    archive_write_close(a); archive_write_free(a);

writeFile (0x1541040) wraps this: guard empty outPath → assert "foundActJson" (cpp:160) → log "Neff will be written to: <outPath>" → initializeNeffHeader → writeArchiveFile → MD5::final → hex → log "IR signature: <hex> for neff artifacts" / "IR signature (MD5): <hex>" → dump the BOM (between literals "The NeffPacakger BOM:" … "Entry <path> written to <entry>" … "End of NeffPacakger BOM" — the Pacakger typo is in the binary, confirmed at file offset 29912342).

NOTE — the identity hash is MD5 over a subset. The "IR signature" is boost::uuids::detail::md5 (16-byte digest), not SHA-1, and it covers only the IR-sig-set members (info.json + every *.dbg + every *.npy), not the whole archive. The MD5 is computed during the tar write (per-chunk md5_update), so it hashes the raw file bytes, not the compressed tar.

The Member Roster

A *.neff is one PAX tar of these members (one on-disk file each), emitted in BOM key order. "Sig" = in the MD5 IR-signature set; "Per" = top-level (T) or per-core (C, under nc<core>/sg<subgraph>/sgLnk/).

NOTE — cpu.so is a real ELF — but it is a custom-op host kernel carried verbatim as a tar member, unrelated to the (non-existent) NEFF ELF header. The member-name literals above (def.json, neff.json, tensor_map.json, ucode_lib.json, kelf-, debug_info_backend, debug_info_asm, _asm_dbg, cpu.so, cpu.params) were all re-verified present via grep -aob in the cp312 libwalrus.so; the engine .bin/.json basenames are confirmed by analyze_neff_artifacts.py line 74.

Evidence Anchors and Limits

The page's structural claims trace to the cp312 libwalrus.so:

Limits. The cp312 .so exports the full writer/packager symbol roster, and every rodata string cited here is present — file.neff, the NeffPacakger BOM typo, the IR-signature logs, the engine tokens, the member-name literals, the ChaCha sigma. What cannot be re-walked from the shipped .so are the in-function byte offsets inside initializeNeffHeader (+168, +480, +204, and their neighbours) and the addToBom IR-sig predicate addresses: that binary carries no local symbols, so those offsets come from the cp310 IDA decompile. The cp312 addresses shift by a small delta (addToBom 0x153fae0 versus cp310 0x153fb80), so the relative layout is consistent across wheels while the absolute cp310 addresses are decompile-derived rather than re-walked byte for byte.

Related Components

Cross-References

• Per-Engine .bin Emission — the instruction streams writeDefJson/addToBom register and the writer packs as <eng>_instr members.
• Part 12.1 — The NEFF Container: the PAX tar/gzip container this writer emits; the byte-level container layout and the no-ELF/no-magic proof.
• Part 12.5 — NEFF Feature Flags: the 64-bit mask at neff_header+192 built by writeNEFFFeatures, bit by bit.