ELF Writer (elfw)

The ELF writer -- internally called elfw -- is nvlink's central data structure for building device ELF binaries. Every cubin the linker produces is constructed inside a 672-byte elfw object that contains the raw ELF header bytes, section and symbol management tables, string tables, program header lists, architecture state, and a battery of boolean flags derived from the merge-flags bitmask. The constructor sub_4438F0 (14,821 bytes at 0x4438F0) allocates this object, initializes the ELF header in place, creates the mandatory sections (.shstrtab, .strtab, .symtab, .symtab_shndx), and wires up the internal data structures. When it is time to emit the final binary, a separate 40-byte polymorphic writer context routes all bytes through one of five backends -- callback, no-op, growable vector, fwrite to FILE*, or memcpy to a pre-allocated buffer.

This page documents the 672-byte elfw struct layout and the 40-byte writer context at reimplementation depth. For the serialization logic that walks the elfw and emits bytes through the writer, see ELF Serialization. For the ELF format semantics (header fields, section types, program headers), see Device ELF Format.

Key Facts

The 672-Byte elfw Struct

The elfw object begins with raw ELF header bytes at offset 0 (the first 52 or 64 bytes are the Elf32_Ehdr / Elf64_Ehdr), followed by metadata fields, boolean flags, and pointers to sub-structures. The constructor zeroes the entire 672 bytes via memset, then populates fields based on its ten parameters.

Constructor Parameters

elfw_t* elfw_create(
    uint16_t  elf_type,        // a1: ET_REL(1), ET_EXEC(2), 0xFF00 (Mercury relocatable)
    bool      is_64bit,        // a2: 0 = ELF32, nonzero = ELF64
    uint8_t   abi_version,     // a3: EI_ABIVERSION value
    uint8_t   sm_major,        // a4: SM major version (e.g., 89 for Ada)
    uint8_t   sm_minor,        // a5: SM minor version (e.g., 0x45 = 69)
    bool      debug_flag,      // a6: generate debug sections
    int       api_version,     // a7: CUDA API version
    bool      verbose_flag,    // a8: verbose output
    uint32_t  merge_flags,     // a9: bitmask controlling many behaviors
    bool      mercury_flag     // a10: Mercury-mode indicator
);

The merge_flags bitmask (a9) is the primary behavioral control. Individual bits are unpacked into boolean fields throughout the struct. When bit 0x400 is set, the constructor creates a dedicated "elfw memory space" arena (via sub_432020) and stores it at offsets +608/+616; otherwise, the global arena is used.

Field Layout

The layout below is derived from the decompiled constructor (sub_4438F0), destructor (sub_4475B0), serializer (sub_45BF00), and debug dump function (sub_4478F0). Offsets are in bytes; qword[N] notation references the decompiled v17[N] 8-byte array indexing.

ELF Header Region (offsets 0--63)

The first bytes overlay the standard ELF header. The constructor writes the magic number and identification bytes directly:

+0    uint32_t  e_ident[0..3]      = 0x464C457F  (ELF magic: 7F 45 4C 46)
+4    uint8_t   EI_CLASS           = (is_64bit != 0) + 1  (1=ELF32, 2=ELF64)
+5    uint8_t   EI_DATA            = 1  (ELFDATA2LSB, little-endian)
+6    uint8_t   EI_VERSION         = 1  (EV_CURRENT)
+7    uint8_t   EI_OSABI           = 0x41 (device 64-bit) or 0x33 (device 32-bit)
+8    uint8_t   EI_ABIVERSION      = abi_version parameter

The remaining header fields (e_type, e_machine, e_version, e_entry, e_phoff, e_shoff, e_flags, e_ehsize, e_phentsize, e_phnum, e_shentsize, e_shnum, e_shstrndx) follow the standard ELF32/ELF64 layout. The constructor sets e_machine = 190 (EM_CUDA), e_type from the elf_type parameter, and packs sm_major/sm_minor into e_flags using encoding that depends on the OSABI.

For the full header encoding, see Device ELF Format -- ELF Identification.

Metadata and Flags (offsets 64--103)

Dynamic Array Regions (offsets 108--172)

The constructor initializes two dynamic arrays at offsets +108 and +140 using sub_43E490:

These are only allocated when is_device_elf is true (OSABI 0x41 path). Each is a 24-byte NVIDIA note header containing the padded name string "NVIDIA Corp" and a 12-byte note descriptor. They are not dynamic arrays with capacities.

String Table Pointers (offsets 216--228)

Hash Tables (offsets 288--303)

Both are created via sub_4489C0 with hash/compare functions sub_44E000 / sub_44E180 and an initial capacity of 512 buckets. The symbol name hash at +288 is read by sub_440BE0 (add-symbol-with-data) for symbol name lookup; the section name hash at +296 is read by sub_441AC0 (add-reloc-section) for section name lookup. They provide O(1) lookup during the merge phase.

Section/Symbol Counters and Pointers (offsets 304--376)

Three sorted arrays at +344, +352, and +360 are created via sub_464AE0 with element sizes of 64, 64, and 64 bytes respectively. The constructor creates a 104-byte null section record (section index 0, SHN_UNDEF) and appends it to the section array at +360, and a 48-byte null symbol entry appended to both the positive symbol array at +344 and the negative symbol array at +352. The dispatcher sub_440590 uses a2 < 0 to select between +352 (negative indices) and +344 (positive indices).

Symbol Management (offsets 376--464)

The architecture vtable is a 632-byte function pointer table created by sub_45AC50 (Mercury targets, when mercury_flag is true) or sub_459640 (non-Mercury targets). If neither returns a valid vtable, the constructor calls fatal_error("couldn't initialize arch state").

Arch State and Named Section Indices (offsets 488--512)

These indices are written during construction after the initial sections are created. They enable fast O(1) access to the mandatory sections without hash table lookups.

Ordered Lists for ELF Segments (offsets 520--560)

Six ordered lists created via sub_465020 with hash/compare functions and element size 16. These manage ELF segment assignments -- the six lists correspond to different segment categories (text, data, rodata, bss, etc.) used during the layout phase.

Hash Table for Section Resolution (offsets 576--584)

Merge State (offsets 592--624)

When bit 0x400 of merge_flags is set, the constructor creates a dedicated "elfw memory space" arena with 4096-byte pages via sub_432020. This arena is stored at +608/+616 and used exclusively for this elfw instance. The destructor checks offset +608 and destroys this private arena if present; otherwise it tears down the sub-structures individually.

Construction Finalization (offsets 624--672)

The constructor ends by calling sub_4504B0(elfw, 0) which performs additional section setup (populating the well-known section name hash table from the static string list at off_1D3A9C0).

Input File Record (offset +512)

A 16-byte input file record is created and appended to the list at v17[64]:

struct input_file_record {
    const char*  filename;       // +0: "<input>" placeholder
    uint32_t     sm_minor;       // +8: sm_minor parameter
    uint32_t     flags;          // +12: initialized to 0
};

This tracks the input files that contributed to this elfw. During the merge phase, each merged cubin adds an entry to this list.

The 40-Byte Writer Context

The polymorphic writer is a small strategy-pattern object that decouples the serialization engine from the output destination. All serialized bytes flow through sub_45B6D0, which dispatches on a mode field at offset 0 of the 40-byte context.

Struct Layout

struct elf_writer {              // 40 bytes
    int32_t   mode;              // +0:  backend selector (0..4)
    int32_t   flags;             // +4:  always 0 in observed paths
    void*     callback_or_state; // +8:  function pointer (mode 0) or reserved
    void*     rewind_fn;         // +16: function pointer for stream rewind
    void*     cleanup_fn;        // +24: destructor called by sub_45B6A0
    void*     dest;              // +32: target -- FILE*, buffer ptr, vector, or callback context
};

Five Dispatch Modes

The central dispatch function sub_45B6D0 handles a NULL writer pointer as a special case (writes to stdout), then switches on the mode field:

int64_t elf_write(elf_writer* w, void* data, size_t len) {
    if (w == NULL)
        return fwrite(data, 1, len, stdout);

    switch (w->mode) {
    case 0:  // Callback
        return w->callback_or_state(w->dest, data, len);
    case 1:  // No-op (size counting)
        return len;
    case 2:  // Growable vector
        vector_append(w->dest, data, len);   // sub_44FC10
        return len;
    case 3:  // FILE* via fwrite
        if (w->dest)
            return fwrite(data, 1, len, w->dest);
        // NULL dest: byte-by-byte putc to stdout
        for (size_t i = 0; i < len; i++)
            _IO_putc(((uint8_t*)data)[i], stdout);
        return len;
    case 4:  // memcpy with advancing cursor
        memcpy(w->dest, data, len);
        w->dest += len;
        return len;
    default:
        return -1;
    }
}

Factory Functions

sub_45B950 -- File-Mode Factory (Mode 3)

Allocates 40 bytes from the elfw's arena, sets mode = 3, stores a pointer to libc rewind() at offset +16 (for potential stream rewinding), and places the FILE* at offset +32. The cleanup function at +24 is NULL because main() manages the file descriptor lifetime.

elf_writer* create_file_writer(FILE* file, elfw_t* elfw) {
    elf_writer* w = arena_alloc(get_arena(file, elfw), 40);
    w->mode      = 3;
    w->flags     = 0;
    w->rewind_fn = &rewind;    // libc rewind()
    w->cleanup_fn = NULL;
    w->dest      = file;
    return w;
}

sub_45BA30 -- Memory-Mode Factory (Mode 4)

Allocates 40 bytes, sets mode = 4, and stores the buffer pointer at offset +32. Both rewind_fn and cleanup_fn are NULL. The dest pointer advances during serialization: each memcpy call advances it by len bytes, so after serialization completes, dest points past the end of the buffer.

elf_writer* create_memory_writer(void* buffer, elfw_t* elfw) {
    elf_writer* w = arena_alloc(get_arena(buffer, elfw), 40);
    w->mode      = 4;
    w->flags     = 0;
    w->rewind_fn = NULL;
    w->cleanup_fn = NULL;
    w->dest      = buffer;
    return w;
}

Both factories take two parameters whose first is the destination and second is the elfw. They call sub_44F410 to retrieve the arena metadata pointer from the elfw (at qword offset +3, i.e., byte offset +24 of the arena header), then allocate the 40 bytes via sub_4307C0.

Writer Cleanup: sub_45B6A0

void destroy_writer(elf_writer* w, void* unused) {
    if (w) {
        if (w->cleanup_fn)       // offset +24
            w->cleanup_fn(w->dest);
        arena_free(w, unused);   // sub_431000
    }
}

Checks offset +24 for a cleanup function. In both observed paths (modes 3 and 4), cleanup_fn is NULL, so only the arena deallocation runs.

Mode 2: Vector-Backed Writer

Mode 2 uses sub_44FC10 (vector_append) to write into a growable arena-backed chunk list. This mode is used for intermediate buffering when the final output size is not known in advance.

The vector is a 40-byte header at dest:

struct vec_header {
    int64_t   default_chunk_size;  // +0:  minimum allocation for new chunks
    int64_t   total_written;       // +8:  cumulative bytes appended
    void*     chain_head;          // +16: first 16-byte wrapper {next, chunk_ptr} (NULL until first append)
    void**    tail_cursor;         // +24: pointer-to-pointer; init = &chain_head, advances to each new wrapper's next-field
    chunk_t*  current_chunk;       // +32: active chunk being filled
};

Each chunk linked through the chain is held by a separate 16-byte wrapper allocated by sub_464460 -- {next at +0, chunk_t* at +8}. The wrapper's +0 is the singly-linked-list link; the wrapper's +8 is the pointer to the 24-byte chunk header. The chain_head field at vec_header +16 is the head of this wrapper list, not a field embedded inside any chunk. The tail_cursor at +24 is the standard self-referencing-tail trick: at init it points at the head slot itself (result + 2, i.e. vec_header+16), so the first append's *tail_cursor = new_wrapper writes the new wrapper pointer directly into the head field; subsequent appends advance tail_cursor to the new wrapper, so the next write lands in that wrapper's next-field at offset +0.

Each chunk itself is a 24-byte header:

struct chunk_t {
    int64_t   capacity;     // +0:  total bytes this chunk can hold
    int64_t   remaining;    // +8:  bytes still available
    void*     data;         // +16: pointer to the data buffer
};

When a write exceeds the current chunk's remaining capacity, the function fills the current chunk with as many bytes as possible, then allocates a new chunk (sized to the larger of default_chunk_size and the remaining write size), copies the rest, and links the new chunk into the chain via a fresh {next, chunk_t*} wrapper.

How the Writer is Used in the Serialization Pipeline

The serialization pipeline follows a strict three-step pattern at both entry points:

1. Create writer  -->  sub_45B950 (file) or sub_45BA30 (memory)
2. Serialize ELF  -->  sub_45BF00 (13,258 bytes -- walks the entire elfw)
3. Destroy writer -->  sub_45B6A0

File output (sub_45C920): Called by main() for non-Mercury targets. The FILE* is opened by main() with fopen(output_path, "wb") before calling this function.

Memory output (sub_45C950): Called by main() for Mercury targets (sm >= 100). The buffer is pre-allocated to the exact size computed by sub_45C980 (size computation using mode 1 dry-run logic). After serialization, the buffer holds the complete ELF image ready for FNLZR post-link transformation.

The serialization engine sub_45BF00 writes the ELF in a strict sequential order through the polymorphic writer:

1. ELF header (52 or 64 bytes -- the raw bytes from the elfw struct)
2. Null padding byte
3. .shstrtab contents (section name strings)
4. .strtab contents (symbol name strings)
5. Alignment padding to .symtab offset
6. Program headers (compact internal format)
7. Section data (sections 4..N-1 with fragment-list traversal)
8. Post-section padding to e_shoff
9. Section header table (40 or 64 bytes per entry)
10. ELF program header table (conditional, via sub_45BAA0)

Every single write operation checks the return value against the expected byte count. Any mismatch triggers sub_467460 with "writing file" -- a fatal error that terminates the linker.

Destructor: sub_4475B0

The destructor handles two cases based on whether the elfw has a private memory arena (offset +608):

Private arena path (offset +608 is non-NULL): Releases the arena metadata via sub_45CAE0, then destroys the entire arena with sub_431C70. This single operation frees all memory allocated from that arena, including all section records, symbol data, and the elfw struct itself.

Shared arena path (offset +608 is NULL): Individually tears down every sub-structure:

1. Frees DCE remap arrays at offsets +472, +464, +456 (v17[59], v17[58], v17[57])
2. Walks and destroys the section name hash table at +296 and symbol name hash table at +288 (via sub_448C00 with callback sub_440080, then sub_448A40)
3. Frees data at offsets +336, +328 (v17[42], v17[41])
4. Destroys six ordered lists at +520..+560 (v17[65]..v17[70]) via sub_466E00
5. Destroys symbol lists at +376, +384, +392 (v17[47], v17[48], v17[49])
6. Walks positive symbol array at +344 (v17[43]), freeing each symbol's associated data
7. Walks negative symbol array at +352 (v17[44]), freeing symbol records from index 1 onward
8. Destroys merged symbol array at +592 and extended symbol store at +600 (v17[74], v17[75]) if non-NULL
9. Walks input file list at +512 (v17[64]), freeing each record
10. Walks section array at +360 (v17[45]), freeing fragment lists and section data
11. Frees linked-list chain at +480 (v17[60]), walking next pointers
12. Destroys section name hash at +496 (v17[62]) and resolution hash at +576 (v17[72])
13. Frees additional lists and the arch vtable at +488 (v17[61])
14. Finally frees the 672-byte elfw struct itself via sub_431000

The private-arena path is far simpler (two calls) because the arena deallocator bulk-frees everything. The shared-arena path runs approximately 30 individual deallocation calls.

Merge-Flags Bitmask Reference

The merge_flags parameter (a9) is a 32-bit bitmask that controls the elfw's behavior. Each bit is unpacked into a boolean field during construction:

When mercury_flag is true or bits 19-20 are set, the constructor sets the mercury_reloc bit (a9 |= 0x80000) in the internal flag word at +76 and routes through the Mercury initialisation path (sub_45AC50 arch state). e_type itself at +16 is not rewritten by these gates -- it remains the value of the elf_type parameter passed in by the caller (set in main at line 391 from (byte_2A5F1E8 == 0) + 1). The Mercury 0xFF00 e_type only appears when the caller passes that value explicitly (e.g. intermediate Mercury serialisations); standard nvlink output is ET_EXEC=2 or ET_REL=1 regardless of Mercury flags.

Bit 12 / std_smem_mode: Consumer-Side Trace

The byte at elfw+99 (std_smem_mode, written as the inverse of merge_flags bit 12 -- see the Metadata and Flags row for the producer-side derivation) is read by exactly two downstream sites. Both branches treat +99 == 1 as "standard smem layout" and +99 == 0 as "user passed --disable-smem-reservation (byte_2A5F210), so do not reserve smem". Confidence: HIGH (direct decompile cross-reference).

Reader 1: Shared-Memory Variable Rebasing (sub_445000:347)

In sub_445000_0x445000.c:340-349 the shared-memory finalisation pass guards a call to sub_439640 with a two-clause predicate:

v3 = *(_WORD *)(a1 + 16);            // e_type
if ( v3 == 2 )                        // ET_EXEC only
{
    v227 = 0x80000000;
    if ( *(_BYTE *)(a1 + 7) == 65 )   // OSABI == 0x41 (device ELF)
        v227 = 1;
    if ( (v227 & *(_DWORD *)(a1 + 48)) == 0   // e_flags relocatable bit clear
       && *(_BYTE *)(a1 + 99) )               // std_smem_mode == 1
        sub_439640(a1);
}

sub_439640 walks the linker's symbol list (a1 + 256) and rebases every shared-memory symbol's value by the device-side smem segment base produced by an arch-vtable callback at *(_QWORD *)(a1 + 488) + 584. The pass exists only for executable device ELFs that are not relocatable and where the standard smem layout was requested. When --disable-smem-reservation is set, +99 is 0 and sub_439640 is skipped -- symbol values remain at their input offsets, which is the behaviour the CLI flag advertises.

Reader 2: OSABI/std_smem Joint Check (sub_451D80:2709)

sub_451D80_0x451d80.c:2706-2711 reads the byte during a symbol-iteration loop that diagnoses cross-section references:

v35 = *((_BYTE *)v34 + 1);                              // symbol's section OSABI
v37 = v35 == 0 || v35 == 38;
if ( !v37 && v35 == 65 && !*(_BYTE *)(a1 + 99) )        // OSABI=0x41 AND !std_smem_mode
{
    v48 = sub_442270(a1, v36);
    ...
    sub_467460(dword_2A5B8D0, *(_QWORD *)(v51 + 32));   // emit diagnostic via section name
}

The diagnostic fires only when the linker is producing a device ELF (v35 == 65) and --disable-smem-reservation was passed (+99 == 0). The combined OSABI=0x41 && !std_smem_mode gate is the only place in the binary where the inverted-bit-12 semantics flips a diagnostic path rather than a layout path -- it tells the user that a referenced section is unexpected once smem reservation is opted out.

Why this matters for the wiki

The producer-side derivation of +99 (constructor sub_4438F0_0x4438f0.c:229) has been documented exhaustively across four audit waves. The consumer side -- the two readers above -- is what actually gives bit 12 its meaning; without them, the byte is unobservable. Together with the Metadata and Flags row and the Merge-Flags Bitmask Reference entry, this section closes the producer/consumer loop for bit 12.

Function Reference

Cross-References

Internal (nvlink wiki):

• ELF Serialization -- The serialization engine (sub_45BF00) that walks the elfw struct and emits bytes through the writer
• Program Headers -- Program header construction (sub_45BAA0) using section classification from the elfw
• Device ELF Format -- ELF header encoding at elfw offsets 0--63, e_flags semantics, and OSABI values
• Output Writing -- Pipeline dispatch between write_elf_to_file (mode 3) and write_elf_to_memory (mode 4)
• Mercury FNLZR -- Mercury path: serialize to memory buffer via compute_elf_size + mode 4, then pass to FNLZR
• Relocation Engine -- Architecture vtable at elfw+488 dispatches relocation application
• R_CUDA Relocations -- Non-Mercury relocation vtable created by sub_459640
• R_MERCURY Relocations -- Mercury relocation vtable created by sub_45AC50
• Section Record -- 104-byte section records stored in the section array at elfw+360
• Symbol Record -- Symbol management via the lists at elfw+376/+384/+392
• Memory Arenas -- Arena allocator (elfw+608/+616) and the "elfw memory space" private arena
• Hash Tables -- Symbol name hash table at elfw+288 and section name hash table at elfw+296 providing O(1) lookup
• Linker Context -- The broader linker state that contains and manages elfw instances

Sibling wikis:

• ptxas: ELF Emitter -- ptxas-side ELF writer for comparison with nvlink's elfw struct and serialization

Confidence Assessment

Each claim below was verified against decompiled functions (sub_4438F0 at /decompiled/sub_4438F0_0x4438f0.c, sub_4475B0, sub_45B6D0, sub_45B950, sub_45BA30, sub_440BE0, sub_441AC0, sub_443260, sub_443500, sub_42F8B0, sub_43E490), string references in nvlink_strings.json, and raw research reports W080, W081.

Struct Size and Allocation

ELF Header (offsets 0--63)

Metadata and Flags (offsets 64--103)

Note Headers and Section Index Cache (offsets 108--210)

Hash Tables and Sorted Arrays (offsets 288--576)

Pointer Fields (offsets 456--624)

Writer Context (40 bytes)

Control-Flow and Behavior

Summary

• Total claims: 89
• HIGH confidence: 85
• MEDIUM confidence: 4

Previously identified labeling errors (now corrected in wiki body):

1. +108/+140 corrected from "dynamic array capacities" to 24-byte NVIDIA note headers
2. +288/+296 corrected: +288 is symbol name hash, +296 is section name hash
3. +344/+352 corrected from "section data/header lists" to positive/negative symbol arrays
4. +592/+600 corrected from "section/symbol remap tables" to merged_symbol_array/extended_symbol_store

Remaining known issues:

1. Section index offsets at +504--+514 are approximate — should be +62/+202/+204/+206/+208/+210
2. "Arch class value = 5" at +624 is a hardcoded constant from sub_42F8B0, not an option parser result | 4096-byte arena page size for "elfw memory space" | HIGH | sub_432020((pthread_mutex_t *)"elfw memory space", 0, 4096) |