Mercury Encoder Pipeline

All addresses in this page apply to ptxas v13.0.88 (CUDA 13.0). Other versions will differ.

Mercury is NVIDIA's intermediate encoding layer between the optimizer's Ori IR and native SASS machine code. It is not a direct binary encoding of SASS -- it is a separate representation that contains pseudo-instructions, lacks dependency barriers, and requires multiple transformation passes before it becomes executable GPU code. The Mercury pipeline occupies phases 113--122 of the 159-phase PhaseManager, forming a six-stage sub-pipeline: encode/decode verification, pseudo-instruction expansion, two WAR-hazard passes (one before and one after operation expansion), scoreboard/latency generation ("opex"), and final SASS microcode emission. All recent GPU architectures (SM 75+) use Mercury as the encoding backend; SM 100+ (Blackwell) defaults to "Capsule Mercury" (capmerc), a variant that embeds additional metadata for relocatable patching.

Architecture

Phase 113  PostFixForMercTargets          Late Ori fixups for Mercury targets
Phase 114  FixUpTexDepBarAndSync          Texture dependency bars + sync fixups
Phase 115  AdvancedScoreboardsAndOpexes   Arch hook point (noop by default)
Phase 116  ProcessO0WaitsAndSBs           -O0 scoreboard insertion
                                          ──────────────────────────────
Phase 117  MercEncodeAndDecode            ┐
Phase 118  MercExpandInstructions         │  Six-stage Mercury core
Phase 119  MercGenerateWARs1              │
Phase 120  MercGenerateOpex               │
Phase 121  MercGenerateWARs2              │
Phase 122  MercGenerateSassUCode          ┘

sub_6F52F0 (23KB orchestrator, 18 params)
  │
  ├─ [1] Decode:     sub_6F2BF0 (59KB)  — Encode Ori→Mercury binary, decode back
  │      └─ sub_6D9690 (94KB master encoder switch)
  │           ├─ sub_6D2750 — append operand word
  │           ├─ sub_6D28C0 — commit encoded instruction
  │           ├─ sub_6D9580 — encode literal values
  │           └─ sub_931690 — create instruction record
  │
  ├─ [2] Expansion:  sub_C3CC60 (26KB)  — Expand pseudo-instructions to SASS
  │      ├─ sub_C37A10 (16KB) — expandInstruction (jump table dispatch)
  │      ├─ sub_C39B40 (10KB) — expandMemoryOp
  │      ├─ sub_C3A460 (6KB)  — expandAtomicOp
  │      ├─ sub_C3B560 (8KB)  — expandTexture
  │      ├─ sub_C3BCD0 (19KB) — expandControlFlow
  │      └─ sub_C3E030 (18KB) — finalizeExpansion
  │
  ├─ [3] WAR pass 1: sub_6FBC20 (7.4KB) — DEPBAR/scoreboard for pre-opex hazards
  │      ├─ sub_6FA5B0 — detect WAR hazard per instruction
  │      ├─ sub_6FA930 — insert scoreboard barrier (opcode 54)
  │      ├─ sub_6FA7B0 — insert WAITDP (opcode 246)
  │      └─ sub_6FAA90 — insert stall cycles
  │
  ├─ [4] Opex:       sub_6FFDC0 (66KB)  — Generate scoreboards + latency waits
  │      └─ sub_703480 (1.4KB entry) or sub_7032A0 (2.3KB MercOpex entry)
  │
  ├─ [5] WAR pass 2: sub_6FBC20          — Same pass, re-run for opex-introduced hazards
  │
  └─ [6] SASS emit:  sub_6E4110 (24KB)  — Final SASS microcode generation
         └─ sub_735290 — per-instruction encoding pipeline
              ├─ sub_733FA0 — encode instruction operands
              ├─ sub_734370 — encode immediates
              ├─ sub_734820 — encode predicates
              ├─ sub_734AD0 — encode memory operands
              └─ sub_734D20 — encode complex operands (texture/surface/barrier)

Each stage logs its completion via trace infrastructure: "After Decode", "After Expansion", "After WAR post-expansion", "After Opex", "After WAR post-opexing".

Mercury vs SASS vs Capsule Mercury

The ptxas CLI (sub_703AB0) accepts --binary-kind with three values:

Additional CLI flags:

• --cap-merc -- force Capsule Mercury generation
• --self-check -- roundtrip verification: reconstitute SASS from capmerc, compare with original
• --out-sass -- dump reconstituted SASS from capmerc

Mercury mode is flagged at *(DWORD*)(context+385) == 2. The function sub_10ADF10 queries the target descriptor to determine whether Mercury encoding is active for the current architecture.

MercConverter -- Operand Reorganization for Encoding

Phase 141 runs the MercConverter infrastructure a second time, after the full optimization pipeline has completed. While phase 5 (ConvertUnsupportedOps) performs the initial PTX-to-SASS opcode conversion early in the pipeline, phase 141 re-invokes the same machinery to handle instructions that were introduced or modified by optimization passes (rematerialization, peephole, loop transformations) and may contain PTX-derived opcodes that were never legalized. After phase 141 completes, the "After MercConverter" diagnostic string appears, and every instruction in the IR carries a valid SASS opcode ready for Mercury encoding.

The orchestrator sub_9F3340 runs two steps sequentially:

1. Opcode conversion (sub_9F1A90, 35KB): the main MercConverter dispatch documented in ISel. Converts any remaining PTX-derived opcodes to SASS equivalents via the master switch in sub_9ED2D0. Gated by *(BYTE*)(*(context+8) + 1398) & 0x20.

2. Operand reorganization (sub_9EF5E0, 27KB): post-conversion lowering that restructures operand lists into a form the Mercury encoder can consume directly. Gated by *(BYTE*)(*(context+16) + 1048) != 0 AND *(context+104) != 0 (non-empty instruction BST).

Gate flag byte_1398 & 0x20 -- who sets it. The flag lives in the SM-backend profile object at byte offset 1398. It is set by the architecture initialization function sub_7DD480, which reads the architecture capability word at profile+1413 and programs byte 1398 based on the encoding-mode bits. Specifically, bit 0x20 is set (byte_1398 = old & 0x1F | 0x60) when capability bits 3 (cap & 0x08, Mercury-capable) or bits 4-5 (cap & 0x30 == 0x20, Capsule-Mercury-capable) are present. When the capability word has cap & 0x30 == 0x10 (legacy non-Mercury mode), bit 0x20 is left unset (byte_1398 &= ~0x80; bit 5 untouched in the 0x10 path). The net effect: bit 0x20 at offset 1398 means "this SM target supports Mercury opcode conversion," and when clear, both phase 5 and phase 141 skip the opcode conversion step entirely, running only operand reorganization (if applicable).

Post-Conversion Lowering -- sub_9EF5E0 (27KB)

This function transforms the BST (binary search tree) of converted instructions produced by step 1 into encoding-ready conversion nodes. For each instruction record in the BST, it performs three operations:

1. Operand sort. Calls sub_9EC160, a linked-list merge sort (Floyd's slow/fast pointer midpoint, recursive split-and-merge) that sorts the operand chain by the operand index at entry+16. This establishes a canonical ordering required by the encoder.

2. Contiguous/gap partitioning. Walks the sorted operand list and classifies each operand into one of two sublists:

// Simplified partitioning logic (lines 215-348 of decompilation)
for (op = first; op != sentinel; op = op->next) {
    int cur_idx  = *(DWORD*)(op + 16);
    int next_idx = *(DWORD*)(op->next + 16);

    if (next_idx - cur_idx == 32) {
        // Consecutive register indices -> contiguous sublist
        append_to_contiguous_list(node, cur_idx);
    } else {
        // Non-consecutive -> gap sublist (stores both cur and next index)
        append_to_gap_list(node, cur_idx, prev_idx);
    }
}

The stride of 32 reflects the operand index encoding: index = register_number * 32 + modifier_bits. Contiguous operands (stride-32 sequences like R0, R1, R2, R3) represent packed register groups -- common in wide loads (LDG.128), GMMA matrix operands, and multi-register moves. The encoder can represent these as a single register-range specifier. Gap operands break the stride and require individual encoding slots.

3. Conversion node construction. Allocates a 168-byte conversion node per instruction, inserts it into a per-record BST sorted by (block_id, sub_block_id), and links the two operand sublists:

Conversion Node (168 bytes):
  +0     8B    BST left child
  +8     8B    BST right child
  +16    8B    BST parent
  +24    4B    block_id
  +28    4B    sub_block_id
  +32    48B   Contiguous operand doubly-linked list (6 pointers)
  +80    4B    Contiguous operand count
  +88    8B    Contiguous list ref-counted handle
  +96    48B   Gap operand doubly-linked list (6 pointers)
  +144   4B    Gap operand count
  +152   8B    Gap list ref-counted handle
  +160   1B    Flags

BST insertion calls sub_7C11F0 for red-black tree rebalancing. The record tracks min/max block IDs at record+32 and record+40 for range queries.

Encoding Validation and Fallback

After building the conversion node, the function attempts encoding:

// Lines 949-982 of decompilation
nullsub_644(*(a1+16), node, "CONVERTING");      // diagnostic trace
int result = sub_7BFC30(node);                   // encoding validation

if (result == -1) {
    // Encoding failed: recursive fallback
    sub_9CE210(a1, node);
    // Continue with next instruction in BST
} else {
    // Encoding succeeded: emit to output
    *(node + 4) = result;                        // store encoding index
    output_slot = vtable_alloc(*(a1+24), 120);   // allocate output record
    *(output_slot + 96) = node;                  // link conversion node
    sub_9314F0(&scratch, *(a1+8), 0xF, 1, 1,    // emit SASS instruction
               &control_word);                   // control = 0x60000000
}

sub_7BFC30 walks the conversion node's BST, accumulates a bit-budget for all operand partitions, and returns an encoding-unit index (0 or 1) or -1 on overflow:

validate_encoding(record):                    // sub_7BFC30
    node = record->bst_root          (+32)    // first conversion node
    if node == NULL: return 1                  // empty -> trivially valid
    bits = 4                                   // base overhead (4 bits)

    for each node in BST in-order:
        contig_count = node->contig_count (+80) - 2    // minus 2 sentinels
        gap_count    = build_gap_list(node)     - 2    // sub_7BEEC0; also minus 2

        // Per-partition limit: max 15 contiguous, max 15 gap operands
        if contig_count > 15 OR gap_count > 15:
            return -1                          // partition too wide

        // Accumulate bit cost: 2-bit node header
        //   + ceil(10 * contig_count / 8) for contiguous encoding
        //   + ceil(15 * gap_count    / 8) for gap encoding
        bits += 2 + ((10 * contig_count + 7) >> 3)
                  + ((15 * gap_count    + 7) >> 3)

    // Convert total bits to encoding-unit index
    rounded = bits + 17                        // +17 = 16-bit unit alignment + 1
    if rounded < 0: rounded = bits + 32        // negative guard (large bit counts)
    unit_index = rounded >> 4                  // divide by 16
    return unit_index < 2 ? unit_index : -1    // must fit in units 0 or 1

The 10-bit and 15-bit per-operand costs reflect the contiguous and gap encoding widths: a contiguous operand needs 10 bits (register base + range length), while a gap operand needs 15 bits (two register indices + modifier delta). The ceiling division (N*W+7)>>3 byte-aligns each partition's contribution.

On failure, sub_9CE210 (a recursive fallback) re-processes the instruction by splitting the operand group into smaller sub-groups that each fit within the 2-unit encoding budget.

Relationship to Phase 5

Phase 5 and phase 141 share the same code (sub_9F3340 orchestrator, sub_9ED2D0 dispatch, sub_9EF5E0 post-conversion). The difference is context:

Stage 1: MercEncodeAndDecode -- Roundtrip Verification

This phase encodes the Ori IR instruction stream into Mercury binary form, then immediately decodes it back and verifies that the decoded result matches the original. This is a self-consistency check that catches encoding bugs early -- if the roundtrip fails, the instruction cannot be correctly represented in Mercury format.

The orchestrator sub_6F52F0 passes the entire pipeline state (18 parameters) to sub_6F2BF0, which performs the actual encode-decode cycle using the master encoder sub_6D9690.

Master Encoder -- sub_6D9690 (94KB)

The central SASS instruction encoding function and the single largest function in the ptxas backend. It reads the instruction type from *(int*)(instruction+8), initializes the encoding base constant 0x2000000000LL, calls sub_C00BF0 for the opcode table index, then dispatches through a 119-case switch covering Ori instruction types 61--544. Unhandled types hit the default branch and return 0 (failure). The epilog at LABEL_3 frees the dynamic word buffer if it grew beyond the 264-byte inline array.

Operand Word Type Prefix -- bits [31:28]

Every operand word appended via sub_6D2750 carries a 4-bit type tag in its upper nibble. The encoder constructs these with literal OR masks:

Case Enumeration (119 cases, 103 distinct types)

The switch groups by instruction category. Cases sharing a body are listed together. The format ID column gives the 3rd argument to sub_931690 (primary encoder) or sub_934630 (secondary/multi-record encoder).

Encoding details:

• Instructions are encoded as sequences of 8-byte words
• Operand word type prefix in bits [31:28]: 0x1 = register, 0x5 = immediate/constant, 0x6 = control/modifier, 0x7 = literal, 0x9 = special
• Control words carry the 0x60000000 prefix
• Architecture-specific bits accumulated in a flags variable, with SM 100+ extensions via knob 4176
• sub_7D6860 handles data type encoding (FP32/FP64/INT, etc.)
• sub_C00BF0 provides opcode lookup from the encoding tables
• sub_91D160 handles register operand encoding

Instruction Word Format

The Mercury instruction word is a 1280-bit (160-byte, 20-QWORD) structure located at offset +544 in the encoder object. All bit-field insertions use sub_7B9B80:

// sub_7B9B80 -- bitfield insert (216 bytes, 18,347 callers)
// Signature: (encoder_obj, bit_offset, bit_width, value)
void bitfield_insert(void *a1, int bit_offset, int bit_width, uint64_t value) {
    uint64_t mask = (1ULL << bit_width) - 1;
    int qword_idx = bit_offset >> 6;
    int bit_pos   = bit_offset & 63;
    *(uint64_t *)(a1 + 8 * qword_idx + 544) |= (value & mask) << bit_pos;
    // handles cross-QWORD boundary cases in a loop up to bit 1280
}

Two companion helpers run before operand encoding:

• sub_7B9D30 (38 bytes) -- clears the 16-entry constant buffer slot table at a1+468 to 0xFF
• sub_7B9D60 (408 bytes) -- encodes reuse flags (1 bit) and predicate register index (5 bits) into the instruction word

Encoding Table Functions (530 functions)

The range 0xC66000--0xD27000 contains 530 functions that each initialize one row of the instruction format table. Every function calls sub_7B9B80 multiple times to describe the SASS bit layout for one instruction format variant:

// Example: sub_C6CF40 — one instruction format initializer
void init_format_XYZ(void *a1) {
    sub_7B9B80(a1, 0,    4, 1);       // bits[0:3]   = opcode field = 1
    sub_7B9B80(a1, 4,    3, 0);       // bits[4:6]   = format = 0
    sub_7B9B80(a1, 8,    9, 0xC);     // bits[8:16]  = subopcode = 12
    sub_7B9B80(a1, 0x11, 8, 0x13);    // bits[17:24] = modifier = 19
    sub_7B9B80(a1, 0x19, 7, 5);       // bits[25:31] = unit = 5
}

Function sizes are remarkably uniform (1000--1600 bytes), reflecting mechanical code generation -- roughly 10 functions per ISA opcode group, covering all SASS formats for SM 100+.

Stage 2: MercExpandInstructions -- Pseudo-Instruction Expansion

Mercury uses abstract instruction forms that may map to multiple real SASS instructions. This phase expands every pseudo-instruction into its concrete SASS equivalent sequence. The expansion is type-dispatched:

sub_C3CC60 iterates over every instruction in the function, dispatching to the appropriate handler. Handlers create new instruction nodes, link them into the list, and delete the original pseudo-instruction. After all expansions, sub_C3E030 (18KB) performs finalization and cleanup.

sub_C37A10 -- expandInstruction jump table

This handler processes general pseudo-instructions. It reads the operand record at instr+16, extracts the Ori opcode from field [2] (offset +8), and maps it through byte_22B7B60[opc] (52-entry LUT, range 0..0x33) to a slot index (default 27 for out-of-range). It then applies predication from field [4]: mode 2 = conditional-true (sub_8F0DC0(slot, 1)), mode 3 = conditional-false (sub_8F0DC0(slot, 0)), mode 4 = uniform (sub_8F0DF0(slot)). The main switch dispatches by opcode:

Overflow to def_C37B2E (at 0xC38180, 13KB) handles complex cases that create new instruction nodes rather than modifying slot state.

sub_C3BCD0 -- expandControlFlow SASS sequence

This handler expands a single control-flow pseudo-instruction into an 8-instruction SASS sequence. It first hashes instr+32 with FNV-1a (prime 16777619, basis 0x811C9DC5) to look up a dedup cache at ctx+488. Each new instruction is allocated by sub_10B1EE0, configured by sub_10AE590 (field setter) and sub_10AE3F0/sub_10AE640 (operand setters), linked by vtable+136 (list splice), then committed by sub_10AF260.

The emitted sequence (8 instructions, in order of creation):

After the 8-instruction sequence, sub_C35F90 (architecture validator) runs on the result, and sub_10ADF90 splices the new nodes into the instruction list replacing the original pseudo-instruction. The field IDs (118, 98, 480, 29, 332, 398) correspond to the Mercury field namespace documented in Stage 1; the values (519, 452, 2481, 126, 1853, 2117) are encoding-table indices that select the concrete SASS bit-field layout for each instruction.

The expansion engine also uses sub_719D00 (50KB), which builds output for expanded instructions across different operand widths (32/64/128-bit, predicate). The four nearly identical code blocks within that function correspond to template instantiations over operand width types.

Stage 3: WAR Hazard Resolution (Phases 119, 121)

Write-After-Read hazards occur when an instruction reads a register that a later instruction will overwrite -- the hardware pipeline can execute them out of order, causing the read to see the wrong value. The WAR pass inserts explicit DEPBAR (dependency barrier) instructions and scoreboard annotations to force correct ordering.

Two passes are needed: WAR1 runs after expansion but before opex, and WAR2 runs after opex. The second pass exists because opex itself introduces new instructions (scoreboard waits, synchronization barriers) that create additional WAR hazards not present in the pre-opex stream.

WAR Pass Algorithm -- sub_6FBC20

// Simplified WAR generation pass
void GenerateWARs(context) {
    // Guard conditions
    if (!(context->instr_flags & 1))  return;  // no WAR-sensitive instrs
    if (context->mode != 2)           return;  // not Mercury mode

    // Per-instruction walk
    for (instr = first; instr != end; instr = instr->next) {
        // Detect hazard
        int severity = DetectWARHazard(state, instr);  // sub_6FA5B0

        if (severity >= 3) {
            InsertScoreboardBarrier(state, instr);       // sub_6FA930, opcode 54
            InsertWAITDP(state, instr);                  // sub_6FA7B0, opcode 246
            InsertWARStalls(state, instr, severity);     // sub_6FAA90
        }
    }

    PostWARAdjustment(state);    // sub_6FB850
    FinalizeWARPass(state);      // sub_6FB350
}

WAR Hazard Detection -- sub_6FA5B0 (2.5KB)

The detector uses a three-stage opcode filter to classify each instruction.

Stage 1 -- first bitmask 0x100000400001 (opcodes 34--78). The range check (opcode - 34) > 0x2C admits opcodes 34--78. Within that window the 45-bit bitmask selects three opcodes for architecture-specific vetting via vtable +968 / +1008:

All 42 other opcodes in 34--78 (I2I, I2IP, IMNMX, POPC, FLO, FCHK, IPA, MUFU, F2F, F2F_X, F2I, F2I_X, I2F, I2F_X, FRND, FRND_X, AL2P, AL2P_INDEXED, BREV, BMOV_B, BMOV_R, S2R, B2R, R2B, LEPC, BAR, BAR_INDEXED, SETCTAID, SETLMEMBASE, GETLMEMBASE, DEPBAR, BRA, BRX, JMP, JMX, CALL, RET, BSSY, BREAK, BPT, KILL, EXIT) have their bitmask bit clear and proceed directly to stage 2.

Stage 2 -- second bitmask 0x800200000100001 (opcodes 71--130) plus opcode 235. An explicit equality test marks opcode 235 (UBLKRED) as never-hazardous. The range check (opcode - 71) > 0x3B admits opcodes 71--130. Within that window the 60-bit bitmask flags four opcodes as unconditionally non-hazardous:

If the combined flag (opcode==235 OR second-bitmask bit set) is nonzero the detector returns immediately with counter = 0 (no hazard). All opcodes outside both bitmask windows that are not 235 fall through to the vtable +688 check.

Stage 3 -- opcode-specific hazard classification. After the vtable +688 filter (arch-specific blanket non-hazard check), the remaining opcodes are classified:

The detector maintains per-instruction state:

• *(DWORD*)(state+8) -- WAR counter (incremented per detected hazard)
• *(DWORD*)(state+12) -- severity level (3 = medium, 4 = high)

For severity 3 or 4, sub_6FA430 inserts (severity - counter) additional stall slots before the instruction, then advances the counter to match.

Inserted Instructions

DEPBAR / Scoreboard barrier (opcode 54) -- sub_6FA930:

• Created when *(BYTE*)(instr+48) & 0x10 is set (barrier-needed flag)
• Barrier type extracted from bits 7:5 of the flag byte
• Encoding: *(DWORD*)(new_instr+56) = 4 (barrier format)
• Control bits: *(DWORD*)(new_instr+48) &= 0xFFF83FFF | 0x50000

WAITDP (opcode 246) -- sub_6FA7B0:

• Skipped if a WAITDP already exists at the insertion point
• Operands configured with codes 102/467 and 301/1520
• Uses FNV-1a hash lookup for instruction deduplication

FNV-1a dedup cache -- sub_6E4110 offset +128:

The emission pass maintains a hash map that caches previously emitted byte sequences keyed by (instr_offset, block_base_addr). When emitting the same logical instruction at the same block-relative position, the cache returns the prior encoded result and avoids redundant encoding work. The cache lives in the emitter context at offsets +128 (entry count), +136 (bucket array pointer), +144 (bucket count, always a power of two).

// --- FNV-1a hash over (instr_offset, block_addr) ---
// instr_offset = *(DWORD*)(instr+144)    // 4 bytes: assigned byte position
// block_addr   = current block base addr  // 8 bytes
fn fnv1a_dedup_key(instr_offset: u32, block_addr: u64) -> u32:
    h = 0x811C9DC5                          // FNV offset basis
    for byte in instr_offset.to_le_bytes(): // 4 rounds
        h = (h ^ byte) * 16777619           // 0x01000193
    for byte in block_addr.to_le_bytes():   // 8 rounds
        h = (h ^ byte) * 16777619
    return h

// --- cache lookup / insert ---
// bucket_array: pointer to array of 24-byte chained entries
// bucket_count: power-of-two bucket count (stored at ctx+144)
// entry layout (24 bytes):
//   +0  next_ptr   (u64) -- singly-linked chain within bucket
//   +8  block_addr (u64) -- key part 1
//   +16 instr_off  (u32) -- key part 2
//   +24 cached_val (i32) -- cached emission result (signed delta)
fn dedup_lookup(ctx, instr_offset, block_addr) -> Option<i32>:
    if ctx.entry_count == 0:
        return None                         // empty cache → miss
    h    = fnv1a_dedup_key(instr_offset, block_addr)
    slot = ctx.bucket_array + 24 * ((ctx.bucket_count - 1) & h)
    node = *(u64*)slot                      // head of chain
    while node != 0:
        if *(u64*)(node+8) == block_addr && *(u32*)(node+16) == instr_offset:
            return Some(*(i32*)(node+24))   // hit → cached delta
        node = *(u64*)node                  // follow chain
    return None                             // miss

// --- bypass conditions ---
// The cache is skipped entirely when:
//   1. instr_offset == -1  (instruction not yet assigned a position)
//   2. The instruction is a branch target (opcode == -1, pseudo-label)
//   3. The instruction carries a relocation flag (*(BYTE*)(instr+148) & 0x10)

The cached delta is used to compute a relocation adjustment: delta = cached_val - block_addr. Both sub_6E4110 and sub_726E00 use the identical FNV-1a constants and per-byte folding pattern; sub_726E00 hashes only the 4-byte instruction ID for its own microcode sequence cache at context offsets +744/+752/+760.

Stall cycles -- sub_6FAA90 (7.9KB):

• Computes required stall cycles from architecture-specific latency tables
• Vtable methods at +888, +896, +904 for stall calculation
• GPU family dispatch: v8[14] == 9 triggers specific handling
• Adjusts stall count fields in the instruction control word

Stage 4: Opex -- Operation Expansion

"Operation expansion" is the most complex stage. It generates the dependency scoreboards, computes latency waits, and inserts synchronization barriers that the hardware needs to manage instruction-level parallelism. After opex, the instruction stream contains all the scheduling metadata required for correct execution.

Entry Points

Two entry paths exist, both calling the same sub_6FFDC0 body:

sub_703480 (RunOpexPass, 1.4KB):

1. Creates pipeline context via sub_6FC280
2. Queries knob #17 to disable WAR penalty flags: *(context->flags+52) &= ~0x10
3. Architecture check: *(DWORD*)(context+52) == 20481 (SM 100a)
4. For SM 100a: queries knob at offset 1296/1304 for loop unroll factor
5. Sets Mercury mode: *(DWORD*)(context+385) = 2
6. Calls sub_6FFDC0 for actual expansion

sub_7032A0 (RunMercOpexPass, 2.3KB):

• Nearly identical to sub_703480
• Additionally calls sub_10ADF10 to verify Mercury mode is active
• Allocates 40-byte + 24-byte records for Mercury-specific context
• Calls sub_6FED20 to destroy previous Mercury context before creating new one

Opex Body -- sub_6FFDC0 (66KB)

This 66KB function (200+ locals, ~0x7A0 bytes stack frame) is the core of Mercury scheduling. It walks every basic block, computes latencies, allocates scoreboards, inserts waits and barriers, and sets stall counts. The six algorithmic steps execute in sequence within a per-block loop.

OpexBody(ctx):                                         // sub_6FFDC0
  // ── Initialization ────────────────────────────────
  func_obj    = *ctx
  is_mercury  = sub_10ADF10(func_obj)                  // Mercury mode?
  sb_count    = 0;  flags = {0,0,0,0}
  sub_6FC810(ctx, &sb_count, &flags[0..3], is_mercury) // query knobs #17/#743/#747
  if !ctx->byte_29:  goto fast_path                    // skip scoreboard if disabled

  // Allocate 4 tracking arrays sized by *(func+984) (max scoreboard index):
  //   A[0..N]=1 (occupied), B[0..N]=1 (refcount), C[0..N]=0 (pending waits)
  sub_BE21D0(tracking_state)                           // finalize array D from A+B+C

fast_path:
  if ctx->mode == 1:                                   // trivial single-block kernel
    sub_6FC3A0(ctx, 1, 1)                              // single-pass expansion
    goto done

  // ── Build block iteration order ───────────────────
  sub_8A4F20(block_order, func_obj)                    // RPO block list
  sub_6FF070(dep_ctx, func_obj, block_order, ctx->i8, sb_count)
  ref_A = alloc_refcount(3);  ref_B = alloc_refcount(3)
  sub_6FD5D0(ref_B);  sub_6FD550(ref_A)               // producer/consumer set init
  if is_mercury:
    merc_state = alloc(272)
    sub_8BE320(merc_state, func_obj, block_order, ctx->i8, ctx->end_ptr)

  // ── Step 1: Per-instruction latency lookup ────────
  //   sub_8A9D80: for each instruction in every block:
  //     latency_idx = instruction_legality[opcode * 4 + variant]
  //       (60416-entry table @ VA 0x22FEE00; 4 bytes/entry; 68% zero)
  //     if latency_idx & 0x08000000:  special-case handling (259 opcodes)
  //     raw_latency = latency_idx & 0xFFFF (cycle count)
  //     pipe_class  = latency_idx >> 16    (functional unit ID)
  //   Arch-specific overrides via vtable at func+39, offsets +72/+120
  //   Knob #743: compresses latency to reduce register lifetime
  //   Knob #747: applies DynBatch scaling factors (see SCHED-36)
  sub_8A9D80(tracking_state, func_obj, dep_ctx, block_order,
             ctx->end_ptr, &dep_root, flags[0..3],
             is_mercury, merc_state, ref_context)

  // ── Prepare per-block processing state ────────────
  sb_limit  = dep_ctx->i22 + 1                         // max scoreboard entries
  has_cap10 = vtable_query(func+39, 10);  has_cap87 = vtable_query(func+39, 87)
  // Allocate 7 scoreboard tables (sub_6FDD30): sb_tables[0]=256x56B (active
  // descriptors), [1..3]=8x56B (barrier groups), [4..5]=256x56B (wait state),
  // [6]=8x56B (membar). Auxiliary: hash_A/B 8x56B (dedup), reg_map 36x8B,
  // stall_vec 36x8B, dep_bitmap 773x8B (per_sm_scoreboard_configs: SM100 has
  // 75 entries x 88B, triplet={scoreboard_id, threshold=56, mask=0xFFFFFFFF})
  sub_BFAFC0(dep_bitmap, func_obj, ctx->end_ptr)      // init dependency bitmap
  sub_719D00(block_iter)                               // begin block iteration

  // ── Steps 2-6: Per-block loop ─────────────────────
  for block_id = 0 .. func_obj->i246:
    block_desc = func_obj->ptr_976[block_id]           // 40-byte block record
    sub_718C10(ctx->end_ptr, &block_desc)              // load block

    if is_mercury:
      cap = *(func+39 + 72)                           // vtable-derived knob state
      if !cap || (cap == 1 && *(func+39)->i4904):
        sub_8B6860(merc_state, &block_desc)            // Mercury block init
      else:
        sub_8A5CA0(merc_state)                         // simplified init

    // Step 2 ── Scoreboard entry allocation ──────────
    //   sub_8BDC40: for each producer instruction in block:
    //     key = encoding_bitfield_lookup[opcode_key]
    //       (4096 entries @ VA 0x23F2E00, pair {field_a, field_b})
    //     if key.field_b != 0xFFFFFFFF:
    //       sb_entry = alloc from sb_tables[0] (LRU evict if full)
    //       sb_entry.id chosen from per_sm_scoreboard_configs triplets
    //       mark register outputs as tracked by sb_entry
    sub_8BDC40(func_obj, dep_ctx, &opex_state, block_order,
               ctx->end_ptr, &block_desc)

    // Step 3 ── Wait insertion ───────────────────────
    //   sub_BFC850: scan consumer operands vs active scoreboards:
    //     if consumer reads reg owned by sb_entry with remaining_lat > 0:
    //       insert DEPBAR.WAIT before consumer (opcode 54, format=4)
    //       wait_mask |= (1 << sb_entry.scoreboard_id)
    //   Called twice: flag=1 (pre-scan), flag=0 (commit)
    if tracking_state[148]:
      sub_BFC850(dep_bitmap, &opex_root, &block_desc, 1, 0)  // pre-scan
    sub_8BB4D0(func_obj, tracking_state, &block_desc)  // update tracking
    sub_BFC850(dep_bitmap, &opex_root, &block_desc, 0, 0)    // commit waits

    // Step 4 ── Stall count computation ──────────────
    //   sub_BFE320: for each instruction:
    //     stall = max(0, producer_latency - distance)
    //     stall = min(stall, 15)          // 4-bit field in control word
    //     Knob #17: if WAR penalty disabled, skip penalty additions
    //     Arch vtable +888/+896/+904: per-pipe stall overrides
    //     Writes stall into control word bits [3:0]
    sub_BFE320(dep_bitmap, &block_desc)

    // Step 5 ── Memory barrier placement ─────────────
    //   sub_717C50: inserts MEMBAR/FENCE at block boundaries and before
    //   memory ops crossing consistency domains.  Only when mode > 2.
    //   Uses sb_tables[6] to coalesce redundant barriers per block.
    if ctx->mode > 2:
      sub_717C50(block_iter, &block_desc)

  // ── Cleanup (reverse-order destruction of all tables/ref-counted nodes) ──
done:
  func_obj->i385 = 2                                  // mark Mercury mode = done

Step 6 -- Knob interaction: knobs #17, #743, and #747 are not separate phases but modulate steps 1-5 throughout. sub_6FC810 reads all three at initialization, storing derived flags in flags[0..3]:

• Knob #17 (expansion options): clears bit 4 of ctx->flags+52, disabling WAR penalty additions in step 4. When set, sub_BFE320 skips the WAR stall penalty term.
• Knob #743 (reduce-reg): passed to sub_8A9D80 in step 1. Compresses producer latencies so consumers schedule sooner, reducing live register count at the cost of more stalls.
• Knob #747 (dynamic batch): also passed to sub_8A9D80. Applies DynBatch priority scaling factors (see SCHED-36) to latency values, favouring throughput over latency.

New instructions created by opex use sub_10B1F90 (instruction allocator) and sub_10AE590 (operand configuration).

Stage 5: SASS Microcode Emission

The final stage converts the fully expanded, WAR-resolved, scoreboard-annotated Mercury stream into native SASS binary. This is the point of no return -- after this phase, the output is executable GPU machine code.

sub_6E4110 takes 8 parameters (context, instruction list, descriptors, format info, ...) and dispatches to the per-instruction encoding pipeline:

sub_6E4110 (24KB, final SASS emission)
  ├─ sub_735290 — per-instruction encoding pipeline
  │    ├─ sub_733FA0 (5.1KB)  — encode instruction operands
  │    │    └─ sub_733870 (10KB) — source operand encoder
  │    ├─ sub_734370 (6.1KB)  — encode immediates
  │    ├─ sub_734820 (4.1KB)  — encode predicates
  │    ├─ sub_734AD0 (3.3KB)  — encode memory operands
  │    └─ sub_734D20 (8.1KB)  — encode complex operands (texture/surface/barrier)
  ├─ sub_726E00 (30.6KB) — instruction encoding with FNV-1a dedup cache
  │    └─ sub_7266A0 (11.7KB) — hash table lookup (24-byte entries, separate chaining)
  ├─ sub_6E3F80 (2.2KB) — encode branch offsets
  ├─ sub_6E3560 (2.6KB) — finalize scheduling control words
  └─ sub_712E70 (9.6KB) — handle relocations (cross-BB branch targets)

The encoding pipeline uses FNV-1a hashing (seed 0x811C9DC5, multiplier 16777619) to cache instruction encodings and avoid re-encoding identical instructions.

Architecture-Specific Dispatch

Architecture selection reads *(int*)(config + 372) >> 12 to determine the SM generation. A vtable at *(context+416) (equivalently *(compilation_state+1584)) with 400+ slots provides per-architecture behavior for encoding, latency tables, resource queries, and hazard rules. 41 slots have confirmed call sites; the highest observed is slot 402 (+3216).

The encoder state initializer sub_6E8EB0 (64KB) sets architecture-specific flags and populates the opcode descriptor table (40+ entries mapping internal opcode IDs to encoding words). For SM 80 (0x5000) it sets bits 1 and 8; for SM 84 (0x5004) it sets bits 16 and 64.

Vtable dispatch helpers at 0xC65530--0xC656E0:

• sub_C65530 -- 3-key dispatch (opcode, subop1, subop2), binary search through 24-byte table entries
• sub_C65600 -- instruction-keyed dispatch, reads keys from instr+12/14/15
• sub_C656E0 -- instruction-keyed dispatch with fallback to default handler sub_9B3020

Architecture Vtable Method Catalog

The vtable is accessed as **ptr where ptr = *(compilation_state + 1584) or *(pipeline_context + 416). Each slot is 8 bytes (function pointer). Methods are grouped by functional area; "refs" is the number of distinct decompiled call sites.

Scheduling class / latency (most heavily used):

Resource query (read-only architecture parameters):

Hazard detection (accessed through *(pipeline_context+416) in WAR pass sub_6FA5B0):

Operand / encoding support:

Instruction legalization / rewrite:

The separate scheduling vtable at 0x21DBC80 (77 entries, 32 unique functions) is a per-pipe-class dispatch table. It has three pipeline groups (A/B/C) of 23 entries each mapping pipe index to latency/throughput query functions, preceded by 8 core scheduling methods at 0x8DA680--0x8DC620 (stall computation, barrier allocation, yield threshold).

Data Structures

Mercury Instruction Word

Offset  Size    Field
------  ------  --------------------------------------------------
+0      8B      vtable pointer (encoder object)
+468    64B     Constant buffer slot table (16 x DWORD, cleared to 0xFF)
+532    4B      Constant buffer slot count
+544    160B    Instruction word (1280 bits = 20 QWORDs)
                — populated by sub_7B9B80 bitfield inserts
                — max addressable bit: 1280

SASS Encoding Record (~264 bytes)

Output of sub_6D9690. Contains the encoded instruction words, operand data, and metadata. The encoding base constant is 0x2000000000LL.

Pipeline Context

Offset  Size    Field
------  ------  --------------------------------------------------
+52     4B      Architecture ID (20481 = sm100a)
+236    1B      Uses shared memory flag
+284    4B      Function flags (bits 0, 3, 7 checked by WAR pass)
+385    4B      Mercury mode flag (2 = Mercury/Capsule mode)
+416    8B      Architecture vtable pointer (400+ slots, 41 confirmed -- see catalog above)

Scheduling Control Word (per SASS instruction)

Offset  Size    Field
------  ------  --------------------------------------------------
+48     4B      Control bits (barrier flags at bits 17:13)
+56     4B      Encoding format (4 = barrier format)
+144    4B      Scheduling slot
+164    4B      Resource class
+168    1B      Stall bits
+236    4B      Latency value

Mercury Instruction Node Layout

The Mercury pipeline (phases 117--122) operates on its own instruction representation, distinct from the 296-byte Ori IR instruction node documented in Instructions & Opcodes. The master encoder sub_6D9690 (phase 117) reads Ori IR nodes and produces Mercury instruction nodes; all subsequent phases -- expansion, WAR resolution, opex, and SASS emission -- operate exclusively on Mercury nodes.

Allocation

Mercury instruction nodes are allocated by sub_10AF8C0 (92 lines), which either recycles a node from a per-block free list or allocates exactly 160 bytes from the arena. The primary API wrappers are sub_10B1F90 and sub_10B1EE0, which call sub_10AF8C0 and perform additional bookkeeping (FNV-1a deduplication cache registration, scheduling state propagation).

Node Layout (160 bytes)

The opcode field at +28 carries the Mercury/SASS opcode number. Known values include: 0xFFFF (sentinel, BB boundary marker), 54 (DEPBAR -- dependency barrier), 246 (WAITDP -- wait for dependency pipeline). All other values are SASS instruction opcodes.

Scheduling Control Record (60 bytes)

Each Mercury instruction node points (via +128) to a separately allocated 60-byte scheduling control record. This record carries barrier state, stall counts, and encoding format metadata that the WAR and opex passes read and modify.

The control_bits field at sched+48 is the primary target of WAR pass modifications:

Bits 17:13  — barrier type (masked via 0xFFF83FFF then OR'd with type << 13)
Bit  4      — barrier-needed flag (in byte at sched+50)
Bits  7:5   — barrier sub-type (in byte at sched+50)

WAR insertion functions modify this field with specific patterns:

• sub_6FA930 (InsertScoreboardBarrier): sched[48] = (sched[48] & 0xFFF83FFF) | 0x50000; clears bit 4 of sched[50]; sets sched[56] = 4
• sub_6FA430 (InsertNOP): sched[48] = (sched[48] & 0xFFF83FFF) | 0x44000; clears bit 4 of sched[50]; sets sched[56] = 1
• sub_6FAFD0 (InsertStall): sched[48] = (sched[48] & 0xFFF83FFF) | 0x3C000; sets bit 4 of sched[50]; sets sched[56] = 15

Linked-List Structure

Mercury nodes form a doubly-linked list per basic block, managed through the next (+0) and prev (+8) pointers:

           head (ctx+40)                          tail (ctx+32)
              |                                      |
              v                                      v
         [node_0]  <-->  [node_1]  <-->  ...  <-->  [node_N]
         next=node_1     next=node_2                 next=0
         prev=0          prev=node_0                 prev=node_{N-1}

New nodes are inserted before the reference node by sub_10AF8C0. The WAR pass (sub_6FBC20) iterates forward through the list; sub_6FB850 (PostWARAdjustment) iterates backward, skipping sentinel nodes (opcode == 0xFFFF).

FNV-1a Deduplication

The sequence_id at +32 serves as the FNV-1a hash key for the instruction deduplication cache. The hash is computed over the 4-byte ID using the standard FNV-1a parameters (seed 0x811C9DC5, multiplier 16777619). The cache resides at context+488 (hash table pointer) with capacity at context+496 and entry count at context+480. Each hash table entry is 24 bytes with separate chaining via pointer at entry+0, key at entry+8, and value (Mercury encoding record pointer) at entry+16.

Relationship to Ori IR Instruction Node

The Mercury node is distinct from the Ori IR instruction node:

Phase 117 (MercEncodeAndDecode) reads Ori IR nodes via the master encoder sub_6D9690 and produces Mercury nodes. All subsequent Mercury pipeline phases operate on Mercury nodes exclusively.

Configuration

Diagnostic Strings

Function Map

Cross-References

• ISel & Opcode Selection -- MercConverter opcode dispatch table (sub_9ED2D0), handler details
• Instructions & Opcodes -- Ori IR instruction node layout (296 bytes, input to Mercury encoder)
• Code Generation Overview -- high-level codegen pipeline context
• SASS Instruction Encoding -- detailed bit-level encoding format
• Capsule Mercury & Finalization -- capmerc variant and --self-check
• Scoreboards & Dependency Barriers -- DEPBAR/WAITDP semantics
• Phase Manager -- 159-phase pipeline infrastructure
• Optimization Levels -- -O0 vs higher-level scoreboard behavior
• Knobs System -- knob #16, #17, #743, #747 details