PTX-to-Ori Lowering

All addresses in this page apply to ptxas v13.0.88 (CUDA 13.0). Other versions will differ.

The PTX-to-Ori lowering is the transition from parsed PTX assembly into the Ori internal representation -- the SASS-level, virtual-register IR that all subsequent optimization operates on. Unlike a traditional compiler where the parser builds an AST and a separate lowering pass consumes it, ptxas has no materialized AST: the Bison parser's reduction actions directly construct Ori IR nodes, basic blocks, and CFG edges inline. What the --compiler-stats timer calls "DAGgen-time" measures this inline construction phase. The result is a raw Ori IR that still uses PTX-derived opcodes and has unresolved architecture-dependent constructs. Fourteen "bridge phases" (pipeline indices 0--13) then transform this raw IR into the optimizer-ready form where every instruction carries its final SASS opcode, the CFG is fully annotated, and architecture-incompatible operations have been legalized.

The key architectural consequence of this design: there is no separate "lowering" function that you can point at and say "this converts PTX to Ori." The conversion is distributed across (1) the Bison parser's 443 reduction actions, (2) a 44 KB operand processing function, (3) the MercConverter instruction legalization pass, and (4) six additional bridge phases that handle FP16 promotion, control flow canonicalization, macro fusion, and recipe application.

Architecture

PTX source text
     |
     v
[Flex scanner]  sub_720F00 (15.8KB, 552 rules)
     |  token stream
     v
[Bison parser]  sub_4CE6B0 (48KB, 512 productions)
     |  NO AST -- reduction actions build IR directly:
     |    - allocate instruction nodes from pool
     |    - set opcode field (instruction +72)
     |    - build operand array (instruction +84)
     |    - link into doubly-linked list per basic block
     |    - create basic block entries (40B each)
     |    - populate CFG hash maps (Code Object +648, +680)
     |
     v                                             "DAGgen-time"
[Operand processing]  sub_6273E0 (44KB)            boundary
     |  6-bit type switch (v12 & 0x3F)             ----------
     |  address computation, state space annotation
     v
+----------------------------------------------------------+
|  RAW ORI IR (PTX-derived opcodes, virtual registers)     |
|  Instructions: PTX-level names (add.f32, ld.global, etc) |
|  Registers: virtual R-file, typed descriptors             |
|  CFG: basic blocks + edge hash maps (partially formed)    |
+----------------------------------------------------------+
     |
     |  Phase 0: OriCheckInitialProgram (validate)
     |  Phase 1: ApplyNvOptRecipes      (configure opt levels)
     |  Phase 2: PromoteFP16            (FP16 -> FP32 where needed)
     |  Phase 3: AnalyzeControlFlow     (finalize CFG + RPO + backedges)
     |  Phase 4: AdvancedPhaseBeforeConvUnSup (arch hook, no-op default)
     |  Phase 5: ConvertUnsupportedOps  (MercConverter: PTX ops -> SASS ops)
     |  Phase 6: SetControlFlowOpLastInBB (CFG structural fixup)
     |  Phase 7: AdvancedPhaseAfterConvUnSup (arch hook, no-op default)
     |  Phase 8: OriCreateMacroInsts    (fuse instruction sequences)
     |  Phase 9: ReportInitialRepresentation (diagnostic dump)
     |  Phase 10: EarlyOriSimpleLiveDead (dead code elimination)
     |  Phase 11: ReplaceUniformsWithImm (fold known constants)
     |  Phase 12: OriSanitize           (validate post-bridge IR)
     |  Phase 13: GeneralOptimizeEarly  (bundled copy-prop + const-fold)
     v                                             "OCG-time"
+----------------------------------------------------------+               begins
|  OPTIMIZER-READY ORI IR                                  |
|  Instructions: SASS opcodes (FADD, IMAD, LDG, STG, ...) |
|  Registers: virtual R/UR/P/UP files                       |
|  CFG: complete with RPO, backedge map, loop headers       |
+----------------------------------------------------------+
     |
     v
[Phase 14+: main optimization pipeline]

Inline IR Construction (Bison -> Ori)

The Bison parser at sub_4CE6B0 has 512 grammar productions with 443 reduction-action cases. Each reduction action constructs IR directly -- no intermediate AST is ever materialized. The instruction table builder (sub_46E000, 93 KB, 1,141 per-opcode registration calls to sub_46BED0) runs during parser initialization and registers the legal type combinations for every PTX instruction. The instruction lookup subsystem (sub_46C690 entry, sub_46C6E0 matcher at 6.4 KB) classifies operands into 12 categories at parse time.

When the parser encounters a PTX instruction like add.f32 %r1, %r2, %r3, it:

1. Looks up add.f32 in the opcode table to get the internal opcode index and validate the type qualifier .f32
2. Allocates an Ori instruction node from the memory pool
3. Writes the opcode into the instruction field at offset +72
4. Processes each operand through sub_6273E0 to build the packed operand array at offset +84
5. Links the instruction into the current basic block's doubly-linked list
6. If the instruction is a branch/jump/return, creates a CFG edge in the successor hash map at Code Object +648

Special PTX registers (%ntid, %laneid, %smid, %ctaid, %clock64, etc.) are mapped to internal identifiers during parser initialization at sub_451730. The mapping table is built from the ROT13-encoded opcode table populated by ctor_003 at 0x4095D0.

Operand Processing -- sub_6273E0

The 44 KB operand processing function handles all PTX operand forms. It reads v12 = *(_BYTE *)a2 (the first byte of the PTX AST node) and switches on v12 & 0x3F. The 11 active case values and the default are:

String references in callees of sub_6273E0 (not in the function itself, but in its direct callees sub_627380, sub_623AB0, sub_4CE6B0):

• ".nv.reservedSmem.{begin,end,cap,offset0,offset1}" -- reserved shared memory region symbols resolved by sub_627380 (called from case 4, storage class 3, state-space codes 'e'--'i')
• "COARSEOFFSET" -- coarse-grained offset label created by sub_623AB0 for large address space indexing
• "__$endLabel$__%s" -- end-of-scope label synthesized by the Bison parser (sub_4CE6B0) for structured control flow

The function bridges PTX's explicitly-typed operand model (where .u32, .f32, .b64 qualifiers are part of the syntax) to Ori's implicitly-typed model where the operand type is determined by the SASS opcode. Type resolution at sub_61C9B0 maps PTX type qualifiers to a 5-entry size class via dword_20249C0[min(typeClass, 4)], producing a size-category byte (255 for out-of-range) that is packed into the high word of the Ori operand descriptor.

Bridge Phases (0--13)

Phase 0: OriCheckInitialProgram -- Validation

Validates the raw Ori IR produced by the Bison parser for structural correctness: all basic blocks have valid entry/exit points, instruction operand counts match opcode requirements, register references are within bounds, and CFG edges are consistent. This is a pure validation pass that produces no IR transformations. It catches malformed IR early, before any optimization pass can amplify a structural error into a hard-to-diagnose miscompile.

Phase 1: ApplyNvOptRecipes -- Optimization Level Configuration

Applies NvOptRecipe transformations controlled by option 391. When enabled, the PhaseManager's constructor (sub_C62720) allocates a 440-byte NvOptRecipe sub-manager at PhaseManager+56. This sub-manager configures per-phase behavior based on the NvOpt level (0--5), controlling which later phases are active and their aggressiveness:

The string "Invalid nvopt level : %d." in sub_C173E0 confirms the valid range. The recipe data lives at NvOptRecipe+312 with per-phase records at stride 584 bytes. The sub-manager maintains its own sorted array (+376) and hash table (+400..+416) for fast recipe lookup by phase index.

NvOptRecipe Sub-Manager (440 bytes, at PhaseManager+56)
  +0      compilation_unit
  +8      phase_manager back-reference
  +16     ref_counted_list_1
  +312    recipe_data
  +336    allocator
  +344    timing_records (stride = 584 per entry)
  +376    sorted_array (for binary search by phase index)
  +400    hash_bucket_count
  +408    hash_buckets
  +432    shared_list_ptr (ref-counted)

Phase 2: PromoteFP16 -- Half-Precision Type Promotion

Promotes half-precision (FP16) operations where hardware support is insufficient or promotion yields better throughput. The promotion strategy is architecture-dependent:

• Pre-sm_53: no native FP16 ALUs. All FP16 arithmetic is expanded to FP32 with narrowing conversions at stores.
• sm_53+: native FP16 support. Only operations that require expensive multi-instruction sequences in FP16 (certain transcendentals, complex comparisons) are promoted.
• sm_89+ (Ada, Blackwell): wide FP16 tensor paths. Promotion is minimal; most FP16 stays native.

The phase walks the instruction linked list, inspects each instruction's type encoding at offset +72, and rewrites FP16 operations to FP32 equivalents by replacing the opcode and inserting conversion instructions (F2F in SASS terminology) at use/def boundaries.

Phase 3: AnalyzeControlFlow -- CFG Finalization

Builds and finalizes the control flow graph data structures that the optimizer requires:

• Successor edges: populates the FNV-1a hash table at Code Object +648
• Backedge map: computes backedges and stores them at Code Object +680
• RPO array: builds the reverse post-order traversal at Code Object +720
• Loop identification: marks loop headers and backedge targets for later loop optimization passes (phases 18, 22, 24, 59)

The Bison parser constructs basic blocks and edges incrementally as it processes PTX instructions, but the CFG is not guaranteed to be fully consistent until this phase runs. For example, forward branch targets may reference blocks that were not yet created at parse time. This phase resolves all pending edges and ensures the CFG is complete.

Phases 4 and 7: Architecture Hook Points

Phases 4 (AdvancedPhaseBeforeConvUnSup) and 7 (AdvancedPhaseAfterConvUnSup) are no-op-by-default hook points that bracket ConvertUnsupportedOps. Architecture backends override their vtables to inject target-specific processing:

• Phase 4 (before): prepare target-specific state, mark instructions that need special handling on this architecture
• Phase 7 (after): clean up after legalization, fix architecture-specific edge cases introduced by the generic lowering

These hooks are part of the 16 AdvancedPhase injection points distributed throughout the 159-phase pipeline. The architecture vtable factory at sub_1CCEEE0 (17 KB, 244 callees) selects which overrides are active based on the sm_version.

Phase 5: ConvertUnsupportedOps -- Instruction Legalization

The most substantial bridge phase. Lowers PTX operations that have no direct SASS equivalent for the target architecture. This phase runs the MercConverter engine (see next section) and handles:

• 64-bit integer arithmetic on architectures with 32-bit ALUs: splits add.s64, mul.lo.s64 into hi/lo 32-bit instruction pairs using carry chains
• Complex addressing modes: decomposes multi-component addresses into separate arithmetic instructions
• PTX-specific operations: converts PTX instructions that have no 1:1 SASS mapping (e.g., bfe, bfi, prmt variants not supported on all targets)
• Architecture availability: gates instructions by SM version (an instruction added in sm_80 is lowered to a multi-instruction sequence on sm_70)
• Texture/surface operations: legalizes texture sampling and surface access patterns (sub_9E8B20, 17 KB)
• Memory operations: legalizes load/store patterns, address register handling (sub_9D76D0/sub_9D80E0, 17--18 KB each)

After ConvertUnsupportedOps completes, every instruction in the IR has a valid SASS opcode for the target architecture.

The late phase 132 (UpdateAfterConvertUnsupportedOps) runs cleanup for edge cases introduced by this phase that are only detectable after optimization.

Phase 6: SetControlFlowOpLastInBB -- CFG Structural Fixup

Enforces a critical structural invariant: control flow operations must be the last instruction in their basic block. If a branch, jump, return, or exit instruction is followed by other instructions in the same block (which can happen during lowering when a PTX instruction expands to a sequence ending in a branch), this phase splits the block at the control flow point.

The invariant is required by the scheduler (which assumes only the last instruction in a block can transfer control) and the register allocator (which computes live-out sets at block boundaries). The phase rewrites the instruction linked list and allocates new 40-byte basic block entries as needed.

Phase 8: OriCreateMacroInsts -- Macro Fusion

Identifies and fuses instruction sequences into macro instructions for hardware efficiency. The phase scans the instruction linked list for patterns that the GPU hardware can execute as a single macro-op:

• Compare + branch: fused into a conditional branch macro instruction
• Multiply + add: fused into FMA where not already (different from PTX fma -- this catches mul followed by add on the same operands)
• Address computation + memory access: fused sequences for coalesced access patterns

The fused macro instructions carry composite semantics in a single IR node. They are expanded back into individual SASS instructions much later at phase 118 (MercExpandInstructions), after scheduling has determined the optimal placement. This late expansion allows the optimizer to treat the fused sequence as atomic, preventing passes from inserting unrelated instructions between the components.

Phase 9: ReportInitialRepresentation -- Diagnostic Dump

Dumps the Ori IR state for debugging, active when DUMPIR or --ftrace diagnostics are enabled. The stats emitter at sub_A3A7E0 prints a per-function profile:

# 142 instructions, 24 R-regs
# [inst=142] [texInst=0] [tepid=0] [rregs=24]
# [est latency = 87] [LSpillB=0]
# [Occupancy = 0.750000]
# [issue thru=0.888889] [fp thru=0.000000]
# [worstcaseLat=87.000000]
# [avgcaseLat=52.500000]
# [SharedMem Alloc thru=0.000000]
# [instHint=0] [instPairs=0]

This snapshot provides the pre-optimization baseline. Comparing it against ReportBeforeScheduling (phase 96) and ReportFinalMemoryUsage (phase 126) shows the optimizer's impact on instruction count, register pressure, and estimated latency.

Phases 10--13: Early Cleanup

The MercConverter Engine

The MercConverter (sub_9F1A90, 35 KB) is the instruction conversion engine at the heart of ConvertUnsupportedOps. Despite its name referencing "Mercury" (NVIDIA's SASS encoding format), it operates purely at the IR level -- converting instruction semantics, not binary encodings.

Call Chain

sub_9F3340 (orchestrator, 7KB)
  |
  +-- sub_9F1A90 (MercConverter main pass, 35KB)
  |     |
  |     +-- sub_9ED2D0 (opcode dispatch, 25KB)
  |     |     |
  |     |     |  Large switch on (*(instr+72)) with byte-1 mask:
  |     |     |    BYTE1(opcode) &= 0xCF  -- strips modifier bits 4-5
  |     |     |
  |     |     +-- 130 explicit cases covering 181 opcode IDs
  |     |     +-- see "Opcode Dispatch Table" below for full map
  |     |     +-- default: emit_opcode(0xFFFF) -- unsupported marker
  |     |
  |     +-- sub_934630 (instruction creation utility, called N times)
  |
  +-- sub_9EF5E0 (post-conversion lowering, 27KB)
        |  string "CONVERTING"
        +-- sub_9EC160, sub_7C11F0, sub_7BFC30 (intrinsic expansion)

Opcode Dispatch Table (sub_9ED2D0)

The dispatch reads *(instr+72), masks byte 1 with &= 0xCF (strips modifier bits 4-5), then switches. v[N] = vtable call at byte offset N*8 from *(a1).

Legend. v[N] = *(*(a1)+N*8)(a1,a2). "emit(X)" = inline replacement opcode with no handler. Cases 15, 52, 54, 72, 97 set v8=1, skipping the default post-dispatch predication call to sub_9CD420. Case 47 (I2F) checks *(*(instr+40)+174) & 1; only calls sub_9E74E0 if set, else falls to default. Case 277 (ACQSHMINIT) inspects operand type: (operand & 7) == 5 selects v[65], otherwise v[11]. Cases 58/148/269 emit replacement opcodes inline (162/45/0xFFFF).

Per-Category Handlers

Intrinsic Descriptor Loading

sub_9EE390 (20 KB) loads architecture-specific instruction descriptions from a file ("IntrinsicDescrFile=%s"). This allows the MercConverter to query which intrinsic operations are natively supported on the target SM and which require multi-instruction expansion. The descriptor file is architecture-versioned and loaded once during the first compilation of a kernel targeting that architecture.

The PTX-to-SASS Opcode Transition

The fundamental semantic transformation during lowering: PTX uses high-level, explicitly-typed opcodes; Ori uses SASS-level opcodes where the type is encoded in the mnemonic. All SASS opcode strings in the binary are ROT13-encoded.

Lowering Rules Reference (95 common PTX instructions)

The table below lists the primary SASS opcode(s) each PTX instruction lowers to during the bridge phases. "1:1" means a single SASS instruction; "1:N" means a multi-instruction expansion whose size depends on type width, rounding mode, or target SM. The lowering phase column indicates where the conversion happens: P = parser inline (Bison reduction action writes the SASS opcode directly), 5 = Phase 5 ConvertUnsupportedOps / MercConverter, 45/78 = later legalization passes. Entries marked with an SM gate are only lowered on architectures that lack native hardware support.

Phase legend: P = parser (Bison reduction action, measured by DAGgen-time). 2 = Phase 2 PromoteFP16. 5 = Phase 5 ConvertUnsupportedOps (MercConverter). 45 = Phase 45 MidExpansion. 78 = Phase 78 LateExpansionUnsupportedOps. Entries marked 45/78 are kept as single Ori instructions through optimization and expanded late to preserve optimization opportunities (LICM, CSE, strength reduction can operate on the unexpanded form).

ROT13 encoding in the binary:

SNQQ  = FADD       VZNQ  = IMAD       SSZN  = FFMA
VNQQ3 = IADD3      QZHY  = DMUL       YQT   = LDG
FGT   = STG        OEN   = BRA        RKVG  = EXIT
ERG   = RET        ONE   = BAR        FGF   = STS

Key semantic differences at the transition:

1. Type moves into the opcode: PTX add.f32 becomes FADD (the "F" encodes float); PTX add.s32 becomes IADD3 (the "I" encodes integer). The type qualifier disappears from the instruction syntax.

2. Register namespace unification: PTX's typed virtual registers (%r for int, %f for float, %rd for 64-bit, %p for predicate) merge into Ori's four register files (R, UR, P, UP) with type tracked in the register descriptor at offset +64.

3. Operand count changes: SASS IADD3 takes 3 source operands where PTX add takes 2 -- the third source defaults to RZ (the hardware zero register). This is handled by the expansion in sub_9E6600.

4. Multi-instruction expansion: Complex PTX operations expand to multiple SASS instructions. A PTX div.f32 may become a Newton-Raphson sequence of RCP + FMUL + correction iterations.

5. Predication mapping: PTX @%p0 instruction maps to an Ori predicate operand in the P register file, attached to the instruction node's predicate slot.

Mapping Tables

Table 1 -- PTX type qualifier to SASS opcode prefix. The type qualifier is consumed during MercConverter dispatch and encoded into the SASS mnemonic prefix. Peephole field 0x7E (slot 126) carries the data type qualifier internally (547 = .f32, 548 = .f64); the ISel field 22 encodes integer sign (99 = unsigned, 101 = signed). The U prefix denotes the uniform-register variant, not a type.

Signed vs. unsigned .s/.u distinction does not change the SASS mnemonic -- both map to I-prefixed instructions. The sign is encoded in a modifier bit (ISel field 22: value 99 for .u, 101 for .s), which the SASS printer emits as .U32 vs no suffix on ISETP.

Table 2 -- PTX state space to SASS memory instruction. The state space qualifier on ld/st selects the SASS opcode suffix. Peephole field 0x119 (slot 281) carries the address space qualifier with enum values 1435--1440. Atomics follow the same suffix pattern (ATOM/ATOMG/ATOMS).

Additional memory instructions: LDSM (load shared matrix, sm_75+), LDGSTS (async global-to-shared, sm_80+), LDGDEPBAR (load with dependency barrier), LDTRAM (load TMEM, sm_100+).

Table 3 -- PTX comparison/rounding modifiers to SASS encoding. Modifiers that survive as SASS instruction suffixes are encoded through the modifier-encode functions in the 0x10B region. Field 345 (slot 0x159) selects rounding; field 150 (slot 0x096) selects comparison mode. The SASS printer emits these via vtable[1760] (rounding) and vtable[1392]/vtable[3528] (comparison).

Rounding modes .rz and .rm share encoding value 1515 because the hardware distinguishes them by a secondary bit in field 301 (1520 vs 1521). The PTX comparison operators .lo .ls .hi .hs are unsigned aliases for .lt .le .gt .ge and produce identical SASS encoding.

Error Detection During Lowering

The bridge phases include two error detection mechanisms:

Internal compiler error assertion (sub_9EB990, 1.4 KB): three references to "Internal compiler error.". Called when a bridge phase encounters an impossible IR state (e.g., an opcode value outside the known range in the MercConverter dispatch switch). Triggers longjmp-based fatal abort via sub_42F590 back to the driver's error recovery point in sub_446240.

Uninitialized register detector (sub_A0B5E0, 7 KB): "Found %d potentially uninitialized register(s) in function %s". Walks the instruction list per block, checks register descriptor flags at offset +48 (bit 5 = "defined"). Reports registers that appear as sources without any prior definition. This detector fires after the bridge phases to catch conversion errors that leave registers undefined.

Key Data Structures

Instruction Node

Instruction (variable size, linked list node)
  +0     prev_ptr           // doubly-linked list: previous instruction
  +8     next_ptr           // doubly-linked list: next instruction
  +16    child_ptr          // child/expanded instruction chain
  +32    control_word_ptr   // set later during scheduling (initially NULL)
  +72    opcode             // byte 0: primary opcode
                            // byte 1 bits 4-5: modifier (masked with 0xCF)
  +80    operand_count      // number of operands
  +84    operand_array      // packed operand descriptors

Operand Encoding

Each operand is a packed 32-bit value:
  Bits 28-30: operand kind ((value >> 28) & 7)
    1 = register operand
    5 = predicate register
    (other values for immediate, constant bank, label, etc.)

  Lower bits: operand-kind-specific payload (register ID, immediate value, etc.)

Register Descriptor

Register descriptor (accessed via *(ctx+88) + 8*regId)
  +12    register number (int)
  +48    flags (bit 5 = "defined", other bits for liveness state)
  +64    type (3=address, 6=GPR, 7=predicate)

Timing Boundary

The lowering spans two --compiler-stats timer phases:

The boundary between "lowering" and "optimization" is therefore between phase 13 (GeneralOptimizeEarly, the last bridge phase) and phase 14 (DoSwitchOptFirst, the first pure optimization). After phase 13, the IR is in its final SASS-opcode form with validated structure, ready for the main optimization pipeline.

Cross-References

• PTX Parser -- Flex scanner + Bison LALR(1) parser (the source of raw Ori IR)
• Ori IR -- IR design: Code Object, basic blocks, instruction format, register files
• Optimization Pipeline -- 159-phase pipeline (phases 0--13 are the bridge)
• Phase Manager -- PhaseManager object, phase factory, dispatch loop
• Optimization Levels -- NvOpt levels 0--5 and their effect on recipes
• SASS Opcodes -- target SASS instruction set after lowering

Function Map