Optimization Pipeline (159 Registered / 157 Dispatched)

Addresses apply to ptxas v13.0.88 (CUDA 13.0). VA base 0x400000 (non-PIE).

The ptxas optimizer is a fixed-order pipeline that transforms Ori IR from its initial post-lowering form into scheduled, register-allocated SASS machine code.

• Registered = 159, dispatched = 157. The PhaseManager registers 159 phase objects (IDs 0–158), but the default driver dispatches exactly 157 of them (IDs 0–156) in a fixed identity order. Both numbers are real and describe different objects:
  • 159 is the size of the phase-name registry and the inclusive ceiling for explicit phase selection.
  • 157 is the length of the default phase-schedule the dispatch loop actually walks.
• The two trailing slots — ID 157 DebuggerBreak and ID 158 NOP — are constructed but never scheduled on a default compile; they enter a schedule only through the recipe/named-phases override path (OCG knob 298).
• No dependency scheduling. Unlike LLVM's PassManager — which uses dependency-driven scheduling and analysis preservation — ptxas dispatches every scheduled phase unconditionally and in a predetermined order; each phase decides internally (via opt-level, knob, and predicate checks inside its own execute() body) whether to do anything.
• Architecture-specific behavior is injected through "AdvancedPhase" hook points whose vtables are overridden per target.

Each phase is a polymorphic C++ object exactly 16 bytes in size, allocated from a memory pool by a 159-case factory switch. The PhaseManager constructs all 159 phase objects up front during initialization, stores them in a flat array, and iterates a separate 157-entry order array in a simple dispatch loop. Per-phase timing and memory consumption are optionally tracked for --stat=phase-wise output.

157 vs 159 — definitive. The order accessor sub_C60D20 returns a pair: the order-table pointer (0x22BEEA0) in rax and the count 0x9D = 157 in rdx. Hex-Rays drops rdx because the inferred prototype returns a single pointer, so the C view shows only return &unk_22BEEA0;. The driver passes (ptr, count) straight into the dispatch loop sub_C64F70, whose bound is &schedule[count] with count = 157. The order table at 0x22BEEA0 is the identity array [0..156] followed by unrelated rodata; sub_C62720 sizes the name registry at 159 (*((_DWORD*)PM+27) = 159, copies 1272 = 159*8 bytes from off_22BD0C0). Registered = 159; dispatched by default = 157.

Key Facts

Phase Object Layout

Every phase is a 16-byte polymorphic object created by the factory:

struct Phase {
    void** vtable;       // +0: pointer to phase-specific vtable in .data.rel.ro
    void*  allocator;    // +8: memory pool used for allocation
};

The vtable provides three virtual methods common to all phases:

Vtable slots +24 and +32 are NULL in all 159 vtable instances. Memory allocation uses standalone pool_alloc (sub_424070) / pool_free (sub_4248B0), not vtable dispatch.

Dispatch Loop

The dispatch loop at sub_C64F70 drives execution. The key fact: execute() runs unconditionally on every entry of the 157-element order array; isNoOp() gates only whether the "Before "/"After " diagnostic banner is printed, not whether the phase body runs. A phase that should do nothing returns early inside its own execute() body (an opt-level, knob, or predicate check), not by being skipped here.

// sub_C64F70 -- simplified
void dispatch(PhaseManager* pm, int* order, int count /* = 157 */) {
    MemorySnapshot baseline = take_snapshot();

    for (int i = 0; i < count; i++) {
        int idx = order[i];
        Phase* phase = pm->phase_list[idx];

        const char* name = pm->name_table[phase->getIndex()];   // vtable+8

        MemorySnapshot before = take_snapshot();                // always

        // isNoOp() (vtable+16) gates ONLY the diagnostic banner, checked
        // once before execute and once after. It never skips execute().
        if (!phase->isNoOp())
            diagnostic("Before " + name);

        phase->execute(pm->compilation_unit);                   // vtable+0, ALWAYS

        if (!phase->isNoOp())
            diagnostic("After " + name);

        if (pm->timing_enabled)
            report_phase_stats(pm, name, &before);              // unconditional
    }

    if (pm->timing_enabled) {
        report_summary(pm, "All Phases Summary", &baseline);
        report_pool_consumption(pm);
    }
}

Because execute() always runs and the timing record + pre-snapshot are written before the first isNoOp() check, --ftime output contains a row for every dispatched phase, including phases that immediately early-returned; gated-off phases simply show near-zero elapsed time rather than being omitted.

Timing output format (to stderr when --stat=phase-wise):

  <phase_name>  ::  [Total 42 KB ]   [Freeable 8 KB ]   [Freeable Leaked 0 KB ] (0%)

Complete Phase Table

Group 1 — Initial Setup (phases 0–13)

Program validation, recipe application, FP16 promotion, control flow analysis, macro instruction creation.

Phase 0 validates the initial Ori IR for structural correctness. Phase 1 applies NvOptRecipe transformations (controlled by option 391, which allocates a 440-byte sub-manager at PhaseManager+56). Phase 2 promotes FP16 operations where profitable. Phases 4 and 7 are architecture hooks that bracket ConvertUnsupportedOps — backends override them to inject target-specific pre/post-legalization logic.

Group 2 — Early Optimization (phases 14–32)

Branch optimization, loop canonicalization, strength reduction, software pipelining, SSA formation.

The GeneralOptimize* phases (13, 29, 37, 46, 58, 65) are compound passes that bundle multiple small optimizations (copy propagation, constant folding, algebraic simplification) into a single fixed-point iteration. They appear at multiple pipeline positions to re-clean the IR after major transformations. Liveness/DCE also runs repeatedly (OriPerformLiveDead at phases 16, 33, 61, 84) to remove dead code exposed by intervening passes.

Group 3 — Mid-Level Optimization (phases 33–52)

GVN-CSE, reassociation, shader constant extraction, CTA expansion, argument enforcement.

Shader constant extraction (phases 34, 51) identifies uniform values that can be loaded from constant memory rather than recomputed per-thread. GvnCse (phase 49) combines global value numbering with common subexpression elimination in a single pass. The MidExpansion (phase 45) performs target-dependent lowering of operations that must be expanded before register allocation but after high-level optimizations have had their chance.

Group 4 — Late Optimization (phases 53–77)

Predication, rematerialization, loop fusion, varying propagation, sync optimization, phi destruction, uniform register conversion.

Predication (phase 63) converts short conditional branches into predicated instruction sequences, eliminating branch divergence. Rematerialization runs twice (phases 54 and 69) — the early pass targets values that are cheap to recompute, while the late pass handles cases exposed by predication and loop fusion. Phase 73 (ConvertAllMovPhiToMov) destroys SSA form by converting phi nodes into move instructions, preparing the IR for register allocation. Phase 74 converts qualifying values to uniform registers (UR), reducing general register pressure.

Group 5 — Legalization (phases 78–96)

Late unsupported-op expansion, backward copy propagation, GMMA fixup, register attribute setting, final inspection.

GMMA (phases 85, 87) handles WGMMA (warp group matrix multiply-accumulate) instruction sequences that require specific register arrangements and ordering constraints. OriSetRegisterAttr (phase 90) annotates registers with scheduling attributes (latency class, bank assignment) consumed by the downstream scheduler. FinalInspectionPass (phase 94) is a validation gate that catches illegal IR patterns before the irreversible scheduling/RA phases.

Group 6 — Pre-Scheduling and Register Allocation (phases 97–103)

Synchronization insertion, WAR fixup, register allocation, 64-bit register handling.

Phase 99 inserts the synchronization instructions (BAR, DEPBAR, MEMBAR) required by the GPU memory model. Phase 100 fixes write-after-read hazards exposed by sync insertion. Register allocation is driven through the hook at phase 101 — the actual allocator is architecture-specific and invoked from the AdvancedPhase override. Phase 103 splits 64-bit register pairs into their 32-bit components for architectures that require it.

Group 7 — Post-RA and Post-Scheduling (phases 104–116)

Post-expansion, NOP removal, hot/cold optimization, block placement, scoreboards.

Highlights of this group:

• Hot/cold partitioning (phases 108–109) separates frequently executed blocks from cold paths, improving instruction cache locality.
• PlaceBlocksInSourceOrder (phase 112) determines the final layout of basic blocks in the emitted binary.
• Scoreboard sub-system — two paths by opt level:
  • At -O1 and above, AdvancedScoreboardsAndOpexes (phase 115) performs full dependency analysis to compute the 23-bit control word per instruction (4-bit stall count, 1-bit yield, 3-bit write barrier, 6-bit read barrier mask, 6-bit wait barrier mask, plus reuse flags).
  • At -O0, phase 115 is a no-op and ProcessO0WaitsAndSBs (phase 116) inserts conservative waits.

Group 8 — Mercury Backend (phases 117–122)

SASS instruction encoding, expansion, WAR generation, opex computation, microcode emission.

"Mercury" is NVIDIA's internal name for the SASS encoding framework. The six phases run in order:

• Phase 117 converts Ori instructions into Mercury's intermediate encoding, then decodes them back to verify round-trip correctness.
• Phase 118 expands pseudo-instructions into their final SASS sequences.
• WAR generation runs in two passes (119, 121) because expansion in phase 118 can introduce new write-after-read hazards.
• Phase 120 generates "opex" (operation extension) annotations.
• Phase 122 produces the final SASS microcode bytes.

The MercConverter infrastructure (sub_9F1A90, 35KB) drives the instruction-level legalization using a visitor pattern dispatched through a large opcode switch (sub_9ED2D0, 25KB).

Group 9 — Post-Mercury (phases 123–131)

Register map, diagnostics, debug output.

CalcRegisterMap (phase 124) computes the final physical-to-logical register mapping emitted as EIATTR metadata in the output ELF. DumpNVuCodeText and DumpNVuCodeHex (phases 129–130) produce the human-readable SASS text and raw hex dumps used by cuobjdump and debugging workflows. DebuggerBreak (phase 131) is a development-only hook that triggers a breakpoint when a specific phase is reached.

Group 10 — Finalization (phases 132–158)

Late merge operations, late unsupported-op expansion, high-pressure live range splitting, architecture-specific fixups.

Phases 132–138 handle late-breaking transformations that must run after the Mercury backend but before finalization. OriSplitHighPressureLiveRanges (phase 138) is a last-resort live range splitter that fires when register pressure exceeds hardware limits after the main allocation pass.

Phases 139–158 are 20 additional registered slots:

• All named. They are named in the static name table at off_22BD0C0, which holds all 159 entries (indices 0–158). There is no "139 named + 20 unnamed" split; every phase ID resolves to a name through the table.
• IDs 139–156 (e.g. ProcessO0WaitsAndSBs=139 … DumpNVuCodeHex=156) are part of the default 157-entry schedule.
• IDs 157 (DebuggerBreak) and 158 (NOP) are registered but dispatched only via the recipe path.
• Tail vtables run off_22BEB08..off_22BEE78.

Optimization Level Gating

AdvancedPhase Hook Points

Sixteen phases serve as conditional extension points. Their isNoOp() returns true by default — which suppresses their Before/After banner but does not remove them from the schedule: their execute() still runs every compile and returns early until a backend installs a non-stub override. Architecture backends and optimization-level configurations override the vtable (or the per-architecture profile slot the stub dispatches to) to activate these hooks:

The pattern is consistent: AdvancedPhase hooks bracket major pipeline stages, allowing backends to insert target-specific transformations without altering the fixed phase ordering. Phase 101 (AdvancedPhaseAllocReg) is notable because register allocation itself is entirely driven through this hook — the base pipeline has no hardcoded allocator.

O0 vs O1+ Behavior

At -O0, the pipeline skips most optimization phases via their individual isNoOp() checks. The critical difference is in scoreboard generation:

• -O1 and above: Phase 115 (AdvancedScoreboardsAndOpexes) runs the full dependency analysis using sub_A36360 (52KB control word encoder) and sub_A23CF0 (54KB DAG list scheduler heuristic). Phase 116 is a no-op.
• -O0: Phase 115 is a no-op. Phase 116 (ProcessO0WaitsAndSBs) inserts conservative stall counts and wait barriers — every instruction gets the maximum stall, and barriers are placed at every potential hazard point. This produces correct but slow code.

Individual phases also check the optimization level internally via the compilation context. The scheduling infrastructure (sub_8D0640) reads the opt-level via sub_7DDB50 and selects between forward-pass scheduling (opt-level <= 2, register-pressure-reducing) and reverse-pass scheduling (opt-level > 2, latency-hiding).

The OCG-Knob Mechanism

Every control-flow decision the optimizer driver makes is gated by an OCG knob. An OCG knob is one 72-byte entry in a flat array at config+0x48, where config is the OCG config object reached as ctx+0x680 (equivalently the options view ctx+0x1664). The config offset of knob N is exactly N*72 (0x48-relative). Each 72-byte entry has a presence/type tag byte at entry[0] (0 = unset, 5 = "set, value present") and a value pointer at entry[8] (read when the tag is 5).

Two inlined leaf accessors recur, compared by address throughout the driver:

When the live accessor object's vtable slot does not match these fast leaves, the driver does a virtual call (*slot)(obj, idx) with the knob index. The knobs the driver and optimizer actually consult:

All offsets id*72 are byte-exact (298*72 = 0x53D0, 391*72 = 0x6DF8, 249*72 = 0x4608, 499*72 = 0x8C58, remainder 0).

Recipe / NamedPhases Override (OCG Knob 298)

The recipe mechanism allows replacement of the default 157-phase schedule with a custom sequence, used for debugging and engineering investigation. It is gated by an internal knob, not a documented user-facing ptxas flag; default compiles never reach it.

Activation

The pipeline orchestrator (sub_7FB6C0) reads OCG knob 298 (presence flag at config+0x53D0). When clear (the default), it builds the PhaseManager and dispatches the default 157-entry schedule. When set, it delegates to sub_9F63D0, which builds a custom schedule and feeds it to the same dispatch loop:

// sub_7FB6C0 -- simplified
void orchestrate(Context* ctx) {
    ocg = *(ctx + 0x680);                       // OCG knob/config object
    if (knob_active(ocg, 298)) {                // config+0x53D0 != 0
        recipe_run(ctx);                        // sub_9F63D0 (custom schedule)
    } else {
        PhaseManager pm;
        PhaseManager_construct(&pm, ctx);       // sub_C62720 -> registers 159
        auto [order, count] = get_default_order(); // sub_C60D20 -> (&0x22BEEA0, 157)
        dispatch(&pm, order, count);            // sub_C64F70 -> runs IDs 0..156
        PhaseManager_destroy(&pm);              // sub_C61B20
    }
    // ... teardown ctx+1880 / +1872 / +1864 scratch objects ...
}

Recipe path and DSL

sub_9F63D0 builds the PhaseManager (same sub_C62720), seeds a 256-int buffer with [0] = 158 (NOP), then calls sub_9F4040 to compute a custom schedule and length. sub_9F4040 starts from the default 157-identity schedule (sub_C60D20()), tokenizes the recipe string (sub_798B60) into parallel name/value arrays, and applies a small DSL. Explicit phase IDs are clamped to [0, 159] (if (v121 > 159) v121 = 159;), and unknown phase names map to 158/NOP via sub_C641D0. This is the only path that can schedule IDs 157/158.

OCG knob 297 is pulsed (write-style virtual call) once per applied recipe element — a diagnostic "recipe-applied" event/counter. Every value is parsed with strtol and bounds-clamped to [0, 256] or [0, 159].

NvOptRecipe (Knob 391) — the user-facing counterpart

When OCG knob 391 is present, the PhaseManager ctor reads its value (391*72+8 = +0x6E00) and constructs a 440-byte recipe-apply object, stashing the recipe pointer at its +312. That object is consumed by phase ID 1, ApplyNvOptRecipes — the second phase in the default schedule. This is the supported path for applying recipe transforms without diverting the whole pipeline.

Pass-Disable Integration

Individual passes can be disabled without reordering the pipeline. The check function sub_799250 (IsPassDisabled, 68 bytes) performs a case-insensitive substring match against the PTXAS_DISABLED_PASSES string at context offset 13328:

// sub_799250 -- simplified
bool is_pass_disabled(Context* ctx, const char* pass_name) {
    if (ctx->pass_disable_flag == 0) return false;  // offset 13320
    if (ctx->pass_disable_flag == 5) {
        return strcasestr(ctx->pass_disable_string, pass_name);  // offset 13328
    }
    return false;
}

This check is called from 16+ sites across the codebase, guarding passes like LoopMakeSingleEntry and SinkCodeIntoBlock. A more thorough variant (sub_7992A0, IsPassDisabledFull) uses FNV-1a hashing for function-specific override tables.

PhaseManager Data Structures

PhaseManager Object (~112 bytes)

Offset  Type       Field
------  ----       -----
+0      int64      compilation_unit pointer
+8      int64*     allocator
+16     void*      sorted_name_table (for binary search)
+24     int32      sorted_name_count
+28     int32      sorted_name_capacity
+32     int64*     allocator_copy
+40     void*      phase_list (array of 16-byte Phase entries)
+48     int32      phase_list_count
+52     int32      phase_list_capacity
+56     int64      nvopt_recipe_ptr (NvOptRecipe sub-manager, or NULL)
+64     int64      (reserved)
+72     bool       timing_enabled (from options[17928])
+76     int32      (flags)
+80     bool       flag_byte
+88     int64*     timing_allocator
+96     void*      phase_name_raw_table
+104    int32      phase_name_raw_count
+108    int32      phase_name_raw_capacity

Timing Record (32 bytes)

Offset  Type       Field
------  ----       -----
+0      int32      phase_index (-1 = sentinel)
+8      int64      phase_name_or_magic (0x2030007 = sentinel)
+16     int64      timing_value
+24     int32      memory_flags

NvOptRecipe Sub-Manager (440 bytes, at PhaseManager+56)

Created when option 391 is set. Contains timing records with 584-byte stride, a hash table for recipe lookup, sorted arrays, and ref-counted shared lists. The sub-manager inherits the phase chain from the previous execution context, enabling recipe-based pipeline modification across compilation units.

Function Map

Diagnostic Strings

Cross-References

• Phase Manager Infrastructure — detailed PhaseManager internals
• Pass Inventory & Ordering — per-pass documentation index
• GeneralOptimize Bundles — compound optimization passes
• Scheduler Architecture — scheduling phases 97–116
• Mercury Encoder — Mercury backend phases 117–122
• Scoreboards & Dependency Barriers — control word generation
• Optimization Levels — O-level gating details
• DUMPIR & NamedPhases — NamedPhases configuration reference
• Memory Pool Allocator — pool allocator used by phase objects