XStat Metadata IDs

All addresses, offsets, and string VAs on this page apply to libtpu.so from the libtpu-0.0.40-cp314 wheel (build-id 89edbbe81c5b328a958fe628a9f2207d). The binary ships full C++ symbols; .text/.rodata VMA == file offset. Other versions will differ.

Abstract

An XStat is the annotation primitive of the profiler's XSpace: a typed key/value pair that hangs off an XEvent (or off an XLine/XPlane). The key is an XStatMetadata.id — an index into the owning XPlane's stat_metadata map — and the value is one of six protobuf oneof arms (double/uint64/int64/string/bytes/ref). This page is the stat catalog: the interned XStatMetadata name strings libtpu attaches to TPU device and host events, grouped by category, each with its value type and the trace-point or payload field that produces it. It is the sibling of XEvent Metadata IDs, which catalogs the event dictionary — stats annotate events, so the two dictionaries are disjoint and live in separate maps on the same plane.

The shape maps onto a frame an LLVM/MLIR engineer owns. If an XEvent is an instruction, an XStat is an operand-attribute: metadata_id is the attribute kind (interned once per plane, like an MLIR attribute name), and the oneof value is the attribute payload. As with XEvent Metadata IDs, there is no single global stat-id enum — XStat.metadata_id is a plane-local interning key, and the same integer means different things on different planes. What is stable is the name: the canonical tsl StatType enum (GetStatTypeStrMap()::stat_type_str_map) provides the well-known names, and XPlaneBuilder::GetOrCreateStatMetadata(name) interns each on first sight, assigning the next free plane-local id. So this page catalogs by name + value-type + producer, not by fixed integer — exactly as the TraceEntry to XEvent page catalogs the per-subscriber stat emission.

Two stats are special and are the only ones with a de facto fixed integer in this binary: device_offset_ps (StatType 147) and device_duration_ps (StatType 148), the absolute device-time pair that the universal TpuXLineBuilder::AddEvent(GtcSpan) stamps on every device XEvent. Their XStatMetadata* are cached at TpuXLineBuilder+0x38/+0x40 at line-builder construction; every other stat is added dynamically through the generic XStatsBuilder::AddStatValue<T> path, with the oneof arm selected by the C++ value type T.

This page is a reference catalog. It covers: the XStatMetadata model and the six value arms; how T selects the arm at the AddStatValue call site; the universal device_*_ps pair; the per-subscriber dynamic stats (sync wait reason, DMA byte count, counter sample); the canonical tsl StatType name catalog grouped by category; and the protobuf-message-valued stats (CoreDetails, PowerComponentMetrics) that ride the bytes arm.

The XStatMetadata Model

Purpose

XStatMetadata is the per-plane dictionary entry that types a stat. An XStat carries only an integer metadata_id and a value; the human name and description live once in XStatMetadata, keyed by that id in the plane's stat_metadata map. This is the same intern-once / reference-by-id design as XEventMetadata, and for the same reason: a 64-byte HLO-op name or a 34-byte wait-reason string is interned once per plane and every event that needs it references the small integer id.

Schema

Byte-confirmed proto2 layout (from xplane.proto, identical structure across host and device planes):

message XStatMetadata { int64 id = 1; string name = 2; string description = 3; }

message XStat {
  int64 metadata_id = 1;            // KEY into XPlane.stat_metadata
  oneof value {
    double  double_value = 2;       // oneof case 2
    uint64  uint64_value = 3;       // oneof case 3   (also used for ref by tsl convention)
    int64   int64_value  = 4;       // oneof case 4
    string  str_value    = 5;       // oneof case 5
    bytes   bytes_value  = 6;       // oneof case 6
    uint64  ref_value    = 7;       // oneof case 7   (an interned StatMetadata id)
  }
}

GOTCHA — the XStat oneof case discriminant numbers (2..7) are field numbers, and they do not line up with the integer the AddStatValue decompilation writes into the discriminant slot. The TraceEntry to XEvent page records the runtime discriminant values byte-confirmed at the call sites: int64 writes case 3, uint64 writes case 2, ref writes case 7. The proto field numbering (int64=f4, uint64=f3) and the in-memory oneof discriminant (int64→3, uint64→2) are two different encodings of the same arm; a reimplementer must not assume they are equal. The catalog below lists the proto field number; cross-reference the trace-entry page for the in-memory discriminant.

The ref_value indirection

ref_value (f7) is the one non-obvious arm: its value is another XStatMetadata.id on the same plane, used when a stat's value is itself an interned string (so the string is stored once in stat_metadata and many events point at it by id rather than each carrying a copy). The sync wait-reason stat uses this — see the wait-reason row. The AddStatValue<XStatMetadata&> instantiation (AddStatValueIRNS3_13XStatMetadataEEE, confirmed symbol) is exactly this path: pass an interned-string XStatMetadata&, get a ref_value arm pointing at its id.

Value Type Selection — AddStatValue<T>

Purpose

A stat's value arm is not chosen by a per-name table — it is chosen by the C++ type T of the argument at the XStatsBuilder<XEvent>::AddStatValue<T> call site. The template is instantiated once per value type used in the binary, and each instantiation hard-codes the oneof arm it writes. This is the single most important fact for reimplementing the stat layer: to know an arm, look at the call-site type, not the name.

Confirmed instantiations

Eight AddStatValue<T> instantiations are present as mangled symbols on the XStatsBuilder<tensorflow::profiler::XEvent> (per-event) and XStatsBuilder<XEventMetadata> (per-event-type) builders. The arm each writes:

NOTE — the two message-typed instantiations (CoreDetails, PowerComponentMetrics) serialize the protobuf message into the bytes arm. There is no dedicated arm for sub-messages; a structured stat is a serialized blob in bytes_value (f6), the same arm the device-event TraceEntry payload uses in XEventMetadata.metadata. A reimplementer must serialize these to bytes, not invent a new arm.

QUIRK — there is an XStatsBuilder<XEvent>::CopyStatValue(XStat&, XPlane&, XStat*) symbol — stat copy during plane merge re-interns the source stat's metadata name into the destination plane and rewrites metadata_id. This is why stat ids are not stable across a merge but stat names are: merge keys on the name, not the integer. Build the catalog on names.

The Universal Device-Time Pair

Purpose

Two stats are stamped on every device XEvent that carries timing, unconditionally (subject only to both metadata pointers being interned). They are the only stats with a fixed StatType integer in this binary, because the line builder caches their XStatMetadata* at construction rather than looking them up by name per event.

The pair

Both carry absolute device wall time in picoseconds, computed by the GTC→ps math (round(gtc_ticks × 1e9 / (clock × 16)), 128-bit). They duplicate the XEvent.offset_ps/duration_ps fields, which are line-relative. The full mechanism — why both exist, the conditional gate, the 128-bit divide — is documented on TraceEntry to XEvent. What this page adds: these two are the catalog's anchor row, the only stat names whose integer id is observable in the binary (as the +0x28 offset into the cached XStatMetadata written into XStat.metadata_id).

GOTCHA — a TpuXLineBuilder constructed without both StatTypes 147/148 interned (e.g. a counter-only line) emits events with line-relative timing but no device_*_ps stats. The pair is conditional on this+0x38 && this+0x40 being non-null. Intern both at line-builder construction or lose the absolute timebase on every event.

Sync and Control Stats

Purpose

The stateful sync subscriber surfaces why a wait happened and which sync flag, as dynamic stats on the SyncWait:<n> duration event (begin/end pairing). These are the highest-value TPU-specific device stats.

NOTE — for the id==255 dummy/sentinel sync entry the wait-reason XStatMetadata.name is the empty string (len 0) — the stat is still emitted, but its interned name is empty, so a consumer sees a ref_value arm pointing at a nameless stat. A reimplementation that asserts non-empty stat names will trip on the sentinel.

Memory / DMA Stats

Purpose

The DMA subscriber surfaces the transferred byte count of a matched DMA span; the OCI/HDE/ICI bands surface byte counts and chunk counts read straight from the decoded TraceEntry variant fields. These ride the uint64/int64 arms.

NOTE — the DMA match key (dma_id, from GetDmaId() @ 0xf698180) is the pairing key, not a stat — it is never interned into stat_metadata. The string "dma_id" is absent from the binary (confirmed MISS), consistent with it being an in-memory FlatHashMap key rather than a serialized stat name. Do not look for a dma_id stat; the surfaced DMA scalar is the byte count.

Counter / Power / Throttle Stats

Purpose

Counter lines and the firmware power/thermal subsystem surface samples as stats. The counter sample uses a fixed rodata label; power components ride the message-valued bytes arm.

NOTE — the counter-sample line uses a constant rodata stat name ("Available Count"), interned via GetOrCreateStatMetadata(string_view), in contrast to the canonical-tsl-named stats above. Fixed-label stats and tsl-enum stats share the same stat_metadata map — they differ only in where the name string originates (a rodata literal vs the GetStatTypeStrMap table).

HLO / XLA-Op Stats

Purpose

The HLO subscriber and the post-hoc symbolizer attach compiler-side identity to XLA-op events on the "XLA Ops" line. These are the canonical tsl names, all string-valued except the numeric ids.

NOTE — these stat names are confirmed present as ASCII in the binary, but the exact subscriber→stat-emission for each on each chip family was not byte-traced — only the hlo_op/hlo_module path (via GetOrCreateXlaEventMetadata) and the symbolizer enrichment are confirmed. Treat the value-arm assignment of the canonical-tsl rows as derived from the tsl StatType schema (which fixes the arm per name), HIGH where a producer is named, LOW where only the string is confirmed present.

Identity / Host / Provenance Stats

Purpose

Plane- and host-level identity stats, plus the build-provenance stat. These appear on /host:<n> planes and as plane-level stats; they are interned the same way but are largely host-side (dynamic name-interning through tsl::profiler::XPlaneBuilder).

QUIRK — num_occurrences is also an XEvent oneof data field (f5), not only a stat. An aggregated XEvent (one event-type, many instances folded) stores its count in XEvent.num_occurrences; the num_occurrences stat is the per-event-type variant. A reimplementer must not store the same count in both places — pick the event field for aggregated events and the stat for type-level aggregation.

Stat Categories — At a Glance

The canonical tsl StatType enum (seeded by GetStatTypeStrMap()::stat_type_str_map) plus the TPU-specific names group into these axes. Rather than dump the full ~150-entry tsl enum, the table gives the category, its dominant value arm, and the producing subsystem:

NOTE — every name above is confirmed present as ASCII in libtpu.so. The value arm per name is fixed by the tsl StatType schema (a name has one arm by convention), so the arm column is HIGH confidence even where the per-family producer is only MEDIUM/LOW. The integer ids of the canonical tsl names are not recoverable from strings — they are assigned at build time by the tsl enum and are documented by name + arm + producer, not by fixed integer, exactly as on XEvent Metadata IDs. Only device_offset_ps (147), device_duration_ps (148), and the provenance stat (166) have confirmed integers.

Relevant Struct Offsets

XStatMetadata (proto2)  +0x00  id (field 1)        — the plane-local stat id
                        (name field 2, description field 3 follow)
                        consumers read id at metadata_ptr + 0x28 when stamping XStat

XStat (proto2)          +0x18  metadata_id  (= XStatMetadata + 0x28)
                        +0x20  value (union storage)
                        +0x28  oneof discriminant (2 uint64 / 3 int64 / 7 ref / double)
                               (in-memory discriminant; NOT the proto field number)

TpuXLineBuilder         +0x38  device_offset_ps   XStatMetadata*  (StatType 147)
                        +0x40  device_duration_ps XStatMetadata*  (StatType 148)
                               AddEvent(GtcSpan) gates the pair on both being non-null

Related Components

Cross-References

• Profiling and Telemetry Overview — the five-stage capture pipeline these stats annotate
• XEvent Metadata IDs — the event dictionary; stats annotate events, the two maps are disjoint
• TraceEntry to XEvent — the payload-scalar → XStat emission, the universal device_*_ps stamp, and the in-memory oneof discriminants
• XPlaneBuilder / XStat / TraceMe — the builder primitives (AddStatValue, GetOrCreateStatMetadata) this catalog drives
• TracePoints Master Registry — the trace_point_id enum whose events these stats hang off