protodesc_cold Catalog

All addresses on this page apply to libtpu.so from the libtpu-0.0.40-cp314 wheel (build-id 89edbbe81c5b328a958fe628a9f2207d). Other builds renumber the section and rehash the descriptor symbols.

Abstract

protodesc_cold is a 3.2 MiB read-only ELF section (0xbe8af30–0xc1bf0b0) that holds the serialized google.protobuf.FileDescriptorProto blob for every protobuf schema statically linked into libtpu.so. There is exactly one blob per compiled .proto file. The protobuf C++ codegen emits, for each foo.proto, a descriptor_table_protodef_<hash> byte array (the serialized schema, parked here in the cold section) plus a descriptor_table_<hash> registration struct; a _GLOBAL__sub_I_ constructor in .init_array walks these tables at process start and calls into the descriptor-pool builder so the reflection layer can later resolve message types by name. This page is the consolidated catalog of those blobs: the section facts, the per-entry record layout, and the full taxonomy of the schema set by source root and by domain.

The count is exact and byte-anchored. The symbol table carries 760 descriptor_table_protodef_* blob symbols and 760 matching descriptor_table_* registrar symbols — so there are 760 file descriptors in the pool. The section additionally contains 769 distinct .proto path strings; the nine-string excess over 760 is the set of .proto names that appear only inside some descriptor's dependency (import) list, i.e. imported files whose own descriptor is reached transitively. This is the schema surface a reimplementer would need to regenerate the entire reflection database the runtime expects: it spans the XLA/HLO compiler, the deepsea TPU ISA for five chip families, the TPU runtime topology and program format, the XPlane/xprof profiler, Megascale collectives, PJRT distributed coordination, and a long tail of Google-internal infrastructure pulled in by transitive imports.

The schema set is the reverse-engineer's index to the binary's intent. A flag that appears as a field in tpu_compilation_environment.proto is a flag the compiler implements; an entry_point oneof arm in tpu_sequencer_program.proto names a chip family the runtime can load; a TransferType enum value in megascale_info.proto names a collective the runtime can route. The catalog is therefore consulted alongside the TpuCompilationEnvironment and tpu_telemetry.proto pages, which reconstruct individual high-value schemas in full.

This is an appendix reference page: it consolidates counts and representative names rather than reconstructing message graphs. For the section's place in the ELF layout and the registration mechanism, see Custom Sections, which owns the section header.

NOTE — the section header size is 0x334180 = 3,359,104 bytes, which is 3.20 MiB (3359104 / 1024 / 1024). This page and Custom Sections both use the 3.20 MiB figure.

Entry / Record Layout

protodesc_cold is not a self-describing container — it carries no header, no count, and no per-entry index. It is a flat concatenation of independent byte arrays, each one a serialized FileDescriptorProto. The structure lives entirely in the symbol table and in the .init_array registrars, not in the section bytes.

How an entry is reached

.init_array
  └─ _GLOBAL__sub_I_<tu>.cc          ── one per translation unit
       └─ AddDescriptors(&descriptor_table_<hash>)   ── protobuf runtime call
            descriptor_table_<hash> {               ── in .data.rel.ro
              once_flag,                              ── std::once init guard
              size,                                   ── byte length of the blob
              const char* protodef  ───────────────┐ ── points INTO protodesc_cold
              const DescriptorTable** deps, ...      │
            }                                        │
  protodesc_cold:                                    │
    [ descriptor_table_protodef_<hash> ]  <──────────┘  serialized FileDescriptorProto

Each blob is a standard length-delimited protobuf message. Because every FileDescriptorProto begins with field 1 (name, a string), the first thing in nearly every blob is the proto's own path — the .proto filename — which is why scanning the section for .proto-suffixed strings recovers the catalog directly. Subsequent strings in the same blob are the package, the dependency import paths, message and field names, and the syntax marker ("proto2", "proto3", or absent/"editions").

Record-layout table

NOTE — the <hash> in descriptor_table_protodef_<hash> is content-derived, not a source path. It is stable for a given .proto content + protobuf version but is not human-meaningful; to map a blob to its file you read field 1 of the blob, not the symbol name. The first blob's symbol ..._zzRDQFgX_23 resolves to learning/45eac/research/pjrt/pjrt_tpu_topo_desc_name_mapping.proto.

GOTCHA — counting .proto strings in the section overcounts by exactly the number of import-only paths (769 vs 760). The authoritative entry count is the number of descriptor_table_protodef_* symbols (760), each of which is a file; a path that appears only as another file's dependency has a string in the section but no blob of its own here. Drive the count off the symbol table, not the string scan.

Syntax dialects

The pool deliberately mixes three proto dialects — each FileDescriptorProto carries its own syntax field, so the builder tolerates the mix. Newer TPU-runtime additions (chip-parts topology, memory reservation, sequencer program, persistent compilation cache) use editions; xla, megascale, and PJRT use proto3; the deepsea ISA protos remain on proto2. The split is roughly even (≈285 proto2 / ≈278 proto3 / ≈197 editions) and is a fingerprint of how recently each subsystem's schema was last touched.

Catalog by Source Root

Every .proto path begins with a Google source-tree root. Grouping the 760 entries by that first path segment gives the coarse map of what is compiled into the binary. Counts are over the 769 distinct path strings in the section (which includes import-only paths); the dominant three roots account for ~77% of the pool.

QUIRK — platforms/ outweighs third_party/ even though TensorFlow/XLA are the "headline" frameworks, because each of the five deepsea chip families ships a parallel ISA proto tree (TensorCore + SparseCore + BarnaCore + DMA + memory subsystems). The chip ISA schemas, not the framework schemas, are the bulk of the descriptor pool.

Catalog by Domain

The source-root view is coarse; the domain view is what a reimplementer actually navigates. The 760 entries fall into the following functional groups. Per-group counts are byte-anchored (matched against the section's path strings); representative file names are verbatim.

TPU deepsea ISA — per-chip-family schemas

The largest single domain. Five silicon codenames each carry a parallel proto tree under platforms/asic_sw/lib/deepsea/<codename>/.../isa/ plus driver/config trees under platforms/asic_sw/driver/deepsea/<codename>/. The ISA protos serialize the per-core compiled program AST (TensorCore, SparseCore SCS/TAC/TEC, BarnaCore).

The shared resource enum architectural_resource_ids.proto (asic_sw.deepsea.ResourceId, 232 values) names every hardware resource a sequencer can allocate (VMEM banks, sync flags, IAR slots, DMA channels, ICI ports); it is the spine of the compiler's resource-allocation pass. common_encodings.proto per chip defines the 64-value operand-source enums (VectorSource, ScalarY, CbregMetadata, SparsecoreVmask).

TPU runtime — topology, program format, compilation cache

learning/45eac/tpu/runtime/ and tensorflow/core/tpu/. These are the editions-dialect schemas that describe a physical TPU and the binary a core executes.

NOTE — the entry_point oneof in tpu_sequencer_program.proto enumerates exactly {jellyfish, pufferfish, viperfish, ghostlite, 6acc60406} with their per-core sub-programs (SCS = SparseCore Sequencer, TAC = Tile Access Core, TEC = Tile Execute Core). This oneof is the authoritative list of chip families the runtime can load and is corroborated by the per-chip ISA trees above.

XLA / HLO compiler

133 files span third_party/tensorflow/compiler/xla/ and platforms/xla/. The headline schemas:

The xla.DebugOptions message (reconstructed on its own page) lives in this domain. Note debug_options.proto is present in the section as a distinct path string.

Profiler — XPlane + per-chip trace_entries

54 files under third_party/tensorflow/core/profiler/, third_party/xprof/, and perftools/accelerators/xprof/, plus per-chip trace schemas under the deepsea driver tree.

The five trace_entries.proto carry per-chip TracePointId enums (up to ~135 values for ghostlite) cataloging memory-controller, ICI/DCN, OCI-descriptor, and throttle/power events. The tpu_telemetry.proto runtime-state snapshot is reconstructed on the profiling page.

Megascale — cross-slice collectives runtime

Exactly 15 files under the platforms/xla/megascale/ tree (most under runtime/, with megascale_info.proto/runtime.proto under common/ and addresses.proto/dcn_topology.proto under the third_party/.../xla/megascale/ mirror). The complete control plane above ICI/DCN.

PJRT — plugin C-API + distributed coordination

16 files spanning learning/45eac/research/pjrt/ and third_party/tensorflow/compiler/xla/pjrt/.

NOTE — there are two CoordinationService definitions in the pool — the public gRPC surface (xla/pjrt, 14 RPCs) and a lower-level state machine (tsl/protobuf, 19 RPCs). The overlap is intentional; both provide register/heartbeat/barrier/KV-store, but the tsl one is the implementation the xla one wraps.

TensorFlow / TF2XLA base

84 files of TF/TF2 graph and saved-model schemas (config.proto, rewriter_config.proto, saved_object_graph.proto, meta_graph.proto, journal.proto) plus the TF→XLA bridge (tf2xla.proto, xla_argument.proto, host_compute_metadata.proto, xla_activity.proto). These are present because the PJRT plugin links the TF2XLA legacy bridge and tf.data service replication.

Operations Research (ortools)

28 files under util/operations_research/. CP-SAT + MathOpt solver schemas: sat_parameters.proto (hundreds of CP-SAT tuning knobs), cp_model.proto, math_opt/{rpc,model,model_update,model_parameters,callback}.proto (SolveService, 3 RPCs), plus seven solver-adapter files (glpk, gscip, gurobi, highs, mosek, osqp, xpress). Pulled in because XLA's autosharding cost model uses MIP solving.

Infrastructure tail

NOTE — the CEL schemas (google/api/expr) are present because some compilation-environment switch expressions are CEL-evaluated. The infrastructure tail (security, monitoring, crash-analysis, file/storage) is not TPU-specific; it is dragged in by transitive imports from the framework and runtime libraries and is mostly inert in the PJRT plugin path.

gRPC Service Inventory

The pool defines 27 gRPC services across 184 RPCs. The largest surfaces are the on-device debug/control planes, not the public API. Full list in the per-domain tables above; the standouts:

QUIRK — the biggest service surface is the post-mortem TPU debugger (DebuggerService, 33 RPCs), not any compilation or execution API. The schema pool is shaped as much by the debug/observability tooling baked into the runtime as by the compiler itself.

Architectural Enum Highlights

Several enums in the pool are large enough to be the spine of a subsystem, read from the descriptor blobs:

The two LLO opcode enums (462 / 461 values) enumerate the full deepsea ISA at the LLO abstraction level — the same opcode universe documented in the LLO opcode table. ResourceId (232) is the resource-allocation spine; TpuMemoryReservationTypeProto (51) is the program-descriptor slot dictionary that PJRT reads.

Verification

• Section facts confirmed against the ELF section header [12] and the IDA segment record: protodesc_cold start 0xbe8af30, end 0xc1bf0b0, size = 3,359,104 (0x334180), flags A.
• Entry count 760 confirmed three ways: 760 distinct descriptor_table_protodef_* blob symbols, 760 matching descriptor_table_* registrar symbols, and 769 distinct .proto path strings in the section (760 own + 9 import-only).
• First entry confirmed: descriptor_table_protodef_zzRDQFgX_23 @ 0xbe8af80 → learning/45eac/research/pjrt/pjrt_tpu_topo_desc_name_mapping.proto.
• Per-root / per-domain counts derived by scanning the section's path strings; chip-family ISA dirs, the 15 Megascale files, 16 PJRT files, and the 5 trace_entries.proto were each enumerated against the binary.
• Registration mechanism anchored to the _ZL37descriptor_table_protodef_* mangled symbols and the companion linkarr_upb_AllExts / pb_defaults sections, both documented in Custom Sections.

Cross-References

• Custom Sections — owns the protodesc_cold section header and the linkarr_upb_AllExts / pb_defaults companions
• Static-Init Surface — the .init_array constructors that call AddDescriptors to build the pool
• Trailing zstd Blob — sibling forensic catalog; records_metadata.proto from this pool corroborates the riegeli hypothesis
• Binary Layout Reference — the full ELF section map this section sits in
• filewrapper_toc Catalog — sibling appendix; the embedded runtime-resource table (binary configs, bootloaders)
• Reconstructed-Proto Index — the index of schemas reconstructed in full from these blobs
• LLO Opcode Table — the 462/461-value opcode enums catalogued here, reconstructed
• TpuCompilationEnvironment — the largest single schema in the pool, reconstructed field-by-field
• xla.DebugOptions Proto — a reconstructed HLO-config schema from the xla domain
• tpu_telemetry.proto — a reconstructed TPU runtime-state schema from the profiler/debug domain