IOCTL Catalog

All file:line citations on this page are into the GPL-2.0 source of aws-neuronx-dkms 2.27.4.0, shipped under /usr/src/aws-neuronx-2.27.4.0/. The command macros are read verbatim from neuron_ioctl.h (876 lines); the handlers from neuron_cdev.c (4043 lines). The source is read directly, not reverse-engineered; every _IOC number is anchored to its #define line and every handler to its function line. Other driver versions renumber lines and add/remove commands. Part — Kernel Driver · back to index

Abstract

This page is the authoritative cross-map of the Neuron driver's ioctl command set: the complete table of every command under magic char 'N' (NEURON_IOCTL_BASE, 0x4E, neuron_ioctl.h:656), its _IOC number and direction, its argument struct and size, the kernel handler it dispatches to, the libnrt/NDL ndl_* caller that issues it, and the gate tier that admits it. It is the reference a reimplementer drives the device from: pick a command, read its row, learn exactly which struct to pack, which fd lane to use, which line in neuron_cdev.c consumes it, and whether DEVICE_INIT ownership is required. The dispatch mechanism — the three-tier gate model, the _IOC_NR/_IOC_SIZE/_IOC_DIR overload-resolution discipline, the *64 sub-gate slip — is owned by the dispatch page and is not re-derived here; this page links to it and tabulates the result.

The command space is 96 distinct macros over a sparse _IOC_NR set {1..135}, grouped into five families: mem (allocation, copy, memset, info, zerocopy — handlers traced by the mem page), dma (engine/queue lifecycle, descriptor I/O, H2T queue management — the dma page), nq (notification-queue init, plus semaphore/event/hw-counter — the nq page), pod (UltraServer membership query/control — the pod/misc page), and misc (device info, version, BDF, topology, PRINTK, the deprecated/stub set). Eleven _IOC_NR values are size-overloaded: two or three differently-sized structs share one nr and direction, demultiplexed kernel-side by exact cmd == or _IOC_SIZE(cmd). The gate tier is one of three: PID (the command is in the npid_is_attached attach allow-list, :3210-3219, so it requires the DEVICE_INIT owner on a non-free-access fd), FA (free-access: reachable on the O_WRONLY misc lane through ncdev_misc_ioctl, :3147, with no attach gate), or none (a non-free-access fd, no attach sub-gate). Every command present kernel-side has a libnrt caller except the deprecated/stub set (DEVICE_INIT/RELEASE/RESET, DMA_ENG_INIT, NOTIFICATIONS_DESTROY_V1/QUEUE_INFO, DMA_QUIESCE_QUEUES).

For this reference, the contract is:

• The exact command constant — nr, direction, and the 32-bit cmd the caller passes (strace-visible), recomputed from the _IOC macro and verified against the #define line.
• The arg struct and its byte size — what libnrt packs and the handler copy_from_users; for size-overloaded families, every variant.
• The handler and the caller — the ncdev_* function (file:line) that consumes the command, and the ndl_* userspace caller (@addr, libnrt.so) that issues it.
• The gate tier and variants — PID / FA / none, plus the V1/V2/*64 sibling set per nr, so a reimplementer knows which fd lane and which struct width each command demands.

At a Glance — The Five Families and Their Lanes

The catalogue partitions by family; each family's gate profile is distinctive, and that profile is the first thing a reimplementer needs.

The dispatch cell (ncdev_ioctl, :3188) routes by exact cmd == for single-variant commands and by _IOC_NR(cmd) == for the size-overloaded families; the dispatch page §3 owns that logic. The columns below name the dispatch line so a row can be cross-checked, but the resolution rule itself is not repeated per row.

Reading a row

• cmd name — the NEURON_IOCTL_* macro suffix.
• _IOC nr — the _IOC_NR, exactly as the #define assigns it (header line in the citation).
• dir — _IO (NONE) · _IOR (R) · _IOW (W) · _IOWR (RW), from the macro. Note the driver ignores _IOC_DIR for dispatch except in ncdev_driver_info; several read-back handlers are encoded _IOR yet copy_to_user results — see the dispatch page.
• arg struct / size — the struct neuron_ioctl_* packed by the caller; size is the C sizeof (LP64, natural packing). Pointer-form macros encode _IOC_SIZE = 8 regardless of the pointed-to struct; struct-form macros encode the struct's sizeof (which is what makes a *64 sibling's cmd differ).
• kernel handler (file:line) — the ncdev_* function in neuron_cdev.c, plus the dispatch line in parentheses.
• ndl_* caller — the libnrt NDL function (@addr in libnrt.so) that issues this command, from the DWARF-attributed NDL disassembly.
• gate — PID (attach allow-list :3210-3219) · FA (free-access misc lane) · none.
• variants — sibling commands sharing the nr (/64, /V2, /V2MT*, _EXT0).

QUIRK — the byte size and the encoded _IOC_SIZE are not always equal. A pointer-form macro — _IOR('N', 21, struct neuron_ioctl_mem_alloc *) — encodes _IOC_SIZE = sizeof(void*) = 8, so its cmd is 0x80084E15 even though the struct it points at is 32 bytes. A struct-form macro — _IOR('N', 23, struct neuron_ioctl_mem_copy64) (no *) — encodes _IOC_SIZE = sizeof(struct) = 40, giving 0x80284E17. This is exactly how the two siblings of an overloaded nr get distinct cmd constants: MEM_COPY is pointer-form (8) and MEM_COPY64 is struct-form (40). The size column below reports the struct size; the cmd-hex in prose reports the encoded value.

Family: mem

Allocation, copy, memset, info queries, and the zero-copy fast path. Handlers traced in the memory page; the security boundary for every copy/memset is mc_access_is_within_bounds (neuron_mempool.h:225). The npid_is_attached attach gate covers MEM_ALLOC, MEM_FREE, MEM_COPY (base), MEM_GET_PA, MEM_COPY_ASYNC (base), MEM_COPY_ASYNC_WAIT, MEM_GET_INFO (:3213-3216); the *64 siblings and the buf-copy/zerocopy/memset families are not in the attach list (they still require a non-free-access fd).

GOTCHA — the rows marked × (MEM_COPY64, MEM_COPY_ASYNC64) carry gate none while their 32-bit base siblings carry PID. This is the *64 sub-gate slip: the attach allow-list tests exact cmd values (cmd == NEURON_IOCTL_MEM_COPY, :3214; cmd == NEURON_IOCTL_MEM_COPY_ASYNC, :3215) but the dispatcher reaches the family by _IOC_NR (:3294, :3296), which is true for both widths. The *64 cmd differs in _IOC_SIZE, so the exact compare is false and npid_is_attached is skipped — a non-DEVICE_INIT opener of a non-free-access fd can issue them. Blast radius is bounded by handle ownership and bounds-checking; full analysis is finding S3 on the attack-surface page. DMA_COPY_DESCRIPTORS64 (dma family) shares the slip.

QUIRK — the V2MT64 allocation struct (mem_alloc_v2_mem_type64, 56 B) is byte-identical to V2MT (48 B) plus a trailing __u32 pad (neuron_ioctl.h:52-62). The pad is never read by ncdev_mem_alloc_libnrt; it exists solely to bump _IOC_SIZE so libnrt can feature-probe whether the driver accepts the wider command — i.e. whether it has >=4GB allocation/copy support. The size column reflects this manufactured difference (8 → 48 → 56 across V2/V2MT/V2MT64). The same idea drives the DRIVER_INFO feature_flags1 capability bitmap (misc family).

Family: dma

Engine and queue lifecycle, descriptor staging, and H2T (host-to-tensor) queue management. Handlers traced in the dma page; they are thin marshalling wrappers that delegate to the ring layer (ndmar_*) and the DMA layer (ndma_*). The attach gate covers DMA_ENG_INIT, DMA_ENG_SET_STATE, DMA_QUEUE_INIT, DMA_ACK_COMPLETED, DMA_QUEUE_RELEASE, DMA_COPY_DESCRIPTORS (base) (:3211-3212); state queries and the H2T family are ungated (still non-free-access).

GOTCHA — DMA_QUIESCE_QUEUES (nr 40) is listed in the attach allow-list region but the handler is gone: :3279 returns -ENOIOCTLCMD with a "no longer supported" log. A reimplementer hunting for the queue-quiesce semantics will not find a handler — it was removed. DMA_ENG_INIT (nr 30) is likewise a deprecated inline return 0 (:3257) yet stays in the gate list; both are dead arms preserved for ABI stability.

CORRECTION (XMAP-batch) — DMA_QUEUE_INIT_BATCH's real struct is 8 + 256×48 = 12296 bytes (MAX_DMA_QUEUE_INIT_BATCH = 256, neuron_ioctl.h:271), which exceeds the 14-bit _IOC_SIZE field (max 0x3FFF); the encoded size therefore wraps. NDL was observed issuing 0xB0084E85 and falling back to the _cm path on -EINVAL (P1-L-NDL-03). The kernel dispatch matches this command by exact cmd == (:3263), so userspace and kernel agree on the same wrapped constant — the wrap self-cancels. The 12296-byte figure is the true sizeof; the encoded _IOC_SIZE is the wrapped value, hence Conf MED on the exact 32-bit hex (HIGH on the US↔K pairing).

Family: nq / semaphore / event / hw-counter

Notification-queue init (V1 deprecated, V2, V2-with-realloc) plus the multiplexed semaphore, event, and hardware-counter handlers. Traced in the nq page. Only the three NQ-init commands are in the attach gate (:3217-3219); semaphore, event, and READ_HW_COUNTERS are reachable on any non-free-access fd with no attach check.

NOTE — the EVENT_SET/EVENT_GET macros are encoded against struct neuron_ioctl_semaphore * (:742-743) but the handler copy_from_users into struct neuron_ioctl_event (:1462). The two are layout-identical (3 × __u32), so the wire form matches; the handler simply uses a clearer struct name. SEMAPHORE_READ/EVENT_GET are _IOWR (they read a value back); INCREMENT/DECREMENT/WRITE/SET and the NQ-init + READ_HW_COUNTERS are encoded _IOR despite writing back via explicit copy_to_user — the driver dispatches on the full cmd, not _IOC_DIR.

GOTCHA — NOTIFICATIONS_QUEUE_INFO (nr 58) returns a raw -1 (:3340), which surfaces to userspace as -EPERM, not -ENOSYS — a reimplementer probing for "unsupported" on this command sees a permission error, not an unimplemented one. NOTIFICATIONS_DESTROY_V1 (nr 52) is an inline return 0: NQ teardown is implicit on reset/flush, and libnrt's ndl_notification_destroy does the userspace munmap of the ring rather than relying on a kernel destroy.

CORRECTION (nq/counter sizeof) — the size column for NOTIFICATIONS_INIT_WITH_REALLOC_V2 and READ_HW_COUNTERS previously read 52 and 20; those were the unpadded field sums. The true sizeof on LP64 is 56 and 24 (verified gcc -m64): the realloc struct is 9 × __u32 (36) + 4 pad + 2 × __u64 (16) = 56 (neuron_ioctl.h:366-378), where the trailing __u64 mmap_offset forces 8-byte alignment after the odd ninth __u32 force_alloc_mem; read_hw_counters is __u64 * + __u32 * + __u32 count + 4 pad = 24 (:405-409). The handler copy_from_users sizeof(arg) (the padded value), so the kernel reads 56/24. See ioctl-nq §correction, which already documented this and flagged the catalog.

Family: pod

UltraServer / pod membership query and control. All three commands dispatch on the free-access misc lane (:3173-3178), so they are reachable on an O_WRONLY fd with no attach gate. Handlers traced in the pod/misc page; they thunk to the pod-election engine ndhal->ndhal_npe.npe_*. POD_STATUS and POD_CTRL are V1/V2 size-overloaded.

GOTCHA — POD_CTRL is state-changing on the ungated free-access lane. It thunks to npe_pod_ctrl (:2902), which can trigger a pod-election state transition (ctrl ∈ {REQ_POD, REQ_SINGLE_NODE, REQ_KILL, SET_MODE}, neuron_driver_shared.h:38). An O_WRONLY opener that never attached can drive pod control. The V1 path forces mode = NEURON_ULTRASERVER_MODE_UNSET (:2886); V2 carries an explicit mode. This is finding S5/S8-adjacent on the attack-surface page. The V1/V2 split is resolved by _IOC_SIZE (static_assert(POD_CTRL != POD_CTRL_V2), :2875), and the handlers memset the arg after copy_from_user then only honour .sz — all other [in] fields of POD_INFO/POD_STATUS are discarded by design.

Family: misc / info / lifecycle

Device info, version negotiation, BDF, topology discovery, CRWL core reservation, PRINTK, datastore acquire/release, the metric/power/HBM commands, and the deprecated/stub set. This is the largest and most heterogeneous family; gate tier varies per command. The free-access (FA) members are the topology/discovery/diagnostic subset the misc lane was built for; the rest run on the main path (none), with a handful PID-gated.

QUIRK — nr 106 is a genuine _IOC_NR collision: DEVICE_RESET_STATUS (__u8, size 1, :800) and DEVICE_BDF_EXT (ptr, size 8, :803) share it. They are disambiguated by both _IOC_SIZE (different cmd constants) and access-mode lane: RESET_STATUS is matched by exact cmd on the main path (:3239), BDF_EXT by exact cmd on the misc lane (:3156). Separately, DEVICE_READY (nr 2) must be tested before RESET_STATUS (:3229 ahead of :3239) because older drivers assigned both to ioctl 2; the dispatch order preserves the legacy contract. Neither collision is ambiguous as shipped — see dispatch §3.

NOTE — DRIVER_INFO (nr 110) is the lone command keyed on _IOC_DIR. GET (_IOR, :816) is dispatched in the misc lane by _IOC_NR (:3163) and so is reachable free-access; SET (_IOW, :817) is in the attach allow-list (:3218) on the main path, but ncdev_driver_info returns -ENOTSUPP for the write direction (:1902-1903) — so SET is effectively unimplemented regardless of lane. The † on the gate marks this dir-demux: a SET-form cmd arriving on a free-access fd reaches ncdev_driver_info by _IOC_NR and is rejected by _IOC_DIR. The GET response advertises feature_flags1 = 0x1FF (all nine capability bits, :1912-1916) with version = 0.

GOTCHA — the FA rows in this family are the free-access lane's full membership: CRWL_NC_RANGE_MARK/UNMARK (+_EXT0), COMPATIBLE_VERSION, DEVICE_BASIC_INFO, DEVICE_BDF_EXT, DMABUF_FD (mem family), DRIVER_INFO_GET, PRINTK, HOST_DEVICE_ID_TO_RID_MAP, NC_PID_STATE_DUMP, GET_LOGICAL_TO_PHYSICAL_NC_MAP, POD_INFO/STATUS/CTRL (pod family), GET_VA_PLACEMENT. Most are read-only topology/info, but CRWL_NC_RANGE_MARK/UNMARK mutate the device-spanning core-reservation map and PRINTK carries the size-0 OOB stack-read bug (finding S1). A reimplementation that treats "O_WRONLY ⇒ harmless query" is wrong — see attack-surface §1.

CORRECTION (crwl sizeof) — the size column for the four CRWL_{READER,WRITER}_* rows and the two CRWL_NC_RANGE_{MARK,UNMARK} rows previously read 8 (the pointer-form encoded _IOC_SIZE), which violates this page's convention (§"Reading a row": the size column reports the struct sizeof). The pointed-to structs are 36 and 24 (verified gcc -m64): struct neuron_ioctl_crwl { __u32 nc_id; struct neuron_uuid uuid; } = 4 + 32 = 36 (neuron_ioctl.h:411-414, neuron_uuid = __u8[32] at share/neuron_driver_shared.h:150-152); struct neuron_ioctl_crwl_nc_map = 4 × __u32 (16) + volatile long unsigned int bitmap (8) = 24 (:416-422). The _EXT0 row's 80 (= 4 × __u32 + long unsigned int bitmap[8], :424-430) was already struct-form and is unchanged.

Cross-References

• IOCTL Dispatch and the Privilege-Gate Model — the dispatch mechanism this catalogue tabulates: the three-tier gate model, _IOC_NR/_IOC_SIZE/_IOC_DIR overload resolution, and the *64 sub-gate slip (not re-derived here)
• Memory IOCTL Handlers — per-handler detail for the mem family: mem_chunk lifecycle, the 2-level handle table, mc_access_is_within_bounds, and the BAR4 zero-copy path
• DMA IOCTL Handlers — per-handler detail for the dma family: ring/queue lifecycle, descriptor staging, and the H2T queue-management commands
• NQ and Semaphore IOCTL Handlers — per-handler detail for NQ init (V1/V2/realloc), the multiplexed semaphore/event handlers, and READ_HW_COUNTERS
• Pod and Misc IOCTL Handlers — per-handler detail for the pod family and the misc info/version/BDF/topology commands
• The IOCTL Attack Surface (14 Findings) — the security projection of this catalogue: which gate tier admits each finding (the *64 slip S3, the ungated free-access state-changers S5/S8, the PRINTK OOB read S1)