vllm.models.minimax_m3.amd.specialized_decode.mono.lamport
¶
Lamport (sentinel) inboxes: a cross-GPU hand-off whose payload is its own flag.
A sender stores plain data into an inbox in a peer's buffer (PeerBuffer,
uncached). Every 16-bit half-word of an inbox holds the sentinel 0xFFFF until a
push lands. Finite bf16 data never holds it (bf16 0xFFFF is a NaN), nor does an
E8M0 scale (at most 254) or a finite e4m3 code; a sentinel half-word in pushed
data is rewritten to 0x7FFF first (canon: still a NaN). The reader re-loads
its slots until none of the half-words it uses is the sentinel: one round trip
after the last byte lands, no flag hop and no tag bytes on the wire.
The protocol the users keep (see glue/kernel.py, the MiniMax-M3.1 decode glue's reduce-scatter and all-gather):
- A slot is polled and re-armed (the sentinel written back) by exactly one lane, after that lane consumed it: a slot re-armed by a fast reader before a slow one saw it would lose the hand-off.
- Inboxes are double-buffered by launch parity (the caller's epoch or colour): a slot of parity p is written again two collective launches later, which a peer reaches only after this rank finished the next launch (it needs this rank's next pushes), i.e. after the re-arm stores completed at a kernel boundary.
- The host arms every inbox once at setup and again after a fault reset
(
arm), with no launch in flight.
Every wait is bounded: ticks of the 100 MHz s_memrealtime clock, and it gives
up once any rank set the error record (word 0 at err_base, see
Mailbox.fail in sync.py); a lane that timed out calls fail(site). The error word
and the clock are read only after a first miss, so a hand-off that already
landed costs one load round trip.
Functions:
-
arm–Every half-word of the inbox bytes [offset, offset + nbytes) of the uint8
-
canon–Sentinel half-words 0xFFFF -> 0x7FFF (a NaN stays a NaN): what a sender
-
gptr–The global pointer of type
tyat the Int64 addressaddr. -
ld32–32-bit system-scope monotonic load: uncached, never hoisted or merged
-
ld64–64-bit system-scope monotonic load (a poll of two words).
-
load16–Four dwords at dword
dwof a (cached) device buffer, as a Vector. -
load_dwords–n(1, 2 or 4) dwords at dworddwof a (cached) device buffer. -
poll_triples–Triple tl's two dwords at each address of
addrs(+triple_off(tl)) -
poll_vecs–16-byte slots at the per-lane addresses
addrs-> 4 Int32 words each, -
probe–Every lane of the wave spins on lane 63's word at
addr(one request a -
sent_mask–Bit 15 of every half-word of
wthat is the sentinel 0xFFFF (AITER -
spin–Re-load (
load_all) untilpendingclears; bounded byticks, gives up -
st16–Four Int32 words at a 16-byte aligned address: one plain 16-byte store
-
triple_off–Byte offset, in a run of packed bf16, of the two dwords that hold triple
-
wave_last–Lane 63's value of the per-lane Int64
a(both halves through -
words–The first
nelements of a Vector as Int32 words.
arm(buf, offset, nbytes)
¶
Every half-word of the inbox bytes [offset, offset + nbytes) of the uint8
tensor buf := the sentinel. Host side, no launch in flight.
Source code in vllm/models/minimax_m3/amd/specialized_decode/mono/lamport.py
canon(ws)
¶
Sentinel half-words 0xFFFF -> 0x7FFF (a NaN stays a NaN): what a sender pushes.
gptr(addr, ty, align)
¶
The global pointer of type ty at the Int64 address addr.
Source code in vllm/models/minimax_m3/amd/specialized_decode/mono/lamport.py
ld32(addr)
¶
32-bit system-scope monotonic load: uncached, never hoisted or merged (a poll).
Source code in vllm/models/minimax_m3/amd/specialized_decode/mono/lamport.py
ld64(addr)
¶
64-bit system-scope monotonic load (a poll of two words).
load16(base, dw)
¶
Four dwords at dword dw of a (cached) device buffer, as a Vector.
load_dwords(base, dw, n)
¶
n (1, 2 or 4) dwords at dword dw of a (cached) device buffer.
poll_triples(addrs, tl, lives, err_word, ticks, fail, site)
¶
Triple tl's two dwords at each address of addrs (+ triple_off(tl))
-> [(lo, hi)] with the 3 bf16 in bits 0 .. 47, once none of the lane's 3
half-words is the sentinel (the 4th half-word loaded belongs to a neighbour
triple and is not checked). lives: an address the lane does not need is
loaded once and never checked.
Source code in vllm/models/minimax_m3/amd/specialized_decode/mono/lamport.py
poll_vecs(addrs, live, err_word, ticks, fail, site)
¶
16-byte slots at the per-lane addresses addrs -> 4 Int32 words each,
once no half-word is the sentinel (two 64-bit loads a slot). live false:
loaded once, never checked.
Source code in vllm/models/minimax_m3/amd/specialized_decode/mono/lamport.py
probe(addr, wave_live, err_word, ticks)
¶
Every lane of the wave spins on lane 63's word at addr (one request a
wave) until it is sentinel-free: a hint before the full poll, bounded, with
no fault of its own. wave_live: false for a wave with no live lane.
Source code in vllm/models/minimax_m3/amd/specialized_decode/mono/lamport.py
sent_mask(w)
¶
Bit 15 of every half-word of w that is the sentinel 0xFFFF (AITER
one_shot_allreduce's test).
Source code in vllm/models/minimax_m3/amd/specialized_decode/mono/lamport.py
spin(load_all, pending, err_word, ticks, fail, site)
¶
Re-load (load_all) until pending clears; bounded by ticks, gives up
once err_word() is set, calls fail(site) on a timeout. The error word and
the clock are read only after a first miss.
Source code in vllm/models/minimax_m3/amd/specialized_decode/mono/lamport.py
st16(addr, ws)
¶
Four Int32 words at a 16-byte aligned address: one plain 16-byte store (a push into an inbox).
triple_off(tl)
¶
Byte offset, in a run of packed bf16, of the two dwords that hold triple tl (values 3 tl .. 3 tl + 2).
wave_last(a)
¶
Lane 63's value of the per-lane Int64 a (both halves through
ds_bpermute).