vllm.models.deepseek_v41.nvidia.specialized_decode.ampere.codecs
¶
CuTe DSL device helpers of the SM80 kernels: global loads by address and FP8/FP4 decoding with integer operations.
SM80 has no instruction that converts E4M3 or E2M1 to a wider type (they arrive with SM89 and SM100), so these decoders place a code's bits in an FP32 word and scale it by a power of two:
- E4M3
s eeee mmm: the FP32 words | eeeemmm << 20has exponent fieldeeeeand mantissammm; times2^120it is the E4M3 value, the subnormals (eeee = 0) included, which are FP32 subnormals before the multiplication. FP32 arithmetic keeps subnormals unless a kernel opts into flush-to-zero, which these kernels do not. - E2M1
s eem: the FP32 words | eem << 22times2^126.
Every product is a power-of-two scaling of a value with at most four significant bits, so it is exact, and so is its BF16 rounding.
Kernels that only need BF16 decode two codes at a time into the halves of a
32-bit word (e4m3x2_bf16, e2m1x2_bf16): the same bit placement in
BF16, whose exponent field is FP32's, then the power of two and the block
scale applied with two fma.rn.bf16x2. Each product is exact, so the
result is the bits the FP32 decoders round to.
Functions:
-
bits_from_f32–The 32-bit word of an FP32 value.
-
byte_perm–prmt.b32: byteiof the result is the byte of(a, b) -
ceil_log2–ceil(log2(x))of a positive normal FP32 value, exactly. -
e2m1–An E2M1 nibble (the low 4 bits of
code) as FP32. -
e2m1x2_bf16–E2M1 nibbles
2 byteand2 byte + 1ofwordtimes the -
e4m3–An E4M3 byte (the low 8 bits of
code) as FP32. -
e4m3_bf16x2–An E4M3 byte as BF16 in both halves.
-
e4m3x2_bf16–E4M3 bytes
2 pairand2 pair + 1ofwordtimes the BF16x2 -
exp2_to_f32–2^exponentfor a normal FP32 exponent. -
f32_from_bits–The FP32 value of a 32-bit word.
-
fma_bf16x2–a * b + con both BF16 halves of 32-bit words, rounded to -
issue_s32–load_s32issued where it is written (see_issue). -
issue_u32–load_u32issued where it is written (see_issue). -
load_s32–One 32-bit global load of a signed index.
-
load_u32–One 32-bit global load.
-
store_u16–Store two bytes (2-byte aligned).
-
store_u32–Store four bytes (4-byte aligned).
-
store_u8–Store one byte.
-
to_e2m1–FP32 to an E2M1 nibble, nearest-even and saturating: the magnitude code
-
to_e4m3–A positive FP32 value in
[2^-9, 448]to an E4M3 byte, nearest-even. -
ue8m0–A UE8M0 scale byte as FP32 (the byte is the exponent field).
-
ue8m0_bf16x2–A UE8M0 scale byte as BF16 in both halves (the byte is the exponent
_asm(result, operands, text, constraints, *, loc, ip)
¶
A side-effect-free inline PTX statement.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
_issue(kind, address, *, loc, ip)
¶
A 32-bit global load that stays where it is written.
load_u32 is free to move: the compiler may sink it into the branch
that uses its value, behind the loads of earlier branches. A kernel that
issues a batch of these before decoding them keeps the whole batch in
flight at once.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
_scale_bf16x2(x, power, scale)
¶
BF16x2 x * power * scale, both products exact.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
_store(width, address, value, *, loc, ip)
¶
One global store of the low width bits of value.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
bits_from_f32(value, *, loc=None, ip=None)
¶
The 32-bit word of an FP32 value.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
byte_perm(a, b, selector, *, loc=None, ip=None)
¶
prmt.b32: byte i of the result is the byte of (a, b)
(bytes 0-3 of a, then 4-7 of b) named by nibble i of
selector.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
ceil_log2(x)
¶
ceil(log2(x)) of a positive normal FP32 value, exactly.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
e2m1(code)
¶
An E2M1 nibble (the low 4 bits of code) as FP32.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
e2m1x2_bf16(word, byte, scale)
¶
E2M1 nibbles 2 byte and 2 byte + 1 of word times the
BF16x2 scale, as BF16x2: code s eem becomes the BF16
s | eem << 6 (the value times 2^-126).
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
e4m3(code)
¶
An E4M3 byte (the low 8 bits of code) as FP32.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
e4m3_bf16x2(code)
¶
An E4M3 byte as BF16 in both halves.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
e4m3x2_bf16(word, pair, scale)
¶
E4M3 bytes 2 pair and 2 pair + 1 of word times the BF16x2
scale, as BF16x2: code s eeee mmm becomes the BF16
s | eeeemmm << 4 (the value times 2^-120, subnormals included).
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
exp2_to_f32(exponent)
¶
f32_from_bits(bits, *, loc=None, ip=None)
¶
The FP32 value of a 32-bit word.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
fma_bf16x2(a, b, c, *, loc=None, ip=None)
¶
a * b + c on both BF16 halves of 32-bit words, rounded to
nearest; subnormals are kept.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
issue_s32(address, *, loc=None, ip=None)
¶
load_s32 issued where it is written (see _issue).
issue_u32(address, *, loc=None, ip=None)
¶
load_u32 issued where it is written (see _issue).
load_s32(address, *, loc=None, ip=None)
¶
One 32-bit global load of a signed index.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
load_u32(address, *, loc=None, ip=None)
¶
One 32-bit global load.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
store_u16(address, value, *, loc=None, ip=None)
¶
Store two bytes (2-byte aligned).
store_u32(address, value, *, loc=None, ip=None)
¶
Store four bytes (4-byte aligned).
store_u8(address, value, *, loc=None, ip=None)
¶
to_e2m1(x)
¶
FP32 to an E2M1 nibble, nearest-even and saturating: the magnitude code
counts the rounding thresholds below |x| (at 0.75, 1.75 and 3.5 the
even code is the upper one).
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
to_e4m3(x)
¶
A positive FP32 value in [2^-9, 448] to an E4M3 byte, nearest-even.
Normal codes round the FP32 exponent-and-mantissa field to three mantissa
bits (a carry moves to the exponent); below 2^-6 the code is the
multiple of 2^-9, rounded by the FP32 magic-number addition.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/codecs.py
ue8m0(code)
¶
A UE8M0 scale byte as FP32 (the byte is the exponent field).
ue8m0_bf16x2(code)
¶
A UE8M0 scale byte as BF16 in both halves (the byte is the exponent field).