vllm.models.deepseek_v32.nvidia.glm52_low_latency_gemm
¶
GLM-5.2 decode GEMM selection for unquantized BF16 on SM100/SM103/SM107.
Dispatch is by local (N, K) weight shape and token count M.
GLM52_PROJECTIONS was measured on B300 (SM103),
GLM52_PROJECTIONS_SM107 on VRNVL72 (SM107) and
GLM52_PROJECTIONS_SM100 on B200 (SM100). The SM107 table also covers
the TP4 projections (q_b, o_proj, shared expert) and the replicated indexer
projections, which run as BF16 GEMMs at decode batch sizes.
Functions:
-
build_glm52_plan–Plan for a weight the walk below cannot reach (a plain
nn.Linear).
_WarpSplitKGemm
¶
Bases: VllmJitKernel['_WarpSplitKGemm.CompileKey']
Compile keys of FlashInfer's warp split-K: M is dynamic, so one per
(N, K, tactic). Compiling fills FlashInfer's own cache, which
run_warp_splitk_dense reads, without launching the kernel.
Source code in vllm/models/deepseek_v32/nvidia/glm52_low_latency_gemm.py
_low_latency_table()
¶
Measured dispatch table for the current device, or None if unsupported.
Source code in vllm/models/deepseek_v32/nvidia/glm52_low_latency_gemm.py
_warp_splitk_module()
cached
¶
FlashInfer's CuTe DSL warp split-K BF16 GEMM, called directly so every
(N, K, M) runs its measured tactic: flashinfer.mm_bf16 takes no tactic
and would use its autotuner or heuristic default instead.
Source code in vllm/models/deepseek_v32/nvidia/glm52_low_latency_gemm.py
_warp_splitk_rows(measured, first_m=1)
¶
Serve every M up to the largest swept M with the tactic swept at the smallest swept M' >= M.
Source code in vllm/models/deepseek_v32/nvidia/glm52_low_latency_gemm.py
build_glm52_plan(weight, dtype)
¶
Plan for a weight the walk below cannot reach (a plain nn.Linear).