vllm.model_executor.kernels.linear.nvfp4.flashinfer
¶
Classes:
-
FlashInferB12xNvFp4LinearKernel–NVFP4 GEMM via FlashInfer's b12x CuTe DSL warp-level MMA kernel (SM12x).
-
FlashInferCudnnNvFp4LinearKernel–NVFP4 GEMM via FlashInfer's cuDNN wrapper.
-
FlashInferCuteDslNvFp4LinearKernel–NVFP4 GEMM via FlashInfer's cutedsl backend.
-
FlashInferCuteDslNvFp4W4A16LinearKernel–BF16 x NVFP4 GEMM via FlashInfer's CuTe-DSL backend.
-
FlashInferCutlassNvFp4LinearKernel–NVFP4 GEMM via FlashInfer's CUTLASS wrapper.
-
FlashInferTrtllmNvFp4LinearKernel–NVFP4 GEMM via FlashInfer's TensorRT-LLM wrapper.
FlashInferB12xNvFp4LinearKernel
¶
Bases: NvFp4LinearKernel
NVFP4 GEMM via FlashInfer's b12x CuTe DSL warp-level MMA kernel (SM12x).
Source code in vllm/model_executor/kernels/linear/nvfp4/flashinfer.py
FlashInferCudnnNvFp4LinearKernel
¶
Bases: NvFp4LinearKernel
NVFP4 GEMM via FlashInfer's cuDNN wrapper.
Source code in vllm/model_executor/kernels/linear/nvfp4/flashinfer.py
FlashInferCuteDslNvFp4LinearKernel
¶
Bases: NvFp4LinearKernel
NVFP4 GEMM via FlashInfer's cutedsl backend.
Source code in vllm/model_executor/kernels/linear/nvfp4/flashinfer.py
FlashInferCuteDslNvFp4W4A16LinearKernel
¶
Bases: NvFp4LinearKernel
BF16 x NVFP4 GEMM via FlashInfer's CuTe-DSL backend.
Source code in vllm/model_executor/kernels/linear/nvfp4/flashinfer.py
FlashInferCutlassNvFp4LinearKernel
¶
Bases: NvFp4LinearKernel
NVFP4 GEMM via FlashInfer's CUTLASS wrapper.
Methods:
-
input_quant_key–This kernel supports dynamic quantization of the input. By
Source code in vllm/model_executor/kernels/linear/nvfp4/flashinfer.py
input_quant_key()
¶
This kernel supports dynamic quantization of the input. By convention, pre-quantized blockscales must use the swizzled layout.
FlashInferTrtllmNvFp4LinearKernel
¶
Bases: NvFp4LinearKernel
NVFP4 GEMM via FlashInfer's TensorRT-LLM wrapper.