Skip to content

vllm.model_executor.layers.fusion.act_nvfp4_quant

Functions:

  • act_nvfp4_quant –

    Fuse ReLU2 or exact GELU, selected by act ("relu2" or "gelu"), and block16

act_nvfp4_quant(x, linear, act)

Fuse ReLU2 or exact GELU, selected by act ("relu2" or "gelu"), and block16 NVFP4 with 128x4 swizzled scales.

Source code in vllm/model_executor/layers/fusion/act_nvfp4_quant.py
def act_nvfp4_quant(
    x: torch.Tensor, linear: LinearBase, act: str
) -> QuantizedActivation:
    """Fuse ReLU2 or exact GELU, selected by act ("relu2" or "gelu"), and block16
    NVFP4 with 128x4 swizzled scales."""
    from vllm.model_executor.layers.fusion.quant_activation import QuantizedActivation
    from vllm.model_executor.layers.quantization.utils.quant_utils import kNvfp4Dynamic

    output, block_scale = torch.ops.vllm.act_nvfp4_quant(
        x, linear.input_global_scale_inv, act
    )
    return QuantizedActivation(
        data=output,
        scale=block_scale,
        orig_dtype=x.dtype,
        orig_shape=x.shape,
        quant_key=kNvfp4Dynamic,
    )