vllm.model_executor.layers.mamba.kda_checkpoint
¶
Classes:
-
FlashKDAPrefillCheckpointExporter–Store KDA (FlashKDA or Triton) recurrent and convolution checkpoint states.
Functions:
-
get_triton_kda_checkpoint_buffer–The
checkpoint_stateandcheckpoint_offsetsto pass the Triton chunk -
reserve_triton_kda_checkpoint_buffer–Reserve the
[max_num_seqs, H, V, K]workspace buffer the Triton chunk
FlashKDAPrefillCheckpointExporter
dataclass
¶
Bases: MambaPrefillCheckpointExporter
Store KDA (FlashKDA or Triton) recurrent and convolution checkpoint states.
Source code in vllm/model_executor/layers/mamba/kda_checkpoint.py
get_triton_kda_checkpoint_buffer(checkpoint, spec, num_seqs)
¶
The checkpoint_state and checkpoint_offsets to pass the Triton chunk
kernel, or Nones when the batch has no checkpoint.
Source code in vllm/model_executor/layers/mamba/kda_checkpoint.py
reserve_triton_kda_checkpoint_buffer(vllm_config, num_heads, head_dim, dtype)
¶
Reserve the [max_num_seqs, H, V, K] workspace buffer the Triton chunk
kernel stores checkpoint states into; profiling skips attention, so the
layer reserves it before the KV cache is sized.