vllm.models.qwen4_exp.nvidia.ops.qsa_indexer
¶
Triton kernels for Qwen4Exp QSA index selection.
Functions:
-
expand_qsa_block_indices–Expand compressed blocks and compact the causal tail of the open group.
-
qsa_select_paged_decode–Score and select compressed blocks for a request-major decode batch.
-
qsa_select_paged_prefill–Score and select compressed prefill blocks in bounded chunks.
-
warmup_qsa_mqa_paged_decode–Compile every reachable decode specialization without launching it.
expand_qsa_block_indices(block_indices, query_positions, visible_blocks, compress_ratio, token_topk, out)
¶
Expand compressed blocks and compact the causal tail of the open group.
Source code in vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py
qsa_prefill_logits_width(max_seq_len, compress_ratio, max_columns)
¶
Compressed columns any prefill row can score, rounded up to 64 for the top-k kernels' row alignment.
Source code in vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py
qsa_select_paged_decode(q, k_cache, page_table, visible_blocks, token_topk, compress_ratio, decode_query_len, block_indices, topk_backend='auto')
¶
Score and select compressed blocks for a request-major decode batch.
Parameters:
-
(q¶Tensor) –Query tensor shaped
[num_requests * decode_query_len, heads, head_dim]. -
(k_cache¶Tensor) –Compressed key cache shaped
[blocks, page_size, 1, head_dim]. -
(page_table¶Tensor) –Request block table shaped
[num_requests, max_pages]. -
(visible_blocks¶Tensor) –Number of visible compressed blocks per query.
-
(token_topk¶int) –Number of logical tokens selected per query.
-
(compress_ratio¶int) –Number of logical tokens represented by a cache row.
-
(decode_query_len¶int) –Number of query tokens per request.
-
(block_indices¶Tensor) –Compressed-index output buffer.
-
(topk_backend¶str, default:'auto') –One of QSA_TOPK_BACKENDS.
Source code in vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py
qsa_select_paged_prefill(q, k_cache, page_table, query_start_loc, visible_blocks, token_topk, compress_ratio, max_query_len, block_indices, max_seq_len, topk_backend='auto')
¶
Score and select compressed prefill blocks in bounded chunks.
Parameters:
-
(q¶Tensor) –Packed prefill query tensor shaped
[num_tokens, heads, head_dim]. -
(k_cache¶Tensor) –Compressed key cache shaped
[blocks, page_size, 1, head_dim]. -
(page_table¶Tensor) –Block table shaped
[num_requests, max_pages]. -
(query_start_loc¶Tensor) –Packed prefill query offsets with a terminal offset. Offsets may share the base of a larger backing tensor.
-
(visible_blocks¶Tensor) –Number of visible compressed blocks per query.
-
(token_topk¶int) –Number of logical tokens selected per query.
-
(compress_ratio¶int) –Number of logical tokens represented by a cache row.
-
(max_query_len¶int) –Maximum number of query tokens in one request.
-
(block_indices¶Tensor) –Compressed-index output buffer.
-
(max_seq_len¶int) –Longest context length in the batch this step.
-
(topk_backend¶str, default:'auto') –One of QSA_TOPK_BACKENDS.
Source code in vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py
704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 | |
use_gvr2_topk(topk_backend, block_topk, width)
¶
Whether _topk uses gvr_2. QSA logits are contiguous fp32, so only k and the width can rule it out.
Source code in vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py
warmup_gvr2_topk(topk_backend, block_topk, decode_width, max_decode_rows, prefill_width)
¶
Pre-compile gvr_2 for every serving geometry. Decode compiles per exact width, prefill per pow2 width bucket, so prefill_width covers every narrower chunk.
Source code in vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py
warmup_qsa_mqa_paged_decode(k_cache, page_table, *, num_heads, head_dim, max_decode_query_len, max_num_reqs, max_num_batched_tokens)
¶
Compile every reachable decode specialization without launching it.