vllm.kernels.triton.fused_pre_fc_norms
¶
Functions:
-
fused_pre_fc_norms–Returns cat([norm_e(inputs_embeds), norm_h(hidden_states)]) -> [N, 2H].
-
fused_pre_fc_norms_split–Two-width variant of
fused_pre_fc_normswith separate outputs.
fused_pre_fc_norms(inputs_embeds, hidden_states, emb_weight, hid_weight, eps)
¶
Returns cat([norm_e(inputs_embeds), norm_h(hidden_states)]) -> [N, 2H].
Both norms are Gemma-style RMSNorm (scale by 1 + weight, fp32
accumulation, multiply in fp32 before the cast back to the input dtype).
Source code in vllm/kernels/triton/fused_pre_fc_norms.py
fused_pre_fc_norms_split(inputs_embeds, hidden_states, emb_weight, hid_weight, eps)
¶
Two-width variant of fused_pre_fc_norms with separate outputs.
For drafters whose pre-FC norms have different widths and feed separate
FC GEMMs (e.g. Qwen4Exp MTP), so one launch writes two contiguous
outputs instead of a concat. Returns (normed_embeds [N, H_EMB],
normed_hidden [N, H_HID]).