vllm.models.glm5next.nvidia.specialized_decode.runtime
¶
GLM-5.3-Flash specialized decode: a decode executor over the stock backbone for
GB300 TP4, enabled by default; decode_backend=stock disables it.
Modules:
-
decode–The specialized decode executor over the stock GLM-5.3-Flash backbone: one
-
draft–The specialized decode of the MTP draft layer: the main model's stages on the
Functions:
-
kda_dtypes–(conv, recurrent) KDA state dtypes the deployment's cache config selects.
-
specialized_decode_config–The deployment's specialized decode settings, None for the stock decode.
-
unsupported–Every reason this deployment cannot run the specialized decode.
kda_dtypes(vllm_config)
¶
(conv, recurrent) KDA state dtypes the deployment's cache config selects.
Source code in vllm/models/glm5next/nvidia/specialized_decode/runtime/__init__.py
specialized_decode_config(vllm_config)
¶
The deployment's specialized decode settings, None for the stock decode. Auto falls back for unsupported deployments; latency_specialized raises.
Source code in vllm/models/glm5next/nvidia/specialized_decode/runtime/__init__.py
unsupported(vllm_config)
¶
Every reason this deployment cannot run the specialized decode.
Source code in vllm/models/glm5next/nvidia/specialized_decode/runtime/__init__.py
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | |