vllm.utils.host_memory
¶
Host memory registration for CUDA/HIP DMA transfers, and core dump exclusion.
Functions:
-
exclude_from_core_dump–Leave the mapping out of core dumps (MADV_DONTDUMP, Linux 3.4+).
-
host_registration_ranges–Split a shared pool without bisecting any tensor's host block.
-
pin_tensor–Register one CPU range, bypassing the caching host allocator's rounding.
-
unpin_tensor–Unregister exactly the range passed to pin_tensor.
exclude_from_core_dump(mmap_obj)
¶
Leave the mapping out of core dumps (MADV_DONTDUMP, Linux 3.4+).
A CPU KV cache can be hundreds of GiB, so a crashing worker would otherwise write all of it to disk. Failure only costs a larger dump.
Source code in vllm/utils/host_memory.py
host_registration_ranges(tensor_sizes, num_blocks, total_size, max_chunk_bytes=HOST_REGISTER_CHUNK_BYTES, *, page_size=mmap.PAGESIZE)
¶
Split a shared pool without bisecting any tensor's host block.
Keep registration boundaries aligned to the backing pages. CUDA batch copies reject descriptors spanning adjacent registrations.
Source code in vllm/utils/host_memory.py
pin_tensor(tensor)
¶
Register one CPU range, bypassing the caching host allocator's rounding.
Source code in vllm/utils/host_memory.py
unpin_tensor(tensor)
¶
Unregister exactly the range passed to pin_tensor.