This is an experimental hybrid GLM-5.2 checkpoint for vLLM/B12X serving. — the dense, attention, shared-expert, special-head, and MTP tensors from QuantTrio/GLM-5.2-Int4-Int8Mix; — the non-shared routed MoE expert MLP projections from lukealonso/GLM-5.2-NVFP4; — a vLLM-compatible compressed-tensors config update for the fused GLM-5.2 runtime module names used by current vLLM. The repository name currently contains MTPFix because the first upload used that internal working name. The actual checkpoint identity is better described as GLM-5.2 NVFP4 Int8Mix. This is not a full re-quantization from BF16. It is a merged checkpoint: — QuantTrio supplies the W8A16 dense/attention/shared/MTP parts and BF16 unquantized tensors. — Luke NVFP4 supplies the routed expert MLP projections for layers 3-77. — The config was adjusted so vLLM can load the fused MTP names (mtpblock, fusedqkvaproj) used at runtime. The effective quantizationconfig uses compressed-tensors with format: nvfp4-pack-quantized`. Compared with the original QuantTrio checkpoint, the routed expert tensors are not INT4 group-size-128 weights anymore. They are replaced by Luke’s NVFP4 expert tensors. Compared with Luke’s NVFP4…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: festr2
Теги: glm_moe_dsa, GLM-5.2, vLLM, compressed-tensors, NVFP4, INT8, W8A16, W4A16
Лайков: 3 | Загрузок: 31
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
