This repo quantizes the model using a data-free quantization tool. This release is prepared for vLLM with compressed-tensors weight-only A16 inference. It does not claim compatibility with SGLang or other runtimes. The default reasoningeffort is changed to medium-high to reduce thinking-token cost. You can override it per request with «chattemplatekwargs»: {«reasoningeffort»: «max»}, or edit chattemplate.jinja` to set a different default. Accuracy reference, using the SGLang GLM-5.2 FP8 H200 / default / low-latency / single-node AIME25 recipe: This is a lightweight reproduction reference rather than a full formal benchmark. As of 2026-06-21, this model has been verified on an 8 x H200 machine with a Python 3.12 virtual environment, vLLM 0.23.0, and Transformers 5.12.1. The first launch may spend several minutes on vLLM compilation and warmup. 👋 Join our WeChat or Discord community. 📖 Check out the GLM-5.2 blog and GLM-5 Technical report. 📍 Use GLM-5.2 API services on Z.ai API Platform. 🔜 Try GLM-5.2 here. We’re introducing GLM-5.2, our latest flagship model for long-horizon tasks. It marks a substantial leap in long-horizon task capability over its predecessor GLM-5.1 and, for…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: QuantTrio
Теги: glm_moe_dsa, vLLM, compressed-tensors, INT8, W8A16, conversational, endpoints_compatible
Лайков: 3 | Загрузок: 146
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
