— Model Architecture: GlmMoeDsaForCausalLM — Input: Text — Output: Text — Model Optimizations: — Weight quantization: FP4 — Activation quantization: FP4 — Release Date: 2026-09-15 — Version: 1.0 — Model Developers: RedHatAI This model is a quantized version of zai-org/GLM-5.3. It was evaluated on several tasks to assess its quality in comparison to the FP8 checkpoint. This model was obtained by quantizing the weights and activations of zai-org/GLM-5.3 to FP4 (MXFP4, OCP Microscaling) data type, ready for inference with vLLM. Weights use FP4 (e2m1) with group size 32 and power-of-2 (E8M0) scales, while activations use FP4 with dynamic per-group scaling. This optimization reduces the number of bits per parameter from 8 (FP8) to 4, reducing the disk size and GPU memory requirements by approximately 50%. Only the weights and activations of the linear operators within transformer blocks are quantized using LLM Compressor. This model was created by applying LLM Compressor with the MXFP4 scheme in model-free post-training quantization (no calibration data required), exported in compressed-tensors format. This model was evaluated on GSM8K Platinum, IFEval, MATH-500, AIME 2025, and GPQA…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: RedHatAI
Теги: glm_moe_dsa, mxfp4, fp4, vllm, llm-compressor, compressed-tensors, conversational, en
Лайков: 3 | Загрузок: 4,440
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
