W4AFP8 quantization of zai-org/GLM-5.3-Flash: routed MoE expert weights in INT4 (group-128, symmetric, AWQ-calibrated) with static per-tensor FP8 activation scales; attention, dense MLPs and shared experts retain the source block-FP8 weights. Served by SGLang’s native CUTLASS W4A8 grouped GEMM on Hopper (SM90). 177.4 GB, down from 328.3 GB (FP8 source) and 642.6 GB (BF16 master). SGLang built from source. INVALID LANGUAGE PAIR SPECIFIED. EXAMPLE: LANGPAIR=EN|IT USING 2 LETTER ISO OR RFC3066 LIKE ZH-CN. ALMOST ALL LANGUAGES SUPPORTED BUT SOME MAY HAVE NO CONTENT The flag set above was verified together on 2xH200: server loads, generation is clean, both parsers work. If you find GLM-5.3-Flash useful in your research, please cite our technical report:
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: graphistry
Теги: glm5_next, glm, w4afp8, quantized, moe, conversational, 8-bit
Лайков: 3 | Загрузок: 245
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
