This is a 4-bit quantized (GPTQ) version of the GLM-4.6-REAP-268B-A32B model. It is able to run on 8x RTX 3090 (24GB) with -tp 8 and 200K tokens context window using fp8 KV cache. Framework: Quantized using GPTQModel. Experimental Optimization: This model uses an experimental modification where the whole dataset was fed to each expert during quantization to ensure improved output quality and expert stability. Calibration Dataset: 1808 samples mixed from c4/en (1024), arc (300), gsm8k (300), humaneval (164), and alpaca` (20). Shows percent passed. Aider polyglot by default uses 2 tries. > Make an html animation of fishes in an aquarium. The aquarium is pretty, the fishes vary in colors and sizes and swim realistically. You can left click to place a piece of fish food in aquarium. Each fish chases a food piece closest to it, trying to eat it. Once there are no more food pieces, fishes resume swimming as usual. Result: The model generated a working artifact using Kilo Code in Code mode with Temperature = 0.6 View the Result on JSFiddle Special thanks to GPTQModel team for the quantization tools and support. — Base Model: GLM-4.6 — Compression Method: REAP (Router-weighted Expert…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: avtc
Теги: vllm, glm4_moe, glm, MOE, pruning, compression, quantization, gptq
Лайков: 3 | Загрузок: 19
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
