EAQ v1 (EXL3 format) quantization of MiniMaxAI/MiniMax-M3.1-Flash, 60 layers, 128 routed experts with top-4 routing, 3.55 bits per weight on average (target 3.5) and a 6-bit LM head. The weights are 179.4 GiB (192.6 GB) in 25 shards. > Status: private build, quality measured, not yet run end to end. > The pack has passed structure, finiteness and hash checks, and it has been measured against the original model by teacher-forced scoring (below). Loading it in an inference engine and generating text are NOT TESTED. No engine we run today has verified support for this architecture, and the pack does not fit on a single 96 GB GPU. The indexer chooses which tokens attention reads, and the router chooses which experts run. Small errors there change what the model looks at and which weights it uses, so both are kept at source precision in every layer. This was checked on the finished pack: all 60 layers. Method: teacher-forced next-token scoring. The original checkpoint (its MXFP8 / NVFP4 blocks dequantized) and this pack both read the same token ids; for every assistant token we compute KL(original || quantized) over the full vocabulary, and whether the two models pick the same top-1…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Mia-AiLab
Теги: exllamav3, minimax_m3_vl, exl3, eaq, quantized, moe, minimax, conversational
Лайков: 3 | Загрузок: 1,196
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
