Mirai’s Qwen3.8-27B Large Quantization A large, high-quality quantization of Qwen3.8-27B prepared for efficient local inference on Apple silicon. Mirai-L is on the size-KL Pareto frontier: we found no checkpoint that is smaller while also having lower KL divergence. Evaluation data mixture: 45% public agentic, 30% public SFT/long-context, 25% private chat data. If you are on macOS, the easiest way is to install the mirai Homebrew package and then run the CLI: Currently only Apple silicon inference is supported. If you want to build things from source, read this overview. Mirai Large uses 8-bit symmetric integer quantization with bfloat16 scales and group size 64. Block-diagonal Random Hadamard Transforms with block size 32 are used to reduce activation and weight outliers. The checkpoint was prepared with post-training quantization. In quantization and model-size tier, this checkpoint is comparable to Unsloth’s UD-Q8KXL GGUF. This is a quantized version of Qwen/Qwen3.8-27B. For architecture details, intended use, evaluations, and limitations, see the original model card.
Модальности:
Генерация текста
Задача: Генерация текста
Автор: trymirai
Теги: uzu, mirai, 8-bit
Лайков: 3 | Загрузок: 936
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
