Unofficial MLX conversion of Aleph-Alpha/Kolibri-1 by Aleph Alpha GmbH. Not affiliated with or endorsed by Aleph Alpha. Modifications: the original FP8 (e4m3, 128×128 block-scaled) weights were dequantized and re-quantized to MLX affine quantization (group size 64). Routed experts use the bit width below; attention and the shared expert are 4-bit, embeddings and LM head 8-bit, and the MoE router is unquantized. Expect quality differences vs. the original, growing at lower bit widths. *First 4,096 tokens of the German «Kolibris» / English «Hummingbird» Wikipedia articles; lower is better; a small sample, so only roughly indicative. Generation runs at ~52–56 tok/s on an M1 Max for all three. On a 48 GB Mac, raise the GPU memory limit for the 3-bit variant: sudo sysctl -w iogpu.wiredlimitmb=40960 (resets on reboot). Don’t run another large model at the same time. Relative weight-reconstruction RMSE vs. INVALID LANGUAGE PAIR SPECIFIED. EXAMPLE: LANGPAIR=EN|IT USING 2 LETTER ISO OR RFC3066 LIKE ZH-CN. ALMOST ALL LANGUAGES SUPPORTED BUT SOME MAY HAVE NO CONTENT
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: velaia
Теги: mlx, kolibri1, moe, reasoning, apple-silicon, 4-bit, conversational, de
Лайков: 3 | Загрузок: 1,567
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
