An MLX 8-bit version of Aleph-Alpha/Kolibri-1, for Apple Silicon. Aleph Alpha’s Kolibri 1 is a German/English reasoning MoE: 78B total and 3.46B active parameters, 384 experts with 6 routed and 1 shared. See the original model card for details, evaluations and intended use. — Source: Aleph-Alpha/Kolibri-1-BF16, converted with mlxlm.convert -q —q-bits 8 —q-group-size 64 — Router (mlp.gate) left unquantized (bf16), as in the official FP8 release; router expertbias kept in fp32 — Size: ~77G on disk; needs at least that much unified memory plus KV cache mlx-lm has no built-in kolibri1 architecture yet. The repo ships kolibri1.py, which mlx-lm loads through «modelfile» in config.json`. It was ported from Aleph Alpha’s vLLM plugin (aleph-alpha-inference): — 4:1 sliding-window (512 + current token, RoPE) to full attention (no positional encoding) — Sandwich norms after attention and after MoE — Top-6 routing on logits + expertbias, with weights sigmoid(logits)` (not renormalized) The port was checked against an independent PyTorch reference of the vLLM semantics on a random tiny checkpoint. Relative error was ~2e-6, both for the full forward pass and for cached decoding past the…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: tobiasoberrauch
Теги: mlx, kolibri1, moe, reasoning, 8-bit, conversational, de, en
Лайков: 3 | Загрузок: 821
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
