An Apple-Silicon MLX pack of Recursive Flint, a 39.8B-total / 2.5B-active coding-agent MoE. Pack size: ~31 GB. Recommended RAM: 40 GB+ unified memory. Requires mlx-serve with Flint support (in progress). Routed expert banks are affine 4-bit at group size 64. Attention, routers, shared-expert weights, embeddings, and LM head are affine 8-bit at group size 64; norms, vectors, and matrices with fewer than 32 rows remain BF16. The checkpoint was converted shard-by-shard on CPU from RecursiveSI/flint, retaining the upstream Apache-2.0 license. Teacher-forced fidelity vs BF16 (256 tokens of prose): top-1 agreement 87.1% (pass threshold ≥85%), mean full-vocabulary KL 0.097 nats. ✅ Passes. For comparison, the uniform 4-bit pack scores 61.3% / 0.73 nats, so this mixed pack is the recommended Flint quant.
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: pgmharikrishnan
Теги: mlx-serve, flint, mlx, mixed-4-8bit, moe, apple-silicon, conversational, 8-bit
Лайков: 3 | Загрузок: 2,867
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.