> Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs A mixed-precision MLX quant of XHToken/Spark-X2.6-4B, the on-device agent model that follows Spark-X2.5 with improved post-training for coding, tool use, long context and reasoning. 2.8 GB on disk, down from 7.7 GB bf16. See the original card for what the model is trained to do and how XHToken evaluates it. Spark is a new architecture for MLX, and mlx-optiq adds support for it, so import optiq before mlxlm.load: stock mlx-lm does not know the spark25 model type. Following llama.cpp’s naming for mixed quants, the «4bit» label denotes the predominant precision, not the weighted average. The per-layer allocation was not measured on this model. It was transferred from mlx-community/Spark-X2.5-4B-OptiQ-4bit, whose allocation came from a KL-divergence sensitivity sweep over a six-domain calibration mix. Spark-X2.6 keeps exactly the same architecture and modeling code as X2.5, and all 217 tensors matched with none unmatched. Run through this quant and its bf16 parent on Apple Silicon with greedy decoding: The one…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: mlx-community
Теги: mlx, spark2_5, quantized, mixed-precision, 4bit, 8bit, optiq, apple-silicon
Лайков: 3 | Загрузок: 1,237
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
