> Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs A 4-bit mixed-precision MLX quant of XHToken/Spark-X2.5-4B, a dense 4B trained for conversation, coding, tool use and agent work, with a native 1M-token context and 200+ languages. Sensitive layers are kept at 8-bit and robust ones at 4-bit, so the on-disk size stays within a few percent of a stock uniform 4-bit quant while the parts that matter keep their precision. Spark-X2.5 is a new architecture for MLX, and mlx-optiq adds support for it, so import optiq before mlxlm.load: stock mlx-lm does not know the spark25 model type. We follow the same naming convention llama.cpp uses for Q4KM and similar mixed-precision quants: the «4-bit» label is for the predominant precision, not the weighted average. Measured on this quant against a uniform 4-bit quant of the same model, same harness, same prompts (MMLU 5-shot 1000, GSM8K, IFEval strict, BFCL V3 simple, HumanEval pass@1, HashHop; the Capability Score is their mean). Serving, with the mixed-precision KV cache, tool calling and the OpenAI, Responses and Anthropic…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: mlx-community
Теги: mlx, spark2_5, quantized, mixed-precision, 4bit, 8bit, optiq, apple-silicon
Лайков: 3 | Загрузок: 827
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
