> Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs An OptiQ mixed-precision 8-bit quant of codelion/dhara-250m, the second member of OptiQ’s Diffusion LLM family, for Apple Silicon. dhara is a tri-mode 250M model: one set of weights that decodes three ways, standard autoregressive (left-to-right), block-diffusion (fill a block of tokens and iteratively un-mask it), and self-speculation (draft a block with the diffusion forward, verify with the AR forward). It is a custom architecture stock mlx-lm can’t load (it adds Canon depthwise-conv layers, QK-norm after RoPE, and a logit soft-cap); OptiQ ships a vendored, mlx-native port that registers with mlx-lm and is bit-exact to the reference. At 250M, dhara is a base to fine-tune, the way Google’s Gemma-270M is, small enough to LoRA on-device for one task, not a general assistant. Demo: dhara-chat Space — chat with it and compare all three decoding modes. Block-diffusion and self-speculation are handled by the OptiQ runtime. optiq serve —model mlx-community/dhara-250m-OptiQ-8bit serves an…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: mlx-community
Теги: mlx, dhara_ar, optiq, diffusion-llm, dhara, quantized, conversational, custom_code
Лайков: 3 | Загрузок: 85
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
