An MLX quantization of Qwen/Qwen3.8-27B for running locally on Apple Silicon — the default model for chad, a Claude-Code-style local coding agent. — Params: 27B (dense qwen35 hybrid: 64 layers, 48 GatedDeltaNet + 16 full attention) — Scheme: 3-bit group-64 body (MLP gate/up/down + attention) and 3-bit embeddings, with lmhead held at 5-bit — Footprint: ~12.1 GB weights + a bundled 4-bit MTP head; 262k native context — RAM: Built for 24 GB Apple Silicon. chad sizes its context window from the live Metal budget, so expect tens of thousands of tokens rather than the full 262k on a 24 GB box. The naming follows Unsloth’s Dynamic 2.0 convention (UD-Q3KXL = a dynamic quant that spends extra bits on the layers that matter) so the scheme is recognizable at a glance. It is not literally a llama.cpp k-quant — this is an MLX group-64 affine quant produced by our own per-module predicate (the bulk low-bit, sensitive layers high), with the bit allocation chosen by measured held-out perplexity per gigabyte across candidate recipes. mtp.safetensors is the checkpoint’s own trained MTP head, quantized to 4-bit group-64 and bundled alongside the weights. chad loads it as a sidecar and uses it for…
Модальности:
Генерация текста
Области применения:
Генерация кода Диалог / чат
Задача: Генерация текста
Автор: nathansutton
Теги: mlx, qwen3_5, apple-silicon, coding, unsloth-style-dynamic-quant, conversational, 3-bit
Лайков: 3 | Загрузок: 495
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
