This is a 4-bit MLX-quantized port of talkie-lm/talkie-1930-13b-it — a 13B language model trained on pre-1931 English text — for use on Apple Silicon (M1/M2/M3/M4) via MLX. — ~7.4 GB on disk, fits comfortably in 16 GB of unified memory. — ~26 tok/s decode on an M4 Max. — The original CUDA-only package needs ≥28 GB of VRAM. This makes it run on any modern Mac. > Dear Friend, It is raining to-day, and the roads are heavy. The weather is very unsettled. The news of the day is the report that the King is ill, and that his life is in danger. Believe me, Yours sincerely, JOHN BROWN. — 4-bit weight-only quantization via mx.quantize, group size 64. — Quantized: every nn.Linear (attnquery, attnkey, attnvalue, attnresid, mlpgate, mlplinear, mlpresid) and the token embedding. — Not quantized: lmhead (kept in bf16, ~700 MB), per-head/-layer scalar gains, and the lmheadgain scalar. — 7 sharded safetensors files plus model.safetensors.index.json. — 40 heads, headdim 128 (MHA, no GQA). — RoPE with base 1000000, NeoX split-half rotation (with the upstream’s specific sign convention). — Per-head QK rms-norm after RoPE, learned per-head gain on q. — SwiGLU MLP, nmlp = 13696. — Embedding skip:…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: joseluissaorin
Теги: mlx, talkie, vintage, pre-1931, apple-silicon, 4-bit, en
Лайков: 3 | Загрузок: 82
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
