WAL-Ternary-8B V77 is an independently constructed low-bit version of Qwen/Qwen3-8B. The complete checkpoint occupies 2.898417 bits per original parameter (BPW). All 252 Transformer-body matrices execute directly from the packed representation; dense Qwen or Neutrino weights are not needed. Weights and a copy of the runtime are hosted together at armanibadboy/WAL-Ternary-8B. > Research release. CUDA and x86-64 CPU are validated. Apple ARM64/NEON code is > included, but physical Apple Silicon validation is still pending. — 252/252 body matrices: T3 + sparse-k8 base. — 213 matrices: packed binary low-rank WALB2 correction. — 39 matrices: base only. — Embedding: symmetric INT3/g128. — LM head: symmetric INT4/g128. — Norm and small tensors: BF16. — Model payload: 2.882034 BPW; complete tree: 2.898417 BPW. This is a ternary-derived hybrid, not a pure 1.58-bit BitNet checkpoint. Additional reference results: GSM8K flexible 77.10%, IFEval strict 51.57%, IFEval loose 54.90%. These are hardware-specific measurements, not universal guarantees. The cache stores packed ternary symbols, sparse positions/signs and FP16 scales—not a reconstructed dense weight tree. Parent, ABI, converter,…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: armanibadboy
Теги: wal-tat, qwen3, ternary, low-bit, quantization, cuda, cpu, research
Лайков: 3 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
