The standard profile. 8-bit attention/routers/shared experts, 4-bit routed experts. JANG mixed-precision affine (JANG4M, group size 64) conversion of nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16` for Apple silicon (MLX). NVIDIA Nemotron 3.5 Lightning is a 30B-total / 3B-active hybrid model: 52 layers of Mamba-2 + MoE + attention (23 Mamba, 23 MoE, 6 full-attention), 128 routed experts with 6 active plus 1 shared expert, and a 256K context window. Only 6 of 52 layers hold a KV cache, which is what makes long context affordable on a laptop. The exact directory uploaded to this repository was loaded and generated on an M5 Max MacBook on 2026-08-11. Greedy decoding, prompts rendered through this bundle’s own chattemplate.jinja`. — Model class: nemotronh — Weight shards: 26 — Weight bytes: 19024579336 (17.718 GiB) — Average bits/weight: 4.12 — Load time: 1.9 seconds — Reasoning ON: 121.0 tok/s — Reasoning OFF: 47.5 tok/s — Stop behaviour: normal stop on every turn (no length stops) — Visible output: coherent on every turn — Tool calling: emitted a well-formed block Probes: a factual question with reasoning on, the same with reasoning off, and a single-function tool-calling…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: OsaurusAI
Теги: mlx, nemotron_h, vmlx-swift, osaurus, jang, nemotron, nemotron-3.5, moe
Лайков: 3 | Загрузок: 288
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
