Ready-to-serve artifact of Qwen3.8-Flash-Next (MoE 125B-A6B) for capy-flash, the engine that runs it on two 16 GB consumer GPUs with the routed experts that do not fit in VRAM computed on the CPU. It is NVIDIA’s nvidia/Qwen3.8-Flash-Next-NVFP4 converted with capy-flash’s own recipe, with an MTP draft head re-distilled on capy-flash’s own output (v0.6; the previous revision, with NVIDIA’s MTP head, is tagged v0.5.4). Want images too? See Feyd89/Qwen3.8-Flash-Next-NVFP4-Vision-capy-flash. Requires capy-flash v0.5.4 or newer (v0.7 recommended), Linux (or WSL2 / Docker on Windows), CUDA 13.1, two sm120 GPUs with 16 GB (RTX 50 series), a CPU with AVX-512 (F/BW/VL, VNNI) or, from v0.7, AVX2 + FMA + F16C, and about 75 GiB of free RAM** (the experts that stay on the host are one 64.6 GiB page-locked allocation; the server checks this at startup). Smaller machine? The 64 GB edition runs on 64 GB of RAM and one or two GPUs, and the 3-bit 27B on a single 16 GB card. > Chat template updated on 23/10/2026: this artifact now embeds the maintained NInfer Qwen3.8 template (developer role, Responses instructions, system turns after the first message: what Codex CLI sends); the weights are…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: Feyd89
Теги: ninfer, capy-flash, nvfp4, moe, qwen3.8, cpu-offload
Лайков: 3 | Загрузок: 1,306
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
