ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ - Каталог нейросетей
Генерация текста

ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ

Добавлено:
ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ

A W4A4 + paired-4:8 sparse compressed checkpoint of moonshotai/Kimi-K2.5, produced with MoESQ. The MoE expert weights are NVFP4 with paired-4:8 structured sparsity, and they are stored sparse: only the kept values plus a small mask are on disk, not a dense NVFP4 tensor with zeros in place. The target is NVIDIA Blackwell (SM100 and SM120) sparse tensor cores. — Base model: moonshotai/Kimi-K2.5 (MoE, 384 routed experts, 61 layers) — Compression: NVFP4 W4A4 plus paired-4:8 sparsity on the routed MoE experts — Effective weight precision: about 2 bits/weight on routed-expert linears — Checkpoint size: 347.6 GiB, vs 524.8 GiB for the same weights stored as dense NVFP4 (0.66×) — Kernel: paired-4:8 sparse NVFP4 grouped GEMM (CUTLASS, SM100 and SM120) through vLLM’s paired48nvfp4` MoE backend Links — Paper: Hardware-Native Joint Sparse-Qu It contains the compression code, the vLLM integration and the kernels. — Other MoESQ checkpoints: ISTA-DASLab/moesq collection This checkpoint does not load in upstream vLLM. The MoESQ repository installs a patched vLLM v0.30.0 (the paired48nvfp4` MoE backend and…

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: ISTA-DASLab
Теги: kimi_k25, feature-extraction, kimi-k2.5, moe, nvfp4, w4a4, semi-structured-sparsity, sparse-storage
Лайков: 3  |  Загрузок: 271

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.