qtum/Hy4-preview-GGUF - Каталог нейросетей
Генерация текста

qtum/Hy4-preview-GGUF

Добавлено:
qtum/Hy4-preview-GGUF

GGUF quantizations of tencent/Hy4-preview, made with llama.cpp. 780B total parameters, 78 layers. DeepSeek-style MLA attention (q-LoRA rank 2048, kv-LoRA rank 512) with a gated output path and learnable attention sinks, wrapped in iHC hyper-connections on every attention and MLP block. 256 routed MoE experts with top-8 routing plus one shared expert. DSA sparse attention: 21 of the 78 layers own a full indexer, the rest share the preceding full layer’s top-k. Vocabulary 120,832. Quantized from the official BF16 weights. Every tier is imatrix-calibrated and ships as 39 shards. The master row is not a file in this repo. It is listed so the numbers above have a reference point — the 1.4 TiB BF16 master was actually loaded and measured on the same machine, so every tier is compared against a real baseline rather than an estimate. Pick IQ4XS unless you cannot fit it. It keeps the same top-1 token as the BF16 master on 90 % of positions and its KL divergence is 4.4× lower than the next tier down. Q2K is the balanced choice and the best option for CPU-only inference. IQ2XS` exists for when the footprint is hard-capped. Every tier here is calibrated with an importance matrix. To quantify…

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: qtum
Теги: gguf, llama.cpp, hunyuan, hy4, moe, quantized, imatrix, en
Лайков: 3  |  Загрузок: 318

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.