GGUF quantized versions of inclusionAI/Ling-3.0-tiny, a lightweight hybrid reasoning MoE model with 7.9B total parameters and 1.3B active parameters per token, supporting English and Chinese, with a 256K context window, tool-calling, and a built-in thinking/reasoning mode. Ling-3.0-tiny is a hybrid-linear MoE model built on a native hybrid linear attention architecture, featuring a 3:1 alternating stacking of Kimi Delta Attention (KDA) and Multi-Head Latent Attention (MLA), combined with a sparse MoE FFN comprising 128 routed experts. With 7.9B total parameters and 1.3B activated parameters per token, it balances long-context modeling capability and computational efficiency for local and resource-constrained deployment. The model uses a Bailing V3-style conversation format with role-based delimiters (SYSTEM, HUMAN, ASSISTANT, OBSERVATION) separated by . It supports special tokens for thinking/reasoning (controlled via detailed thinking on / detailed thinking off in the system prompt, with reasoning output between think/answer tokens) and tool calling** (function blocks with parameter tags). > Recommended: Q4KM or Q5KM offer the best quality-to-size trade-off for most use cases.…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: WhiskyAKM
Теги: llama-cpp, gguf, ling-3.0, hybrid-linear, moe, quantized, conversational, endpoints_compatible
Лайков: 3 | Загрузок: 1,156
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
