baa-ai/Qwen3.8-27B-RAM-13GB-GGUF - Каталог нейросетей
Генерация текста

baa-ai/Qwen3.8-27B-RAM-13GB-GGUF

Добавлено:
baa-ai/Qwen3.8-27B-RAM-13GB-GGUF

GGUF build of Qwen/Qwen3.8-27B, capability-validated by baa.ai for faithful, retrieval-grounded reading — a full 27B reader that runs on a single 16 GB GPU (e.g. AWS g4dn.xlarge / NVIDIA T4) with a 32K-token context window at full-precision KV cache (64K and 128K with quantized KV). An Apple Silicon build of the same allocation is at baa-ai/Qwen3.8-27B-RAM-13GB-MLX (13.0 GB, vision tower retained). Built for the case where the retrieved documents are the source of truth and the model’s job is to interpret them — not to answer from its own memory. It was selected and validated to keep that behaviour intact under aggressive compression. > v2 (2026-09-27). This release replaces the v1 IQ3M build with a RAM per-tensor mixed-precision allocation at the same 12.6 GB: per-tensor GGUF types (Q2K to Q8_0) with importance-matrix calibration. WikiText-2 perplexity drops from 6.34 to 6.02 (−5.0%, better on 37 of 40 paired text chunks) with no change in size, and the build passes the full five-certificate bundle below. The v1 file remains available in the repository history. > ⚠️ Requires a recent llama.cpp (build b10360 / Aug 2026 or newer). Qwen3.8 is a hybrid Gated Delta Net…

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: baa-ai
Теги: gguf, quantized, mixed-precision, imatrix, faithful-retrieval, endpoints_compatible, conversational
Лайков: 3  |  Загрузок: 380

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.