ahmedelkilani01/Swift-Qwen3.8-27B-i1-IQ4_XS-Smaller-GGUF - Каталог нейросетей
Генерация текста

ahmedelkilani01/Swift-Qwen3.8-27B-i1-IQ4_XS-Smaller-GGUF

Добавлено:
ahmedelkilani01/Swift-Qwen3.8-27B-i1-IQ4_XS-Smaller-GGUF

A 13.5 GB mixed 4-bit GGUF of UkisAI’s Swift-Qwen3.8-27B, made for 16 GB GPUs. It keeps the MTP head and leaves room for a 64K q4_0 KV cache with MTP speculative decoding. It uses the same per-tensor recipe as jrell/Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller, so the two files compare directly: same size and tensor types, different weights. As of 2026-09-15, the published 4-bit Swift GGUFs are 15.3 GB or larger, which leaves too little of a 16 GB card for long context. Everything published between 12.5 and 14.8 GB is 3-bit (Q3KM, IQ3_XS and similar). This file keeps 4-bit attention, SSM and embedding tensors and takes the savings from the FFN. — Sampling follows UkisAI’s recommendation. The chat template accepts reasoningeffort low, medium or xhigh (default xhigh) and enablethinking. — MTP needs —parallel 1. — For more context, drop MTP: about 104K fits on the same card without —spec-type, at lower decode speed (measured on the jrell base quant, which has the identical layout). AMD RX 6900 XT (16 GB, ROCm), llama.cpp build 10712 (daef7b687), with a desktop session running. Every run compares this file with the jrell Qwen3.8 quant under identical settings on the card above. These are…

Модальности:
Генерация текста

Области применения:
Диалог / чат

Языки программирования:
Swift


Задача: Генерация текста
Автор: ahmedelkilani01
Теги: gguf, llama.cpp, imatrix, mtp, qwen3.8, 16gb-vram, endpoints_compatible, conversational
Лайков: 3  |  Загрузок: 203

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.