Official GGUF quantized builds for ewinregirgojr/Qwen3.8-19B-Pruned. The model is compressed to 19B effective parameters using SparseGPT 30% structured sparsity and quantized into single-file GGUF formats (Q40, Q6K, and Q8_0) for local inference on consumer hardware. Base Model (BF16 Safetensors): ewinregirgojr/Qwen3.8-19B-Pruned Effective Parameters: ~18.9B active non-zero parameters (30% sparse pruned, 866 text layers retained) Quantization Formats: Single-file Q40 (17.79 GB), Q6K (23.20 GB), and Q80 (29.30 GB) Context Length: Up to 262,144 tokens Compatibility: Ollama, LM Studio, Jan, llama.cpp, Unsloth, LocalAI, Open WebUI 1-Click Google Colab Notebook: qwen3.819bggufinferencecolab.ipynb` Run directly from Hugging Face Hub withou Calibration subset evaluation: Benchmark scores in the table were measured on calibration subsets (512 samples from UltraChat and C4) during the pruning phase to quantify layer reconstruction error. 2. Academic baseline alignment: The 1% to 2% degradation delta reflects published empirical degradation curves for 20B to 30B parameter models at 30% unstructured sparsity documented in the SparseGPT (arXiv:2301.00774),…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат Вызов функций (Tool use)
Задача: Генерация текста
Автор: ewin-reg
Теги: gguf, llama.cpp, ollama, unsloth, qwen3.8, qwen3_5, qwen3.5, qwen3
Лайков: 3 | Загрузок: 569
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.