This repository contains the official 4-bit quantized version of the Apertus-8B-2509 model using the calibrated version of SINQ (Sinkhorn-Normalized Quantization) method. SINQ is a novel, fast and high-quality quantization method designed to make any Large Language Models smaller while keeping their accuracy almost intact. To support the project please put a star ⭐ in the official SINQ github repository. — Model Name: Apertus-8B-2509-4bit-ASINQ — Base Model: swiss-ai/Apertus-8B-2509 — Task: Text Generation — Framework: PyTorch / Transformers — License: Apache-2.0 — Quantized By: Huawei — Computing Systems Lab — Quantization Method: A-SINQ (Sinkhorn-Normalized Quantization) — Precision: INT4 — Group Size: 64 — Framework: PyTorch — Quantization Library: sinq If you find SINQ useful in your research or applications, please — Put a star ⭐ in the official SINQ github repository. — Cite our paper:
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: huawei-csl
Теги: apertus, quantization, sinq, int4, efficient-inference, llm, compression, conversational
Лайков: 3 | Загрузок: 33
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
