This repository contains the official 4-bit quantized version of the Kimi-Linear-48B model using the SINQ (Sinkhorn-Normalized Quantization) method presented in SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights. SINQ is a novel, fast and high-quality quantization method designed to make any Large Language Models smaller while keeping their accuracy almost intact. To support the project please put a star ⭐ in the official SINQ github repository. — Model Name: Kimi-Linear-48B-A3B-Instruct-4bit-SINQ — Base Model: Kimi-Linear-48B — Task: Text Generation — Framework: PyTorch / Transformers — License: Apache-2.0 — Quantized By: Huawei — Computing Systems Lab — Quantization Method: SINQ (Sinkhorn-Normalized Quantization) — Precision: INT4 — Group Size: 64 — Framework: PyTorch — Quantization Library: sinq If you find SINQ useful in your research or applications, please — Put a star ⭐ in the official SINQ github repository. — Cite our paper:
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: huawei-csl
Теги: kimi_linear, quantization, sinq, int4, efficient-inference, llm, compression, conversational
Лайков: 3 | Загрузок: 25
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
