This repository provides the Q4KM quantization of sarvamai/sarvam-30b, an advanced Mixture-of-Experts (MoE) model. By using the GGUF format and 4-bit quantization, the VRAM requirement is reduced significantly, making it runnable on consumer hardware like RTX 3090, or RTX 4090. — Method: Q4KM (Medium, K-Quants) — Original Size: ~120 GB — Quantized Size: ~19 GB — Architecture: Sarvam MoE This model is a quantized version of sarvamai/sarvam-30b. Both the original model and these weights are released under the Apache License 2.0.
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: ThatCultivator
Теги: gguf, endpoints_compatible, conversational
Лайков: 3 | Загрузок: 42
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
