ExLlamaV3 quantizations of Devstral-2-123B-Instruct-2512 with tensor-level (L3) optimization. Maximum effort applied towards the goal of achieving the best possible quantizations at the expense of time and compute. Using this measurement.json file and the base quants provided, additional highly-optimized quantizations can be made in seconds at any reasonable bpw by anyone. All work done with ExLlamaV3 v0.0.18. VRAM-targeted quants using exl3’s measure.py → optimize.py pipeline. The 5.70bpw quant hit the optimization ceiling — requesting 6.75bpw produced 5.70bpw output, indicating no further beneficial tensor swaps available.
Модальности:
Генерация текста
Области применения:
Генерация кода Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: amanwalksdownthestreet
Теги: ministral3, mistral, devstral, code, exl3, exllamav3, quantized, conversational
Лайков: 3 | Загрузок: 42
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
