BigStationW/llama-13b-pretrained-sft-do2-4bit-128g-TRITON

This is the gptq 4bit quantization of this model: https://huggingface.co/dvruette/llama-13b-pretrained-sft-do2 This quantization was made by using this repository: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton And I used the triton branch with all the gptq implementations available (truesequential + actorder + groupsize 128) CUDAVISIBLEDEVICES=0 python llama.py ./llama-13b-pretrained-sft-do2-4bit-128g-TRITON c4 —wbits 4 —true-sequential —act-order —groupsize 1

Модальности:
Генерация текста

Задача: Генерация текста
Автор: BigStationW
Теги: llama, text-generation-inference, endpoints_compatible
Лайков: 2  |  Загрузок: 16

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.