Authors: Erik Nijkamp, Hiroaki Hayashi, Yingbo Zhou, Caiming Xiong CodeGen2.5 is a family of autoregressive language models for program synthesis. This model is a 4-bit 128 group size AWQ quantized model. For more information about AWQ quantization, please click here. This model was successfully tested on CUDA driver v530.30.02 and runtime v11.7 with Python v3.10.11. Please note that AWQ requires NVIDIA GPUs with compute capability of 8.0 or higher. For Docker users, the nvcr.io/nvidia/pytorch:23.06-py3 image is runtime v12.1 but otherwise the same as the configuration above and has also been verified to work. The model was quantized with AWQ technique. If you find AWQ useful or relevant to your research, please kindly cite the paper:
Модальности:
Генерация текста
Области применения:
Генерация кода Следование инструкциям
Задача: Генерация текста
Автор: abhinavkulkarni
Теги: llama, AWQ, text-generation-inference
Лайков: 3 | Загрузок: 11
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
