michaelfeil/ct2fast-gpt_bigcode-santacoder

Speedup inference while reducing memory by 2x-4x using int8 inference in C++ on CPU or GPU. Checkpoint compatible to ctranslate2>=3.14.0 and hf-hub-ctranslate2>=2.0.8 — computetype=int8float16 for device=»cuda» — computetype=int8 for device=»cpu»` This is just a quantized version. Licence conditions are intended to be idential to original huggingface repo. 1. Model Summary 2. Use 3. Limitations 4. Training 5. License 6. Citation This is the Megatron-version of SantaCoder. We refer the reader to the SantaCoder model page for full documentation about this model — Repository: bigcode/Megatron-LM — Project Website: bigcode-project.org — Paper: 🎅SantaCoder: Don’t reach for the stars!🌟 — Point of Contact: contact@bigcode-project.org — Languages: Python, Java, and JavaScript There are two versions (branches) of the model: main: Uses the gptbigcode model. Requires the bigcode fork of transformers. maincustom: Packaged with its modeling code. Requires transformers>=4.27. Alternatively, it can run on older versions by setting the configuration parameter activationfunction = «gelupytorchtanh»`. The model was trained on GitHub code. As such it is not an instruction model and commands like…

Модальности:
Генерация текста

Области применения:
Генерация кода

Задача: Генерация текста
Автор: michaelfeil
Теги: ctranslate2, int8, float16, code, model-index
Лайков: 2  |  Загрузок: 38

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.