Speedup inference while reducing memory by 2x-4x using int8 inference in C++ on CPU or GPU. Checkpoint compatible to ctranslate2>=3.16.0 and hf-hub-ctranslate2>=2.10.0 — computetype=int8float16 for device=»cuda» — computetype=int8 for device=»cpu»` This is just a quantized version. Licence conditions are intended to be idential to original huggingface repo. This model is a fine-tuning of TII’s Falcon 7B LLM. It was trained with 11,123 top-1 (high-quality) demonstrations of the OASST data set (exported on June 2, 2023) with a batch size of 128 for 8 epochs with LIMA style dropout (p=0.2) and a context-length of 2048 tokens. — Finetuned from: tiiuae/falcon-7b — Model type: Causal decoder-only transformer language model — Language: English, German, Spanish, French (and limited capabilities in Italian, Portuguese, Polish, Dutch, Romanian, Czech, Swedish); — Weights & Biases: Training log (Checkpoint: 696 steps) — Code: Open-Assistant/model/modeltraining — Demo: Continuations for 250 random prompts — License: Apache 2.0 — Contact:** Open-Assistant Discord Two special tokens are used to mark the begi Each turn ends with a token. The input ends…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: michaelfeil
Теги: ctranslate2, int8, float16, sft, en, de, es, fr
Лайков: 3 | Загрузок: 18
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
