Falcon-RW-1B is a 1B parameters causal decoder-only model built by TII and trained on 350B tokens of RefinedWeb. It is made available under the Apache 2.0 license. RefinedWeb is a high-quality web dataset built by leveraging stringent filtering and large-scale deduplication. Falcon-RW-1B, trained on RefinedWeb only, matches or outperforms comparable models trained on curated data. ⚠️ Falcon is now available as a core model in the transformers library! To use the in-library version, please install the latest version of transformers with pip install git+https://github.com/huggingface/transformers.git, then simply remove the trustremotecode=True argument from frompretrained()`. ⚠️ This model is intended for use as a research artifact, to study the influence of training on web data alone. If you are interested in state-of-the-art models, we recommend using Falcon-7B/40B, both trained on >1,000 billion tokens. 💥 Falcon LLMs require PyTorch 2.0 for use with transformers! — Developed by: https://www.tii.ae; — Model type: Causal decoder-only; — Language(s) (NLP): English; — License: Apache 2.0. Research on large language models, specifically the influence of adequately filtered and…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: petals-team
Теги: falcon, custom_code, en, text-generation-inference
Лайков: 2 | Загрузок: 122
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.