RWKV/RWKV7-Goose-World3-1.5B-HF

— Developed by: Bo Peng, Yu Zhang, Songlin Yang, Ruichong Zhang — Funded by: RWKV Project (Under LF AI & Data Foundation) — Model type: RWKV7 — Language(s) (NLP): English, Chinese, Japanese, Korean, French, Arabic, Spanish, Portuguese — License: Apache-2.0 — Parameter count: 1.52B — Tokenizer: RWKV World tokenizer — Vocabulary size: 65,536 — Repository: https://github.com/fla-org/flash-linear-attention ; https://github.com/BlinkDL/RWKV-LM — Paper: https://huggingface.co/paper — Training regime: bfloat16, lr 4e-4 to 1e-5 «delayed» cosine decay, wd 0.1 (with increasing batch sizes during the middle) — Final Loss: 1.9965 — Token Count: 3.119 trillion A: upgrade transformers to >=4.48.0: pip install ‘transformers>=4.48.0’

Модальности:
Генерация текста

Области применения:
Диалог / чат

Задача: Генерация текста
Автор: RWKV
Теги: rwkv7, conversational, custom_code, en, zh, ja, ko, fr
Лайков: 4  |  Загрузок: 155

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.