RWKV/RWKV7-Goose-World2.8-0.1B-HF

— Developed by: Bo Peng, Yu Zhang, Songlin Yang, Ruichong Zhang — Funded by: RWKV Project (Under LF AI & Data Foundation) — Model type: RWKV7 — Language(s) (NLP): English — License: Apache-2.0 — Parameter count: 191M — Tokenizer: RWKV World tokenizer — Vocabulary size: 65,536 — Repository: https://github.com/fla-org/flash-linear-attention ; https://github.com/BlinkDL/RWKV-LM — Paper: https://arxiv.org/abs/2503.14456 Install flash-linear-attention and the latest version of tra — Training regime: bfloat16, lr 8e-4 to 1e-5 «delayed» cosine decay, wd 0.1, bsz 8x30x4096 (Please contact Peng Bo for details) A: upgrade transformers to >=4.48.0: pip install ‘transformers>=4.48.0’

Модальности:
Генерация текста

Области применения:
Диалог / чат

Задача: Генерация текста
Автор: RWKV
Теги: rwkv7, conversational, custom_code, en, zh, ja, ko, fr
Лайков: 3  |  Загрузок: 273

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.