fla-hub/rwkv7-168M-pile

— Developed by: Bo Peng, Yu Zhang, Songlin Yang, Ruichong Zhang — Funded by: RWKV Project (Under LF AI & Data Foundation) — Model type: RWKV7 — Language(s) (NLP): English — License: Apache-2.0 — Parameter count: 168M — Tokenizer: GPT-NeoX 20B tokenizer — Repository: https://github.com/fla-org/flash-linear-attention ; https://github.com/BlinkDL/RWKV-LM — Paper: https://huggingface.co/papers/2503.14456 — Weights: Converted from https://modelscope.cn/models/RWKV/rwkv-7-pile/file — Training regime: bfloat16, lr 8e-4 to 3e-5 cosine decay, wd 0.1, bsz 8x30x4096 A: upgrade transformers to >=4.48.0: pip install ‘transformers>=4.48.0’

Модальности:
Генерация текста

Задача: Генерация текста
Автор: fla-hub
Теги: rwkv, rwkv7, custom_code, en
Лайков: 4  |  Загрузок: 124

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.