This model is a fine-tuned version of kaizerBox/retnet-summarization on the xsum dataset. It achieves the following results on the evaluation set: — Loss: 3.1397 The following hyperparameters were used during training: — learningrate: 0.001 — trainbatchsize: 4 — evalbatchsize: 4 — seed: 42 — gradientaccumulationsteps: 4 — totaltrainbatchsize: 16 — optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 — lrschedulertype: cosine — lrschedulerwarmupsteps: 100 — numepochs: 3 — mixedprecisiontraining: Native AMP — Transformers 4.35.2 — Pytorch 2.1.0+cu118 — Datasets
Модальности:
Генерация текста
Задача: Генерация текста
Автор: kaizerBox
Теги: tensorboard, retnet, generated_from_trainer, endpoints_compatible
Лайков: 2 | Загрузок: 13
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.