This is a fine tuned GPT-2 model trained on a corpus of all available English language Magic the Gathering card flavour texts. This is intended only for use in generating new, novel, and sometimes surprising, MtG like flavour texts. The training corpus was surprisingly small, only ~29000 cards, I had suspected there were more. This might mean there is a real limit to the number of entirely original strings this will generate. This is also only based on the 117M parameter GPT2, it’s a pretty obvious upgrade to retrain with medium, large or XL models. However, despite this, the outputs I tested were very convincing! The data was 29222 MtG card flavour texts. The model was based on the «gpt2» pretrained transformer: https://huggingface.co/gpt2. Only English language MtG flavour texts were scraped from the Scryfall API. Empty strings and any non-UTF-8 encoded tokens were removed leaving 29222 entries. This was trained using google Colab with a T4 instance. 4 epochs, adamW optimizer with default parameters and a batch size of 32. Token embedding lengths were capped at 98 tokens as this was the longest string and an attention mask was added to the training model to ignore all padding…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: rjbownes
Теги: jax, gpt2, text-generation-inference, endpoints_compatible
Лайков: 2 | Загрузок: 22
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.