This model is currently being fine-tuned with deepspeed+bf16 weights using the dataset from Robert A. Gonsalves’ article «I Once Trained an AI to Rhyme, and It Took GPT-J a Long Time. Since the Colab was slow, I upgraded to Pro. Each limerick cost me a dime.» https://towardsdatascience.com/i-once-trained-an-ai-to-rhyme-and-it-took-gpt-j-a-long-time-de1f98925e17 Unfortunately, support for 8-bit FTing doesn’t seem widely available yet on HF: «8-bit state dicts cannot currently be loaded directly into the 8-bit model after being pushed on the Hub. This is due to the fact that the statistics (remember weight.CB and weight.SCB) computed by the model are not currently stored or taken into account inside the state dict, and the Linear8bitLt module does not support this feature yet. INVALID LANGUAGE PAIR SPECIFIED. EXAMPLE: LANGPAIR=EN|IT USING 2 LETTER ISO OR RFC3066 LIKE ZH-CN. ALMOST ALL LANGUAGES SUPPORTED BUT SOME MAY HAVE NO CONTENT
Модальности:
Генерация текста
Задача: Генерация текста
Автор: pcalhoun
Теги: gptj, endpoints_compatible
Лайков: 2 | Загрузок: 11
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.