A language model that reads and writes English backwards. Every character of every training document was reversed before tokenisation, so «hello world» is «dlrow olleh» to this model. 81.5M parameters, trained from scratch on 5.73B reversed tokens. > An experiment, not a product. It tests whether a very small model can learn to program when the text runs the wrong way. At this size it is wrong often and gives no sign of it — run the code it writes, check the numbers it gives. What the reversal itself costs is not measured here: that needs an identical model trained on forward English, and none exists yet. Executed, never judged by a model. Temperature 0.6, 3 samples per task; pass@1 is the mean over all samples, pass@3 is best-of-3. Intervals are Wilson 95%; the suites are small. Eight of the fifteen easy tasks have their function name in the training data and the model solves those twice as often, so the harder suite is the one to read. The chat template reverses your text on the way in; the [::-1] turns the answer back round on the way out. That is the only thing to remember. — Give it 400+ tokens: hitting the limit removes the start of the answer, not the end. — Use a…
Модальности:
Генерация текста
Области применения:
Генерация кода Диалог / чат
Задача: Генерация текста
Автор: eventvoid
Теги: llama, reversed-text, code, small-model, trained-from-scratch, conversational, en, text-generation-inference
Лайков: 3 | Загрузок: 440
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.