A LoRA fine-tune of Steerling-8B — a model that breaks from virtually every convention in modern large language models. This is not GPT. This is not Llama. This is something fundamentally different. Nearly every language model you’ve used — GPT-4, Llama, Mistral, Qwen, Phi — works the same way: it predicts the next token, one at a time, left to right. That paradigm has dominated NLP since 2017. Steerling throws it out entirely. Steerling is a causal diffusion language model. Instead of predicting tokens sequentially, it generates text through iterative denoising — the same fundamental process that powers image generators like Stable Diffusion, but applied to language. Tokens are generated in parallel within 64-token blocks, refined over multiple diffusion steps until the output crystallizes. This means: — Generation isn’t strictly left-to-right — the model can revise and refine tokens within a block simultaneously — The training objective is masked diffusion (predict randomly masked tokens), not next-token prediction — The model has a fundamentally different relationship with uncertainty — it can express and resolve ambiguity across multiple positions at once Standard…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: cahlen
Теги: peft, steerling, causal-diffusion, interpretable, concept-steering, lora, fine-tuned, nlp
Лайков: 3 | Загрузок: 15
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
