FlashNorm-prepared checkpoint of HuggingFaceTB/SmolLM2-135M. Mathematically equivalent to the source model. The per-channel RMSNorm weight tensors (inputlayernorm.weight, postattentionlayernorm.weight, model.norm.weight`) are folded into the following linear layers and then removed from the state dict entirely. > Framework support note. Stock vLLM currently does not load this checkpoint because the norm weight tensors are absent. The upstream patch to accept missing tensors is tracked at: TBD (vLLM issue link). Until the patch lands, use HuggingFace Transformers; it loads this with a warning that norm weights were not initialized and defaults them to ones, which is the correct behavior for FlashNorm. > > Two additional Llama-family verification checkpoints are published as Llama-3.2-1B-FlashNorm-test and Llama-3.1-8B-FlashNorm-test. These retain the norm tensors as all-ones (compatibility layout) so they load in stock vLLM today and are intended for experimentation. They will be republished as weightless variants once vLLM’s loader supports absent norm tensors. — Fold the per-channel normalization weight g into the following linear layer: Wstar = W @ diag(g), computed once at…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: open-machine
Теги: llama, flashnorm, transformer-tricks, efficient-inference, weightless-rmsnorm
Лайков: 3 | Загрузок: 46
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
