An 8.9M-parameter question-answering model that runs entirely offline on an ESP32-S3 microcontroller, answering espresso questions a piece at a time at roughly 50 ms per position. This is the Barista model from the esp32-ai project, which pairs it with a portable C inference runtime that runs on the device. One question in, one answer out. No chat history, no retrieval. The model writes the answer itself; it does not select from a catalogue of canned replies. The third answer is the model declining a request outside what it was built to handle, which is a trained behaviour rather than a filter. This is the design point that separates Barista from a conventional small model. It reads 8,057 input tokens so it can accept varied typing, and writes only its 854-class output alphabet. — The output head is 854 rows instead of 8,057, so it is 109K parameters rather than a million. A full-vocabulary head would be the largest single matvec the model runs per position; this makes it one of the smallest. — Words outside the 854 are not merely unlikely, they are unsayable. There are no digit tokens at all, so the model cannot invent grind 2 steps finer. This restriction is a property of the…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: slvDev
Теги: esp32, esp32-s3, microcontroller, embedded, edge, per-layer-embeddings, int4, constrained-decoding
Лайков: 4 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
