A Q80 GGUF file of basal-omni-pico (0.8B, built on Qwen3.5-0.8B-Base) for llama.cpp, on the CPU or on a GPU. Same model and the same typed-decision API through basal-serve`: it reads a state and answers typed questions about it with a probability for each allowed option, in one prefill, without generating text. Results, limitations and training are on the main model card. Agreement is the share of decisions where the file picks the same option as the bf16 model served as released (basal 2.3.2, —mode eager, bf16, one H100), on 1,700 items of our development sets (one option order): — The differences are near-ties. No probability was non-finite on either device. — Automation with this folder’s CALIBRATION.gguf.json (fitted on Q80 readouts with —mode gguf, one option order, our calibration set; evaluated on a frozen test split of 8,560 decisions): at maxerror 1%, 35.4% of decisions are decided automatically (observed error 0.96%); at 5%, 58.5% (4.51%). ECE after the temperature: choice 0.009, yes/no 0.013, score 0.060. The bf16 model’s own file gives 35.0% (0.93%) and 58.2% (4.38%) on the same split. — These thresholds hold on our held-out test split. On Werdykt, an out-of-domain…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Remek
Теги: gguf, typed-decisions, decision-making, classification, calibration, polish, llama.cpp, edge
Лайков: 3 | Загрузок: 690
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
