Held-out loss: 0.696 (epoch 0.2) -> 0.651 (0.6) -> 0.622 (epoch 2). Training loss ended at about 0.55. These are language-modelling losses, not task accuracy. Same tasks and conditions for all models; deterministic checkers (executed code and SQL, schema checks, exact values). Few tasks per domain (6), so differences of one task are noise. What this means: recipe 2 repairs the damage of recipe 1 (tool calls, thinking mode, prompt-injection resistance) and keeps the datainfra domain at the base model’s level. It does not beat the base model on these probes; the datainfra tasks are already saturated by the base model, so the probe cannot show a gain there. Known weaknesses: it guesses a missing tool argument instead of asking (2/2 tasks), summaries run too long, and it can invent citations and article numbers (research, governance). Treat answers as drafts to verify. Held-out language-modelling loss: 0.696 (epoch 0.2) -> 0.622 (epoch 2). ChatML with an empty thinking block (the model was trained in non-thinking mode): SHA256SUMS lists the checksums. The previous version (recipe 1) stays available under the git tag v1. Native 262,144-token context usable in full on multi-GPU pools…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: h3rb3rn
Теги: gguf, compound-ai, domain-expert, datainfra, lora, lumi-g, moe-sovereign, hybrid-attention
Лайков: 3 | Загрузок: 198
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
