Granite 4.0 Micro (≈3B) tuned for medical education & instruction following. Recipe: JEPA-LLM SFT on medmcqa-hard + personas augmentation → GRPO on medmcqa-hard; finalized with Arcee Fusion merge back into the IBM base. > ⚠️ Medical safety > This model is not a clinician and may hallucinate. Do not use for diagnosis or treatment. Use under qualified medical supervision only. — JEPA-LLM objective, see repo: mkurman/jepa-llm, used as an auxiliary signal during SFT to bias toward stable, representation-level learning rather than pure next-token fitting; run for 400 steps on MedMCQA-hard with Personas augmentation from Tulu-3 personas (adds constraint-following behaviors and improves coverage of IFEval-style requirements). — GRPO replaces the critic with group baselines, enabling efficient multi-sample training; we generate 8 candidates per item and reward answer correctness / format checks. — Arcee Fusion in MergeKit to selectively fuse with the original Granite 4.0 Micro (avoids over-averaging from naive merges and tends to keep base calibration). Author’s harness notes: EleutherAI lm-evaluation-harness with Granite’s chat template and batch size 8. > Why gains are modest:…
Модальности:
Генерация текста
Области применения:
Диалог / чат Медицина
Задача: Генерация текста
Автор: openmed-community
Теги: gguf, medical, instruction-tuned, jepa-llm, grpo, dpo-like, personas, mergekit
Лайков: 4 | Загрузок: 128
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
