Starting from nothing but 5 search queries, we used the Lightning Rod SDK to automatically generate 2,108 forecasting questions from news articles, label them using real outcomes, and train this model via RL. No expertise required. No manual labeling. No domain-specific engineering. The result beats GPT-5 on held-out questions. You can do this in any domain — just change the search queries. See how we built the dataset. This repo contains a LoRA adapter for gpt-oss-120b. A standalone merge.py script is included to merge it into a full model. — Brier Score: Mean squared error between predicted probability and outcome (0 or 1). Lower is better. Brier Skill Score (BSS) expresses this as improvement over always predicting the base rate—positive means the model learned something useful beyond historical frequency. — Expected Calibration Error (ECE): Measures whether predicted probabilities match actual frequencies. «70%» predictions should resolve «yes» 70% of the time. Lower is better. This repo contains a LoRA adapter trained with Tinker. The adapter uses Tinker’s module naming convention, so it requires a merge step before inference. A standalone merge.py script is included. This…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: LightningRodLabs
Теги: peft, forecasting, prediction, reinforcement-learning, grpo, lora, mixture-of-experts, politics
Лайков: 4 | Загрузок: 1
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
