HealthGPT-LoRA is a biomedical question-answering model built by fine-tuning Meta Llama 3.2 3B Instruct using QLoRA (PEFT) on the PubMedQA dataset. This repository contains only the LoRA adapter, which can be loaded on top of the original Llama 3.2 3B Instruct model. The model is trained on the PubMedQA dataset containing biomedical question-answer pairs. — Training Samples: ~150,000 — Dataset Completion: Approximately 75% — Training is currently in progress, with plans to continue training on the remaining dataset and further improve performance. The current model achieves approximately 12% higher accuracy than the base Llama 3.2 3B Instruct model on the evaluation dataset. HealthGPT is an ongoing project focused on building a production-ready biomedical AI assistant. The current release represents approximately 75% of the planned fine-tuning process, with Retrieval-Augmented Generation (RAG) and deployment planned in future updates.
Модальности:
Генерация текста
Области применения:
Ответы на вопросы Медицина
Задача: Генерация текста
Автор: llmithull
Теги: peft, llama, lora, qlora, biomedical, healthcare, pubmed, medical
Лайков: 3 | Загрузок: 6
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
