A fine-tuned Llama 3.2 1B Instruct model, distilled for question-answering over Siemens SIMATIC S7-1200 PLC documentation. Designed to be paired with a RAG pipeline so that dense technical manuals — alarm codes, signal addresses, parameter tables — are accessible at the point of use, even on edge hardware without GPU. Industrial environments with strict security requirements (e.g. Purdue model network segmentation) cannot easily call cloud LLMs, and hosting large open-source models on-prem requires expensive GPUs. This model demonstrates that fine-tuned small language models offer a practical alternative: a 1B parameter model that, after distillation, matches or exceeds a 3B base model on domain-specific technical QA. All models were evaluated on 144 held-out questions from the S7-1200 system manual using an LLM-as-a-Judge binary score. Fine-tuning improved the 1B model by +16 percentage points, bringing it to parity with the 3x larger 3B base model. On 6 out of 144 test questions, this model answered correctly where both the 1B and 3B base models failed. — Architecture: LlamaForCausalLM (1.24B parameters) — Base model: meta-llama/Llama-3.2-1B-Instruct — Context length: 131,072…
Модальности:
Генерация текста
Области применения:
Диалог / чат RAG (поиск + генерация)
Задача: Генерация текста
Автор: distil-labs
Теги: llama, industrial-automation, siemens, plc, rag, documentation-qa, fine-tuned, distillation
Лайков: 3 | Загрузок: 99
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
