> These are the GGUF builds of version 2 (released 2026-09-23), kept for reproducibility. The current version is v3 at inboxpraveen/Kavach-PII-270M-GGUF. The GGUF files and figures are unchanged from the original v2 upload; kavachextract.py`, missing from that upload, is now included in its v2 form. GGUF builds of Kavach PII 270M v2, a 270M parameter model that extracts 52 kinds of personal data from text in 29 languages. Runs in llama.cpp, Ollama, LM Studio, and anything else that reads GGUF. Start with Q4KM. It matches the F16 reference exactly at 89.2 F1, is half the size, and produced zero parse failures. Measured on 400 stratified documents from the held-out test set. Each quant was served by llama-server with full GPU offload, greedy decoding, and identical prompts. Speed on an RTX 5060 Laptop was 8.2 to 9.2 documents per second across all quants, with peak resident memory between 1.9 and 2.5 GB. This model has an unusual shape that changes the normal quantisation advice. The token embedding of Gemma-3-270M is 63% of its parameters (262,144 x 640 = 168M of 268M). llama.cpp keeps embeddings at high precision by default, so a nominal Q2K build only reaches 6.88 bits per weight…
Модальности:
Генерация текста
Области применения:
Диалог / чат Мультиязычность Распознавание сущностей
Задача: Генерация текста
Автор: inboxpraveen
Теги: gguf, llama.cpp, quantized, pii, pii-detection, privacy, redaction, anonymization
Лайков: 3 | Загрузок: 565
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
