A 165M parameter Memory-Augmented Language Model (MALM) for semantic code search, trained on CodeParrot. MALM combines a transformer with learned memory retrieval for semantic code search: 1. Query encoder — Encodes natural language queries into embeddings 2. Value encoder — Encodes function signatures/docstrings 3. Retrieval — Attention-based lookup from query to memory 4. Memory bank — 2000 Python functions from CodeParrot MALM uses a memory-augmented architecture different from standard LLMs: — Separate query and value encoders for retrieval — Requires a memory bank of functions — Inference is retrieval-based, not autoregressive generation This architecture doesn’t fit mlx-lm generate, so we provide a custom inference script. Trained on CodeParrot with a focus on Python function retrieval: — Encodes natural language queries into embedding space — Learns semantic similarity between queries and function signatures — Uses attention-based retrieval over a memory bank Part of the HashHop project exploring long-context evaluation and memory-augmented architectures.
Модальности:
Генерация текста
Области применения:
Генерация кода
Языки программирования:
Python
Задача: Генерация текста
Автор: codelion
Теги: mlx, memory-augmented, code-generation, retrieval-augmented, python, code-search
Лайков: 4 | Загрузок: 31
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
