Multilingual sentence embedding model based on Gemma 2, designed for semantic similarity and retrieval. It is used in the Mitra alignment stack to embed sentences in languages such as Sanskrit, Tibetan, Pali, Chinese, and English for cross-lingual sentence alignment and similarity search. — Semantic similarity: Encode sentences and compare them via cosine similarity (embeddings are L2-normalized). — Retrieval: Encode queries with the query template and corpus passages with the corpus (no instruction) format; retrieve by nearest-neighbor search (e.g. FAISS). — Multilingual alignment: Used in this repo to embed source and target sentences for sentence-level alignment (e.g. Buddhist texts across Sanskrit, Tibetan, Pali, Chinese, English). — Integration into alignment pipelines (e.g. Bertalign-style alignment with this embedder). — RAG or search systems that need multilingual, instruction-aware query/corpus embeddings. — Any application that consumes L2-normalized sentence vectors from this model. — Use the exact prompt format (see “How to Get Started”) for queries and corpus. — Users should be aware of potential biases and limitations; evaluate on their own data and languages before…
Модальности:
Генерация текста
Области применения:
Мультиязычность
Задача: Генерация текста
Автор: buddhist-nlp
Теги: gemma2, embedding, semantic-similarity, multilingual, mitra, text-generation-inference, endpoints_compatible
Лайков: 4 | Загрузок: 205
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
