Zero-Shot Multilingual Text Embeddings by Connecting an Encoder to an LLM
An architecture for producing high-quality multilingual sentence/text embeddings without any multilingual training data, by fusing two pre-trained models: a multilingual encoder that yields language-agnostic representations and an English-optimized LLM-based embedding model. A small set of trainable connector parameters maps the encoder's language-neutral vectors into the LLM's input space, transferring multilingual understanding to the LLM while keeping its strong embedding ability. The result is a shared language-universal representation space that lifts performance most for medium- and low-resource languages.
LUSIFER: Language Universal Space Integration for Enhanced Multilingual Embeddings with Large
LUSIFER (SIGIR 2025) is a zero-shot method for building multilingual text-embedding models without any multilingual training data. Its architecture wires a strong multilingual encoder (XLM-R), which …