1Cademy - RePlug Retrieval-Augmented Black-Box Language Model

Learn Before

Research

Concept

RePlug Retrieval-Augmented Black-Box Language Model

RePlug is a retrieval-augmented language modeling framework introduced by Shi et al. (2023) that treats the language model as a frozen black box: the LM's parameters and architecture are never modified, and the only interface used is input-output. Given a query, a dense retriever fetches the top- $k$ documents from an external corpus, each retrieved document is independently prepended to the query, and the resulting per-document LM next-token distributions are combined via a retrieval-score-weighted ensemble. An optional extension, RePlug LSR (LM-Supervised Retrieval), tunes the retriever using the language model's own likelihood of the target output as a supervision signal, so the retriever learns to surface documents that actually improve LM predictions. Because the LM is untouched, RePlug can be paired with any off-the-shelf retriever (e.g. a late-interaction retriever such as ColBERTv2) and any black-box LM, which is why the combined ColBERTv2/RePlug system can appear as a strict-parity baseline in retrieval-augmented QA evaluations.