A general method for generating diverse, high-quality retrieval-augmented generation (RAG) instruction-tuning data from any source corpus. It is built on five RAG query paradigms that span differing document usefulness and numbers of useful documents, plus 'Instruction Simulation' that reuses exemplar data from existing instruction datasets to raise diversity and quality. Applied to Wikipedia, it yields a 40K-example dataset whose fine-tuning measurably improves LLMs' RAG performance across knowledge-intensive QA and multi-hop reasoning benchmarks, surpassing methods such as Self-RAG and RQ-RAG.
RAG-Instruct is a general method for synthesizing diverse, high-quality retrieval-augmented generation (RAG) instruction-tuning data from any source corpus. It defines five RAG query paradigms spanni…