DRR (Distillation-Reinforcement-Reasoning) is an externalist framework that improves the reliability of large language model reasoning without altering the base model. Rather than having a model critique its own chain-of-thought (which inherits the same biases, the introspection illusion), DRR judges the reasoner's observable behavior: it distills the reasoner's behavioral traces, trains a lightweight external Discriminative Model, and at inference time uses that model as a critic to flag and reject suspicious reasoning steps, forcing the LLM to discard flawed paths and explore alternatives. The approach is annotation-free and lightweight and outperforms prominent self-critique methods on multiple reasoning benchmarks.
This paper introduces DRR (Distillation-Reinforcement-Reasoning), an externalist framework for improving the reliability of large language model (LLM) reasoning. Inference-time chain-of-thought reaso…