Learned Reward Models Where No Verifier Exists

safety, helpfulness and taste have no automatic checker, so a learned reward signal is still the only option there

jeremy Video

Reinforcement Learning With Human Feedback in Large Language Models

Steps through the RLHF pipeline: supervised fine-tuning, collecting ranked human preference data, fitting a reward model, and optimizing the policy with reinforcement learning while a KL term keeps i…