Bootstrapped Reward Shaping
Bootstrapped Reward Shaping (BSRS) is a potential-based reward shaping method that uses the reinforcement-learning agent's own current estimate of the state-value function as the shaping potential, densifying a sparse reward without a hand-designed task-specific potential function. The shaped Bellman operator is proven to remain a contraction for shape-scale eta in (-1,(1-gamma)/(1+gamma)), with closed-form asymptotic value and potential functions, and BSRS is shown to be inequivalent to any static potential; it accelerates training in tabular gridworlds, the Atari suite, and continuous control.
Bootstrapped Reward Shaping Jacob Adamczyk1,2, Volodymyr Makarenko3, Stas Tiomkin4, Rahul V.
Bootstrapped Reward Shaping (BSRS) is a method for potential-based reward shaping (PBRS) in reinforcement learning that removes the need to hand-design a task-specific potential function. Ng, Harada …