Browse Teach Curate

Graphical Identification Criteria for Off-Policy Learning in Sequential Decision Processes

A set of graphical conditions, built on acyclic directed mixed graphs, that determine when the value of a target decision policy can be identified from data collected under a different policy. The criteria extend Pearl's backdoor criterion to sequential decision problems and unify the identification assumptions of dynamic treatment regimes and Markov decision processes, giving principled guidance for selecting state variables in off-policy learning.

Data Text

A Graphical Approach to State Variable Selection in Off-policy Learning Joakim Blach Andersen∗1,2

This paper develops a unifying causal framework for off-policy learning - estimating the value of a decision policy from data generated under a different policy - that encompasses both dynamic treatm…