A reward function recovered by inverse reinforcement learning is usually fitted to one task's raw state variables, so it silently encodes that task's coordinates and dynamics and stops working the moment the environment changes shape - which forces fresh expert demonstrations for every new instance. The fix is to move the reward off the ground states entirely: learn a single encoder shared across several related source tasks that maps each task's states into one common abstract space, and define the reward only there. Two pressures shape that space. A variational autoencoder with one shared encoder and one decoder per source task forces the latent code to retain what every task has in common; a discriminator that tries to tell expert trajectories from learner trajectories inside the latent space forces it to also encode optimality, which pure reconstruction would ignore. The reward is then fitted on the frozen abstract space by driving the learner's abstract state density toward the expert's, and freezing the encoder at that stage is precisely what stops the reward from re-absorbing source-task idiosyncrasies. Because the abstract states are dynamics-invariant, a state-only reward defined over them can be dynamics-agnostic even when no such reward exists over the ground states, and the transfer succeeds exactly when the target task's abstract state density lands close to the sources'. The payoff is a reward that trains a competent policy in an unseen task with no demonstrations there at all.
Inverse reinforcement learning recovers a reward function from expert demonstrations, but a reward fitted to one task's raw state space rarely transfers: it entangles the reward with that task's dyna…