This method learns a representation of robot behaviors whose features align with user preferences by treating a user's natural exploratory search (selecting interesting behaviors, ignoring irrelevant ones) as an implicit source of contrastive labels, eliminating the separate proxy-labeling task usually required for user-aligned features. A contrastive loss trains feature-generating networks so that behaviors a user engaged with are pulled together and separated from those they skipped. Students learn how implicit signals from ordinary interaction can substitute for explicit annotation in preference learning, and how such features improve downstream reward learning on measures like completeness, simplicity, minimality, and explainability.
This human-robot-interaction paper tackles preference elicitation: learning a representation of robot behavior whose features align with what a user actually cares about, so a robot can be customized…