A parameter-efficient tuning method that automatically determines, per transformer layer, how many continuous (soft) prompt tokens to insert and how deep to insert them, by relaxing the discrete configuration choice into a differentiable objective in the style of differentiable neural architecture search. This lets a vision-language model such as CLIP tailor its prompt configuration to each downstream dataset instead of using a single hand-fixed context length, improving few-shot accuracy while remaining composable with other prompt-tuning designs.
Differentiable Prompt Learning (DPL) is a method for adapting large pre-trained vision-language models such as CLIP to downstream tasks by automatically optimizing the configuration of continuous (so…