Browse Teach Curate

Multi-Task Model Merging via Adaptive Projective Gradient Descent

A data-free method that merges independently fine-tuned expert models into one multi-task model by framing merging as a constrained optimization problem: minimize the gap between the merged model and each task-specific model while preserving knowledge shared across tasks. It projects task-vector gradient updates into a subspace shared by all tasks and treats the per-task merging coefficients as adaptive, training-free learning rates.

Data Text

Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent Yongxian Wei 1 Anke Tang

Model merging combines several independently fine-tuned expert models into a single multi-task model without retraining or access to their training data, operating on task vectors (each fine-tuned mo…