This deep-learning-systems paper introduces GRASP (Gradient-based Retention of Adaptive Singular Parameters), a framework for compressing large language models. Prior work shows many transformer layers are functionally redundant and can be pruned to cut inference cost, but indiscriminate layer pruning degrades performance sharply. Instead of deleting whole layers, GRASP replaces redundant layers with only a minimal set of sensitivity-aware singular components: it uses gradient-based attribution over a small calibration dataset to identify and retain the singular values (from an SVD-style decomposition of the layers) that matter most for the model's behavior, discarding the rest. This yields efficient compression with minimal added overhead while preserving accuracy. Across multiple LLMs, GRASP consistently outperforms existing compression baselines, retaining about 90% of the original model's performance at a 20% compression ratio. Source code is released.
This deep-learning-systems paper introduces GRASP (Gradient-based Retention of Adaptive Singular Parameters), a framework for compressing large language models. Prior work shows many transformer laye…