Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models.

Tue, 01 Apr 2025 01:00:00 +0200

Junjie Yang, Junhao Song, Xudong Han, Ziqian Bi, Tianyang Wang, Chia Xin Liang, Xinyuan Song, Yichao Zhang, Qian Niu, Benji Peng, Keyu Chen, Ming Liu:
Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models. CoRR abs/2504.13825 (2025)