What is model compression distillation?
by SMEBOOK (admin) · February 28, 2021
Model compression distillation or Knowledge distillation (KD) is commonly deemed as an effective model compression technique in which a compact model (student) is trained under the supervision of a larger pre-trained model or an ensemble of models (teacher). In knowledge distillation, a large, complex model is trained on a large dataset. When this large model can generalize and perform well on unseen data, it is transferred to a smaller network. The larger model is also known as the teacher model and the smaller network is also known as the student network.
SMEBOOK is reinventing the management of tech companies’ assets by providing a matchmaking algorithm capable of recommending partnerships according to needs and interests.