Philipp Schmid • 2/1/2022

Task-specific knowledge distillation for BERT using Transformers and Amazon SageMaker

This technical guide provides an end-to-end example of task-specific knowledge distillation for text classification. It demonstrates how to train a compact BERT-Tiny student model to mimic a larger BERT-base teacher model using the SST-2 dataset, PyTorch, Hugging Face Transformers, and Amazon SageMaker.

0 comments

#Transformers #Text Classification #Amazon Sagemaker