O que é Self-supervised learning, pre-training, fine-tuning?

Self-supervised learning trains on a pretext task whose targets are made from the unlabelled data themselves: a masked part of the input, the input’s future, or another view of the same example. Pre-training is such a fit, done before the task’s labels are used; fine-tuning continues training the pre-trained network, with a new output head, on the labelled task, usually at a small learning rate.