Targeted Supervised Contrastive Learning for Long-Tailed Recognition
Authors
Authors
- Dina Katabi
- Piotr Indyk
- Rogerio Feris
- Tianhong Li
- Peng Cao
- Yuan Yuan
- Lijie Fan
- Yuzhe Yang
Authors
- Dina Katabi
- Piotr Indyk
- Rogerio Feris
- Tianhong Li
- Peng Cao
- Yuan Yuan
- Lijie Fan
- Yuzhe Yang
Published on
06/24/2022
Real-world data often exhibits long tail distributions with heavy class imbalance, where the majority classes can dominate the training process and alter the decision boundaries of the minority classes. Recently, researchers have investigated the potential of supervised contrastive learning for long-tailed recognition, and demonstrated that it provides a strong performance gain. In this paper, we show that while supervised contrastive learning can help improve performance, past baselines suffer from poor uniformity brought in by imbalanced data distribution. This poor uniformity manifests in samples from the minority class having poor separability in the feature space. To address this problem, we propose targeted supervised contrastive learning (TSC), which improves the uniformity of the feature distribution on the hypersphere. TSC first generates a set of targets uniformly distributed on a hypersphere. It then makes the features of different classes converge to these distinct and uniformly distributed targets during training. This forces all classes, including minority classes, to maintain a uniform distribution in the feature space, improves class boundaries, and provides better generalization even in the presence of long-tail data. Experiments on multiple datasets show that TSC achieves state-of-the-art performance on long-tailed recognition tasks.
Please cite our work using the BibTeX below.
@InProceedings{Li_2022_CVPR,
author = {Li, Tianhong and Cao, Peng and Yuan, Yuan and Fan, Lijie and Yang, Yuzhe and Feris, Rogerio S. and Indyk, Piotr and Katabi, Dina},
title = {Targeted Supervised Contrastive Learning for Long-Tailed Recognition},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2022},
pages = {6918-6928}
}