A parameter-efficient method for efficient model adaptation — Updates: [Sep. 27th, 2022] Add CSP [Sep.29th, 2022] Add V-VL Vision-Language models have been impactive in computer vision community. With the alignment learning between image-text pairs, the trained image encoders are capable of few/zero-shot learning abilities, which could achieve quick model adaptation in a data-efficient manner. …

Explicit and implicit methods for aligning object-level vision and language features without human annotations — Contrastive pre-training is proposed as a self-supervised learning method in computer vision to boost performance for the subsequent supervised fine-tuning. My former blog was a general introduction on this topic: Understanding Contrastive Learning and MoCo What is important in contrastive learning and how it can helpmedium.com

Some methods that make your contrastive pre-training implementation light and simple — Contrastive learning is important for unsupervised pre-training, an approach to initialize the model weights before fine-tuning on labeled datasets. With this initialization, the fine-tuned model could achieve higher performance and label-efficiency. Contrastive learning usually leverages positive and negative pairs of input samples. Positive pairs are formed by different data augmentations…

Appropriate data sampling methods matter for training a good model — Data sampling in model training is an overlooked area in machine learning. Machine learning has two sides: modeling and data. Both of them are equally important and should be considered carefully in a real-life machine learning project. However, many textbooks, papers, blogs talk about the modeling side, only few of…

Boost your model performance via multimodal masked auto-encoders — Self-supervised pre-training is a main approach to improve the performance to traditional supervised learning, in which large amount of labeled data is necessary and costly. Among self-supervised learning methods, contrastive learning is popular for its simplicity and efficacy. However, most contrastive learning methods use global vectors in which the details…

Fully leveraging supervision signals in contrastive perspectives — Contrastive pre-training has been widely applied in deep learning. One reason for this is that contrastive pre-training can improve the efficiency of labeled data. During unsupervised contrastive pre-training, the unlabeled images are clustered in the latent space, forming fairly good decision boundaries between different classes. Based on this clustering, the…

Be aware of the differences in training large and tiny neural networks — Training deep neural networks (NN) is difficult, sometimes tricky even for veteran practitioners. In order to reach the highest potential performance of a model given a specific dataset, we need to consider many aspects: hyper-parameter tuning, regularization, validation metrics, normalizations and so on. However, the considerations we should take are…

Dense contrastive learning with active sampling strategy — Contrastive learning (CL) is a self-supervised learning process without labels. Since it can improve model performance economically and effectively, it is applied as a pre-training process in more and more deep learning projects recently, forming the unsupervised pre-training and supervised fine-tuning model training paradigm. However, almost all well applied CL…

Deep Learning without Pixels and Voxels Deep learning is widely applied by many areas, with their representative data formats. For example, in computer vision, deep learning can consume images and videos with convolutional neural networks; in NLP, deep learning can consume text data with the main help from LSTM or…