Selective Prediction in the presence of Out-of-Distribution data

Most machine learning methods assume that the training and test data are drawn from the same distribution. However, this assumption is often violated in practice. Test data frequently include so-called out-of-distribution (OOD) samples, i.e. inputs generated from a distribution different from the in-distribution (ID) that produced the training data. This issue can be addressed by constructing a selective predictor that abstains from making predictions when the input is either (i) likely to be an OOD sample or (ii) an in-distribution (ID) sample that is nonetheless likely to be misclassified. This type of predictor is referred to as a SCOD predictor. We make three contributions to the SCOD problem: First, we formulate the optimal SCOD strategy and characterize its structure. Second, we prove that SCOD predictors cannot be reliably learned in PAC sense using only in-distribution data. Third, we propose an effective method for learning SCOD predictors using an unlabeled mixture of ID and OOD data.


Reject option prediction

Machine learning (ML) models typically make a prediction even when they are likely to be wrong, which creates serious risks in practice. ML with a reject option solves this by abstaining when the prediction is unreliable. Our paper makes the first unifying contribution to this field: it shows that previously fragmented formulations of reject-option prediction are equivalent and introduces the first statistically consistent algorithm applicable across classification, regression, and structured outputs.


Learning Markov Network Classifiers

Markov Network classifier is a generic model for structured output prediction which provides easy way to encode prior knowledge about label interactions. We propose Maximum-Margin based algorithms for learning MN classifier from examples given arbitrary graph of label interactions. Our work extends SOTA in two ways. First, we show how to efficiently learn MN from examples with missing labels by converting the problem into a convex optimization tractable by standard gradient methods. Second, we show how that the proposed learning algorithm is statistically consistent, i.e. with enough data it converges to the optimal MN classifier.


CMP Tomato Taste Comparison Challenge (TTCC)

The Tomato Taste Comparison Challenge evaluates the taste of tomato samples provided by researchers at Center for Machine Perception. One high-level motivation is to allow researchers to compare progress in the cultivation of delicious tomatoes over a span of several years. Another motivation is to compete for the glory of winning. No competition, no progress!


Hairstyle Transfer between Face Images

We propose a neural network which takes two inputs, a hair image and a face image, and produces an output image having the hair of the hair image seamlessly merged with the inner face of the face image. Our architecture consists of neural networks mapping the input images into a latent code of a pretrained StyleGAN2 which generates the output high-definition image. We propose an algorithm for training parameters of the architecture solely from synthetic images generated by the StyleGAN2 itself without the need of any annotations or external dataset of hairstyle images. We empirically demonstrate the effectiveness of our method in applications including hair-style transfer, hair generation for 3D morphable models, and hair-style interpolation. Fidelity of the generated images is verified by a user study and by a novel hairstyle metric proposed in the paper.


License Plate recognition and Super-resolution from Low-Resolution Videos

We developed CNN architecture recognizing license plates from a sequence of low-resolution videos. Our system works reliably on videos which are unreadable by humans. We also show how to a generate super-resolution LP images from low-res videos.


Learning CNNs from weakly annotated facial images

We show how to learn CNNs for face recognition using weakly annotated images where the annotation is assigned to a set of candidate faces rather than a single face like in the standard supervised setting. We use our method to create a database containing more than 300k faces of celebrities each annotated with his/her biological age, gender and identity.