Planar Object Tracking via Weighted Optical Flow

Abstract

We propose WOFT - a novel method for planar object tracking that estimates a full 8 degrees-of-freedom pose, i.e. the homography w.r.t. a reference view. The method uses a novel module that leverages dense optical flow and assigns a weight to each optical flow correspondence, estimating a homography by weighted least squares in a fully differentiable manner. The trained module assigns zero weights to incorrect correspondences (outliers) in most cases, making the method robust and eliminating the need of the typically used non-differentiable robust estimators like RANSAC. The proposed weighted optical flow tracker (WOFT) achieves state-of-the-art performance on two benchmarks, POT-210 and POIC, tracking consistently well across a wide range of scenarios.

WOFT results

We provide the WOFT outputs on POT-280 (also valid for its subset POT-210), and POIC. The results are in the standard format for each dataset: Each line contains 4 pairs of numbers, specifying the x, y coordinates of the target corners. The POIC result format additionally includes frame name on each line and a header line. WOFT Alignment error plot on the POT-210 dataset

WOFT Alignment error plot on the POT-210 dataset

Qualitative comparison with other methods

Here is an example with reflections obstructing the target. Our WOFT method (bottom right) works, while competing methods lose the target or flicker.

POT-210 re-annotation

We have carefully manually re-annotated a subset of the POT-210 dataset as described in the paper and the supplementary. We provide the re-annotation here.

Bibtex

Please cite our paper in case you use its source code, results, or the re-annotation.

@inproceedings{serych2023planar,
               title={Planar Object Tracking via Weighted Optical Flow},
               author={{\v{S}}er{\'y}ch, Jon{\'{a}}{\v{s}} and Matas, Ji{\v{r}}{\'{i}}},
	       booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
               pages={1593--1602},
               year={2023}
}

Acknowledgments

This work was supported by Toyota Motor Europe, by CTU student grant SGS20/171/OHK3/3T/13, and by the Research Center for Informatics project CZ.02.1.01/0.0/0.0/16_019/0000765 funded by OP VVV.