Interleaved 3D Object Modeling and Tracking

Karel Zimmermann, Tomas Svoboda, Jiri Matas
Center for Machine Perception (CMP)


We propose a multiview tracking method for rigid objects. Assuming that a part of the object is visible in at least two cameras, a partial 3D model is reconstructed in terms of a collection of small 3D planar patches of arbitrary topology. The 3D representation, recovered fully automatically, allows to formulate tracking as gradient minimization in pose (translation, rotation) space. As the object moves, the 3D model is incrementally updated. A virtuous circle emerges: tracking enables composition of the partial 3D model; the 3D model facilitates and robustifies the multiview tracking.

We demonstrate experimentally that the interleaved track-and-reconstruct approach successfully tracks a 360 degrees turn-around and a wide range of motions. Monocular tracking is also possible after the model is constructed. Using more cameras, however, significantly increases stability in critical poses and moves. We demonstrate how to exploit the 3D model to increases stability in the presence of uneven and/or changing illumination.



[1] K.Zimmermann, T. Svoboda, J. Matas Multiview 3D Tracking with an Incrementally Constructed 3D Model, Third International Symposium on 3D Data Processing, Visualization and Transmission, Chapel Hill, USA, 2006, to appear.