Karel Zimmermann, Tomas Svoboda, Jiri Matas: Multiview 3D Tracking with an Incrementally Constructed 3D Model

We propose a multiview tracking method for approximately rigid objects. Assuming that a part of the object is visible in at least two cameras, a partial 3D model is reconstructed in terms of a collection of small 3D planar patches of arbitrary topology. The 3D representation, recovered fully automatically, allows to formulate tracking as gradient minimization in pose (translation, rotation) space. As the object moves, the 3D model is incrementally updated. A virtuous circle emerges: tracking enables composition of the partial 3D model; the 3D model facilitates and robustifies the multiview tracking. Experimentally, we demonstrate that the interleaved track-and-reconstruct approach successfully tracks a 360 degrees turn-around and a wide range of motions. As a side product, a 3D model is delivered. Monocular tracking is also possible after the model is constructed. Using more cameras, however, significantly increases stability in critical poses and moves.