Linear Formulations for Rigid and Non-Rigid 2D-to-3D Registration

Francesc Moreno-Noguer (UPC Barcelona, Spain)

Abstract:

In this talk, I will first present an approach to the PnP problem -the estimation of the pose of a calibrated camera from n point correspondences between an image and a 3D model— whose computational complexity grows linearly with n. Our central idea is to express the 3D points as a weighted sum of four virtual control points. The problem then reduces to estimating the coordinates of these control points in the camera referential, which can be done in O(n) using simple linearization techniques.

I will then show that the same type of approach can be applied to register non-rigid 3D surfaces. However, since monocular non-rigid reconstruction is severely under-constrained we will have to consider additional constraints, either based on local rigidity (to reconstruct deformable and inextensible surfaces), or based on shading coherence (to reconstruct deformable and stretchable surfaces).

In the final part of the talk, I will discuss the major limitations of these linear formulations and propose a novel and alternative stochastic exploration strategy. I will present results both for non-rigid shape and human pose recovery.