Depth and uncertainty

Assume a 3D surface point ${\tt M}$ that is projected onto its corresponding image points ${\tt m}_k = {\bf P}_k {\tt M} , {\tt m}_{l} = {\bf P}_{l} {\tt M}$. The inverse process holds for triangulating ${\tt M}$ from the corresponding point pair $({\tt m}_k, {\tt m}_{l})$. We can in fact exploit the calibrated camera geometry and express the 3D point ${\tt M}$ as a depth value $d_{\tt M}$ along the known line of sight ${\tt L}_{{\tt m}_k}$ that extends from the camera projection center through the image correspondence ${\tt m}_k$. Triangulation computes the depth as the length of ${\tt L}_{{\tt m}_k}$ connecting the camera projection center and the locus of minimum distance between the corresponding lines of sight. The triangulation is computed for each image point and stored in a dense depth map associated with the viewpoint.

The depth for each reference image point ${\tt x}_k$ is improved by the correspondence linking that delivers two lists of image correspondences relative to the reference, one linking down from $k \rightarrow 1$ and one linking up from $k \rightarrow N$. For each valid corresponding point pair $({\tt m}_i, {\tt m}_k)$ we can triangulate a consistent depth estimate $d({\tt m}_k, {\tt m}_l)$ along ${\tt L}_{{\tt m}_k}$ with $e_l$ representing the depth uncertainty. Figure 7.17(left) visualizes the decreasing uncertainty interval during linking. While the disparity measurement resolution $\Delta D$ in the image is kept constant (at 1 pixel), the reprojected depth error $e_l$ decreases with the baseline.

Figure 7.17: Depth fusion and uncertainty reduction from correspondence linking (left). Detection of correspondence outliers by depth interval testing (right).