Exploiting scene constraints

The epipolar constraint restricts the search range for a corresponding point ${\tt m}_k$ in one image to the epipolar line in the other image. It imposes no restrictions on the object geometry other that the reconstructed object point ${\tt M}$ lays on the line of sight ${\tt L}_k$ from the projection center of ${\bf P}_k$ and through the corresponding point ${\tt m}_k$ as seen in Figure 7.1(left). The search for the corresponding point ${\tt m}_l$ is restricted to the epipolar line but no restrictions are imposed along the search line.

If we now think of the epipolar constraint as being a plane spanned by the line of sight ${\tt L}_k$ and the baseline connecting the camera projection centers, then we will find the epipolar line by intersecting the image plane ${\bf I}_l$ with this epipolar plane.

This plane also intersects the image plane ${\bf I}_k$ and it cuts a 3D profile out of the surface of the scene objects. The profile projects onto the corresponding epipolar lines in ${\bf I}_k$ and ${\bf I}_l$ where it forms an ordered set of neighboring correspondences, as indicated in Figure 7.1 (right).

For well behaved surfaces this ordering is preserved and delivers an additional constraint, known as 'ordering constraint'. Scene constraints like this can be applied by making weak assumptions about the object geometry. In many real applications the observed objects will be opaque and composed out of piecewise continuous surfaces. If this restriction holds then additional constraints can be imposed on the correspondence estimation. Koschan[92] listed as many as 12 different constraints for correspondence estimation in stereo pairs. Of them, the most important apart from the epipolar constraint are:

1. Ordering Constraint: For opaque surfaces the order of neighboring correspondences on the corresponding epipolar lines is always preserved. This ordering allows the construction of a dynamic programming scheme which is employed by many dense disparity estimation algorithms [53,26,39].

2. Uniqueness Constraint: The correspondence between any two corresponding points is bidirectional as long as there is no occlusion in one of the images. A correspondence vector pointing from an image point to its corresponding point in the other image always has a corresponding reverse vector pointing back. This test is used to detect outliers and occlusions.

   Figure 7.1: Object profile triangulation from ordered neighboring correspondences (left). Rectification and correspondence between viewpoints $k$ and $l$ (right).

   

3. Disparity Limit: The search band is restricted along the epipolar line because the observed scene has only a limited depth range (see Figure 7.1, right).

4. Disparity continuity constraint: The disparities of the correspondences vary mostly continuously and step edges occur only at surface discontinuities. This constraint relates to the assumption of piecewise continuous surfaces. It provides means to further restrict the search range. For neighboring image pixels along the epipolar line one can even impose an upper bound on the possible disparity change. Disparity changes above the bound indicate a surface discontinuity.

All above mentioned constraints operate along the epipolar lines which may have an arbitrary orientation in the image planes. The matching procedure is greatly simplified if the image pair is rectified to a standard geometry. How this can be achieved for an arbitrary image pair is explained in the Section 7.2.2. In standard geometry both image planes are coplanar and the epipoles are projected to infinity. The rectified image planes are oriented such that the epipolar lines coincide with the image scan lines. This corresponds to a camera translated in the direction of the $x$-axis of the image. An example is shown in figure 7.2. In this case the image displacements between the two images or disparities are purely horizontal.

Figure 7.2: Standard stereo setup