Correspondence Linking Algorithm

The correspondence linking is described in this section. It concatenates corresponding image points over multiple viewpoints by correspondence tracking over adjacent image pairs. This of course implies that the individually measured pair matches are accurate. To account for outliers in pair matches, some robust control strategies need to be employed to check the validity of the correspondence linking. Consider an image sequence taken from $k = [1,N]$ viewpoints. Assume that the sequence is taken by a camera moving sideways while keeping the object in view. For any view point ${\tt k}$ let us consider the image triple $[{\bf I}_{k-1}, {\bf I}_k, {\bf I}_{k+1}]$. The image pairs (${\bf I}_{k-1}$, ${\bf I}_k$) and (${\bf I}_{k}$, ${\bf I}_{k+1}$) form two stereoscopic image pairs with correspondence estimates as described above. We have now defined 3 representations of image and camera matrices for each viewpoint: the original image ${\bf I}_k$ and projection matrix ${\bf P}_k$, their transformed versions ${\bf I}^{k-1}_{k} , {\bf P}^{k-1}_{k}$ rectified towards view point $k-1$ with transformation ${\bf R}^{k-1}_{k}$ and the transformed ${\bf I}^{k+1}_{k}, P^{k+1}_{k}$ rectified towards viewpoint $k+1$ with mapping ${\bf R}^{k+1}_{k}$ . The Disparity map $D_{(k,k-1)}$ holds the downward correspondences from ${\bf I}^{k-1}_k$ to ${\bf I}^{k}_{k-1}$ while the map $D_{(k,k+1)}$ contains the upward correspondences from ${\bf I}^{k+1}_k$ to ${\bf I}^{k}_{k+1}$. We can now create two chains of correspondence links for an image point ${\tt m}_k$, one up and one down the image index $k$.

Upwards linking:	${{\tt m_{k+1}} = {({\bf R}^{k}_{k+1})}^{-1}} D_{(k,k+1)}[{\bf R}^{k+1}_k {\tt m}_k]$
Downwards linking:	${{\tt m_{k-1}} = {({\bf R}^{k}_{k-1})}^{-1}} D_{(k,k-1)}[{\bf R}^{k-1}_k {\tt m}_k]$

This linking process is repeated along the image sequence to create a chain of correspondences upwards and downwards. Every correspondence link requires 2 mappings and 1 disparity lookup. Throughout the sequence of N images, $2(N-1)$ disparity maps are computed. The multi-viewpoint linking is then performed efficiently via fast lookup functions on the pre-computed estimates.

Due to the rectification mapping transformed image point will normally not fall on integer pixel coordinates in the rectified image. The lookup of an image disparity in the disparity map D will therefore require an interpolation function. Since disparity maps for piecewise continuous surfaces have a spatially low frequency content, a bilinear interpolation between pixels suffices.