Castle sequence

The Castle sequence consists of images of 720x576 pixel resolution taken with a standard semi-professional camcorder that was moved freely in front of a building. The quantitative performance of correspondence linking can be tested in different ways. One measure already mentioned is the visibility of an object point. In connection with correspondence linking, we have defined visibility $V$ as the number of views linked to the reference view. Another important feature of the algorithm is the density and accuracy of the depth maps. To describe its improvement over the 2-view estimator, we define the fill rate $F$ and the average relative depth error $E$ as additional measures.

Visibility $V[views]$:	average number of views linked to the reference image.
Fill Rate $F[\%]$:	$\frac{\mbox{Number of valid pixels}}{\mbox{Total number of pixels}}$
Depth error $E[\%]$:	standard deviation of relative depth error $e_d$ for all valid pixels.

The 2-view disparity estimator is a special case of the proposed linking algorithm, hence both can be compared on an equal basis. The 2-view estimator operates on the image pair $(k,k+1)$ only, while the multi-view estimator operates on a sequence $1 < k < N$ with $N >= 3$. The above defined statistical measures were computed for different sequence lengths N. Figure 7.18 displays visibility and relative depth error for sequences from 2 to 15 images, chosen symmetrically around the reference image. The average visibility $V$ shows that for up to 5 images nearly all views are utilized. For 15 images, at average 9 images are linked. The amount of linking is reflected in the relative depth error that drops from 5% in the 2 view estimator to about 1.2% for 15 images.

Figure 7.18: Statistics of the castle sequence. Influence of sequence length $N$ on visibility $V$ and relative depth error $E$. (left) Influence of minimum visibility $V_{min}$ on fill rate $F$ and depth error $E$ for $N=11$ (center). Depth map (above: dark=near, light=far) and error map (below: dark=large error, light=small error) for $N=11$ and $V_{min}=3$ (right).

Linking two views is the minimum case that allows triangulation. To increase the reliability of the estimates, a surface point should occur in more than two images. We can therefore impose a minimum visibility $V_{min}$ on a depth estimate. This will reject unreliable depth estimates effectively, but will also reduce the fill rate of the depth map.

The graphs in figure 7.18(center) show the dependency of the fill rate and depth error on minimum visibility for N=11. The fill rate drops from 92% to about 70%, but at the same time the depth error is reduced to 0.5% due to outlier rejection. The depth map and the relative error distribution over the depth map is displayed in Figure 7.18(right). The error distribution shows a periodic structure that in fact reflects the quantization uncertainty of the disparity resolution when it switches from one disparity value to the next.