Relating to other views

The procedure to add a view described in the previous section only relates the image to the previous image. In fact it is implicitly assumed that once a point gets out of sight, it will not come back. Although this is true for many sequences, this assumptions does not always hold. Assume that a specific 3D point got out of sight, but that it is visible again in the last two views. In this case a new 3D point will be instantiated. This will not immediately cause problems, but since for the system these two 3D points are unrelated, nothing enforces their position to correspond.

This is especially crucial for longer image sequences where the errors accumulate. It results in a degraded calibration or even causes the failure of the algorithm after a certain number of views.

A possible solution consists of relating every new view with all previous views using the procedure of Section 4.3. It is clear that this would require a considerable computational effort. We propose a more pragmatic approach. This approach worked well in the cases we encountered (see Section 8.3).

Let $\tilde{\bf P}_i$ be the initial estimate of the camera pose obtained as described in the previous section. A criterion is then used to define which views are close to the actual view. All these close views are matched with the actual view (as described in Section 4.3). For every close view a set of potential 2D-3D correspondences is obtained. These sets are merged and the camera projection matrix ${\bf P}_i$ is reestimated using the same robust procedure as described in the previous section. Figure 5.3 illustrates this approach.

We applied a very simple criterion to decide if views were close or not. It worked well for the applications we had in mind, but it could easily be refined if needed. The position ${\tt t}_i$ of the camera is extracted from $\tilde{\bf P}_i$ and the distance $d_{ij}$ to all the other camera positions is computed. Close views are selected as views for which $d_{ij}<1.6d_{i(i-1)}$. Note that strictly speaking such measure is meaningless in projective space, but since a quasi-metric initialization was carried out and since only local qualitative comparisons are made the obtained results are good.

Figure 5.3: Sequential approach (left) and extended approach (right). In the traditional scheme view 8 would be matched with view 7 and 9 only. A point ${\tt M}$ which would be visible in views 2,3,4,7,8,9,12,13 and 14 would therefore result in 3 independently reconstructed points. With the extended approach only one point will be instantiated. It is clear that this results in a higher accuracy for the reconstructed point while it also dramatically reduces the accumulation of calibration errors.