Intensity cross-correlation

A better approach should take the information of more pixels into account. This could be achieved by comparing local neighborhoods of corners through intensity cross-correlation. As a neighborhood a small window of $(2N+1)\times (2N+1)$ pixels centered around the corner could be taken. For the points $(x,y)$ and $(x',y')$ the similarity measure is obtained as follows:

$$C = \sum_{i=-N}^{N} \sum_{j=-N}^{N} (I(x-i,y-j)-\overline{I\,})(I'(x'-i,y'-j)-\overline{I'})$$ (D3)

with $I$ and $I'$ the intensity values at a certain point and $\overline{I}$ and $\overline{I'}$ the mean intensity value of the considered neighborhood. Typically $N=3$ which yields $7 \times 7$ windows.

In figure 4.2 corresponding parts of two images are shown. In each the position of 5 corners is indicated. In figure 4.3 the neighborhood of each of these corners is shown. The intensity cross-correlation was computed for every possible combination. This is shown in Table 4.1. It can be seen that in this case the correct pair matches all yield the highest cross-correlation values (i.e. highest values on diagonal). However, the combination 2-5, for example, comes very close to 2-2. In practice, one can certainly not rely on the fact that all matches will be correct and automatic matching procedures should therefore be able to deal with important fraction of outliers. Therefore, further on robust matching procedures will be introduced.

Figure 4.2: Detail of the images of figure 4.1 with 5 corresponding corners.

Figure 4.3: Local neighborhoods of the 5 corners of figure 4.2.

Table 4.1: Intensity cross-correlation values for all possible combinations of the 5 corners indicated figure 4.2.

0.9639	-0.3994	-0.1627	-0.3868	0.1914
-0.0533	0.7503	-0.4677	0.5115	0.7193
-0.1826	-0.3905	0.7730	0.1475	-0.7457
-0.2724	0.4878	0.1640	0.7862	0.2077
0.0835	0.5044	-0.4541	0.2802	0.9876

If one can assume that the motion between two images is small (which is needed anyway for the intensity cross-correlation measure to yield good results), the location of the feature can not change widely between two consecutive views. This can therefore be used to reduce the combinatorial complexity of the matching. Only features with similar coordinates in both images will be compared. For a corner located at $(x,y)$, only the corners of the other image with coordinates located in the interval $[x-w_i,x+w_i]\times[y-h_i,y+h_i]$. $w_i$ and $h_i$ are typically $10\%$ or $20\%$ of the image.