Matching using affinely invariant regions

One can note that the similarity measure presented in the previous section is only invariant to translation and offsets in the intensity values. If important rotation or scaling takes place the similarity measure is not appropriate. The same is true when the lighting conditions differ too much. Therefore the cross-correlation based approach can only be used between images for which the camera poses are not too far apart.

In this section a more advanced matching procedure is presented that can deal with much larger changes in viewpoint and illumination [208]. As should be clear from the previous discussion, it is important that pixels corresponding to the same part of the surface are used for comparison during matching. By assuming that the surface is locally planar and that there is no perspective distortion, local transformations of pixels from one image to the other are described by 2D affine transformations. Such a transformation is defined by three points. At this level we only have one, i.e. the corner under consideration, and therefore need two more. The idea is to go look for them along edges which pass through the point of interest. It is proposed to only use corners having two edges connected to them, as in figure 4.4.

Figure 4.4: Based on the edges in the neighborhood of a corner point $p$ an affinely invariant region is determined up to two parameters $l_1$ and $l_2$.

For curved edges it is possible to uniquely relate a point on one edge with a point on the other edge (using an affine invariant parameterization $l_1$ can be linked to $l_2$), yielding only one degree of freedom. For straight edges two degrees of freedom are left. Over the parallelogram-shaped region (see figure 4.4) functions that reach their extrema in an invariant way for both geometric and photometric changes, are evaluated. Two possible functions are:

$$\frac{\int I(x,y) dx dy}{\int dx dy} \mbox{ and } \frac{\ve... ...f p-g}\vert} {\vert {\bf p - p}_1 \enspace {\bf p - p}_2\vert}$$ (D4)

with $I(x,y)$ the image intensity, ${\bf g}$ the center of gravity of the region, weighted with image intensity and the other points defined as in figure 4.4.

$${\bf g} = (\frac{\int I(x,y)\, x\, dxdy}{\int I(x,y) \,dxdy} , \frac{\int I(x,y) \,y \,dxdy}{\int I(x,y) \,dxdy})$$ (D5)

The regions for which such an extremum is reached will thus also be determined invariantly. In practice it turns out that the extrema are often not very well defined when two degrees of freedom are left (i.e. for straight edges), but occur in shallow ``valleys''. In these cases more than one function is used at the same time and intersections of these ``valleys'' are used to determine invariant regions.

Now that we have a method at hand for the automatic extraction of local, affinely invariant image regions, these can easily be described in an affinely invariant way using moment invariants [212]. As in the region finding steps, invariance both under affine geometric changes and linear photometric changes, with different offsets and different scale factors for each of the three color bands, is considered.

For each region, a feature vector of moment invariance is composed. These can be compared quite efficiently with the invariant vectors computed for other regions, using a hashing-technique. It can be interesting to take the region type (curved or straight edges? Extrema of which function?) into account as well. Once the corresponding regions have been identified, the cross-correlation between them (after normalization to a square reference region) is computed as a final check to reject false matches.