Relation between projection matrices and image homographies

The homographies that will be discussed here are collineations from ${\cal P}^2 \rightarrow {\cal P}^2$. A homography ${\bf H}$ describes the transformation from one plane to another. A number of special cases are of interest, since the image is also a plane. The projection of points of a plane into an image $i$ can be described through a homography ${\bf H}_{{\tt\Pi} i}$. The matrix representation of this homography is dependent on the choice of the projective basis in the plane.

As an image is obtained by perspective projection, the relation between points ${\tt M}_{\tt\Pi}$ belonging to a plane ${\tt\Pi }$ in 3D space and their projections ${\tt m}_{{\tt\Pi}i}$ in the image is mathematically expressed by a homography ${\bf H}_{{\tt\Pi} i}$. The matrix of this homography is found as follows. If the plane ${\tt\Pi }$ is given by ${\tt\Pi} \sim [{\tt\pi}^\top \, 1]^\top$ and the point ${\tt M}_{\tt\Pi}$ of ${\tt\Pi }$ is represented as ${\tt M}_{\tt\Pi} \sim [{\tt m}^\top_{\tt\Pi} \, 1]^\top$, then ${\tt M}_{\tt\Pi}$ belongs to ${\tt\Pi }$ if and only if $0 = {\tt\Pi}^\top {\tt M}_{\tt\Pi} = {\tt\pi}^\top {\tt m}_{\tt\Pi}+1$. Hence,

$${\tt M}_{\tt\Pi} \sim \left[ \begin{array}{c} {\tt m}_{\tt\P... ...{\tt\pi}^\top \end{array} \right] {\tt m}_{\tt\Pi} \enspace .$$ (C13)

Now, if the camera projection matrix is ${\bf P}_i=[{\bf A}_i \vert {\tt a}_i ]$, then the projection ${\tt m}_{{\tt\Pi}i}$ of ${\tt M}_{\tt\Pi}$ onto the image is

$${\tt m}_{{\tt\Pi}i} \sim {\bf P}_{i} {\tt M}_{\tt\Pi} = [{\bf A}_i \vert {\tt a}_i ] \left[ \begin{array}{c} {\bf I}_{3 \times 3} \\ -{\tt\pi}^\top \end{array} \right] {\tt m}_{\tt\Pi}$$

$$= [{\bf A}_i-{\tt a}_i {\tt\pi}^\top ] {\tt m}_{\tt\Pi} \enspace .$$ (C14)

Consequently, ${\bf H}_{{\tt\Pi}i} \sim {\bf A}_i - {\tt a}_i {\tt\pi}^\top$.

Note that for the specific plane ${\tt\Pi}_{\tt REF}= [0\,0\,0\,1]^\top$ the homographies are simply given by ${\bf H}_{{\tt REF}i} \sim {\bf A}_i$.

It is also possible to define homographies which describe the transfer from one image to the other for points and other geometric entities located on a specific plane. The notation ${\bf H}_{ij}^{\tt\Pi}$ will be used to describe such a homography from view $i$ to $j$ for a plane ${\tt\Pi }$. These homographies can be obtained through the following relation ${\bf H}_{ij}^{\tt\Pi} = {\bf H}_{{\tt\Pi}j}{\bf H}_{{\tt\Pi}i}^{-1}$ and are independent to reparameterizations of the plane (and thus also to a change of basis in ${\cal P}^3$).

In the metric and Euclidean case, ${\bf A}_i={\bf K}_i {\bf R}_i^\top$ and the plane at infinity is ${\tt\Pi}_\infty=[0 0 0 1]^\top$. In this case, the homographies for the plane at infinity can thus be written as:

$${\bf H}^\infty_{ij} = {\bf K}_i {\bf R}_{ij}^\top {\bf K}_i^{-1} \, ,$$ (C15)

where ${\bf R}_{ij}= {\bf R}_i^\top {\bf R}_j$ is the rotation matrix that describes the relative orientation from the $j^{th}$ camera with respect top the $i^{th}$ one.

In the projective and affine case, one can assume that ${\bf P}_1 = [{\bf I}_{3 \times 3} \vert {\tt0}_3]$ (since in this case ${\bf K}_i$ is unknown). In that case, the homographies ${\bf H}_{{\tt\Pi}1} \sim {\bf I}_{3 \times 3}$ for all planes; and thus, ${\bf H}_{1i}^{\tt REF} = {\bf H}_{{\tt REF}i}$. Therefore ${\bf P}_i$ can be factorized as

$${\bf P}_i = [ {\bf H}_{1i}^{\tt REF} \vert {\tt e}_{1i} ]$$ (C16)

where ${\tt e}_{1i}$ is the projection of the center of projection of the first camera (in this case, $[0 \, 0\, 0\, 1]^\top$) in image $i$. This point ${\tt e}_{1i}$ is called the epipole, for reasons which will become clear in Section 3.3.1.

Note that this equation can be used to obtain ${\bf H}_{1i}^{\tt REF}$ and ${\tt e}_{1i}$ from ${\bf P}_i$, but that due to the unknown relative scale factors ${\bf P}_i$ can, in general, not be obtained from ${\bf H}_{1i}^{\tt REF}$ and ${\tt e}_{1i}$. Observe also that, in the affine case (where ${\tt\Pi}_\infty=[0 0 0 1]^\top$), this yields ${\bf P}_i = [ {\bf H}_{1i}^\infty \vert {\tt e}_{1i} ]$.

Combining equations (3.14) and (3.16), one obtains

$${\bf H}_{1i}^{\tt\Pi} = {\bf H}_{1i}^{\tt REF} - {\tt e}_{1i} {\tt\pi}^\top$$ (C17)

This equation gives an important relationship between the homographies for all possible planes. Homographies can only differ by a term ${\tt e}_{1i} [{\tt l-\pi}']^\top$. This means that in the projective case the homographies for the plane at infinity are known up to 3 common parameters (i.e. the coefficients of ${\tt\pi}_\infty$ in the projective space).

Equation (3.16) also leads to an interesting interpretation of the camera projection matrix:

$${\tt m}_1 \sim [ {\bf I}_{3 \times 3} \vert {\tt0}_3] \left[\begin{array}{c} {\tt m} \\ 1 \end{array}\right] = {\tt m}$$ (C18)

$${\tt m}_i \sim [ {\bf H}_{1i}^{\tt REF} \vert {\tt e}_{1i} ] \left[\begin{array}... ...\tt m} \\ 1 \end{array}\right] = {\bf H}_{1i}^{\tt REF} {\tt m} + {\tt e}_{1i}$$ (C19)

$$= \lambda {\bf H}_{1i}^{\tt REF} {\tt m}_1 + {\tt e}_{1i} = {\bf P}... ...\end{array}\right] + \left[\begin{array}{c} {\tt0}_3 \\ 1 \end{array}\right] )$$ (C20)

In other words, a point can thus be parameterized as being on the line through the optical center of the first camera (i.e. $[0 0 0 1]^\top$) and a point in the reference plane ${\tt\Pi }_{\tt REF}$. This interpretation is illustrated in Figure 3.4.

Figure 3.4: A point ${\tt M}$ can be parameterized as ${\tt C}_1+\lambda {\tt M}_{\tt REF}$. Its projection in another image can then be obtained by transferring ${\tt m}_1$ according to ${\tt \Pi}_{\tt REF}$ (i.e. with ${\bf H}^{\tt REF}_{1i}$) to image $i$ and applying the same linear combination with the projection ${\tt e}_{1i}$ of ${\tt C}_1$ (i.e. ${\tt m}_i \sim {\tt e}_{1i} + \lambda {\bf H}^{\tt REF}_{1i} {\tt m}_1$).