Detailed description

Recomended scenario:

• point correspondences

  Extraction and searching for corresponding image points in different images is complicated problem itself. In your work you should define point correspondences manualy by mouse.

• geometry

  From epipolar geometry to projective reconstruction

  start with epipolar geometry - expressed by fundamental matrix F. F constraints two corresponding image points x' and x with equation x'^TFx = 0 (i).
  
  Rearange epipolar constraint (i) to system of linear equations Af = 0 where A is 9xN matrix for N corresponding pairs and f is fundamental matrix written in 9x1 vector. Find f using SVD decomposition on A.
  F matrix has rank 2. Force it to be rank 2 using SVD by zeroing smallest singular value.
  
  Calculate canonical camera pair from fundamental matrix.
  This can be done by setting first camera to P1 = [I | 0] where I is 3x3 identity matrix.
  Second camera is then P2 = [ -[e']_xF | e'] where e' is right epipole (e'F = 0). [v]_x denotes skew symetric cross product matrix for vector v = (x y z):
  

  [v]x =

  [ 0 -z y  ] [ z 0 -x  ] [ -y x 0  ]

  
  
  Warning! normalization required
  As pointed in RI Hartley, "In Defence of the 8-point Algorithm", ICCV 1995, pp. 1064-1070 not normalized data does not lead to precise result. In order to get good result for linear algorithm for solving F-matrix (described above) you have to perform data normalization as follows:
  
  

  • rescale the camera points to zero mean and root 2 variance: x_normalized=H₁x and x'_normalized=H₂x'
  • calculate least squares fit Fundamental Matrix: using normalized coordinates, SVD and setting last singular value to 0
  • remove the normalization: H₂^T F H₁

  
  
  For detailed information see Hartley, Zisserman: Multiple View Geometry in Computer Vision (online)

• calibration

  Note that if you had at least two cameras then you would calculate 3D position of every corresponding pair of 2D points. For this purpose rearrange equation x_i = P_iX where the only unknown is X.
  
  Simple observation : a x_ij = P_iX_j =  = P_i I X_j =  = P_iHH^-1X_j
  It is easy to see now that there are infinitely many projective reconstructions with respect to 4x4 homography H.
  
  Calibration means to put reconstructed projective space into correct metric or euclidean space. This is done be claiming that camera has real physical properties - like focal length can be varying but CCD chip is not skewed and principal point is constant...
  Again there are many possible ways how to do it. It is possible to calculate calibration directly from images (3 and more, according to camera model).
  
  For your purpose it is sufficient to calibrate cameras from partial knowledge of observed space.
  
  2 possibilities :

  • Calibration from knowledge of part of your image.
    

    If you are able to measure distances or angles between points you can simply calculate H. Len X_i and X_j be two points for which we have measured the distances in real space. We have constraint ||H*(X_i - X_j)|| = d. Similary if we know that two lines defined by points X_i, X_j, X_k, X_l are perpendicular we have constraint ( H*(X_i - X_j) , H*(X_k - X_l) )= 0 , where ( , ) denotes scalar product.
    Calculate H and use it to align reconstructed space with real world.

  • Calibration by independent object.
  Camera P can be decomposed to calibration 3x3 matrix K, rotation 3x3 matrix R and translation vector t such that P = K[R -Rt]. K is upper triangular matrix. Basic idea is : calculate calibration matrix K from pictures of known object. Then, without changing internal camera parameters (zoom...) move (R, t) to the desired place and take pictures of unknown object.
  Such calibration can be divided into following steps:
  
  • take pictures of known object,
  • create projective reconstruction: cameras P_i and 3D points Xⁱ,
  • calculate 4x4 homography H such that a X_i-known = H X_i. For more the details see Marc Pollefeys online tutorial,
  • transform cameras with H^-1 and 3D structure with H,
  • decompose cameras using RQ decomposition to obtain calibration matrix K,
    
    
  • calculate homography H which will change calibration matrix of the other reconstruction to desired form (obtained from pictures of the known object) K.
    
    Note: If camera calibration matrix was know, you would "undistort" measured 2D points - x'=K^-1x. Epipolar geometry for calibrated data is described by essential matrix E with x'^TEx = 0. Unlike F, E matix can be decomposed to rotation matrix and translation vector directly using SVD decomposition see Hartley, Zisserman : Multiple View Geometry in Computer Vision (online)

• result - measure distance between two specified points.