We demonstrate that it is possible to track omnidirectional camera in space from image matches and to compute structure from motion to facilitate scene modeling and object recognition.

We implemented the calibration techniques into a single SW platform in MATLAB.

The SW platform is modular and in the present state it supports two different approaches to calibration, the first one is a target based calibration and the second one is an autocalibration from image features. As a first step for both methods, an initial guess of the image center and field of view is obtained using a robust circle fitting algorithm. The target based calibration supports two different calibration targets, the PLANETARIUM object, shown in the first row of the following table and the BOX object, depicted in the second row of the table.

The platform also supports auto calibration using 15-point RANSAC and employing correspondences obtained externally from a matching tool.

The setup: 2 x Kyocera Camera with Nikon FC-E9 fisheye adapter mounted on a car. (+ 2 x DV cam for another experiment)

Images from the sequence

Feature points detected in the omnidirectional images, used for camera motion and scene structure estimation. Also available as a video sequence. The features were detected and camera the motion with scene structure was estimated using a KUL software, into which was an omnidirectional module implemented in cooperation between KUL and CTU.

Horizon line (projection of the ground plane) in the images computed from the camera motion. Also available as a video sequence.

Reconstructed camera path and 3D points, also available as VRML model

The setup: 2 x Kyocera Camera with Nikon FC-E9 fisheye adapter, hand held.

Images from the sequence

Features detected in the images. The fatures were detected using the Wide Baseline Stereo tool at CTU, red and green areas correspond to MSERs (Maximally stable extremal regions) and blue ellipses to APTS (affine invariant points).

Epipolar estimation results (two consecutive images shown, also available as a video sequence.):

• Red circle: the ground truth
• Blue upward triangle: the epipole selected by the number of supports (angular error)
• Green downward triangle: the epipole selected by soft voting (angular error)
• Green dots and numbers: the five points which are used to compute the green epipole.
• Magenta square: the epipole found by PROSAC soft voting but using the reprojection error.

Omnidirectional image unwarped to 3 perspective views (left, front, right) for processing using techniques for detection of cars and pedestrians developed for perspective images, without the need for adapting them to omnidirectional case. Also available as a video sequence.