Results obtained in the previous section can be used to merge virtual objects with the input video sequence. One can import the final calibration of each single image of the video sequence and the reconstructed crude 3D environment into a computer graphics system to generate augmented images.
In a computer graphics system virtual cameras can be instantiated which correspond to the retrieved calibrations of each image. The image calibrations include translation, rotation, focal length, principal point and skew of the actual real camera that took the image at that time. Typically computer graphics systems do not support skew of the camera. This can easily be adapted in the software of the computer graphics system by including a skew transformation after performing the typical perspective transformation as explained in [51]. We use the Visualization ToolKit [170] as our computer graphics package. The virtual cameras can now be used to create images of virtual objects.
These virtual objects need to be properly registered in the real 3D environment. This is achieved in the following manner. First virtual objects are placed roughly within the 3D environment using its crude reconstruction. Fine tuning of the position is achieved by viewing the result of a rough positioning by several virtual cameras and overlaying the rendering results from these virtual cameras on their corresponding real images in the video sequence. See Figure 8.17. Using specific features in the real video images that were not reconstructed in the crude 3D environment a better and final placement of all virtual objects can be obtained. Note that at this stage of the implementation we don't take into account occlusions when rendering virtual objects.
 |