What is image-based rendering (IBR)?

• A photorealistic approach to representation of a real environment in a computer, for example for use in virtual reality (VR).

  VR generated by computer graphics	VR using image based rendering

• IBR advantages:
  • Photorealistic output.
  • No 3D model is required.
  • Real-time performance.
  • Stereo output possible.
  • No need for specialized 3D graphics hardware.
• IBR disadvantages:
  • Memory requirements.

The method - X-slits

• X-slits method is used.
• A set of images is the input.
• Novel images are generated by composition of parts of the input images, such as columns or stripes of columns.
• Input images can be captured on a linear or circular path.
• Input images are composed into an image volume, novel image is created by a cut through this volume.
• Geometry of X-slits images

Linear input path

• Linear cut through the image volume.
• Pinhole camera model holds for position of the virtual camera on the input path, pushbroom camera model holds for virtual camera at infinity. X-slits model is used between these two extremes.
• Is used for simulation of a "virtual window" where a computer screen act as a window through which a viewer can look at the scene, demo at CeBit 2005.

Circular input path

• Simple cutting function.
• Each virtual image is a panorama, ususally only a small part is presented to a viewer.
• Is used with a head mounted display - demo "Door of perception" at CeBit 2005.

Why omnidirectional images?

• They capture a wide field of view (FOV). For example, the Nikon FC-E8 lens has FOV 183 degrees.
• There is no limit in the field of view of the IBR images.

Videos

• "Virtual window" generated from a linear input path, demonstrated at CeBit 2005
• "Door of perception generated from a circular input path, demonstrated at CeBit 2005, an overview of the installation.
• "Door of perception generated from a circular input path, demonstrated at CeBit 2005, viewer's view.

References

The BeNoGo project.

Bakstein, Hynek and Pajdla, Tomas. Rendering novel views from a set of omnidirectional mosaic images, in Proceedings of the IEEE Workshop on Omnidirectional Workshop (Omnivis 2003), Madison, USA, 2003. [pdf]

Bakstein, Hynek, Pajdla, Tomas, and Vecerka, Daniel. Rendering Almost Perspective Views from a Sparse Set of Omnidirectional Images, in Proceedings of the 14th British Machine Vision Conference (BMVC 2003), volume 1, pages 241-250, 2003. [pdf]

Bakstein, Hynek, Pajdla, Tomas, and Diaz, Federico A window into reality, in Proceedings of Electronic Imaging and the Visual Arts Conference (EVA 2005), pages 142-147, 2005. [pdf]