Regular grid representation

The original 4D lightfield [99] data structure employs a two-plane parameterization. Each light ray passes through two parallel planes with plane coordinates $(s,t)$ and $(u,v)$ (see Figure 8.8).

Figure 8.8: 4D representation of viewing rays in the regular grid representation of the lightfield.

Thus the ray is uniquely described by the 4-tuple $(s,t,u,v)$. The $(s,t)$-plane is the viewpoint plane in which all camera focal points are placed on regular grid points. The cameras are constructed such, that the $(u,v)$-plane is their common image plane and that their optical axes are perpendicular to it. From the two-plane parameterization new views can be rendered by placing a virtual camera on an arbitrary viewing position with arbitrary parameters (e.g. focal length) and intersecting each viewing ray with the two planes at $(s,t,u,v)$. The resulting radiance is a look-up into the regular grid. For rays passing in between the $(s,t)$ and $(u,v)$ grid coordinates an interpolation is applied that will degrade the rendering quality depending on the scene geometry. In fact, the lightfield contains an implicit geometrical assumption: The scene geometry is planar and coincides with the focal plane (Figure 8.9). Deviation of the scene geometry from the focal plane causes image warping. Figure 8.9 shows that the radiance of the viewing ray $r$ is interpolated from radiance values $l_{k-1}$ and $l_k$ of neighboring camera viewpoints, depending on the geometrical deviation from the focal plane.

Figure 8.9: Viewpoint interpolation between $s_k$ and $s_{k-1}$.

Linear interpolation between the viewpoints in $(s,t)$ and $(u,v)$ introduces a blurred image with ghosting artifacts. In reality we will always have to choose between high density of stored viewing rays with high data volume and high fidelity, or low density with poor image quality.

If we have a sequence of images taken with a hand-held camera, in general the camera positions are not placed at the grid points of the viewpoint plane. In [54] a method is shown for resampling this regular two plane parameterization from real images recorded from arbitrary positions (rebinning). The required regular structure is resampled and gaps are filled by applying a multi-resolution approach, considering depth corrections. The disadvantage of this rebinning step is that the interpolated regular structure already contains inconsistencies and ghosting artifacts because of errors in the scantily approximated geometry. To render views a depth corrected look-up is performed. During this step the effect of ghosting artifacts is repeated so duplicate ghosting effects occur.