Computer vision on rolling shutter cameras

Per-Erik Forssen (Linkoping University, Sweden)

The majority of cameras sold today have CMOS sensors with electronic rolling shutters. In the absence of a mechanical shutter this means that instead of a global frame exposure, frames are exposed in a line-by-line fashion, where the lag between the first and last row readout (the readout-time) is measured in tens of milliseconds. A rolling shutter causes geometric distortions, whenever either the camera, or objects in the scene are moving, and this can have severe implications for any computer vision algorithm that is geometry based.

In this talk I will describe our efforts to design novel algorithms that take the rolling shutter into account. I will cover versions of video rectification, video stabilization, and bundle adjustment of video. Finally I will show some recent results on how to improve 3D mapping using the Microsoft Kinect sensor, by taking its rolling shutter into account.