Video Synopsis of Surveillance Cameras

Shmuel Peleg
The Hebrew University of Jerusalem, Israel

Millions of surveillance cameras record video around the clock, producing huge video archives. Even when a video archive is known to include critical activities, finding them is like finding a needle in a haystack, making the archive almost worthless. Two main approaches were proposed to address this problem: automatic action recognition and video summarization. Since automatic detection of activities still faces problems in many scenarios, human inspection of video is necessary. This can be enabled by video synopsis, which is an effective tool for browsing and indexing of massive amounts of recorded video. It provides a short video representation, while preserving the essential activities of the original video.

In video synopsis, the activity in the video is condensed into a shorter period by simultaneously showing multiple activities, even when they originally occurred at different times. The synopsis is also an index of the original video by pointing to the original time of each activity. Video synopsis can be applied to create a synopsis of video streams generated by surveillance cameras. It can address queries like “Show in one minute the synopsis of the video captured during the past day.”

This process includes two major phases: i) a conversion of the video stream into a database of objects and activities (rather than frames) and ii) The generation of the video synopsis as a response to the user’s query. Video synopsis can become more concise and coherent by clustering together similar activities. Objects with similar activities are easy to watch simultaneously, and outliers can be spotted instantly. Having clusters of objects enables to view only the objects of interest, ignoring irrelevant activities.