One of the grand goals of computer vision is to interpret a scene
semantically given an image. It involves various individual tasks, such as
object recognition, image segmentation, object detection, and 3D scene
recovery. Substantial progress has been made in each of these tasks in the
past few years. In light of these successes, the challenging problem now is
to put these individual elements together to achieve the grand goal --
"scene understanding", a problem which has received increasing attention
recently, with the introduction of applications such as Google Street View,
Microsoft Bing maps. We address the problems of "what", "where", and "how
many" in scenes: we recognize objects, find their location and spatial
extent, segment them, and also provide the number of instances of objects.
We formulate this problem in an energy minimization framework, defined on
pixels, segments, and objects, and give each of them a label. In the context
of such labelling problems, the talk will present: (i) How to model the
problem; (ii) How to learn the parameters of the energy function; and (iii)
How to solve the problem efficiently for gigapixel images. We will also look
at some recent extensions to this work.