In recent years, because cameras have become inexpensive and ever more prevalent, there has been increasing interest in modeling human shape and motion from image data. Such an ability has many applications, such as electronic publishing, entertainment, sports medicine and athletic training. This, however, is an inherently difficult task, both because the body is very complex and because the data that can be extracted from images is often incomplete, noisy and ambiguous.

In this talk, I will present the approach we have developed to overcome these difficulties. We start from sophisticated 3-D animation models and reformulate them so that they can be used for data analysis. We use them, not only to represent faces and bodies in motion, but also to guide the interpretation of the image data, thereby substantially improving performance. Using complex video sequences, I will highlight the effectiveness of our approach to video-based shape and motion capture and demonstrate the applicability of our technology for Augmented Reality purposes. Finally, I will present some open research issues and discuss our plans for future developments.