11:00-11:10: Jiri Matas

Opening

11:10-11:45: Rafal Mantiuk (University of Cambridge, UK)

How should we measure progress in computer vision and computational photography? slides - The improvemen - When working in the field of autonomous driving applications and mobility in general, there is a significant challenge to transfer results from the more controlled research environment into real-world scenarios. Deep learning approaches showcase impressive capabilities and advancements for particular benchmarks, but we tempt to hit a wall when rying to deal with the diverse and unpredictable conditions in real life. For one thing Autonomous vehicles encounter a wide array of scenarios, from varied weather conditions to complex traffic situations, changes in scenery etc. Furthermore, for autonomous systems to be safe, it needs to provide some 3D awareness, requiring accurate distance estimation. And last but not least, there is the expectation of the industry to reduce annotation costs, and the models to be applicable for various mobility setups and configurations, even in circumstances where the setup conditions are unknown. In this talk we will discuss some of the pragmatic approaches that we use to tackle some of those problems. We focus on the important task of vehicle and pedestrian detection. We disentangle the calibration aspects, image space segmentation and detection, and 3D monocular uplifting. And we introduce a concept of "camera independence" to make sure the applicability of the model can extend the domains it has been trained for. We illustrate with a variety of examples. briefly introduce a family of visual difference predictors (VDPs), which are image and video metrics that are derived from psychophysical models of human vision and offer additional capabilities lacking in popular metrics. [abstract]

11:45-12:20: Gul Varol (Ecole des Ponts ParisTech, FR)

Audio Description Generation for Movies using Vision-Language Models slides Audio Description (AD) is the narration to describe visual elements in movies or TV series, originally to assist visually impaired audiences. Creating high-quality AD annotations demands significant time and effort, posing a challenging task even for humans. This is due to the necessity for long-term memory, detailed visual understanding, and accurate character recognition. This talk will introduce a series of works, named AutoAD, that leverages video-language models to achieve the automatic generation of AD. [abstract]

12:20-13:30: Lunch break

13:30-14:05: Fatma Guney (Koc University, TR)

Learning Object-Centric Representations for Self-Driving slides - The choice of representation plays a key role in self-driving. Bird’s eye view (BEV) representations have shown remarkable performance in recent years. In this talk, I will introduce learned object-centric representations in BEV to distill a complex scene into more actionable information for self-driving. We first learn to place objects into slots with a slot attention model on BEV sequences. Based on these object-centric representations, we then train a transformer to learn to drive as well as reason about the future of other vehicles. We found that object-centric slot representations outperform both scene-level and object-level approaches that use the exact attributes of objects. Slot representations naturally incorporate information about objects from their spatial and temporal context such as position, heading, and speed without explicitly providing it. Our model with slots achieves increased coverage of the provided routes and, consequently, a higher driving score, with a lower variance across multiple runs, affirming slots as a reliable alternative in object-centric approaches. Additionally, we validate our model’s performance as a world model through forecasting experiments, demonstrating its capability to accurately predict future slot representations. [abstract]

14:05-14:40: Bill Psomas (NTUA, GR)

Leveraging Attention in Masked Image Modeling and Pooling slides - This talk explores innovative methodologies that leverage attention mechanisms to refine the process of masked image modeling (MIM) and enhance spatial pooling strategies in both vision transformers and convolutional networks. The first segment of the talk delves into MIM, pivoting away from the conventional random masking strategies towards an informed masking technique guided by attention—called AttMask. AttMask makes use of the inherent self-attention maps generated by transformer encoders to identify and mask the most salient image tokens. Through this, AttMask facilitates a more challenging pretext task in MIM, prompting the network to hone in on the most discriminative visual cues. The second part discusses the evolution of spatial pooling in vision models. By extracting a more distilled and spatially aware representation at the end of the network, SimPool is introduced, a versatile attention-based pooling mechanism. SimPool transcends the boundaries of supervised and self-supervised learning, providing high-quality attention maps that accurately outline object boundaries. This "universal" pooling illustrates that quality attention maps in vision transformers can be achieved without the need for self-supervised specificity or architectural modifications [abstract]

14:40-15:20: Coffee break

15:20-15:55: Tomas Hodan (Meta, CH)

Object pose estimation for augmented reality slides - I will motivate applications of object pose estimation for extended (virtual, augmented, mixed) reality, introduce devices and data we use at Meta Reality Labs, and describe our recent works. I will primarily focus on FoundPose, our recent method for unseen object pose estimation based on DINOv2. Besides, I will give an update on the BOP bechmark and introduce HOT3D, our new dataset for 3D tracking of hands and objects that will be publicly released in May. [abstract]

15:55-16:30: Jakub Paplham (CTU in Prague, CZ)

A Call to Reflect on Evaluation Practices for Age Estimation: Comparative Analysis of the State-of-the-Art and a Unified Benchmark. - Comparing different age estimation methods poses a challenge due to the unreliability of published results stemming from inconsistencies in the benchmarking process. Previous studies have reported continuous performance improvements over the past decade using specialized methods; however, our findings challenge these claims. This paper identifies two trivial, yet persistent issues with the currently used evaluation protocol and describes how to resolve them. We describe our evaluation protocol in detail and provide specific examples of how the protocol should be used. We utilize the protocol to offer an extensive comparative analysis for state-of-the-art facial age estimation methods. Surprisingly, we find that the performance differences between the methods are negligible compared to the effect of other factors, such as facial alignment, facial coverage, image resolution, model architecture, or the amount of data used for pretraining. We use the gained insights to propose using FaRL as the backbone model and demonstrate its efficiency. The results emphasize the importance of consistent data preprocessing practices for reliable and meaningful comparisons. We make our source code public. [abstract]

10:55-11:00: Jiri Matas

Opening

11:00-11:35: Marc Proesmans (KU Leuven, BE)

In search for camera and data independence for mobility applications - When working in the field of autonomous driving applications and mobility in general, there is a significant challenge to transfer results from the more controlled research environment into real-world scenarios. Deep learning approaches showcase impressive capabilities and advancements for particular benchmarks, but we tempt to hit a wall when rying to deal with the diverse and unpredictable conditions in real life. For one thing Autonomous vehicles encounter a wide array of scenarios, from varied weather conditions to complex traffic situations, changes in scenery etc. Furthermore, for autonomous systems to be safe, it needs to provide some 3D awareness, requiring accurate distance estimation. And last but not least, there is the expectation of the industry to reduce annotation costs, and the models to be applicable for various mobility setups and configurations, even in circumstances where the setup conditions are unknown. In this talk we will discuss some of the pragmatic approaches that we use to tackle some of those problems. We focus on the important task of vehicle and pedestrian detection. We disentangle the calibration aspects, image space segmentation and detection, and 3D monocular uplifting. And we introduce a concept of "camera independence" to make sure the applicability of the model can extend the domains it has been trained for. We illustrate with a variety of examples.[abstract]

11:35-12:10: Oriane Siméoni (VALEO.AI, FR)

Object localization (almost) for free harnessing self-supervised features. slides - The localization of objects in images is at the heart of many perception systems today. However, training object detectors requires large and expensive campaigns of annotation for a finite and pre-defined vocabulary. Instead, being able to discover objects in images without knowing in advance what objects populate the dataset is an exciting prospect. In this talk we will discuss solutions to exploit self-supervised pre-trained features to perform class-agnostic object localization with zero supervision, and such without requiring object proposals or the exploration of image collection. We will also explore means to unite the class-agnostic object localizations with open-vocabulary features in order to perform open-vocabulary semantic segmentation.[abstract]

12:10-13:10: Lunch break

13:10-13:45: Federico Tombari (Google/TU Munich, DE/CH)

3D scene understanding for real world applications - 3D scene understanding is a key enabler for applications in augmented reality, autonomous driving and robotics. The ability for 3D scene understanding methods to work in real world scenarios and to generalize across various domains is of fundamental importance for their applicability. In this talk we will look at two recent trends that aim for this goal. The first one is open vocabulary 3D scene understanding: we will introduce the core ideas and illustrate some of the main works in the literature for open vocabulary semantic segmentation from explicit and implicit 3D representations. The second one is using scene graphs as underlying representation for 3D scenes. Also in this case we will briefly introduce and discuss the main concepts, methodologies and datasets for 3D scene graph inference from 3D and RGB data.[abstract]

13:45-14:20: Lea Müller (Max Planck Institute, DE)

3D Human Mesh Reconstruction for Poses with Contact - Humans are social creatures and interaction plays an important role in our everyday life. The field of computer vision should be able to reconstruct such interaction in 3D from an image. However, this is a challenging task that requires tailored datasets and reconstruction methods. We will introduce BUDDI, a generative model that learns the joint distribution of 3D poses and shapes during interactions. Besides sampling new meshes from BUDDI, we can use this model as a prior during optimisation. To do this, we draw inspiration from the 3D content creation community and propose using a score distillation loss when fitting SMPL meshes to images. We evaluate our method of various benchmark and demonstrate that we can reconstruct more realistic human interactions from images such as hugs, people posing for pictures, or shaking hands.[abstract]

14:20-14:50: Coffee break

14:50-15:25: Tomas Jakab (University of Oxford, UK)

Learning Articulated 3D Animals in the Wild. slides - Reconstructing articulated objects, such as animals, from a single image has numerous practical applications, from robotics to 3D virtual asset creation. Predicting the 3D shape, articulation, viewpoint, and texture from a single image necessitates prior knowledge of the possible shapes and appearances of the object. Typically, learning this requires multi-view images or 3D scans. While such data are available for some types of objects like humans, it is unlikely that this approach will scale to the long tail of objects that appear in the world, including countless different animals. In this talk, we will explore how we can learn this 3D prior using only 2D internet images, which are available in abundance.[abstract]

15:25-16:00: Jonas Kulhanek (CIIRC - CTU in Prague, CZ)

Learning 3D scene representations for novel view synthesis. slides - Novel view synthesis and 3D scene representation are central topics of 3D vision, with many applications spanning areas such as AR/VR, robotics, and gaming. Volumetric representations, such as Neural radiation fields (NeRFs), became a dominant direction in the field, providing photorealistic rendering results at real-time rendering speeds. A popular scene representation used by NeRFs is to combine a uniform, voxel-based subdivision of the scene with an MLP. However, these representations do not use geometry priors (even though point clouds are often available), which hurts the learning time or the memory requirements. An alternative is to use an adaptive representation based on tetrahedra obtained by Delaunay triangulation instead of uniform subdivisions or point-based representations. Such a representation can enable efficient training and match the state-of-the-art results. This approach elegantly combines concepts from 3D geometry processing, triangle-based rendering, and modern neural radiance fields, and compared to voxel-based representations, it can provide more detail around parts of the scene likely to be close to the surface.[abstract]