CMP Stereo Algorithm Evaluation	Center for Machine Perception Czech Technical University Prague http://cmp.felk.cvut.cz/
	Jana Kostliva   Radim Sara   Jan Cech

Introduction

This website presents material for stereo matching (dense or semi-dense) algorithm evaluation described in [1,2]. It contains matlab toolbox for evaluating your algorithm, comparison of already tested algorithms, downloads with scenes or results and upload form for uploading your results.

Our methodology evaluates algorithms over a wide range of parameter settings which allows to get the algorithm performance as good as possible. The results are presented as error-plots as well as numerical characteristics describing the algorithm performance.

Our evaluation uses also scenes from Middlebury dataset (Cones, Teddy, Tsukuba, and Venus), courtesy of D. Scharstein, which has been introduced in [3,4].

We invite other researches to contribute their results to this page to create wider a study. To this end, we provide a matlab toolbox which helps you to run the evaluation on your algorithm. If you are interested in contributing, you can download it on page "Downloads".

News:

• 23-Jan-2009: new toolbox version ROC_Toolbox_v1.2 allowing time monitoring is released. The information about running times is added to the evaluation. It has only informative character since it is not possible to correctly rank the algorithms based on the running time until they all are tested on a single computer.
• 23-Jan-2009: a function for MATLAB interface sroc_Matlab_Interf_ex.m for algorithms running from a command line can be downloaded to help researchers to transfer input and output of their methods from and to MATLAB.

The website has four main pages:

• Introduction - this page with description and instructions how to use it.
• Results - a page with results of tested methods. Main page shows one table with overall evaluation. Detailed evaluation (plots and additional tables) on one scene only is shown after clicking on the scene name. Clicking on points in plots shows disparity map which generated that point.
• Downloads - a page, where you can download toolbox, current Stereo Feasibility Boundary, images with ground truth (if you do not want to download the toolbox, since it is included there), or other method results.
• Upload - a page for uploading your results generated by the toolbox.

How to cite:

We give permission to use and publish all images and results on this website. Nevertheless, if you use our toolbox and/or results created based on this method, please cite the appropriate articles [1,2].

References: