Proposal full title: OMNIVIEWS "Omni-directional Visual System"
Proposal acronym: OMNIVIEWS
Date of preparation: 14 May 2000
Proposal number: -
In this assessment phase the consortium involves three academic institutions from three different states. All the participants have a long standing experience and reputation in fundamental and applied research as well as close links to strong industrial partners particularly with SME spin-off companies.
LIRA-Lab is at the Department of Informatics, Systems Science and Telecommunication of the University of Genova in Italy and has a vast experience with constructing and implementing smart CMOS sensors with a retina-like (foveal) layout [1-4]. The first CMOS sensor with a retina-like layout was implemented within the EU-funded IBIDEM project [5-7] and had as a result the first CMOS retina-like sensor in the world, with about 8,000 pixels. Within the ESPRIT project SVAVISCA  LIRA-Lab lead the team to realize first a colour version of the 8,000 pixels sensor and, successively, a new version with 33,000 pixels has been realized and its utility in remote monitoring and surveillance has been demonstrated[a1]. In the side figure the layout of the sensors developed in the past are presented. The first industrial product based on this sensor, a digital camera realised by an Italian SME, was awarded the 1999 European IST prize.
Besides the expertise on retina-like sensor's technology, LIRA-Lab has a long standing experience on Computer Vision and Robotics. On these aspects LIRA-Lab has a long standing collaboration with the group at the ISR in Lisbon with whom they collaborate within a TMR network (SMART) and an LTR Esprit Project (NARVAL) in the field of underwater robotics [9-11].
The Center for Machine Perception of the Czech Technical University, Prague has the expertise in the construction of panoramic cameras, their mathematical modelling , and in exploiting them in research as well as in various industrial applications. Many different versions of such sensors have been realised and their use in application areas such as robotics, surveillance and quality checking and control has been demonstrated. Also in this respect, CMP has strong links with a spin-off SME fabricating and commercialising panoramic mirrors.
In the next figure examples of different panoramic mirrors realisations are shown; hyperbolic of different “sharpness” and spherical of different sizes.
The VisLab - Computer Vision Laboratory, at the Instituto de Sistemas e Robotica of the Instituto Superior Tecnico in Lisbon has a long-term experience in low-level vision for mobile robotics and active vision including visual navigation of autonomous vehicles [13-17]. The VisLab has been involved in research projects particularly in the areas of vision based navigation, often inspired in principles found in biology. Various techniques for tracking, egomotion estimation and navigation have been developed for un-conventional imaging geometries, including the log-polar geometry and panoramic mirrors. Recently, such ideas have been applied to a small robot with an insect-like compound eye, which used analogue filters running in parallel, similar to the Elementary Motion Detectors found in some insect eyes. In all this cases research has been directed towards both the theoretical aspects of geometry, image processing and control, as well as looking for application domains, primarily in the areas of mobile robot navigation in land, air and underwater environments.
All the partners have an extend experience with EC projects. LIRA laboratory has been working (and in some cases coordinating) in a number of EU-supported projects (ESPRIT projects VOILA, VAP, NARVAL, ROBVISION). It collaborates with other European research institutions through a Human Capital and Mobility Network. The CMP has participated in RECCAD project, which was funded through the EC Copernicus scheme, in VIMP project financed by EU EUREKA scheme, and it is active in the Erasmus/Leonardo network at present. The IST/ISR is currently involved in several national EU-funded projects including ESPRIT, MAST, INCO-COPERNICUS and BRITE/EURAM. acting as coordinator in some cases. There exists a long-term successful collaboration on EU-funded projects between LIRA-Lab and IST/ISR, e.g. currently running project NARVAL. Recently, a link was established between IST and a CMP spin-off company Neovision Ltd. regarding the panoramic vision technology for mobile robot navigation.
The main role of LIRA laboratory:
The main role of CMP:
The main role of IST/ISR:
There is no overlapping among the partners’ responsibilities. The objectives that have been set for this project are fulfilled with that distribution of the roles. Each partner in an expert at the area that has undertaken.
LIRA-Lab, DIST - University Genova
Professor Giulio Sandini
DIST - University of Genova
Viale Causa 13
16145 Genova Italy
Phone: +39 10 353 2779
Fax: +39 10 353 2948
LIRA-Lab operates within the Dipartimento di Informatica, Sistemistica e Telematica (DIST) of the University of Genova which is composed of approximately 47 persons including 32 persons with permanent teaching and research position, and 15 persons providing administrative and technical support. The participation to the project will be through the LIRA-Lab (Laboratory for Integrated Advanced Robotics). LIRA-Lab main research themes are in the field of Robotics in un-conventional applications and un-structured environments, but also more classical robotic & automation topics are carried out in cooperation with various automation industries in Italy and outside. As to the present project the activities will be based on past activities in design and use of space-variant image sensors. This expertise results from past participation to EC-supported projects (ESPRIT project SVAVISCA, TIDE project IBIDEM) and also from a very successful collaboration with IST within a Human Capital and Mobility Network.
Giulio Sandini is an Associate Professor at DIST since 1986. He coordinates the activity of researchers at LIRA Laboratory. He is Associate Editor of Image and Vision Computing Journal and Co-Editor of Videre: A Journal of Computer Vision Research (MIT Press). He is also reviewer of projects supported by the EU and of many international journals in the field of Computer Vision and Robotics. Giulio Sandini has been a member of programme committees of international conferences and chairman and co-chairman of international conferences and workshops. He is/was principal investigator of ESPRIT Projects: P419, P2502 (VOILA) and SVAVISCA, BRA project P3274 (FIRST) and VAP-II, TIDE project IBIDEM, TMR projects VIRGO and SMART and founding Member of ECVNet.
CMP-Czech Technical University, Prague
Assistant Professor Tomas Pajdla
Center for Machine Perception
Department of Cybernetics
Faculty of Electrical Engineering
Czech Technical University, Prague
12135 Karlovo namesti 13
Praha, Czech Republic
Phone: +420 2 2435 7348
Fax: +420 2 2435 7385
The CMP is a research unit existing at the Czech Technical University and dealing mainly with research in computer vision and mathematics of uncertainty. CMP has altogether about 10 professors and researchers and about 12 PhD students. CMP is financed mainly by governmental money, research grants (Czech Grant Agency, EU Copernicus, EU COST, EU Eureka), and contracts with Czech and foreign industries (Rockwell Automation, Texas Instruments, Boeing, Neovision s.r.o.)
CMP has expertise in calibrating optical systems for 3D measurement, modelling and correcting optical distortions, and projective geometry for optical measurement. CMP has a technology, which allows acquiring 3D faces from images of persons via stereo vision and is active in face recognition as well.
CMP was one of the places where panoramic cameras based on mirrors were proposed and constructed for the first time. The research on panoramic cameras started in 1996 and since then the construction of the cameras as well as their mathematical models were developed. In 1998, the importance of central panoramic cameras was clearly recognized and their epipolar geometry was proposed. It made clear how to use classical mathematical theories developed for ordinary cameras for central panoramic cameras. Some results related to the panoramic camera construction are also used by the CMP spin-off company Neovision s.r.o. which can provide manufacturing of the optics needed within the project.
CV of Tomas Pajdla
Born in Prague, Czech Republic on May 10, 1969; in 1992, Ing. (MSc.) in Electrical Engineering from the CTU, Prague; in 1994-1995 ESAT, Katholieke Universiteit, Leuven, Belgium working at VISICS group; 1995 Assistant Professor at the CTU, Prague; lecturer of computer vision and robotics. Research projects: RECCAD-Reconstruction and Analysis of Complex Objects for Building CAD Models Based on Measured Data, EU Copernicus No.1068/94, and Representation of 3-D Scene by 2-D Images, GACR No. 102/97/0855, the responsible investigator of the project Construction of Complete 3D Models from Range Images, GACR No. 102/97/0480. Research in geometrical aspects of computer vision, camera systems calibration, scene reconstruction from images, 3D data acquisition and processing, and panoramic vision. He published more than 20 scientific works at prestigious conferences. In 1998 he was awarded OAGM Prize 1998 for the best paper.
IST/ISR – Istituto Superior Técnico / Instituto de Sistemas e Robótica
Professor Jose Santos-Victor
Instituto Superior Tecnico
ISR - Torre Norte
Av. Rovisco Pais 1
1049-001 Lisboa, Portugal
Phone: +351 21 8418 294
Fax: +351 21 8418 291
The Instituto Superior Técnico (IST) is the largest and oldest engineering school in Portugal, with a large record of participation in international researcher projects. At the research level, the work described in this project will be carried out at the Computer Vision Lab (VisLab) of the Instituto de Sistemas e Robótica (ISR).
ISR is a national research organization established in 1991 involving a total of 127 scientists, 61 of which hold a PhD degree. ISR is an institution that intervenes in the areas of Computer Vision, Robotics, Automation, Control, Signal Processing, Aeronautics, Physical Acoustics and Energy Management and Production, with an emphasis on Systems Theory. It comprises 3 laboratories: Lisbon, Porto and Coimbra. The Lisbon laboratory is integrated in the Instituto Superior Técnico (IST).
The VisLab has been involved in various research projects in areas related to Computer Vision and Robotics. It has solid expertise in several problems, particularly in the areas of vision based control, active vision and vision-based navigation.
Jose Santos-Victor serves as an Assistant Professor at the Instituto Superior Tecnico (IST), Lisbon and as researcher at the Instituto de Sistemas e Robotica (ISR). He has founded the Computer Vision Lab (VisLab) at IST/ISR. His main research topics are Computer Vision and Robotics with emphasis to Active Vision and the relationship between visual perception and action. He was the principal investigator in a number of national and international R&D projects in the areas of Computer Vision and Robotics. He is a member of the editorial board of the Journal of Robotics and Autonomous Systems (Elsevier) and member of the program committee of various international conferences on computer vision and robotics.
A Note on the consortium of phase-2 of the project
If the first assessment phase of the project will be successful, it is the intention of the proposers to extend the consortium to include two or three SMEs. In particular, according to the results of the assessment phase we intend to include the following SMEs:
1) Neovision S.r.o. (manufacturer of panoramic mirrors)
2) Aitek S.r.l. (manufacturer of retina-like sensors and cameras)
3) T.B.D. Manufacture of processing Chips
This extension will assure the follow up project to have the necessary industrial background to study and realise an innovative device with sound commercial potential.
 G. Sandini and V. Tagliasco, “An Anthropomorphic Retina-like Structure for Scene Analysis,” CVGIP, vol. 14, pp. 365-372, 1980.
 G. Sandini, P. Dario, M. DeMicheli, and M. Tistarelli, “Retina-like CCD Sensor for Active Vision,” presented at NATO ARW on Robots and Biological Systems, Il Ciocco, Tuscany, Italy, 1989.
 M. Tistarelli and G. Sandini, “On the Estimation of Depth from Motion Using a CCD Retinal Sensor,” presented at Proc. of first European Conference on Computer Vision, Antibes (France), 1990.
 J. V. d. Spiegel, G. Kreider, C. Claeys, I. Debusschere, G. Sandini, P. Dario, F. Fantini, P. Bellutti, and G. Soncini, “A Foveated Retina-like Sensor Usinf CCD Technology,”: De Kluwer, 1989.
 G. Sandini and et.al, “Image-based Personal Communication Using an Innovative Space-variant CMOS Sensor,” presented at Proc. Roman '96, Tsukuba, Japan, 1996.
 W. W. Woelders, H. W. Frowein, J. Nielsen, P. Questa, and G. Sandini, “New Developments in Low-bit Rate Videotelephony for People Who are Deaf,” Journal of Speech, Language and Hearing Reserach, 1997.
 F. Ferrari, J. Nielsen, P. Questa, and G. Sandini, “Space variant imaging,” Sensor Review, vol. 15, pp. 17-20, 1995.
 G. Sandini, A. Alaerts, B. Dierickx, F. Ferrari, L. Hermans, A. Mannucci, B. Parmentier, P. Questa, G. Meynants, and D. Sheffer, “The Project SVAVISCA: a Space-Variant Color CMOS Sensor,” presented at Proc. AFPAEC'98, Zurich, 1998.
 J. Santos-Victor and G. Sandini, “Visual Behaviors for Docking,” Computer Vision and Image Processing,, vol. 67, pp. 223-238, 1997.
 J. Santos-Victor and G. Sandini, “Embedded Visual Behaviors for Navigation,” Robotics and Autonomous Systems, vol. 19, pp. 299-313, 1997.
 J. Santos-Victor, G. Sandini, F. Curotto, and S. Garibaldi, “Divergent Stereo for Robot Navigation: From Bees to Robots,” International Journal of Computer Vision, vol. 14, pp. 159-177, 1995.
 T. Svoboda, T. Pajdla, and V. Hlavac, “Epipolar Geometry for Panoramic Cameras,” in 5th European Conference on Computer Vision, Lecture Notes in Computer Science, vol. 1406: Springer Verlag, 1998, pp. 218-232.
 C. Silva and J. Santos-Victor, “Egomotion Estimation Using Log-Polar Images,” presented at Internationnal Conference on Computer Vision - ICCV98, Bombay, India, 1998.
 C. Silva and J. Santos-Victor, “Robust Egomotion Estimation from the Normal Flow, Using Search Subspaces,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, 1997.
 J. Gaspar and J. Santos-Victor, “Visual Path Following with a Catadioptric Panoramic Camera,” presented at 7th International Symposium on Intelligent Robotic Systems, Coimbra, Portugal, 1999.
 N. Winters and J. Santos-Victor, “Mobile Robot Navigation using Omni-directional Vision,” presented at Irish Machine Vision and Image Processing Conference, IMVIP, Dublin, Ireland, 1999.
 S. v. d. Zwaan and J. Santos-Victor, “An Insect Inspired Visual Sensor for the Autonomous Navigation of a Mobile Robot,” presented at 7th International Symposium on Intelligent Robotic Systems, Coimbra, Portugal, 1999.