Interaction with Real and Virtual Worlds

The scientific problems posed by interaction between the real and virtual worlds involve perception (analysis, recognition, categorization, and semantic interpretation capacities), action (looped command for perception, decision, planning and learning) and comprehension mechanisms (with the new man-machine interfaces based on our five senses) enable this interaction to take place. Progress relating to these issues requires advances in numerous fields: algorithms, statistical and probabilistic techniques, signal processing, speech and imaging, geometric algorithms and image processing, vision, modeling of the environment, planning, digital technology, natural language processing, as well as decision-making and planning algorithms.

In the field of robotics, the major challenge is to combine the diversity of tasks to be carried out by the machine with the variability of environments. Models combine with those in cognitive sciences, and those with controllable self-organized systems. Integration of perception and movement, planning, action, learning and exploration are all key points. We need to develop gesture, attitude and human behavior recognition. The user must be able to benefit from increased reality and sensory return. One concept of direct interaction is the enhanced human, which could be equipped with such devices as exoskeletons, ortheses and prostheses, as well as a range of proprioceptive and exteroceptive sensors.

«  These researches also open new options for interactions between individuals and social mediation, for example – within collaborative activities. »

Augmented or virtual worlds combine computed pictures and artificial vision, and assume calculations and data searches. New methods of visualization using geometric and functional modeling, based on algorithmic progress, are needed to fully use this data. They consist of improving movement capturing techniques for visual, auditory and haptic animation and multimodal rendering to improve the realism of scenes. Three-dimensional interaction with virtual universes must all experience progress, whether it be multimodal, mobile, collaborative or cognitive.

These researches also open new options for interactions between individuals and social mediation, for example – within collaborative activities. The expansion of new tools, services and uses has enabled new concepts to emerge, such as collective intelligence, which is based on data from hundreds of millions of internet users. INRIA will work on these issues, most specifically in partnership with researchers in human and social sciences.