Mixed Interactive Systems
Mixed Interactive Systems are Interactive systems that combine the use of physical artefacts and digital entities. A number of terms is used to denote this kind of interactive systems: Augmented Reality, Mixed Reality, Embedded Reality, Mixed Environment, Augmented Video, etc.
We detail below:
Definition of Mixed Interactive Systems
A Mixed Interactive System is an interactive system, involving:
- a user
- a set of digital entities, representing the different software ressources, methods, digital data, etc. manipulated by the application
- and a set of physical entities, such as bricks, tools, documents, objects, etc., manipulated by the user to communicate with the system or used by the system as a support to provide the user with computerised data.
The originality of these systems is the presence of physical artefacts on top of the other entities, also present in any traditionnal interactive system.
These interactive systems are also called Augmented Reality, Augmented Surfaces,
Tangible User Interface, Mixed Reality, Augmented User, Augmented Video,
Enriched Environment, pervasive or ubiquitous systems, etc.
As opposed to some other approaches, we do not promote the idea that "one entity (either physical or digital) is augmented by some others". We rather consider that the user's interaction with the interactive system is a enrich by the use of a combination of digital and physical entities: this is the reason why we use the terms Mixed Interaction, Mixed Interactive Systems, and Mixed Interactive Techniques.
For a more precise and illustrated introduction to Mixed Systems, please refer to one of my papers, published in Interact'99.
Taxonomy of Mixed Interactive Systems
To structure the wide domain of Mixed Interactive Systems and support the comparison and classification of such systems, we propose a first distinction between two kinds of mixed interactive systems :
- Augmented Reality systems (AR): in these systems, the user's interaction with the physical world is enriched with computerised data and services in the physical user's environment.
- Augmented Virtuality systems (AV): in these systems, the user's interaction with the digital world is enriched with the use of tools, objects and actions of the physical world.
Furhtermore, we consider that a relevant distinction to consider when comparing two Mixed Interactive Systems, is the kind of augmentation provided by the MIS. Inspired from the work of D. Norman that identified the gulfs of execution and perception as the set of problems a user will face when executing a command and perceiving the feedback from the system, three different kind of augmentation have been identified:
- augmented execution: the system helps the user in performing his/her task
- augmented perception: the system provides the user with additional information
- both augmentation simulataneously:
The resulting classification space thus consists in 6 different groups as illustrated in the figure below.
For more explanations about this classification of Mixed Interactive Systems, please refer to one of my papers, published in Interact'99.
Illustration of Mixed Interactive Systems
Augmented Virtuality systems are sometinmes based on the use of physical bricks, localised by video or magnetic devices (application for the design of building), and lead sometimes to the development of new inteaction devices, such as the Cubic Mouse.
Augmented Reality Systems may display data in a See-Through Head Mounted Display (museum application) or project data on top of the physical world (Digital Desk). Data may also be superimposed on a video of the real world (surgical application example) or displayed on a deviced encrusted in the physical places or objects (fridge) (cf. illustrations : PDF).
Gestures and tangible palette also constitute natural way to interact with 3D spaces. Visualising data in a see-through Head-Mounted Display is also adaptable to mobile situations (hall) and data projection may be used on top of physical artefacts (desk, videotape, magazine, etc.) (cf. illustrations : PDF et PDF).
Initially dedicated to specific application domains with specific and clearly identified needs and constraints such as surgery and maintenance, new application domains emerged such as teaching environement, architecture, games, etc. (cf. illustration: PDF).
Nowadays, many "consummer applications" are interrested in including this king of interaction: public inforation terminal, museum and even artistic creation process (cf. illustrations : PDF).