Modality (human–computer interaction)

Uses your system voice; voice and rate in settings.

Snapshot

≈ 5 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In the context of human–computer interaction, a modality is the classification of a single independent channel of input/output between a computer and a human. Such channels may differ based on sensory nature (e.g., visual vs. auditory), or other significant differences in processing (e.g., text vs. image). A system is designated unimodal if it has only one modality implemented, and multimodal if it has more than one. When multiple modalities are available for some tasks or aspects of a task, the system is said to have overlapping modalities. If multiple modalities are available for a task, the system is said to have redundant modalities. Multiple modalities can be used in combination to provide complementary methods that may be redundant but convey information more effectively. Modalities can be generally defined in two forms: computer-human and human-computer modalities.

In the context of human–computer interaction, a modality is the classification of a single independent channel of input/output between a computer and a human. Such channels may differ based on sensory nature (e.g., visual vs. auditory), or other significant differences in processing (e.g., text vs. image). A system is designated unimodal if it has only one modality implemented, and multimodal if it has more than one. When multiple modalities are available for some tasks or aspects of a task, the system is said to have overlapping modalities. If multiple modalities are available for a task, the system is said to have redundant modalities. Multiple modalities can be used in combination to provide complementary methods that may be redundant but convey information more effectively. Modalities can be generally defined in two forms: computer-human and human-computer modalities.

Computer–human modalities

Computers utilize a wide range of technologies to communicate and send information to humans: Common modalities Vision – computer graphics typically through a screen Audition – various audio outputs Tactition – vibrations or other movement Uncommon modalities Gustation (taste) Olfaction (smell) Thermoception (heat) Nociception (pain) Equilibrioception (balance) Any human sense can be used as a computer to human modality. However, the modalities of seeing and hearing are the most commonly employed since they are capable of transmitting information at a higher speed than other modalities, 250 to 300 and 150 to 160 words per minute, respectively. Though not commonly implemented as computer-human modality, tactition can achieve an average of 125 wpm through the use of a refreshable Braille display. Other more common forms of tactition are smartphone and game controller vibrations.

Human–computer modalities

Computers can be equipped with various types of input devices and sensors to allow them to receive information from humans. Common input devices are often interchangeable if they have a standardized method of communication with the computer and afford practical adjustments to the user. Certain modalities can provide a richer interaction depending on the context, and having options for implementation allows for more robust systems. Simple modalities Keyboard Pointing device Touchscreen Complex modalities Computer vision Speech recognition Motion Orientation With the increasing popularity of smartphones, the general public are becoming more comfortable with the more complex modalities. Motion and orientation are commonly used in smartphone mapping applications. Speech recognition is widely used with Virtual Assistant applications. Computer Vision is now common in camera applications that are used to scan documents and QR codes.

Using multiple modalities

Having multiple modalities in a system gives more affordance to users and can contribute to a more robust system. Having more also allows for greater accessibility for users who work more effectively with certain modalities. Multiple modalities can be used as backup when certain forms of communication are not possible. This is especially true in the case of redundant modalities in which two or more modalities are used to communicate the same information. Certain combinations of modalities can add to the expression of a computer-human or human-computer interaction because the modalities each may be more effective at expressing one form or aspect of information than others. There are six types of cooperation between modalities, and they help define how a combination or fusion of modalities work together to convey information more effectively. Equivalence: information is presented in multiple ways and can be interpreted as the same information Specialization: when a specific kind of information is always processed through the same modality Redundancy: multiple modalities process the same information Complementarity: multiple modalities take separate information and merge it Transfer: a modality produces information that another modality consumes Concurrency: multiple modalities take in separate information that is not merged Complementary-redundant systems are those which have multiple sensors to form one understanding or dataset, and the more effectively the information can be combined without duplicating data, the more effectively the modalities cooperate. Having multiple modalities for communication is common, particularly in smartphones, and often their implementations work together towards the same goal, for example gyroscopes and accelerometers working together to track movement.

See also

References

Text derived from https://en.wikipedia.org/w/index.php?oldid=1353974009 under CC BY-SA 4.0; modifications have been made.

Provenance

snapshot enwiki-001353974009 · captured 1778674815.0 · version history

Cite this page as: encyclopegia.org, “Modality (human–computer interaction)”, snapshot enwiki-001353974009. https://encyclopegia.org/a/modality-human-computer-interaction

Sources cited in this revision

Extracted from the cited revision's reference blocks; absent citations are not invented.