The Handbook of Multimodal-Multisensor Interfaces, Volume 1 , livre ebook

Association for Computing Machinery and Morgan & Claypool Publishers - Gerasimos Potamianos , Daniel Sonntag , Philip Cohen , Björn Schuller , Sharon Oviatt

Découvre YouScribe en t'inscrivant gratuitement

Je m'inscris

Obtenez un accès à la bibliothèque pour le consulter en ligne
En savoir plus

260 pages

English

Vous pourrez modifier la taille du texte de cet ouvrage

Obtenez un accès à la bibliothèque pour le consulter en ligne
En savoir plus

A propos
Informations
Extrait

Description

The Handbook of Multimodal-Multisensor Interfaces provides the first authoritative resource on what has become the dominant paradigm for new computer interfaces— user input involving new media (speech, multi-touch, gestures, writing) embedded in multimodal-multisensor interfaces. These interfaces support smart phones, wearables, in-vehicle and robotic applications, and many other areas that are now highly competitive commercially. This edited collection is written by international experts and pioneers in the field. It provides a textbook, reference, and technology roadmap for professionals working in this and related areas. This first volume of the handbook presents relevant theory and neuroscience foundations for guiding the development of high-performance systems. Additional chapters discuss approaches to user modeling and interface designs that support user choice, that synergistically combine modalities with sensors, and that blend multimodal input and output. This volume also highlights an in-depth look at the most common multimodal-multisensor combinations—for example, touch and pen input, haptic and non-speech audio output, and speech-centric systems that co-process either gestures, pen input, gaze, or visible lip movements. A common theme throughout these chapters is supporting mobility and individual differences among users. These handbook chapters provide walk-through examples of system design and processing, information on tools and practical resources for developing and evaluating new systems, and terminology and tutorial support for mastering this emerging field. In the final section of this volume, experts exchange views on a timely and controversial challenge topic, and how they believe multimodal-multisensor interfaces should be designed in the future to most effectively advance human performance.
Table of Contents:Introduction / Theory and Neuroscience Foundations / Theoretical Foundations of Multimodal Interfaces and Systems / The Impact of Multimodal-Multisensory Learning on Human Performance and Brain Activation Patterns / Approaches to Design and User Modeling / Multisensory Haptic Interactions: Understanding the Sense and Designing for It / A Background Perspective on Touch as a Multimodal / Understanding and Supporting Modality Choices / Using Cognitive Models to Understand Multimodal Processes: The Case for Speech and Gesture Production / Multimodal Feedback in HCI: Haptics, Non-Speech Audio, and Their Applications / Multimodal Technologies for Seniors: Challenges and Opportunities / Common Modality Combinations / Gaze Informed Multimodal Interaction / Multimodal Speech and Pen Interfaces / Multimodal Gesture Recognition / Audio and Visual Modality Combination in Speech Processing Applications / Multidisciplinary Challenge Topic: Perspectives on Learning with Multimodal Technology / Contributors’ Brief Biographies: Editors, Authors and Challenge Discussants / Index

Sujets

User Interfaces

Computers

Informatique

Informations

Publié par	Association for Computing Machinery and Morgan & Claypool Publishers
Date de parution	01 juin 2017
Nombre de lectures	0
EAN13	9781970001662
Langue	English
Poids de l'ouvrage	35 Mo

Informations légales : prix de location à la page 0,4000€. Cette information est donnée uniquement à titre indicatif conformément à la législation en vigueur.

Extrait

The Handbook of Multimodal-Multisensor Interfaces, Volume 1
ACM Books
Editor in Chief
M. Tamer zsu, University of Waterloo
ACM Books is a new series of high-quality books for the computer science community, published by ACM in collaboration with Morgan Claypool Publishers. ACM Books publications are widely distributed in both print and digital formats through booksellers and to libraries (and library consortia) and individual ACM members via the ACM Digital Library platform.
The Handbook of Multimodal-Multisensor Interfaces, Volume 1: Foundations, User Modeling, and Common Modality Combinations
Editors: Sharon Oviatt, Incaa Designs
Bj rn Schuller, University of Passau and Imperial College London
Philip R. Cohen, Voicebox Technologies
Daniel Sonntag, German Research Center for Artificial Intelligence (DFKI)
Gerasimos Potamianos, University of Thessaly
Antonio Kr ger, German Research Center for Artificial Intelligence (DFKI)
2017
Communities of Computing: Computer Science and Society in the ACM
Thomas J. Misa, Editor, University of Minnesota
2017
Text Data Management and Analysis: A Practical Introduction to Information Retrieval and Text Mining
ChengXiang Zhai, University of Illinois at Urbana-Champaign
Sean Massung, University of Illinois at Urbana-Champaign
2016
An Architecture for Fast and General Data Processing on Large Clusters
Matei Zaharia, Massachusetts Institute of Technology
2016
Reactive Internet Programming: State Chart XML in Action
Franck Barbier, University of Pau, France
2016
Verified Functional Programming in Agda
Aaron Stump, The University of Iowa
2016
The VR Book: Human-Centered Design for Virtual Reality
Jason Jerald, NextGen Interactions
2016
Ada s Legacy: Cultures of Computing from the Victorian to the Digital Age
Robin Hammerman, Stevens Institute of Technology
Andrew L. Russell, Stevens Institute of Technology
2016
Edmund Berkeley and the Social Responsibility of Computer Professionals
Bernadette Longo, New Jersey Institute of Technology
2015
Candidate Multilinear Maps
Sanjam Garg, University of California, Berkeley
2015
Smarter Than Their Machines: Oral Histories of Pioneers in Interactive Computing
John Cullinane, Northeastern University; Mossavar-Rahmani Center for Business and Government, John F. Kennedy School of Government, Harvard University
2015
A Framework for Scientific Discovery through Video Games
Seth Cooper, University of Washington
2014
Trust Extension as a Mechanism for Secure Code Execution on Commodity Computers
Bryan Jeffrey Parno, Microsoft Research
2014
Embracing Interference in Wireless Systems
Shyamnath Gollakota, University of Washington
2014
The Handbook of Multimodal-Multisensor Interfaces, Volume 1
Foundations, User Modeling, and Common Modality Combinations
Sharon Oviatt
Incaa Designs
Bj rn Schuller
University of Passau and Imperial College London
Philip R. Cohen
Voicebox Technologies
Daniel Sonntag
German Research Center for Artificial Intelligence (DFKI)
Gerasimos Potamianos
University of Thessaly
Antonio Kr ger
German Research Center for Artificial Intelligence (DFKI)
ACM Books #14
Copyright 2017 by the Association for Computing Machinery and Morgan Claypool Publishers
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means-electronic, mechanical, photocopy, recording, or any other except for brief quotations in printed reviews-without the prior permission of the publisher.
Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In all instances in which Morgan Claypool is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration.
The Handbook of Multimodal-Multisensor Interfaces, Volume 1
Sharon Oviatt, Bj rn Schuller, Philip R. Cohen, Daniel Sonntag, Gerasimos Potamianos, Antonio Kr ger, editors
books.acm.org
www.morganclaypoolpublishers.com
ISBN: 978-1-97000-167-9 hardcover
ISBN: 978-1-97000-164-8 paperback
ISBN: 978-1-97000-165-5 ebook
ISBN: 978-1-97000-166-2 ePub
Series ISSN: 2374-6769 print 2374-6777 electronic
DOIs:
10.1145/3015783 Book
10.1145/3015783.3015784 Preface
10.1145/3015783.3015785 Introduction
10.1145/3015783.3015786 Chapter 1
10.1145/3015783.3015787 Chapter 2
10.1145/3015783.3015788 Chapter 3
10.1145/3015783.3015789 Chapter 4
10.1145/3015783.3015790 Chapter 5
10.1145/3015783.3015791 Chapter 6
10.1145/3015783.3015792 Chapter 7
10.1145/3015783.3015793 Chapter 8
10.1145/3015783.3015794 Chapter 9
10.1145/3015783.3015795 Chapter 10
10.1145/3015783.3015796 Chapter 11
10.1145/3015783.3015797 Chapter 12
10.1145/3015783.3015798 Chapter 13
10.1145/3015783.3015799 Index , Bios , Glossary
A publication in the ACM Books series, #14
Editor in Chief: M. Tamer zsu, University of Waterloo
First Edition
10 9 8 7 6 5 4 3 2 1
This book is dedicated to our families, whose patience and support sustained the year-long effort required to organize, write, and manage different stages of this multi-volume project .
Contents
Preface
Figure Credits
Introduction: Scope, Trends, and Paradigm Shift in the Field of Computer Interfaces
Why Multimodal-Multisensor Interfaces Have Become Dominant
Flexible Multiple-Component Tools as a Catalyst for Performance
More Expressively Powerful Tools Are Capable of Stimulating Cognition
One Example of How Multimodal-Multisensor Interfaces Are Changing Today
Insights in the Chapters Ahead
Expert Exchange on Multidisciplinary Challenge Topic
References
PART I THEORY AND NEUROSCIENCE FOUNDATIONS
Chapter 1 Theoretical Foundations of Multimodal Interfaces and Systems
Sharon Oviatt
1.1 Gestalt Theory: Understanding Multimodal Coherence, Stability, and Robustness
1.2 Working Memory Theory: Performance Advantages of Distributing Multimodal Processing
1.3 Activity Theory, Embodied Cognition, and Multisensory-Multimodal Facilitation of Cognition
Focus Questions
References
Chapter 2 The Impact of Multimodal-Multisensory Learning on Human Performance and Brain Activation Patterns
Karin H. James, Sophia Vinci-Booher, Felipe Munoz-Rubke
2.1 Introduction
2.2 The Multimodal-Multisensory Body
2.3 Facilitatory Effects on Learning as a Function of Active Interactions
2.4 Behavioral Results in Children
2.5 Neuroimaging Studies in Adults
2.6 Neuroimaging Studies in Developing Populations
2.7 Theoretical Implications-Embodied Cognition
2.8 Implications for Multimodal-Multisensor Interface Design
Focus Questions
References
PART II APPROACHES TO DESIGN AND USER MODELING
Chapter 3 Multisensory Haptic Interactions: Understanding the Sense and Designing for It
Karon E. MacLean, Oliver S. Schneider, Hasti Seifi
3.1 Introduction
3.2 Interaction Models for Multimodal Applications
3.3 Physical Design Space of Haptic Media
3.4 Making Haptic Media
3.5 Frontiers for Haptic Design
Focus Questions
References
Chapter 4 A Background Perspective on Touch as a Multimodal (and Multisensor) Construct
Ken Hinckley
4.1 Introduction
4.2 The Duality of Sensors and Modalities
4.3 A Model of Foreground and Background Interaction
4.4 Seven Views of Touch Interaction
4.5 Summary and Discussion
Focus Questions
References
Chapter 5 Understanding and Supporting Modality Choices
Anthony Jameson, Per Ola Kristensson
5.1 Introduction
5.2 Synthesis of Research on Modality Choices
5.3 Brief Introduction to the ASPECT and ARCADE Models
5.4 Consequence-Based Choice
5.5 Trial-and-Error-Based Choice
5.6 Policy-Based Choice
5.7 Experience-Based Choice
5.8 Other Choice Patterns
5.9 Recapitulation and Ideas for Future Research
Focus Questions
References
Chapter 6 Using Cognitive Models to Understand Multimodal Processes: The Case for Speech and Gesture Production
Stefan Kopp, Kirsten Bergmann
6.1 Introduction
6.2 Multimodal Communication with Speech and Gesture
6.3 Models of Speech and Gesture Production
6.4 A Computational Cognitive Model of Speech and Gesture Production
6.5 Simulation-based Testing
6.6 Summary
Focus Questions
References
Chapter 7 Multimodal Feedback in HCI: Haptics, Non-Speech Audio, and Their Applications
Euan Freeman, Graham Wilson, Dong-Bach Vo, Alex Ng, Ioannis Politis, Stephen Brewster
7.1 Overview of Non-Visual Feedback Modalities
7.2 Applications of Multimodal Feedback: Accessibility and Mobility
7.3 Conclusions and Future Directions
Focus Questions
References
Chapter 8 Multimodal Technologies for Seniors: Challenges and Opportunities
Cosmin Munteanu, Albert Ali Salah
8.1 Introduction
8.2 Senior Users and Challenges
8.3 Specific Application Areas
8.4 Available Multimodal-Multisensor Technologies
8.5 Multimodal Interaction for Older Adults-Usability, Design, and Adoption Challenges
8.6 Conclusions
Focus Questions
References
PART III COMMON MODALITY COMBINATIONS
Chapter 9 Gaze-Informed Multimodal Interaction
Pernilla Qvarfordt
9.1 Introduction
9.2 Eye Movements and Eye Tracking Data Analysis
9.3 Eye Movements in Relation to Other Modalities
9.4 Gaze in Multimodal Interaction and Systems
9.5 Conclusion and Outlook
Focus Questions
References
Chapter 10 Multimodal Speech and Pen Interfaces
Philip R. Cohen, Sharon Oviatt
10.1 Introduction
10.2 Empirical Research on Multimodal Speech and Pen Interaction
10.3 Design Prototyping and Data Collection
10.4 Flow of Signal and Information Processing
10.5 Distributed Architectural Components
10.6 Multimodal Fusion and Semantic Integration Architectures
10.7 Multimodal Speech and Pen Systems
10.8 Conclusion and Future Directions
Focus Questions
References
Chapter 11 Multimodal Gesture Recognition
Atha