The Experts below are selected from a list of 9414 Experts worldwide ranked by ideXlab platform
Marian Kepesi - One of the best experts on this subject based on the ideXlab platform.
-
Multimodal mobile robot control using speech application language tags
Ambient Intelligence, 2003Co-Authors: Michael Pucher, Marian KepesiAbstract:This paper describes the design and architecture of a Multimodal Interface for controlling a mobile robot. The architecture is build up from standardized components and uses Speech Application Language Tags. We show how these components can be used to build complex Multimodal Interfaces. Basic design patterns for such Interfaces are presented and discussed.
-
EUSAI - Multimodal Mobile Robot Control Using Speech Application Language Tags
Lecture Notes in Computer Science, 2003Co-Authors: Michael Pucher, Marian KepesiAbstract:This paper describes the design and architecture of a Multimodal Interface for controlling a mobile robot. The architecture is build up from standardized components and uses Speech Application Language Tags. We show how these components can be used to build complex Multimodal Interfaces. Basic design patterns for such Interfaces are presented and discussed.
Mark Billinghurst - One of the best experts on this subject based on the ideXlab platform.
-
Designing an Augmented Reality Multimodal Interface for 6DOF Manipulation Techniques
Intelligent Systems and Applications, 2019Co-Authors: Ajune Wanis Ismail, Mohd Shahrizal Sunar, Mark Billinghurst, Cik Suhaimi YusofAbstract:Augmented Reality (AR) supports natural interaction in physical and virtual worlds, so it has recently given rise to a number of novel interaction modalities. This paper presents a method for using hand-gestures with speech input for Multimodal interaction in AR. It focuses on providing an intuitive AR environment which supports natural interaction with virtual objects while sustaining accessible real tasks and interaction mechanisms. The paper reviews previous Multimodal Interfaces and describes recent studies in AR that employ gesture and speech inputs for Multimodal input. It describes an implementation of gesture interaction with speech input in AR for virtual object manipulation. Finally, the paper presents a user evaluation of the technique, showing that it can be used to improve the interaction between virtual and physical elements in an AR environment.
-
mozart a Multimodal Interface for conceptual 3d modeling
International Conference on Multimodal Interfaces, 2011Co-Authors: Anirudh Sharma, Sriganesh Madhvanath, Ankit Shekhawat, Mark BillinghurstAbstract:There is a need for computer aided design tools that support rapid conceptual level design. In this paper we explore and evaluate how intuitive speech and multitouch input can be combined in a Multimodal Interface for conceptual 3D modeling. Our system, MozArt, is based on a user's innate abilities - speaking and touching, and has a toolbar/button-less Interface for creating and interacting with computer graphics models. We briefly cover the hardware and software technology behind MozArt, and present a pilot study comparing our Multimodal system with a conventional multitouch modeling Interface with first time CAD users. While a larger study is required to obtain statistically significant comparison regarding efficiency and accuracy of the two Interfaces, a majority of the participants preferred the Multimodal Interface over the multitouch. We summarize lessons learned and discuss directions for future research.
-
ICMI - MozArt: a Multimodal Interface for conceptual 3D modeling
Proceedings of the 13th international conference on multimodal interfaces - ICMI '11, 2011Co-Authors: Anirudh Sharma, Sriganesh Madhvanath, Ankit Shekhawat, Mark BillinghurstAbstract:There is a need for computer aided design tools that support rapid conceptual level design. In this paper we explore and evaluate how intuitive speech and multitouch input can be combined in a Multimodal Interface for conceptual 3D modeling. Our system, MozArt, is based on a user's innate abilities - speaking and touching, and has a toolbar/button-less Interface for creating and interacting with computer graphics models. We briefly cover the hardware and software technology behind MozArt, and present a pilot study comparing our Multimodal system with a conventional multitouch modeling Interface with first time CAD users. While a larger study is required to obtain statistically significant comparison regarding efficiency and accuracy of the two Interfaces, a majority of the participants preferred the Multimodal Interface over the multitouch. We summarize lessons learned and discuss directions for future research.
-
a wizard of oz study for an ar Multimodal Interface
International Conference on Multimodal Interfaces, 2008Co-Authors: Minkyung Lee, Mark BillinghurstAbstract:In this paper we describe a Wizard of Oz (WOz) user study of an Augmented Reality (AR) Interface that uses Multimodal input (MMI) with natural hand interaction and speech commands. Our goal is to use a WOz study to help guide the creation of a Multimodal AR Interface which is most natural to the user. In this study we used three virtual object arranging tasks with two different display types (a head mounted display, and a desktop monitor) to see how users used Multimodal commands, and how different AR display conditions affect those commands. The results provided valuable insights into how people naturally interact in a Multimodal AR scene assembly task. For example, we discovered the optimal time frame for fusing speech and gesture commands into a single command. We also found that display type did not produce a significant difference in the type of commands used. Using these results, we present design recommendations for Multimodal interaction in AR environments.
-
ISMAR - "Move the couch where?" : developing an augmented reality Multimodal Interface
2006 IEEE ACM International Symposium on Mixed and Augmented Reality, 2006Co-Authors: Sylvia Irawati, Scott A. Green, Andreas Duenser, Mark Billinghurst, Heedong KoAbstract:This paper describes an augmented reality (AR) Multimodal Interface that uses speech and paddle gestures for interaction. The application allows users to intuitively arrange virtual furniture in a virtual room using a combination of speech and gestures from a real paddle. Unlike other Multimodal AR applications, the Multimodal fusion is based on the combination of time-based and semantic techniques to disambiguate a users speech and gesture input. We describe our AR Multimodal Interface architecture and discuss how the Multimodal inputs are semantically integrated into a single interpretation by considering the input time stamps, the object properties, and the user context.
Michael Pucher - One of the best experts on this subject based on the ideXlab platform.
-
Multimodal mobile robot control using speech application language tags
Ambient Intelligence, 2003Co-Authors: Michael Pucher, Marian KepesiAbstract:This paper describes the design and architecture of a Multimodal Interface for controlling a mobile robot. The architecture is build up from standardized components and uses Speech Application Language Tags. We show how these components can be used to build complex Multimodal Interfaces. Basic design patterns for such Interfaces are presented and discussed.
-
EUSAI - Multimodal Mobile Robot Control Using Speech Application Language Tags
Lecture Notes in Computer Science, 2003Co-Authors: Michael Pucher, Marian KepesiAbstract:This paper describes the design and architecture of a Multimodal Interface for controlling a mobile robot. The architecture is build up from standardized components and uses Speech Application Language Tags. We show how these components can be used to build complex Multimodal Interfaces. Basic design patterns for such Interfaces are presented and discussed.
Heedong Ko - One of the best experts on this subject based on the ideXlab platform.
-
ISMAR - "Move the couch where?" : developing an augmented reality Multimodal Interface
2006 IEEE ACM International Symposium on Mixed and Augmented Reality, 2006Co-Authors: Sylvia Irawati, Scott A. Green, Andreas Duenser, Mark Billinghurst, Heedong KoAbstract:This paper describes an augmented reality (AR) Multimodal Interface that uses speech and paddle gestures for interaction. The application allows users to intuitively arrange virtual furniture in a virtual room using a combination of speech and gestures from a real paddle. Unlike other Multimodal AR applications, the Multimodal fusion is based on the combination of time-based and semantic techniques to disambiguate a users speech and gesture input. We describe our AR Multimodal Interface architecture and discuss how the Multimodal inputs are semantically integrated into a single interpretation by considering the input time stamps, the object properties, and the user context.
-
An evaluation of an augmented reality Multimodal Interface using speech and paddle gestures
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2006Co-Authors: Sylvia Irawati, Scott Green, Andreas Duenser, Mark Billinghurst, Heedong KoAbstract:This paper discusses an evaluation of an augmented reality (AR) Multimodal Interface that uses combined speech and paddle gestures for interaction with virtual objects in the real world. We briefly describe our AR Multimodal Interface architecture and Multimodal fusion strategies that are based on the combination of time-based and domain semantics. Then, we present the results from a user study comparing using Multimodal input to using gesture input alone. The results show that a combination of speech and paddle gestures improves the efficiency of user interaction. Finally, we describe some design recommendations for developing other Multimodal AR Interfaces.
-
"Move the couch where?" : developing an augmented reality Multimodal Interface
2006 IEEE ACM International Symposium on Mixed and Augmented Reality, 2006Co-Authors: Sylvia Irawati, Scott Green, Andreas Duenser, Mark Billinghurst, Heedong KoAbstract:This paper describes an augmented reality (AR) Multimodal Interface that uses speech and paddle gestures for interaction. The application allows users to intuitively arrange virtual furniture in a virtual room using a combination of speech and gestures from a real paddle. Unlike other Multimodal AR applications, the Multimodal fusion is based on the combination of time-based and semantic techniques to disambiguate a users speech and gesture input. We describe our AR Multimodal Interface architecture and discuss how the Multimodal inputs are semantically integrated into a single interpretation by considering the input time stamps, the object properties, and the user context.
Maja Pantic - One of the best experts on this subject based on the ideXlab platform.
-
Gaze-X: Adaptive, affective, Multimodal Interface for single-user office scenarios
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2007Co-Authors: Leendert Maat, Maja PanticAbstract:This paper describes an intelligent system that we developed to support affective Multimodal human-computer interaction (AMM-HCI) where the user’s actions and emotions are modeled and then used to adapt the interaction and support the user in his or her activity. The proposed system, which we named Gaze-X, is based on sensing and interpretation of the human part of the computer’s context, known as W5+ (who, where, what, when, why, how). It integrates a number of natural human communicative modalities including speech, eye gaze direction, face and facial expression, and a number of standard HCI modalities like keystrokes, mouse movements, and active software identification, which, in turn, are fed into processes that provide decision making and adapt the HCI to support the user in his or her activity according to his or her preferences. A usability study conducted in an office scenario with a number of users indicates that Gaze-X is perceived as effective, easy to use, useful, and affectively qualitative.
-
gaze x adaptive affective Multimodal Interface for single user office scenarios
International Conference on Multimodal Interfaces, 2006Co-Authors: Ludo Maat, Maja PanticAbstract:This paper describes an intelligent system that we developed to support affective Multimodal human-computer interaction (AMM-HCI) where the user's actions and emotions are modeled and then used to adapt the HCI and support the user in his or her activity. The proposed system, which we named Gaze-X, is based on sensing and interpretation of the human part of the computer's context, known as W5+ (who, where, what, when, why, how). It integrates a number of natural human communicative modalities including speech, eye gaze direction, face and facial expression, and a number of standard HCI modalities like keystrokes, mouse movements, and active software identification, which, in turn, are fed into processes that provide decision making and adapt the HCI to support the user in his or her activity according to his or her preferences. To attain a system that can be educated, that can improve its knowledge and decision making through experience, we use case-based reasoning as the inference engine of Gaze-X. The utilized case base is a dynamic, incrementally self-organizing event-content-addressable memory that allows fact retrieval and evaluation of encountered events based upon the user preferences and the generalizations formed from prior input. To support concepts of concurrency, modularity/scalability, persistency, and mobility, Gaze-X has been built as an agent-based system where different agents are responsible for different parts of the processing. A usability study conducted in an office scenario with a number of users indicates that Gaze-X is perceived as effective, easy to use, useful, and affectively qualitative.
