The Experts below are selected from a list of 7425 Experts worldwide ranked by ideXlab platform
Michael J. Proulx - One of the best experts on this subject based on the ideXlab platform.
-
MultiSensory inclusive design with Sensory Substitution.
Cognitive research: principles and implications, 2020Co-Authors: Tayfun Lloyd-esenkaya, Vanessa Lloyd-esenkaya, Eamonn O'neill, Michael J. ProulxAbstract:Sensory Substitution techniques are perceptual and cognitive phenomena used to represent one Sensory form with an alternative. Current applications of Sensory Substitution techniques are typically focused on the development of assistive technologies whereby visually impaired users can acquire visual information via auditory and tactile cross-modal feedback. But despite their evident success in scientific research and furthering theory development in cognition, Sensory Substitution techniques have not yet gained widespread adoption within Sensory-impaired populations. Here we argue that shifting the focus from assistive to mainstream applications may resolve some of the current issues regarding the use of Sensory Substitution devices to improve outcomes for those with disabilities. This article provides a tutorial guide on how to use research into multiSensory processing and Sensory Substitution techniques from the cognitive sciences to design new inclusive cross-modal displays. A greater focus on developing inclusive mainstream applications could lead to innovative technologies that could be enjoyed by every person.
-
Sensory Substitution and Augmentation - The Processing of What, Where, and How: Insights from Spatial Navigation via Sensory Substitution
Sensory Substitution and Augmentation, 2018Co-Authors: Michael J. Proulx, David J. Brown, Achille PasqualottoAbstract:Vision is the default Sensory modality for normal spatial navigation in humans. Touch is restricted to providing information about peripersonal space, whereas detecting and avoiding obstacles in extrapersonal space is key for efficient navigation. Hearing is restricted to the detection of objects that emit noise, yet many obstacles such as walls are silent. Sensory Substitution devices provide a means of translating distal visual information into a form that visually impaired individuals can process through either touch or hearing. Here we will review findings from various Sensory Substitution systems for the processing of visual information that can be classified as what (object recognition), where (localization), and how (perception for action) processing. Different forms of Sensory Substitution excel at some tasks more than others. Spatial navigation brings together these different forms of information and provides a useful model for comparing Sensory Substitution systems, with important implications for rehabilitation, neuroanatomy, and theories of cognition.
-
Other ways of seeing: From behavior to neural mechanisms in the online “visual” control of action with Sensory Substitution
Restorative neurology and neuroscience, 2015Co-Authors: Michael J. Proulx, James M. Gwinnutt, Sara Dell’erba, Shelly Levy-tzedek, Alexandra A. De Sousa, David J. BrownAbstract:Vision is the dominant sense for perception-for-action in humans and other higher primates. Advances in sight restoration now utilize the other intact senses to provide information that is normally sensed visually through Sensory Substitution to replace missing visual information. Sensory Substitution devices translate visual information from a sensor, such as a camera or ultrasound device, into a format that the auditory or tactile systems can detect and process, so the visually impaired can see through hearing or touch. Online control of action is essential for many daily tasks such as pointing, grasping and navigating, and adapting to a Sensory Substitution device successfully requires extensive learning. Here we review the research on Sensory Substitution for vision restoration in the context of providing the means of online control for action in the blind or blindfolded. It appears that the use of Sensory Substitution devices utilizes the neural visual system; this suggests the hypothesis that Sensory Substitution draws on the same underlying mechanisms as unimpaired visual control of action. Here we review the current state of the art for Sensory Substitution approaches to object recognition, localization, and navigation, and the potential these approaches have for revealing a metamodal behavioral and neural basis for the online control of action.
-
MultiSensory integration, Sensory Substitution and visual rehabilitation
Neuroscience and biobehavioral reviews, 2014Co-Authors: Michael J. Proulx, Maurice Ptito, Amir AmediAbstract:Sensory Substitution has advanced remarkably over the past 35 years since first introduced to the scientific literature by Paul Bach-y-Rita. In this issue dedicated to his memory, we describe a collection of reviews that assess the current state of neuroscience research on Sensory Substitution, visual rehabilitation, and multiSensory processes.
-
how well do you see what you hear the acuity of visual to auditory Sensory Substitution
Frontiers in Psychology, 2013Co-Authors: Alastair Haigh, Peter Meijer, David J. Brown, Michael J. ProulxAbstract:Sensory Substitution devices (SSDs) aim to compensate for the loss of a Sensory modality, typically vision, by converting information from the lost modality into stimuli in a remaining modality. “The vOICe” is a visual-to-auditory SSD which encodes images taken by a camera worn by the user into “soundscapes” such that an experienced user can extract information about their surroundings. Here we investigated how much detail was resolvable during the early induction stages by testing the acuity of blindfolded sighted, naive vOICe users. Initial performance was well above chance. Participants who took the test twice as a form of minimal training showed a marked improvement on the second test. Acuity was slightly but not significantly impaired when participants wore a camera and judged letter orientations “live”. A positive correlation was found between participants’ musical training and their acuity. The relationship between auditory expertise via musical training and the lack of a relationship with visual imagery, suggests that early use of a Sensory Substitution device draws primarily on the mechanisms of the Sensory modality being used rather than the one being substituted. If vision is lost, audition represents the Sensory channel of highest bandwidth of those remaining. The level of acuity found here, and the fact it was achieved with very little experience in Sensory Substitution by naive users is promising.
Malika Auvray - One of the best experts on this subject based on the ideXlab platform.
-
Mixing up the Senses: Sensory Substitution Is Not a Form of Artificially Induced Synaesthesia
Multisensory Research, 2020Co-Authors: Louise P. Kirsch, Xavier E. Job, Malika AuvrayAbstract:Abstract Sensory Substitution Devices (SSDs) are typically used to restore functionality of a Sensory modality that has been lost, like vision for the blind, by recruiting another Sensory modality such as touch or audition. Sensory Substitution has given rise to many debates in psychology, neuroscience and philosophy regarding the nature of experience when using SSDs. Questions first arose as to whether the experience of Sensory Substitution is represented by the substituted information, the substituting information, or a multiSensory combination of the two. More recently, parallels have been drawn between Sensory Substitution and synaesthesia, a rare condition in which individuals involuntarily experience a percept in one Sensory or cognitive pathway when another one is stimulated. Here, we explore the efficacy of understanding Sensory Substitution as a form of ‘artificial synaesthesia’. We identify several problems with previous suggestions for a link between these two phenomena. Furthermore, we find that Sensory Substitution does not fulfil the essential criteria that characterise synaesthesia. We conclude that Sensory Substitution and synaesthesia are independent of each other and thus, the ‘artificial synaesthesia’ view of Sensory Substitution should be rejected.
-
MultiSensory and spatial processes in Sensory Substitution.
Restorative neurology and neuroscience, 2019Co-Authors: Malika AuvrayAbstract:Sensory Substitution devices aim at compensating Sensory deficits by converting stimuli coming from a deficient Sensory modality (e.g., vision) into stimuli accessible through another modality (e.g., touch or audition). Studies conducted with these devices revealed the central nervous system to be very plastic. Various laboratories have conducted studies investigating such plasticity by means of behavioural and brain-imaging techniques. At the ISIR Laboratory, we focused on the factors underlying the learning of Sensory Substitution devices, their adequacy to the target population, and we explored ways of improving their design by the use of crossmodal correspondences and by taking into account individual differences in the used reference frames. We also investigated the nature of the experience with Sensory Substitution. In particular, we suggested moving beyond positions reducing experience to that of a single Sensory modality. Rather, Sensory Substitution is considered as a multiSensory experience, involving not only visual, but also auditory or tactile processes as well as cognitive processes. In this framework, individual differences do have an influence on the extent to which the different Sensory modalities influence the experience with the devices.
-
Sensory Substitution and Augmentation - Limits of the Classical Functionalist Perspective on Sensory Substitution
Sensory Substitution and Augmentation, 2018Co-Authors: Maurice Ptito, Malika Auvray, Katrine Iversen, Ophelia Deroy, Ron KupersAbstract:The tongue display unit (TDU) is a Sensory Substitution device that translates visual images into electrotactile stimulation that is transmitted to the tongue and leads to new perceptual skills following training. Trained users, including blind individuals, become capable of orientation discrimination, motion detection, shape recognition and they can also successfully use the TDU to navigate in an environment, locate objects and avoid obstacles. Many studies and discussions have focused on the effects of training at the behavioural level, and assumed that the effects shown in training blindfolded sighted individuals are similar to those observed in blind people. In doing so, we argue that behavioural research on Sensory Substitution shows a functionalist bias. Functionalism claims that mental processes can be individuated by their characteristic inputs and outputs, and that the physical realization of a given function introduces no relevant difference, as long as the function is the same. We emphasize here why this assumption biases the interpretation of Sensory Substitution devices.
-
Individual Differences in Sensory Substitution.
Multisensory research, 2017Co-Authors: Gabriel Arnold, Jacques Pesnot-lerousseau, Malika AuvrayAbstract:Sensory Substitution devices were developed in the context of perceptual rehabilitation and they aim at compensating one or several functions of a deficient Sensory modality by converting stimuli that are normally accessed through this deficient Sensory modality into stimuli accessible by another Sensory modality. For instance, they can convert visual information into sounds or tactile stimuli. In this article, we review those studies that investigated the individual differences at the behavioural, neural, and phenomenological levels when using a Sensory Substitution device. We highlight how taking into account individual differences has consequences for the optimization and learning of Sensory Substitution devices. We also discuss the extent to which these studies allow a better understanding of the experience with Sensory Substitution devices, and in particular how the resulting experience is not akin to a single Sensory modality. Rather, it should be conceived as a multiSensory experience, involving both perceptual and cognitive processes, and emerging on each user's pre-existing Sensory and cognitive capacities.
-
The state of the art of Sensory Substitution.
Multisensory research, 2014Co-Authors: Malika Auvray, Laurence R. HarrisAbstract:As an introduction to the special issue of MultiSensory Research on Sensory Substitution (SS) we would like to give an overview of the limits and promises of this growing field of research. It seems that a large part of the scientific community considers that Sensory Substitution does not work. Yet many research groups claim that it does, and are making advances and obtaining interesting results, many of which are described in this special issue, that support their case. Why is there this discrepancy? It is likely that the term ‘Sensory Substitution’ induces a narrowing of the field of research, linked to the original idea of fully restoring a lost Sensory modality by artificially providing the information through another sense (e.g., Bach-y-Rita et al., 1969; see also Bach-y-Rita, 1972). However, such an ambitious claim has long been abandoned, and many researchers agree that SS is never going to be the way to achieve full recovery of the use of a lost sense. Many researchers do however agree that a restricted form of Sensory Substitution can be achieved. Thus, it is undeniably the case that many abilities of people with Sensory loss can be significantly enhanced by the use of SS devices. To some extent, therefore, Sensory Substitution does work. Reflecting the fact that SS researchers have had to retract from their original ambitious aim, several alternative names for this cross-modal approach have been proposed such as ‘perceptual supplementation’ (Lenay et al., 2003) and ‘sensorimotor extension’ (Auvray and Myin, 2008). However, none has so far received universal acceptance. This may reflect the fact that, although SS
Achille Pasqualotto - One of the best experts on this subject based on the ideXlab platform.
-
Sensory Substitution and Augmentation - The Processing of What, Where, and How: Insights from Spatial Navigation via Sensory Substitution
Sensory Substitution and Augmentation, 2018Co-Authors: Michael J. Proulx, David J. Brown, Achille PasqualottoAbstract:Vision is the default Sensory modality for normal spatial navigation in humans. Touch is restricted to providing information about peripersonal space, whereas detecting and avoiding obstacles in extrapersonal space is key for efficient navigation. Hearing is restricted to the detection of objects that emit noise, yet many obstacles such as walls are silent. Sensory Substitution devices provide a means of translating distal visual information into a form that visually impaired individuals can process through either touch or hearing. Here we will review findings from various Sensory Substitution systems for the processing of visual information that can be classified as what (object recognition), where (localization), and how (perception for action) processing. Different forms of Sensory Substitution excel at some tasks more than others. Spatial navigation brings together these different forms of information and provides a useful model for comparing Sensory Substitution systems, with important implications for rehabilitation, neuroanatomy, and theories of cognition.
-
Sensory Substitution: The Spatial Updating of Auditory Scenes "Mimics" the Spatial Updating of Visual Scenes.
Frontiers in behavioral neuroscience, 2016Co-Authors: Achille Pasqualotto, Tayfun EsenkayaAbstract:Visual-to-auditory Sensory Substitution is used to convey visual information through audition, and it was initially created to compensate for blindness; it consists of software converting the visual images captured by a video-camera into the equivalent auditory images, or “soundscapes”. Here, it was used by blindfolded sighted participants to learn the spatial position of simple shapes depicted in images arranged on the floor. Very few studies have used Sensory Substitution to investigate spatial representation, while it has been widely used to investigate object recognition. Additionally, with Sensory Substitution we could study the performance of participants actively exploring the environment through audition, rather than passively localizing sound sources. Blindfolded participants egocentrically learnt the position of six images by using Sensory Substitution and then a judgment of relative direction task (JRD) was used to determine how this scene was represented. This task consists of imagining being in a given location, oriented in a given direction, and pointing towards the required image. Before performing the JRD task, participants explored a map that provided allocentric information about the scene. Although spatial exploration was egocentric, surprisingly we found that performance in the JRD task was better for allocentric perspectives. This suggests that the egocentric representation of the scene was updated. This result is in line with previous studies using visual and somatoSensory scenes, thus supporting the notion that different Sensory modalities produce equivalent spatial representation(s). Moreover, our results have practical implications to improve training methods with Sensory Substitution devices (SSD).
-
MultiSensory perceptual learning and Sensory Substitution.
Neuroscience and biobehavioral reviews, 2012Co-Authors: Michael J. Proulx, David J. Brown, Achille Pasqualotto, Peter MeijerAbstract:One of the most exciting recent findings in neuroscience has been the capacity for neural plasticity in adult humans and animals. Studies of perceptual learning have provided key insights into the mechanisms of neural plasticity and the changes in functional neuroanatomy that it affords. Key questions in this field of research concern how practice of a task leads to specific or general improvement. Although much of this work has been carried out with a focus on a single Sensory modality, primarily visual, there is increasing interest in multiSensory perceptual learning. Here we will examine how advances in perceptual learning research both inform and can be informed by the development and advancement of Sensory Substitution devices for blind persons. To allow ‘sight’ to occur in the absence of visual input through the eyes, visual information can be transformed by a Sensory Substitution device into a representation that can be processed as sound or touch, and thus give one the potential to ‘see’ through the ears or tongue. Investigations of auditory, visual and multiSensory perceptual learning can have key benefits for the advancement of Sensory Substitution, and the study of Sensory deprivation and Sensory Substitution likewise will further the understanding of perceptual learning in general and the reverse hierarchy theory in particular. It also has significant importance for the developing understanding of the brain in metamodal terms, where functional brain areas might be best defined by the computations they carry out rather than by their Sensory-specific processing role.
David J. Brown - One of the best experts on this subject based on the ideXlab platform.
-
Sensory Substitution and Augmentation - The Processing of What, Where, and How: Insights from Spatial Navigation via Sensory Substitution
Sensory Substitution and Augmentation, 2018Co-Authors: Michael J. Proulx, David J. Brown, Achille PasqualottoAbstract:Vision is the default Sensory modality for normal spatial navigation in humans. Touch is restricted to providing information about peripersonal space, whereas detecting and avoiding obstacles in extrapersonal space is key for efficient navigation. Hearing is restricted to the detection of objects that emit noise, yet many obstacles such as walls are silent. Sensory Substitution devices provide a means of translating distal visual information into a form that visually impaired individuals can process through either touch or hearing. Here we will review findings from various Sensory Substitution systems for the processing of visual information that can be classified as what (object recognition), where (localization), and how (perception for action) processing. Different forms of Sensory Substitution excel at some tasks more than others. Spatial navigation brings together these different forms of information and provides a useful model for comparing Sensory Substitution systems, with important implications for rehabilitation, neuroanatomy, and theories of cognition.
-
Other ways of seeing: From behavior to neural mechanisms in the online “visual” control of action with Sensory Substitution
Restorative neurology and neuroscience, 2015Co-Authors: Michael J. Proulx, James M. Gwinnutt, Sara Dell’erba, Shelly Levy-tzedek, Alexandra A. De Sousa, David J. BrownAbstract:Vision is the dominant sense for perception-for-action in humans and other higher primates. Advances in sight restoration now utilize the other intact senses to provide information that is normally sensed visually through Sensory Substitution to replace missing visual information. Sensory Substitution devices translate visual information from a sensor, such as a camera or ultrasound device, into a format that the auditory or tactile systems can detect and process, so the visually impaired can see through hearing or touch. Online control of action is essential for many daily tasks such as pointing, grasping and navigating, and adapting to a Sensory Substitution device successfully requires extensive learning. Here we review the research on Sensory Substitution for vision restoration in the context of providing the means of online control for action in the blind or blindfolded. It appears that the use of Sensory Substitution devices utilizes the neural visual system; this suggests the hypothesis that Sensory Substitution draws on the same underlying mechanisms as unimpaired visual control of action. Here we review the current state of the art for Sensory Substitution approaches to object recognition, localization, and navigation, and the potential these approaches have for revealing a metamodal behavioral and neural basis for the online control of action.
-
how well do you see what you hear the acuity of visual to auditory Sensory Substitution
Frontiers in Psychology, 2013Co-Authors: Alastair Haigh, Peter Meijer, David J. Brown, Michael J. ProulxAbstract:Sensory Substitution devices (SSDs) aim to compensate for the loss of a Sensory modality, typically vision, by converting information from the lost modality into stimuli in a remaining modality. “The vOICe” is a visual-to-auditory SSD which encodes images taken by a camera worn by the user into “soundscapes” such that an experienced user can extract information about their surroundings. Here we investigated how much detail was resolvable during the early induction stages by testing the acuity of blindfolded sighted, naive vOICe users. Initial performance was well above chance. Participants who took the test twice as a form of minimal training showed a marked improvement on the second test. Acuity was slightly but not significantly impaired when participants wore a camera and judged letter orientations “live”. A positive correlation was found between participants’ musical training and their acuity. The relationship between auditory expertise via musical training and the lack of a relationship with visual imagery, suggests that early use of a Sensory Substitution device draws primarily on the mechanisms of the Sensory modality being used rather than the one being substituted. If vision is lost, audition represents the Sensory channel of highest bandwidth of those remaining. The level of acuity found here, and the fact it was achieved with very little experience in Sensory Substitution by naive users is promising.
-
MultiSensory perceptual learning and Sensory Substitution.
Neuroscience and biobehavioral reviews, 2012Co-Authors: Michael J. Proulx, David J. Brown, Achille Pasqualotto, Peter MeijerAbstract:One of the most exciting recent findings in neuroscience has been the capacity for neural plasticity in adult humans and animals. Studies of perceptual learning have provided key insights into the mechanisms of neural plasticity and the changes in functional neuroanatomy that it affords. Key questions in this field of research concern how practice of a task leads to specific or general improvement. Although much of this work has been carried out with a focus on a single Sensory modality, primarily visual, there is increasing interest in multiSensory perceptual learning. Here we will examine how advances in perceptual learning research both inform and can be informed by the development and advancement of Sensory Substitution devices for blind persons. To allow ‘sight’ to occur in the absence of visual input through the eyes, visual information can be transformed by a Sensory Substitution device into a representation that can be processed as sound or touch, and thus give one the potential to ‘see’ through the ears or tongue. Investigations of auditory, visual and multiSensory perceptual learning can have key benefits for the advancement of Sensory Substitution, and the study of Sensory deprivation and Sensory Substitution likewise will further the understanding of perceptual learning in general and the reverse hierarchy theory in particular. It also has significant importance for the developing understanding of the brain in metamodal terms, where functional brain areas might be best defined by the computations they carry out rather than by their Sensory-specific processing role.
Tayfun Esenkaya - One of the best experts on this subject based on the ideXlab platform.
-
Sensory Substitution: The Spatial Updating of Auditory Scenes "Mimics" the Spatial Updating of Visual Scenes.
Frontiers in behavioral neuroscience, 2016Co-Authors: Achille Pasqualotto, Tayfun EsenkayaAbstract:Visual-to-auditory Sensory Substitution is used to convey visual information through audition, and it was initially created to compensate for blindness; it consists of software converting the visual images captured by a video-camera into the equivalent auditory images, or “soundscapes”. Here, it was used by blindfolded sighted participants to learn the spatial position of simple shapes depicted in images arranged on the floor. Very few studies have used Sensory Substitution to investigate spatial representation, while it has been widely used to investigate object recognition. Additionally, with Sensory Substitution we could study the performance of participants actively exploring the environment through audition, rather than passively localizing sound sources. Blindfolded participants egocentrically learnt the position of six images by using Sensory Substitution and then a judgment of relative direction task (JRD) was used to determine how this scene was represented. This task consists of imagining being in a given location, oriented in a given direction, and pointing towards the required image. Before performing the JRD task, participants explored a map that provided allocentric information about the scene. Although spatial exploration was egocentric, surprisingly we found that performance in the JRD task was better for allocentric perspectives. This suggests that the egocentric representation of the scene was updated. This result is in line with previous studies using visual and somatoSensory scenes, thus supporting the notion that different Sensory modalities produce equivalent spatial representation(s). Moreover, our results have practical implications to improve training methods with Sensory Substitution devices (SSD).
