The Experts below are selected from a list of 18537 Experts worldwide ranked by ideXlab platform
Rufin Vanrullen - One of the best experts on this subject based on the ideXlab platform.
-
Predictive Coding feedback results in perceived illusory contours in a recurrent neural network
Neural Networks, 2021Co-Authors: Zhaoyang Pang, Callum Biggs Omay, Bhavin Choksi, Rufin VanrullenAbstract:Abstract Modern feedforward convolutional neural networks (CNNs) can now solve some computer vision tasks at super-human levels. However, these networks only roughly mimic human visual perception. One difference from human vision is that they do not appear to perceive illusory contours (e.g. Kanizsa squares) in the same way humans do. Physiological evidence from visual cortex suggests that the perception of illusory contours could involve feedback connections. Would recurrent feedback neural networks perceive illusory contours like humans? In this work we equip a deep feedforward convolutional network with brain-inspired recurrent dynamics. The network was first pretrained with an unsupervised reconstruction objective on a natural image dataset, to expose it to natural object contour statistics. Then, a classification decision head was added and the model was finetuned on a form discrimination task: squares vs. randomly oriented inducer shapes (no illusory contour). Finally, the model was tested with the unfamiliar “illusory contour” configuration: inducer shapes oriented to form an illusory square. Compared with feedforward baselines, the iterative “Predictive Coding” feedback resulted in more illusory contours being classified as physical squares. The perception of the illusory contour was measurable in the luminance profile of the image reconstructions produced by the model, demonstrating that the model really “sees” the illusion. Ablation studies revealed that natural image pretraining and feedback error correction are both critical to the perception of the illusion. Finally we validated our conclusions in a deeper network (VGG): adding the same Predictive Coding feedback dynamics again leads to the perception of illusory contours.
-
the rhythms of Predictive Coding pre stimulus phase modulates the influence of shape perception on luminance judgments
Scientific Reports, 2017Co-Authors: Biao Han, Rufin VanrullenAbstract:Predictive Coding is an influential model emphasizing interactions between feedforward and feedback signals. Here, we investigated the temporal dynamics of these interactions. Two gray disks with different versions of the same stimulus, one enabling Predictive feedback (a 3D-shape) and one impeding it (random-lines), were simultaneously presented on the left and right of fixation. Human subjects judged the luminance of the two disks while EEG was recorded. The choice of 3D-shape or random-lines as the brighter disk was used to assess the influence of feedback signals on sensory processing in each trial (i.e., as a measure of post-stimulus Predictive Coding efficiency). Independently of the spatial response (left/right), we found that this choice fluctuated along with the pre-stimulus phase of two spontaneous oscillations: a ~5 Hz oscillation in contralateral frontal electrodes and a ~16 Hz oscillation in contralateral occipital electrodes. This pattern of results demonstrates that Predictive Coding is a rhythmic process, and suggests that it could take advantage of faster oscillations in low-level areas and slower oscillations in high-level areas.
-
the rhythms of Predictive Coding pre stimulus phase modulates the influence of shape perception on luminance judgments
bioRxiv, 2016Co-Authors: Biao Han, Rufin VanrullenAbstract:Predictive Coding is an influential model emphasizing interactions between feedforward and feedback signals. Here, we investigated its temporal dynamics. Two gray disks with different versions of the same stimulus, one enabling Predictive feedback (a 3D-shape) and one impeding it (random-lines), were simultaneously presented on the left and right of fixation. Human subjects judged the luminance of the two disks while EEG was recorded. Independently of the spatial response (left/right), we found that the choice of 3D-shape or random-lines as the brighter disk (our measure of post-stimulus Predictive Coding efficiency on each trial) fluctuated along with the pre-stimulus phase of two spontaneous oscillations: a ~5Hz oscillation in contralateral frontal electrodes and a ~16Hz oscillation in contralateral occipital electrodes. This pattern of results demonstrates that Predictive Coding is a rhythmic process, and suggests that it could take advantage of faster oscillations in low-level areas and slower oscillations in high-level areas.
Jun Tani - One of the best experts on this subject based on the ideXlab platform.
-
a novel Predictive Coding inspired variational rnn model for online prediction and recognition
Neural Computation, 2019Co-Authors: Ahmadreza Ahmadi, Jun TaniAbstract:This study introduces PV-RNN, a novel variational RNN inspired by Predictive-Coding ideas. The model learns to extract the probabilistic structures hidden in fluctuating temporal patterns by dynami...
-
Goal-Directed Behavior under Variational Predictive Coding: Dynamic organization of Visual Attention and Working Memory
2019 IEEE RSJ International Conference on Intelligent Robots and Systems (IROS), 2019Co-Authors: Minju Jung, Takazumi Matsumoto, Jun TaniAbstract:Mental simulation is a critical cognitive function for goal-directed behavior because it is essential for assessing actions and their consequences. When a self-generated or externally specified goal is given, a sequence of actions that is most likely to attain that goal is selected among other candidates via mental simulation. Therefore, better mental simulation leads to better goal-directed action planning. However, developing a mental simulation model is challenging because it requires knowledge of self and the environment. The current paper studies how adequate goal-directed action plans of robots can be mentally generated by dynamically organizing top-down visual attention and visual working memory. For this purpose, we propose a neural network model based on variational Bayes Predictive Coding, where goal-directed action planning is formulated by Bayesian inference of latent intentional space. Our experimental results showed that cognitively meaningful competencies, such as autonomous top-down attention to the robot end effector (its hand) as well as dynamic organization of occlusion-free visual working memory, emerged. Furthermore, our analysis of comparative experiments indicated that the introduction of visual working memory and the inference mechanism using variational Bayes Predictive Coding significantly improved the performance in planning adequate goal-directed actions.
-
a novel Predictive Coding inspired variational rnn model for online prediction and recognition
arXiv: Learning, 2018Co-Authors: Ahmadreza Ahmadi, Jun TaniAbstract:This study introduces PV-RNN, a novel variational RNN inspired by the Predictive-Coding ideas. The model learns to extract the probabilistic structures hidden in fluctuating temporal patterns by dynamically changing the stochasticity of its latent states. Its architecture attempts to address two major concerns of variational Bayes RNNs: how can latent variables learn meaningful representations and how can the inference model transfer future observations to the latent variables. PV-RNN does both by introducing adaptive vectors mirroring the training data, whose values can then be adapted differently during evaluation. Moreover, prediction errors during backpropagation, rather than external inputs during the forward computation, are used to convey information to the network about the external data. For testing, we introduce error regression for predicting unseen sequences as inspired by Predictive Coding that leverages those mechanisms. The model introduces a weighting parameter, the meta-prior, to balance the optimization pressure placed on two terms of a lower bound on the marginal likelihood of the sequential data. We test the model on two datasets with probabilistic structures and show that with high values of the meta-prior the network develops deterministic chaos through which the data's randomness is imitated. For low values, the model behaves as a random process. The network performs best on intermediate values, and is able to capture the latent probabilistic structure with good generalization. Analyzing the meta-prior's impact on the network allows to precisely study the theoretical value and practical benefits of incorporating stochastic dynamics in our model. We demonstrate better prediction performance on a robot imitation task with our model using error regression compared to a standard variational Bayes model lacking such a procedure.
Evelina Fedorenko - One of the best experts on this subject based on the ideXlab platform.
-
fmri reveals language specific Predictive Coding during naturalistic sentence comprehension
Neuropsychologia, 2020Co-Authors: Cory Shain, Idan Blank, Marten Van Schijndel, William Schuler, Evelina FedorenkoAbstract:Much research in cognitive neuroscience supports prediction as a canonical computation of cognition across domains. Is such Predictive Coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that Predictive Coding in the brain's response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed continuous-time deconvolutional regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found effects of prediction measures in the language network but not in the domain-general multiple-demand network, which supports executive control processes and has been previously implicated in language comprehension. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.
-
fmri reveals language specific Predictive Coding during naturalistic sentence comprehension
bioRxiv, 2019Co-Authors: Cory Shain, Evelina Fedorenko, Idan Blank, Marten Van Schijndel, William SchulerAbstract:Abstract Much research in cognitive neuroscience supports prediction as a canonical computation of cognition in many domains. Is such Predictive Coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that Predictive Coding in the brain’s response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed deconvolutional time series regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found we found effects of prediction measures in the language network but not in the domain-general, multiple-demand network. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.
Karl J Friston - One of the best experts on this subject based on the ideXlab platform.
-
repetition suppression and its contextual determinants in Predictive Coding
Cortex, 2016Co-Authors: Ryszard Auksztulewicz, Karl J FristonAbstract:This paper presents a review of theoretical and empirical work on repetition suppression in the context of Predictive Coding. Predictive Coding is a neurobiologically plausible scheme explaining how biological systems might perform perceptual inference and learning. From this perspective, repetition suppression is a manifestation of minimising prediction error through adaptive changes in predictions about the content and precision of sensory inputs. Simulations of artificial neural hierarchies provide a principled way of understanding how repetition suppression - at different time scales - can be explained in terms of inference and learning implemented under Predictive Coding. This formulation of repetition suppression is supported by results of numerous empirical studies of repetition suppression and its contextual determinants.
-
active inference Predictive Coding and cortical architecture
In: Recent Advances On The Modular Organization Of The Cortex. (pp. 97-121). (2015), 2015Co-Authors: Rick A Adams, Karl J Friston, Andre M BastosAbstract:This chapter discusses how many features of cortical anatomy and physiology can be understood in the light of a Predictive Coding theory of brain function. In Sect. 7.1, we briefly discuss the theoretical reasons to suppose that the brain is likely to use Predictive Coding. One key theoretical underpinning of Predictive Coding is the free energy principle, which argues that brains must maximize the evidence for their (generative) model of sensory inputs: a process of ‘active inference’. In Sect. 7.2, we discuss how active inference predicts commonalities in the extrinsic connections of sensory and motor systems. Such commonalities are found in their hierarchical structure (shown by laminar characteristics), their topography, their pharmacology and physiology. In Sect. 7.3, we show how the equations describing hierarchical message passing within a Predictive Coding scheme can be mapped on to key features of intrinsic connections, namely the canonical cortical microcircuit, and their implications for the oscillatory dynamics of different cell populations. In Sect. 7.4, we briefly review some empirical evidence for Predictive Coding in the brain.
-
bayesian inference Predictive Coding and delusions
Avant 5 (3) pp. 51-88. (2015), 2015Co-Authors: Rick A Adams, Harriet R Brown, Karl J FristonAbstract:This paper considers psychotic symptoms in terms of false inferences or beliefs. It is based on the notion that the brain is an organ of inference that actively constructs hypotheses to explain or predict its sensations. This perspective provides a normative (Bayes optimal) account of action and perception that emphasises probabilistic representations; in particular, the confidence or precision of beliefs about the world. We consider sensory attenuation deficits, catatonia and delusions as various expressions of the same core pathology: namely, an aberrant enCoding of precision in a Predictive Coding hierarchy. In Predictive Coding, precision is thought to be encoded by the postsynaptic gain of neurons reporting prediction error. This suggests that both pervasive trait abnormalities and florid failures of inference in the psychotic state can be linked to factors controlling postsynaptic gain-such as NMDA receptor function and (dopaminergic) neuromodulation. We illustrate these points using a biologically plausible simulation of attribution of agency-showing how a reduction in the precision of prior beliefs, relative to sensory evidence, can lead to false inference.
-
reflections on agranular architecture Predictive Coding in the motor cortex
Trends in Neurosciences, 2013Co-Authors: Stewart Shipp, Rick A Adams, Karl J FristonAbstract:The agranular architecture of motor cortex lacks a functional interpretation. Here, we consider a 'Predictive Coding' account of this unique feature based on asymmetries in hierarchical cortical connections. In sensory cortex, layer 4 (the granular layer) is the target of ascending pathways. We theorise that the operation of Predictive Coding in the motor system (a process termed 'active inference') provides a principled rationale for the apparent recession of the ascending pathway in motor cortex. The extension of this theory to interlaminar circuitry also accounts for a sub-class of 'mirror neuron' in motor cortex--whose activity is suppressed when observing an action--explaining how Predictive Coding can gate hierarchical processing to switch between perception and action.
-
Predictive Coding and pitch processing in the auditory cortex
Journal of Cognitive Neuroscience, 2011Co-Authors: Karl J Friston, Sukhbinder Kumar, William Sedley, Kirill V Nourski, Hiroto Kawasaki, Hiroyuki Oya, Roy D Patterson, Matthew A Howard, Timothy D GriffithsAbstract:In this work, we show that electrophysiological responses during pitch perception are best explained by distributed activity in a hierarchy of cortical sources and, crucially, that the effective connectivity between these sources is modulated with pitch strength. Local field potentials were recorded in two subjects from primary auditory cortex and adjacent auditory cortical areas along the axis of Heschl's gyrus (HG) while they listened to stimuli of varying pitch strength. Dynamic causal modeling was used to compare system architectures that might explain the recorded activity. The data show that representation of pitch requires an interaction between nonprimary and primary auditory cortex along HG that is consistent with the principle of Predictive Coding.
Marcel C M Bastiaansen - One of the best experts on this subject based on the ideXlab platform.
-
a Predictive Coding perspective on beta oscillations during sentence level language comprehension
Frontiers in Human Neuroscience, 2016Co-Authors: Ashley Glen Lewis, Marcel C M Bastiaansen, Janmathijs Schoffelen, Herbert SchriefersAbstract:Oscillatory neural dynamics have been steadily receiving more attention as a robust and temporally precise signature of network activity related to language processing. We have recently proposed that oscillatory dynamics in the beta and gamma frequency ranges measured during sentence-level comprehension might be best explained from a Predictive Coding perspective. Under our proposal we related beta oscillations to both the maintenance/change of the neural network configuration responsible for the construction and representation of sentence-level meaning, and to top-down predictions about upcoming linguistic input based on that sentence-level meaning. Here we zoom in on these particular aspects of our proposal, and discuss both old and new supporting evidence. Finally, we present some preliminary MEG data from an experiment comparing Dutch subject- and object-relative clauses that was specifically designed to test our Predictive Coding framework. Initial results support the first of the two suggested roles for beta oscillations in sentence-level language comprehension.
-
a Predictive Coding framework for rapid neural dynamics during sentence level language comprehension
Cortex, 2015Co-Authors: Ashley Glen Lewis, Marcel C M BastiaansenAbstract:There is a growing literature investigating the relationship between oscillatory neural dynamics measured using electroencephalography (EEG) and/or magnetoencephalography (MEG), and sentence-level language comprehension. Recent proposals have suggested a strong link between Predictive Coding accounts of the hierarchical flow of information in the brain, and oscillatory neural dynamics in the beta and gamma frequency ranges. We propose that findings relating beta and gamma oscillations to sentence-level language comprehension might be unified under such a Predictive Coding account. Our suggestion is that oscillatory activity in the beta frequency range may reflect both the active maintenance of the current network configuration responsible for representing the sentence-level meaning under construction, and the top-down propagation of predictions to hierarchically lower processing levels based on that representation. In addition, we suggest that oscillatory activity in the low and middle gamma range reflect the matching of top-down predictions with bottom-up linguistic input, while evoked high gamma might reflect the propagation of bottom-up prediction errors to higher levels of the processing hierarchy. We also discuss some of the implications of this Predictive Coding framework, and we outline ideas for how these might be tested experimentally.
