The Experts below are selected from a list of 86499 Experts worldwide ranked by ideXlab platform
Xiao Jing Wang - One of the best experts on this subject based on the ideXlab platform.
-
inter areal balanced amplification enhances Signal Propagation in a large scale circuit model of the primate cortex
Neuron, 2018Co-Authors: Madhura R Joglekar, Jorge F Mejias, Guangyu Robert Yang, Xiao Jing WangAbstract:Summary Understanding reliable Signal transmission represents a notable challenge for cortical systems, which display a wide range of weights of feedforward and feedback connections among heterogeneous areas. We re-examine the question of Signal transmission across the cortex in a network model based on mesoscopic directed and weighted inter-areal connectivity data of the macaque cortex. Our findings reveal that, in contrast to purely feedforward Propagation models, the presence of long-range excitatory feedback projections could compromise stable Signal Propagation. Using population rate models as well as a spiking network model, we find that effective Signal Propagation can be accomplished by balanced amplification across cortical areas while ensuring dynamical stability. Moreover, the activation of prefrontal cortex in our model requires the input strength to exceed a threshold, which is consistent with the ignition model of conscious processing. These findings demonstrate our model as an anatomically realistic platform for investigations of global primate cortex dynamics.
-
inter areal balanced amplification enhances Signal Propagation in a large scale circuit model of the primate cortex
bioRxiv, 2017Co-Authors: Madhura R Joglekar, Jorge F Mejias, Guangyu Robert Yang, Xiao Jing WangAbstract:Reliable Signal transmission represents a fundamental challenge for cortical systems, which display a wide range of weights of feedforward and feedback connections among heterogeneous areas. We re-examine the question of Signal transmission across the cortex in network models based on recently available mesoscopic, directed- and weighted- inter-areal connectivity data of the macaque cortex. Our findings reveal that, in contrast to feed-forward Propagation models, the presence of long-range excitatory feedback projections could compromise stable Signal Propagation. Using population rate models as well as a spiking network model, we find that effective Signal Propagation can be accomplished by balanced amplification across cortical areas while ensuring dynamical stability. Moreover, the activation of prefrontal cortex in our model requires the input strength to exceed a threshold, in support of the ignition model of conscious processing, demonstrating our model as an anatomically-realistic platform for investigations of the global primate cortex dynamics.
Herman Kamper - One of the best experts on this subject based on the ideXlab platform.
-
critical initialisation for deep Signal Propagation in noisy rectifier neural networks
arXiv: Machine Learning, 2018Co-Authors: Arnu Pretorius, Elan Van Biljon, Steve Kroon, Herman KamperAbstract:Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influences Signal Propagation in deep neural networks remains limited. By extending recent work based on mean field theory, we develop a new framework for Signal Propagation in stochastic regularised neural networks. Our noisy Signal Propagation theory can incorporate several common noise distributions, including additive and multiplicative Gaussian noise as well as dropout. We use this framework to investigate initialisation strategies for noisy ReLU networks. We show that no critical initialisation strategy exists using additive noise, with Signal Propagation exploding regardless of the selected noise distribution. For multiplicative noise (e.g. dropout), we identify alternative critical initialisation strategies that depend on the second moment of the noise distribution. Simulations and experiments on real-world data confirm that our proposed initialisation is able to stably propagate Signals in deep networks, while using an initialisation disregarding noise fails to do so. Furthermore, we analyse correlation dynamics between inputs. Stronger noise regularisation is shown to reduce the depth to which discriminatory information about the inputs to a noisy ReLU network is able to propagate, even when initialised at criticality. We support our theoretical predictions for these trainable depths with simulations, as well as with experiments on MNIST and CIFAR-10
-
critical initialisation for deep Signal Propagation in noisy rectifier neural networks
Neural Information Processing Systems, 2018Co-Authors: Arnu Pretorius, Elan Van Biljon, Steve Kroon, Herman KamperAbstract:Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influences Signal Propagation in deep neural networks remains limited. By extending recent work based on mean field theory, we develop a new framework for Signal Propagation in stochastic regularised neural networks. Our \textit{noisy Signal Propagation} theory can incorporate several common noise distributions, including additive and multiplicative Gaussian noise as well as dropout. We use this framework to investigate initialisation strategies for noisy ReLU networks. We show that no critical initialisation strategy exists using additive noise, with Signal Propagation exploding regardless of the selected noise distribution. For multiplicative noise (e.g.\ dropout), we identify alternative critical initialisation strategies that depend on the second moment of the noise distribution. Simulations and experiments on real-world data confirm that our proposed initialisation is able to stably propagate Signals in deep networks, while using an initialisation disregarding noise fails to do so. Furthermore, we analyse correlation dynamics between inputs. Stronger noise regularisation is shown to reduce the depth to which discriminatory information about the inputs to a noisy ReLU network is able to propagate, even when initialised at criticality. We support our theoretical predictions for these trainable depths with simulations, as well as with experiments on MNIST and CIFAR-10.
Arnu Pretorius - One of the best experts on this subject based on the ideXlab platform.
-
critical initialisation for deep Signal Propagation in noisy rectifier neural networks
arXiv: Machine Learning, 2018Co-Authors: Arnu Pretorius, Elan Van Biljon, Steve Kroon, Herman KamperAbstract:Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influences Signal Propagation in deep neural networks remains limited. By extending recent work based on mean field theory, we develop a new framework for Signal Propagation in stochastic regularised neural networks. Our noisy Signal Propagation theory can incorporate several common noise distributions, including additive and multiplicative Gaussian noise as well as dropout. We use this framework to investigate initialisation strategies for noisy ReLU networks. We show that no critical initialisation strategy exists using additive noise, with Signal Propagation exploding regardless of the selected noise distribution. For multiplicative noise (e.g. dropout), we identify alternative critical initialisation strategies that depend on the second moment of the noise distribution. Simulations and experiments on real-world data confirm that our proposed initialisation is able to stably propagate Signals in deep networks, while using an initialisation disregarding noise fails to do so. Furthermore, we analyse correlation dynamics between inputs. Stronger noise regularisation is shown to reduce the depth to which discriminatory information about the inputs to a noisy ReLU network is able to propagate, even when initialised at criticality. We support our theoretical predictions for these trainable depths with simulations, as well as with experiments on MNIST and CIFAR-10
-
critical initialisation for deep Signal Propagation in noisy rectifier neural networks
Neural Information Processing Systems, 2018Co-Authors: Arnu Pretorius, Elan Van Biljon, Steve Kroon, Herman KamperAbstract:Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influences Signal Propagation in deep neural networks remains limited. By extending recent work based on mean field theory, we develop a new framework for Signal Propagation in stochastic regularised neural networks. Our \textit{noisy Signal Propagation} theory can incorporate several common noise distributions, including additive and multiplicative Gaussian noise as well as dropout. We use this framework to investigate initialisation strategies for noisy ReLU networks. We show that no critical initialisation strategy exists using additive noise, with Signal Propagation exploding regardless of the selected noise distribution. For multiplicative noise (e.g.\ dropout), we identify alternative critical initialisation strategies that depend on the second moment of the noise distribution. Simulations and experiments on real-world data confirm that our proposed initialisation is able to stably propagate Signals in deep networks, while using an initialisation disregarding noise fails to do so. Furthermore, we analyse correlation dynamics between inputs. Stronger noise regularisation is shown to reduce the depth to which discriminatory information about the inputs to a noisy ReLU network is able to propagate, even when initialised at criticality. We support our theoretical predictions for these trainable depths with simulations, as well as with experiments on MNIST and CIFAR-10.
Mike Roper - One of the best experts on this subject based on the ideXlab platform.
-
finite difference time domain modelling of through the earth radio Signal Propagation
Computers & Geosciences, 2015Co-Authors: M Ralchenko, Markus Svilans, C Samson, Mike RoperAbstract:This research seeks to extend the knowledge of how a very low frequency (VLF) through-the-Earth (TTE) radio Signal behaves as it propagates underground, by calculating and visualizing the strength of the electric and magnetic fields for an arbitrary geology through numeric modelling. To achieve this objective, a new software tool has been developed using the finite-difference time-domain method. This technique is particularly well suited to visualizing the distribution of electromagnetic fields in an arbitrary geology. The frequency range of TTE radio (400-9000Hz) and geometrical scales involved (1m resolution for domains a few hundred metres in size) involves processing a grid composed of millions of cells for thousands of time steps, which is computationally expensive. Graphics processing unit acceleration was used to reduce execution time from days and weeks, to minutes and hours. Results from the new modelling tool were compared to three cases for which an analytic solution is known. Two more case studies were done featuring complex geologic environments relevant to TTE communications that cannot be solved analytically. There was good agreement between numeric and analytic results. Deviations were likely caused by numeric artifacts from the model boundaries; however, in a TTE application in field conditions, the uncertainty in the conductivity of the various geologic formations will greatly outweigh these small numeric errors. HighlightsSoftware to simulate through-the-Earth radio Signal Propagation has been developed.Finite-difference time-domain method is used to directly generate a time series.A graphics processing unit was used to parallelize and accelerate computations.Model results were validated against three analytic solutions.Two complex geologic environments that are not solvable analytically were modelled.
Michael A Scarpulla - One of the best experts on this subject based on the ideXlab platform.
-
Signal Propagation through piecewise transmission lines for interpretation of reflectometry in photovoltaic systems
IEEE Journal of Photovoltaics, 2019Co-Authors: Mashad Uddin Saleh, Josiah Lacombe, Naveen Kumar Tumkur Jayakumar, Samuel Kingston, Joel B Harley, Cynthia Furse, Michael A ScarpullaAbstract:We present a framework for analyzing electromagnetic Signal Propagation through piecewise-defined transmission lines with arbitrary, series-connected impedances. While the formulation is general and scalable, we apply it here to Propagation through a photovoltaic module with cables on either side acting, with a home run cable, as a section of an inhomogeneous transmission line. Understanding Propagation through this unit of a series-connected string of photovoltaic modules is necessary to enable the use of time-domain reflectometry techniques for monitoring the status of individual components in series-connected strings within large photovoltaic arrays.
