The Experts below are selected from a list of 204504 Experts worldwide ranked by ideXlab platform

Ugur Yigit - One of the best experts on this subject based on the ideXlab platform.

  • Approche théorie de l’information à l’apprentissage statistique : codage distribué de sources sous mesure de fidélité logarithmique
    HAL CCSD, 2019
    Co-Authors: Ugur Yigit
    Abstract:

    One substantial question, that is often argumentative in Learning theory, is how to choose a `good' loss function that measures the fidelity of the reconstruction to the original. Logarithmic loss is a natural distortion measure in the settings in which the reconstructions are allowed to be `soft', rather than `hard' or deterministic. Logarithmic loss is widely used as a penalty criterion in various contexts, including clustering and classification, pattern Recognition, Learning and prediction, and image processing. Considering the high amount of research which is done recently in these fields, the logarithmic loss becomes a very important metric and will be the main focus as a distortion metric in this thesis.In this thesis, we investigate a distributed setup, so-called the Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, agents observe independently corrupted noisy versions of a remote source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO also has its own noisy observation of the source and wants to reconstruct the remote source to within some prescribed distortion level where the incurred distortion is measured under the logarithmic loss penalty criterion. One of the main contributions of the thesis is the explicit characterization of the rate-distortion region of the vector Gaussian CEO problem, in which the source, observations and side information are jointly Gaussian. For the proof of this result, we first extend Courtade-Weissman's result on the rate-distortion region of the discrete memoryless (DM) CEO problem to the case in which the CEO has access to a correlated side information stream which is such that the agents' observations are independent conditionally given the side information and remote source. Next, we obtain an outer bound on the region of the vector Gaussian CEO problem by evaluating the outer bound of the DM model by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. The approach is similar to Ekrem-Ulukus outer bounding technique for the vector Gaussian CEO problem under quadratic distortion measure, for which it was there found generally non-tight; but it is shown here to yield a complete characterization of the region for the case of logarithmic loss measure. Also, we show that Gaussian test channels with time-sharing exhaust the Berger-Tung inner bound, which is optimal. Furthermore, application of our results allows us to find the complete solutions of three related problems: the quadratic vector Gaussian CEO problem with determinant constraint, the vector Gaussian distributed hypothesis testing against conditional independence problem and the vector Gaussian distributed Information Bottleneck problem.With the known relevance of the logarithmic loss fidelity measure in the context of Learning and prediction, developing algorithms to compute the regions provided in this thesis may find usefulness in a variety of applications where Learning is performed distributively. Motivated from this fact, we develop two type algorithms : i) Blahut-Arimoto (BA) type iterative numerical algorithms for both discrete and Gaussian models in which the joint distribution of the sources are known; and ii) a variational inference type algorithm in which the encoding mappings are parametrized by neural networks and the variational bound approximated by Markov sampling and optimized with stochastic gradient descent for the case in which there is only a set of training data is available. Finally, as an application, we develop an unsupervised generative clustering framework that uses the variational Information Bottleneck (VIB) method and models the latent space as a mixture of Gaussians. This generalizes the VIB which models the latent space as an isotropic Gaussian which is generally not expressive enough for the purpose of unsupervised clusteringUne question de fond, souvent discutée dans l’apprentissage de la théorie, est de savoir comment choisir une fonction de `bonne' perte qui mesure la fidélité de la reconstruction à l’originale. La perte logarithmique est largement utilisée comme critère de pénalité dans divers contextes, notamment le regroupement et la classification, la reconnaissance de formes, l'apprentissage et la prédiction, ainsi que le traitement d'images. Compte tenu du nombre élevé de recherches effectuées récemment dans ces domaines, la perte logarithmique devient une métrique très importante et sera l’axe principal en tant que métrique de distorsion dans cette thèse. Dans cette thèse, nous étudions une configuration distribuée, appelée problème de Chief Executive Officer (CEO) dans le cadre d’une mesure de distorsion de perte logarithmique. Plus précisément, les agents observent les versions bruitées d'une source distante indépendamment corrompues et communiquent indépendamment avec un décodeur ou un chef de la direction via des liaisons sans bruit à contrainte de débit. Le CEO a également sa propre observation bruyante de la source et souhaite reconstruire la source distante dans les limites du niveau de distorsion prescrit, où la distorsion induite est mesurée selon le critère de pénalité de perte logarithmique. Une des contributions principales de la thèse est la caractérisation explicite de la région de distorsion du taux du problème vectoriel du PDG gaussien, dans laquelle la source, les observations et les informations complémentaires sont conjointement gaussiennes. Pour la preuve de ce résultat, nous étendons d'abord résultat de Courtade-Weissman sur la région de distorsion des taux du problème de CEO discret sans mémoire au cas où le CEO a accès à un flux d'informations secondaires corrélé qui est tel que les observations des agents sont indépendantes conditionnellement compte tenu des informations secondaires et de la source distante. Ensuite, nous obtenons une borne externe sur la région du problème de vecteur du CEO gaussien en évaluant la limite extérieure du modèle discret sans mémoire au moyen d'une technique qui s'appuie sur l'identité de Bruijn et les propriétés des informations de Fisher. L'approche est similaire à technique de la délimitation externe d'Ekrem-Ulukus pour le problème de vecteur du CEO gaussien sous mesure de distorsion quadratique, pour lequel il était là trouvé généralement non serré; mais il est montré ici pour donner une caractérisation complète de la région pour le cas de mesure de perte logarithmique. L’application de nos résultats nous permet de trouver des solutions complètes de trois problèmes connexes: le problème du CEO du vecteur quadratique gaussien avec contrainte déterminante, le test d'hypothèses gaussiennes distribuées par vecteur contre le problème de l'indépendance conditionnelle et le problème de goulot d'étranglement d'information distribué gaussien de vecteur. Avec la pertinence connue de la mesure logarithmique de fidélité à la perte dans le contexte de l’apprentissage et de la prédiction, développer des algorithmes pour calculer les régions fournies dans cette thèse peut trouver une utilité dans diverses applications où l'apprentissage est effectué de manière distributive. Motivés par ce fait, nous développons deux algorithmes de type: i) algorithmes numériques itératifs de type Blahut-Arimoto pour les modèles discrets et gaussiens dans lesquels la distribution conjointe des sources est connue; ii) un algorithme de type inférence variationnelle dans lequel les mappages de codage sont paramétrés par des réseaux de neurones et la borne de variation approchée par échantillonnage de Markov et optimisé avec descente de gradient stochastique pour le cas où il n'y a qu'un ensemble de données de formation disponible (...

  • Approche théorie de l’information à l’apprentissage statistique. Codage distribué de sources sous mesure de fidélité logarithmique
    HAL CCSD, 2019
    Co-Authors: Ugur Yigit
    Abstract:

    One substantial question, that is often argumentative in Learning theory, is how to choose a `good' loss function that measures the fidelity of the reconstruction to the original. Logarithmic loss is a natural distortion measure in the settings in which the reconstructions are allowed to be `soft', rather than `hard' or deterministic. In other words, rather than just assigning a deterministic value to each sample of the source, the decoder also gives an assessment of the degree of confidence or reliability on each estimate, in the form of weights or probabilities. This measure has appreciable mathematical properties which establish some important connections with lossy universal compression. Logarithmic loss is widely used as a penalty criterion in various contexts, including clustering and classification, pattern Recognition, Learning and prediction, and image processing. Considering the high amount of research which is done recently in these fields, the logarithmic loss becomes a very important metric and will be the main focus as a distortion metric in this thesis. In this thesis, we investigate a distributed setup, so-called the Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, agents observe independently corrupted noisy versions of a remote source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO also has its own noisy observation of the source and wants to reconstruct the remote source to within some prescribed distortion level where the incurred distortion is measured under the logarithmic loss penalty criterion.One of the main contributions of the thesis is the explicit characterization of the rate-distortion region of the vector Gaussian CEO problem, in which the source, observations and side information are jointly Gaussian. For the proof of this result, we first extend Courtade-Weissman's result on the rate-distortion region of the discrete memoryless (DM) $K$-encoder CEO problem to the case in which the CEO has access to a correlated side information stream which is such that the agents' observations are independent conditionally given the side information and remote source. Next, we obtain an outer bound on the region of the vector Gaussian CEO problem by evaluating the outer bound of the DM model by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. The approach is similar to Ekrem-Ulukus outer bounding technique for the vector Gaussian CEO problem under quadratic distortion measure, for which it was there found generally non-tight; but it is shown here to yield a complete characterization of the region for the case of logarithmic loss measure. Also, we show that Gaussian test channels with time-sharing exhaust the Berger-Tung inner bound, which is optimal. Furthermore, application of our results allows us to find the complete solutions of three related problems: the quadratic vector Gaussian CEO problem with \textit{determinant} constraint, the vector Gaussian distributed hypothesis testing against conditional independence problem and the vector Gaussian distributed Information Bottleneck problem.With the known relevance of the logarithmic loss fidelity measure in the context of Learning and prediction, developing algorithms to compute the regions provided in this thesis may find usefulness in a variety of applications where Learning is performed distributively. Motivated from this fact, we develop two type algorithms: i) Blahut-Arimoto (BA) type iterative numerical algorithms for both discrete and Gaussian models in which the joint distribution of the sources are known; and ii) a variational inference type algorithm in which the encoding mappings are parameterized by neural networks and the variational bound approximated by Monte Carlo sampling and optimized with stochastic gradient descent for the case in which there is only a set of training data is available. Finally, as an application, we develop an unsupervised generative clustering framework that uses the variational Information Bottleneck (VIB) method and models the latent space as a mixture of Gaussians. This generalizes the VIB which models the latent space as an isotropic Gaussian which is generally not expressive enough for the purpose of unsupervised clustering. We illustrate the efficiency of our algorithms through some numerical examples.Une problème de fond, qui suscite un intérêt croissant en théorie de l’apprentissage statistique, est de déterminer des limites d'apprentissage d'une architecture donnée. Aussi, comment choisir la structure la mieux adaptée a un problème donné? Quelle fonction 'risque' fait le plus sens? Ces questions sont d'autant plus importantes que les principales critiques faites aux approches actuelles reposent sur le fait que celles-ci sont très largement expérimentales, et donc sans garanties réelles de reproductibilité ('black-box' approaches). Dans cette thèse, nous adoptons une approche théorie de l'information au problème. En réalité, le problème d'inférence statistique est fondamentalement un problème de codage de source sous mesure de distorsion logarithmique ('logarithmic loss'). Cela peut être démontre a partir des principes de base. La mesure de distorsion logarithmique est largement utilisée comme critère de pénalité dans divers contextes, notamment la prédiction linéaire, la classification, reconnaissance de formes, ainsi que le traitement d'images.Motivés par le problème d'apprentissage a partir de données partielles, nous étudions les limites d'apprentissage du problème de 'Multiview Learning'. Nous établissons des limites fondamentales sur la capacité d'apprentissage dans ce contexte en étudiant, et résolvent, le problème de codage de source distribue sous contrainte de distorsion logarithmique. Dans ce problème, dit 'Chief Executive Officer (CEO) problem', des agents observent les versions bruitées d'une source distante. Les agents communiquent de façon indépendante avec un décodeur commun via des liens a capacité finie. Le décodeur commun, dit aussi CEO, a également sa propre observation bruite de la source et souhaite reconstruire la source distante avec un niveau de fidélité donne, ou la fidélité est mesurée selon le critère de pénalité de perte logarithmique.Une des contributions principales de cette thèse est la caractérisation explicite de la région taux de compression / distorsion du problème de codage distribue de sources (CEO problem) sous perte logarithmique dans le cas ou les sources sont vectorielles et Gaussiennes. Pour la preuve de ce résultat, nous commençons par étendre un résultat important de Courtade-Weissman pour le cas spécifique de même modèle ou le CEO ne possède pas sa propre observation dans le cas discret sans mémoire. Ensuite, nous établissons une borne externe sur la région taux de compression / distorsion du problème du problème CEO avec sources vectorielles Gaussiennes en évaluant la limite extérieure du modèle discret sans mémoire au moyen d'une technique qui s'appuie sur l'identité de Bruijn et les propriétés des informations de Fisher. L'application de nos résultats au problème d'apprentissage via la technique Information Bottleneck nous permet d'établir le compromis optimal entre niveau de précision et capacité de généralisation. De plus, nous développons des algorithmes d'apprentissage distribue, qui sont applicables aussi bien au cas de données discrètes (exemple, classification) et que de données continues (exemple régression linéaire). En particulier, un de nos algorithme est de type inférence variationnelle dans lequel les mappings sont paramétrés par des réseaux de neurones et la fonction coût approchée par échantillonnage de Monte Carlo et optimisé avec descente de gradient stochastique.Nous montrons que nos algorithmes convergent; et nous proposons des structures d'apprentissage basées sur les réseaux de neurones qui permettent d'implémenter efficacement ces algorithmes. Les algorithmes et architectures développés dans cette these sont valides par des résultats expérimentales. Enfin, en tant qu'application, nous développons un cadre de clustering génératif non supervisé qui utilise le modèle VIB et modélise l'espace latent comme un mélange de gaussiennes

Michael T Williams - One of the best experts on this subject based on the ideXlab platform.

  • mouse plasmacytoma expressed transcript 1 knock out induced 5 ht disruption results in a lack of cognitive deficits and an anxiety phenotype complicated by hypoactivity and defensiveness
    Neuroscience, 2009
    Co-Authors: Tori L Schaefer, Charles V Vorhees, Michael T Williams
    Abstract:

    Abstract Serotonin (5-HT) is involved in many developmental processes and influences behaviors including anxiety, aggression, and cognition. Disruption of the serotonergic system has been implicated in human disorders including autism, depression, schizophrenia, and ADHD. Although pharmacological, neurotoxin, and dietary manipulation of 5-HT and tryptophan hydroxylase has added to our understanding of the serotonergic system, the results are complicated by multiple factors. A newly identified ETS domain transcription factor, Pet-1, has direct control of major aspects of 5-HT neuronal development. Pet-1 is the only known factor that is restricted in the brain to 5-HT neurons during development and adulthood and exerts dominant control over 5-HT neuronal phenotype. Disruption of Pet-1 produces an ∼80% loss of 5-HT neurons and content and results in increased aggression in male Pet-1 −/− mice [ Hendricks TJ, Fyodorov DV, Wegman LJ, Lelutiu NB, Pehek EA, Yamamoto B, Silver J, Weeber EJ, Sweatt JD, Deneris ES (2003) Neuron 37:233–247]. We hypothesized that Pet-1 −/− mice would also exhibit changes in anxiety and cognition. Pet-1 −/− mice were hypoactive which may have affected the observed lack of anxious behavior in the elevated zero maze and light-dark test. Pet-1 −/− mice, however, were more defensive during marble burying and showed acoustic startle hyper-reactivity. No deficits in spatial, egocentric, or novel object Recognition Learning were found in Pet-1 −/− mice. These findings were unexpected given that 5-HT depleting drugs given to adult or developing animals result in Learning deficits [ Mazer C, Muneyyirci J, Taheny K, Raio N, Borella A, Whitaker-Azmitia P (1997) Brain Res 760:68–73; Morford LL, Inman-Wood SL, Gudelsky GA, Williams MT, Vorhees CV (2002) Eur J Neurosci 16:491–500; Vorhees CV, Schaefer TL, Williams MT (2007) Synapse 61:488–499]. Lack of differences may be the result of compensatory mechanisms in reaction to a constitutive knock out of Pet-1 or 5-HT may not be as important in Learning and memory as previously suspected.

  • mouse plasmacytoma expressed transcript 1 knock out induced 5 ht disruption results in a lack of cognitive deficits and an anxiety phenotype complicated by hypoactivity and defensiveness
    Neuroscience, 2009
    Co-Authors: Tori L Schaefer, Charles V Vorhees, Michael T Williams
    Abstract:

    Serotonin (5-HT) is involved in many developmental processes and influences behaviors including anxiety, aggression, and cognition. Disruption of the serotonergic system has been implicated in human disorders including autism, depression, schizophrenia, and ADHD. Although pharmacological, neurotoxin, and dietary manipulation of 5-HT and tryptophan hydroxylase has added to our understanding of the serotonergic system, the results are complicated by multiple factors. A newly identified ETS domain transcription factor, Pet-1, has direct control of major aspects of 5-HT neuronal development. Pet-1 is the only known factor that is restricted in the brain to 5-HT neurons during development and adulthood and exerts dominant control over 5-HT neuronal phenotype. Disruption of Pet-1 produces an approximately 80% loss of 5-HT neurons and content and results in increased aggression in male Pet-1(-/-) mice [Hendricks TJ, Fyodorov DV, Wegman LJ, Lelutiu NB, Pehek EA, Yamamoto B, Silver J, Weeber EJ, Sweatt JD, Deneris ES (2003) Neuron 37:233-247]. We hypothesized that Pet-1(-/-) mice would also exhibit changes in anxiety and cognition. Pet-1(-/-) mice were hypoactive which may have affected the observed lack of anxious behavior in the elevated zero maze and light-dark test. Pet-1(-/-) mice, however, were more defensive during marble burying and showed acoustic startle hyper-reactivity. No deficits in spatial, egocentric, or novel object Recognition Learning were found in Pet-1(-/-) mice. These findings were unexpected given that 5-HT depleting drugs given to adult or developing animals result in Learning deficits [Mazer C, Muneyyirci J, Taheny K, Raio N, Borella A, Whitaker-Azmitia P (1997) Brain Res 760:68-73; Morford LL, Inman-Wood SL, Gudelsky GA, Williams MT, Vorhees CV (2002) Eur J Neurosci 16:491-500; Vorhees CV, Schaefer TL, Williams MT (2007) Synapse 61:488-499]. Lack of differences may be the result of compensatory mechanisms in reaction to a constitutive knock out of Pet-1 or 5-HT may not be as important in Learning and memory as previously suspected.

Douglas P Chivers - One of the best experts on this subject based on the ideXlab platform.

  • fathead minnows pimephales promelas acquire predator Recognition when alarm substance is associated with the sight of unfamiliar fish
    Animal Behaviour, 1994
    Co-Authors: Douglas P Chivers, Jan R F Smith
    Abstract:

    Abstract Abstract. To determine whether fathead minnows can learn to recognize potential predators through releaser-induced Recognition Learning, predator-naive minnows were simultaneously exposed to a neutral visual stimulus, the sight of a northern pike, Esox lucius , or a goldfish, Carassius auratus , paired with either minnow alarm substance (Schreckstoff) or water. Two days after this initial conditioning trial the visual stimuli were presented alone and minnows previously conditioned with alarm substance exhibited an appropriate anti-predator response, while those conditioned with water did not. The conditioned minnows were tested approximately 2 months later and again the minnows that were previously conditioned with alarm substance showed an appropriate anti-predator response, while those conditioned with water did not. A comparison of the reaction of minnows conditioned to a natural predator (the pike) and those conditioned to a non-piscivorous exotic (the goldfish) revealed a similar response intensity when tested 2 days after the initial conditioning trial. However, approximately 2 months after the conditioning trial, the reaction of minnows conditioned to pike was stronger than that of minnows conditioned to goldfish, indicating that Learning may be constrained to favour a response to the natural predator. Minnows that were initially conditioned to pike did not show an anti-predator response to goldfish nor did minnows that were conditioned to goldfish respond to pike, demonstrating that the learned response was specific to the species used in the conditioning trials and not to any large fish. These results extend the known benefits to alarm-signal receivers.

  • The role of experience and chemical alarm signalling in predator Recognition by fathead minnows, Pimephales promelas
    Journal of Fish Biology, 1994
    Co-Authors: Douglas P Chivers, R J F Smith
    Abstract:

    Young-of-the-year, predator-naive fathead minnows, Pimephales promelas, from a pikesympatric population did not respond to chemical stimuli from northern pike, Esox Indus, while wild-caught fish of the same age and size did. These results suggest that chemical predator Recognition is a result of previous experience and not genetic factors, Wild young-of-the-year minnows responded to pike odour with a response intensity that was similar to that of older fish, demonstrating that the ability to recognize predators is learned within the first year. The intensity of response of wild minnows which had been maintained in a predator free environment for 1 year was similar to that of recently caught minnows of the same age, suggesting that reinforcement was not required for predator Recognition to be retained. Naive minnows that were exposed simultaneously to chemical stimuli from pike (a neutral stimulus) and minnow alarm substance exhibited a fright response upon subsequent exposure to the pike stimulus alone. Predator-naive minnows exposed simultaneously to chemical stimuli from pike and glass-distilled water did not exhibit a fright response to the pike stimulus alone. These results demonstrate that fathead minnows can acquire predator Recognition through releaserinduced Recognition Learning, thus confirming a known mechanism through which alarm substance may benefit the receivers of an alarm signal.

Jianfeng Feng - One of the best experts on this subject based on the ideXlab platform.

  • a novel extended granger causal model approach demonstrates brain hemispheric differences during face Recognition Learning
    PLOS Computational Biology, 2009
    Co-Authors: Tian Ge, Keith M Kendrick, Jianfeng Feng
    Abstract:

    Two main approaches in exploring causal relationships in biological systems using time-series data are the application of Dynamic Causal model (DCM) and Granger Causal model (GCM). These have been extensively applied to brain imaging data and are also readily applicable to a wide range of temporal changes involving genes, proteins or metabolic pathways. However, these two approaches have always been considered to be radically different from each other and therefore used independently. Here we present a novel approach which is an extension of Granger Causal model and also shares the features of the bilinear approximation of Dynamic Causal model. We have first tested the efficacy of the extended GCM by applying it extensively in toy models in both time and frequency domains and then applied it to local field potential recording data collected from in vivo multi-electrode array experiments. We demonstrate face discrimination Learning-induced changes in inter- and intra-hemispheric connectivity and in the hemispheric predominance of theta and gamma frequency oscillations in sheep inferotemporal cortex. The results provide the first evidence for connectivity changes between and within left and right inferotemporal cortexes as a result of face Recognition Learning.

Jan R F Smith - One of the best experts on this subject based on the ideXlab platform.

  • fathead minnows pimephales promelas acquire predator Recognition when alarm substance is associated with the sight of unfamiliar fish
    Animal Behaviour, 1994
    Co-Authors: Douglas P Chivers, Jan R F Smith
    Abstract:

    Abstract Abstract. To determine whether fathead minnows can learn to recognize potential predators through releaser-induced Recognition Learning, predator-naive minnows were simultaneously exposed to a neutral visual stimulus, the sight of a northern pike, Esox lucius , or a goldfish, Carassius auratus , paired with either minnow alarm substance (Schreckstoff) or water. Two days after this initial conditioning trial the visual stimuli were presented alone and minnows previously conditioned with alarm substance exhibited an appropriate anti-predator response, while those conditioned with water did not. The conditioned minnows were tested approximately 2 months later and again the minnows that were previously conditioned with alarm substance showed an appropriate anti-predator response, while those conditioned with water did not. A comparison of the reaction of minnows conditioned to a natural predator (the pike) and those conditioned to a non-piscivorous exotic (the goldfish) revealed a similar response intensity when tested 2 days after the initial conditioning trial. However, approximately 2 months after the conditioning trial, the reaction of minnows conditioned to pike was stronger than that of minnows conditioned to goldfish, indicating that Learning may be constrained to favour a response to the natural predator. Minnows that were initially conditioned to pike did not show an anti-predator response to goldfish nor did minnows that were conditioned to goldfish respond to pike, demonstrating that the learned response was specific to the species used in the conditioning trials and not to any large fish. These results extend the known benefits to alarm-signal receivers.