The Experts below are selected from a list of 45348 Experts worldwide ranked by ideXlab platform
Olivier Sigaud - One of the best experts on this subject based on the ideXlab platform.
-
combining Latent Learning with dynamic programming in the modular anticipatory classifier system
European Journal of Operational Research, 2005Co-Authors: Pierre Gérard, Jean-arcady Meyer, Olivier SigaudAbstract:Learning Classifier Systems (LCS) are rule based Reinforcement Learning (RL) systems which use a generalization capability. In this paper, we highlight the differences between two kinds of LCSs. Some are used to directly perform RL while others Latently learn a model of the interactions between the agent and its environment. Such a model can be used to speed up the core RL process. Thus, these two kinds of Learning processes are complementary. We show here how the notion of generalization differs depending on whether the system anticipates (like Anticipatory Classifier System (ACS) and Yet Another Classifier System (YACS)) or not (like XCS). Moreover, we show some limitations of the formalism common to ACS and YACS, and propose a new system, called Modular Anticipatory Classifier System (MACS), which allows the Latent Learning process to take advantage of new regularities. We describe how the model can be used to perform active exploration and how this exploration may be aggregated with the policy resulting from the reinforcement Learning process. The different algorithms are validated experimentally and some limitations in presence of uncertainties are highlighted.
-
yacs combining dynamic programming with generalization in classifier systems
Lecture Notes in Computer Science, 2001Co-Authors: Pierre Gérard, Olivier SigaudAbstract:This paper describes our work on the use of anticipation in Learning Classifier Systems (LCS) applied to Markov problems. We present YACS 1 , a new kind of Anticipatory Classifier System. It calls upon classifiers with a [Condition], an [Action] and an [Effect] part. As in the traditional LCS framework, the classifier discovery process relies on a selection and a creation mechanism. As in the Anticipatory Classifier System (ACS), YACS looks for classifiers which anticipate well rather than for classifiers which propose an optimal action. The creation mechanism does not rely on classical genetic operators but on a specialization operator, which is explicitly driven by experience. Likewise, the action qualities of the classifiers are not computed by a classical bucket-brigade algorithm, but by a variety of the value iteration algorithm that takes advantage of the effect part of the classifiers. This paper presents the Latent Learning process of YACS. The description of the reinforcement Learning process is focussed on the problem induced by the joint use of generalization and dynamic programming methods.
Pierre Gérard - One of the best experts on this subject based on the ideXlab platform.
-
combining Latent Learning with dynamic programming in the modular anticipatory classifier system
European Journal of Operational Research, 2005Co-Authors: Pierre Gérard, Jean-arcady Meyer, Olivier SigaudAbstract:Learning Classifier Systems (LCS) are rule based Reinforcement Learning (RL) systems which use a generalization capability. In this paper, we highlight the differences between two kinds of LCSs. Some are used to directly perform RL while others Latently learn a model of the interactions between the agent and its environment. Such a model can be used to speed up the core RL process. Thus, these two kinds of Learning processes are complementary. We show here how the notion of generalization differs depending on whether the system anticipates (like Anticipatory Classifier System (ACS) and Yet Another Classifier System (YACS)) or not (like XCS). Moreover, we show some limitations of the formalism common to ACS and YACS, and propose a new system, called Modular Anticipatory Classifier System (MACS), which allows the Latent Learning process to take advantage of new regularities. We describe how the model can be used to perform active exploration and how this exploration may be aggregated with the policy resulting from the reinforcement Learning process. The different algorithms are validated experimentally and some limitations in presence of uncertainties are highlighted.
-
yacs combining dynamic programming with generalization in classifier systems
Lecture Notes in Computer Science, 2001Co-Authors: Pierre Gérard, Olivier SigaudAbstract:This paper describes our work on the use of anticipation in Learning Classifier Systems (LCS) applied to Markov problems. We present YACS 1 , a new kind of Anticipatory Classifier System. It calls upon classifiers with a [Condition], an [Action] and an [Effect] part. As in the traditional LCS framework, the classifier discovery process relies on a selection and a creation mechanism. As in the Anticipatory Classifier System (ACS), YACS looks for classifiers which anticipate well rather than for classifiers which propose an optimal action. The creation mechanism does not rely on classical genetic operators but on a specialization operator, which is explicitly driven by experience. Likewise, the action qualities of the classifiers are not computed by a classical bucket-brigade algorithm, but by a variety of the value iteration algorithm that takes advantage of the effect part of the classifiers. This paper presents the Latent Learning process of YACS. The description of the reinforcement Learning process is focussed on the problem induced by the joint use of generalization and dynamic programming methods.
Toshitaka Nabeshima - One of the best experts on this subject based on the ideXlab platform.
-
involvement of a dysfunctional dopamine d1 n methyl d aspartate nr1 and ca2 calmodulin dependent protein kinase ii pathway in the impairment of Latent Learning in a model of schizophrenia induced by phencyclidine
Molecular Pharmacology, 2007Co-Authors: Akihiro Mouri, Yukihiro Noda, Akihiro Noda, Tomonobu Nakamura, Takanobu Tokura, Yoshimitsu Yura, Atsumi Nitta, Hiroshi Furukawa, Toshitaka NabeshimaAbstract:Continuous ingestion of phencyclidine (PCP) in humans produces long-lasting schizophrenic-like cognitive dysfunction. Although a malfunction of dopaminergic and/or glutamatergic neurotransmission is implicated in the etiology of schizophrenia, involvement of the dopaminergic-glutamatergic neurotransmission in the cognitive dysfunction induced by repeated PCP treatment is minor. We demonstrated that mice treated with PCP (10 mg/kg/day s.c.) for 14 days displayed an impairment of Latent Learning in a water-finding task and of Learning-associated phosphorylation of Ca 2+ /calmodulin-dependent protein kinase II (CaMKII) and NR1 in the prefrontal cortex even after drug withdrawal. The infusion of a CaMKII inhibitor and NR1 antisense oligonucleotide into the prefrontal cortex produced an impairment of Latent Learning and decrease of Learning-associated phosphorylation of CaMKII, which were observed in the PCP-treated mice. Exogenous NMDA-induced CaMKII activation was not observed in slices of the prefrontal cortex prepared from mice treated repeatedly with PCP. The potentiation of NMDA receptor function by the infusion of glycine into the prefrontal cortex ameliorated these impairments in mice treated repeatedly with PCP. The high potassium-stimulated release of dopamine from the prefrontal cortex was less extensive in the PCP-treated than saline-treated mice. The infusion of a dopamine-D1 receptor agonist into the prefrontal cortex attenuated the impairment of Latent Learning and decrease of Learning-associated NR1 phosphorylation in the PCP-treated mice, suggesting a functional linkage between glutamatergic and dopaminergic signaling. These findings indicate that repeated PCP treatment impairs Latent Learning through a prefrontal cortical dysfunction of NMDA-CaMKII signaling, which is associated with dopaminergic hypofunction.
-
enhancement of spatial attention in nociceptin orphanin fq receptor knockout mice
Brain Research, 1998Co-Authors: Takayoshi Mamiya, Hiroshi Takeshima, Miyuki Nishi, Yukihiro Noda, Toshitaka NabeshimaAbstract:We isolated genes for the opioid receptor homologue MOR-C, namely nociceptin receptor (designated alternatively as orphanin FQ receptor) and generated nociceptin receptor-knockout mice. Previously, we have reported that the nociceptin system appears to participate in the regulation of the auditory system. However, the behavior of the nociceptin receptor-knockout mice has yet to be fully characterized. In the present study, we investigated changes in several behavioral performances in mice which lack nociceptin receptor. Nociceptive thresholds of nociceptin receptor-knockout mice were unchanged in the hot-plate and electric foot-shock tests as well as tail-flick and acetic acid-induced writhing tests compared to those of wild-type mice. The nociceptin receptor-knockout mice did not show any behavioral changes in the elevated plus-maze task. Surprisingly, in the water-finding test, the nociceptin receptor-knockout mice showed an enhanced retention of spatial attention (Latent Learning) compared to wild-type mice. In a biochemical study, dopamine content in the frontal cortex was lower in nociceptin receptor-knockout mice than wild-type mice. These results suggest that nociceptin receptor plays an important role in spatial attention by regulating the dopaminergic system in the brain.
-
eburnamenine 14 carboxylic acid 2 nitroxyethyl ester va 045 a putative cognitive enhancer facilitates recovery from concussive brain injury induced Learning and memory impairments in mice
Behavioural Brain Research, 1997Co-Authors: Ya Ping Tang, Yukihiro Noda, Takaaki Hasegawa, Toshitaka NabeshimaAbstract:Abstract We characterized alterations in the ability of concussive brain injury (CBI) models to perform a water-finding task and examined effects of (+)-eburnamenine-14-carboxylic acid (2-nitroxyethyl) ester (VA-045), a novel apovincaminic acid derivative, on post-traumatic impairments in Latent Learning and memory processes. Two types of CBI-induced amnesia (retrograde and anterograde) were produced by means of post- or preacquisition head impact using a simple weight-drop device. Profound impairments of Latent Learning and memory processes related to retention and retrieval were observed in the CBI mice. In the CBI-induced retrograde amnesia model, VA-045 (0.5–4.0 mg/kg) significantly ameliorated impairments of Latent Learning and retention in both the retention test and the retest. In the CBI-induced anterograde amnesia model, the protective effects of the compound on impairments in Latent Learning and retention or retrieval were significant in both the retention test and the retest. These results suggested that VA-045 may be a novel cognitive enhancer for attenuating or protecting against the Learning and memory dysfunction associated with CBI.
Yukihiro Noda - One of the best experts on this subject based on the ideXlab platform.
-
involvement of a dysfunctional dopamine d1 n methyl d aspartate nr1 and ca2 calmodulin dependent protein kinase ii pathway in the impairment of Latent Learning in a model of schizophrenia induced by phencyclidine
Molecular Pharmacology, 2007Co-Authors: Akihiro Mouri, Yukihiro Noda, Akihiro Noda, Tomonobu Nakamura, Takanobu Tokura, Yoshimitsu Yura, Atsumi Nitta, Hiroshi Furukawa, Toshitaka NabeshimaAbstract:Continuous ingestion of phencyclidine (PCP) in humans produces long-lasting schizophrenic-like cognitive dysfunction. Although a malfunction of dopaminergic and/or glutamatergic neurotransmission is implicated in the etiology of schizophrenia, involvement of the dopaminergic-glutamatergic neurotransmission in the cognitive dysfunction induced by repeated PCP treatment is minor. We demonstrated that mice treated with PCP (10 mg/kg/day s.c.) for 14 days displayed an impairment of Latent Learning in a water-finding task and of Learning-associated phosphorylation of Ca 2+ /calmodulin-dependent protein kinase II (CaMKII) and NR1 in the prefrontal cortex even after drug withdrawal. The infusion of a CaMKII inhibitor and NR1 antisense oligonucleotide into the prefrontal cortex produced an impairment of Latent Learning and decrease of Learning-associated phosphorylation of CaMKII, which were observed in the PCP-treated mice. Exogenous NMDA-induced CaMKII activation was not observed in slices of the prefrontal cortex prepared from mice treated repeatedly with PCP. The potentiation of NMDA receptor function by the infusion of glycine into the prefrontal cortex ameliorated these impairments in mice treated repeatedly with PCP. The high potassium-stimulated release of dopamine from the prefrontal cortex was less extensive in the PCP-treated than saline-treated mice. The infusion of a dopamine-D1 receptor agonist into the prefrontal cortex attenuated the impairment of Latent Learning and decrease of Learning-associated NR1 phosphorylation in the PCP-treated mice, suggesting a functional linkage between glutamatergic and dopaminergic signaling. These findings indicate that repeated PCP treatment impairs Latent Learning through a prefrontal cortical dysfunction of NMDA-CaMKII signaling, which is associated with dopaminergic hypofunction.
-
enhancement of spatial attention in nociceptin orphanin fq receptor knockout mice
Brain Research, 1998Co-Authors: Takayoshi Mamiya, Hiroshi Takeshima, Miyuki Nishi, Yukihiro Noda, Toshitaka NabeshimaAbstract:We isolated genes for the opioid receptor homologue MOR-C, namely nociceptin receptor (designated alternatively as orphanin FQ receptor) and generated nociceptin receptor-knockout mice. Previously, we have reported that the nociceptin system appears to participate in the regulation of the auditory system. However, the behavior of the nociceptin receptor-knockout mice has yet to be fully characterized. In the present study, we investigated changes in several behavioral performances in mice which lack nociceptin receptor. Nociceptive thresholds of nociceptin receptor-knockout mice were unchanged in the hot-plate and electric foot-shock tests as well as tail-flick and acetic acid-induced writhing tests compared to those of wild-type mice. The nociceptin receptor-knockout mice did not show any behavioral changes in the elevated plus-maze task. Surprisingly, in the water-finding test, the nociceptin receptor-knockout mice showed an enhanced retention of spatial attention (Latent Learning) compared to wild-type mice. In a biochemical study, dopamine content in the frontal cortex was lower in nociceptin receptor-knockout mice than wild-type mice. These results suggest that nociceptin receptor plays an important role in spatial attention by regulating the dopaminergic system in the brain.
-
eburnamenine 14 carboxylic acid 2 nitroxyethyl ester va 045 a putative cognitive enhancer facilitates recovery from concussive brain injury induced Learning and memory impairments in mice
Behavioural Brain Research, 1997Co-Authors: Ya Ping Tang, Yukihiro Noda, Takaaki Hasegawa, Toshitaka NabeshimaAbstract:Abstract We characterized alterations in the ability of concussive brain injury (CBI) models to perform a water-finding task and examined effects of (+)-eburnamenine-14-carboxylic acid (2-nitroxyethyl) ester (VA-045), a novel apovincaminic acid derivative, on post-traumatic impairments in Latent Learning and memory processes. Two types of CBI-induced amnesia (retrograde and anterograde) were produced by means of post- or preacquisition head impact using a simple weight-drop device. Profound impairments of Latent Learning and memory processes related to retention and retrieval were observed in the CBI mice. In the CBI-induced retrograde amnesia model, VA-045 (0.5–4.0 mg/kg) significantly ameliorated impairments of Latent Learning and retention in both the retention test and the retest. In the CBI-induced anterograde amnesia model, the protective effects of the compound on impairments in Latent Learning and retention or retrieval were significant in both the retention test and the retest. These results suggested that VA-045 may be a novel cognitive enhancer for attenuating or protecting against the Learning and memory dysfunction associated with CBI.
Jessica E Yoder - One of the best experts on this subject based on the ideXlab platform.
-
middle aged 12 month old male rats show selective Latent Learning deficit
Neurobiology of Aging, 2011Co-Authors: Eric M Stouffer, Jessica E YoderAbstract:While many cognitive aging studies have been conducted using old (20+ months old) rats, few have demonstrated cognitive deficits in middle-aged (12 months old) rats. The present study was conducted to determine if deficits in Latent Learning (the acquisition of neutral information that does not immediately influence behavior) arise during middle age in rats. Twelve young (3 months old) and 12 middle-aged male Sprague-Dawley rats completed the Latent cue preference (LCP) task, a conditioned cue preference (CCP) task in the same apparatus, and a reinforced spatial Learning task using the Barnes maze. Results showed that the middle-aged rats were impaired on the Latent Learning (LCP) task relative to the young rats, but were not impaired on the CCP task or the spatial Learning task. This may be because Latent Learning requires a functional entorhinal cortex, and the entorhinal cortex is one brain region that shows early age-related functional degeneration.
