The Experts below are selected from a list of 11397 Experts worldwide ranked by ideXlab platform
Tamer Basar - One of the best experts on this subject based on the ideXlab platform.
-
finite sample analysis for decentralized batch multi agent reinforcement learning with networked agents
IEEE Transactions on Automatic Control, 2021Co-Authors: Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, Tamer BasarAbstract:Despite the increasing interest in multi-agent reinforcement learning (Marl) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this paper, we address this problem by providing a finite-sample analysis for decentralized batch Marl. Specifically, we consider a type of mixed Marl setting with both cooperative and competitive agents, where two teams of agents compete in a zero-sum game setting, while the agents within each team collaborate by communicating over a time-varying network. This setting covers many conventional Marl settings in the literature. We then develop batch Marl algorithms that can be implemented in a decentralized fashion, and quantify the finite-sample errors of the estimated action-value functions. Our error analysis captures how the function class, the number of samples within each iteration, and the number of iterations determine the statistical accuracy of the proposed algorithms. Our results, compared to the finite-sample bounds for single-agent RL, involve additional error terms caused by decentralized computation, which is inherent in our decentralized Marl setting. This work provides the
-
finite sample analysis for decentralized batch multi agent reinforcement learning with networked agents
arXiv: Learning, 2018Co-Authors: Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, Tamer BasarAbstract:Despite the increasing interest in multi-agent reinforcement learning (Marl) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this work, we address this problem by providing a finite-sample analysis for decentralized batch Marl with networked agents. Specifically, we consider two decentralized Marl settings, where teams of agents are connected by time-varying communication networks, and either collaborate or compete in a zero-sum game setting, without any central controller. These settings cover many conventional Marl settings in the literature. For both settings, we develop batch Marl algorithms that can be implemented in a decentralized fashion, and quantify the finite-sample errors of the estimated action-value functions. Our error analysis captures how the function class, the number of samples within each iteration, and the number of iterations determine the statistical accuracy of the proposed algorithms. Our results, compared to the finite-sample bounds for single-agent RL, involve additional error terms caused by decentralized computation, which is inherent in our decentralized Marl setting. This work appears to be the first finite-sample analysis for batch Marl, a step towards rigorous theoretical understanding of general Marl algorithms in the finite-sample regime.
Bodil W Lauridsen - One of the best experts on this subject based on the ideXlab platform.
-
episodic dysoxia during late cretaceous cyclic chalk Marl deposition evidence from framboidal pyrite distribution in the upper maastrichtian rordal mb danish basin
Cretaceous Research, 2020Co-Authors: Mattia Tagliavento, Bodil W Lauridsen, Lars StemmerikAbstract:Abstract This study investigates the use of pyrite framboid data to increase insight in sea floor environmental conditions during deposition of chalk-Marl cycles in the Upper Cretaceous of the Danish Basin, NW Europe. Framboid data have been used successfully to differentiate stages of oxygen reduction in siliciclastic mud rocks, and the data presented here indicate that they can be used to identify temporary events of oxygen deficiency, otherwise hidden by the intense bioturbation of the chalk and Marl. The study is based on investigation of 23 samples from the 30 m thick chalk-Marl alternations in the upper Maastrichtian Rordal Member in the Dalbyover-1 core, Denmark. Framboid pyrite data are integrated with sulphur isotope data, observations of sedimentary facies, bioturbation and macro fossils in core, and well log data. The study reveals that framboids are completely absent in the chalk units and present in all Marl layers. Their size and size distribution are indicative of different stages of dysoxia during Marl deposition. The dysoxic events were probably short-lived so successive generations of burrowing animals destroyed any primary depositional signals, like e.g. lamination.
-
trace fossils of a cyclic chalk Marl succession the upper maastrichtian rordal member denmark
Cretaceous Research, 2011Co-Authors: Bodil W Lauridsen, Finn Surlyk, R G BromleyAbstract:Abstract Trace fossils from an upper Maastrichtian cyclic chalk–Marl succession, the Rordal Member, exposed in the Rordal quarry, Denmark, are analysed in order to test whether the changes in substrate lithology exerted any control over the ichnodiversity, tiering complexity, and density of the infauna. The cyclicity is interpreted as caused by orbital changes within the Milankovitch frequency band. The carbonate content varies between 71 and 82 weight% in the Marl and 82–92 weight% in the chalk beds. The material is based on 19 samples collected from six chalk and Marl beds. The investigated bedding-normal sample surfaces vary in area between 29 and 155 cm 2 . Eight ichnogenera and two undetermined ichnogenera are recognised. The member is characterised by three ichnofabrics (A, B and C). The ichnofabric analysis is based on texture and internal structure of the sediments resulting from bioturbation. Ichnofabric A is found only in chalk samples and shows a poor preservation of trace fossils, whereas ichnofabric C is found in a few chalk and all Marl samples and comprises a very dense, diverse and well preserved ichnofauna representing a high tiering complexity. Ichnofabric B represents an intermediate situation between ichnofabrics A and C and occurs in chalk samples immediately adjacent to Marl beds. The observed changes in ichnofabrics between chalk and Marl are related to the amount of clay in the samples and the differences in the occurrence of trace fossils are interpreted as due to differences in the visibility of traces between chalk and Marl and not due to differences in ecological stress upon the endobenthic community of the two lithologies. The study thus provides an excellent example of how the effect of taphonomic factors may give a misleading and biased impression of apparent differences in the endobenthic community between chalk and Marl.
-
benthic macrofauna variations and community structure in cenomanian cyclic chalk Marl from southerham grey pit se england
Journal of the Geological Society, 2009Co-Authors: Bodil W Lauridsen, Andrew S Gale, Finn SurlykAbstract:Abstract: Cenomanian chalk–Marl couplets from England represent the 20 ka Milankovitch precession cycle. Fossil communities from both chalk and Marl are identified to test if the orbital fluctuations and the associated changes in substrate lithology and climate exerted any control on the benthic macrofauna. The material comprises washing residues of 24 bulk samples collected from chalk and Marl half-cycles. A total of 5055 invertebrate specimens were retrieved and referred to 68 species, forming the basis for the recognition of six guilds. In general, the fauna is more diverse in Marl than in chalk, but it is remarkable and somewhat surprising that the guild assemblages of Marl and chalk show no difference. Thus the substrate character of the two lithologies did not exert any influence on the modes of life of the benthic fauna in spite of a difference in clay content of about 20%. The fauna was clearly well adapted to both facies and thus to the fine grain size of the substrate rather than to lithology. The systematic difference in diversity between chalk and Marl samples was possibly caused by long-term climatic and oceanographic changes and thus could represent a biological response to Milankovitch cyclicity.
-
benthic faunal response to late maastrichtian chalk Marl cyclicity at rordal denmark
Palaeogeography Palaeoclimatology Palaeoecology, 2008Co-Authors: Bodil W Lauridsen, Finn SurlykAbstract:Abstract Chalk–Marl cyclicity is a common and characteristic feature of the Upper Cretaceous Chalk Group of NW Europe. The mainly small-sized benthic fauna from an upper Maastrichtian cyclic chalk–Marl succession at Rordal, Denmark, is analysed in order to interpret the composition and structure of the benthic fauna, and to test if faunal density, diversity, composition, and mode of life substrate were controlled by changes in lithology. The material comprises 21 bulk samples weighing 5 kg each, collected from all chalk and Marl half-cycles. The samples were washed and sieved and yielded a total of 25,974 specimens. These are referred to at least 75 species and 15 bryozoan morphotypes and form the basis for the establishment of 10 guilds. The lithological cyclicity is not reflected by the distribution of individual species or by changes in average adult size, but there is generally a higher species diversity and density in chalk compared to Marl. The chalk thus represents times of more favourable living conditions, higher environmental stability, and probably higher nutrient influx. No changes in guilds between Marl and chalk are found. The variation in substrate character between the two lithologies was thus too small to have exerted any significant influence on the mode of life of the benthic fauna in spite of a difference in clay content, averaging 10%. Two different fossil associations are recognized, each covering one to two chalk–Marl cycles and characterized by a distinct fauna with similar ecological requirements. If the chalk–Marl couplets represent precession cycles within the Milankovitch band, then the change from one association to the other and back again may represent the 100,000 years eccentricity cycle based on a calculated sedimentation rate of 6 cm ka − 1 .
Jieyu Lin - One of the best experts on this subject based on the ideXlab platform.
-
efficient communication in multi agent reinforcement learning via variance based control
arXiv: Learning, 2019Co-Authors: Sai Qian Zhang, Qi Zhang, Jieyu LinAbstract:Multi-agent reinforcement learning (Marl) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in Marl. In this work, we propose Variance Based Control (VBC), a simple yet efficient technique to improve communication efficiency in Marl. By limiting the variance of the exchanged messages between agents during the training phase, the noisy component in the messages can be eliminated effectively, while the useful part can be preserved and utilized by the agents for better performance. Our evaluation using a challenging set of StarCraft II benchmarks indicates that our method achieves $2-10\times$ lower in communication overhead than state-of-the-art Marl algorithms, while allowing agents to better collaborate by developing sophisticated strategies.
-
efficient communication in multi agent reinforcement learning via variance based control
Neural Information Processing Systems, 2019Co-Authors: Sai Qian Zhang, Qi Zhang, Jieyu LinAbstract:Multi-agent reinforcement learning (Marl) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in Marl. In this work, we propose Variance Based Control (VBC), a simple yet efficient technique to improve communication efficiency in Marl. By limiting the variance of the exchanged messages between agents during the training phase, the noisy component in the messages can be eliminated effectively, while the useful part can be preserved and utilized by the agents for better performance. Our evaluation using multiple Marl benchmarks indicates that our method achieves $2-10\times$ lower in communication overhead than state-of-the-art Marl algorithms, while allowing agents to achieve better overall performance.
Kaiqing Zhang - One of the best experts on this subject based on the ideXlab platform.
-
finite sample analysis for decentralized batch multi agent reinforcement learning with networked agents
IEEE Transactions on Automatic Control, 2021Co-Authors: Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, Tamer BasarAbstract:Despite the increasing interest in multi-agent reinforcement learning (Marl) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this paper, we address this problem by providing a finite-sample analysis for decentralized batch Marl. Specifically, we consider a type of mixed Marl setting with both cooperative and competitive agents, where two teams of agents compete in a zero-sum game setting, while the agents within each team collaborate by communicating over a time-varying network. This setting covers many conventional Marl settings in the literature. We then develop batch Marl algorithms that can be implemented in a decentralized fashion, and quantify the finite-sample errors of the estimated action-value functions. Our error analysis captures how the function class, the number of samples within each iteration, and the number of iterations determine the statistical accuracy of the proposed algorithms. Our results, compared to the finite-sample bounds for single-agent RL, involve additional error terms caused by decentralized computation, which is inherent in our decentralized Marl setting. This work provides the
-
finite sample analysis for decentralized batch multi agent reinforcement learning with networked agents
arXiv: Learning, 2018Co-Authors: Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, Tamer BasarAbstract:Despite the increasing interest in multi-agent reinforcement learning (Marl) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this work, we address this problem by providing a finite-sample analysis for decentralized batch Marl with networked agents. Specifically, we consider two decentralized Marl settings, where teams of agents are connected by time-varying communication networks, and either collaborate or compete in a zero-sum game setting, without any central controller. These settings cover many conventional Marl settings in the literature. For both settings, we develop batch Marl algorithms that can be implemented in a decentralized fashion, and quantify the finite-sample errors of the estimated action-value functions. Our error analysis captures how the function class, the number of samples within each iteration, and the number of iterations determine the statistical accuracy of the proposed algorithms. Our results, compared to the finite-sample bounds for single-agent RL, involve additional error terms caused by decentralized computation, which is inherent in our decentralized Marl setting. This work appears to be the first finite-sample analysis for batch Marl, a step towards rigorous theoretical understanding of general Marl algorithms in the finite-sample regime.
Finn Surlyk - One of the best experts on this subject based on the ideXlab platform.
-
trace fossils of a cyclic chalk Marl succession the upper maastrichtian rordal member denmark
Cretaceous Research, 2011Co-Authors: Bodil W Lauridsen, Finn Surlyk, R G BromleyAbstract:Abstract Trace fossils from an upper Maastrichtian cyclic chalk–Marl succession, the Rordal Member, exposed in the Rordal quarry, Denmark, are analysed in order to test whether the changes in substrate lithology exerted any control over the ichnodiversity, tiering complexity, and density of the infauna. The cyclicity is interpreted as caused by orbital changes within the Milankovitch frequency band. The carbonate content varies between 71 and 82 weight% in the Marl and 82–92 weight% in the chalk beds. The material is based on 19 samples collected from six chalk and Marl beds. The investigated bedding-normal sample surfaces vary in area between 29 and 155 cm 2 . Eight ichnogenera and two undetermined ichnogenera are recognised. The member is characterised by three ichnofabrics (A, B and C). The ichnofabric analysis is based on texture and internal structure of the sediments resulting from bioturbation. Ichnofabric A is found only in chalk samples and shows a poor preservation of trace fossils, whereas ichnofabric C is found in a few chalk and all Marl samples and comprises a very dense, diverse and well preserved ichnofauna representing a high tiering complexity. Ichnofabric B represents an intermediate situation between ichnofabrics A and C and occurs in chalk samples immediately adjacent to Marl beds. The observed changes in ichnofabrics between chalk and Marl are related to the amount of clay in the samples and the differences in the occurrence of trace fossils are interpreted as due to differences in the visibility of traces between chalk and Marl and not due to differences in ecological stress upon the endobenthic community of the two lithologies. The study thus provides an excellent example of how the effect of taphonomic factors may give a misleading and biased impression of apparent differences in the endobenthic community between chalk and Marl.
-
benthic macrofauna variations and community structure in cenomanian cyclic chalk Marl from southerham grey pit se england
Journal of the Geological Society, 2009Co-Authors: Bodil W Lauridsen, Andrew S Gale, Finn SurlykAbstract:Abstract: Cenomanian chalk–Marl couplets from England represent the 20 ka Milankovitch precession cycle. Fossil communities from both chalk and Marl are identified to test if the orbital fluctuations and the associated changes in substrate lithology and climate exerted any control on the benthic macrofauna. The material comprises washing residues of 24 bulk samples collected from chalk and Marl half-cycles. A total of 5055 invertebrate specimens were retrieved and referred to 68 species, forming the basis for the recognition of six guilds. In general, the fauna is more diverse in Marl than in chalk, but it is remarkable and somewhat surprising that the guild assemblages of Marl and chalk show no difference. Thus the substrate character of the two lithologies did not exert any influence on the modes of life of the benthic fauna in spite of a difference in clay content of about 20%. The fauna was clearly well adapted to both facies and thus to the fine grain size of the substrate rather than to lithology. The systematic difference in diversity between chalk and Marl samples was possibly caused by long-term climatic and oceanographic changes and thus could represent a biological response to Milankovitch cyclicity.
-
benthic faunal response to late maastrichtian chalk Marl cyclicity at rordal denmark
Palaeogeography Palaeoclimatology Palaeoecology, 2008Co-Authors: Bodil W Lauridsen, Finn SurlykAbstract:Abstract Chalk–Marl cyclicity is a common and characteristic feature of the Upper Cretaceous Chalk Group of NW Europe. The mainly small-sized benthic fauna from an upper Maastrichtian cyclic chalk–Marl succession at Rordal, Denmark, is analysed in order to interpret the composition and structure of the benthic fauna, and to test if faunal density, diversity, composition, and mode of life substrate were controlled by changes in lithology. The material comprises 21 bulk samples weighing 5 kg each, collected from all chalk and Marl half-cycles. The samples were washed and sieved and yielded a total of 25,974 specimens. These are referred to at least 75 species and 15 bryozoan morphotypes and form the basis for the establishment of 10 guilds. The lithological cyclicity is not reflected by the distribution of individual species or by changes in average adult size, but there is generally a higher species diversity and density in chalk compared to Marl. The chalk thus represents times of more favourable living conditions, higher environmental stability, and probably higher nutrient influx. No changes in guilds between Marl and chalk are found. The variation in substrate character between the two lithologies was thus too small to have exerted any significant influence on the mode of life of the benthic fauna in spite of a difference in clay content, averaging 10%. Two different fossil associations are recognized, each covering one to two chalk–Marl cycles and characterized by a distinct fauna with similar ecological requirements. If the chalk–Marl couplets represent precession cycles within the Milankovitch band, then the change from one association to the other and back again may represent the 100,000 years eccentricity cycle based on a calculated sedimentation rate of 6 cm ka − 1 .