The Experts below are selected from a list of 3435 Experts worldwide ranked by ideXlab platform
Urbashi Mitra - One of the best experts on this subject based on the ideXlab platform.
-
On Solving MDPs With Large State Space: Exploitation of Policy Structures and Spectral Properties
IEEE Transactions on Communications, 2019Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:In this paper, a point-to-point network transmission control problem is formulated as a Markov decision process (MDP). Classical dynamic programming techniques such as value iteration, policy iteration, and linear programming can be employed to solve the optimization problem, but they suffer from high-computational complexity in networks with large state space. To achieve complexity reduction, the structure of the optimal policy can be exploited and incorporated into standard algorithms. In addition, function approximation can also be applied, where the value function is approximated by the linear combination of some basis vectors in a lower dimensional Subspace. The main challenge for function approximation lies in the absence of general guidelines for Subspace construction. In this paper, a Proper Subspace for projection is first generated based on system information, and more general construction methods are proposed using tools from graph signal processing (GSP). Graph symmetrization methods are also used to tackle the directed nature of the probability transition graph so that the well-developed GSP theory for undirected graphs can be employed. The numerical results for a typical wireless system show that standard algorithms with structural information incorporated can achieve 50% complexity reduction without performance loss. The Subspace generated from the system can achieve zero policy error with faster runtime, and the GSP approach can also provide a Proper Subspace for perfect reconstruction of the optimal policy. It is also shown that how the proposed method can be applied to other MDP problems.
-
on exploiting spectral Properties for solving mdp with large state space
Allerton Conference on Communication Control and Computing, 2017Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:A large number of systems are well-modeled by Markov Decision Processes (MDPs). In particular, certain wireless communication networks and biological networks admit such models. Herein, moderate complexity strategies are proposed for computing the optimal policy for a large state space with long run discounted cost MDP, by exploiting spectral Properties of the probability transition matrices (PTM). Methods such as value iteration and policy iteration for such problems are computationally prohibitive for large state spaces. Reduced dimensional policy iteration can be achieved by projecting the value function on a Proper Subspace. However there is no clear method for determining the optimal Subspace. To this end, Graph signal processing methods have the potential to provide a solution. In order to use spectral techniques, an appropriate positive semi-definite (PSD) matrix is generated from the PTM and the single stage cost vector. Low complexity computation of the value function is enabled by the bases of this dominant Subspace. The proposed projections are combined with policy iteration to find the optimal policy. Finally, numerical results on a wireless system are provided to highlight the performances and trade-offs of these various algorithms, and we found that direct spectral decomposition of outer product of PTM gives us best performance in general.
-
Allerton - On exploiting spectral Properties for solving MDP with large state space
2017 55th Annual Allerton Conference on Communication Control and Computing (Allerton), 2017Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:A large number of systems are well-modeled by Markov Decision Processes (MDPs). In particular, certain wireless communication networks and biological networks admit such models. Herein, moderate complexity strategies are proposed for computing the optimal policy for a large state space with long run discounted cost MDP, by exploiting spectral Properties of the probability transition matrices (PTM). Methods such as value iteration and policy iteration for such problems are computationally prohibitive for large state spaces. Reduced dimensional policy iteration can be achieved by projecting the value function on a Proper Subspace. However there is no clear method for determining the optimal Subspace. To this end, Graph signal processing methods have the potential to provide a solution. In order to use spectral techniques, an appropriate positive semi-definite (PSD) matrix is generated from the PTM and the single stage cost vector. Low complexity computation of the value function is enabled by the bases of this dominant Subspace. The proposed projections are combined with policy iteration to find the optimal policy. Finally, numerical results on a wireless system are provided to highlight the performances and trade-offs of these various algorithms, and we found that direct spectral decomposition of outer product of PTM gives us best performance in general.
Paolo Facchi - One of the best experts on this subject based on the ideXlab platform.
-
Hamiltonian Purification
Journal of Mathematical Physics, 2015Co-Authors: Davide Orsucci, Paolo Facchi, Saverio Pascazio, Daniel Burgarth, Hiromichi Nakazato, Kazuya Yuasa, Vittorio GiovannettiAbstract:The problem of Hamiltonian purification introduced by Burgarth et al. [Nat. Commun. 5, 5173 (2014)] is formalized and discussed. Specifically, given a set of non-commuting Hamiltonians {h1, ?, hm} operating on a d-dimensional quantum system ?d, the problem consists in identifying a set of commuting Hamiltonians {H1, ?, Hm} operating on a larger dE-dimensional system ?dE which embeds ?d as a Proper Subspace, such that hj = PHjP with P being the projection which allows one to recover ?d from ?dE. The notions of spanning-set purification and generator purification of an algebra are also introduced and optimal solutions for ?(d) are providedpublishersversionPeer reviewe
-
The classical limit of the quantum Zeno effect
Journal of Physics A: Mathematical and Theoretical, 2009Co-Authors: Paolo Facchi, Sandro Graffi, Marilena LigabòAbstract:The evolution of a quantum system subjected to infinitely many measurements in a finite time interval is confined in a Proper Subspace of the Hilbert space. This phenomenon is called 'quantum Zeno effect': a particle under intensive observation which does not evolve. This effect is at variance with the classical evolution, which obviously is not affected by any observations. By a semiclassical analysis, we will show that the quantum Zeno effect vanishes at all orders, when the Planck constant tends to zero, and thus it is a purely quantum phenomenon without classical analog, at the same level of tunneling.
-
Quantum Zeno Subspaces and Dynamical Superselection Rules
The Physics of Communication, 2003Co-Authors: Paolo Facchi, Saverio PascazioAbstract:The quantum Zeno evolution of a quantum system takes place in a Proper Subspace of the total Hilbert space. The physical and mathematical features of the “Zeno Subspaces” depend on the measuring apparatus: when this is included in the quantum description, the Zeno effect becomes a mere consequence of the dynamics and, remarkably, can be cast in terms of an adiabatic theorem, with a dynamical superselection rule. We look at several examples and focus on quantum computation and decoherence-free Subspaces.
-
Quantum Zeno effect, adiabaticity and dynamical superselection rules
Fundamental Aspects of Quantum Physics, 2003Co-Authors: Paolo FacchiAbstract:The evolution of a quantum system undergoing very frequent measurements takes place in a Proper Subspace of the total Hilbert space (quantum Zeno effect). When the measuring apparatus is included in the quantum description, the Zeno effect becomes a pure consequence of the dynamics. We show that for continuous measurement processes the quantum Zeno evolution derives from an adiabatic theorem. The system is forced to evolve in a set of orthogonal Subspaces of the total Hilbert space and a dynamical superselection rule arises. The dynamical Properties of this evolution are investigated and several examples are considered.
Libin Liu - One of the best experts on this subject based on the ideXlab platform.
-
On Solving MDPs With Large State Space: Exploitation of Policy Structures and Spectral Properties
IEEE Transactions on Communications, 2019Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:In this paper, a point-to-point network transmission control problem is formulated as a Markov decision process (MDP). Classical dynamic programming techniques such as value iteration, policy iteration, and linear programming can be employed to solve the optimization problem, but they suffer from high-computational complexity in networks with large state space. To achieve complexity reduction, the structure of the optimal policy can be exploited and incorporated into standard algorithms. In addition, function approximation can also be applied, where the value function is approximated by the linear combination of some basis vectors in a lower dimensional Subspace. The main challenge for function approximation lies in the absence of general guidelines for Subspace construction. In this paper, a Proper Subspace for projection is first generated based on system information, and more general construction methods are proposed using tools from graph signal processing (GSP). Graph symmetrization methods are also used to tackle the directed nature of the probability transition graph so that the well-developed GSP theory for undirected graphs can be employed. The numerical results for a typical wireless system show that standard algorithms with structural information incorporated can achieve 50% complexity reduction without performance loss. The Subspace generated from the system can achieve zero policy error with faster runtime, and the GSP approach can also provide a Proper Subspace for perfect reconstruction of the optimal policy. It is also shown that how the proposed method can be applied to other MDP problems.
-
on exploiting spectral Properties for solving mdp with large state space
Allerton Conference on Communication Control and Computing, 2017Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:A large number of systems are well-modeled by Markov Decision Processes (MDPs). In particular, certain wireless communication networks and biological networks admit such models. Herein, moderate complexity strategies are proposed for computing the optimal policy for a large state space with long run discounted cost MDP, by exploiting spectral Properties of the probability transition matrices (PTM). Methods such as value iteration and policy iteration for such problems are computationally prohibitive for large state spaces. Reduced dimensional policy iteration can be achieved by projecting the value function on a Proper Subspace. However there is no clear method for determining the optimal Subspace. To this end, Graph signal processing methods have the potential to provide a solution. In order to use spectral techniques, an appropriate positive semi-definite (PSD) matrix is generated from the PTM and the single stage cost vector. Low complexity computation of the value function is enabled by the bases of this dominant Subspace. The proposed projections are combined with policy iteration to find the optimal policy. Finally, numerical results on a wireless system are provided to highlight the performances and trade-offs of these various algorithms, and we found that direct spectral decomposition of outer product of PTM gives us best performance in general.
-
Allerton - On exploiting spectral Properties for solving MDP with large state space
2017 55th Annual Allerton Conference on Communication Control and Computing (Allerton), 2017Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:A large number of systems are well-modeled by Markov Decision Processes (MDPs). In particular, certain wireless communication networks and biological networks admit such models. Herein, moderate complexity strategies are proposed for computing the optimal policy for a large state space with long run discounted cost MDP, by exploiting spectral Properties of the probability transition matrices (PTM). Methods such as value iteration and policy iteration for such problems are computationally prohibitive for large state spaces. Reduced dimensional policy iteration can be achieved by projecting the value function on a Proper Subspace. However there is no clear method for determining the optimal Subspace. To this end, Graph signal processing methods have the potential to provide a solution. In order to use spectral techniques, an appropriate positive semi-definite (PSD) matrix is generated from the PTM and the single stage cost vector. Low complexity computation of the value function is enabled by the bases of this dominant Subspace. The proposed projections are combined with policy iteration to find the optimal policy. Finally, numerical results on a wireless system are provided to highlight the performances and trade-offs of these various algorithms, and we found that direct spectral decomposition of outer product of PTM gives us best performance in general.
Mohammed El Aidi - One of the best experts on this subject based on the ideXlab platform.
-
On the Riesz-means of negative eigenvalues for a fractional Schrödinger operator
Integral Transforms and Special Functions, 2016Co-Authors: Mohammed El AidiAbstract:ABSTRACTWe furnish an upper bound for the moment of negative eigenvalues for a relativistic Schrodinger operator defined in a Proper Subspace of square-integrable functions.
-
CLR-type inequality on a suitable smooth manifold
Ricerche di Matematica, 2016Co-Authors: Mohammed El AidiAbstract:For a real potential belonging to a Proper Subspace of the set of integrable functions, we provide a new upper bound of the number of negative eigenvalues of a Schrodinger operator defined in a suitable Riemannian manifold.
-
on a new embedding theorem and the clr type inequality for euclidean and hyperbolic spaces
Bulletin Des Sciences Mathematiques, 2014Co-Authors: Mohammed El AidiAbstract:Abstract The goal of this note is to provide a new embedding theorem and to derive from this embedding the CLR-type inequality for a potential belonging to a Proper Subspace of integrable functions.
Arpan Chattopadhyay - One of the best experts on this subject based on the ideXlab platform.
-
On Solving MDPs With Large State Space: Exploitation of Policy Structures and Spectral Properties
IEEE Transactions on Communications, 2019Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:In this paper, a point-to-point network transmission control problem is formulated as a Markov decision process (MDP). Classical dynamic programming techniques such as value iteration, policy iteration, and linear programming can be employed to solve the optimization problem, but they suffer from high-computational complexity in networks with large state space. To achieve complexity reduction, the structure of the optimal policy can be exploited and incorporated into standard algorithms. In addition, function approximation can also be applied, where the value function is approximated by the linear combination of some basis vectors in a lower dimensional Subspace. The main challenge for function approximation lies in the absence of general guidelines for Subspace construction. In this paper, a Proper Subspace for projection is first generated based on system information, and more general construction methods are proposed using tools from graph signal processing (GSP). Graph symmetrization methods are also used to tackle the directed nature of the probability transition graph so that the well-developed GSP theory for undirected graphs can be employed. The numerical results for a typical wireless system show that standard algorithms with structural information incorporated can achieve 50% complexity reduction without performance loss. The Subspace generated from the system can achieve zero policy error with faster runtime, and the GSP approach can also provide a Proper Subspace for perfect reconstruction of the optimal policy. It is also shown that how the proposed method can be applied to other MDP problems.
-
on exploiting spectral Properties for solving mdp with large state space
Allerton Conference on Communication Control and Computing, 2017Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:A large number of systems are well-modeled by Markov Decision Processes (MDPs). In particular, certain wireless communication networks and biological networks admit such models. Herein, moderate complexity strategies are proposed for computing the optimal policy for a large state space with long run discounted cost MDP, by exploiting spectral Properties of the probability transition matrices (PTM). Methods such as value iteration and policy iteration for such problems are computationally prohibitive for large state spaces. Reduced dimensional policy iteration can be achieved by projecting the value function on a Proper Subspace. However there is no clear method for determining the optimal Subspace. To this end, Graph signal processing methods have the potential to provide a solution. In order to use spectral techniques, an appropriate positive semi-definite (PSD) matrix is generated from the PTM and the single stage cost vector. Low complexity computation of the value function is enabled by the bases of this dominant Subspace. The proposed projections are combined with policy iteration to find the optimal policy. Finally, numerical results on a wireless system are provided to highlight the performances and trade-offs of these various algorithms, and we found that direct spectral decomposition of outer product of PTM gives us best performance in general.
-
Allerton - On exploiting spectral Properties for solving MDP with large state space
2017 55th Annual Allerton Conference on Communication Control and Computing (Allerton), 2017Co-Authors: Libin Liu, Arpan Chattopadhyay, Urbashi MitraAbstract:A large number of systems are well-modeled by Markov Decision Processes (MDPs). In particular, certain wireless communication networks and biological networks admit such models. Herein, moderate complexity strategies are proposed for computing the optimal policy for a large state space with long run discounted cost MDP, by exploiting spectral Properties of the probability transition matrices (PTM). Methods such as value iteration and policy iteration for such problems are computationally prohibitive for large state spaces. Reduced dimensional policy iteration can be achieved by projecting the value function on a Proper Subspace. However there is no clear method for determining the optimal Subspace. To this end, Graph signal processing methods have the potential to provide a solution. In order to use spectral techniques, an appropriate positive semi-definite (PSD) matrix is generated from the PTM and the single stage cost vector. Low complexity computation of the value function is enabled by the bases of this dominant Subspace. The proposed projections are combined with policy iteration to find the optimal policy. Finally, numerical results on a wireless system are provided to highlight the performances and trade-offs of these various algorithms, and we found that direct spectral decomposition of outer product of PTM gives us best performance in general.
