The Experts below are selected from a list of 8358 Experts worldwide ranked by ideXlab platform
Oscar Vegaamaya - One of the best experts on this subject based on the ideXlab platform.
-
solutions of the average Cost Optimality equation for markov decision processes with weakly continuous kernel the fixed point approach revisited
Journal of Mathematical Analysis and Applications, 2018Co-Authors: Oscar VegaamayaAbstract:Abstract This paper shows the existence of lower semicontinuous solutions of the average Cost Optimality equation for Markov decision processes with Borel spaces, possible unbounded Cost function and weakly continuous transition kernel. This is done imposing a growth condition on the Cost function, a Lyapunov stability condition on the transition kernel and a set of standard compactness-continuity conditions. The solution of the average Cost Optimality equation is obtained by means of the Banach fixed-point theorem.
-
time and ratio expected average Cost Optimality for semi markov control processes on borel spaces
Communications in Statistics-theory and Methods, 2004Co-Authors: Fernando Luquevasquez, Oscar VegaamayaAbstract:We deal with semi-Markov control models with Borel state and control spaces, and unbounded Cost functions under the ratio and the time expected average Cost criteria. Under suitable growth conditions on the Costs and the mean holding times together with stability conditions on the embedded Markov chains, we show the following facts: (i) the ratio and the time average Costs coincide in the class of the stationary policies; (ii) there exists a stationary policy which is optimal for both criteria. Moreover, we provide a generalization of the classical Wald's Lemma to semi-Markov processes. These results are obtained combining the existence of solutions of the average Cost Optimality equation and the Optional Stopping Theorem.
-
sample path average Cost Optimality for semi markov control processes on borel spaces unbounded Costs and mean holding times
Applicationes Mathematicae, 2000Co-Authors: Oscar Vegaamaya, Fernando LuquevasquezAbstract:We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded Cost and mean holding time. Under suitable growth conditions on the Cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average Cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria.
Anna Jaśkiewicz - One of the best experts on this subject based on the ideXlab platform.
-
a fixed point approach to solve the average Cost Optimality equation for semi markov decision processes with feller transition probabilities
Communications in Statistics-theory and Methods, 2007Co-Authors: Anna JaśkiewiczAbstract:Semi-Markov control processes with Feller (also known as weakly continuous) transition probabilities and unbounded Cost functions are considered. We give sufficient conditions for the existence of a lower semicontinuous and continuous solutions to the average Cost Optimality equation. The proof is based on a fixed point argument.
-
on the Optimality equation for average Cost markov control processes with feller transition probabilities
Journal of Mathematical Analysis and Applications, 2006Co-Authors: Anna Jaśkiewicz, Andrzej S NowakAbstract:We consider Markov control processes with Borel state space and Feller transition probabilities, satisfying some generalized geometric ergodicity conditions. We provide a new theorem on the existence of a solution to the average Cost Optimality equation.
-
An approximation approach to ergodic semi-Markov control processes
Mathematical Methods of Operations Research, 2001Co-Authors: Anna JaśkiewiczAbstract:We consider semi-Markov control models (SMCMs) with a Borel state space satisfying certain stochastic stability assumptions on the transition structure which imply the so-called V-uniform geometric ergodicity of the state process. We deal with a class of e-perturbations of transition probability functions of the original model. First, we determine the rate of convergence of the optimal expected Costs in in perturbed models to the optimal expected Cost in the orginal SMCM. Next, we present a new algorithm for finding the solution to the average Cost Optimality equation (ACOE). The algorithm makes use of a sequence of solutions to the ACOE for the perturbed models, which can be found by a simple iterative procedure. Copyright Springer-Verlag Berlin Heidelberg 2001
E. Fernandez-gaucherand - One of the best experts on this subject based on the ideXlab platform.
-
Markov decision processes with risk-sensitive criteria: dynamic programming operators and discounted stochastic games
Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228), 2001Co-Authors: Rolando Cavazos-cadena, E. Fernandez-gaucherandAbstract:We study discrete-time Markov decision processes with denumerable state space and bounded Costs per stage. It is assumed that the decision maker exhibits a constant sensitivity to risk, and that the performance of a control policy is measured by a (long-run) risk-sensitive average Cost criterion. Besides standard continuity-compactness conditions, the basic structural constraint on the decision model is that the transition law satisfies a simultaneous Doeblin condition. Within this framework, the main objective is to study the existence of bounded solutions to the risk-sensitive average Cost Optimality equation. Our main result guarantees a bounded solution to the Optimality equation only if the risk sensitivity coefficient /spl lambda/ is small enough and, via a detailed example, it can be shown that such a conclusion cannot be extended to arbitrary values of /spl lambda/. Our results are in opposition to previous claims in the literature, but agree with recent results obtained via a direct probabilistic analysis. A key analysis tool developed in the paper is the definition of an appropriate operator with contractive properties, analogous to the dynamic programming operator in Bellman's equation, and a family of (value) functions with a discounted stochastic games interpretation.
-
Convex stochastic control problems
[1992] Proceedings of the 31st IEEE Conference on Decision and Control, 1992Co-Authors: E. Fernandez-gaucherand, A. Arapostathis, S.i. MarcusAbstract:The solution of the infinite horizon stochastic control problem under certain criteria, the functional characterization and computation of optimal values and policies, is related to two dynamic programming-like functional equations: the discounted Cost Optimality equation (DCOE) and the average Cost Optimality equation (ACOE). The authors consider what useful properties, shared by large and important problem classes, can be used to show that an ACOE holds, and how these properties can be exploited to aid in the development of tractable algorithmic solutions. They address this issue by concentrating on structured solutions to stochastic control models. By a structured solution is meant a model for which value functions and/or optimal policies have some special dependence on the (initial) state. The focus is on convexity properties of the value function.
Andrzej S Nowak - One of the best experts on this subject based on the ideXlab platform.
-
on the Optimality equation for average Cost markov control processes with feller transition probabilities
Journal of Mathematical Analysis and Applications, 2006Co-Authors: Anna Jaśkiewicz, Andrzej S NowakAbstract:We consider Markov control processes with Borel state space and Feller transition probabilities, satisfying some generalized geometric ergodicity conditions. We provide a new theorem on the existence of a solution to the average Cost Optimality equation.
Rolando Cavazoscadena - One of the best experts on this subject based on the ideXlab platform.
-
solution to the risk sensitive average Cost Optimality equation in a class of markov decision processes with finite state space
Mathematical Methods of Operations Research, 2003Co-Authors: Rolando CavazoscadenaAbstract:This work concerns discrete-time Markov decision processes with finite state space and bounded Costs per stage. The decision maker ranks random Costs via the expectation of the utility function associated to a constant risk sensitivity coefficient, and the performance of a control policy is measured by the corresponding (long-run) risk-sensitive average Cost criterion. The main structural restriction on the system is the following communication assumption: For every pair of states x and y, there exists a policy π, possibly depending on x and y, such that when the system evolves under π starting at x, the probability of reaching y is positive. Within this framework, the paper establishes the existence of solutions to the Optimality equation whenever the constant risk sensitivity coefficient does not exceed certain positive value. Copyright Springer-Verlag Berlin Heidelberg 2003
-
controlled markov chains with risk sensitive criteria average Cost Optimality equations and optimal solutions
Mathematical Methods of Operations Research, 1999Co-Authors: Rolando Cavazoscadena, E FernandezgaucherandAbstract:We study controlled Markov chains with denumerable state space and bounded Costs per stage. A (long-run) risk-sensitive average Cost criterion, associated to an exponential utility function with a constant risk sensitivity coefficient, is used as a performance measure. The main assumption on the probabilistic structure of the model is that the transition law satisfies a simultaneous Doeblin condition. Working within this framework, the main results obtained can be summarized as follows: If the constant risk-sensitivity coefficient is small enough, then an associated Optimality equation has a bounded solution with a constant value for the optimal risk-sensitive average Cost; in addition, under further standard continuity-compactness assumptions, optimal stationary policies are obtained. However, it is also shown that the above conclusions fail to hold, in general, for large enough values of the risk-sensitivity coefficient. Our results therefore disprove previous claims on this topic. Also of importance is the fact that our developments are very much self-contained and employ only basic probabilistic and analysis principles. Copyright Springer-Verlag Berlin Heidelberg 1999
-
a counterexample on the Optimality equation in markov decision chains with the average Cost criterion
Systems & Control Letters, 1991Co-Authors: Rolando CavazoscadenaAbstract:Abstract We consider Markov decision processes with denumerable state space and finite control sets; the performance index of a control policy is a long-run expected average Cost criterion and the Cost function is bounded below. For these models, the existence of average optimal stationary policies was recently established in [11] under very general assumptions. Such a result was obtained via an Optimality inequality. Here, we use a simple example to prove that the conditions in [11] do not imply the existence of a solution to the average Cost Optimality equation.
