The Experts below are selected from a list of 153 Experts worldwide ranked by ideXlab platform

Olivier Pietquin - One of the best experts on this subject based on the ideXlab platform.

  • An algorithmic Survey of Parametric Value Function Approximation
    IEEE Transactions on Neural Networks and Learning Systems, 2013
    Co-Authors: Matthieu Geist, Olivier Pietquin
    Abstract:

    Reinforcement learning is a machine learning answer to the optimal control problem. It consists in learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called Value function. A recurrent subtopic of reinforcement learning is to compute an approximation of this Value function when the system is too large for an exact representation. This survey reviews state-of-the-art methods for (Parametric) Value function approximation by grouping them into three main categories: bootstrapping, residual and projected fixed-point approaches. Related algorithms are derived by considering one of the associated cost functions and a specific minimization method, generally a stochastic gradient descent or a recursive least-squares approach.

  • Algorithmic Survey of Parametric Value Function Approximation
    IEEE Transactions on Neural Networks and Learning Systems, 2013
    Co-Authors: Matthieu Geist, Olivier Pietquin
    Abstract:

    Reinforcement learning (RL) is a machine learning answer to the optimal control problem. It consists of learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called Value function. A recurrent subtopic of RL concerns computing an approximation of this Value function when the system is too large for an exact representation. This survey reviews state-of-the-art methods for (Parametric) Value function approximation by grouping them into three main categories: bootstrapping, residual, and projected fixed-point approaches. Related algorithms are derived by considering one of the associated cost functions and a specific minimization method, generally a stochastic gradient descent or a recursive least-squares approach.

  • Parametric Value function approximation: A unified view
    2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 2011
    Co-Authors: Matthieu Geist, Olivier Pietquin
    Abstract:

    Reinforcement learning (RL) is a machine learning answer to the optimal control problem. It consists of learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called Value function. An important RL subtopic is to approximate this function when the system is too large for an exact representation. This survey reviews and unifies state of the art methods for Parametric Value function approximation by grouping them into three main categories: bootstrapping, residuals and projected fixed-point approaches. Related algorithms are derived by considering one of the associated cost functions and a specific way to minimize it, almost always a stochastic gradient descent or a recursive least-squares approach.

  • ADPRL - Parametric Value function approximation: A unified view
    2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 2011
    Co-Authors: Matthieu Geist, Olivier Pietquin
    Abstract:

    Reinforcement learning (RL) is a machine learning answer to the optimal control problem. It consists of learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called Value function. An important RL subtopic is to approximate this function when the system is too large for an exact representation. This survey reviews and unifies state of the art methods for Parametric Value function approximation by grouping them into three main categories: bootstrapping, residuals and projected fixed-point approaches. Related algorithms are derived by considering one of the associated cost functions and a specific way to minimize it, almost always a stochastic gradient descent or a recursive least-squares approach.

Matthieu Geist - One of the best experts on this subject based on the ideXlab platform.

  • An algorithmic Survey of Parametric Value Function Approximation
    IEEE Transactions on Neural Networks and Learning Systems, 2013
    Co-Authors: Matthieu Geist, Olivier Pietquin
    Abstract:

    Reinforcement learning is a machine learning answer to the optimal control problem. It consists in learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called Value function. A recurrent subtopic of reinforcement learning is to compute an approximation of this Value function when the system is too large for an exact representation. This survey reviews state-of-the-art methods for (Parametric) Value function approximation by grouping them into three main categories: bootstrapping, residual and projected fixed-point approaches. Related algorithms are derived by considering one of the associated cost functions and a specific minimization method, generally a stochastic gradient descent or a recursive least-squares approach.

  • Algorithmic Survey of Parametric Value Function Approximation
    IEEE Transactions on Neural Networks and Learning Systems, 2013
    Co-Authors: Matthieu Geist, Olivier Pietquin
    Abstract:

    Reinforcement learning (RL) is a machine learning answer to the optimal control problem. It consists of learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called Value function. A recurrent subtopic of RL concerns computing an approximation of this Value function when the system is too large for an exact representation. This survey reviews state-of-the-art methods for (Parametric) Value function approximation by grouping them into three main categories: bootstrapping, residual, and projected fixed-point approaches. Related algorithms are derived by considering one of the associated cost functions and a specific minimization method, generally a stochastic gradient descent or a recursive least-squares approach.

  • Parametric Value function approximation: A unified view
    2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 2011
    Co-Authors: Matthieu Geist, Olivier Pietquin
    Abstract:

    Reinforcement learning (RL) is a machine learning answer to the optimal control problem. It consists of learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called Value function. An important RL subtopic is to approximate this function when the system is too large for an exact representation. This survey reviews and unifies state of the art methods for Parametric Value function approximation by grouping them into three main categories: bootstrapping, residuals and projected fixed-point approaches. Related algorithms are derived by considering one of the associated cost functions and a specific way to minimize it, almost always a stochastic gradient descent or a recursive least-squares approach.

  • ADPRL - Parametric Value function approximation: A unified view
    2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 2011
    Co-Authors: Matthieu Geist, Olivier Pietquin
    Abstract:

    Reinforcement learning (RL) is a machine learning answer to the optimal control problem. It consists of learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called Value function. An important RL subtopic is to approximate this function when the system is too large for an exact representation. This survey reviews and unifies state of the art methods for Parametric Value function approximation by grouping them into three main categories: bootstrapping, residuals and projected fixed-point approaches. Related algorithms are derived by considering one of the associated cost functions and a specific way to minimize it, almost always a stochastic gradient descent or a recursive least-squares approach.

Jose Olmo - One of the best experts on this subject based on the ideXlab platform.

  • backtesting Parametric Value at risk with estimation risk
    Journal of Business & Economic Statistics, 2010
    Co-Authors: Juan Carlos Escanciano, Jose Olmo
    Abstract:

    One of the implications of the creation of the Basel Committee on Banking Supervision was the implementation of Value-at-Risk (VaR) as the standard tool for measuring market risk. Since then, the capital requirements of commercial banks with trading activities are based on VaR estimates. Therefore, appropriately constructed tests for assessing the out-of-sample forecast accuracy of the VaR model (backtesting procedures) have become of crucial practical importance. In this article we show that the use of the standard unconditional and independence backtesting procedures to assess VaR models in out-of-sample composite environments can be misleading. These tests do not consider the impact of estimation risk, and therefore, may use wrong critical Values to assess market risk. The purpose of this article is to quantify such estimation risk in a very general class of dynamic Parametric VaR models and to correct standard backtesting procedures to provide valid inference in out-of-sample analyses. A Monte Carlo s...

  • Specification tests in Parametric Value-at-risk models
    2009
    Co-Authors: Juan Carlos Escanciano, Jose Olmo
    Abstract:

    One of the implications of the creation of Basel Committee on Banking Supervision was the implementation of Value-at-Risk (VaR) as the standard tool for measuring market risk and of out-of-sample backtesting for banking risk monitoring. We stress in this article that the results derived from this exercise can be spurious if one does not carry out a previous in-sample specification test to determine the adequacy of the VaR model. We study in this paper specification tests that, unlike the existing ones, are able to control the type-I error probability. More concretely, we show that not taking into account the effect of estimating the parameters of the VaR model in the in-sample specification tests can lead to invalid inferences, which in turn may imply wrong conclusions about the out-of-sample backtesting procedures. The first aim of this article is to quantify the effect of estimating the parameters of the model and to stress its impact in specification tests, and the second is then to propose a corrected method taking into account such risk, and thereby to provide a valid econometric framework for measuring and evaluating market risk. The results are given for general dynamic Parametric models and illustrated with a Monte-Carlo simulation for location-scale models and with an empirical application for S&P500 Index.

  • estimation risk effects on backtesting for Parametric Value at risk models
    2007
    Co-Authors: Juan Carlos Escanciano, Jose Olmo
    Abstract:

    One of the implications of the creation of Basel Committee on Banking Supervision was the implementation of Value-at-Risk (VaR) as the standard tool for measuring market risk. Thereby the correct specification of Parametric VaR models became of crucial importance in order to provide accurate and reliable risk measures. If the underlying risk model is not correctly specified, VaR estimates understate/overstate risk exposure. This can have dramatic consequences on stability and reputation of financial institutions or lead to sub-optimal capital allocation. We show that the use of the standard unconditional backtesting procedures to assess VaR models is completely misleading. These tests do not consider the impact of estimation risk and therefore use wrong critical Values to assess market risk. The purpose of this paper is to quantify such estimation risk in a very general class of dynamic Parametric VaR models and to correct standard backtesting procedures to provide valid inference in specification analyses. A Monte Carlo study illustrates our theoretical findings in finite-samples. Finally, an application to S&P500 Index shows the importance of this correction and its impact on capital requirements as imposed by Basel Accord, and on the choice of dynamic Parametric models for risk management.

Juan Carlos Escanciano - One of the best experts on this subject based on the ideXlab platform.

  • backtesting Parametric Value at risk with estimation risk
    Journal of Business & Economic Statistics, 2010
    Co-Authors: Juan Carlos Escanciano, Jose Olmo
    Abstract:

    One of the implications of the creation of the Basel Committee on Banking Supervision was the implementation of Value-at-Risk (VaR) as the standard tool for measuring market risk. Since then, the capital requirements of commercial banks with trading activities are based on VaR estimates. Therefore, appropriately constructed tests for assessing the out-of-sample forecast accuracy of the VaR model (backtesting procedures) have become of crucial practical importance. In this article we show that the use of the standard unconditional and independence backtesting procedures to assess VaR models in out-of-sample composite environments can be misleading. These tests do not consider the impact of estimation risk, and therefore, may use wrong critical Values to assess market risk. The purpose of this article is to quantify such estimation risk in a very general class of dynamic Parametric VaR models and to correct standard backtesting procedures to provide valid inference in out-of-sample analyses. A Monte Carlo s...

  • Specification tests in Parametric Value-at-risk models
    2009
    Co-Authors: Juan Carlos Escanciano, Jose Olmo
    Abstract:

    One of the implications of the creation of Basel Committee on Banking Supervision was the implementation of Value-at-Risk (VaR) as the standard tool for measuring market risk and of out-of-sample backtesting for banking risk monitoring. We stress in this article that the results derived from this exercise can be spurious if one does not carry out a previous in-sample specification test to determine the adequacy of the VaR model. We study in this paper specification tests that, unlike the existing ones, are able to control the type-I error probability. More concretely, we show that not taking into account the effect of estimating the parameters of the VaR model in the in-sample specification tests can lead to invalid inferences, which in turn may imply wrong conclusions about the out-of-sample backtesting procedures. The first aim of this article is to quantify the effect of estimating the parameters of the model and to stress its impact in specification tests, and the second is then to propose a corrected method taking into account such risk, and thereby to provide a valid econometric framework for measuring and evaluating market risk. The results are given for general dynamic Parametric models and illustrated with a Monte-Carlo simulation for location-scale models and with an empirical application for S&P500 Index.

  • estimation risk effects on backtesting for Parametric Value at risk models
    2007
    Co-Authors: Juan Carlos Escanciano, Jose Olmo
    Abstract:

    One of the implications of the creation of Basel Committee on Banking Supervision was the implementation of Value-at-Risk (VaR) as the standard tool for measuring market risk. Thereby the correct specification of Parametric VaR models became of crucial importance in order to provide accurate and reliable risk measures. If the underlying risk model is not correctly specified, VaR estimates understate/overstate risk exposure. This can have dramatic consequences on stability and reputation of financial institutions or lead to sub-optimal capital allocation. We show that the use of the standard unconditional backtesting procedures to assess VaR models is completely misleading. These tests do not consider the impact of estimation risk and therefore use wrong critical Values to assess market risk. The purpose of this paper is to quantify such estimation risk in a very general class of dynamic Parametric VaR models and to correct standard backtesting procedures to provide valid inference in specification analyses. A Monte Carlo study illustrates our theoretical findings in finite-samples. Finally, an application to S&P500 Index shows the importance of this correction and its impact on capital requirements as imposed by Basel Accord, and on the choice of dynamic Parametric models for risk management.

Åśaunak Sen - One of the best experts on this subject based on the ideXlab platform.

  • Kernel-based reinforcement learning
    Machine Learning, 2002
    Co-Authors: Dirk Ormoneit, Åśaunak Sen
    Abstract:

    We present a kernel-based approach to reinforcement learning that overcomes the stability problems of temporal-difference learning in continuous state-spaces. First, our algorithm converges to a unique solution of an approximate Bellman's equation regardless of its initialization Values. Second, the method is consistent in the sense that the resulting policy converges asymptotically to the optimal policy. Parametric Value function estimates such as neural networks do not possess this property. Our kernel-based approach also allows us to show that the limiting distribution of the Value function estimate is a Gaussian process. This information is useful in studying the bias-variance tradeoff in reinforcement learning. We find that all reinforcement learning approaches to estimating the Value function, Parametric or non-Parametric, are subject to a bias. This bias is typically larger in reinforcement learning than in a comparable regression problem.