Risk-Constrained Forward Trading Optimization through Stochastic Approximate Dynamic Programming

GIL-PUGLIESE, M.; OLSINA, F.

Dynamic Programming and Bayesian Inference, Concepts and Applications

IN-TECH

Lugar: Rijeka; Año: 2014; p. 91 - 124

Since the mid-20th century the Dynamic Programming (DP) algorithms proved as a flexible and powerful technique to address optimal decisions problems. Nevertheless, a drawback of conventional DP methods is the requirement of exploring the whole state space in order to find a solution. The immense amount of mathematical operations involved to solve complex high dimensional problems using DP limited their use to small and or highly simplified real problems. Indeed, in the case of multivariate optimization problems, state space grows exponentially with the number of variables. The curses of dimensionality are a well-known limitation of conventional DP algorithms for tackling large-scale problems ubiquitous in real science and engineering applications. In the last decades and trying to overcome the inherent limitations of DP, many new algorithms of Approximate Dynamic Programming (ADP) emerged in different branches of science. The ADP algorithms do not enumerate and calculate every possible state of a system during the optimization process as DP algorithms do. Instead, they perform approximations of relevant features over the state space, which are iteratively improved by means Monte Carlo simulation methods and statistical regression techniques. These techniques allow the ADP algorithms to overcome the dimensionality limitations of the conventional DP while retaining many of its benefits. In this chapter is considered the applications of stochastic approximate dynamic programming techniques to the problem of the optimization of the forward sell strategy of a power generator subjected to deliver risk, which is allowed to rebalance the portfolio during the period of analysis. In electricity markets, a power generator can sell in advance part or all its future energy production at a fixed price, hedging against the high price volatility of the spot market. Intuitively, a strategy by which a generator entirely sells all its energy production in the forward market leads to the minimum variance of profits. Nonetheless, it can be proven that this is not the case for most generators. Unplanned outages of the generation units and transmission links, as well as shortages in the primary energy supply introduce delivery risk into forward contracts. Delivery risk considerably modifies the probability distribution of profits, shifting the minimal-variance strategy to a portfolio mixing forward contracting and selling in the spot market. Due to the size of the probability state space and the current computing technology, the determination of an optimal trading strategy is matter of current study without a closed form solution. Despite that, the increase in computing power and recent developments in Operational Research give new insights into the path to the proper solution of such problems. In the past decades and by virtue of the always increasing computational power, many methods appeared in different scientific fields with several different names: Reinforced Learning, Q-Learning, Neuro-Dynamic Programming, etc. All these methods were later brought together in what is known as Approximated Dynamic Programming. Although ADP algorithms are being used in several other fields of science, the use to find optimal trading strategies in power markets has not been used before. In this work, ADP techniques are used to optimize the portfolio strategy of a generator working in a frictional market with transaction costs, considering two available markets (spot and yearly forward contracting), maximizing the expected profit while limiting financial risk associated to delivery failure and price volatility. The model considers a 12-stage decision scheme. At each monthly decision stage, the current trading position can be changed at a cost in order to rebalance the portfolio. The consequences that a present decision has on future decisions and the associated cost of these decisions along the optimization horizon are taken into account. The model considers a perfectly competitive two-settlement market. The stochastic nature of the electricity prices on spot and futures markets are modeled using a spectral representation algorithm. The stochastic availability of the generator unit is properly described through a 4-state Markovian chronological model. The objective function is formulated as the cumulated profit over all stages of the optimization period, considering revenues in the forward position and spot markets as well as fuel and transaction costs. The problem is subjected to generation capacity and maximum acceptable risk of monetary losses. For the risk constraint, a high moment risk metric is used to approximate the conditional value at risk (CVaR), which is a standard downside risk metric in the financial industry. The implemented ADP algorithm is validated against a conventional DP algorithm for a simplified case and then used to solve a complete model of decision. The generality of the developed approach allow the further application without modifications to the optimal future trading of other commodities such as oil, gas, metals, crops, etc.