Risk-sensitive Reinforcement Learning for Portfolio Allocation

Abstract

This study explores the application of risk-sensitive Reinforcement Learning (RL) in portfolio optimization, aiming to integrate asset pricing and portfolio construction into a unified, end-to-end RL framework. While RL has shown promise in various domains, its traditional risk-neutral approach is unsuitable for financial contexts where risk sensitivity is crucial. This research focuses on risk-sensitive RL methods that incorporate different risk measures to manage uncertainty and volatility in financial markets better. The project extends existing RL techniques by adapting the cross-sectional approach to risk-sensitive settings and introducing new variants like PPO-CVaR and PPO-Expectile for portfolio management. A comparative study of these methods is conducted to assess their performance with real market data and simulated environments. The research addresses key questions related to how different risk measures impact learned portfolio strategies, the influence of risk appetite on decision-making, and the performance gap between simulated and real market data. The findings aim to provide insights for practitioners looking to implement risk-sensitive RL in financial asset management.