Reinforcement Learning for Optimizing Investment Portfolios in Robo-Advisory Platforms
DOI:
https://doi.org/10.63282/3117-5481/AIJCST-V5I4P103Keywords:
Reinforcement Learning, Robo-Advisory Platforms, Portfolio Optimization, Algorithmic Trading, Deep Reinforcement Learning, Risk-Adjusted Returns, Financial Decision-Making, Dynamic Asset Allocation, Proximal Policy Optimization (PPO), Deep Q-Networks (DQN), AI in FintechAbstract
Artificial Intelligence has quickly revolutionized fintech, but perhaps most notably in automated investment management. Automated advisory services that use algorithms for financial planning while reducing or eliminating human intervention have gained popularity over the years amongst both retail and institutional investors. But conventional approaches to portfolio construction like Modern Portfolio Theory MPT fail to adjust for a rapidly changing world with a flux of constantly changing investor needs. To resolve the aforementioned limitations, this paper studies machine learning, specifically reinforcement learning, which are “teach” the agent to find the best possible actions by utilizing the environment the agent interacts with in order to improve portfolio optimization on robo-advisors. We consider the investment process as a problem of sequential decisions, in which an RL agent allocates the assets at different points in time in order to improve returns and control the risk. Utilizing cutting edge RL algorithms like DQN and PPO, we show in simulations that RL models can achieve superior risk adjusted returns compared to traditional models while also being more adept at handling periods of market volatility. The results point to the applicability of reinforcement learning in developing intelligent, adaptive and personalized robo-advisors as a new paradigm of AI in finance.
References
[1] Jiang, Z., Xu, D., & Liang, J. (2017). A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059.
[2] Moody, J., & Saffell, M. (2001). Learning to trade via direct reinforcement. IEEE Transactions on Neural Networks, 12(4), 875–889. https://doi.org/10.1109/72.935097
[3] Li, Y., & Hoi, S. C. H. (2014). Online portfolio selection: A survey. ACM Computing Surveys, 46(3), 1–36. https://doi.org/10.1145/2512962
[4] Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2016). Deep direct reinforcement learning for financial signal representation and trading. IEEE Transactions on Neural Networks and Learning Systems, 28(3), 653–664. https://doi.org/10.1109/TNNLS.2016.2522401
[5] Liang, Z., Chen, H., Zhu, J., Jiang, K., & Li, Y. (2018). Adversarial deep reinforcement learning in portfolio management. arXiv preprint arXiv:1808.09940.
[6] Almahdi, S., & Yang, S. Y. (2017). An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown. Expert Systems with Applications, 87, 267–279. https://doi.org/10.1016/j.eswa.2017.06.023
[7] Lee, D., & O, J. (2012). A reinforcement learning approach to portfolio management. Journal of Applied Mathematics, 2012, 1–16. https://doi.org/10.1155/2012/160247
[8] Corazza, M., & Sangalli, I. (2020). Portfolio optimization with reinforcement learning. Journal of Economic Dynamics and Control, 113, 103855. https://doi.org/10.1016/j.jedc.2020.103855
[9] Xiong, Z., Liu, X., Zhong, S., Yang, H., & Walid, A. (2018). Practical deep reinforcement learning approach for stock trading. arXiv preprint arXiv:1811.07522.
[10] Wang, Z., Liu, Y., & Li, Z. (2020). Deep reinforcement learning for dynamic portfolio optimization. Neurocomputing, 385, 159–171. https://doi.org/10.1016/j.neucom.2019.12.002
[11] Bertoluzzo, F., & Corazza, M. (2012). Testing different reinforcement learning configurations for financial trading. Journal of Economic Dynamics and Control, 36(10), 1494–1506. https://doi.org/10.1016/j.jedc.2012.04.007
[12] Kolm, P. N., Tütüncü, R., & Fabozzi, F. J. (2014). 60 years of portfolio optimization: Practical challenges and current trends. European Journal of Operational Research, 234(2), 356–371. https://doi.org/10.1016/j.ejor.2013.10.060
[13] Nevmyvaka, Y., Feng, Y., & Kearns, M. (2006). Reinforcement learning for optimized trade execution. Proceedings of the 23rd International Conference on Machine Learning, 673–680. https://doi.org/10.1145/1143844.1143929
[14] Yu, P., Yan, X., & Zhang, C. (2019). A deep reinforcement learning framework for the financial portfolio management problem. IEEE Access, 7, 129789–129799. https://doi.org/10.1109/ACCESS.2019.2940221
[15] Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.
