Deep Reinforcement Learning-Driven Optimization of Multi-Agent Cyber-Physical Systems for Autonomous Decision Making
DOI:
https://doi.org/10.63282/3117-5481/AIJCST-V3I4P101Keywords:
Deep Reinforcement Learning, Multi-Agent Systems, Cyber-Physical Systems, Autonomous Decision Making, Optimization, Agent Coordination, High-Dimensional State Spaces, Reward ShapingAbstract
Cyber-physical systems (CPS) implementation into multi-agent systems (MAS) is a paradigm shift to the autonomous decision making process in multi-dynamic and complicated environments. The recent development of deep reinforcement learning (DRL) has proposed new approaches towards maximising the performance of MAS in CPS, which enables better adaptability, coordination, and efficiency of decision-making. This article represents an extensive study of both the algorithms and methods of DRL-based optimization specifically to multi-agent CPS, emphasizing the interaction between algorithms and agent cooperation with system dynamics. We investigate the problems of real-time decision making, state-action space that is high-dimensional and dynamic interaction among heterogeneous agents. The effectiveness of DRL-based strategies is proven through the large-scale simulation work and the evaluation of their effectiveness, with the rates of accomplishing tasks, using resources, and robustness improvement being highly favorable over the utilization of traditional optimization solutions. In addition, the paper highlights scalable structure, incentive directing, and communication procedures that boost coordination between agents. Results of analysis, supported by tables, figures, and flowcharts, demonstrate the relevance of the given approach to practice in such areas as autonomous vehicular networks, smart grids, and industrial automation. The article adds to the increasing amount of literature concerning autonomous CPS optimization and offers a system of ways to implement DRL methods in practice in multi-agent systems
References
[1] Arturo Servin & Daniel Kudenko. “Multi agent Reinforcement Learning for Intrusion Detection”. In Adaptive Agents and Multi Agent Systems III: Adaptation and Multi‐Agent Learning (eds. K. Tuyls, A. Nowé, Z. Guessoum, D. Kudenko), Springer, 2008, pp. 211‐223.
[2] Valeria Javalera, Bernardo Morcego & Vicenç Puig. “A Multi Agent MPC Architecture for Distributed Large Scale Systems”. Proceedings of the 2010 (Scitepress) Conference. 2010.
[3] Konstantinos Karydis, Prasanna Kannappan, Herbert G. Tanner, Adam Jardine & Jeffrey Heinz. “Resilience through Learning in Multi Agent Cyber Physical Systems”. Frontiers in Robotics and AI, Vol.3 (2016) – published online June 2016.
[4] Caroline Claus & Craig Boutilier (1998). The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI 1998), pp. 746 752.
[5] Michael L. Littman (1994). Markov Games as a Framework for Multi Agent Reinforcement Learning. In Proceedings of the 11th International Conference on Machine Learning (ICML 1994), pp. 157 163..
[6] S. M. Shafiul Alam. “Multi agent Estimation and Control of Cyber Physical Systems”. Kansas State University, December 2015.
[7] Ronald J. Williams (1992). “Simple statistical gradient‐following algorithms for connectionist reinforcement learning” (Machine Learning, Vol 8, pp 229 256)
[8] Scalable Centralized Deep Multi Agent Reinforcement Learning via Policy Gradients (Khan, Zhang, Lee, Kumar & Ribeiro, 2018)
[9] Multi Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (Zhang, Yang & Başar, 2019)
[10] Wooldridge, M., & Jennings, N. R. (1995). Intelligent Agents: Theory and Practice. The Knowledge Engineering Review, 10(2), 115–152.
[11] Busoniu, L., Babuska, R., & De Schutter, B. (2008). A Comprehensive Survey of Multi-Agent Reinforcement Learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 38(2), 156–172.
[12] Rajkumar, R., Lee, I., Sha, L., & Stankovic, J. (2010). Cyber-Physical Systems: The Next Computing Revolution. Proceedings of the Design Automation Conference.
[13] Mnih, V., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.
[14] Li, C., & Qiu, M. (2019). Reinforcement Learning for Cyber-Physical Systems: With Cybersecurity Case Studies. Routledge.
[15] Lowe, R., et al. (2017). Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. Advances in Neural Information Processing Systems (NeurIPS).
[16] Shannon, C. E. (1950) – Programming a Computer for Playing Chess. Philosophical Magazine.
[17] Widrow, B., & Hoff, M. E. (1960). Adaptive Switching Circuits.
[18] Thallam, N. S. T. (2020). Comparative Analysis of Data Warehousing Solutions: AWS Redshift vs. Snowflake vs. Google BigQuery. European Journal of Advances in Engineering and Technology, 7(12), 133-141.
[19] Designing LTE-Based Network Infrastructure for Healthcare IoT Application - Varinder Kumar Sharma - IJAIDR Volume 10, Issue 2, July-December 2019. DOI 10.71097/IJAIDR.v10.i2.1540
