A Multi-Agent Reinforcement Framework for Autonomous Cloud Resource Scheduling and Optimization

Abstract

Cloud computing has reached the core of the digital ecosystems and provides high-availability and scale-able access to the services of computational services. Nevertheless, the active resource requirements, enforcement of service-level agreement (SLA), energy usage restrictions, and a variation in the cost-performance represent significant challenges to the cloud resource management. The conventional heuristic-based schedulers in clouds orchestrators like Kube and OpenStack fail to adjust to real-time workload variability, fractional resources, violation of SLA and inflated operational costs frequently occur. The adaptive, reward-driven decision-making properties of reinforcement learning (RL) have been discovered to be an interesting alternative to a heuristic policy. However, centralized RL has scalability and state observability scale in distributed clouds. As a way of overcoming these limitations, this study hypothesizes a Multi-Agent Reinforcement Learning (MARL) model of autonomous resource scheduling and optimization in cloud infrastructures. The offered structure suggests the use of decentralized intelligent agents executing a coordinated management of computational, storage, and network allocations. The agents are optimized to adapt local behaviors under environment feedback with the aid of cooperative communication to enhance global optimization. The system uses hybrid in reinforcement system consisting of Deep Q-Learning (DQN) and Proximal Policy Optimization (PPO) to trade between exploration, stability, and on-policy improvement. The goal of an adaptive reward model is to maximize important performance indicators (KPIs): SLA adherence, task completion latency, energy efficiency and balance resource utilization. The Cloud sim Plus experimental measurements were performed in a simulated multi-cluster environment and supported by extra features of container virtualization. The proposed version of the MARL scheduler showed a better throughput (18.7%), SLA violations reduction (27.4%) and energy consumption (14.3) than the baseline strategies like Round-Robin, Min-Min, and centralized DQN models. Convergence time was reduced and decision correlation under the workload bursts was improved due to the incorporation of the inter-agent communication. The research paper has a tremendous impact in the field by proposing a foresight, smart cloud orchestration framework that has the capability to adapt itself in the distributed large scale systems. Additionally, we offer an improvement in algorithms, performance analysis, system workflow optimization, and theoretical support of a cooperative reinforcement learning in clouds. The implication of these findings in the future is that the applicability of the research to edge and fog computing settings can be applied to Industry 5.0 automation