AI-Powered Optimization of Networked Computing Infrastructures for Low-Latency Data Processing

Abstract

Modern applications from real-time analytics to cyber-physical control demand millisecond-level responsiveness across heterogeneous, networked infrastructures. This paper presents an AI-driven optimization stack that jointly orchestrates compute, network, and data paths to minimize end-to-end latency under dynamic workloads. We model the system as a constrained Markov decision process and employ reinforcement learning for SLO-aware autoscaling, operator placement, and flow steering across edge cloud tiers. A complementary learning-to-rank scheduler prioritizes critical microservices using online features (queue depths, p95 latency, tail-loss risk) derived from kernel-level telemetry. To shrink inference delays, we integrate model-aware compilation and compression (quantization/distillation) with zero-copy data planes and adaptive batching. Network latency is reduced via intent-based routing that couples RL policies with programmable switches for congestion- and jitter-aware path selection. A control-theoretic safety layer enforces stability and cost caps, while a digital-twin emulator enables fast policy evaluation before deployment. Prototype implementation across containerized clusters demonstrates consistent reductions in tail latency and recovery time during bursty and faulted conditions, with improvements driven by closed-loop decisions rather than static heuristics. The framework is modular, explainable via attribution on scheduling actions, and portable to diverse hardware. We conclude with guidelines for production roll-out and discuss open challenges in cross-layer observability and multi-tenant fairness