Safety-Constrained Multi-Agent Reinforcement Learning for Coordinated UAV-UGV Control under Model and Delay Uncertainty

Abstract

The cooperative deployment of Unmanned Aerial Vehicles and Unmanned Ground Vehicles has emerged as a transformative paradigm for complex field operations, offering complementary capabilities in mobility, payload capacity, and sensing range. However, designing effective control policies for such heterogeneous multi-agent systems remains profoundly challenging due to the inherent presence of communication delays, unpredictable environmental dynamics, and stringent safety requirements. Traditional control paradigms often fail to scale in high-dimensional state spaces or struggle to adapt to unmodeled disturbances. This paper proposes a novel Safety-Constrained Multi-Agent Reinforcement Learning framework explicitly designed to address coordinated Unmanned Aerial Vehicle and Unmanned Ground Vehicle control under simultaneous model and delay uncertainty. By reformulating the cooperative task as a Delay-Aware Constrained Markov Decision Process, we introduce a decentralized actor-centralized critic architecture that incorporates recurrent belief-state estimators to compensate for delayed state observations. Furthermore, to ensure strict adherence to safety constraints such as collision avoidance and energy limitations, we integrate a probabilistic Lagrangian optimization technique that dynamically adjusts penalty multipliers based on the estimated uncertainty of the transition dynamics. Extensive simulations in realistically disturbed environments demonstrate that the proposed framework significantly outperforms existing baselines, maintaining near-zero constraint violation rates while achieving superior task completion efficiency. The findings provide a robust foundation for deploying heterogeneous autonomous teams in communication-degraded and highly uncertain real-world environments.