在无人机环境监测问题中,通过协同分布式无人机的角度、速度、方向等,能够有效提升监测的覆盖率和准确率。以往的工作通过组合优化、博弈建模等来实现无人机的协同,但是这些方法往往采用了集中式的调度策略,或者假设环境是静态的;而在实际情况中,由于环境的复杂性,不同的无人机会动态调整自己的策略,因此很难通过寻找博弈均衡来设置无人机协同策略。针对上述问题,提出一种基于值分布多智能体强化学习的无人机协同方法,将每个无人机看做一个智能体,无人机通过与环境交互来最大化集体长期累积收益。在该环境中,由于无人机的移动策略可能是动态的,因此引入了值分布来刻画每个智能体的累积收益。相比于传统的多智能体强化学习,通过值分布学习,可以用概率分布对多智能体的累积收益进行评估,从而可以更全面地取得更稳定的结果。在模拟环境中的实验表明,以上的方法可以有效提升多个无人机协同的长期收益,相比于其他最新的算法,可以提升平均收益约17. 2%。

In the Unmanned Aerial Vehicle (UAV) monitoring problem, distributed UAVs can adjust their speeds, directions, and accelerations to improve the coverage and accuracy. Previous works have adopted centralized methods, or considered the immediate reward only. In practice, due to the complexity of the environment, the UAVs should make decentralized decisions in a dynamic environment, which is challenging to find equilibrium. Regarding to this problem, in this paper, a novel distributional multi-agent reinforcement learning is proposed for coordinating decentralized UAVs. In this algorithm, each UAV is represented as an autonomous agent, which interacts with the environment and other UAVs to optimize the return. As the agents′ policies are dynamic, probability distributions are used to characterize the state-action value function of all the agents. The distributional method can provide a complete assessment of the state of the agent. Extensive experiments show that the proposed algorithm can improve the returns of the UAVs significantly. Compared with other state-of-the-art algorithms, it achieves a higher performance of 17. 2%.

TP312

国家自然科学基金资助项目(62202238)