基于强化学习的AUV对接控制算法研究

Research on AUV docking control algorithm based on reinforcement learning

摘要:

自主式水下航行器（AUV）是人类探索和利用海洋的重要装备，能否足够智能化地解决路径规划控制问题是AUV完成其他复杂任务的基础。考虑终端姿态约束下的局部路径规划问题，结合AUV的自主对接控制这一实际使用场景，基于改进的深度强化学习算法（DRL）开发了一种对接控制器，使其具备自主对接能力，延长其续航时间。考虑实际工作场景中的复杂海浪干扰因素，使用了非线性扰动观测器（NDO）来估计AUV三维运动中各自由度的外部扰动，并结合可测量的状态量为DRL智能体设计了科学的观测量及奖励函数，使AUV能够在扰动环境中完成三维对接控制任务。仿真结果表明了提出的方法的有效性和鲁棒性。

Abstract:

Autonomous underwater vehicle (AUV) is an important equipment for human to explore and utilize the ocean, and whether it can be intelligent enough to solve the path planning control problem is the basis for AUV to accomplish other complex tasks. Considering the local path planning problem under terminal attitude constraints and combining with the autonomous docking control of AUV, which is a practical use scenario, a docking controller is developed based on the improved Deep Reinforcement Learning (DRL) algorithm, which equips it with the ability of autonomous docking and extends its endurance time. Considering the complex wave disturbance factors in the practical working scenario, a nonlinear disturbance observer (NDO) is used to estimate the external disturbances of each degree of freedom in the three-dimensional motion of the AUV, and scientific observation quantities and reward functions are designed for the DRL agent in combination with measurable state quantities, so as to enable the AUV to accomplish the three-dimensional docking control task in a disturbed environment. The simulation results demonstrate the effectiveness and robustness of the proposed method.