半潜式航行体回收辅助操控技术应用

Application of Auxiliary Control Technology for Semi-submersible Vehicle Recovery

摘要:

半潜式航行体回收过程耗时较长，对操纵者经验、技术门槛高，试错成本大。针对此问题提出 VH-PPO 算法，从收敛性、期望收敛时间的上下界、时间复杂度 3 个方面分析其性能。通过人工操控成功的历史数据给予初始概率分布并在此基础上进行训练，省去了自由探索不断试错的过程，可有效减小期望收敛时间，使训练模型更快地收敛，从而降低算法的时间复杂度。针对训练时不同的阶段，选择更优的超参数，防止超调和欠调现象，可帮助训练模型更好地收敛，降低期望收敛时间的上下界，从而降低算法的时间复杂度。 在 OpenAI Gym 上通过该算法进行强化学习，训练完成后应用于操控软件，在某海域进行验证并进一步调整模型。实验结果表明：随着试验次数增加，智能体在真实环境中的适应能力越来越好，辅助操控指令在总操控指令中占比超过 50%，有效地减缓了操纵者的疲劳，降低了新手训练难度及替换操纵者的门槛。

关键词:半潜式航行体;算法复杂度;可视化界面;OpenAI Gym;强化学习;辅助操控

Abstract:

The recovery process of semi-submersible vehicles takes a long time，requires a high level of experience and technology of the operator，and the cost of trial and error is high. The VH-PPO algorithm is proposed to address this issue，and its performance is analyzed from three aspects，which are convergence，upper and lower bounds of expected convergence time，and time complexity. Using historical data successfully manipulated by human，the initial probability distribution is given and trained on this basis. The process of free exploration and continuous trial and error is eliminated and the expected convergence time can be effectively reduced，so that the training model can converge faster，and time complexity of the algorithm can be reduced. Choosing better hyperparameters for different stages of training to prevent overshoot and undershooting can help the training model converge better，reduce the upper and lower bounds of the expected convergence time，and thus reduce the time complexity of the algorithm. The algorithm is used for reinforcement learning in OpenAI Gym. After training，it is applied to the control software. The model is validated and further adjusted in a certain sea area. The experimental results show that as the number of experiment increases，the adaptability of the intelligent agent in the real environment gets better and better，and auxiliary control commands account for more than 50% of the total control commands，which effectively relieves the fatigue of the operator，and reduces the difficulty of novice training and the threshold for replacing operators.

Key words:semi-submersible vehicle;algorithm complexity;visual interface;OpenAI Gym;reinforcement learning;auxiliary control