Comparing continuous single-agent reinforcement learning controls in a simulated logistic environment using NVIDIA Omniverse

With the transition to Logistics 4.0, the increasing demand for autonomous mobile robots (AMR) in logistics has amplified the complexity of fleet control in dynamic environments. Reinforcement learning (RL), particularly decentralized RL algorithms, has emerged as a potential solution given its ability to learn in uncertain terrains. While discrete RL structures have shown merit, their adaptability in logistics remains questionable due to their inherent limitations. This paper presents a comparative analysis of continuous RL algorithms - Advantage Actor-Critic (A2C), Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO) - in the context of controlling a Turtlebot3 within a warehouse scenario. Our findings reveal A2C as the frontrunner in terms of success rate and training time, while DDPG excels step minimization while PPO distinguishes itself primarily through its relatively short training duration. This study underscores the potential of continuous RL algorithms, especially A2C, in the future of AMR fleet management in logistics. Significant work remains to be done, particularly in the area of algorithmic fine-tuning.

Subjects

artificial intelligence

Autonome Roboter

autonomous mobile robots

Künstliche Intelligenz

logistics 4.0

Logistik 4.0

reinforcement learning

robotics

Robotik

DDC Class

620: Engineering

004: Computer Sciences

Publication version

publishedVersion

Lizenz

https://creativecommons.org/licenses/by/4.0/

Name

wesselhoeft_en_2023.pdf

Type

Main Article

Size

657.34 KB

Format

Adobe PDF

Options

Comparing continuous single-agent reinforcement learning controls in a simulated logistic environment using NVIDIA Omniverse