TY - GEN
T1 - Optimizing Deep Reinforcement Learning for Adaptive Robotic Arm Control
AU - Shianifar, Jonaid
AU - Schukat, Michael
AU - Mason, Karl
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
PY - 2025
Y1 - 2025
N2 - In this paper, we explore the optimization of hyperparameters for the Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO) algorithms using the Tree-structured Parzen Estimator (TPE) in the context of robotic arm control with seven Degrees of Freedom (DOF). Our results demonstrate a significant enhancement in algorithm performance, TPE improves the success rate of SAC by 10.48% points and PPO by 34.28% points, where models trained for 50K episodes. Furthermore, TPE enables PPO to converge to a reward within 95% of the maximum reward 76% faster than without TPE, which translates to about 40K fewer episodes of training required for optimal performance. Also, this improvement for SAC is 80% faster than without TPE. This study underscores the impact of advanced hyperparameter optimization on the efficiency and success of deep reinforcement learning algorithms in complex robotic tasks.
AB - In this paper, we explore the optimization of hyperparameters for the Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO) algorithms using the Tree-structured Parzen Estimator (TPE) in the context of robotic arm control with seven Degrees of Freedom (DOF). Our results demonstrate a significant enhancement in algorithm performance, TPE improves the success rate of SAC by 10.48% points and PPO by 34.28% points, where models trained for 50K episodes. Furthermore, TPE enables PPO to converge to a reward within 95% of the maximum reward 76% faster than without TPE, which translates to about 40K fewer episodes of training required for optimal performance. Also, this improvement for SAC is 80% faster than without TPE. This study underscores the impact of advanced hyperparameter optimization on the efficiency and success of deep reinforcement learning algorithms in complex robotic tasks.
KW - Deep Reinforcement Learning
KW - Hyperparameter Optimization
KW - Robotic Arm Control
UR - https://www.scopus.com/pages/publications/85216107283
U2 - 10.1007/978-3-031-73058-0_24
DO - 10.1007/978-3-031-73058-0_24
M3 - Conference Publication
AN - SCOPUS:85216107283
SN - 9783031730573
T3 - Communications in Computer and Information Science
SP - 293
EP - 304
BT - Highlights in Practical Applications of Agents, Multi-Agent Systems, and Digital Twins
A2 - González-Briones, Alfonso
A2 - Julian Inglada, Vicente
A2 - El Bolock, Alia
A2 - Marco-Detchart, Cedric
A2 - Jordan, Jaume
A2 - Mason, Karl
A2 - Lopes, Fernando
A2 - Sharaf, Nada
PB - Springer Science and Business Media Deutschland GmbH
T2 - International Workshops on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2024
Y2 - 26 June 2024 through 28 June 2024
ER -