Custom Quadruped Robot: Design, Build & Reinforcement Learning

Author: mgr inż. Witold Pietroń Thesis supervisor: dr inż. Patryk Chaber Master's Diploma Thesis: Design, construction, and reinforcement learning of a custom quadruped robot using a digital twin Contents: 0:00 Intro 1:04 3D Model 1:57 Simulation 2:40 Training in Simulation 5:20 Sim-to-Real Transfer The subject of this thesis is the design, construction and learning of stable locomotion for a custom-built legged machine using reinforcement learning methods and the digital twin. The considered problem is limited to motion in a single direction. As part of the thesis, a low-cost quadrupedal platform with two degrees of freedom per leg was developed using 3D printing technology. The drive system is based on hobby-grade servomechanisms without position feedback or rotational speed control, while a Raspberry Pi Pico microcontroller was used as the central control unit. In the next stage, a digital twin of the robot was developed in the CoppeliaSim environment, with particular focus placed on accurately reproducing the dynamic and structural parameters of the platform. Simultaneously, a multi-component gait quality metric was developed, taking into account locomotion speed and stability. Reinforcement learning process was then performed using the Proximal Policy Optimization algorithm, an asymmetric actor-critic architecture, and curriculum learning. Trained policy achieved velocity more than three times higher than that obtained using the reference trajectory developed with a white-box approach. To transfer the trained policy to the physical platform, a custom inference mechanism was implemented on the microcontroller using the MicroPython environment and optimized with the ulab module. The reality gap between the simulation and the physical system was reduced by introducing a penalty for abrupt changes in actions into the reward function. Experimental tests conducted on the physical robot confirmed the successful transfer of the policy and demonstrated the ability to achieve stable and efficient locomotion on the physical platform. Linkedin: www.linkedin.com/in/witold-pietroń-56a882256 E-mail: witek.pietron@gmail.com