Double Inverted Pendulum Reinforcement Learning | PPO, Pymunk & Physics Simulation

Keywords: Double Inverted Pendulum, Reinforcement Learning, PPO, Proximal Policy Optimization, Python, Pymunk, Pygame, Gymnasium, Stable-Baselines3, Physics Simulation, Reward Shaping, Continuous Control, Docker, Robotics, Control Systems 🚀 PROJECT OVERVIEW This project implements a custom Reinforcement Learning (RL) environment for a Double Inverted Pendulum using Python, Pymunk for 2D rigid-body physics, and Pygame for visualization. A Proximal Policy Optimization (PPO) agent, trained with Stable-Baselines3, learns to balance two interconnected poles by applying continuous horizontal forces to a moving cart. The environment uses a six-dimensional observation space and supports baseline and shaped reward functions to explore balancing performance, control stability, and reward design. ✨ KEY FEATURES Custom Gymnasium-compatible environment using Pymunk physics Two interconnected pendulum poles mounted on a horizontally constrained cart Six-dimensional observation space containing cart position and velocity, and both poles' angles and angular velocities Continuous action space for force-based cart control PPO agent training using Stable-Baselines3 and PyTorch Baseline and shaped reward formulations for balancing behavior Center-drift, angular-velocity, and action-magnitude penalties for improved control Linear and angular damping to address physics instability Training logs, reward comparison plots, and agent behavior GIF recording Docker and Docker Compose workflows for training, evaluation, and reproducible execution 🛠️ TECHNOLOGY STACK Python, Gymnasium, Pymunk, Pygame, Stable-Baselines3 (PPO), PyTorch, NumPy, Pandas, Matplotlib, ImageIO, pytest, Docker, Docker Compose 📂 GITHUB REPOSITORY https://github.com/SabbellaLaharika/R... 🎥 ABOUT THIS DEMO This technical walkthrough demonstrates the implementation of a custom reinforcement learning system for a nonlinear control problem. It covers physics environment design, observation and action spaces, PPO training, reward shaping, physics stabilization, model evaluation, training logs, visualization, and Docker-based execution. The project also explores potential applications of reinforcement learning in robotics, industrial automation, and aerospace control. The focus is on the engineering decisions, implementation, and behavior of the trained agent rather than a step-by-step tutorial. If you find this project useful, please consider starring the GitHub repository and subscribing for more software engineering, AI, and machine learning project demonstrations. #️⃣ HASHTAGS #ReinforcementLearning #Python #PPO #Pymunk #Pygame #Gymnasium #StableBaselines3 #PyTorch #PhysicsSimulation #RewardShaping #ContinuousControl #MachineLearning #Robotics #ControlSystems #Docker