Maximum Power Point Tracker Controller for Solar Photovoltaic Based on Reinforcement Learning Agent with a Digital Twin

Abstract

Photovoltaic (PV) energy, representing a renewable source of energy, plays a key role in the reduction of greenhouse gas emissions and the achievement of a sustainable mix of energy generation. To achieve the maximum solar energy harvest, PV power systems require the implementation of Maximum Power Point Tracking (MPPT). Traditional MPPT controllers, such as P&O, are easy to implement, but they are by nature slow and oscillate around the MPP losing efficiency. This work presents a Reinforcement learning (RL)-based control to increase the speed and the efficiency of the controller. Deep Deterministic Policy Gradient (DDPG), the selected RL algorithm, works with continuous actions and space state to achieve a stable output at MPP. A Digital Twin (DT) enables simulation training, which accelerates the process and allows it to operate independent of weather conditions. In addition, we use the maximum power achieved in the DT to adjust the reward function, making the training more efficient. The RL control is compared with a traditional P&O controller to validate the speed and efficiency increase both in simulations and real implementations. The results show an improvement of 10.45% in total power output and a settling time 24.54 times faster in simulations. Moreover, in real-time tests, an improvement of 51.45% in total power output and a 0.25 s settling time of the DDPG compared with 4.26 s of the P&O is obtained.