This paper investigates the application of deep reinforcement learning in the composite energy management problem of microgrid, establishes the DQN algorithm model for the characteristics of microgrid, and reduces the system operation cost and improves the renewable energy consumption by arranging the time-sequential charging and discharging states of the energy storage system. On this basis, a controller based on Q learning algorithm is designed, and the Q learning algorithm is utilized to dynamically correct the sag parameter, coordinate multiple distributed power sources of the grid for frequency restoration control, and verify the stability of the grid. The results show that the multi-source coordinated frequency control method proposed in this paper can fully exploit the economic optimization potential of demand-side response and use the energy optimization allocation capability of the energy storage system. It effectively improves the load-side power consumption, enhances the stability and reliability of the system, and reduces the system power cost. It is verified that the sag control at the primary control layer has the effect of reasonably allocating the system output power and stabilizing the output voltage and frequency, and the Q-learning frequency and voltage secondary controllers based on reinforcement learning can effectively improve the frequency and voltage deviation caused by the primary sag control, and improve the quality of the grid output power.
