不确定性环境下园区风光储互动运行的PPO强化学习策略

PPO-based reinforcement learning strategy for interactive operation of wind-photovoltaic-storage in an uncertain environment

中文关键词: 园区能源管理系统  微电网  风光储互动  电池储能系统

英文关键词: energy management system in parks  microgrid  interaction of wind- photovoltaic- storage  battery energy storage system

中文摘要:

      随着能源结构的不断升级，部署有新能源发电的新型园区将在未来新型电力系统中扮演重要的角色。需求的随机性、风光出力的间歇性、电力市场电价的波动性等不确定性因素耦合在一起，使得风光能源与电池储能系统的合理互动运行变得十分困难。考虑到传统优化方法的局限性，提出了一种基于PPO算法的深度强化学习方法，用于解决不确定环境下园区风光储互动运行问题。基于强化学习的理论框架，为园区互动运行构建了一个具有连续状态空间和连续动作空间且转移概率未知的马尔可夫决策模型，通过与新型负荷控制系统对接的能源管理系统控制园区微电网中的电池储能系统及柔性资源，在充分考虑电池退化成本的基础上，实现园区经济运行。

英文摘要:

      With the continuous upgrading of the energy structure, the new parks with new energy power generation will play an important role in the future new power system. Uncertainties such as the randomness of demand, intermittency of wind and solar output, and volatility of electricity prices in electricity market are coupled together, making it difficult to achieve the reasonable operation between wind and solar energy and battery energy storage system. Considering the limitations of traditional optimization methods, a deep reinforcement learning method based on the PPO algorithm is proposed to solve the problem of interactive operation of wind-solar-storage in parks under uncertain environments. Based on the theoretical framework of reinforcement learning, a Markov decision model with continuous state space and continuous action space and unknown transition probability is constructed for the interactive operation of the park. The new load control system controls the battery energy storage system and flexible resources in the microgrid of the park to realize the economic operation, fully considering battery degradation.