Article(id=1200482408840687769, tenantId=1146029695717560320, journalId=1189987059142926344, issueId=1200482403828495344, articleNumber=null, orderNo=null, doi=10.19457/j.1001-2095.dqcd24592, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1661702400000, receivedDateStr=2022-08-29, revisedDate=1666800000000, revisedDateStr=2022-10-27, acceptedDate=null, acceptedDateStr=null, onlineDate=1764146910151, onlineDateStr=2025-11-26, pubDate=1718812800000, pubDateStr=2024-06-20, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1764146910151, onlineIssueDateStr=2025-11-26, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1764146910151, creator=13701087609, updateTime=1764146910151, updator=13701087609, issue=Issue{id=1200482403828495344, tenantId=1146029695717560320, journalId=1189987059142926344, year='2024', volume='54', issue='6', pageStart='3', pageEnd='96', issueExtLink='null', onlineDate='null', pubDate='null', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1764146908957, creator=13701087609, updateTime=1764224882080, updator=13701087609, preIssue=null, nextIssue=null, ext={EN=IssueExt(id=1200809446868898278, tenantId=1146029695717560320, journalId=1189987059142926344, issueId=1200482403828495344, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1200809446868898279, tenantId=1146029695717560320, journalId=1189987059142926344, issueId=1200482403828495344, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null}, startPage=54, endPage=59, ext={EN=ArticleExt(id=1200482409268506790, articleId=1200482408840687769, tenantId=1146029695717560320, journalId=1189987059142926344, language=EN, title=Optimal Dispatch of Microgrid Based on Proximal Policy Optimization, columnId=null, journalTitle=Electric Drive, columnName=null, runingTitle=null, highlight=null, articleAbstract=

Microgrid is an effective method to integrate a large number of distributed generators into the power grid. Aiming at the optimal dispatch problem of microgrid,an optimal dispatch method based on proximal policy optimization algorithm was proposed.Firstly,the optimal diapatch model of microgrid was constructed by considering the operation cost of microgrid and operation constraints of various equipment. Secondly,the problem was formulated as a reinforcement learning framework,and the elements of reinforcement learning such as state,action and reward function were designed. Finally,the solution flow based on the proximal policy optimization algorithm was designed,and the effectiveness of the proposed method was verified by simulation.

, correspAuthors=null, authorNote=null, correspAuthorsNote=null, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, authorCompany=null, fund=null, authors=null, authorsList=Yuefeng MA, Suqi ZHAO, Kai WANG), CN=ArticleExt(id=1200482411189498119, articleId=1200482408840687769, tenantId=1146029695717560320, journalId=1189987059142926344, language=CN, title=基于近端策略优化算法的微网优化调度, columnId=1190325328129131131, journalTitle=电气传动, columnName=综合能源与现代电网, runingTitle=null, highlight=null, articleAbstract=

微网是实现大量分布式电源集成到电网中的有效手段。针对微网优化调度问题,提出了一种基于近端策略优化算法的优化调度方法。首先,综合考虑微网运行成本和各类设备运行约束构建微网优化调度模型;其次,将该问题表述为强化学习框架,设计了强化学习状态、动作和奖励函数等要素;最后,设计了基于近端策略优化算法的求解流程,通过仿真验证了所提方法的有效性。

, correspAuthors=null, authorNote=null, correspAuthorsNote=null, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=XUk6Z7CLuo8ANEkz6ufW8w==, magXml=NYAZFpjIHTaO3O5nN+rmog==, pdfUrl=null, pdf=3QktFkfgi9nK8vUnKZiSkA==, pdfFileSize=2034149, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=NoAGGpUlzjyuNeMaxzDNFQ==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=zBsgTK8UFZeedV8E9pKlfg==, mapNumber=null, authorCompany=null, fund=null, authors=

马跃锋(1985—),男,本科,工程师,研究方向为电力系统继电保护,Email:

, authorsList=马跃锋, 招苏颀, 王凯)}, authors=[Author(id=1200501564671643827, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=82765829@qq.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1200501564818444471, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, authorId=1200501564671643827, language=EN, stringName=Yuefeng MA, firstName=Yuefeng, middleName=null, lastName=MA, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1 Chifeng Power Supply Company of State Grid East Inner Mongolia Power Co.,Ltd.,Chifeng 024000,Nei Monggol,China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1200501564923302078, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, authorId=1200501564671643827, language=CN, stringName=马跃锋, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1 国网内蒙古东部电力有限公司赤峰供电公司,内蒙古 赤峰 024000, bio={"content":"

马跃锋(1985—),男,本科,工程师,研究方向为电力系统继电保护,Email:

"}, bioImg=null, bioContent=

马跃锋(1985—),男,本科,工程师,研究方向为电力系统继电保护,Email:

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1200501564453540003, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, xref=1, ext=[AuthorCompanyExt(id=1200501564461928613, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564453540003, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 Chifeng Power Supply Company of State Grid East Inner Mongolia Power Co.,Ltd.,Chifeng 024000,Nei Monggol,China), AuthorCompanyExt(id=1200501564470317222, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564453540003, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 国网内蒙古东部电力有限公司赤峰供电公司,内蒙古 赤峰 024000)])]), Author(id=1200501565070102723, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1200501565288206537, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, authorId=1200501565070102723, language=EN, stringName=Suqi ZHAO, firstName=Suqi, middleName=null, lastName=ZHAO, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1 Chifeng Power Supply Company of State Grid East Inner Mongolia Power Co.,Ltd.,Chifeng 024000,Nei Monggol,China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1200501565422424270, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, authorId=1200501565070102723, language=CN, stringName=招苏颀, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1 国网内蒙古东部电力有限公司赤峰供电公司,内蒙古 赤峰 024000, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1200501564453540003, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, xref=1, ext=[AuthorCompanyExt(id=1200501564461928613, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564453540003, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 Chifeng Power Supply Company of State Grid East Inner Mongolia Power Co.,Ltd.,Chifeng 024000,Nei Monggol,China), AuthorCompanyExt(id=1200501564470317222, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564453540003, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 国网内蒙古东部电力有限公司赤峰供电公司,内蒙古 赤峰 024000)])]), Author(id=1200501565665693911, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1200501565875409122, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, authorId=1200501565665693911, language=EN, stringName=Kai WANG, firstName=Kai, middleName=null, lastName=WANG, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2 School of Electric Power Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1200501565992849642, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, authorId=1200501565665693911, language=CN, stringName=王凯, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2 华南理工大学 电力学院,广东 广州 510640, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1200501564575174828, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, xref=2, ext=[AuthorCompanyExt(id=1200501564583563437, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564575174828, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 School of Electric Power Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China), AuthorCompanyExt(id=1200501564591952046, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564575174828, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 华南理工大学 电力学院,广东 广州 510640)])])], keywords=[Keyword(id=1200501566240313589, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, orderNo=1, keyword=microgrid), Keyword(id=1200501566353559803, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, orderNo=2, keyword=optimal dispatch), Keyword(id=1200501566445834495, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, orderNo=3, keyword=artificial intelligence), Keyword(id=1200501566517137669, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, orderNo=4, keyword=reinforcement learning), Keyword(id=1200501566609412362, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, orderNo=1, keyword=微电网), Keyword(id=1200501566693298446, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, orderNo=2, keyword=优化调度), Keyword(id=1200501566802350356, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, orderNo=3, keyword=人工智能), Keyword(id=1200501566911402264, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, orderNo=4, keyword=强化学习)], refs=[Reference(id=1200501569197298062, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2018, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=[1], rfOrder=0, authorNames=吕天光, journalName=含多微网配电系统多元主体互动运行策略研究, refType=null, unstructuredReference=吕天光. 含多微网配电系统多元主体互动运行策略研究[D]. 上海: 上海交通大学, 2018., articleTitle=null, refAbstract=null), Reference(id=1200501569314738580, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2018, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=[1], rfOrder=1, authorNames=LÜ Tianguang, journalName=Study on interactive operation strategies for multiple agents in power distribution systems with multi-microgrids, refType=null, unstructuredReference= Tianguang. Study on interactive operation strategies for multiple agents in power distribution systems with multi-microgrids[D]. Shanghai: Shanghai Jiao Tong University, 2018., articleTitle=null, refAbstract=null), Reference(id=1200501569465733530, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2020, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=[2], rfOrder=2, authorNames=沈儒茹, journalName=多微网系统的优化调度策略研究, refType=null, unstructuredReference=沈儒茹. 多微网系统的优化调度策略研究[D]. 哈尔滨: 哈尔滨工业大学, 2020., articleTitle=null, refAbstract=null), Reference(id=1200501569545425311, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2020, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=[2], rfOrder=3, authorNames=SHEN Ruru, journalName=Research on optimal scheduling strategy of multi- microgrid system, refType=null, unstructuredReference=SHEN Ruru. Research on optimal scheduling strategy of multi- microgrid system[D]. Harbin: Harbin Industrial University, 2020., articleTitle=null, refAbstract=null), Reference(id=1200501569646088613, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=46, issue=1, pageStart=130, pageEnd=138, url=null, language=null, rfNumber=[3], rfOrder=4, authorNames=帅轩越, 王秀丽, 吴雄, journalName=电网技术, refType=null, unstructuredReference=帅轩越, 王秀丽, 吴雄, 等. 计及条件风险价值下基于合作博弈的多微网协同优化调度[J]. 电网技术, 2022, 46(1):130-138., articleTitle=计及条件风险价值下基于合作博弈的多微网协同优化调度, refAbstract=null), Reference(id=1200501569755140521, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=46, issue=1, pageStart=130, pageEnd=138, url=null, language=null, rfNumber=[3], rfOrder=5, authorNames=SHUAI Xuanyue, WANG Xiuli, WU Xiong, journalName=Power System Technology, refType=null, unstructuredReference=SHUAI Xuanyue, WANG Xiuli, WU Xiong, et al. Cooperative optimal scheduling of multi-microgrids based on cooperative game considering conditional value at risk[J]. Power System Technology, 2022, 46(1):130-138., articleTitle=Cooperative optimal scheduling of multi-microgrids based on cooperative game considering conditional value at risk, refAbstract=null), Reference(id=1200501569893552557, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2021, volume=41, issue=4, pageStart=1307, pageEnd=1321, url=null, language=null, rfNumber=[4], rfOrder=6, authorNames=李鹏, 吴迪凡, 李雨薇, journalName=中国电机工程学报, refType=null, unstructuredReference=李鹏, 吴迪凡, 李雨薇, 等. 基于综合需求响应和主从博弈的多微网综合能源系统优化调度策略[J]. 中国电机工程学报, 2021, 41(4):1307-1321,1538., articleTitle=基于综合需求响应和主从博弈的多微网综合能源系统优化调度策略, refAbstract=null), Reference(id=1200501570036158898, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2021, volume=41, issue=4, pageStart=1307, pageEnd=1321, url=null, language=null, rfNumber=[4], rfOrder=7, authorNames=LI Peng, WU Difan, LI Yuwei, journalName=Proceedings of the CSEE, refType=null, unstructuredReference=LI Peng, WU Difan, LI Yuwei, et al. Optimal dispatch of multi-microgrids integrated energy system based on integrated demand response and stackelberg game[J]. Proceedings of the CSEE, 2021, 41(4):1307-1321,1538., articleTitle=Optimal dispatch of multi-microgrids integrated energy system based on integrated demand response and stackelberg game, refAbstract=null), Reference(id=1200501570203931061, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=52, issue=15, pageStart=53, pageEnd=60, url=null, language=null, rfNumber=[5], rfOrder=8, authorNames=许琼果, 张炜焘, 刘光伟, journalName=电气传动, refType=null, unstructuredReference=许琼果, 张炜焘, 刘光伟, 等. 计及储能调节特性的直流微电网优化调度研究[J]. 电气传动, 2022, 52(15):53-60., articleTitle=计及储能调节特性的直流微电网优化调度研究, refAbstract=null), Reference(id=1200501570321371575, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=52, issue=15, pageStart=53, pageEnd=60, url=null, language=null, rfNumber=[5], rfOrder=9, authorNames=XU Qiongguo, ZHANG Weitao, LIU Guangwei, journalName=Electric Drive, refType=null, unstructuredReference=XU Qiongguo, ZHANG Weitao, LIU Guangwei, et al. Research on optimal scheduling of DC microgrid considering energy sto-rage regulation characteristics[J]. Electric Drive, 2022, 52(15):53-60., articleTitle=Research on optimal scheduling of DC microgrid considering energy sto-rage regulation characteristics, refAbstract=null), Reference(id=1200501570455589310, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2014, volume=42, issue=5, pageStart=85, pageEnd=89, url=null, language=null, rfNumber=[6], rfOrder=10, authorNames=宋晓英, 王艳松, journalName=电力系统保护与控制, refType=null, unstructuredReference=宋晓英, 王艳松. 基于协同进化遗传算法的微网经济环保调度[J]. 电力系统保护与控制, 2014, 42(5):85-89., articleTitle=基于协同进化遗传算法的微网经济环保调度, refAbstract=null), Reference(id=1200501570594001345, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2014, volume=42, issue=5, pageStart=85, pageEnd=89, url=null, language=null, rfNumber=[6], rfOrder=11, authorNames=SONG Xiaoying, WANG Yansong, journalName=Power System Protection and Control, refType=null, unstructuredReference=SONG Xiaoying, WANG Yansong. Economic and environmental dispatch of microgrid using co-evolutionary genetic algorithm[J]. Power System Protection and Control, 2014, 42(5):85-89., articleTitle=Economic and environmental dispatch of microgrid using co-evolutionary genetic algorithm, refAbstract=null), Reference(id=1200501570828882374, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2019, volume=32, issue=1, pageStart=67, pageEnd=81, url=null, language=null, rfNumber=[7], rfOrder=12, authorNames=WANG L, LAN X G, ZHANG H B, journalName=Pattern Recognition and Artificial Intelligence, refType=null, unstructuredReference=WANG L, LAN X G, ZHANG H B. The theory application of deep reinforcement learning[J]. Pattern Recognition and Artificial Intelligence, 2019, 32(1):67-81., articleTitle=The theory application of deep reinforcement learning, refAbstract=null), Reference(id=1200501570958905803, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2021, volume=47, issue=10, pageStart=2321, pageEnd=2340, url=null, language=null, rfNumber=[8], rfOrder=13, authorNames=熊珞琳, 毛帅, 唐漾, journalName=自动化学报, refType=null, unstructuredReference=熊珞琳, 毛帅, 唐漾, 等. 基于强化学习的综合能源系统管理综述[J]. 自动化学报, 2021, 47(10):2321-2340., articleTitle=基于强化学习的综合能源系统管理综述, refAbstract=null), Reference(id=1200501571118289359, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2021, volume=47, issue=10, pageStart=2321, pageEnd=2340, url=null, language=null, rfNumber=[8], rfOrder=14, authorNames=XIONG Luolin, MAO Shuai, TANG Yang, journalName=Acta Automatica Sinica, refType=null, unstructuredReference=XIONG Luolin, MAO Shuai, TANG Yang, et al. Reinforcement learning based integrated energy system management:a survey[J]. Acta Automatica Sinica, 2021, 47(10):2321-2340., articleTitle=Reinforcement learning based integrated energy system management:a survey, refAbstract=null), Reference(id=1200501571235729877, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=29, issue=7, pageStart=1162, pageEnd=1172, url=null, language=null, rfNumber=[9], rfOrder=15, authorNames=吴利刚, 张梁, 周倩, journalName=控制工程, refType=null, unstructuredReference=吴利刚, 张梁, 周倩, 等. 基于强化学习的微电网能量调度优化策略研究[J]. 控制工程, 2022, 29(7):1162-1172., articleTitle=基于强化学习的微电网能量调度优化策略研究, refAbstract=null), Reference(id=1200501571386724825, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=29, issue=7, pageStart=1162, pageEnd=1172, url=null, language=null, rfNumber=[9], rfOrder=16, authorNames=WU Ligang, ZHANG Liang, ZHOU Qian, journalName=Control Engineering of China, refType=null, unstructuredReference=WU Ligang, ZHANG Liang, ZHOU Qian, et al. Research on energy scheduling optimization strategy of micro-grid based on reinforcement learning[J]. Control Engineering of China, 2022, 29(7):1162-1172., articleTitle=Research on energy scheduling optimization strategy of micro-grid based on reinforcement learning, refAbstract=null), Reference(id=1200501571470610908, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2021, volume=null, issue=3, pageStart=150, pageEnd=153, url=null, language=null, rfNumber=[10], rfOrder=17, authorNames=邢敖, 李云, journalName=自动化与仪器仪表, refType=null, unstructuredReference=邢敖, 李云. 面向主动配电网的调峰需求系统调度学习优化[J]. 自动化与仪器仪表, 2021(3):150-153,157., articleTitle=面向主动配电网的调峰需求系统调度学习优化, refAbstract=null), Reference(id=1200501571600634337, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2021, volume=null, issue=3, pageStart=150, pageEnd=153, url=null, language=null, rfNumber=[10], rfOrder=18, authorNames=XING Ao, LI Yun, journalName=Automation & Instrumentation, refType=null, unstructuredReference=XING Ao, LI Yun. System scheduling learning optimization for peak load regulation in active distribution network[J]. Automation & Instrumentation, 2021(3):150-153,157., articleTitle=System scheduling learning optimization for peak load regulation in active distribution network, refAbstract=null), Reference(id=1200501571713880552, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=51, issue=3, pageStart=141, pageEnd=147, url=null, language=null, rfNumber=[11], rfOrder=19, authorNames=谢启跃, 应雨龙, journalName=热力发电, refType=null, unstructuredReference=谢启跃, 应雨龙. 基于深度确定性策略梯度的电-气综合能源微网优化调度研究[J]. 热力发电, 2022, 51(3):141-147., articleTitle=基于深度确定性策略梯度的电-气综合能源微网优化调度研究, refAbstract=null), Reference(id=1200501571831321070, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=51, issue=3, pageStart=141, pageEnd=147, url=null, language=null, rfNumber=[11], rfOrder=20, authorNames=XIE Qiyue, YING Yulong, journalName=Thermal Power Generation, refType=null, unstructuredReference=XIE Qiyue, YING Yulong. Research on optimal dispatch of electricity-gas integrated energy microgrid based on deep deterministic policy gradient[J]. Thermal Power Generation, 2022, 51(3):141-147., articleTitle=Research on optimal dispatch of electricity-gas integrated energy microgrid based on deep deterministic policy gradient, refAbstract=null), Reference(id=1200501571919401460, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2019, volume=10, issue=4, pageStart=3698, pageEnd=3708, url=null, language=null, rfNumber=[12], rfOrder=21, authorNames=MOCANU E, MOCANU D C, NGUYEN P H, journalName=IEEE Transactions on Smart Grid, refType=null, unstructuredReference=MOCANU E, MOCANU D C, NGUYEN P H, et al. On-line building energy optimization using deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2019, 10(4):3698-3708., articleTitle=On-line building energy optimization using deep reinforcement learning, refAbstract=null), Reference(id=1200501572024259063, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=609, issue=null, pageStart=750, pageEnd=765, url=null, language=null, rfNumber=[13], rfOrder=22, authorNames=ZHANG J W, ZHANG Z H, HAN S, journalName=Information Sciences, refType=null, unstructuredReference=ZHANG J W, ZHANG Z H, HAN S, et al. Proximal policy optimization via enhanced exploration efficiency[J]. Information Sciences, 2022,609:750-765., articleTitle=Proximal policy optimization via enhanced exploration efficiency, refAbstract=null), Reference(id=1200501572120728058, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=46, issue=1, pageStart=110, pageEnd=119, url=null, language=null, rfNumber=[14], rfOrder=23, authorNames=叶宇剑, 王卉宇, 汤奕, journalName=电力系统自动化, refType=null, unstructuredReference=叶宇剑, 王卉宇, 汤奕, 等. 基于深度强化学习的居民实时自治最优能量管理策略[J]. 电力系统自动化, 2022, 46(1):110-119., articleTitle=基于深度强化学习的居民实时自治最优能量管理策略, refAbstract=null), Reference(id=1200501572322054655, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=46, issue=1, pageStart=110, pageEnd=119, url=null, language=null, rfNumber=[14], rfOrder=24, authorNames=YE Yujian, WANG Huiyu, TANG Yi, journalName=Automation of Electric Power System, refType=null, unstructuredReference=YE Yujian, WANG Huiyu, TANG Yi, et al. Real-time autonomous optimal energy management strategy for residents based on deep reinforcement learning[J]. Automation of Electric Power System, 2022, 46(1):110-119., articleTitle=Real-time autonomous optimal energy management strategy for residents based on deep reinforcement learning, refAbstract=null), Reference(id=1200501572510798340, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=43, issue=8, pageStart=22, pageEnd=32, url=null, language=null, rfNumber=[15], rfOrder=25, authorNames=闫庆友, 史超凡, 秦光宇, journalName=电力建设, refType=null, unstructuredReference=闫庆友, 史超凡, 秦光宇, 等. 基于近端策略优化算法的电化学/氢混合储能系统双层配置及运行优化[J]. 电力建设, 2022, 43(8):22-32., articleTitle=基于近端策略优化算法的电化学/氢混合储能系统双层配置及运行优化, refAbstract=null), Reference(id=1200501572636627463, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=43, issue=8, pageStart=22, pageEnd=32, url=null, language=null, rfNumber=[15], rfOrder=26, authorNames=YAN Qingyou, SHI Chaofan, QIN Guangyu, journalName=Electric Power Construction, refType=null, unstructuredReference=YAN Qingyou, SHI Chaofan, QIN Guangyu, et al. Research on two-layer configuration and operation optimization based on proximal policy optimization for electrochemical/hydrogen hybrid energy storage system[J]. Electric Power Construction, 2022, 43(8):22-32., articleTitle=Research on two-layer configuration and operation optimization based on proximal policy optimization for electrochemical/hydrogen hybrid energy storage system, refAbstract=null), Reference(id=1200501572766650891, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2020, volume=46, issue=7, pageStart=2286, pageEnd=2296, url=null, language=null, rfNumber=[16], rfOrder=27, authorNames=李咸善, 陈敏睿, 程杉, journalName=高电压技术, refType=null, unstructuredReference=李咸善, 陈敏睿, 程杉, 等. 基于双重激励协同博弈的含电动汽车微电网优化调度策略[J]. 高电压技术, 2020, 46(7):2286-2296., articleTitle=基于双重激励协同博弈的含电动汽车微电网优化调度策略, refAbstract=null), Reference(id=1200501572913451536, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2020, volume=46, issue=7, pageStart=2286, pageEnd=2296, url=null, language=null, rfNumber=[16], rfOrder=28, authorNames=LI Xianshan, CHEN Minrui, CHENG Shan, journalName=High Voltage Engineering, refType=null, unstructuredReference=LI Xianshan, CHEN Minrui, CHENG Shan, et al. Research on optimal scheduling strategy of microgrid with electric vehicles based on dual incentive cooperative game[J]. High Voltage Engineering, 2020, 46(7):2286-2296., articleTitle=Research on optimal scheduling strategy of microgrid with electric vehicles based on dual incentive cooperative game, refAbstract=null), Reference(id=1200501573098000918, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2021, volume=45, issue=12, pageStart=4729, pageEnd=4738, url=null, language=null, rfNumber=[17], rfOrder=29, authorNames=董雷, 刘雨, 乔骥, journalName=电网技术, refType=null, unstructuredReference=董雷, 刘雨, 乔骥, 等. 基于多智能体深度强化学习的电热联合系统优化运行[J]. 电网技术, 2021, 45(12):4729-4738., articleTitle=基于多智能体深度强化学习的电热联合系统优化运行, refAbstract=null), Reference(id=1200501573215441433, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2021, volume=45, issue=12, pageStart=4729, pageEnd=4738, url=null, language=null, rfNumber=[17], rfOrder=30, authorNames=DONG Lei, LIU Yu, QIAO Ji, journalName=Power System Technology, refType=null, unstructuredReference=DONG Lei, LIU Yu, QIAO Ji, et al. Optimal dispatch of combined heat and power system based on multi-agent deep reinforcement learning[J]. Power System Technology, 2021, 45(12):4729-4738., articleTitle=Optimal dispatch of combined heat and power system based on multi-agent deep reinforcement learning, refAbstract=null), Reference(id=1200501573324493341, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=52, issue=22, pageStart=38, pageEnd=43, url=null, language=null, rfNumber=[18], rfOrder=31, authorNames=于新海, 王鑫, 苏日古格, journalName=电气传动, refType=null, unstructuredReference=于新海, 王鑫, 苏日古格, 等. 基于GA和IPSO的微电网调度优化与对比分析[J]. 电气传动, 2022, 52(22):38-43., articleTitle=基于GA和IPSO的微电网调度优化与对比分析, refAbstract=null), Reference(id=1200501573496459810, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=52, issue=22, pageStart=38, pageEnd=43, url=null, language=null, rfNumber=[18], rfOrder=32, authorNames=YU Xinhai, WANG Xin, SU Riguge, journalName=Electric Drive, refType=null, unstructuredReference=YU Xinhai, WANG Xin, SU Riguge, et al. Microgrid dispatching optimization and comparative analysis based on GA and IPSO[J]. Electric Drive, 2022, 52(22):38-43., articleTitle=Microgrid dispatching optimization and comparative analysis based on GA and IPSO, refAbstract=null), Reference(id=1200501573592928805, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=16, issue=1, pageStart=117, pageEnd=126, url=null, language=null, rfNumber=[19], rfOrder=33, authorNames=陈卫东, 吴宁, 黄彦璐, journalName=南方电网技术, refType=null, unstructuredReference=陈卫东, 吴宁, 黄彦璐, 等. 基于深度学习的微电网优化调度辅助决策方法[J]. 南方电网技术, 2022, 16(1):117-126., articleTitle=基于深度学习的微电网优化调度辅助决策方法, refAbstract=null), Reference(id=1200501573710369324, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2022, volume=16, issue=1, pageStart=117, pageEnd=126, url=null, language=null, rfNumber=[19], rfOrder=34, authorNames=CHEN Weidong, WU Ning, HUANG Yanlu, journalName=Southern Power Technology, refType=null, unstructuredReference=CHEN Weidong, WU Ning, HUANG Yanlu, et al. Auxiliary decision-making method of optimal dispatching for microgrid based on deep learning[J]. Southern Power Technology, 2022, 16(1):117-126., articleTitle=Auxiliary decision-making method of optimal dispatching for microgrid based on deep learning, refAbstract=null), Reference(id=1200501573806838319, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2020, volume=44, issue=8, pageStart=2932, pageEnd=2940, url=null, language=null, rfNumber=[20], rfOrder=35, authorNames=袁桂丽, 贾新潮, 房方, journalName=电网技术, refType=null, unstructuredReference=袁桂丽, 贾新潮, 房方, 等. 虚拟电厂源荷双侧热电联合随机优化调度[J]. 电网技术, 2020, 44(8):2932-2940., articleTitle=虚拟电厂源荷双侧热电联合随机优化调度, refAbstract=null), Reference(id=1200501575010603570, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2020, volume=44, issue=8, pageStart=2932, pageEnd=2940, url=null, language=null, rfNumber=[20], rfOrder=36, authorNames=YUAN Guili, JIA Xinchao, FANG Fang, journalName=Power System Technology, refType=null, unstructuredReference=YUAN Guili, JIA Xinchao, FANG Fang, et al. Joint stochastic optimal scheduling of heat and power considering source and load sides of virtual power plant[J]. Power System Technology, 2020, 44(8):2932-2940., articleTitle=Joint stochastic optimal scheduling of heat and power considering source and load sides of virtual power plant, refAbstract=null), Reference(id=1200501575111266869, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2023, volume=38, issue=7, pageStart=1864, pageEnd=1881, url=null, language=null, rfNumber=[21], rfOrder=37, authorNames=陈明昊, 孙毅, 谢志远, journalName=电工技术学报, refType=null, unstructuredReference=陈明昊, 孙毅, 谢志远. 基于双层深度强化学习的园区综合能源系统多时间尺度优化管理[J]. 电工技术学报, 2023, 38(7):1864-1881., articleTitle=基于双层深度强化学习的园区综合能源系统多时间尺度优化管理, refAbstract=null), Reference(id=1200501575203541559, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, doi=null, pmid=null, pmcid=null, year=2023, volume=38, issue=7, pageStart=1864, pageEnd=1881, url=null, language=null, rfNumber=[21], rfOrder=38, authorNames=CHEN Minghao, SUN Yi, XIE Zhiyuan, journalName=Transactions of China Electrotechnical Society, refType=null, unstructuredReference=CHEN Minghao, SUN Yi, XIE Zhiyuan. The multi-time-scale management optimization mehod for park integrated energy system based on the bi-layer deep reinforcement learning[J]. Transactions of China Electrotechnical Society, 2023, 38(7):1864-1881., articleTitle=The multi-time-scale management optimization mehod for park integrated energy system based on the bi-layer deep reinforcement learning, refAbstract=null)], funds=[Fund(id=1200501568949834113, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, awardId=52077083, language=CN, fundingSource=国家自然科学基金(52077083), fundOrder=null, country=null)], companyList=[AuthorCompany(id=1200501564453540003, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, xref=1, ext=[AuthorCompanyExt(id=1200501564461928613, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564453540003, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 Chifeng Power Supply Company of State Grid East Inner Mongolia Power Co.,Ltd.,Chifeng 024000,Nei Monggol,China), AuthorCompanyExt(id=1200501564470317222, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564453540003, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 国网内蒙古东部电力有限公司赤峰供电公司,内蒙古 赤峰 024000)]), AuthorCompany(id=1200501564575174828, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, xref=2, ext=[AuthorCompanyExt(id=1200501564583563437, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564575174828, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 School of Electric Power Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China), AuthorCompanyExt(id=1200501564591952046, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, companyId=1200501564575174828, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 华南理工大学 电力学院,广东 广州 510640)])], figs=[ArticleFig(id=1200501567133700386, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, label=Fig.1, caption=Reinforcement learning framework of microgrid optimal dispatch, figureFileSmall=fhMuEIEakWQuo4nNNDN8Pw==, figureFileBig=K4uBXgcxboKOdKsSYiWgWg==, tableContent=null), ArticleFig(id=1200501567221780777, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, label=图1, caption=微网优化调度的强化学习框架, figureFileSmall=fhMuEIEakWQuo4nNNDN8Pw==, figureFileBig=K4uBXgcxboKOdKsSYiWgWg==, tableContent=null), ArticleFig(id=1200501567339221299, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, label=Fig.2, caption=Solution flow based on PPO, figureFileSmall=4EoLB8/k2tjCW7R7kJdsYQ==, figureFileBig=h+qno4bwfQWBs1CyeVZntg==, tableContent=null), ArticleFig(id=1200501567444078902, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, label=图2, caption=基于PPO算法的求解流程, figureFileSmall=4EoLB8/k2tjCW7R7kJdsYQ==, figureFileBig=h+qno4bwfQWBs1CyeVZntg==, tableContent=null), ArticleFig(id=1200501567561519420, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, label=Fig.3, caption=Mathematical model of neuron network and neuron, figureFileSmall=/pO0RX8Id7B1JejrD9SZqg==, figureFileBig=lWjWhvaaO/fKvrI4xKEk/g==, tableContent=null), ArticleFig(id=1200501567657988419, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, label=图3, caption=神经网络及神经元数学模型, figureFileSmall=/pO0RX8Id7B1JejrD9SZqg==, figureFileBig=lWjWhvaaO/fKvrI4xKEk/g==, tableContent=null), ArticleFig(id=1200501567750263109, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, label=Fig.4, caption=PV and load data of a typical day, figureFileSmall=dtuGayCJ1d8TGauDAjEeJQ==, figureFileBig=XAkP+AGRChVouARdCgXUCQ==, tableContent=null), ArticleFig(id=1200501567871897928, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, label=图4, caption=典型日光伏和负荷数据, figureFileSmall=dtuGayCJ1d8TGauDAjEeJQ==, figureFileBig=XAkP+AGRChVouARdCgXUCQ==, tableContent=null), ArticleFig(id=1200501567980949839, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, label=Fig.5, caption=Reward change curve, figureFileSmall=fuv4Zshzx/1auClt0yRLSA==, figureFileBig=jzpqkBpDd1nt68g60OSDYA==, tableContent=null), ArticleFig(id=1200501568060641620, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, label=图5, caption=奖励变化曲线, figureFileSmall=fuv4Zshzx/1auClt0yRLSA==, figureFileBig=jzpqkBpDd1nt68g60OSDYA==, tableContent=null), ArticleFig(id=1200501568161304922, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, label=Fig.6, caption=Optimization dispatch results of microgrid, figureFileSmall=Vpa/2NN0duf19ywlde7yrA==, figureFileBig=nzgPLQwTekUswyNeP9PqsQ==, tableContent=null), ArticleFig(id=1200501568261968227, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, label=图6, caption=微网优化调度结果, figureFileSmall=Vpa/2NN0duf19ywlde7yrA==, figureFileBig=nzgPLQwTekUswyNeP9PqsQ==, tableContent=null), ArticleFig(id=1200501568350048616, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, label=Fig.7, caption=Comparison of optimal cost of different methods, figureFileSmall=SrhJPI2K026/LEiP0/GORA==, figureFileBig=qb8eSA3zYfVz0c4n0M7Lfw==, tableContent=null), ArticleFig(id=1200501568459100526, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, label=图7, caption=不同方法的优化成本对比, figureFileSmall=SrhJPI2K026/LEiP0/GORA==, figureFileBig=qb8eSA3zYfVz0c4n0M7Lfw==, tableContent=null), ArticleFig(id=1200501568555569523, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=EN, label=Tab.1, caption=

The equipment paraments

, figureFileSmall=null, figureFileBig=null, tableContent=
设备 参数 数值
燃气轮机 功率上/下限 1 200/100 kW
爬坡功率上/下限 150/-150 kW
成本系数 0.33
电储能 功率上/下限 600/-600 kW
荷电状态上下限 0.9/0.1
充放电效率 0.97
度电成本系数 0.15
), ArticleFig(id=1200501568673010040, tenantId=1146029695717560320, journalId=1189987059142926344, articleId=1200482408840687769, language=CN, label=表1, caption=

设备参数

, figureFileSmall=null, figureFileBig=null, tableContent=
设备 参数 数值
燃气轮机 功率上/下限 1 200/100 kW
爬坡功率上/下限 150/-150 kW
成本系数 0.33
电储能 功率上/下限 600/-600 kW
荷电状态上下限 0.9/0.1
充放电效率 0.97
度电成本系数 0.15
)], attaches=null, journal=Journal(id=1149695973913305096, delFlag=0, nameCn=电气传动, nameEn=Electric Drive, nameHistory1=null, nameHistory2=null, issn=1001-2095, eissn=null, cn=12-1067/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=MX9kQyxauhRIcWSHz1lsfg==, journalPrice=null, startedYear=null, abbrevIsoEn=null, journalRemark=null, publicationField=null, createdTime=1752038480501, updatedTime=1761735785885, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=E, firstLetterEn=E, subjectCode=Engineering, subjectName=Engineering, subjectCodeEn=Engineering, subjectNameEn=null, picCn=MX9kQyxauhRIcWSHz1lsfg==, picEn=R8RQJTk995s1dk9HSLANAQ==, jcr=null, cjcr=null, exts=[JournalExt(id=1190369420775096487, language=CN, name=电气传动, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1761735785907, updatedTime=1761735785907, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://tg.au365.cn/Login/index.html, submissionEditorUrl=https://tg.au365.cn/Login/index/cate/3.html, submissionReviewUrl=https://tg.au365.cn/Login/index/cate/2.html, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1190369420829622440, language=EN, name=Electric Drive, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1761735785920, updatedTime=1761735785920, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://tg.au365.cn/Login/index.html, submissionEditorUrl=https://tg.au365.cn/Login/index/cate/3.html, submissionReviewUrl=https://tg.au365.cn/Login/index/cate/2.html, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1189987059142926344, websiteList=[Website(id=1189988338468713189, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1189987059142926344, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/dqcd/CN, language=CN, createTime=1761644928805, createBy=18614031015, updateTime=1761644944247, updateBy=18614031015, name=电气传动-中文, tplId=1146099689490845704, title=电气传动, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1190236467705254391, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=articleTextType, value=kx, createTime=1761704087425, updateTime=1761704087425, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467671699956, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=banner, value=null, createTime=1761704087417, updateTime=1761704087417, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467763974650, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=grayFlag, value=0, createTime=1761704087439, updateTime=1761704087439, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467663311347, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=logo, value=https://castjournals.cast.org.cn/joweb/dqcd/CN/file/pic?fileId=C+51VGe7It8gHMdS0MAlXg==, createTime=1761704087415, updateTime=1761704087415, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467784946172, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=minRunFlag, value=0, createTime=1761704087444, updateTime=1761704087444, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467692671478, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/dqcd/CN/file/pic, createTime=1761704087422, updateTime=1761704087422, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467776557563, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=silenceFlag, value=0, createTime=1761704087442, updateTime=1761704087442, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467680088565, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1761704087419, updateTime=1761704087419, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467713643000, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=themeColor, value=null, createTime=1761704087427, updateTime=1761704087427, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236467726225913, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338468713189, code=themeStyle, value=null, createTime=1761704087430, updateTime=1761704087430, creator=18614031015, updator=18614031015)]), Website(id=1189988338527433447, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1189987059142926344, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/dqcd/EN, language=EN, createTime=1761644928819, createBy=18614031015, updateTime=1761644963810, updateBy=18614031015, name=电气传动-英文, tplId=1146101810881728533, title=Electric Drive, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1190236496637563393, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=articleTextType, value=kx, createTime=1761704094323, updateTime=1761704094323, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496612397566, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=banner, value=null, createTime=1761704094317, updateTime=1761704094317, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496666923524, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=grayFlag, value=0, createTime=1761704094330, updateTime=1761704094330, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496604008957, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=logo, value=https://castjournals.cast.org.cn/joweb/dqcd/EN/file/pic?fileId=C+51VGe7It8gHMdS0MAlXg==, createTime=1761704094315, updateTime=1761704094315, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496683700742, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=minRunFlag, value=0, createTime=1761704094334, updateTime=1761704094334, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496629174784, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/dqcd/EN/file/pic, createTime=1761704094321, updateTime=1761704094321, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496675312133, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=silenceFlag, value=0, createTime=1761704094332, updateTime=1761704094332, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496620786175, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1761704094319, updateTime=1761704094319, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496650146306, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=themeColor, value=null, createTime=1761704094326, updateTime=1761704094326, creator=18614031015, updator=18614031015), WebsiteProps(id=1190236496658534915, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1189988338527433447, code=themeStyle, value=null, createTime=1761704094328, updateTime=1761704094328, creator=18614031015, updator=18614031015)])], journalTitle=电气传动, weixinUrl=null, journalUrl=https://www.au365.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Electric Drive, journalPhotoCn=MX9kQyxauhRIcWSHz1lsfg==, journalPhotoEn=R8RQJTk995s1dk9HSLANAQ==, journalFirstLetter=E, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false), detailUrlCn=https://castjournals.cast.org.cn/joweb/dqcd/CN/10.19457/j.1001-2095.dqcd24592, detailUrlEn=https://castjournals.cast.org.cn/joweb/dqcd/EN/10.19457/j.1001-2095.dqcd24592, pdfUrlCn=https://castjournals.cast.org.cn/joweb/dqcd/CN/PDF/10.19457/j.1001-2095.dqcd24592, pdfUrlEn=https://castjournals.cast.org.cn/joweb/dqcd/EN/PDF/10.19457/j.1001-2095.dqcd24592, aliStartDate=null, aliEndDate=null, collectionFlag=false, citedCount=null, citedUrl=null, reference=null)
收藏切换
基于近端策略优化算法的微网优化调度
收藏切换
PDF下载
马跃锋 1 , 招苏颀 1 , 王凯 2
电气传动 | 综合能源与现代电网 2024,54(6): 54-59
收起
收藏切换
电气传动 | 综合能源与现代电网 2024, 54(6): 54-59
基于近端策略优化算法的微网优化调度
全屏
马跃锋1 , 招苏颀1, 王凯2
作者信息
  • 1 国网内蒙古东部电力有限公司赤峰供电公司,内蒙古 赤峰 024000
  • 2 华南理工大学 电力学院,广东 广州 510640
  • 马跃锋(1985—),男,本科,工程师,研究方向为电力系统继电保护,Email:

Optimal Dispatch of Microgrid Based on Proximal Policy Optimization
Yuefeng MA1 , Suqi ZHAO1, Kai WANG2
Affiliations
  • 1 Chifeng Power Supply Company of State Grid East Inner Mongolia Power Co.,Ltd.,Chifeng 024000,Nei Monggol,China
  • 2 School of Electric Power Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China
出版时间: 2024-06-20 doi: 10.19457/j.1001-2095.dqcd24592
文章导航
收藏切换

微网是实现大量分布式电源集成到电网中的有效手段。针对微网优化调度问题,提出了一种基于近端策略优化算法的优化调度方法。首先,综合考虑微网运行成本和各类设备运行约束构建微网优化调度模型;其次,将该问题表述为强化学习框架,设计了强化学习状态、动作和奖励函数等要素;最后,设计了基于近端策略优化算法的求解流程,通过仿真验证了所提方法的有效性。

微电网  /  优化调度  /  人工智能  /  强化学习

Microgrid is an effective method to integrate a large number of distributed generators into the power grid. Aiming at the optimal dispatch problem of microgrid,an optimal dispatch method based on proximal policy optimization algorithm was proposed.Firstly,the optimal diapatch model of microgrid was constructed by considering the operation cost of microgrid and operation constraints of various equipment. Secondly,the problem was formulated as a reinforcement learning framework,and the elements of reinforcement learning such as state,action and reward function were designed. Finally,the solution flow based on the proximal policy optimization algorithm was designed,and the effectiveness of the proposed method was verified by simulation.

microgrid  /  optimal dispatch  /  artificial intelligence  /  reinforcement learning
马跃锋, 招苏颀, 王凯. 基于近端策略优化算法的微网优化调度. 电气传动, 2024 , 54 (6) : 54 -59 . DOI: 10.19457/j.1001-2095.dqcd24592
Yuefeng MA, Suqi ZHAO, Kai WANG. Optimal Dispatch of Microgrid Based on Proximal Policy Optimization[J]. Electric Drive, 2024 , 54 (6) : 54 -59 . DOI: 10.19457/j.1001-2095.dqcd24592
分布式发电具有明显的随机性、波动性,近年来大规模分布式电源接入电网,给电网的调度带来了极大的挑战[1-2]。微电网是各种分布式电源、储能装置、负荷和能量转换设备整合而成的小型发配电系统,既可独立运行,也可并网运行作为大电网的有益补充[3]。微网因其环境友好性、运行方式灵活等优点得到了广泛的研究。文献[4]以多微网系统为领导者、各微网负荷聚合商为跟随者建立一主多从的主从博弈优化调度模型,采用混沌粒子群算法嵌套gurobi求解器求解以上模型;文献[5]以微网总运行成本最小为目标建立优化调度模型,采用基于天牛群搜索算法的改进智能算法对模型进行求解;文献[6]采用协同进化算法求解微网经济调度模型,有效降低了微网运行成本。上述方法均为启发式算法,易陷入局部最优解。
随着人工智能技术的快速发展,基于数据驱动的强化学习方法为微网优化调度提供了新的求解思路[7]。强化学习中智能体以试错的方式对环境不断探索,以整个训练回合奖励最大化为目标逐渐学习得到最优策略,这与微网优化调度以整个调度周期运行成本最低为目标是契合的[8]。文献[9-11]采用Q学习方法求解优化调度策略,但需将各电源功率离散化,离散化不仅会带来调度误差,而且随着变量的增多易出现“维数灾难”。文献[12-13]均采用深度强化学习方法进行优化调度,采用神经网络进行决策,智能体通过更新神经网络参数的形式探索得到最优调度决策。近端策略优化算法(proximal policy optimization,PPO)[14]是一种典型的深度强化学习方法,具有收敛速度快、数据利用率高等优点。文献[15]提出基于PPO算法的家庭实时能量管理策略,在最优化居民日用电成本的同时提升了对不确定性光伏发电的适应性;文献[16]利用PPO算法有效降低了储能系统运行成本。
本文针对包含光伏、电储能、燃气轮机、负荷的典型微网系统的日前优化调度问题,首先以日运行成本最低为目标构建调度模型;然后采用PPO算法进行日前优化调度策略求解,并对仿真结果进行了分析;最后通过与粒子群算法进行对比验证了所提方法的有效性。
本文所考虑的微网系统包括燃气轮机、电储能、光伏、负荷,微网优化目标为日运行成本最低,如下式所示:
m i n t = 1 T [ C M T ( t ) + C E S ( t ) + C b u y ( t ) ]
式中:T为调度周期时段数; C M T ( t ) C E S ( t )分别为微燃气轮机运行成本、电储能运行成本; C b u y ( t )为外网购电成本。
微燃气轮机运行成本与发电功率呈线性关系:
C M T ( t ) = α P M T ( t )
式中: α为成本系数; P M T ( t )t时刻发电功率。
对于电储能考虑其度电成本,度电成本表达式为
C E S ( t ) = ρ P E S ( t ) |
式中: ρ为度电成本系数; P E S ( t )t时刻储能充/放电功率,为正时处于放电状态,为负时处于充电状态。
微网系统从主网的购电成本如下式所示:
C b u y ( t ) = λ b u y ( t ) P b u y ( t )           P b u y ( t ) 0 λ s e l l ( t ) P b u y ( t )           P b u y ( t ) 0
式中: P b u y ( t )t时刻从主网的购电成本; λ b u y ( t ) λ s e l l ( t )分别为t时刻主网购电、售电价格。
微网的约束包括功率平衡约束和设备运行约束。
功率平衡约束。在时段t微网内电能供应与负荷须达到平衡,如下式所示:
P P V ( t ) + P M T ( t ) + P E S ( t ) + P b u y ( t ) = P l o a d ( t )
式中: P P V ( t ) P l o a d ( t )分别为t时段微网内光伏、总负荷功率。
设备运行约束。微网中各设备均有运行上、下限范围,燃气轮机运行约束如下式:
P M T m i n P M T ( t ) P M T m a x
- R d o w n P M T ( t ) - P M T ( t - 1 ) R u p
式中: P M T m a x P M T m i n分别为燃气轮机输出功率的上、下限; R u p R d o w n分别为燃气轮机最大向上、向下爬坡功率。
电储能运行约束如下式所示:
- P d i s m a x P E S ( t ) P c h m a x
式中: P c h m a x P d i s m a x分别为最大充、放电功率。
为避免过度充放电对电储能设备造成的损害,需将电储能荷电状态 S O C ( t )限制在一定范围内:
S O C m i n S O C ( t ) S O C m a x
其中
S O C ( t + 1 ) = S O C ( t ) + P E S ( t ) d t × η / E
式中: S O C m a x S O C m i n分别为最大、最小荷电状态;E为电储能容量; η为电储能的充放电效率。
微网优化调度的强化学习框架如图1所示,其中智能体由深度神经网络构成,具有调度决策能力。在调度时刻t,具有决策能力的智能体根据微网系统状态st探索性做出调度决策at,并获得下一时刻微网系统给出的奖励rt,智能体进入下一时刻状态。智能体不断对微网系统探索,根据经验调整调度决策,使一个调度周期累积奖励之和最大化。
图1中所示状态、动作和奖励函数等强化学习要素设计如下:
1)状态ststt时刻智能体接收到的微网系统运行状态,包括光伏发电功率、负荷功率、上一时刻微燃气轮机功率和电储能荷电状态,状态st可表示如下:
s t = { P P V ( t ) , P l o a d ( t ) , P M T ( t - 1 ) , S O C ( t ) }
2)动作at。微网优化调度问题的目标是最优地确定其微燃气轮机和电储能功率,动作at即为决策变量,可表示为
a t = { P M T ( t ) , P E S ( t ) }
3)奖励函数rt。微网优化问题优化目标为最小化整个调度周期运行成本,而强化学习以整个调度周期获得的奖励最大化为更新方向。因此,智能体在时刻t获得的奖励可表示为该时刻负的运行成本:
r t = - [ C M T ( t ) + C E S ( t ) + C b u y ( t ) ] + r 0
式中:r0为偏置常数,使奖励函数由负转正,加快收敛速度。
智能体的最终目标为寻找一策略 π *,使其获得的累积奖励最大:
π * = a r g m a x π E ( t = 0 γ t r t )
其中, γ为折扣因子,表示当前所选动作对未来收益的影响程度, γ [ 0,1 ]。由于本文的优化目标为最小化整个调度周期的运行成本,因此 γ取值为1。
针对上述优化调度模型,设计其基于PPO算法的求解流程如图2所示。该方法中智能体不断对微网优化调度策略进行探索,以调度成本最优为目标指导神经网络参数更新,直到找到最优策略。具体流程如下:
1)随机初始化智能体神经网络参数 θ,包括如图3所示的神经网络中每个神经元的权重系数和偏置项;
2)初始化环境,即确定式(10)中各变量的初始值;
3)在调度时刻t,智能体神经网络输入st,输出调度决策at,并获得下一时刻状态st+1和奖励rt。判断t是否达到调度周期时段数,如果没有则继续该步骤,否则进入神经网络更新;
4)更新智能体神经网络参数 θ。PPO算法除采用参数为 θ的策略网络进行调度决策外,还采用价值网络对策略网络所采取策略进行评估,以指导策略网络更新。价值网络输入st,输出价值函数Vst),Vst)表示当前状态下可获取的累积奖励的期望,即
V ( s t ) = r t + γ r t + 1 + + γ T - t r T
式中:T为决策序列的长度。
为更准确地对Vst)进行估计,价值网络的更新目标为最小化时序差分误差:
m i n [ γ V ( s t + 1 ) + r t - V ( s t ) ]
n次迭代时策略网络以梯度下降法进行更新,更新的目标函数为
L ( θ ) = E t { m i n { r t ( θ ) A π ( s t , a t ) , c l i p [ r t ( θ ) , 1 - ε , 1 + ε ] A π ( s t , a t ) } }
其中
A π ( s t , a t ) = r t + γ r t + 1 + + γ T - t + 1 r T - 1 + γ T - t V ( s T ) - V ( s t )
式中:Et(·)为期望函数; r t ( θ )为新策略 π θ n和旧策略 π θ n - 1的比值; ε为超参数; A π ( s t , a t )为优势函数,表示状态st下选取动作at相对平均动作的优势。
clip函数将新旧策略比值限定在[ 1 - ε , 1 + ε],防止了训练过程中智能体获得的奖励波动过大,其具体数学表达为
c l i p ( x , a , b ) = x         a x b a         x a b         x b
5)判断是否达到最大训练轮数N,如果达到则结束训练,否则返回步骤2)。
本文以成都市某地区微网为例进行仿真研究,该地区某一典型单日的光伏、负荷功率如图4所示。一天共设置24个调度时刻,各个时刻的分时电价参考文献[17]。微网中设备参数如表1所示,本文假设光伏发电成本为0。
对于智能体策略网络设置4层隐藏层,每层神经元的个数往往按经验选取,本文参考文献[18]的设计,从输入到输出层每层神经元数量分别为128,128,32和2;对智能体价值网络也设置4层隐藏层,由于价值网络仅输出Vst),因此其与策略网络不同的是输出层神经元数量为1,从输入到输出层每层神经元数量分别为128,128,32和1。神经元激活函数均为Tanh,该函数为双曲正切函数,其输出均值为0,相较选择其他函数具有更快的收敛速度;超参数 ε设置为0.01;最大训练轮数N为1 700;奖励偏置常数设为1 200。
本文仿真程序基于Pytorch框架编写,硬件条件为i7 11 800H CPU,2.3 GHz,24 G内存。
智能体更新过程中每个调度周期所获得的奖励之和变化规律如图5所示,奖励值最终收敛于8 700。训练开始时,奖励变化幅值变化较大,这是因为智能体尚未寻得最优策略,尚处于探索阶段;随着智能体神经网络更新,智能体逐渐探索出最优策略,因此奖励趋于收敛。
智能体对该日的优化调度结果如图6所示。可见在01:00—04:00电价低谷时刻,系统通过外网购电给电储能充电,以用于其他电价较高时刻负荷供电,降低购电成本和燃气轮机发电成本。在13:00—17:00,光伏发电功率高于负荷功率,此时燃气轮机发电成本始终维持在下限,多余的发电功率全部存储于储能中,有效提升了可再生能源利用率。
为了验证本文所提出的基于PPO的微网优化调度方法的有效性,将本文方法与文献[19]提出的改进粒子群算法(improved particle swarm optimization,IPSO)、文献[20]提出的基于深度双向长短期记忆网络(bidirectional long-short term memory,Bi-LSTM)的优化调度方法和CPLEX优化器进行对比。多次对该日进行优化调度,3种方法的最优运行成本如图7所示。
图7可见,CPLEX优化结果始终保持不变,本文方法由于每次需重新更新神经网络参数,其结果具有波动性,但经过大量的训练,其调度结果始终与最优的CPLEX基本一致。本文方法下第3次优化结果最优,仅比最优的运行成本多0.001 8%。
由于基于Bi-LSTM的优化调度方法需大量样本进行神经网络的训练,本文在图4所示光伏、负荷功率基础上叠加服从正态分布的随机性[21],生成100个样本,利用求解器求解各样本对应的最优调度结果。通过100个样本对深度神经网络进行训练,可见经过训练后其优化成本始终高于本文所提方法,原因是基于Bi-LSTM的优化调度方法的决策精度取决于样本容量大小,当实际调度情况与训练样本差异较大时,其优化结果会与最优解产生偏差。
IPSO方法容易陷入局部最优解,本文方法优化成本始终低于IPSO方法,验证了本文所提方法的有效性。
针对微网优化调度问题,本文提出了一种基于PPO算法的优化调度方法。首先构建了以微网运行经济性为目标的优化模型,并结合深度强化学习框架,对微网优化智能体的状态、动作和奖励进行设计,在此基础上设计了优化调度算法流程,通过实验仿真结果可知本文优化方法优于粒子群算法。本文为人工智能方法在电力系统中的应用提供了参考。
  • 国家自然科学基金(52077083)
参考文献 引证文献
排序方式:
[1]
吕天光. 含多微网配电系统多元主体互动运行策略研究[D]. 上海: 上海交通大学, 2018.
Tianguang. Study on interactive operation strategies for multiple agents in power distribution systems with multi-microgrids[D]. Shanghai: Shanghai Jiao Tong University, 2018.
[2]
沈儒茹. 多微网系统的优化调度策略研究[D]. 哈尔滨: 哈尔滨工业大学, 2020.
SHEN Ruru. Research on optimal scheduling strategy of multi- microgrid system[D]. Harbin: Harbin Industrial University, 2020.
[3]
帅轩越, 王秀丽, 吴雄, 等. 计及条件风险价值下基于合作博弈的多微网协同优化调度[J]. 电网技术, 2022, 46(1):130-138.
SHUAI Xuanyue, WANG Xiuli, WU Xiong, et al. Cooperative optimal scheduling of multi-microgrids based on cooperative game considering conditional value at risk[J]. Power System Technology, 2022, 46(1):130-138.
[4]
李鹏, 吴迪凡, 李雨薇, 等. 基于综合需求响应和主从博弈的多微网综合能源系统优化调度策略[J]. 中国电机工程学报, 2021, 41(4):1307-1321,1538.
LI Peng, WU Difan, LI Yuwei, et al. Optimal dispatch of multi-microgrids integrated energy system based on integrated demand response and stackelberg game[J]. Proceedings of the CSEE, 2021, 41(4):1307-1321,1538.
[5]
许琼果, 张炜焘, 刘光伟, 等. 计及储能调节特性的直流微电网优化调度研究[J]. 电气传动, 2022, 52(15):53-60.
XU Qiongguo, ZHANG Weitao, LIU Guangwei, et al. Research on optimal scheduling of DC microgrid considering energy sto-rage regulation characteristics[J]. Electric Drive, 2022, 52(15):53-60.
[6]
宋晓英, 王艳松. 基于协同进化遗传算法的微网经济环保调度[J]. 电力系统保护与控制, 2014, 42(5):85-89.
SONG Xiaoying, WANG Yansong. Economic and environmental dispatch of microgrid using co-evolutionary genetic algorithm[J]. Power System Protection and Control, 2014, 42(5):85-89.
[7]
WANG L, LAN X G, ZHANG H B. The theory application of deep reinforcement learning[J]. Pattern Recognition and Artificial Intelligence, 2019, 32(1):67-81.
[8]
熊珞琳, 毛帅, 唐漾, 等. 基于强化学习的综合能源系统管理综述[J]. 自动化学报, 2021, 47(10):2321-2340.
XIONG Luolin, MAO Shuai, TANG Yang, et al. Reinforcement learning based integrated energy system management:a survey[J]. Acta Automatica Sinica, 2021, 47(10):2321-2340.
[9]
吴利刚, 张梁, 周倩, 等. 基于强化学习的微电网能量调度优化策略研究[J]. 控制工程, 2022, 29(7):1162-1172.
WU Ligang, ZHANG Liang, ZHOU Qian, et al. Research on energy scheduling optimization strategy of micro-grid based on reinforcement learning[J]. Control Engineering of China, 2022, 29(7):1162-1172.
[10]
邢敖, 李云. 面向主动配电网的调峰需求系统调度学习优化[J]. 自动化与仪器仪表, 2021(3):150-153,157.
XING Ao, LI Yun. System scheduling learning optimization for peak load regulation in active distribution network[J]. Automation & Instrumentation, 2021(3):150-153,157.
[11]
谢启跃, 应雨龙. 基于深度确定性策略梯度的电-气综合能源微网优化调度研究[J]. 热力发电, 2022, 51(3):141-147.
XIE Qiyue, YING Yulong. Research on optimal dispatch of electricity-gas integrated energy microgrid based on deep deterministic policy gradient[J]. Thermal Power Generation, 2022, 51(3):141-147.
[12]
MOCANU E, MOCANU D C, NGUYEN P H, et al. On-line building energy optimization using deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2019, 10(4):3698-3708.
[13]
ZHANG J W, ZHANG Z H, HAN S, et al. Proximal policy optimization via enhanced exploration efficiency[J]. Information Sciences, 2022,609:750-765.
[14]
叶宇剑, 王卉宇, 汤奕, 等. 基于深度强化学习的居民实时自治最优能量管理策略[J]. 电力系统自动化, 2022, 46(1):110-119.
YE Yujian, WANG Huiyu, TANG Yi, et al. Real-time autonomous optimal energy management strategy for residents based on deep reinforcement learning[J]. Automation of Electric Power System, 2022, 46(1):110-119.
[15]
闫庆友, 史超凡, 秦光宇, 等. 基于近端策略优化算法的电化学/氢混合储能系统双层配置及运行优化[J]. 电力建设, 2022, 43(8):22-32.
YAN Qingyou, SHI Chaofan, QIN Guangyu, et al. Research on two-layer configuration and operation optimization based on proximal policy optimization for electrochemical/hydrogen hybrid energy storage system[J]. Electric Power Construction, 2022, 43(8):22-32.
[16]
李咸善, 陈敏睿, 程杉, 等. 基于双重激励协同博弈的含电动汽车微电网优化调度策略[J]. 高电压技术, 2020, 46(7):2286-2296.
LI Xianshan, CHEN Minrui, CHENG Shan, et al. Research on optimal scheduling strategy of microgrid with electric vehicles based on dual incentive cooperative game[J]. High Voltage Engineering, 2020, 46(7):2286-2296.
[17]
董雷, 刘雨, 乔骥, 等. 基于多智能体深度强化学习的电热联合系统优化运行[J]. 电网技术, 2021, 45(12):4729-4738.
DONG Lei, LIU Yu, QIAO Ji, et al. Optimal dispatch of combined heat and power system based on multi-agent deep reinforcement learning[J]. Power System Technology, 2021, 45(12):4729-4738.
[18]
于新海, 王鑫, 苏日古格, 等. 基于GA和IPSO的微电网调度优化与对比分析[J]. 电气传动, 2022, 52(22):38-43.
YU Xinhai, WANG Xin, SU Riguge, et al. Microgrid dispatching optimization and comparative analysis based on GA and IPSO[J]. Electric Drive, 2022, 52(22):38-43.
[19]
陈卫东, 吴宁, 黄彦璐, 等. 基于深度学习的微电网优化调度辅助决策方法[J]. 南方电网技术, 2022, 16(1):117-126.
CHEN Weidong, WU Ning, HUANG Yanlu, et al. Auxiliary decision-making method of optimal dispatching for microgrid based on deep learning[J]. Southern Power Technology, 2022, 16(1):117-126.
[20]
袁桂丽, 贾新潮, 房方, 等. 虚拟电厂源荷双侧热电联合随机优化调度[J]. 电网技术, 2020, 44(8):2932-2940.
YUAN Guili, JIA Xinchao, FANG Fang, et al. Joint stochastic optimal scheduling of heat and power considering source and load sides of virtual power plant[J]. Power System Technology, 2020, 44(8):2932-2940.
[21]
陈明昊, 孙毅, 谢志远. 基于双层深度强化学习的园区综合能源系统多时间尺度优化管理[J]. 电工技术学报, 2023, 38(7):1864-1881.
CHEN Minghao, SUN Yi, XIE Zhiyuan. The multi-time-scale management optimization mehod for park integrated energy system based on the bi-layer deep reinforcement learning[J]. Transactions of China Electrotechnical Society, 2023, 38(7):1864-1881.
2024年第54卷第6期
PDF下载
212
101
引用本文
BibTeX
文章信息
doi: 10.19457/j.1001-2095.dqcd24592
  • 接收时间:2022-08-29
  • 首发时间:2025-11-26
  • 出版时间:2024-06-20
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2022-08-29
  • 修回日期:2022-10-27
基金
国家自然科学基金(52077083)
作者信息
    1 国网内蒙古东部电力有限公司赤峰供电公司,内蒙古 赤峰 024000
    2 华南理工大学 电力学院,广东 广州 510640
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/dqcd/CN/10.19457/j.1001-2095.dqcd24592
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数

Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)

Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏