Article(id=1240655139355291924, tenantId=1146029695717560320, journalId=1240618002186551303, issueId=1240655131507740845, articleNumber=null, orderNo=null, doi=10.3969/j.issn.1000-4653.2025.03.016, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1725120000000, receivedDateStr=2024-09-01, revisedDate=null, revisedDateStr=null, acceptedDate=null, acceptedDateStr=null, onlineDate=1773724835480, onlineDateStr=2026-03-17, pubDate=1758729600000, pubDateStr=2025-09-25, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1773724835480, onlineIssueDateStr=2026-03-17, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1773724835480, creator=13701087609, updateTime=1773724835480, updator=13701087609, issue=Issue{id=1240655131507740845, tenantId=1146029695717560320, journalId=1240618002186551303, year='2025', volume='48', issue='3', pageStart='1', pageEnd='184', issueExtLink='null', onlineDate='null', pubDate='null', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1773724833608, creator=13701087609, updateTime=1773725188865, updator=13701087609, preIssue=null, nextIssue=null, ext={EN=IssueExt(id=1240656621609742880, tenantId=1146029695717560320, journalId=1240618002186551303, issueId=1240655131507740845, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1240656621613937185, tenantId=1146029695717560320, journalId=1240618002186551303, issueId=1240655131507740845, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null}, startPage=129, endPage=136, ext={EN=ArticleExt(id=1240655141804765560, articleId=1240655139355291924, tenantId=1146029695717560320, journalId=1240618002186551303, language=EN, title=Trajectory planning and control for unmanned surface vehicle based on deep reinforcement learning, columnId=1240655021340160960, journalTitle=Navigation of China, columnName=Intelligent Shipping, runingTitle=null, highlight=null, articleAbstract=

This study aims to apply deep reinforcement learning to address the challenges of trajectory planning and control for unmanned surface vehicles. In trajectory planning, the Q-learning algorithm is employed to generate trajectories in real-world aquatic environments. For the design of the reward function, factors such as shallow water areas are taken into account, with an emphasis on minimizing the number of turning points along the path. For trajectory tracking control, we integrate the Soft Actor-Critic (SAC) algorithm with the Proportional-Integral-Derivative (PID) control method to alleviate the difficulties of manual parameter tuning associated with conventional PID controllers. This hybrid approach also mitigates the interpretability limitations often found in pure deep reinforcement learning methods. Comparative experiments involving the traditional PID algorithm, Genetic Algorithm (GA), and Deep Deterministic Policy Gradient (DDPG) algorithm demonstrate the superiority of the proposed SAC-PID method. Simulation results show that the planned trajectories effectively incorporate multiple factors, including travel distance, shallow water regions, and number of turning point, the SAC-PID method achieves outstanding performance in trajectory tracking.

, correspAuthors=null, authorNote=null, correspAuthorsNote=null, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, authorCompany=null, fund=null, authors=null, authorsList=Wei GUAN, Zhaoyong XI, Zhewen CUI, Xianku ZHANG), CN=ArticleExt(id=1240655142589100472, articleId=1240655139355291924, tenantId=1146029695717560320, journalId=1240618002186551303, language=CN, title=基于深度强化学习的无人艇航迹规划与控制, columnId=1240655021528904650, journalTitle=中国航海, columnName=智能航运, runingTitle=null, highlight=null, articleAbstract=

本研究旨在运用强化学习方法解决无人艇航迹规划与控制问题。在航迹规划方面,采用Q学习(Q-learning)算法,针对真实水域进行航迹规划。在奖励函数设计中考虑了浅水区,并致力于减少航迹的转向点数量。在航迹控制方面,将柔性动作评价(SAC)算法与比例积分微分(PID)控制算法相结合,克服了传统PID控制器参数人工整定、调节困难的问题的同时,也规避了深度强化学习缺乏可解释性的缺点。通过与传统PID算法、遗传算法(GA)和深度确定性策略梯度(DDPG)进行对比试验,展现出所提出SAC-PID方法的优越性。仿真结果表明,所规划的航迹能够综合考虑航迹距离、浅水区、航路转向点数量等优化目标,所提出的SAC-PID方法能够很好实现航迹跟踪效果。

, correspAuthors=null, authorNote=null, correspAuthorsNote=
关巍(1982—),男,博士,教授,博士生导师,研究方向为船舶智能航行决策,船舶运动控制。E-mail:
, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=g9W02oDBMU7GIgTrH2WE2g==, magXml=VPjcDc4sKJUVkkfyOoHbuA==, pdfUrl=null, pdf=LtIpvcFJbgmHuIl2gxwPSg==, pdfFileSize=4430525, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=+Q9N1VCYrRZQutvslEd+jA==, mapNumber=null, authorCompany=null, fund=null, authors=null, authorsList=关巍, 奚赵勇, 崔哲闻, 张显库)}, authors=[Author(id=1240655143021113819, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=gwwtxdy@dlmu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1240655143104999904, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, authorId=1240655143021113819, language=EN, stringName=Wei GUAN, firstName=Wei, middleName=null, lastName=GUAN, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=Navigation College, Dalian Maritime University, Dalian 116026, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1240655143167914468, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, authorId=1240655143021113819, language=CN, stringName=关巍, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=大连海事大学 航海学院,辽宁 大连 116026, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1240655142933033424, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, xref=null, ext=[AuthorCompanyExt(id=1240655142941422034, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=Navigation College, Dalian Maritime University, Dalian 116026, China), AuthorCompanyExt(id=1240655142949810644, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=大连海事大学 航海学院,辽宁 大连 116026)])]), Author(id=1240655143235023339, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1240655143310520816, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, authorId=1240655143235023339, language=EN, stringName=Zhaoyong XI, firstName=Zhaoyong, middleName=null, lastName=XI, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=Navigation College, Dalian Maritime University, Dalian 116026, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1240655144774332913, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, authorId=1240655143235023339, language=CN, stringName=奚赵勇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=大连海事大学 航海学院,辽宁 大连 116026, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1240655142933033424, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, xref=null, ext=[AuthorCompanyExt(id=1240655142941422034, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=Navigation College, Dalian Maritime University, Dalian 116026, China), AuthorCompanyExt(id=1240655142949810644, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=大连海事大学 航海学院,辽宁 大连 116026)])]), Author(id=1240655144874996212, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1240655144950493689, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, authorId=1240655144874996212, language=EN, stringName=Zhewen CUI, firstName=Zhewen, middleName=null, lastName=CUI, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=Navigation College, Dalian Maritime University, Dalian 116026, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1240655145025991166, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, authorId=1240655144874996212, language=CN, stringName=崔哲闻, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=大连海事大学 航海学院,辽宁 大连 116026, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1240655142933033424, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, xref=null, ext=[AuthorCompanyExt(id=1240655142941422034, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=Navigation College, Dalian Maritime University, Dalian 116026, China), AuthorCompanyExt(id=1240655142949810644, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=大连海事大学 航海学院,辽宁 大连 116026)])]), Author(id=1240655145135043076, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1240655145256677899, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, authorId=1240655145135043076, language=EN, stringName=Xianku ZHANG, firstName=Xianku, middleName=null, lastName=ZHANG, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=Navigation College, Dalian Maritime University, Dalian 116026, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1240655145357341198, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, authorId=1240655145135043076, language=CN, stringName=张显库, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=大连海事大学 航海学院,辽宁 大连 116026, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1240655142933033424, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, xref=null, ext=[AuthorCompanyExt(id=1240655142941422034, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=Navigation College, Dalian Maritime University, Dalian 116026, China), AuthorCompanyExt(id=1240655142949810644, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=大连海事大学 航海学院,辽宁 大连 116026)])])], keywords=[Keyword(id=1240655145458004501, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, language=EN, orderNo=1, keyword=trajectory planning), Keyword(id=1240655145533501975, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, language=EN, orderNo=2, keyword=trajectory control), Keyword(id=1240655145604805145, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, language=EN, orderNo=3, keyword=deep reinforcement learning), Keyword(id=1240655145680302619, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, language=EN, orderNo=4, keyword=unmanned surface vehicle), Keyword(id=1240655145739022876, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, language=CN, orderNo=1, keyword=航迹规划), Keyword(id=1240655145797743135, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, language=CN, orderNo=2, keyword=航迹控制), Keyword(id=1240655145860657696, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, language=CN, orderNo=3, keyword=深度强化学习), Keyword(id=1240655145940349474, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, language=CN, orderNo=4, keyword=水面无人艇)], refs=null, funds=[Fund(id=1240655146045207076, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, awardId=52171342, language=CN, fundingSource=国家自然科学基金项目(52171342), fundOrder=null, country=null)], companyList=[AuthorCompany(id=1240655142933033424, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, xref=null, ext=[AuthorCompanyExt(id=1240655142941422034, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=Navigation College, Dalian Maritime University, Dalian 116026, China), AuthorCompanyExt(id=1240655142949810644, tenantId=1146029695717560320, journalId=1240618002186551303, articleId=1240655139355291924, companyId=1240655142933033424, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=大连海事大学 航海学院,辽宁 大连 116026)])], figs=null, attaches=null, journal=Journal(id=1240617866001694726, delFlag=0, nameCn=中国航海, nameEn=Navigation of China, nameHistory1=null, nameHistory2=null, issn=1000-4653, eissn=null, cn=10-1741/U, coden=null, periodic=1, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=noZCpe+7WzqFjMsPMw3Ylg==, journalPrice=null, startedYear=null, abbrevIsoEn=Navigation of China, journalRemark=null, publicationField=null, createdTime=1773715948823, updatedTime=1773716241174, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=N, firstLetterEn=N, subjectCode=Engineering, subjectName=null, subjectCodeEn=Engineering, subjectNameEn=null, picCn=noZCpe+7WzqFjMsPMw3Ylg==, picEn=h3SZYo8W+nhAJz8ZlQjXpw==, jcr=null, cjcr=null, exts=[JournalExt(id=1240619092323922506, language=CN, name=中国航海, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1773716241198, updatedTime=1773716241198, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=http://hhxhbjb.cinnet.cn/authorLogOn.action, submissionEditorUrl=http://hhxhbjb.cinnet.cn/editorLogOn.action, submissionReviewUrl=http://hhxhbjb.cinnet.cn/expertLogOn.action, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1240619092365865547, language=EN, name=Navigation of China, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1773716241208, updatedTime=1773716241208, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=http://hhxhbjb.cinnet.cn/authorLogOn.action, submissionEditorUrl=http://hhxhbjb.cinnet.cn/editorLogOn.action, submissionReviewUrl=http://hhxhbjb.cinnet.cn/expertLogOn.action, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1240618002186551303, websiteList=[Website(id=1240619203443618380, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1240618002186551303, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/zghh/CN, language=CN, createTime=1773716267691, createBy=18614031015, updateTime=1773716289156, updateBy=18614031015, name=中国航海-中文, tplId=1146099689490845704, title=中国航海, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1240620222206505565, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=articleTextType, value=kx, createTime=1773716510583, updateTime=1773716510583, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222185534042, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=banner, value=null, createTime=1773716510578, updateTime=1773716510578, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222223282784, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=grayFlag, value=0, createTime=1773716510587, updateTime=1773716510587, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222177145433, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=logo, value=https://castjournals.cast.org.cn/joweb/zghh/CN/file/pic?fileId=KvlDOS/lCoEQEn0ia9k/vw==, createTime=1773716510576, updateTime=1773716510576, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222244254306, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=minRunFlag, value=0, createTime=1773716510592, updateTime=1773716510592, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222198116956, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/zghh/CN/file/pic, createTime=1773716510581, updateTime=1773716510581, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222235865697, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=silenceFlag, value=0, createTime=1773716510590, updateTime=1773716510590, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222189728347, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1773716510579, updateTime=1773716510579, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222210699870, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=themeColor, value=null, createTime=1773716510584, updateTime=1773716510584, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620222219088479, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203443618380, code=themeStyle, value=null, createTime=1773716510586, updateTime=1773716510586, creator=18614031015, updator=18614031015)]), Website(id=1240619203519115854, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1240618002186551303, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/zghh/EN, language=EN, createTime=1773716267709, createBy=18614031015, updateTime=1773716307444, updateBy=18614031015, name=中国航海-英文, tplId=1146101810881728533, title=Navigation of China, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1240620320386765737, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=articleTextType, value=kx, createTime=1773716533991, updateTime=1773716533991, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320361599910, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=banner, value=null, createTime=1773716533985, updateTime=1773716533985, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320411931564, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=grayFlag, value=0, createTime=1773716533997, updateTime=1773716533997, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320353211301, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=logo, value=https://castjournals.cast.org.cn/joweb/zghh/EN/file/pic?fileId=KvlDOS/lCoEQEn0ia9k/vw==, createTime=1773716533983, updateTime=1773716533983, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320424514478, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=minRunFlag, value=0, createTime=1773716534000, updateTime=1773716534000, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320378377128, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/zghh/EN/file/pic, createTime=1773716533989, updateTime=1773716533989, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320420320173, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=silenceFlag, value=0, createTime=1773716533999, updateTime=1773716533999, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320369988519, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1773716533987, updateTime=1773716533987, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320395154346, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=themeColor, value=null, createTime=1773716533993, updateTime=1773716533993, creator=18614031015, updator=18614031015), WebsiteProps(id=1240620320403542955, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1240619203519115854, code=themeStyle, value=null, createTime=1773716533995, updateTime=1773716533995, creator=18614031015, updator=18614031015)])], journalTitle=中国航海, weixinUrl=null, journalUrl=https://zghh.cinnet.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Navigation of China, journalPhotoCn=noZCpe+7WzqFjMsPMw3Ylg==, journalPhotoEn=h3SZYo8W+nhAJz8ZlQjXpw==, journalFirstLetter=N, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false), detailUrlCn=https://castjournals.cast.org.cn/joweb/zghh/CN/10.3969/j.issn.1000-4653.2025.03.016, detailUrlEn=https://castjournals.cast.org.cn/joweb/zghh/EN/10.3969/j.issn.1000-4653.2025.03.016, pdfUrlCn=https://castjournals.cast.org.cn/joweb/zghh/CN/PDF/10.3969/j.issn.1000-4653.2025.03.016, pdfUrlEn=https://castjournals.cast.org.cn/joweb/zghh/EN/PDF/10.3969/j.issn.1000-4653.2025.03.016, aliStartDate=null, aliEndDate=null, collectionFlag=false, citedCount=null, citedUrl=null, reference=null)
收藏切换
基于深度强化学习的无人艇航迹规划与控制
收藏切换
PDF下载
关巍 , 奚赵勇 , 崔哲闻 , 张显库
中国航海 | 智能航运 2025,48(3): 129-136
收起
收藏切换
中国航海 | 智能航运 2025, 48(3): 129-136
基于深度强化学习的无人艇航迹规划与控制
全屏
关巍 , 奚赵勇, 崔哲闻, 张显库
作者信息
  • 大连海事大学 航海学院,辽宁 大连 116026

通讯作者:

关巍(1982—),男,博士,教授,博士生导师,研究方向为船舶智能航行决策,船舶运动控制。E-mail:
Trajectory planning and control for unmanned surface vehicle based on deep reinforcement learning
Wei GUAN , Zhaoyong XI, Zhewen CUI, Xianku ZHANG
Affiliations
  • Navigation College, Dalian Maritime University, Dalian 116026, China
出版时间: 2025-09-25 doi: 10.3969/j.issn.1000-4653.2025.03.016
文章导航
收藏切换

本研究旨在运用强化学习方法解决无人艇航迹规划与控制问题。在航迹规划方面,采用Q学习(Q-learning)算法,针对真实水域进行航迹规划。在奖励函数设计中考虑了浅水区,并致力于减少航迹的转向点数量。在航迹控制方面,将柔性动作评价(SAC)算法与比例积分微分(PID)控制算法相结合,克服了传统PID控制器参数人工整定、调节困难的问题的同时,也规避了深度强化学习缺乏可解释性的缺点。通过与传统PID算法、遗传算法(GA)和深度确定性策略梯度(DDPG)进行对比试验,展现出所提出SAC-PID方法的优越性。仿真结果表明,所规划的航迹能够综合考虑航迹距离、浅水区、航路转向点数量等优化目标,所提出的SAC-PID方法能够很好实现航迹跟踪效果。

航迹规划  /  航迹控制  /  深度强化学习  /  水面无人艇

This study aims to apply deep reinforcement learning to address the challenges of trajectory planning and control for unmanned surface vehicles. In trajectory planning, the Q-learning algorithm is employed to generate trajectories in real-world aquatic environments. For the design of the reward function, factors such as shallow water areas are taken into account, with an emphasis on minimizing the number of turning points along the path. For trajectory tracking control, we integrate the Soft Actor-Critic (SAC) algorithm with the Proportional-Integral-Derivative (PID) control method to alleviate the difficulties of manual parameter tuning associated with conventional PID controllers. This hybrid approach also mitigates the interpretability limitations often found in pure deep reinforcement learning methods. Comparative experiments involving the traditional PID algorithm, Genetic Algorithm (GA), and Deep Deterministic Policy Gradient (DDPG) algorithm demonstrate the superiority of the proposed SAC-PID method. Simulation results show that the planned trajectories effectively incorporate multiple factors, including travel distance, shallow water regions, and number of turning point, the SAC-PID method achieves outstanding performance in trajectory tracking.

trajectory planning  /  trajectory control  /  deep reinforcement learning  /  unmanned surface vehicle
关巍, 奚赵勇, 崔哲闻, 张显库. 基于深度强化学习的无人艇航迹规划与控制. 中国航海, 2025 , 48 (3) : 129 -136 . DOI: 10.3969/j.issn.1000-4653.2025.03.016
Wei GUAN, Zhaoyong XI, Zhewen CUI, Xianku ZHANG. Trajectory planning and control for unmanned surface vehicle based on deep reinforcement learning[J]. Navigation of China, 2025 , 48 (3) : 129 -136 . DOI: 10.3969/j.issn.1000-4653.2025.03.016
  • 国家自然科学基金项目(52171342)
2025年第48卷第3期
PDF下载
83
18
引用本文
BibTeX
文章信息
doi: 10.3969/j.issn.1000-4653.2025.03.016
  • 接收时间:2024-09-01
  • 首发时间:2026-03-17
  • 出版时间:2025-09-25
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2024-09-01
基金
国家自然科学基金项目(52171342)
作者信息
    大连海事大学 航海学院,辽宁 大连 116026

通讯作者:

关巍(1982—),男,博士,教授,博士生导师,研究方向为船舶智能航行决策,船舶运动控制。E-mail:
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/zghh/CN/10.3969/j.issn.1000-4653.2025.03.016
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数

Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)

Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏