Article(id=1251458155385929833, tenantId=1146029695717560320, journalId=1251194880429441115, issueId=1251458153020342360, articleNumber=null, orderNo=null, doi=10.3979/j.issn.1673-825X.202410140254, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1728835200000, receivedDateStr=2024-10-14, revisedDate=1757865600000, revisedDateStr=2025-09-15, acceptedDate=null, acceptedDateStr=null, onlineDate=1776300475212, onlineDateStr=2026-04-16, pubDate=null, pubDateStr=null, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1776300475212, onlineIssueDateStr=2026-04-16, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1776300475212, creator=13041195026, updateTime=1776300475212, updator=13041195026, issue=Issue{id=1251458153020342360, tenantId=1146029695717560320, journalId=1251194880429441115, year='2025', volume='37', issue='5', pageStart='627', pageEnd='780', issueExtLink='null', onlineDate='null', pubDate='null', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=1, specialIssue=null, createTime=1776300474648, creator=13041195026, updateTime=1776311939434, updator=13041195026, preIssue=null, nextIssue=null, ext={EN=IssueExt(id=1251506239914586238, tenantId=1146029695717560320, journalId=1251194880429441115, issueId=1251458153020342360, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1251506239914586239, tenantId=1146029695717560320, journalId=1251194880429441115, issueId=1251458153020342360, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null}, startPage=668, endPage=676, ext={EN=ArticleExt(id=1251458156765855859, articleId=1251458155385929833, tenantId=1146029695717560320, journalId=1251194880429441115, language=EN, title=Unmanned aerial vehicle assisted RSMA relay communication technology based on deep reinforcement learning, columnId=1251458153846620250, journalTitle=Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), columnName=New-Generation Mobile Communication, runingTitle=null, highlight=null, articleAbstract=

To address the security challenges of relay communication in complex environments with potential eavesdroppers, this paper proposes a multi-UAV-assisted relay communication network that provides secure communication services for users. A multi-agent deep reinforcement learning(MARL)algorithm based on the Q-mixing network(QMIX)is employed to jointly optimize UAV trajectories and power allocation. The goal is to guarantee the minimum transmission rate of low-security-sensitivity users(secondary users)while enhancing the communication security and data rate of high-security-sensitivity users(primary users). Simulation results demonstrate that, compared with the Double Deep Q-Network(Double DQN)and the Dueling Deep Q-Network(Dueling DQN), the proposed algorithm improves the cumulative reward by approximately 15.5% and 1.26%, respectively. Moreover, the proposed rate-splitting multiple access(RSMA)technique significantly outperforms space-division multiple access(SDMA)and non-orthogonal multiple access(NOMA)in terms of overall system performance and information security. The proposed method provides an effective solution for achieving secure and efficient communication in multi-user wireless networks.

, correspAuthors=null, authorNote=null, correspAuthorsNote=null, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, authorCompany=null, fund=null, authors=null, authorsList=Zifu FAN, Kerui ZHANG, Zhengqiang WANG), CN=ArticleExt(id=1251458169877250568, articleId=1251458155385929833, tenantId=1146029695717560320, journalId=1251194880429441115, language=CN, title=基于深度强化学习的无人机辅助RSMA中继通信技术, columnId=1251458154026975325, journalTitle=重庆邮电大学学报(自然科学版), columnName=新一代移动通信, runingTitle=null, highlight=null, articleAbstract=

针对用户在存在窃听者的复杂通信环境进行中继通信的安全问题,提出了一种多无人机辅助的中继通信网络为用户提供通信服务。通过基于Q混合网络(Q-mixing network,QMIX)的多智能体深度强化学习(multi-agent reinforcement learning,MRAL)算法优化无人机轨迹与功率分配,在信息安全敏感度较低用户(次要用户)最低速率得到保障的情况下,提高信息安全敏感较高用户(主要用户)的安全和速率。仿真结果表明,算法相较于双层深度Q网络(double deep Q-network,Double DQN)和对偶深度Q网络(dueling deep Q-network,Dueling DQN),累积奖励分别提高了大约15.5%和1.26%;模型的速率分割多址技术相较于空分多址和非正交多址技术,在系统整体性能和信息安全保障方面都具有显著优势,为多用户通信场景下的安全高效通信提供了更优解决方案。

, correspAuthors=null, authorNote=null, correspAuthorsNote=
王正强
, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=7qPySJA4Fvt7D1ek44ON+Q==, magXml=mvz88Xq8JFVGFrLXQxzv1A==, pdfUrl=null, pdf=UnHFwgvbppNJdRQpUu9CxA==, pdfFileSize=6261518, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=uyFr9IE2bwTCpY4/lmauUQ==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=fIxQ5q27bGpFQnUpRSgIjg==, mapNumber=null, authorCompany=null, fund=null, authors=

樊自甫,教授,主要研究方向为网络优化、绿色通信。Email:

张珂瑞,硕士研究生,主要研究方向为无人机通信、机器学习。E-mail:

王正强,副教授,博士,硕士生导师,主要研究方向为无人机通信、物理层安全。E-mail:

, authorsList=樊自甫, 张珂瑞, 王正强)}, authors=[Author(id=1251458170305069601, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=fanzf@cqupt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1251458170426704425, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, authorId=1251458170305069601, language=EN, stringName=Zifu FAN, firstName=Zifu, middleName=null, lastName=FAN, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P R China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1251458170510590510, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, authorId=1251458170305069601, language=CN, stringName=樊自甫, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=重庆邮电大学 通信与信息工程学院,重庆 400065, bio={"content":"

樊自甫,教授,主要研究方向为网络优化、绿色通信。Email:

"}, bioImg=null, bioContent=

樊自甫,教授,主要研究方向为网络优化、绿色通信。Email:

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1251458170196017686, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, xref=null, ext=[AuthorCompanyExt(id=1251458170212794905, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, companyId=1251458170196017686, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P R China), AuthorCompanyExt(id=1251458170221183514, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, companyId=1251458170196017686, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=重庆邮电大学 通信与信息工程学院,重庆 400065)])]), Author(id=1251458170611253814, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=927533633@qq.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1251458170690945593, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, authorId=1251458170611253814, language=EN, stringName=Kerui ZHANG, firstName=Kerui, middleName=null, lastName=ZHANG, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P R China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1251458170787414589, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, authorId=1251458170611253814, language=CN, stringName=张珂瑞, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=重庆邮电大学 通信与信息工程学院,重庆 400065, bio={"content":"

张珂瑞,硕士研究生,主要研究方向为无人机通信、机器学习。E-mail:

"}, bioImg=null, bioContent=

张珂瑞,硕士研究生,主要研究方向为无人机通信、机器学习。E-mail:

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1251458170196017686, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, xref=null, ext=[AuthorCompanyExt(id=1251458170212794905, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, companyId=1251458170196017686, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P R China), AuthorCompanyExt(id=1251458170221183514, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, companyId=1251458170196017686, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=重庆邮电大学 通信与信息工程学院,重庆 400065)])]), Author(id=1251458170871300674, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=wangzq@cqupt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1251458170976158280, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, authorId=1251458170871300674, language=EN, stringName=Zhengqiang WANG, firstName=Zhengqiang, middleName=null, lastName=WANG, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P R China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1251458171072627276, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, authorId=1251458170871300674, language=CN, stringName=王正强, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=重庆邮电大学 通信与信息工程学院,重庆 400065, bio={"content":"

王正强,副教授,博士,硕士生导师,主要研究方向为无人机通信、物理层安全。E-mail:

"}, bioImg=null, bioContent=

王正强,副教授,博士,硕士生导师,主要研究方向为无人机通信、物理层安全。E-mail:

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1251458170196017686, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, xref=null, ext=[AuthorCompanyExt(id=1251458170212794905, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, companyId=1251458170196017686, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P R China), AuthorCompanyExt(id=1251458170221183514, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, companyId=1251458170196017686, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=重庆邮电大学 通信与信息工程学院,重庆 400065)])])], keywords=[Keyword(id=1251458171202650707, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, orderNo=1, keyword=rate-splitting multiple access ( RSMA )), Keyword(id=1251458171282342487, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, orderNo=2, keyword=wireless communication networks), Keyword(id=1251458171362034266, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, orderNo=3, keyword=deep reinforcement learning (DRL)), Keyword(id=1251458171454308959, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, orderNo=4, keyword=security and rate), Keyword(id=1251458171529806434, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, orderNo=1, keyword=速率分割多址接入), Keyword(id=1251458171617886823, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, orderNo=2, keyword=无线通信网络), Keyword(id=1251458171697578603, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, orderNo=3, keyword=深度强化学习), Keyword(id=1251458171781464687, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, orderNo=4, keyword=安全和速率)], refs=[Reference(id=1251458175094964916, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2021, volume=70, issue=12, pageStart=13424, pageEnd=13429, url=null, language=null, rfNumber=[1], rfOrder=0, authorNames=ZHOU X B, YAN S H, DERRICK W K N, journalName=IEEE Transactions on Vehicular Technology, refType=null, unstructuredReference=ZHOU X B, YAN S H, DERRICK W K N. Three-dimensional placement and transmit power design for UAV covert communications[J]. IEEE Transactions on Vehicular Technology, 2021, 70(12): 13424-13429., articleTitle=Three-dimensional placement and transmit power design for UAV covert communications, refAbstract=null), Reference(id=1251458175183045304, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2019, volume=26, issue=5, pageStart=114, pageEnd=120, url=null, language=null, rfNumber=[2], rfOrder=1, authorNames=LI B, FEI Z S, ZHANG Y, journalName=IEEE Wireless Communication, refType=null, unstructuredReference=LI B, FEI Z S, ZHANG Y, et.al. Secure UAV communication networks over 5G[J]. IEEE Wireless Communication, 2019, 26(5): 114-120., articleTitle=Secure UAV communication networks over 5G, refAbstract=null), Reference(id=1251458175262737085, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2021, volume=70, issue=4, pageStart=3298, pageEnd=3313, url=null, language=null, rfNumber=[3], rfOrder=2, authorNames=NGUYEN P X, NGUYEN V D, NGUYEN H V, journalName=IEEE Transactions on Vehicular Technology, refType=null, unstructuredReference=NGUYEN P X, NGUYEN V D, NGUYEN H V, et al. UAV assisted secure communications in terrestrial cognitive radio networks:Joint power control and 3D trajectory optimization[J]. IEEE Transactions on Vehicular Technology, 2021, 70(4): 3298-3313., articleTitle=UAV assisted secure communications in terrestrial cognitive radio networks:Joint power control and 3D trajectory optimization, refAbstract=null), Reference(id=1251458175367594688, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2021, volume=20, issue=4, pageStart=2383, pageEnd=2397, url=null, language=null, rfNumber=[4], rfOrder=3, authorNames=YANG B, TALEB T, SHEN Y L, journalName=IEEE Transactions on Wireless Communications, refType=null, unstructuredReference=YANG B, TALEB T, SHEN Y L, et al. Performance, fairness,and tradeoff in UAV swarm underlaid mmWave cellular networks with directional antennas[J]. IEEE Transactions on Wireless Communications, 2021, 20(4): 2383-2397., articleTitle=Performance, fairness,and tradeoff in UAV swarm underlaid mmWave cellular networks with directional antennas, refAbstract=null), Reference(id=1251458175459869380, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2020, volume=38, issue=9, pageStart=2057, pageEnd=2073, url=null, language=null, rfNumber=[5], rfOrder=4, authorNames=ZHU L P, ZHANG J, XIAO Z Y, journalName=IEEE Journal on Selected Areas in Communications, refType=null, unstructuredReference=ZHU L P, ZHANG J, XIAO Z Y, et al. Millimeter-wave full-duplex UAV relay: Joint positioning, beamforming, and power control[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(9): 2057-2073., articleTitle=Millimeter-wave full-duplex UAV relay: Joint positioning, beamforming, and power control, refAbstract=null), Reference(id=1251458175568921287, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2022, volume=40, issue=10, pageStart=2962, pageEnd=2979, url=null, language=null, rfNumber=[6], rfOrder=5, authorNames=DU H Y, NIYATO D, XIE Y A, journalName=IEEE Journal on Selected Areas in Communications, refType=null, unstructuredReference=DU H Y, NIYATO D, XIE Y A, et al. Performance analysis and optimization for jammer-aided multiantenna UAV covert communication[J]. IEEE Journal on Selected Areas in Communications, 2022, 40(10): 2962-2979., articleTitle=Performance analysis and optimization for jammer-aided multiantenna UAV covert communication, refAbstract=null), Reference(id=1251458175644418763, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2021, volume=24, issue=2, pageStart=139, pageEnd=153, url=null, language=null, rfNumber=[7], rfOrder=6, authorNames=WANG Z, GUO J H, CHEN Z Q, journalName=Journal of Communications and Networks, refType=null, unstructuredReference=WANG Z, GUO J H, CHEN Z Q, et al. Robust secure UAV relay-assisted cognitive communications with resource allocation and cooperative jamming[J]. Journal of Communications and Networks, 2021, 24(2): 139-153., articleTitle=Robust secure UAV relay-assisted cognitive communications with resource allocation and cooperative jamming, refAbstract=null), Reference(id=1251458175715721934, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2022, volume=null, issue=null, pageStart=1549, pageEnd=1554, url=null, language=null, rfNumber=[8], rfOrder=7, authorNames=SINGH S K, AGRAWAL K, SINGH K, journalName=null, refType=null, unstructuredReference=SINGH S K, AGRAWAL K, SINGH K, et al. RSMA enhanced RIS-FD-UAV-aided short packet communications under imperfect SIC[C]//2022 IEEE Globecom Workshops. Riode Janeiro, Brazil: IEEE, 2022: 1549-1554., articleTitle=RSMA enhanced RIS-FD-UAV-aided short packet communications under imperfect SIC, refAbstract=null), Reference(id=1251458175787025105, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2022, volume=71, issue=1, pageStart=639, pageEnd=652, url=null, language=null, rfNumber=[9], rfOrder=8, authorNames=LIANG Y P, XIAO L, YANG D C, journalName=IEEE Transactions on Vehicular Technology, refType=null, unstructuredReference=LIANG Y P, XIAO L, YANG D C, et al. Joint trajectory and resource optimization for UAV-aided two-way relay networks[J]. IEEE Transactions on Vehicular Technology, 2022, 71(1): 639-652., articleTitle=Joint trajectory and resource optimization for UAV-aided two-way relay networks, refAbstract=null), Reference(id=1251458175883494102, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2022, volume=6, issue=1, pageStart=543, pageEnd=557, url=null, language=null, rfNumber=[10], rfOrder=9, authorNames=ZHANG G C, YAN H Q, ZENG Y, journalName=IEEE Transactions on Green Communications and Networking, refType=null, unstructuredReference=ZHANG G C, YAN H Q, ZENG Y, et al. Cooperative UAV enabled relaying systems: Joint trajectory and transmit power optimization[J]. IEEE Transactions on Green Communications and Networking, 2022, 6(1): 543-557., articleTitle=Cooperative UAV enabled relaying systems: Joint trajectory and transmit power optimization, refAbstract=null), Reference(id=1251458175963185881, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2020, volume=69, issue=9, pageStart=10033, pageEnd=10045, url=null, language=null, rfNumber=[11], rfOrder=10, authorNames=CHEN X Y, YANG Z T, ZHAO N, journalName=IEEE Transactions on Vehicular Technology, refType=null, unstructuredReference=CHEN X Y, YANG Z T, ZHAO N, et al. Secure transmission via power allocation in NOMA-UAV networks with circular trajectory[J]. IEEE Transactions on Vehicular Technology, 2020, 69(9): 10033-10045., articleTitle=Secure transmission via power allocation in NOMA-UAV networks with circular trajectory, refAbstract=null), Reference(id=1251458176072237791, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2025, volume=43, issue=5, pageStart=45, pageEnd=53, url=null, language=null, rfNumber=[12], rfOrder=11, authorNames=程擎, 何汶键, 贾志杰, journalName=贵州师范大学学报(自然科学版), refType=null, unstructuredReference=程擎,何汶键,贾志杰,.基于改进YOLOv7的低空小型无人机识别算法研究[J].贵州师范大学学报(自然科学版), 2025, 43(5): 45-53., articleTitle=基于改进YOLOv7的低空小型无人机识别算法研究, refAbstract=null), Reference(id=1251458176147735266, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2025, volume=43, issue=5, pageStart=45, pageEnd=53, url=null, language=null, rfNumber=[12], rfOrder=12, authorNames=CHENG Q, HE W J, JIA Z J, journalName=Journal of Guizhou Normal University(Natural Sciences), refType=null, unstructuredReference=CHENG Q, HE W J, JIA Z J, et al. Researchon low altitude small UAVs recognition algorithm based on improved YOLOv7[J]. Journal of Guizhou Normal University(Natural Sciences), 2025, 43(5): 45-53., articleTitle=Researchon low altitude small UAVs recognition algorithm based on improved YOLOv7, refAbstract=null), Reference(id=1251458176235815655, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2022, volume=71, issue=10, pageStart=10570, pageEnd=10583, url=null, language=null, rfNumber=[13], rfOrder=13, authorNames=KANG H Y, CHANG X L, MISIC J, journalName=IEEE Transactions on Vehicular Technology, refType=null, unstructuredReference=KANG H Y, CHANG X L, MISIC J, et al. Improving dual-UAV aided ground-UAV bi-directional communication security: Joint UAV trajectory and transmit power optimization[J]. IEEE Transactions on Vehicular Technology, 2022, 71(10): 10570-10583., articleTitle=Improving dual-UAV aided ground-UAV bi-directional communication security: Joint UAV trajectory and transmit power optimization, refAbstract=null), Reference(id=1251458176302924525, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2022, volume=44, issue=8, pageStart=2730, pageEnd=2738, url=null, language=null, rfNumber=[14], rfOrder=14, authorNames=高远, 谭蓉俊, 邓志祥, journalName=电子与信息学报, refType=null, unstructuredReference=高远,谭蓉俊,邓志祥.无人机辅助物理层安全下的保密性能优化[J].电子与信息学报, 2022, 44(8):2730-2738., articleTitle=无人机辅助物理层安全下的保密性能优化, refAbstract=null), Reference(id=1251458176395199215, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2022, volume=44, issue=8, pageStart=2730, pageEnd=2738, url=null, language=null, rfNumber=[14], rfOrder=15, authorNames=GAO Y, TAN R G, DENG Z X, journalName=Journal of Electronics & Information Technology, refType=null, unstructuredReference=GAO Y, TAN R G, DENG Z X. Secrecy performance optimization of unmanned aerial vehicle-aided physical layer security[J]. Journal of Electronics & Information Technology, 2022, 44(8): 2730-2738., articleTitle=Secrecy performance optimization of unmanned aerial vehicle-aided physical layer security, refAbstract=null), Reference(id=1251458176474890995, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2016, volume=54, issue=5, pageStart=98, pageEnd=105, url=null, language=null, rfNumber=[15], rfOrder=16, authorNames=CLERCKX B, JOUDEH H, HAO C, journalName=IEEE Communications Magazine, refType=null, unstructuredReference=CLERCKX B, JOUDEH H, HAO C, et al. Rate splitting for MIMO wireless networks: A promising PHY-layer strategy for LTE-evolution[J]. IEEE Communications Magazine, 2016, 54(5): 98-105., articleTitle=Rate splitting for MIMO wireless networks: A promising PHY-layer strategy for LTE-evolution, refAbstract=null), Reference(id=1251458176554582776, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2020, volume=9, issue=3, pageStart=349, pageEnd=353, url=null, language=null, rfNumber=[16], rfOrder=17, authorNames=CLERCKX B, MAO Y J, SCHOBER R, journalName=IEEE Transactions on Wireless Communications Letters, refType=null, unstructuredReference=CLERCKX B, MAO Y J, SCHOBER R, et al. Rate-splitting unifying SDMA, OMA, NOMA, and multicasting in MISO broadcast channel: A simple two-user rate analysis[J]. IEEE Transactions on Wireless Communications Letters, 2020, 9(3): 349-353., articleTitle=Rate-splitting unifying SDMA, OMA, NOMA, and multicasting in MISO broadcast channel: A simple two-user rate analysis, refAbstract=null), Reference(id=1251458176625885947, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2023, volume=null, issue=null, pageStart=176, pageEnd=180, url=null, language=null, rfNumber=[17], rfOrder=18, authorNames=PANG H R, JI F, NING Z L, journalName=null, refType=null, unstructuredReference=PANG H R, JI F, NING Z L, et al. Enhancing security in RSMA networks with cooperative jamming and relaying[C]//2023 IEEE 24th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). Shanghai, China: IEEE, 2023: 176-180., articleTitle=Enhancing security in RSMA networks with cooperative jamming and relaying, refAbstract=null), Reference(id=1251458176718160639, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2024, volume=45, issue=7, pageStart=1763, pageEnd=1769, url=null, language=null, rfNumber=[18], rfOrder=19, authorNames=许钧智, 李陶深, 葛志辉, journalName=小型微型计算机系统, refType=null, unstructuredReference=许钧智,李陶深,葛志辉.自能量回收的全双工无人机中继资源分配[J].小型微型计算机系统, 2024, 45(7): 1763-1769., articleTitle=自能量回收的全双工无人机中继资源分配, refAbstract=null), Reference(id=1251458176823018242, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2024, volume=45, issue=7, pageStart=1763, pageEnd=1769, url=null, language=null, rfNumber=[18], rfOrder=20, authorNames=XU JZ, LITS, GEZH, journalName=Journal of Chinese Computer Systems, refType=null, unstructuredReference=XU JZ,LITS,GEZH. Resource allocation for full-duplex UAV relaying with self-energized technology[J]. Journal of Chinese Computer Systems, 2024, 45(7):1763-1769., articleTitle=Resource allocation for full-duplex UAV relaying with self-energized technology, refAbstract=null), Reference(id=1251458176885932804, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2023, volume=17, issue=2, pageStart=2586, pageEnd=2589, url=null, language=null, rfNumber=[19], rfOrder=21, authorNames=SINGH S K, AGRAWAL K, SINGH K, journalName=IEEE Systems Journal, refType=null, unstructuredReference=SINGH S K, AGRAWAL K, SINGH K, et al. Ergodic capacity and placement optimization for RSMA-enabled UAV-assisted communication[J]. IEEE Systems Journal, 2023, 17(2): 2586-2589., articleTitle=Ergodic capacity and placement optimization for RSMA-enabled UAV-assisted communication, refAbstract=null), Reference(id=1251458176957235973, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2021, volume=10, issue=11, pageStart=2528, pageEnd=2532, url=null, language=null, rfNumber=[20], rfOrder=22, authorNames=HOANG T M, DUNG L T, NGUYEN B C, journalName=IEEE Wireless Communications Letters, refType=null, unstructuredReference=HOANG T M, DUNG L T, NGUYEN B C, et al. Outage probability and throughput analysis of UAV-assisted rate-splitting multiple access[J]. IEEE Wireless Communications Letters, 2021, 10(11): 2528-2532., articleTitle=Outage probability and throughput analysis of UAV-assisted rate-splitting multiple access, refAbstract=null), Reference(id=1251458177041122055, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2022, volume=24, issue=4, pageStart=2073, pageEnd=2126, url=null, language=null, rfNumber=[21], rfOrder=23, authorNames=MAO Y J, DIZDAR O, CLERCKX B, journalName=IEEE Communications Surveys Tutorials, refType=null, unstructuredReference=MAO Y J, DIZDAR O, CLERCKX B, et al. Rate-splitting multiple access: Fundamentals, survey, and future research trends[J]. IEEE Communications Surveys Tutorials, 2022, 24(4): 2073-2126, articleTitle=Rate-splitting multiple access: Fundamentals, survey, and future research trends, refAbstract=null), Reference(id=1251458178651734794, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2023, volume=44, issue=12, pageStart=50, pageEnd=60, url=null, language=null, rfNumber=[22], rfOrder=24, authorNames=赵柏, 林敏, 肖圣杰, journalName=通信学报, refType=null, unstructuredReference=赵柏,林敏,肖圣杰,.基于速率分割的可重构智能表面辅助星地融合网络鲁棒安全传输方案[J].通信学报, 2023, 44(12): 50-60., articleTitle=基于速率分割的可重构智能表面辅助星地融合网络鲁棒安全传输方案, refAbstract=null), Reference(id=1251458178764981006, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2023, volume=44, issue=12, pageStart=50, pageEnd=60, url=null, language=null, rfNumber=[22], rfOrder=25, authorNames=ZHAO B, LIN M, XIAO S J, journalName=Journal on Communications, refType=null, unstructuredReference=ZHAO B, LIN M, XIAO S J, et al. Rate splitting based robust secure transmission scheme in RIS-assisted satellite-terrestrial integrated network[J]. Journal on Communications, 2023, 44(12): 50-60., articleTitle=Rate splitting based robust secure transmission scheme in RIS-assisted satellite-terrestrial integrated network, refAbstract=null), Reference(id=1251458178911781648, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2023, volume=39, issue=7, pageStart=1214, pageEnd=1221, url=null, language=null, rfNumber=[23], rfOrder=26, authorNames=夏慧云, 韩帅, 李成, journalName=信号处理, refType=null, unstructuredReference=夏慧云,韩帅,李成.人工噪声在速率分拆多址保密通信中的作用[J].信号处理, 2023, 39(7): 1214-1221., articleTitle=人工噪声在速率分拆多址保密通信中的作用, refAbstract=null), Reference(id=1251458179071165202, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2023, volume=39, issue=7, pageStart=1214, pageEnd=1221, url=null, language=null, rfNumber=[23], rfOrder=27, authorNames=XIA H Y, HAN S, LI C, journalName=Journal of Signal Processing, refType=null, unstructuredReference=XIA H Y, HAN S, LI C. The role of artificial noise in RSMA-based secure communications[J]. Journal of Signal Processing, 2023, 39(7): 1214-1221., articleTitle=The role of artificial noise in RSMA-based secure communications, refAbstract=null), Reference(id=1251458179230548759, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2019, volume=3, issue=2, pageStart=47, pageEnd=55, url=null, language=null, rfNumber=[24], rfOrder=28, authorNames=周毅, 马晓勇, 郜富晓, journalName=物联网学报, refType=null, unstructuredReference=周毅,马晓勇,郜富晓,.基于深度强化学习的无人机自主部署及能效优化策略[J].物联网学报, 2019, 3(2): 47-55., articleTitle=基于深度强化学习的无人机自主部署及能效优化策略, refAbstract=null), Reference(id=1251458179310240537, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2019, volume=3, issue=2, pageStart=47, pageEnd=55, url=null, language=null, rfNumber=[24], rfOrder=29, authorNames=ZHOU Y, MA X Y, HAO F X, journalName=Chinese Journal on Internet of Things, refType=null, unstructuredReference=ZHOU Y, MA X Y, HAO F X, et al. Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning[J]. Chinese Journal on Internet of Things, 2019, 3(2): 47-55., articleTitle=Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning, refAbstract=null), Reference(id=1251458179385738012, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2023, volume=22, issue=3, pageStart=1495, pageEnd=1509, url=null, language=null, rfNumber=[25], rfOrder=30, authorNames=ZHANG X C, ZHAO H T, WEI J B, journalName=IEEE Transactions on Wireless Communications, refType=null, unstructuredReference=ZHANG X C, ZHAO H T, WEI J B, et al. Cooperative trajectory design of multiple UAV base stations with heterogeneous graph neural networks[J]. IEEE Transactions on Wireless Communications, 2023, 22(3): 1495-1509., articleTitle=Cooperative trajectory design of multiple UAV base stations with heterogeneous graph neural networks, refAbstract=null), Reference(id=1251458179452846879, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, doi=null, pmid=null, pmcid=null, year=2023, volume=72, issue=3, pageStart=3025, pageEnd=3034, url=null, language=null, rfNumber=[26], rfOrder=31, authorNames=GUO Z H, WU Y, WANG L F, journalName=IEEE Transactions on Vehicular Technology, refType=null, unstructuredReference=GUO Z H, WU Y, WANG L F, et al. Coordination for connected and automated vehicles at non-signalized intersections: A value decomposition-based multiagent deep reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2023, 72(3): 3025-3034., articleTitle=Coordination for connected and automated vehicles at non-signalized intersections: A value decomposition-based multiagent deep reinforcement learning approach, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1251458170196017686, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, xref=null, ext=[AuthorCompanyExt(id=1251458170212794905, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, companyId=1251458170196017686, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P R China), AuthorCompanyExt(id=1251458170221183514, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, companyId=1251458170196017686, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=重庆邮电大学 通信与信息工程学院,重庆 400065)])], figs=[ArticleFig(id=1251458171924071030, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, label=Fig.1, caption=UAV-assisted secure communication model, figureFileSmall=9At+9QZ00/xIEU8He0F/zw==, figureFileBig=uyFr9IE2bwTCpY4/lmauUQ==, tableContent=null), ArticleFig(id=1251458172012151418, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, label=图1, caption=无人机辅助安全通信模型, figureFileSmall=9At+9QZ00/xIEU8He0F/zw==, figureFileBig=uyFr9IE2bwTCpY4/lmauUQ==, tableContent=null), ArticleFig(id=1251458172200895108, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, label=Fig.2, caption=Multi-agent deep reinforcement learning architecture based on QMIX, figureFileSmall=O35az6WSw4aOYKqZRsFAJw==, figureFileBig=agjrk75o4EChzbquG+XmDQ==, tableContent=null), ArticleFig(id=1251458172268003976, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, label=图2, caption=基于QMIX的多智能体深度强化学习架构, figureFileSmall=O35az6WSw4aOYKqZRsFAJw==, figureFileBig=agjrk75o4EChzbquG+XmDQ==, tableContent=null), ArticleFig(id=1251458172351890060, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, label=Fig.3, caption=Comparison of the average safety rate of the three constraints, figureFileSmall=yP71ev6oSl4hHfbgAOYMCA==, figureFileBig=J+D4eJxvRXlMupFwpOY8LQ==, tableContent=null), ArticleFig(id=1251458172469330575, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, label=图3, caption=三种约束的平均安全速率收敛对比, figureFileSmall=yP71ev6oSl4hHfbgAOYMCA==, figureFileBig=J+D4eJxvRXlMupFwpOY8LQ==, tableContent=null), ArticleFig(id=1251458172590965395, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, label=Fig.4, caption=Comparison of UAV trajectory under two highly optimized algorithms, figureFileSmall=QxToKvHiAIECd/LQj6BqaA==, figureFileBig=iarWUpd8P4Qo2htvaq4r+A==, tableContent=null), ArticleFig(id=1251458174214161049, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, label=图4, caption=两种高度优化算法下无人机轨迹对比, figureFileSmall=QxToKvHiAIECd/LQj6BqaA==, figureFileBig=iarWUpd8P4Qo2htvaq4r+A==, tableContent=null), ArticleFig(id=1251458174323212956, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, label=Fig.5, caption=Relay UAV power allocation, figureFileSmall=55OyygzFhHQ5ES6Di9AHXA==, figureFileBig=H863ArzL/5jYi9qjrz9XnQ==, tableContent=null), ArticleFig(id=1251458174419681951, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, label=图5, caption=中继无人机功率分配, figureFileSmall=55OyygzFhHQ5ES6Di9AHXA==, figureFileBig=H863ArzL/5jYi9qjrz9XnQ==, tableContent=null), ArticleFig(id=1251458174587454113, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, label=Fig.6, caption=Relationship between the number of users and the average security rate, figureFileSmall=f/1TJGxYu9Yr6eSsabiblA==, figureFileBig=RW6N0QFACLZHoN6HBTlO0w==, tableContent=null), ArticleFig(id=1251458174679728805, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, label=图6, caption=用户数量与平均安全速率关系, figureFileSmall=f/1TJGxYu9Yr6eSsabiblA==, figureFileBig=RW6N0QFACLZHoN6HBTlO0w==, tableContent=null), ArticleFig(id=1251458174788780712, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, label=Fig.7, caption=System convergence of different algorithms, figureFileSmall=QxfInJdA81lREp+N1Y1C9A==, figureFileBig=FVmct6lLkvgUaVcnALFXQQ==, tableContent=null), ArticleFig(id=1251458174876861099, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, label=图7, caption=不同算法系统收敛图, figureFileSmall=QxfInJdA81lREp+N1Y1C9A==, figureFileBig=FVmct6lLkvgUaVcnALFXQQ==, tableContent=null), ArticleFig(id=1251458174952358575, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=EN, label=Tab.2, caption=

Experiment Parameters

, figureFileSmall=null, figureFileBig=null, tableContent=
参数数值
环境边界(XmaxYmax)/m(4000,4 000)
环境边界(XminYmin)/m(0,0)
最大飞行高度Hmax/m600
最小飞行高度Hmin/m50
水平飞行速度/(m/s)10
垂直飞行速度/(m/s)2
基站功率/dBm30
无人机功率/dBm15
无人机干扰功率/dBm15
信道参数(ab(9.61,0.16)
路径损失(LOS,NLOS)/dB1,20
服务时长T/时间步70
时间步t/s30
环境噪声σ2/dBm-170
中继与基站天线数量4
), ArticleFig(id=1251458175015273137, tenantId=1146029695717560320, journalId=1251194880429441115, articleId=1251458155385929833, language=CN, label=表2, caption=

实验参数设置

, figureFileSmall=null, figureFileBig=null, tableContent=
参数数值
环境边界(XmaxYmax)/m(4000,4 000)
环境边界(XminYmin)/m(0,0)
最大飞行高度Hmax/m600
最小飞行高度Hmin/m50
水平飞行速度/(m/s)10
垂直飞行速度/(m/s)2
基站功率/dBm30
无人机功率/dBm15
无人机干扰功率/dBm15
信道参数(ab(9.61,0.16)
路径损失(LOS,NLOS)/dB1,20
服务时长T/时间步70
时间步t/s30
环境噪声σ2/dBm-170
中继与基站天线数量4
)], attaches=null, journal=Journal(id=1251194162217791579, delFlag=0, nameCn=重庆邮电大学学报(自然科学版), nameEn=Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), nameHistory1=null, nameHistory2=null, issn=1673-825X, eissn=null, cn=50-1181/N, coden=null, periodic=1, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=eIjswk9Qxcq7+V27dEZ90g==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), journalRemark=null, publicationField=null, createdTime=1776237534337, updatedTime=1776238167705, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Natural Sciences, subjectName=null, subjectCodeEn=Natural Sciences, subjectNameEn=null, picCn=eIjswk9Qxcq7+V27dEZ90g==, picEn=PN1NU8XEwLhVlBzwQEoroA==, jcr=null, cjcr=null, exts=[JournalExt(id=1251196818869137642, language=CN, name=重庆邮电大学学报(自然科学版), nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1776238167732, updatedTime=1776238167732, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=http://journal2.cqupt.edu.cn/jcuptnsecn/jcuptnse/author/login, submissionEditorUrl=http://journal2.cqupt.edu.cn/jcuptnsecn/jcuptnse/editor/login, submissionReviewUrl=http://journal2.cqupt.edu.cn/jcuptnsecn/jcuptnse/reviewer/login, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1251196818936246507, language=EN, name=Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1776238167747, updatedTime=1776238167747, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=http://journal2.cqupt.edu.cn/jcuptnsecn/jcuptnse/author/login, submissionEditorUrl=http://journal2.cqupt.edu.cn/jcuptnsecn/jcuptnse/editor/login, submissionReviewUrl=http://journal2.cqupt.edu.cn/jcuptnsecn/jcuptnse/reviewer/login, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1251194880429441115, websiteList=[Website(id=1251197148419797361, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1251194880429441115, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/cqyddxxb/CN, language=CN, createTime=1776238246302, createBy=18614031015, updateTime=1776238652182, updateBy=18614031015, name=重庆邮电大学学报(自然科学版)-中文, tplId=1146099689490845704, title=重庆邮电大学学报(自然科学版), delFlag=0, indexPage=/home, props=[WebsiteProps(id=1251198981565530670, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=articleTextType, value=kx, createTime=1776238683358, updateTime=1776238683358, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981540364843, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=banner, value=null, createTime=1776238683352, updateTime=1776238683352, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981590696497, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=grayFlag, value=0, createTime=1776238683364, updateTime=1776238683364, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981531976234, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=logo, value=https://castjournals.cast.org.cn/joweb/cqyddxxb/CN/file/pic?fileId=X24gYotabwCg03WC1YllbA==, createTime=1776238683350, updateTime=1776238683350, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981607473715, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=minRunFlag, value=0, createTime=1776238683368, updateTime=1776238683368, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981557142061, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/cqyddxxb/CN/file/pic, createTime=1776238683356, updateTime=1776238683356, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981599085106, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=silenceFlag, value=0, createTime=1776238683366, updateTime=1776238683366, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981548753452, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1776238683354, updateTime=1776238683354, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981578113583, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=themeColor, value=null, createTime=1776238683361, updateTime=1776238683361, creator=18614031015, updator=18614031015), WebsiteProps(id=1251198981582307888, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148419797361, code=themeStyle, value=null, createTime=1776238683362, updateTime=1776238683362, creator=18614031015, updator=18614031015)]), Website(id=1251197148516266372, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1251194880429441115, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/cqyddxxb/EN, language=EN, createTime=1776238246325, createBy=18614031015, updateTime=1776238647495, updateBy=18614031015, name=重庆邮电大学学报(自然科学版)-英文, tplId=1146101810881728533, title=Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), delFlag=0, indexPage=/home, props=[WebsiteProps(id=1251199007897371192, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=articleTextType, value=kx, createTime=1776238689636, updateTime=1776238689636, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007872205365, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=banner, value=null, createTime=1776238689630, updateTime=1776238689630, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007914148411, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=grayFlag, value=0, createTime=1776238689640, updateTime=1776238689640, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007863816756, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=logo, value=https://castjournals.cast.org.cn/joweb/cqyddxxb/EN/file/pic?fileId=X24gYotabwCg03WC1YllbA==, createTime=1776238689628, updateTime=1776238689628, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007930925629, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=minRunFlag, value=0, createTime=1776238689644, updateTime=1776238689644, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007888982583, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/cqyddxxb/EN/file/pic, createTime=1776238689634, updateTime=1776238689634, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007922537020, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=silenceFlag, value=0, createTime=1776238689642, updateTime=1776238689642, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007880593974, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1776238689632, updateTime=1776238689632, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007901565497, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=themeColor, value=null, createTime=1776238689637, updateTime=1776238689637, creator=18614031015, updator=18614031015), WebsiteProps(id=1251199007909954106, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148516266372, code=themeStyle, value=null, createTime=1776238689639, updateTime=1776238689639, creator=18614031015, updator=18614031015)])], journalTitle=重庆邮电大学学报(自然科学版), weixinUrl=null, journalUrl=http://journal2.cqupt.edu.cn/jcuptnsecn, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), journalPhotoCn=eIjswk9Qxcq7+V27dEZ90g==, journalPhotoEn=PN1NU8XEwLhVlBzwQEoroA==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false), detailUrlCn=https://castjournals.cast.org.cn/joweb/cqyddxxb/CN/10.3979/j.issn.1673-825X.202410140254, detailUrlEn=https://castjournals.cast.org.cn/joweb/cqyddxxb/EN/10.3979/j.issn.1673-825X.202410140254, pdfUrlCn=https://castjournals.cast.org.cn/joweb/cqyddxxb/CN/PDF/10.3979/j.issn.1673-825X.202410140254, pdfUrlEn=https://castjournals.cast.org.cn/joweb/cqyddxxb/EN/PDF/10.3979/j.issn.1673-825X.202410140254, aliStartDate=null, aliEndDate=null, collectionFlag=false, citedCount=null, citedUrl=null, reference=null)
收藏切换
基于深度强化学习的无人机辅助RSMA中继通信技术
收藏切换
PDF下载
樊自甫 , 张珂瑞 , 王正强
重庆邮电大学学报(自然科学版) | 新一代移动通信 2025,37(5): 668-676
收起
收藏切换
重庆邮电大学学报(自然科学版) | 新一代移动通信 2025, 37(5): 668-676
基于深度强化学习的无人机辅助RSMA中继通信技术
全屏
樊自甫 , 张珂瑞 , 王正强
作者信息
  • 重庆邮电大学 通信与信息工程学院,重庆 400065
  • 樊自甫,教授,主要研究方向为网络优化、绿色通信。Email:

    张珂瑞,硕士研究生,主要研究方向为无人机通信、机器学习。E-mail:

    王正强,副教授,博士,硕士生导师,主要研究方向为无人机通信、物理层安全。E-mail:

通讯作者:

Unmanned aerial vehicle assisted RSMA relay communication technology based on deep reinforcement learning
Zifu FAN , Kerui ZHANG , Zhengqiang WANG
Affiliations
  • School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P R China
doi: 10.3979/j.issn.1673-825X.202410140254
文章导航
收藏切换

针对用户在存在窃听者的复杂通信环境进行中继通信的安全问题,提出了一种多无人机辅助的中继通信网络为用户提供通信服务。通过基于Q混合网络(Q-mixing network,QMIX)的多智能体深度强化学习(multi-agent reinforcement learning,MRAL)算法优化无人机轨迹与功率分配,在信息安全敏感度较低用户(次要用户)最低速率得到保障的情况下,提高信息安全敏感较高用户(主要用户)的安全和速率。仿真结果表明,算法相较于双层深度Q网络(double deep Q-network,Double DQN)和对偶深度Q网络(dueling deep Q-network,Dueling DQN),累积奖励分别提高了大约15.5%和1.26%;模型的速率分割多址技术相较于空分多址和非正交多址技术,在系统整体性能和信息安全保障方面都具有显著优势,为多用户通信场景下的安全高效通信提供了更优解决方案。

速率分割多址接入  /  无线通信网络  /  深度强化学习  /  安全和速率

To address the security challenges of relay communication in complex environments with potential eavesdroppers, this paper proposes a multi-UAV-assisted relay communication network that provides secure communication services for users. A multi-agent deep reinforcement learning(MARL)algorithm based on the Q-mixing network(QMIX)is employed to jointly optimize UAV trajectories and power allocation. The goal is to guarantee the minimum transmission rate of low-security-sensitivity users(secondary users)while enhancing the communication security and data rate of high-security-sensitivity users(primary users). Simulation results demonstrate that, compared with the Double Deep Q-Network(Double DQN)and the Dueling Deep Q-Network(Dueling DQN), the proposed algorithm improves the cumulative reward by approximately 15.5% and 1.26%, respectively. Moreover, the proposed rate-splitting multiple access(RSMA)technique significantly outperforms space-division multiple access(SDMA)and non-orthogonal multiple access(NOMA)in terms of overall system performance and information security. The proposed method provides an effective solution for achieving secure and efficient communication in multi-user wireless networks.

rate-splitting multiple access ( RSMA )  /  wireless communication networks  /  deep reinforcement learning (DRL)  /  security and rate
樊自甫, 张珂瑞, 王正强. 基于深度强化学习的无人机辅助RSMA中继通信技术. 重庆邮电大学学报(自然科学版), 2025 , 37 (5) : 668 -676 . DOI: 10.3979/j.issn.1673-825X.202410140254
Zifu FAN, Kerui ZHANG, Zhengqiang WANG. Unmanned aerial vehicle assisted RSMA relay communication technology based on deep reinforcement learning[J]. Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), 2025 , 37 (5) : 668 -676 . DOI: 10.3979/j.issn.1673-825X.202410140254
面对未来无线网络的海量用户接入与高度多样化需求,如何高效利用频谱为用户提供公平、高质量服务是无线通信技术发展的重要挑战。新一代移动通信系统需要更强大的物理层多用户接入技术。无人机(unmanned aerial vehicle,UAV)辅助通信因其操作灵活、覆盖范围广泛等特点被视为前景广阔的技术之一[1-2],能够为多用户提供灾难救援支援、实时监控和数据采集等服务[3]。无人机通信由于其覆盖范围广且连接规模大,易受窃听和恶意攻击威胁[4-5]。因此,设计安全的无人机辅助系统至关重要[6-7]
使用无人机进行中继通信是一种有效的信息传输方式,中继无人机的工作方式可以分为静态中继和动态中继,静态中继研究主要集中在寻找最佳悬停点以改善系统性能。文献[8]提出了一种集成的解码转发(decode-and-forward,DF)无人机中继通信方案,协助基站与多个用户间的通信。相比之下,动态中继更能充分利用无人机的灵活性,因此备受关注。文献[9]提出了一种时隙配对中继策略,以最大化双向中继网络的总速率。为确保安全传输,文献[10]提出了一种启发式无人机替代方案,通过多无人机协作延长通信时间。借助物理层安全措施,可以保障无人机的通信安全[11-12]。文献[12]提出了一种双无人机非正交多址接入技术(Non-orthogonal multiple access,NOMA)方案,一架无人机负责通信,另一架负责干扰。传统优化方法难以应对大量地面用户、无人机飞行时间和迭代次数的增长,因此多智能体强化学习算法逐渐成为解决问题的有效工具[13-14]。文献[13]在估计窃听者位置后,基于Qlearning算法优化无人机轨迹,在窃听者移动情况下最大化可达安全速率。
在保证无人机物理层安全的同时,速率拆分多址(rate-splitting multiple access,RSMA)作为一种基于分速率技术和线性预编码的多天线多用户多址方案,将用户消息分解为公共和私有部分,并通过编码和预编码技术将其转化为公共和私有流,利用多输入多输出(multipe input multiple output,MIMO)或多输入单输出(multiple input single output,MISO)信道进行叠加传输[15]。RSMA具有灵活管理干扰的能力,允许部分解码干扰信号[16]。已有研究表明,RSMA在5G及后续通信系统中表现优越于空分复用接入(space division multiple access,SDMA)、NOMA和正交多址接入(orthogonal multiple access,OMA)等现有多址方案[17]。文献[18]探讨了在直通和中继协调系统中采用RSMA传输策略,旨在最大化两个用户的最低可达速率。文献[19]研究了RSMA与无人机基站(UAV-mounted BS,UBS)相结合的应用,提升了私有消息的遍历容量和公共消息的预期容量。文献[20]分析了基于RSMA的UBS辅助网络的中断概率和吞吐量。然而,固定速率传输限制了RSMA的潜力,因此自适应速率传输成为发挥RSMA优势的关键[21]。文献[22]在基于RSMA星地融合网络同时还存在多个窃听者的条件下,研究了用户的通信质量要求与最大窃听概率的约束,以及如何最小化系统总发射功率。
上述关于无人机与RSMA安全通信的研究缺乏对用户安全等级划分的考虑,对所有用户提供同一级别的安全保护,这导致极大的资源浪费。本文针对无人机辅助RSMA中继通信系统,考虑不同用户安全等级,在为次要用户提供最低服务质量的情况下,最大化主要用户的安全和速率,主要贡献如下。
1)对安全通信不敏感的用户建立包含多无人机和窃听者的RSMA中继通信系统模型。在无人机发射功率、飞行区域、次要用户服务质量等条件约束下,提出了最大安全和速率资源分配问题。
2)为了处理多智能体联合优化问题,利用Q混合网络(Q-mixing network,QMIX)的多智能体强化学习算法对无人机轨迹和资源分配进行优化。
3)仿真对比RSMA协议与其他基准协议的性能。
包含双无人机、窃听者、保密用户和普通用户的无人机辅助安全通信模型,如图1所示。拥有N个天线的发射基站需要借助带K个天线的无人机中继(Relay)与M个单天线的用户进行通信。窃听者(Eve)一般只会窃听保密用户的通信信息。空中部署的单天线干扰无人机(Jammer)通过发射人工噪声干扰发射基站到Eve的窃听信道,降低窃听信道容量,保护合法用户的通信安全,同时解决人工噪声与信息流处于同一发射机时RSMA协议失效的问题。无被窃听风险的用户也需要通过中继无人机进行通信,Jammer和Relay在保证此类用户的速率高于阈值的情况下通过优化公共速率分配,最大化被窃听用户的安全通信速率。Relay和Jammer部署在高空中,采用RSMA传输协议广播信号给配备单天线的地面用户,设HminHmax是无人机飞行保障飞行安全的最低和最高高度限制。
在时隙t,无人机和地面节点的水平位置分别表示为(XRt),YRt)),(XJt),YJt))和(xiyi),i=1,2,…,K。建立三维笛卡尔坐标系,窃听者Ue的坐标表示为we=(xeye,0)。
本文采用空对地(A2G)模型来描述无人机与地面节点之间的信道系数。传播过程中地面节点与无人机之间的视距信道有一定概率受阻挡物的遮挡,视距信道的概率取决于地面节点与无人机之间的仰角。同时考虑无人机到地面用户之间的链路存在的视距链路(line-of-sight,LoS)分量和非视距链路(non-line-of-sight,NLoS)分量,得到一种空对地(airto-ground,A2G)的LoS信道的概率为
式(1)中:ab是固定的环境参数;θm,u是地面节点m与无人机u之间的仰角。θm,u表达式为
中继无人机、干扰无人机与地面节点m之间的信道平均功率增益分别表示为
本文通信模型包含双无人机、窃听者、保密用户和普通用户,如图1所示。发射基站拥有N个天线,借助一个带有K个天线的无人机中继与M个单天线用户进行通信。窃听者仅尝试窃取保密用户的通信信息。为了保护合法用户的通信安全,模型在空中部署了一个单天线干扰无人机,通过发射可被用户消除的人工噪声干扰窃听信道,降低窃听信道的容量,解决了人工噪声与信息流在同一发射机时RSMA协议的失效问题[23]。没有被窃听风险的用户需要通过中继无人机进行通信。在保证普通用户速率高于阈值的前提下,通过优化公共速率分配、Jammer和Relay的轨迹,最大化被窃听用户的通信安全性。使用预编码对信号进行优化,基站与无人机中继发射端的波束成形向量可以表示为Wi∈ℂ1∈ℂ1。在DF中继通信中,通信过程分为两个阶段。在第一阶段,发射基站将信息发送给中继无人机,后者接收到信号后对其进行解码;在第二阶段,中继无人机将解码后的信号以解码转发方式发送给用户。这种方式可以有效避免中继节点的噪声传递至目的节点,防止目的节点处的噪声过大。对于中继无人机来说,首先需要去除人工噪声,然后通过SIC解码公共信号,将所有私有流视为噪声,一旦成功解码公共消息,就可以删除公共消息,接着继续将其他用户的私有信号视为噪声,以便解码接收方的私有信号。这种解码流程有助于提高通信质量和可靠性。
在第一阶段,基站发送广播信号
式(4)—(5)中:d是数据符号,且满足E[|d2]=1;wcwm为基站发送公共信号和私有信号的预编码矩阵;dc,m为用户m的公共信号;am为用户m的公共信号在系统信号中的占比。dc,mam在无人机中继发送的信号中不变。中继处接收到来自基站的信号为
无人机公有流和私有流可实现速率分别为
中继使用DF协议发送信号到用户m,则m接收的信号为
用户公有流和私有流可实现速率分别为
根据信息论原理,基站到用户的信号速率可以等价于基站到Relay,Relay到用户速率中的较小者。
窃听者可以从发射基站发射信息或从无人机发射消息中获取保密用户的信息,窃听速率小于或等于两种方式中的较大值。窃听者公有流和私有流可实现速率分别为
窃听者在第一阶段的速率可以表示为
在第二阶段,窃听者公有流和私有流可实现速率可以表示为
基站到用户在时刻t的安全速率可以表示为
将式(12)、式(14)—(16)代入式(23)可知,可达安全速率受无人机与地面节点间信道增益及BS、Relay预编码矩阵影响。由式(1)可知,无人机与地面节点间信道增益与视距概率相关;随着无人机与地面节点间距离增加,路径损耗增大,而无人机到地面节点视距概率与无人机相对于地面节点的位置高度相关,预编码矩阵则受到分配给不同数据流的功率分配系数控制。
考虑到无人机飞行高度限制和用户服务质量约束,通过多智能体深度强化学习联合优化Jammer和Relay的轨迹以及Relay和基站的预编码矩阵,在保障次要用户最低通信质量的情况下使得主要用户的安全和速率最大化,在增大用户可达速率与降低窃听者可达速率之间取得平衡,优化问题可以表述为
式(24)中:C1是预编码矩阵功率约束;C2是用户的最小传输速率要求;C3-C5代表无人机飞行区域限制的水平;C6是无人机避免碰撞的限制。由于变量和目标函数高度耦合,问题(P)显然是一个非凸优化问题,求解困难。利用QMIX深度强化学习算法,可以有效解决多无人机的联合轨迹优化。
强化学习(reinforcement learning,RL)主要包括一个智能体和与之交互的环境,其形式表征为马尔可夫决策过程(Markov decision process,MDP)。在MDP中,用S表示所有可能的状态集合,A表示所有可能的动作集合,R表示奖励函数,T表示状态转移概率。智能体从累积经验中学习最佳策略,以最大化长期收益。Q-Learning是一种基于价值的RL算法,使用Q值表来存储动作-价值函数,并根据这些Q值选择能带来最大收益的动作。更新策略为[24-25]
当环境中存在多个智能体时,将标准MDP扩展为(MOART)所描述的马尔可夫博弈,MN个智能体的集合,在本系统模型中,智能体为中继和干扰无人机。在时隙t,每个智能体通过局部观测值选择自己的动作,并获得个体奖励和下一次观测值。每个智能体获得的奖励不仅取决于自己的行为,还取决于其他智能体的行为。将式(24)中的优化问题转换为MARL决策任务,任务环境为位置已知的基站、用户和位置部分已知的窃听者的三维空间。QMIX算法通过在训练学习过程中加入全局状态信息,提高了系统性能。在满足单调性约束情况下,多智能体系统局部动作值函数取最大值等价于联合动作值函数取最大值,从而实现集中式学习,分布式策略[26],表示为
通过端到端的训练最小化损失函数,更新QMIX算法的策略为
式(26)—(28)中,Qtotal为混合网络(Mixing)中每一个Q值与全局状态的混合,在训练时每一个智能体都训练一个独立的卷积神经网络。QMIX网络采用和深度Q网络(deep Q-network,DQN)相同的方式更新,即
式(29)中,θ-为DQN网络中的TD误差。
智能体采用QMIX算法对飞行轨迹和功率分配进行联合优化,以实现最大化系统可达性和速率的目标。在每个时隙,UAV接收观测值,并构建位置和通信关系。每个智能体通过卷积处理局部观测,并与其他UAV交换信息,然后输出动作,更新UAV的位置和功率分配。时间步长结束后,所有智能体立即收到奖励以及下一个观测值。基于QMIX的多智能体深度强化学习架构如图2所示。在每个时间步之后,所有智能体的观察值、动作、奖励和下一次观察值作为经验被存储到经验重播存储器中。算法具体步骤如算法1。
算法1 基于QMIX的无人机轨迹与资源分配联合优化
[1]对于每一轮训练,当Episode小于最大训练次数Episode_limit时,执行算法;
[2]初始化环境,重置用户和窃听者位置;
[3]选择训练智能体i,同时访问链接其他智能体观测空间;
[4]当飞行训练时间tNt时执行:
a)基站进行编码向中继无人机发送信息,中继无人机编码并向用户发送信息,干扰无人机发射人工噪声;
b)用户和窃听者解码接收中继无人机发送的信号;
c)飞行结束,智能体依据ε-贪婪策略选择智能体动作,获得新的奖励值
d)t=t+1;
e)当t=Nt时结束;
[5]存储经验到回放空间;
[6]随机从回放空间中采样一些数据,通过Mixing网络得到Qtotal并计算TD误差,根据式(24)通过反向传播算法更新;
[7] Episode=Episode+1;
[8]当Episode=Episode_limit时,循环结束;
本文通过模拟实验验证基于QMIX的无人机轨迹与功率分配算法的性能,主要设计了两个实验,分别展现不同深度强化学习算法的优点和RSMA通信在不同速率约束下同SDMA、NOMA协议对比的优势。表2为实验环境中的各项参数。BS、Relay和Jammer的初始坐标分别为(0,0)、(0,200)和(200,0)m,主要用户位于[4000±1000,4000± 1000]m范围内的随机位置,窃听者被部署于[4000±750,2000±750]m范围内的随机位置,次要用户位于[1500±1000,4000±1000]m范围内的随机位置。
不同约束条件、不同通信协议对系统性能有着重要的影响。本文对比了SDMA、NOMA通信协议与RSMA通信协议在0、1.1、2.2 bit·Hz-1·s-1三种次要用户QoS约束情况下每个主要用户在整个无人机飞行周期内的平均安全速率的收敛情况,如图3所示。
图3可以看出,RSMA的性能在三种情况下均好于其他两种协议,在双用户情况下和NOMA较为接近。具体来说RSMA算法相较于SDMA、NOMA协议在三种情况下安全用户的平均速率分别提高了大约35%和96%、48%和175%、78%和250%。NOMA和SDMA协议中的主要用户速率显著下降,RSMA协议主要用户速率下降较慢。在收敛速度方面,随着约束速率的增加,三种算法的收敛轮次都会增加,RSMA公共速率可以满足次要用户一部分需求,三种算法的收敛速度接近。
无人机轨迹和系统功率优化具有重要意义,在次要用户最低QoS为2.2 bit·Hz-1·s-1条件下,无人机轨迹与功率分配情况如图4所示。图4a中,无人机固定高度为50 m,图4b中,无人机高度位于[50,600]m范围内。通过图4可以了解到,中继无人机会通过绕开窃听者的方式接近主要用户,优于次要用户最低服务质量约束,其最终位置既不会在主要用户正上方,也不会在次要用户正上方,而会倾向于悬停在主要用户和次要用户之间的区域,具有一定的高度,保障主要用户和次要用户的通信质量。
对信息进行功率分配时,在发射机侧进行预编码对于目标函数最大化具有重要意义。系统会首先抬高信号中次要用户私有信号和公共信息的占比,优先满足次要用户的服务质量约束,在无人机到达一定位置后则会倾向于增加主要用户信息量占比,增加安全和速率,中继无人机功率分配如图5所示。随着主要用户数量的增加,由于RSMA协议中其他用户的私有速率对于本用户和窃听者均为干扰项,用户能够对公共信号进行解码。随着主要用户的增加,系统中的安全和速率会略微降低,稳定在5 bit·Hz-1·s-1左右,用户数量和用户平均安全速率反相关如图6所示。
与传统的单智能体优化算法相比,基于QMIX对多无人机的轨迹与系统的功率分配系数进行优化,可以获得更准确的分配功率并控制无人机轨迹。如图7所示为QMIX,DoubleDQN,DuelingDQN三种强化学习算法的系统收敛图。从图7可以看到,QMIX算法收敛性能最好,其次为DuelingDQN算法,而DoubleDQN算法的性能最差。
本文在次要用户最低服务质量得到保障的前提下最大化主要用户安全和速率,利用无人机部署灵活、与地面通信具有视距信道的优势,为用户提供中继服务。采用基于QMIX的无人机轨迹优化算法,实现系统安全速率最大。仿真结果表明,本文优化算法相较于其他算法性能更优,随着中继无人机发射功率的增加,本文所用RSMA协议获得了更大的安全速率;使用基于QMIX的多智能体强化学习算法可以获得更好性能。未来工作中,将考虑在符合实际的多用户场景下,在存在多个合法接收者和窃听者时,无人机的部署和轨迹、功率优化问题。
参考文献 引证文献
排序方式:
[1]
ZHOU X B, YAN S H, DERRICK W K N. Three-dimensional placement and transmit power design for UAV covert communications[J]. IEEE Transactions on Vehicular Technology, 2021, 70(12): 13424-13429.
[2]
LI B, FEI Z S, ZHANG Y, et.al. Secure UAV communication networks over 5G[J]. IEEE Wireless Communication, 2019, 26(5): 114-120.
[3]
NGUYEN P X, NGUYEN V D, NGUYEN H V, et al. UAV assisted secure communications in terrestrial cognitive radio networks:Joint power control and 3D trajectory optimization[J]. IEEE Transactions on Vehicular Technology, 2021, 70(4): 3298-3313.
[4]
YANG B, TALEB T, SHEN Y L, et al. Performance, fairness,and tradeoff in UAV swarm underlaid mmWave cellular networks with directional antennas[J]. IEEE Transactions on Wireless Communications, 2021, 20(4): 2383-2397.
[5]
ZHU L P, ZHANG J, XIAO Z Y, et al. Millimeter-wave full-duplex UAV relay: Joint positioning, beamforming, and power control[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(9): 2057-2073.
[6]
DU H Y, NIYATO D, XIE Y A, et al. Performance analysis and optimization for jammer-aided multiantenna UAV covert communication[J]. IEEE Journal on Selected Areas in Communications, 2022, 40(10): 2962-2979.
[7]
WANG Z, GUO J H, CHEN Z Q, et al. Robust secure UAV relay-assisted cognitive communications with resource allocation and cooperative jamming[J]. Journal of Communications and Networks, 2021, 24(2): 139-153.
[8]
SINGH S K, AGRAWAL K, SINGH K, et al. RSMA enhanced RIS-FD-UAV-aided short packet communications under imperfect SIC[C]//2022 IEEE Globecom Workshops. Riode Janeiro, Brazil: IEEE, 2022: 1549-1554.
[9]
LIANG Y P, XIAO L, YANG D C, et al. Joint trajectory and resource optimization for UAV-aided two-way relay networks[J]. IEEE Transactions on Vehicular Technology, 2022, 71(1): 639-652.
[10]
ZHANG G C, YAN H Q, ZENG Y, et al. Cooperative UAV enabled relaying systems: Joint trajectory and transmit power optimization[J]. IEEE Transactions on Green Communications and Networking, 2022, 6(1): 543-557.
[11]
CHEN X Y, YANG Z T, ZHAO N, et al. Secure transmission via power allocation in NOMA-UAV networks with circular trajectory[J]. IEEE Transactions on Vehicular Technology, 2020, 69(9): 10033-10045.
[12]
程擎,何汶键,贾志杰,.基于改进YOLOv7的低空小型无人机识别算法研究[J].贵州师范大学学报(自然科学版), 2025, 43(5): 45-53.
CHENG Q, HE W J, JIA Z J, et al. Researchon low altitude small UAVs recognition algorithm based on improved YOLOv7[J]. Journal of Guizhou Normal University(Natural Sciences), 2025, 43(5): 45-53.
[13]
KANG H Y, CHANG X L, MISIC J, et al. Improving dual-UAV aided ground-UAV bi-directional communication security: Joint UAV trajectory and transmit power optimization[J]. IEEE Transactions on Vehicular Technology, 2022, 71(10): 10570-10583.
[14]
高远,谭蓉俊,邓志祥.无人机辅助物理层安全下的保密性能优化[J].电子与信息学报, 2022, 44(8):2730-2738.
GAO Y, TAN R G, DENG Z X. Secrecy performance optimization of unmanned aerial vehicle-aided physical layer security[J]. Journal of Electronics & Information Technology, 2022, 44(8): 2730-2738.
[15]
CLERCKX B, JOUDEH H, HAO C, et al. Rate splitting for MIMO wireless networks: A promising PHY-layer strategy for LTE-evolution[J]. IEEE Communications Magazine, 2016, 54(5): 98-105.
[16]
CLERCKX B, MAO Y J, SCHOBER R, et al. Rate-splitting unifying SDMA, OMA, NOMA, and multicasting in MISO broadcast channel: A simple two-user rate analysis[J]. IEEE Transactions on Wireless Communications Letters, 2020, 9(3): 349-353.
[17]
PANG H R, JI F, NING Z L, et al. Enhancing security in RSMA networks with cooperative jamming and relaying[C]//2023 IEEE 24th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). Shanghai, China: IEEE, 2023: 176-180.
[18]
许钧智,李陶深,葛志辉.自能量回收的全双工无人机中继资源分配[J].小型微型计算机系统, 2024, 45(7): 1763-1769.
XU JZ,LITS,GEZH. Resource allocation for full-duplex UAV relaying with self-energized technology[J]. Journal of Chinese Computer Systems, 2024, 45(7):1763-1769.
[19]
SINGH S K, AGRAWAL K, SINGH K, et al. Ergodic capacity and placement optimization for RSMA-enabled UAV-assisted communication[J]. IEEE Systems Journal, 2023, 17(2): 2586-2589.
[20]
HOANG T M, DUNG L T, NGUYEN B C, et al. Outage probability and throughput analysis of UAV-assisted rate-splitting multiple access[J]. IEEE Wireless Communications Letters, 2021, 10(11): 2528-2532.
[21]
MAO Y J, DIZDAR O, CLERCKX B, et al. Rate-splitting multiple access: Fundamentals, survey, and future research trends[J]. IEEE Communications Surveys Tutorials, 2022, 24(4): 2073-2126
[22]
赵柏,林敏,肖圣杰,.基于速率分割的可重构智能表面辅助星地融合网络鲁棒安全传输方案[J].通信学报, 2023, 44(12): 50-60.
ZHAO B, LIN M, XIAO S J, et al. Rate splitting based robust secure transmission scheme in RIS-assisted satellite-terrestrial integrated network[J]. Journal on Communications, 2023, 44(12): 50-60.
[23]
夏慧云,韩帅,李成.人工噪声在速率分拆多址保密通信中的作用[J].信号处理, 2023, 39(7): 1214-1221.
XIA H Y, HAN S, LI C. The role of artificial noise in RSMA-based secure communications[J]. Journal of Signal Processing, 2023, 39(7): 1214-1221.
[24]
周毅,马晓勇,郜富晓,.基于深度强化学习的无人机自主部署及能效优化策略[J].物联网学报, 2019, 3(2): 47-55.
ZHOU Y, MA X Y, HAO F X, et al. Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning[J]. Chinese Journal on Internet of Things, 2019, 3(2): 47-55.
[25]
ZHANG X C, ZHAO H T, WEI J B, et al. Cooperative trajectory design of multiple UAV base stations with heterogeneous graph neural networks[J]. IEEE Transactions on Wireless Communications, 2023, 22(3): 1495-1509.
[26]
GUO Z H, WU Y, WANG L F, et al. Coordination for connected and automated vehicles at non-signalized intersections: A value decomposition-based multiagent deep reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2023, 72(3): 3025-3034.
2025年第37卷第5期
PDF下载
138
70
引用本文
BibTeX
文章信息
doi: 10.3979/j.issn.1673-825X.202410140254
  • 接收时间:2024-10-14
  • 首发时间:2026-04-16
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2024-10-14
  • 修回日期:2025-09-15
基金
作者信息
    重庆邮电大学 通信与信息工程学院,重庆 400065

通讯作者:

参考文献
分享链接
https://castjournals.cast.org.cn/joweb/cqyddxxb/CN/10.3979/j.issn.1673-825X.202410140254
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数

Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)

Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏