Article(id=1251226688114344374, tenantId=1146029695717560320, journalId=1251194772300279900, issueId=1251226682309423223, articleNumber=null, orderNo=null, doi=10.20079/j.issn.1001-893x.240715002, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1720972800000, receivedDateStr=2024-07-15, revisedDate=1730995200000, revisedDateStr=2024-11-08, acceptedDate=null, acceptedDateStr=null, onlineDate=1776245289114, onlineDateStr=2026-04-15, pubDate=1764259200000, pubDateStr=2025-11-28, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1776245289114, onlineIssueDateStr=2026-04-15, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1776245289114, creator=13041195026, updateTime=1776245289114, updator=13041195026, issue=Issue{id=1251226682309423223, tenantId=1146029695717560320, journalId=1251194772300279900, year='2025', volume='65', issue='11', pageStart='1729', pageEnd='1954', issueExtLink='null', onlineDate='null', pubDate='null', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=1, specialIssue=null, createTime=1776245287729, creator=13041195026, updateTime=1776246742124, updator=13041195026, preIssue=null, nextIssue=null, ext={EN=IssueExt(id=1251232782568080068, tenantId=1146029695717560320, journalId=1251194772300279900, issueId=1251226682309423223, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1251232782568080069, tenantId=1146029695717560320, journalId=1251194772300279900, issueId=1251226682309423223, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null}, startPage=1820, endPage=1827, ext={EN=ArticleExt(id=1251226688412139961, articleId=1251226688114344374, tenantId=1146029695717560320, journalId=1251194772300279900, language=EN, title=An Intelligent Communication Anti-interference Decision Algorithm Based on Multiple Reward Value DDQN, columnId=1251226683223781499, journalTitle=Telecommunication Engineering, columnName=Application Fundamental Research and Advanced Technology, runingTitle=null, highlight=null, articleAbstract=

In satellite communication systems operating in dynamic interference environments,the quality of channels and the interference power vary. Limited spectrum resources and complex interference environments pose challenges for anti-interference communication decisions, particularly in terms of resource allocation and service demands. Specifically, the challenge lies in efficiently utilizing resources while avoiding interference frequencies and optimizing power. To address this issue,a deep reinforcement learning-based anti-interference algorithm with multiple reward functions is proposed. The algorithm models the interaction between the transmitter,receiver,and interferer as a Markov decision process. By optimizing the reward function associated with the costs of channel and power switching,it introduces mechanisms for both frequency and power switching,analyzes the interference characteristics in the spectrum of adjacent time slots, and integrates the interference signal features collected during the interaction with channel information to train an anti-interference strategy. This strategy enables joint anti-interference decision-making in both the frequency and power domains. Simulation results demonstrate that the algorithm effectively reduces the probability of interference,accelerates convergence,and optimizes the utilization of power resources.

, correspAuthors=null, authorNote=null, correspAuthorsNote=null, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, authorCompany=null, fund=null, authors=null, authorsList=Yao LING, Shijun XIE, Hao LIANG, Jiao FENG, Weijie GAO), CN=ArticleExt(id=1251226696519729892, articleId=1251226688114344374, tenantId=1146029695717560320, journalId=1251194772300279900, language=CN, title=基于多奖励值DDQN智能通信抗干扰决策方法, columnId=1251226683383165054, journalTitle=电讯技术, columnName=应用基础与前沿技术, runingTitle=null, highlight=null, articleAbstract=

在动态干扰环境下的卫星通信系统中,各信道的质量和干扰功率存在差异。有限的频谱资源和复杂的干扰环境对抗干扰通信决策提出了资源分配和业务需求的挑战,即如何在避开干扰频率和优化功率的同时,实现资源的高效利用。为解决这一问题,提出了一种基于多奖励值函数的深度强化学习抗干扰算法。该算法将发送方、接收方与干扰方之间的交互建模为马尔可夫决策过程。通过优化信道切换与功率切换代价的奖励函数,引入频率切换与功率切换机制,分析相邻时隙频谱中的干扰特征,并将交互过程中采集到的干扰信号特征与信道信息结合,用于训练抗干扰策略。该策略实现了频率域与功率域的联合抗干扰决策。仿真结果表明,该算法能够有效降低系统的受干扰概率,加快算法收敛速度,并优化功率资源的利用效率。

, correspAuthors=null, authorNote=null, correspAuthorsNote=
谢世珺 Email:
, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=OjZkebcCBlacH/h6GPslkg==, magXml=ToPCxo9JPJTsbKgqghTZeA==, pdfUrl=null, pdf=ALNS3FpozUjX9Wh6uPBYPA==, pdfFileSize=4619958, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=hg8e+XZ/RhMVaHe8N+09/Q==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=j+bDN/7kaJ+QCb6KLj0uVA==, mapNumber=null, authorCompany=null, fund=null, authors=

凌耀 男,1998年生于江苏盐城,2021年获工学学士学位,现为硕士研究生,主要研究方向为智能抗干扰决策。

谢世珺 女,1980年生于贵州凯里,2005年获工学硕士学位,现为副研究员,主要研究方向为卫星通信。

梁豪 男,1993年生于山东泰安,2020年获工学博士学位,现为助理研究员,主要研究方向为卫星通信。

冯姣 女,1984年生于吉林延吉,2014年获工学博士学位,现为副教授,主要研究方向为无线通信。

高伟杰 男,2000年生于江苏无锡,2022年获工学学士学位,现为硕士研究生,主要研究方向为智能抗干扰决策。

, authorsList=凌耀, 谢世珺, 梁豪, 冯姣, 高伟杰)}, authors=[Author(id=1251226697031435020, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1251226697144681237, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226697031435020, language=EN, stringName=Yao LING, firstName=Yao, middleName=null, lastName=LING, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1School of Electronic and Information Engineering,Nanjing University of Information Science & Technology,Nanjing 210044,China
2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1251226697316647709, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226697031435020, language=CN, stringName=凌耀, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1南京信息工程大学 电子与信息工程学院,南京 210044
2国防科技大学第六十三研究所,南京 210007, bio={"content":"

凌耀 男,1998年生于江苏盐城,2021年获工学学士学位,现为硕士研究生,主要研究方向为智能抗干扰决策。

"}, bioImg=null, bioContent=

凌耀 男,1998年生于江苏盐城,2021年获工学学士学位,现为硕士研究生,主要研究方向为智能抗干扰决策。

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1251226696813331192, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=1, ext=[AuthorCompanyExt(id=1251226696825914105, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696813331192, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electronic and Information Engineering,Nanjing University of Information Science & Technology,Nanjing 210044,China), AuthorCompanyExt(id=1251226696834302717, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696813331192, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1南京信息工程大学 电子与信息工程学院,南京 210044)]), AuthorCompany(id=1251226696909800194, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=2, ext=[AuthorCompanyExt(id=1251226696913994499, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China), AuthorCompanyExt(id=1251226696922383108, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2国防科技大学第六十三研究所,南京 210007)])]), Author(id=1251226697434088228, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=xsjxsj_520@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1251226697543140140, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226697434088228, language=EN, stringName=Shijun XIE, firstName=Shijun, middleName=null, lastName=XIE, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1251226697668969268, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226697434088228, language=CN, stringName=谢世珺, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2国防科技大学第六十三研究所,南京 210007, bio={"content":"

谢世珺 女,1980年生于贵州凯里,2005年获工学硕士学位,现为副研究员,主要研究方向为卫星通信。

"}, bioImg=null, bioContent=

谢世珺 女,1980年生于贵州凯里,2005年获工学硕士学位,现为副研究员,主要研究方向为卫星通信。

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1251226696909800194, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=2, ext=[AuthorCompanyExt(id=1251226696913994499, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China), AuthorCompanyExt(id=1251226696922383108, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2国防科技大学第六十三研究所,南京 210007)])]), Author(id=1251226699233444673, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1251226699355079496, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226699233444673, language=EN, stringName=Hao LIANG, firstName=Hao, middleName=null, lastName=LIANG, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1251226699451548493, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226699233444673, language=CN, stringName=梁豪, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2国防科技大学第六十三研究所,南京 210007, bio={"content":"

梁豪 男,1993年生于山东泰安,2020年获工学博士学位,现为助理研究员,主要研究方向为卫星通信。

"}, bioImg=null, bioContent=

梁豪 男,1993年生于山东泰安,2020年获工学博士学位,现为助理研究员,主要研究方向为卫星通信。

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1251226696909800194, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=2, ext=[AuthorCompanyExt(id=1251226696913994499, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China), AuthorCompanyExt(id=1251226696922383108, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2国防科技大学第六十三研究所,南京 210007)])]), Author(id=1251226699535434578, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1251226699619320665, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226699535434578, language=EN, stringName=Jiao FENG, firstName=Jiao, middleName=null, lastName=FENG, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Electronic and Information Engineering,Nanjing University of Information Science & Technology,Nanjing 210044,China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1251226699719983969, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226699535434578, language=CN, stringName=冯姣, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1南京信息工程大学 电子与信息工程学院,南京 210044, bio={"content":"

冯姣 女,1984年生于吉林延吉,2014年获工学博士学位,现为副教授,主要研究方向为无线通信。

"}, bioImg=null, bioContent=

冯姣 女,1984年生于吉林延吉,2014年获工学博士学位,现为副教授,主要研究方向为无线通信。

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1251226696813331192, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=1, ext=[AuthorCompanyExt(id=1251226696825914105, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696813331192, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electronic and Information Engineering,Nanjing University of Information Science & Technology,Nanjing 210044,China), AuthorCompanyExt(id=1251226696834302717, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696813331192, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1南京信息工程大学 电子与信息工程学院,南京 210044)])]), Author(id=1251226699820647275, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1251226699912921977, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226699820647275, language=EN, stringName=Weijie GAO, firstName=Weijie, middleName=null, lastName=GAO, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1School of Electronic and Information Engineering,Nanjing University of Information Science & Technology,Nanjing 210044,China
2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1251226700017779581, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, authorId=1251226699820647275, language=CN, stringName=高伟杰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1南京信息工程大学 电子与信息工程学院,南京 210044
2国防科技大学第六十三研究所,南京 210007, bio={"content":"

高伟杰 男,2000年生于江苏无锡,2022年获工学学士学位,现为硕士研究生,主要研究方向为智能抗干扰决策。

"}, bioImg=null, bioContent=

高伟杰 男,2000年生于江苏无锡,2022年获工学学士学位,现为硕士研究生,主要研究方向为智能抗干扰决策。

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1251226696813331192, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=1, ext=[AuthorCompanyExt(id=1251226696825914105, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696813331192, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electronic and Information Engineering,Nanjing University of Information Science & Technology,Nanjing 210044,China), AuthorCompanyExt(id=1251226696834302717, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696813331192, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1南京信息工程大学 电子与信息工程学院,南京 210044)]), AuthorCompany(id=1251226696909800194, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=2, ext=[AuthorCompanyExt(id=1251226696913994499, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China), AuthorCompanyExt(id=1251226696922383108, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2国防科技大学第六十三研究所,南京 210007)])])], keywords=[Keyword(id=1251226700118442882, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, orderNo=1, keyword=intelligent communication anti-interference), Keyword(id=1251226700219106187, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, orderNo=2, keyword=joint anti-interference decision), Keyword(id=1251226700336546707, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, orderNo=3, keyword=deep reinforcement learning), Keyword(id=1251226700416238488, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, orderNo=4, keyword=multiple reward value functions), Keyword(id=1251226700558844835, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, orderNo=1, keyword=智能通信抗干扰), Keyword(id=1251226700655313834, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, orderNo=2, keyword=联合抗干扰决策), Keyword(id=1251226700785337270, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, orderNo=3, keyword=深度强化学习), Keyword(id=1251226700877611964, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, orderNo=4, keyword=多奖励值函数)], refs=[Reference(id=1251226704602153047, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2024, volume=64, issue=1, pageStart=149, pageEnd=157, url=null, language=null, rfNumber=[1], rfOrder=0, authorNames=陈书恒, 莫嘉倩, 莫小欣, journalName=电讯技术, refType=null, unstructuredReference=陈书恒, 莫嘉倩, 莫小欣.机载低轨卫星通信发展及关键技术综述[J].电讯技术, 2024, 64(1):149-157., articleTitle=机载低轨卫星通信发展及关键技术综述, refAbstract=null), Reference(id=1251226704702816349, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2021, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=[2], rfOrder=1, authorNames=魏鹏, journalName=卫星通信智能抗干扰决策技术研究, refType=null, unstructuredReference=魏鹏.卫星通信智能抗干扰决策技术研究[D].长沙:国防科技大学, 2021., articleTitle=null, refAbstract=null), Reference(id=1251226704816062563, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2021, volume=2, issue=3, pageStart=213, pageEnd=243, url=null, language=null, rfNumber=[3], rfOrder=2, authorNames=FOURATI F, ALOUINI M S, journalName=Intelligent and Converged Networks, refType=null, unstructuredReference=FOURATI F, ALOUINI M S. Artificial intelligence for satellite communication: a review[J]. Intelligent and Converged Networks, 2021, 2(3):213-243., articleTitle=Artificial intelligence for satellite communication: a review, refAbstract=null), Reference(id=1251226704937697386, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2023, volume=9, issue=6, pageStart=1579, pageEnd=1595, url=null, language=null, rfNumber=[4], rfOrder=3, authorNames=ZHANG Y P, ZHAO Z J, ZHENG S L, journalName=IEEE Transactions on Cognitive Communications and Networking, refType=null, unstructuredReference=ZHANG Y P, ZHAO Z J, ZHENG S L, et al. Intelligent anti-jamming decision with continuous action and state in bivariate frequency agility communication system[J]. IEEE Transactions on Cognitive Communications and Networking, 2023, 9(6):1579-1595., articleTitle=Intelligent anti-jamming decision with continuous action and state in bivariate frequency agility communication system, refAbstract=null), Reference(id=1251226705042554992, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2023, volume=22, issue=7, pageStart=4625, pageEnd=4637, url=null, language=null, rfNumber=[5], rfOrder=4, authorNames=LI X C, CHEN J N, LING X, journalName=IEEE Transactions on Wireless Communications, refType=null, unstructuredReference=LI X C, CHEN J N, LING X, et al. Deep reinforcement learning-based anti-jamming algorithm using dual action network[J]. IEEE Transactions on Wireless Communications, 2023, 22(7):4625-4637., articleTitle=Deep reinforcement learning-based anti-jamming algorithm using dual action network, refAbstract=null), Reference(id=1251226705206132853, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2024, volume=73, issue=3, pageStart=3568, pageEnd=3579, url=null, language=null, rfNumber=[6], rfOrder=5, authorNames=QI J, ZHANG H M, QI X L, journalName=IEEE Transactions on Vehicular Technology, refType=null, unstructuredReference=QI J, ZHANG H M, QI X L, et al. Deep reinforcement learning based hopping strategy for wideband anti-jamming wireless communications[J]. IEEE Transactions on Vehicular Technology, 2024, 73(3):3568-3579., articleTitle=Deep reinforcement learning based hopping strategy for wideband anti-jamming wireless communications, refAbstract=null), Reference(id=1251226705277436027, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2020, volume=14, issue=4, pageStart=682, pageEnd=692, url=null, language=null, rfNumber=[7], rfOrder=6, authorNames=NOORI H, SADEGHI VILNI S, journalName=IET Communications, refType=null, unstructuredReference=NOORI H, SADEGHI VILNI S. Jamming and anti-jamming in interference channels:a stochastic game approach[J]. IET Communications, 2020, 14(4):682-692., articleTitle=Jamming and anti-jamming in interference channels:a stochastic game approach, refAbstract=null), Reference(id=1251226705399070852, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2020, volume=69, issue=5, pageStart=5331, pageEnd=5342, url=null, language=null, rfNumber=[8], rfOrder=7, authorNames=HAN C, HUO L Y, TONG X H, journalName=IEEE Transactions on Vehicular Technology, refType=null, unstructuredReference=HAN C, HUO L Y, TONG X H, et al. Spatial anti-jamming scheme for Internet of satellites based on the deep reinforcement learning and Stackelberg game[J].IEEE Transactions on Vehicular Technology, 2020, 69(5):5331-5342., articleTitle=Spatial anti-jamming scheme for Internet of satellites based on the deep reinforcement learning and Stackelberg game, refAbstract=null), Reference(id=1251226705520705672, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2021, volume=2, issue=null, pageStart=775, pageEnd=784, url=null, language=null, rfNumber=[9], rfOrder=8, authorNames=ZHOU Q, LI Y G, NIU Y T, journalName=IEEE Open Journal of the Communications Society, refType=null, unstructuredReference=ZHOU Q, LI Y G, NIU Y T. Intelligent anti-jamming communication for wireless sensor networks:a multi-agent reinforcement learning approach[J]. IEEE Open Journal of the Communications Society, 2021, 2:775-784., articleTitle=Intelligent anti-jamming communication for wireless sensor networks:a multi-agent reinforcement learning approach, refAbstract=null), Reference(id=1251226705642340497, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2022, volume=58, issue=8, pageStart=1, pageEnd=11, url=null, language=null, rfNumber=[10], rfOrder=9, authorNames=王桂胜, 董淑福, 黄国策, journalName=计算机工程与应用, refType=null, unstructuredReference=王桂胜, 董淑福, 黄国策.无人系统认知联合抗干扰通信研究综述[J].计算机工程与应用, 2022, 58(8):1-11., articleTitle=无人系统认知联合抗干扰通信研究综述, refAbstract=null), Reference(id=1251226705734615187, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2013, volume=59, issue=5, pageStart=3237, pageEnd=3248, url=null, language=null, rfNumber=[11], rfOrder=10, authorNames=ZENG X Y, CAI H, TANG X H, journalName=IEEE Transactions on Information Theory, refType=null, unstructuredReference=ZENG X Y, CAI H, TANG X H, et al. Optimal frequency hopping sequences of odd length[J]. IEEE Transactions on Information Theory, 2013, 59(5):3237-3248., articleTitle=Optimal frequency hopping sequences of odd length, refAbstract=null), Reference(id=1251226705839472790, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2021, volume=9, issue=null, pageStart=29980, pageEnd=29992, url=null, language=null, rfNumber=[12], rfOrder=11, authorNames=XU H, CHENG Y F, WANG P Y, journalName=IEEE Access, refType=null, unstructuredReference=XU H, CHENG Y F, WANG P Y. Jamming detection in broadband frequency hopping systems based on multi-segment signals spectrum clustering[J]. IEEE Access, 2021, 9:29980-29992., articleTitle=Jamming detection in broadband frequency hopping systems based on multi-segment signals spectrum clustering, refAbstract=null), Reference(id=1251226705948524700, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2022, volume=null, issue=null, pageStart=734, pageEnd=738, url=null, language=null, rfNumber=[13], rfOrder=12, authorNames=WU J L, GUO F C, journalName=null, refType=null, unstructuredReference=WU J L, GUO F C. Time-frequency parameter estimation method of frequency hopping signal based on morphology method under low SNR[C]//2021 IEEE 6th International Conference on Signal and Image Processing. Nanjing:IEEE, 2022:734-738., articleTitle=Time-frequency parameter estimation method of frequency hopping signal based on morphology method under low SNR, refAbstract=null), Reference(id=1251226706086936738, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2024, volume=2, issue=null, pageStart=956, pageEnd=977, url=null, language=null, rfNumber=[14], rfOrder=13, authorNames=DOANIS P, SPYROPOULOS T, journalName=IEEE Transactions on Machine Learning in Communications and Networking, refType=null, unstructuredReference=DOANIS P, SPYROPOULOS T. Sample-efficient multiagent DQNs for scalable multi-domain 5G+ inter-slice orchestration[J].IEEE Transactions on Machine Learning in Communications and Networking, 2024, 2:956-977., articleTitle=Sample-efficient multiagent DQNs for scalable multi-domain 5G+ inter-slice orchestration, refAbstract=null), Reference(id=1251226706187600036, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, doi=null, pmid=null, pmcid=null, year=2023, volume=53, issue=11, pageStart=7187, pageEnd=7198, url=null, language=null, rfNumber=[15], rfOrder=14, authorNames=LIU G S, DENG W J, XIE X R, journalName=IEEE Transactions on Cybernetics, refType=null, unstructuredReference=LIU G S, DENG W J, XIE X R, et al. Human-level control through directly trained deep spiking Q-networks[J]. IEEE Transactions on Cybernetics, 2023, 53(11):7187-7198., articleTitle=Human-level control through directly trained deep spiking Q-networks, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1251226696813331192, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=1, ext=[AuthorCompanyExt(id=1251226696825914105, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696813331192, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electronic and Information Engineering,Nanjing University of Information Science & Technology,Nanjing 210044,China), AuthorCompanyExt(id=1251226696834302717, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696813331192, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1南京信息工程大学 电子与信息工程学院,南京 210044)]), AuthorCompany(id=1251226696909800194, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, xref=2, ext=[AuthorCompanyExt(id=1251226696913994499, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China), AuthorCompanyExt(id=1251226696922383108, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, companyId=1251226696909800194, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2国防科技大学第六十三研究所,南京 210007)])], figs=[ArticleFig(id=1251226701036995532, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=fIEDl6M068qT5hSv/eckLQ==, figureFileBig=hg8e+XZ/RhMVaHe8N+09/Q==, tableContent=null), ArticleFig(id=1251226701112493013, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=图1, caption=抗干扰通信系统示意, figureFileSmall=fIEDl6M068qT5hSv/eckLQ==, figureFileBig=hg8e+XZ/RhMVaHe8N+09/Q==, tableContent=null), ArticleFig(id=1251226701334791139, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=2Rj8dCuFW7jZlWzBlIl+Tw==, figureFileBig=H40bWhCSKma92T4WTxZZ1g==, tableContent=null), ArticleFig(id=1251226701406094314, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=图2, caption=干扰动态变化二维拓扑, figureFileSmall=2Rj8dCuFW7jZlWzBlIl+Tw==, figureFileBig=H40bWhCSKma92T4WTxZZ1g==, tableContent=null), ArticleFig(id=1251226701515146223, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=3bVzTYdDKdgbu0718Fd8yg==, figureFileBig=o934sGc3I5zSxLnpeoJ6/w==, tableContent=null), ArticleFig(id=1251226701607420920, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=图3, caption=MRV-DDQN策略神经网络的结构, figureFileSmall=3bVzTYdDKdgbu0718Fd8yg==, figureFileBig=o934sGc3I5zSxLnpeoJ6/w==, tableContent=null), ArticleFig(id=1251226701699694594, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=woRl5YKwgSp9/UMEGXC0tg==, figureFileBig=lGiEHO3T5BCzAo4GGWZoyw==, tableContent=null), ArticleFig(id=1251226701775192070, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=图4, caption=MRV-DDQN抗干扰算法框架, figureFileSmall=woRl5YKwgSp9/UMEGXC0tg==, figureFileBig=lGiEHO3T5BCzAo4GGWZoyw==, tableContent=null), ArticleFig(id=1251226701867466765, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=+6KBf9H9q8nqKc4jaXYUZw==, figureFileBig=Jtld3cqLsR9VPAIm6rlRXA==, tableContent=null), ArticleFig(id=1251226701963935764, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=图5, caption=算法收敛性能对比, figureFileSmall=+6KBf9H9q8nqKc4jaXYUZw==, figureFileBig=Jtld3cqLsR9VPAIm6rlRXA==, tableContent=null), ArticleFig(id=1251226702102347806, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=jHugEUiQSsp2e6XyiLsoVg==, figureFileBig=JPwrAEfRCVrgT6uRri2+TQ==, tableContent=null), ArticleFig(id=1251226702194622498, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=图6, caption=算法总奖励值对比, figureFileSmall=jHugEUiQSsp2e6XyiLsoVg==, figureFileBig=JPwrAEfRCVrgT6uRri2+TQ==, tableContent=null), ArticleFig(id=1251226703792652328, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=KMxOKB0qzwcvFExtyiZiUw==, figureFileBig=9xa4GuoHQ4lorpe3Gk4ceA==, tableContent=null), ArticleFig(id=1251226703884927023, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=图7, caption=算法受干扰率对比, figureFileSmall=KMxOKB0qzwcvFExtyiZiUw==, figureFileBig=9xa4GuoHQ4lorpe3Gk4ceA==, tableContent=null), ArticleFig(id=1251226703998173237, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=zfP1J6JpKdNCrw0aiyjC1A==, figureFileBig=om3F+jLt8SrQobpKRpoDJQ==, tableContent=null), ArticleFig(id=1251226704098836538, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=图8, caption=发射功率与干扰功率对比, figureFileSmall=zfP1J6JpKdNCrw0aiyjC1A==, figureFileBig=om3F+jLt8SrQobpKRpoDJQ==, tableContent=null), ArticleFig(id=1251226704199499841, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=null, figureFileBig=null, tableContent=
参数
通信频谱带宽/MHz400
通信子带宽/MHz40
干扰带宽/MHz40
通信子带数10
每回合时隙数10
仿真干扰功率p1, p2, p3/dBm10,19,26
子带间隔/MHz40
), ArticleFig(id=1251226704287580232, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=表1, caption=

仿真参数

, figureFileSmall=null, figureFileBig=null, tableContent=
参数
通信频谱带宽/MHz400
通信子带宽/MHz40
干扰带宽/MHz40
通信子带数10
每回合时隙数10
仿真干扰功率p1, p2, p3/dBm10,19,26
子带间隔/MHz40
), ArticleFig(id=1251226704375660618, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=EN, label=null, caption=null, figureFileSmall=null, figureFileBig=null, tableContent=
参数
训练回合数episode400
更新目标网络的周期δ200个频率切换时隙
折扣因子γ0.9
贪婪因子ε0.8
经验池大小η2000
初始奖励值rewardorigin10
信干噪比奖励值-4,-6,-8
功率切换代价奖励值-3,-4,-5
频率切换代价奖励值rewardF-2
功率切换成功奖励3,4,5
学习率α0.1
每次用于训练的样本数sample32个元组
), ArticleFig(id=1251226704493101136, tenantId=1146029695717560320, journalId=1251194772300279900, articleId=1251226688114344374, language=CN, label=表2, caption=

算法参数

, figureFileSmall=null, figureFileBig=null, tableContent=
参数
训练回合数episode400
更新目标网络的周期δ200个频率切换时隙
折扣因子γ0.9
贪婪因子ε0.8
经验池大小η2000
初始奖励值rewardorigin10
信干噪比奖励值-4,-6,-8
功率切换代价奖励值-3,-4,-5
频率切换代价奖励值rewardF-2
功率切换成功奖励3,4,5
学习率α0.1
每次用于训练的样本数sample32个元组
)], attaches=null, journal=Journal(id=1251193998841266264, delFlag=0, nameCn=电讯技术, nameEn=Telecommunication Engineering, nameHistory1=null, nameHistory2=null, issn=1001-893X, eissn=null, cn=51-1267/TN, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=CpBmHoMzpESavU+iEMTBmw==, journalPrice=null, startedYear=null, abbrevIsoEn=Telecommunication Engineering, journalRemark=null, publicationField=null, createdTime=1776237495387, updatedTime=1776238086301, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=T, firstLetterEn=T, subjectCode=Engineering, subjectName=null, subjectCodeEn=Engineering, subjectNameEn=null, picCn=CpBmHoMzpESavU+iEMTBmw==, picEn=jCOIy2zOaGJZ/y3z2gPZzg==, jcr=null, cjcr=null, exts=[JournalExt(id=1251196477385687352, language=CN, name=电讯技术, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1776238086315, updatedTime=1776238086315, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://www.teleonline.cn/dxjs/ch/author/login.aspx, submissionEditorUrl=https://www.teleonline.cn/dxjs/ch/login.aspx, submissionReviewUrl=https://www.teleonline.cn/dxjs/ch/auditor/login.aspx, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1251196477469573433, language=EN, name=Telecommunication Engineering, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1776238086335, updatedTime=1776238086335, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://www.teleonline.cn/dxjs/ch/author/login.aspx, submissionEditorUrl=https://www.teleonline.cn/dxjs/ch/login.aspx, submissionReviewUrl=https://www.teleonline.cn/dxjs/ch/auditor/login.aspx, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1251194772300279900, websiteList=[Website(id=1251197148327522670, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1251194772300279900, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/dxjs/CN, language=CN, createTime=1776238246280, createBy=18614031015, updateTime=1776238378770, updateBy=18614031015, name=电讯技术-中文, tplId=1146099689490845704, title=电讯技术, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1251197904854135502, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=articleTextType, value=kx, createTime=1776238426650, updateTime=1776238426650, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904833163979, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=banner, value=null, createTime=1776238426645, updateTime=1776238426645, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904870912721, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=grayFlag, value=0, createTime=1776238426654, updateTime=1776238426654, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904824775370, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=logo, value=https://castjournals.cast.org.cn/joweb/dxjs/CN/file/pic?fileId=BBd4SC9puESjyaw04bneig==, createTime=1776238426643, updateTime=1776238426643, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904883495635, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=minRunFlag, value=0, createTime=1776238426657, updateTime=1776238426657, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904845746893, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/dxjs/CN/file/pic, createTime=1776238426648, updateTime=1776238426648, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904875107026, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=silenceFlag, value=0, createTime=1776238426655, updateTime=1776238426655, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904841552588, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1776238426647, updateTime=1776238426647, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904858329807, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=themeColor, value=null, createTime=1776238426651, updateTime=1776238426651, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197904866718416, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148327522670, code=themeStyle, value=null, createTime=1776238426653, updateTime=1776238426653, creator=18614031015, updator=18614031015)]), Website(id=1251197148512072052, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1251194772300279900, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/dxjs/EN, language=EN, createTime=1776238246324, createBy=18614031015, updateTime=1776238398944, updateBy=18614031015, name=电讯技术-英文, tplId=1146101810881728533, title=Telecommunication Engineering, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1251197930175152619, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=articleTextType, value=kx, createTime=1776238432687, updateTime=1776238432687, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930154181096, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=banner, value=null, createTime=1776238432682, updateTime=1776238432682, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930200318446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=grayFlag, value=0, createTime=1776238432693, updateTime=1776238432693, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930141598183, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=logo, value=https://castjournals.cast.org.cn/joweb/dxjs/EN/file/pic?fileId=BBd4SC9puESjyaw04bneig==, createTime=1776238432679, updateTime=1776238432679, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930212901360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=minRunFlag, value=0, createTime=1776238432696, updateTime=1776238432696, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930170958314, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/dxjs/EN/file/pic, createTime=1776238432686, updateTime=1776238432686, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930204512751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=silenceFlag, value=0, createTime=1776238432694, updateTime=1776238432694, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930162569705, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1776238432684, updateTime=1776238432684, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930183541228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=themeColor, value=null, createTime=1776238432689, updateTime=1776238432689, creator=18614031015, updator=18614031015), WebsiteProps(id=1251197930191929837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1251197148512072052, code=themeStyle, value=null, createTime=1776238432691, updateTime=1776238432691, creator=18614031015, updator=18614031015)])], journalTitle=电讯技术, weixinUrl=null, journalUrl=https://www.teleonline.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Telecommunication Engineering, journalPhotoCn=CpBmHoMzpESavU+iEMTBmw==, journalPhotoEn=jCOIy2zOaGJZ/y3z2gPZzg==, journalFirstLetter=T, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false), detailUrlCn=https://castjournals.cast.org.cn/joweb/dxjs/CN/10.20079/j.issn.1001-893x.240715002, detailUrlEn=https://castjournals.cast.org.cn/joweb/dxjs/EN/10.20079/j.issn.1001-893x.240715002, pdfUrlCn=https://castjournals.cast.org.cn/joweb/dxjs/CN/PDF/10.20079/j.issn.1001-893x.240715002, pdfUrlEn=https://castjournals.cast.org.cn/joweb/dxjs/EN/PDF/10.20079/j.issn.1001-893x.240715002, aliStartDate=null, aliEndDate=null, collectionFlag=false, citedCount=null, citedUrl=null, reference=null)
收藏切换
基于多奖励值DDQN智能通信抗干扰决策方法
收藏切换
PDF下载
凌耀 1, 2 , 谢世珺 2 , 梁豪 2 , 冯姣 1 , 高伟杰 1, 2
电讯技术 | 应用基础与前沿技术 2025,65(11): 1820-1827
收起
收藏切换
电讯技术 | 应用基础与前沿技术 2025, 65(11): 1820-1827
基于多奖励值DDQN智能通信抗干扰决策方法
全屏
凌耀1, 2, 谢世珺2 , 梁豪2, 冯姣1, 高伟杰1, 2
作者信息
  • 1南京信息工程大学 电子与信息工程学院,南京 210044
  • 2国防科技大学第六十三研究所,南京 210007
  • 凌耀 男,1998年生于江苏盐城,2021年获工学学士学位,现为硕士研究生,主要研究方向为智能抗干扰决策。

    谢世珺 女,1980年生于贵州凯里,2005年获工学硕士学位,现为副研究员,主要研究方向为卫星通信。

    梁豪 男,1993年生于山东泰安,2020年获工学博士学位,现为助理研究员,主要研究方向为卫星通信。

    冯姣 女,1984年生于吉林延吉,2014年获工学博士学位,现为副教授,主要研究方向为无线通信。

    高伟杰 男,2000年生于江苏无锡,2022年获工学学士学位,现为硕士研究生,主要研究方向为智能抗干扰决策。

通讯作者:

谢世珺 Email:
An Intelligent Communication Anti-interference Decision Algorithm Based on Multiple Reward Value DDQN
Yao LING1, 2, Shijun XIE2 , Hao LIANG2, Jiao FENG1, Weijie GAO1, 2
Affiliations
  • 1School of Electronic and Information Engineering,Nanjing University of Information Science & Technology,Nanjing 210044,China
  • 2The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China
出版时间: 2025-11-28 doi: 10.20079/j.issn.1001-893x.240715002
文章导航
收藏切换

在动态干扰环境下的卫星通信系统中,各信道的质量和干扰功率存在差异。有限的频谱资源和复杂的干扰环境对抗干扰通信决策提出了资源分配和业务需求的挑战,即如何在避开干扰频率和优化功率的同时,实现资源的高效利用。为解决这一问题,提出了一种基于多奖励值函数的深度强化学习抗干扰算法。该算法将发送方、接收方与干扰方之间的交互建模为马尔可夫决策过程。通过优化信道切换与功率切换代价的奖励函数,引入频率切换与功率切换机制,分析相邻时隙频谱中的干扰特征,并将交互过程中采集到的干扰信号特征与信道信息结合,用于训练抗干扰策略。该策略实现了频率域与功率域的联合抗干扰决策。仿真结果表明,该算法能够有效降低系统的受干扰概率,加快算法收敛速度,并优化功率资源的利用效率。

智能通信抗干扰  /  联合抗干扰决策  /  深度强化学习  /  多奖励值函数

In satellite communication systems operating in dynamic interference environments,the quality of channels and the interference power vary. Limited spectrum resources and complex interference environments pose challenges for anti-interference communication decisions, particularly in terms of resource allocation and service demands. Specifically, the challenge lies in efficiently utilizing resources while avoiding interference frequencies and optimizing power. To address this issue,a deep reinforcement learning-based anti-interference algorithm with multiple reward functions is proposed. The algorithm models the interaction between the transmitter,receiver,and interferer as a Markov decision process. By optimizing the reward function associated with the costs of channel and power switching,it introduces mechanisms for both frequency and power switching,analyzes the interference characteristics in the spectrum of adjacent time slots, and integrates the interference signal features collected during the interaction with channel information to train an anti-interference strategy. This strategy enables joint anti-interference decision-making in both the frequency and power domains. Simulation results demonstrate that the algorithm effectively reduces the probability of interference,accelerates convergence,and optimizes the utilization of power resources.

intelligent communication anti-interference  /  joint anti-interference decision  /  deep reinforcement learning  /  multiple reward value functions
凌耀, 谢世珺, 梁豪, 冯姣, 高伟杰. 基于多奖励值DDQN智能通信抗干扰决策方法. 电讯技术, 2025 , 65 (11) : 1820 -1827 . DOI: 10.20079/j.issn.1001-893x.240715002
Yao LING, Shijun XIE, Hao LIANG, Jiao FENG, Weijie GAO. An Intelligent Communication Anti-interference Decision Algorithm Based on Multiple Reward Value DDQN[J]. Telecommunication Engineering, 2025 , 65 (11) : 1820 -1827 . DOI: 10.20079/j.issn.1001-893x.240715002
卫星通信作为一种空间信息基础设施,具有传播距离远、覆盖范围广、部署速度快、不受地理环境限制和可用通信频带宽等特点,被广泛应用于军事、水利和远洋航行等领域[1]。在卫星通信网络中,空间段、地面段和用户段采用电磁波作为信息传输的媒介,并在暴露的无线信道中进行传输。这使得干扰设备可以轻易对通信链路进行干扰,因此在恶劣电磁环境中实现抗干扰的可靠传输成为一个重要的研究课题。随着软件无线电、人工智能等技术的飞速发展,抗干扰正向着智能抗干扰方向发展[2-3]。深度学习、强化学习等技术作为实现智能抗干扰的有效手段,得到了大量研究。
文献[4]提出了一种基于深度确定性策略梯度(Deep Deterministic Policy Gradient,DDPG)的频率间隔选择和跳频设置算法,在干扰环境下找到最优的频率间隔和跳频策略,以最大限度地提高信噪比,并将参数决策问题表述为马尔可夫决策[5]过程(Markov Decision Process,MDP)。文献[6]则对动态干扰环境中的跳频通信系统进行建模,针对多信道干扰的情况下发射机和接收机未知干扰模式,提出了一种深度强化学习(Deep Q Network,DQN)跳频算法来解决跳频决策问题,实现抗干扰。文献[7]通过随机博弈[8]的方法解决干扰问题,提出了一种强化学习(Q-learning)算法,获得干扰机和目标发射机的智能信道跳频序列,高发射功率目标用户利用智能信道跳变来迫使低发射功率用户同时使用目标用户未跳变的信道来混淆干扰器,低发射功率目标用户通过信道跳变来避开干扰器。
然而,上述文献大多只考虑了频率域或功率域的单域干扰,没有考虑到实际信道环境中存在的频率域和功率域的联合干扰[9-10],因此,这些算法通常只能针对单个干扰域制定抗干扰策略。为了提高发送方的通信质量,一方面需要躲避干扰频率,另一方面在遭遇干扰功率时需要以最优的资源效率适当提高发射功率来克服干扰。本文的抗干扰决策算法在考虑功率域和频率域联合抗干扰的同时,优化了资源利用。
本文主要贡献如下:
1)研究了在动态频率域和功率域联合干扰下的抗干扰策略,结合干扰的二维拓扑图建立了马尔可夫抗干扰决策模型,设计了相应的状态集、动作集和代价奖励函数。
2)提出了一种多奖励值函数的深度强化学习(Multiple Reward Value-Double Deep Q Network,MRV-DDQN)抗干扰算法。与以往的DQN抗干扰算法不同,该算法将信干噪比、功率切换代价和频率切换代价等信息分别作为多奖励值函数,通过奖励值函数引导算法以最少的频率切换和最小的功率开销,实现频率域和功率域的联合抗干扰及资源的合理利用。
图1所示,本文研究发送方向接收方发送数据时的抗干扰通信方法,构建了一个由发送方、接收方、若干干扰方和抗干扰决策代理组成的通信系统模型。该模型中,前向通信链路包括多个可用信道。干扰方在每个通信时隙内对多个前向链路信道进行干扰。抗干扰决策代理根据每个通信时隙内信道受到干扰的情况,对下一通信时隙的频率切换和功率切换[11-13]动作做出决策,并通过未被干扰的通信决策控制链路通知发送方。本文仅对单向通信链路的抗干扰进行研究,并假设信息反馈控制链路中没有干扰存在。发送方在下一传输时隙执行该决策,以最优的子带和功率通过前向链路发送信息。
图2模拟了前向通信链路中的干扰,横坐标代表系统通信时隙,纵坐标代表通信链路按频率从低到高划分的若干信道,频率中心为{f1f2f3,…,fN}。若干随机分布的干扰机产生N种不同功率的干扰,干扰随时隙和频率变化的二维拓扑图如图2所示。发送方在传输数据时,根据通信链路的动态干扰情况选择相应的频率和功率切换策略,从而实现抗干扰。抗干扰策略的最优目标是通过最少的频率切换和最小的发送功率开销进行通信。
状态空间定义为St=[fnt],表示在第t个传输时隙占用第n个信道,n∈[1,N],t∈[1,T]。因此,St在时间和频率维度上具有N×T个的可能状态,并且每个状态之间相互独立。由于信道中存在干扰功率,在抗干扰通信系统中,发送机需要根据抗干扰决策代理发送的频率切换动作,决定是否在当前信道上继续传输或跳转到其他N-1个信道之一,同时根据功率切换动作决定当前的发射功率。将频率切换动作和功率切换动作定义为二维元组A= [AFnAPn],AFn表示跳转到第n个子带,APn表示发射功率切换为PnAtA是第t个时隙中发送方采用的频率和功率切换动作。
奖励值函数是引导算法实现频率域和功率域联合抗干扰的重要参数,在本节中建立了奖励初值、信噪比奖励值、频率及功率切换代价奖励值和功率切换成功奖励值。算法训练完成后的奖励值也作为该抗干扰决策算法性能的评价标准。
为了便于仿真,设置了奖励初值rewardorigin,用于表示没有进行频率切换且子带没有干扰功率时的奖励值。接收方的信干噪比和传输速率分别表示为
式中:fU,t为发送方在t时刻的发送频率,发射功率为PU,tfj,t为干扰方在t时刻选择干扰的通信频率,干扰功率为Pj,tn为信道噪声功率;gu表示发送方到接收方之间的信道增益;gj表示干扰方和接收方之间的信道增益;BU是用户发送信道带宽;CU是发送速率。发送方如果选择和干扰方相同的频率即(fj,t=fU,t),则δfj,t=fU,t)= 1,否则δfj,t=fU,t)= 0。为了表征接收端信号质量的好坏,将信干噪比建立为奖励值函数r,根据式(1)计算不同信干噪比设置,表示为
为了衡量功率切换和频率切换对通信过程的影响,将功率域和频率域的干扰量化为抗干扰决策算法中的奖励值函数。系统在频率切换后需要重新同步通信链路,因此产生频率切换代价,根据频率切换代价设置奖励值rewardF,当频率没有发生切换时则代价为0;发射功率大于干扰功率6 dBm以上即功率域抗干扰成功,根据功率切换成功代价设置奖励值rewardPs,否则奖励值为0。
当选择有干扰功率的信道时,发射功率会根据信道中的干扰功率进行提高。在发射功率提高的过程中会产生功率切换代价。根据不同的干扰功率,设置了功率切换代价奖励值,表示为
这样的多奖励值函数机制控制Q值的更新以及神经网络权值的优化,引导算法尽可能切换到空闲信道或干扰功率低的信道,通过发射功率的切换保证传输的可靠性。抗干扰策略的目标是最大化长期累积奖励值,使得发送方能在最小的频率切换和功率切换的代价下实现抗干扰。
在本文中将状态动作值定义为QSA),表示在当前状态S经过神经网络计算输出的Q值元组,Q值元组的索引位置与动作所在集合的索引位置相对应。在MRV-DDQN算法中,Q值元组的更新表示为
式中:QSA)是状态StA对应的状态动作更新值;γ是折扣因子,在0~1范围之间;θ为目标网络权值;θ′是策略网络权值;中先由取得目标网络中最大Q状态动作值对应的动作At+1,再根据下一时刻的St+1At+1动作更新策略网络的Q值。
本节对抗干扰策略的状态、动作、奖励值和状态动作值进行建模,为下一节的算法实现提供理论基础。通过不断优化Q状态动作值,确定最优抗干扰策略的问题可以等效于发射机在每个时隙选择最优动作。
网络结构如图3所示,由输入、输出层和两个隐藏层组成,每层包含30个单元。3种层之间用权重参数θ={θInputθHiddenθOnput}全连接。特征输入层获取状态空间元组里的信道信息。隐藏层中使用relu激活函数如式(6):
式中:x为每层的输出。激活函数使神经网络[14]能够拟合复杂的非线性Q状态动作函数值。输出层的30个Q状态动作函数值单元对应不同的频率和功率切换动作。
基于MRV-DDQN抗干扰算法框架如图4所示。框图中展示了算法的执行过程,其中包含两张结构相同的MRV-DDQN网络分别是策略网络(policy net)和目标网络(target net)。
在发送方、干扰方、接收方交互过程中产生的状态、奖励、动作和下一状态组成的元组保存在经验池中,训练时从经验池中周期性随机取样32个样本来训练策略网络。目标网络输出的QtargetSA)与QSA)的差值作为损失值:
损失越小,MRV-DDQN神经网络的收敛性越好,抗干扰策略掌握得越准确,从而输出最优的抗干扰动作。损失值通过反向梯度计算更新网络中隐藏层的权值,如式(8)所示:
抗干扰策略随着网络权值的动态更新不断优化。为了加快网络收敛速度,周期地将该网络的权值复制给目标网络。在循环动态更新的过程中,目标网络作为最佳抗干扰策略的输出网络,根据QSA)状态动作值来获得最佳动作。
MRV-DDQN抗干扰算法执行流程如下:
输入:
1)初始化算法参数:各奖励值reward,学习率α,折扣因子γ,贪婪策略选择概率ε,目标网络更新周期δ,经验池大小η,用于训练的样本数sample。
2)初始化环境参数:干扰功率P1P2,…,Pn,初始化policy和target网络权值,训练回合数(episode)初始值为0,最大值为400。
输出:最优抗干扰动作[AFn, APn]。
循环:for episode=0 to 400;
1)发送方、接收方和干扰方之间交互形成的状态动作空间存储到经验池η中;
2)从上述的经验池η中随机取出32样本元组输入到policy神经网络中得到状态动作的QS, A)值;
3)根据式(7)来更新QS, A)得到QS, A),并根据式(8)计算损失值Loss;
4)根据式(8)反向梯度算法更新policy网络中神经网络节点的权值θ′;
5)直到损失值收敛,得到最优抗干扰动作[AFnAPn],此时发送方使用最佳通信频率和发射功率。
开始之前,需要初始化算法参数和环境参数。算法参数包括第1.3节介绍的5种奖励值、用于控制网络更新幅度的学习速率α,以及用于控制未来奖励在当前决策中权重的折扣因子γ。在算法初期,发送方、干扰方和接收方需要相互交互,以探索信道中的干扰情况。贪婪策略的选择概率ε决定了算法进行探索的概率,概率越大,干扰情况统计得越全面,经验池也会越丰富。为防止算法出现Q值高估,设置了更新周期δ,该参数控制策略网络向目标网络更新权重的周期数。训练样本数sample表示每次从经验池中提取的用于训练网络的样本数量。环境参数包括信道中设置的动态干扰(P1P2,…,Pn)、初始神经网络权值和训练回合数。
在算法执行过程中,进入训练循环。首先,在算法初期,贪婪策略选择概率ε控制发送方、干扰方和接收方之间的交互,以探索信道中的干扰情况,并将探索得到的状态-动作对存储在经验池中。同时,算法从经验池中随机抽取32个训练样本输入到策略神经网络中,计算得到状态-动作值QSA)。根据第1.4节中的状态-动作值更新公式计算出QSA),并通过计算QSA)和QSA)之间的差异确定算法的损失值。由于QSA)是依据奖励值引导的结果,相比之前的QSA)更加符合算法在频率域和功率域的抗干扰需求。通过两者差值进行反向梯度更新,优化策略网络的权重,从而增强算法的抗干扰性能。最终,随着算法的收敛,输出的抗干扰动作[AFnAPn]达到最优。
本节对基于MRV-DDQN抗干扰算法的性能进行分析,并展示MRV-DDQN与DQN抗干扰算法性能的对比。在训练过程中,定义每10个传输时隙为一个回合,总回合数为400次。为了更好地比较两种算法的性能,在前向链路的10个信道中设置了3个随机分布且不同功率的干扰机对通信链路进行干扰,干扰机以5个时隙为周期不断切换干扰频段,对通信子带动态干扰。仿真环境基于OpenAI提供的gym库[15]。为了模拟实际通信过程中的抗干扰性能,算法中模拟了一些参数,如表1所示。
表2中,参数δ为策略网络向目标网络复制权重的周期数。为了避免MRV-DDQN算法的Q值高估,目标网络的更新周期应适当延长。折扣因子γ决定未来奖励在当前决策中的权重,设为0.9能确保奖励有效引导算法输出最优策略。贪婪因子ε决定算法运行初期的探索力度,以积累足够的数据存储在经验池中,便于神经网络的训练。本文将经验池大小设为2000,每次从中提取32个样本用于训练,能够满足算法需求。初始奖励值设为正值,在频率或功率切换时将奖励值设为负值,以增强算法的抗干扰性能。学习速率控制参数更新幅度,速率过大可能导致参数过度更新和发散;过小则会减缓模型的收敛速度,延长训练时间。在调试过程中,逐步增大学习速率,最终确定0.1为合理值,算法的收敛效果较好。
图5展示了本文MRV-DDQN抗干扰算法与传统DQN抗干扰算法的收敛性能对比,损失值越小,表明神经网络收敛得越好。在算法逐渐收敛的过程中,MRV-DDQN算法损失值最终达到0左右,DQN算法损失值在25附近。从图中可以看出,MRV-DDQN的损失值更低,能够掌握更好的抗干扰策略,输出更优的频率切换和功率切换动作。
由于奖励值函数包含无干扰的初始奖励和若干受干扰的奖励代价,它也反映了系统的数据吞吐能力,奖励值越大,单位时间内的数据吞吐量越高。图6展示了MRV-DDQN和DQN抗干扰算法每回合总奖励值的对比。随着回合数的增加,MRV-DDQN抗干扰算法的总奖励值不断增加,最终达到一个固定值,表明系统的数据吞吐能力不断提高并最终达到稳定状态。相比之下,DQN抗干扰算法每回合的总奖励值较低,传输数据能力较差。
图7展示了MRV-DDQN和DQN抗干扰算法在每个回合中的归一化受干扰率。结果显示,MRV-DDQN抗干扰算法在收敛后每回合的受干扰率为0,表明其能够有效摆脱周期性的动态干扰,而DQN抗干扰算法在收敛后每回合的受干扰率为0.3左右。因此,MRV-DDQN抗干扰算法在频率域的抗干扰性能明显优于DQN抗干扰算法。
在训练的400回合中,系统生成了4000个时隙,展示效果对比不明显。因此,在这4000个时隙中,每隔100个时隙进行一次干扰功率和发射功率的抽样,如图8所示。算法在运行过程中与环境不断交互,当抗干扰策略未完全优化时,频率切换动作可能导致发送方通信频段切换到带有干扰功率的信道中,并且可能出现发射功率低于干扰功率的情况(如抽样8、抽样12)。此时,算法仅掌握部分抗干扰策略,而功率切换动作也可能导致发射功率高于干扰功率6 dBm(如抽样2、抽样5、抽样9),实现部分功率域抗干扰。当算法收敛并完全掌握抗干扰策略时,频率切换动作将使发送方通信频段切换到空闲的子带,并以最低的发射功率(6 dBm)进行通信,以实现资源的最佳利用。
本文研究了无线通信网络中功率域和频率域的联合抗干扰问题,综合考虑了资源利用、频率切换代价和功率切换代价等因素,重点解决了在有限频谱资源内如何选择合适的信道及发射功率以实现有效抗干扰。本文将发送方、接收方和干扰方之间的交互建模为马尔可夫决策过程,通过改进信道切换和功率切换代价的奖励函数,并利用干扰特征和信道信息训练抗干扰策略,实现了频率域和功率域的联合抗干扰决策。在有限资源的情况下,通过引入频率切换和功率切换机制提升了信道和功率资源的利用率,引入双神经网络则提升了系统的收敛性能。
在通信抗干扰领域仍然有许多问题有待分析和探讨:①在通信抗干扰的上下游领域包括智能干扰感知、智能抗干扰波形重构也是值得深入研究的方向;②未来抗干扰领域将由单域、多域向全域发展,这使得抗干扰的策略空间急剧膨胀,因此深度强化学习方面的策略优化是未来研究的重点。
参考文献 引证文献
排序方式:
[1]
陈书恒, 莫嘉倩, 莫小欣.机载低轨卫星通信发展及关键技术综述[J].电讯技术, 2024, 64(1):149-157.
[2]
魏鹏.卫星通信智能抗干扰决策技术研究[D].长沙:国防科技大学, 2021.
[3]
FOURATI F, ALOUINI M S. Artificial intelligence for satellite communication: a review[J]. Intelligent and Converged Networks, 2021, 2(3):213-243.
[4]
ZHANG Y P, ZHAO Z J, ZHENG S L, et al. Intelligent anti-jamming decision with continuous action and state in bivariate frequency agility communication system[J]. IEEE Transactions on Cognitive Communications and Networking, 2023, 9(6):1579-1595.
[5]
LI X C, CHEN J N, LING X, et al. Deep reinforcement learning-based anti-jamming algorithm using dual action network[J]. IEEE Transactions on Wireless Communications, 2023, 22(7):4625-4637.
[6]
QI J, ZHANG H M, QI X L, et al. Deep reinforcement learning based hopping strategy for wideband anti-jamming wireless communications[J]. IEEE Transactions on Vehicular Technology, 2024, 73(3):3568-3579.
[7]
NOORI H, SADEGHI VILNI S. Jamming and anti-jamming in interference channels:a stochastic game approach[J]. IET Communications, 2020, 14(4):682-692.
[8]
HAN C, HUO L Y, TONG X H, et al. Spatial anti-jamming scheme for Internet of satellites based on the deep reinforcement learning and Stackelberg game[J].IEEE Transactions on Vehicular Technology, 2020, 69(5):5331-5342.
[9]
ZHOU Q, LI Y G, NIU Y T. Intelligent anti-jamming communication for wireless sensor networks:a multi-agent reinforcement learning approach[J]. IEEE Open Journal of the Communications Society, 2021, 2:775-784.
[10]
王桂胜, 董淑福, 黄国策.无人系统认知联合抗干扰通信研究综述[J].计算机工程与应用, 2022, 58(8):1-11.
[11]
ZENG X Y, CAI H, TANG X H, et al. Optimal frequency hopping sequences of odd length[J]. IEEE Transactions on Information Theory, 2013, 59(5):3237-3248.
[12]
XU H, CHENG Y F, WANG P Y. Jamming detection in broadband frequency hopping systems based on multi-segment signals spectrum clustering[J]. IEEE Access, 2021, 9:29980-29992.
[13]
WU J L, GUO F C. Time-frequency parameter estimation method of frequency hopping signal based on morphology method under low SNR[C]//2021 IEEE 6th International Conference on Signal and Image Processing. Nanjing:IEEE, 2022:734-738.
[14]
DOANIS P, SPYROPOULOS T. Sample-efficient multiagent DQNs for scalable multi-domain 5G+ inter-slice orchestration[J].IEEE Transactions on Machine Learning in Communications and Networking, 2024, 2:956-977.
[15]
LIU G S, DENG W J, XIE X R, et al. Human-level control through directly trained deep spiking Q-networks[J]. IEEE Transactions on Cybernetics, 2023, 53(11):7187-7198.
2025年第65卷第11期
PDF下载
112
54
引用本文
BibTeX
文章信息
doi: 10.20079/j.issn.1001-893x.240715002
  • 接收时间:2024-07-15
  • 首发时间:2026-04-15
  • 出版时间:2025-11-28
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2024-07-15
  • 修回日期:2024-11-08
基金
作者信息
    1南京信息工程大学 电子与信息工程学院,南京 210044
    2国防科技大学第六十三研究所,南京 210007

通讯作者:

谢世珺 Email:
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/dxjs/CN/10.20079/j.issn.1001-893x.240715002
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数

Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)

Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏