Article(id=1329457774238478473, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329457773609332873, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202440387, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1717084800000, receivedDateStr=2024-05-31, revisedDate=1721145600000, revisedDateStr=2024-07-17, acceptedDate=null, acceptedDateStr=null, onlineDate=1794897033509, onlineDateStr=2026-11-17, pubDate=1727712000000, pubDateStr=2024-10-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794897033509, onlineIssueDateStr=2026-11-17, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794897033509, creator=13701087609, updateTime=1794897033509, updator=13701087609, issue=Issue{id=1329457773609332873, tenantId=1146029695717560320, journalId=1324759914146529478, year='2024', volume='61', issue='10', pageStart='2401', pageEnd='2661', issueExtLink='null', onlineDate='null', pubDate='1727712000000', pubDateStr='2024-10-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794897033359, creator='13701087609', updateTime=1794897033359, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext=null, issueFiles=null, downloadFileDto=null}, startPage=2404, endPage=2416, ext={EN=ArticleExt(id=1329457774469165196, articleId=1329457774238478473, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Dynamic Decision-Driven Threat Detection Method for Data Elements in Industrial Control Networks, columnId=1329457774406250635, journalTitle=Journal of Computer Research and Development, columnName=Secure Circulation of Data Elements, runingTitle=null, highlight=null, articleAbstract=

In recent years, the industrial control network has been developing rapidly. The advantages of digitization, intelligence, and automation have brought significant benefits to the industry while also introducing increasingly complex and variable attack threats. In the context of data element security, timely detection and response to industrial control network threats have become an urgent task to be solved. By continuously monitoring and analyzing the data flow in industrial control networks, the problem of industrial control network threat detection can be transformed into a time series anomaly detection problem. However, the existing time-series anomaly detection methods are limited by the quality of industrial control network datasets and are often sensitive to only a single type of anomaly while ignoring other anomalies. Therefore, in this paper, we propose a deep reinforcement learning and data augmentation based threat detection method in industrial control networks (DELTA). DELTA introduces a novel data augmentation selection technique for time series datasets, which allows for the selection of appropriate data augmentation operations sets tailored to different baseline models to enhance the quality of the industrial control network time series datasets. Simultaneously, deep reinforcement learning algorithms (A2C/PPO) dynamically select candidate models from the baseline models at different time points, leveraging multiple types of anomaly detection models to address the issue of sensitivity to single-type anomalies. The experimental results compared with the existing time series anomaly detection models show that DELTA has a significant improvement in accuracy and F1 value over all baseline models at an acceptable cost of additional time consumption, which verifies the effectiveness and practicality of the method.

, authors=Zepeng Wang, Chao Ma, Zhuangzhuang Zhang, Libing Wu, Xiaochuan Shi*, authorsList=Zepeng Wang, Chao Ma, Zhuangzhuang Zhang, Libing Wu, Xiaochuan Shi, authorCompany=null, correspAuthors=Xiaochuan Shi, authorNote=

born in 1999. PhD candidate. Student member of CCF. His main research interests include reinforcement learning, anomaly detection, and big data analysis

born in 1982. PhD, lecturer. Member of CCF. His main research interests include time series analytics, generative AI, explainable AI, and reinforcement learning

born in 1994. PhD. Student member of CCF. His main research interests include federated learning security and IoV security

born in 1972. PhD, professor, PhD supervisor. Distinguished member of CCF. His main research interests include network security, Internet of things, machine learning, and data security

born in 1984. PhD, associate professor, PhD supervisor. Member of CCF. His main research interests include big data analysis, reinforcement learning, deep learning, and wireless sensor networks

, correspAuthorsNote=null, copyrightStatement=Copyright © 2024 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1329457776922833064, articleId=1329457774238478473, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=动态决策驱动的工控网络数据要素威胁检测方法, columnId=1329457774548856973, journalTitle=计算机研究与发展, columnName=数据要素安全流通, runingTitle=null, highlight=null, articleAbstract=

近年来,工控网络发展势头迅猛. 其数字化、智能化、自动化的优势为工业带来巨大效益的同时,也面临着愈发复杂多变的攻击威胁. 在数据要素安全的背景下,及时发现和应对工控网络威胁成为一项迫切需要得到解决的任务. 通过对工控网络中的数据流进行连续监测和分析,工控网络威胁检测问题可以转化为时间序列异常检测问题. 然而现有时间序列异常检测方法受限于工控网络数据集的质量,且往往仅对单一类型异常敏感而忽略其他异常. 针对上述问题,提出了一种基于深度强化学习和数据增强的工控网络威胁检测方法(deep reinforcement learning and data augmentation based threat detection method in industrial control networks,DELTA). 该方法提出了一种新的时序数据集数据增强选择方法,可以针对不同的基准模型选择合适的数据增强操作集以提升工控网络时间序列数据集的质量;同时使用深度强化学习算法(A2C/PPO)在不同时间点从基线模型中动态选取候选模型,以利用多种类型的异常检测模型解决单一类型异常敏感问题. 与现有时间序列异常检测模型对比的实验结果表明,在付出可接受的额外时间消耗成本下,DELTA在准确率和F1值上比所有基线模型有明显的提升,验证了方法的有效性与实用性.

, authors=王泽鹏, 马超, 张壮壮, 吴黎兵, 石小川*, authorsList=王泽鹏, 马超, 张壮壮, 吴黎兵, 石小川, authorCompany=null, correspAuthors=石小川, authorNote=

1999年生. 博士研究生. CCF学生会员. 主要研究方向为强化学习、异常检测、大数据分析

1982年生. 博士,讲师. CCF会员. 主要研究方向为时间序列分析、生成式人工智能、可解释性人工智能、强化学习

1994年生. 博士. CCF学生会员. 主要研究方向为联邦学习安全、车联网安全

1972年生. 博士,教授,博士生导师. CCF杰出会员. 主要研究方向为网络安全、物联网、机器学习、数据安全

1984年生. 博士,副教授,博士生导师. CCF会员. 主要研究方向为大数据分析、强化学习、深度学习、无线传感器网络

, correspAuthorsNote=
石小川()
, copyrightStatement=版权所有 © 2024 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=LUoqftqsGbQ2oySfH+xAtg==, magXml=+ZAjGoWusZfjvHSIkFMpLA==, pdfUrl=null, pdf=ArVggueWMsIrv3XT+0i9Ag==, pdfFileSize=1292625, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=uVCQ4bjgXq1hNIsqhHdbSA==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=K+WfMKMwkQncfwXv7PPbxQ==, mapNumber=null, fund=null)}, authors=[Author(id=1329457777312903344, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=wangzepeng@whu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329457777384206513, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777312903344, language=EN, stringName=Zepeng Wang, firstName=Zepeng, middleName=null, lastName=Wang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1999. PhD candidate. Student member of CCF. His main research interests include reinforcement learning, anomaly detection, and big data analysis

"}, bioImg=null, bioContent=

born in 1999. PhD candidate. Student member of CCF. His main research interests include reinforcement learning, anomaly detection, and big data analysis

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457777442926770, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777312903344, language=CN, stringName=王泽鹏, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"pIc5lPdqZh2IjtlvIxBwbw==","content":"

1999年生. 博士研究生. CCF学生会员. 主要研究方向为强化学习、异常检测、大数据分析

"}, bioImg=pIc5lPdqZh2IjtlvIxBwbw==, bioContent=

1999年生. 博士研究生. CCF学生会员. 主要研究方向为强化学习、异常检测、大数据分析

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1329457777505841332, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329457777581338805, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777505841332, language=EN, stringName=Chao Ma, firstName=Chao, middleName=null, lastName=Ma, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1982. PhD, lecturer. Member of CCF. His main research interests include time series analytics, generative AI, explainable AI, and reinforcement learning

"}, bioImg=null, bioContent=

born in 1982. PhD, lecturer. Member of CCF. His main research interests include time series analytics, generative AI, explainable AI, and reinforcement learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457777644253366, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777505841332, language=CN, stringName=马超, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"B2F5N9Nw7/vraflvB/BgTg==","content":"

1982年生. 博士,讲师. CCF会员. 主要研究方向为时间序列分析、生成式人工智能、可解释性人工智能、强化学习

"}, bioImg=B2F5N9Nw7/vraflvB/BgTg==, bioContent=

1982年生. 博士,讲师. CCF会员. 主要研究方向为时间序列分析、生成式人工智能、可解释性人工智能、强化学习

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1329457777702973624, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329457777761693881, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777702973624, language=EN, stringName=Zhuangzhuang Zhang, firstName=Zhuangzhuang, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1994. PhD. Student member of CCF. His main research interests include federated learning security and IoV security

"}, bioImg=null, bioContent=

born in 1994. PhD. Student member of CCF. His main research interests include federated learning security and IoV security

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457777816219834, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777702973624, language=CN, stringName=张壮壮, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"yh8lNKeZZ52v2748fdZN4A==","content":"

1994年生. 博士. CCF学生会员. 主要研究方向为联邦学习安全、车联网安全

"}, bioImg=yh8lNKeZZ52v2748fdZN4A==, bioContent=

1994年生. 博士. CCF学生会员. 主要研究方向为联邦学习安全、车联网安全

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1329457777879134396, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329457777958826173, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777879134396, language=EN, stringName=Libing Wu, firstName=Libing, middleName=null, lastName=Wu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1972. PhD, professor, PhD supervisor. Distinguished member of CCF. His main research interests include network security, Internet of things, machine learning, and data security

"}, bioImg=null, bioContent=

born in 1972. PhD, professor, PhD supervisor. Distinguished member of CCF. His main research interests include network security, Internet of things, machine learning, and data security

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457778021740734, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777879134396, language=CN, stringName=吴黎兵, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"Fbk3eERlOefqT7ZSqHOlHA==","content":"

1972年生. 博士,教授,博士生导师. CCF杰出会员. 主要研究方向为网络安全、物联网、机器学习、数据安全

"}, bioImg=Fbk3eERlOefqT7ZSqHOlHA==, bioContent=

1972年生. 博士,教授,博士生导师. CCF杰出会员. 主要研究方向为网络安全、物联网、机器学习、数据安全

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1329457778080460992, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=shixiaochuan@whu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329457778160152769, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457778080460992, language=EN, stringName=Xiaochuan Shi, firstName=Xiaochuan, middleName=null, lastName=Shi, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=null, bio={"content":"

born in 1984. PhD, associate professor, PhD supervisor. Member of CCF. His main research interests include big data analysis, reinforcement learning, deep learning, and wireless sensor networks

"}, bioImg=null, bioContent=

born in 1984. PhD, associate professor, PhD supervisor. Member of CCF. His main research interests include big data analysis, reinforcement learning, deep learning, and wireless sensor networks

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457778244038851, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457778080460992, language=CN, stringName=石小川, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=null, bio={"img":"E2MAcK0kb9xELx9IgnkZhA==","content":"

1984年生. 博士,副教授,博士生导师. CCF会员. 主要研究方向为大数据分析、强化学习、深度学习、无线传感器网络

"}, bioImg=E2MAcK0kb9xELx9IgnkZhA==, bioContent=

1984年生. 博士,副教授,博士生导师. CCF会员. 主要研究方向为大数据分析、强化学习、深度学习、无线传感器网络

, aboutCorrespAuthor=null)}, companyList=null)], keywords=[Keyword(id=1329457778361479364, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, orderNo=1, keyword=industrial control networks), Keyword(id=1329457778420199621, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, orderNo=2, keyword=data element security), Keyword(id=1329457778474725574, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, orderNo=3, keyword=time series), Keyword(id=1329457778546028743, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, orderNo=4, keyword=anomaly detection), Keyword(id=1329457778613137608, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, orderNo=5, keyword=deep reinforcement learning), Keyword(id=1329457778676052169, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, orderNo=6, keyword=data augmentation), Keyword(id=1329457778726383818, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, orderNo=7, keyword=model selection), Keyword(id=1329457778780909771, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, orderNo=1, keyword=工控网络), Keyword(id=1329457778839630028, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, orderNo=2, keyword=数据要素安全), Keyword(id=1329457778898350285, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, orderNo=3, keyword=时间序列), Keyword(id=1329457778948681934, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, orderNo=4, keyword=异常检测), Keyword(id=1329457779007402191, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, orderNo=5, keyword=深度强化学习), Keyword(id=1329457779057733840, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, orderNo=6, keyword=数据增强), Keyword(id=1329457779120648401, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, orderNo=7, keyword=模型选择)], refs=[Reference(id=1329457781108748522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Wu H S. A survey of research on anomaly detection for time series[C]//Proc of 2016 13th Int Computer Conf on Wavelet Active Media Technology and Information Processing (ICCWAMTIP). Piscataway, NJ: IEEE, 2016: 426−431, articleTitle=null, refAbstract=null), Reference(id=1329457781192634603, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Chalapathy R, Chawla S. Deep learning for anomaly detection: A survey[J]. arXiv preprint, arXiv: 1901.03407, 2019, articleTitle=null, refAbstract=null), Reference(id=1329457781251354860, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=徐丽娟,王佰玲,杨美红,等. 工业控制网络多模式攻击检测及异常状态评估方法[J]. 计算机研究与发展,2021,58(11):2333−2349, articleTitle=null, refAbstract=null), Reference(id=1329457781381378285, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Xu Lijuan, Wang Bailing, Yang Meihong, et al. Multi-mode attack detection and evaluation of abnormal states for industrial control network[J]. Journal of Computer Research and Development, 2021, 58(11): 2333−2349 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329457781456875758, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=席亮,王勇,张凤斌. 基于自适应人工鱼群FCM的异常检测算法[J]. 计算机研究与发展,2019,56(5):1048−1059, articleTitle=null, refAbstract=null), Reference(id=1329457781523984623, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Xi Liang, Wang Yong, Zhang Fengbin. Anomaly detection algorithm based on FCM with adaptive artificial Fish-Swarm[J]. Journal of Computer Research and Development, 2019, 56(5): 1048−1059 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329457781591093488, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=陈波冯,李靖东,卢兴见,等. 基于深度学习的图异常检测技术综述[J]. 计算机研究与发展,2021,58(7):1436−1455, articleTitle=null, refAbstract=null), Reference(id=1329457781670785265, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Bofeng, Li Jingdong, Lu Xingjian, et al. Survey of deep learning based graph anomaly detection methods[J]. Journal of Computer Research and Development, 2021, 58(7): 1436−1455 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329457781733699826, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Audibert J, Michiardi P, Guyard F, et al. USAD: Unsupervised anomaly detection on multivariate time series[C]//Proc of the 26th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2020: 3395−3404, articleTitle=null, refAbstract=null), Reference(id=1329457781805002995, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Chuxu, Song Dongjin, Chen Yuncong, et al. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 1409−1416, articleTitle=null, refAbstract=null), Reference(id=1329457781880500468, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Park D, Hoshi Y, Kemp C C. A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder[J]. IEEE Robotics and Automation Letters, 2018, 3(3): 1544−1551, articleTitle=null, refAbstract=null), Reference(id=1329457781964386549, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Su Ya, Zhao Youjian, Niu Chenhao, et al. Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]//Proc of the 25th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2019: 2828−2837, articleTitle=null, refAbstract=null), Reference(id=1329457782039884022, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Xin Ruyue, Liu Hongyun, Chen Peng, et al. Robust and accurate performance anomaly detection and prediction for cloud applications: A novel ensemble learning-based framework[J]. Journal of Cloud Computing, 2023, 12(1): 1−16, articleTitle=null, refAbstract=null), Reference(id=1329457782119575799, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Yubo, Guo Ni, Chen Wei, et al. Multi-step ahead forecasting for electric power load using an ensemble model[J]. Expert Systems with Applications, 2023, 211: 118649, articleTitle=null, refAbstract=null), Reference(id=1329457782174101752, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Aggarwal C C, Sathe S. Theoretical foundations and algorithms for outlier ensembles[J]. ACM Sigkdd Explorations Newsletter, 2015, 17(1): 24−47, articleTitle=null, refAbstract=null), Reference(id=1329457782333485305, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Tama B A, Nkenyereye L, Islam S M R, et al. An enhanced anomaly detection in web traffic using a stack of classifier ensemble[J]. IEEE Access, 2020, 8: 24120−24134, articleTitle=null, refAbstract=null), Reference(id=1329457782429954298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Adeyemo V E, Abdullah A, JhanJhi N Z, et al. Ensemble and deep-learning methods for two-class and multi-attack anomaly intrusion detection: An empirical study[J]. International Journal of Advanced Computer Science and Applications, 2019, 10(9): 520−528, articleTitle=null, refAbstract=null), Reference(id=1329457782509646075, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Ünlü R, Xanthopoulos P. A weighted framework for unsupervised ensemble learning based on internal quality measures[J]. Annals of Operations Research, 2019, 276: 229−247, articleTitle=null, refAbstract=null), Reference(id=1329457782572560636, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Wen Qingsong, Sun Liang, Yang Fan, et al. Time series data augmentation for deep learning: A survey[C]//Proc of the 30th Int Joint Conf on Artificial Intelligence. Freiburg, German: IJCAI, 2021: 4653−4660, articleTitle=null, refAbstract=null), Reference(id=1329457782648058109, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Gao Jingkun, Song Xiaomin, Wen Qingsong, et al. Robusttad: Robust time series anomaly detection via decomposition and convolutional neural networks[C]//Proc of ACM SIGKDD Workshop on Mining and Learning from Time Series (KDD-MiLeTS 2020). New York: ACM, 2020: 1−9, articleTitle=null, refAbstract=null), Reference(id=1329457782715166974, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Steven Eyobu O, Han D S. Feature representation and data augmentation for human activity classification based on wearable IMU sensor data using a deep LSTM neural network[J]. Sensors, 2018, 18(9): 2892−2917, articleTitle=null, refAbstract=null), Reference(id=1329457782820024575, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Park D S, Chan W, Zhang Yu, et al. SpecAugment: A simple data augmentation method for automatic speech recognition[C]. Proc of. Interspeech New York: ACM, 2019: 2613−2617, articleTitle=null, refAbstract=null), Reference(id=1329457782899716352, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Park D S, Chan W, Zhang Yu, et al. RobustSTL: A robust seasonal-trend decomposition algorithm for long time series[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 5409−5416, articleTitle=null, refAbstract=null), Reference(id=1329457782958436609, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Yan, Lu Xinjiang, Wang Yaqing, et al. Generative time series forecasting with diffusion, denoise, and disentanglement[J]. Advances in Neural Information Processing Systems, 2022, 35: 23009−23022, articleTitle=null, refAbstract=null), Reference(id=1329457783046516994, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Kang Yanfei, Hyndman R J, Li Feng. GRATIS: GeneRAting time series with diverse and controllable characteristics[J]. Statistical Analysis and Data Mining: The ASA Data Science Journal, 2020, 13(4): 354−376, articleTitle=null, refAbstract=null), Reference(id=1329457783122014467, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Devries T, Taylor G W. Dataset augmentation in feature space[J]. arXiv preprint, arXiv: 1702.05538, 2017, articleTitle=null, refAbstract=null), Reference(id=1329457783193317636, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Cheung T H, Yeung D Y. Modals: Modality-agnostic automated data augmentation in the latent space[C]//Proc of the 9th Int Conf on Learning Representations. Washington, DC: ICLR, 2021: 1−18, articleTitle=null, refAbstract=null), Reference(id=1329457783256232197, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Yoon J, Jarrett D, Van der Schaar M. Time-series generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2019, 32: 5508−5518, articleTitle=null, refAbstract=null), Reference(id=1329457783314952454, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Fons E, Dawson P, Zeng Xiaojun, et al. Adaptive weighting scheme for automatic time-series data augmentation[J]. arXiv preprint, arXiv: 2102.08310, 2021, articleTitle=null, refAbstract=null), Reference(id=1329457783377867015, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang J E, Wu Di, Boulet B. Time series anomaly detection via reinforcement learning-based model selection[C]//Proc of 2022 IEEE Canadian Conf on Electrical and Computer Engineering (CCECE). Piscataway, NJ: IEEE, 2022: 193−199, articleTitle=null, refAbstract=null), Reference(id=1329457783449170185, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Sutton R S, Barto A G. Reinforcement Learning: An Introduction[M]. Cambridge, MA: MIT press, 2018, articleTitle=null, refAbstract=null), Reference(id=1329457783558222090, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms[J]. arXiv preprint, arXiv: 1707.06347, 2017, articleTitle=null, refAbstract=null), Reference(id=1329457783629525259, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Goh J, Adepu S, Junejo K N, et al. A dataset to support research in the design of secure water treatment systems[C]//Proc of the 11th Int Conf on Critical Information Infrastructures Security (CRITIS 2016). Berlin: Springer, 2016: 88−99, articleTitle=null, refAbstract=null), Reference(id=1329457783751160076, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu F T, Ting Kaiming, Zhou Zhihua. Isolation forest[C]//Proc of 2008 8th IEEE Int Conf on Data Mining. Piscataway, NJ: IEEE, 2008: 413−422, articleTitle=null, refAbstract=null), Reference(id=1329457784027984141, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Xu Hongzuo, Pang Guansong, Wang Yijie, et al. Deep isolation forest for anomaly detection[J]. IEEE Transactions on Knowledge and Data Engineering, 2023: 12591−12604, articleTitle=null, refAbstract=null), Reference(id=1329457784099287310, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Pevný T. Loda: Lightweight on-line detector of anomalies[J]. Machine Learning, 2016, 102: 275−304, articleTitle=null, refAbstract=null), Reference(id=1329457784225116431, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Kingma D P, Welling M. Auto-encoding variational Bayes[J]. arXiv preprint, arXiv: 1312.6114, 2013, articleTitle=null, refAbstract=null), Reference(id=1329457784342556944, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=Higgins I, Matthey L, Pal A, et al. Beta-vae: Learning basic visual concepts with a constrained variational framework[C]//Proc of the 5th Int Conf on Learning Representations. Washington, DC: ICLR, 2017: 1−22, articleTitle=null, refAbstract=null), Reference(id=1329457784485163281, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Zheng, Zhao Yue, Hu Xiyang, et al. Ecod: Unsupervised outlier detection using empirical cumulative distribution functions[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 35(12): 12181−12193, articleTitle=null, refAbstract=null), Reference(id=1329457784590020882, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=37, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Zheng, Zhao Yue, Botta N, et al. COPOD: Copula-based outlier detection[C]//Proc of 2020 IEEE Int Conf on Data Mining (ICDM). Piscataway, NJ: IEEE, 2020: 1118−1123, articleTitle=null, refAbstract=null), Reference(id=1329457784770375955, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Scikit-learn. Scikit-Learn[EB/OL]. 2024 [2024-04-02]. https://github. com/scikit-learn/scikit-learn, articleTitle=null, refAbstract=null), Reference(id=1329457784858456340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=39, rfOrder=41, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Yue. PyOD[EB/OL]. 2024 [2024-04-02]. https://github.com/ yzhao062/pyod, articleTitle=null, refAbstract=null), Reference(id=1329457784967508245, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=40, rfOrder=42, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang J E. RLMSAD[EB/OL]. 2022 [2024-04-02]. https://github. com/elisejiuqizhang/RLMSAD, articleTitle=null, refAbstract=null), Reference(id=1329457785101725974, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=41, rfOrder=43, authorNames=null, journalName=null, refType=null, unstructuredReference=Raffin A, Hill A, Gleave A, et al. Stable-baselines3: Reliable reinforcement learning implementations[J]. Journal of Machine Learning Research, 2021, 22(268): 1−8, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1329457777136742569, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, xref=null, ext=[AuthorCompanyExt(id=1329457777145131178, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, companyId=1329457777136742569, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Cyber Science Engineering-WHU, Wuhan University 430072), AuthorCompanyExt(id=1329457777153519787, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, companyId=1329457777136742569, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=武汉大学国家网络安全学院 武汉 430072)]), AuthorCompany(id=1329457777220628652, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, xref=null, ext=[AuthorCompanyExt(id=1329457777224822957, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, companyId=1329457777220628652, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=Key Laboratory of Aerospace Information Security and Trusted Computing (Wuhan University), Ministry of Education, Wuhan 430072), AuthorCompanyExt(id=1329457777233211566, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, companyId=1329457777220628652, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=空天信息安全与可信计算教育部重点实验室(武汉大学) 武汉 430072)])], figs=[ArticleFig(id=1329457779233894610, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Figure 1, caption=Examples of sensor exceptions in the SWaT dataset, figureFileSmall=Y3wpiueop5VShizP7J4zDQ==, figureFileBig=uVCQ4bjgXq1hNIsqhHdbSA==, tableContent=null), ArticleFig(id=1329457779305197779, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=图1, caption=SWaT数据集中的传感器异常示例, figureFileSmall=Y3wpiueop5VShizP7J4zDQ==, figureFileBig=uVCQ4bjgXq1hNIsqhHdbSA==, tableContent=null), ArticleFig(id=1329457779468775636, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Figure 2, caption=Classification of time series data augmentation methods, figureFileSmall=zqh/exM3Pftz++++pL4fQg==, figureFileBig=wtSpF4yv/fk1dojF3AmnsA==, tableContent=null), ArticleFig(id=1329457779531690197, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=图2, caption=时间序列数据增强方法分类, figureFileSmall=zqh/exM3Pftz++++pL4fQg==, figureFileBig=wtSpF4yv/fk1dojF3AmnsA==, tableContent=null), ArticleFig(id=1329457779607187670, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Figure 3, caption=Overall framework diagram of DELTA, figureFileSmall=6Fv9Ymsjck2e+v2s/qrwNQ==, figureFileBig=N5Z64wepujJDTlgRMY/q8Q==, tableContent=null), ArticleFig(id=1329457779686879447, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=图3, caption=DELTA整体框架图, figureFileSmall=6Fv9Ymsjck2e+v2s/qrwNQ==, figureFileBig=N5Z64wepujJDTlgRMY/q8Q==, tableContent=null), ArticleFig(id=1329457779753988312, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Figure 4, caption=Overall performance results, figureFileSmall=E4JUoNNnJtm9lBsUdy9ziQ==, figureFileBig=Ds/E3fDxbYXhge4tp6MueQ==, tableContent=null), ArticleFig(id=1329457779821097177, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=图4, caption=总体性能结果, figureFileSmall=E4JUoNNnJtm9lBsUdy9ziQ==, figureFileBig=Ds/E3fDxbYXhge4tp6MueQ==, tableContent=null), ArticleFig(id=1329457779909177562, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Figure 5, caption=Impact of different FP on DELTA model’s performance, figureFileSmall=Ax/DDZraLbJ5tOZojLRKEQ==, figureFileBig=9jMMEPpMJFhxpMocb9mV4Q==, tableContent=null), ArticleFig(id=1329457779967897819, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=图5, caption=不同FP对DELTA模型性能的影响, figureFileSmall=Ax/DDZraLbJ5tOZojLRKEQ==, figureFileBig=9jMMEPpMJFhxpMocb9mV4Q==, tableContent=null), ArticleFig(id=1329457780051783900, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Figure 6, caption=Impact of different FN on DELTA model’s performance, figureFileSmall=trtkLejH6uLppa4/gFsFtg==, figureFileBig=QnfKymyzlnR8WDa9zxH+NA==, tableContent=null), ArticleFig(id=1329457780131475677, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=图6, caption=不同FN对DELTA模型性能的影响, figureFileSmall=trtkLejH6uLppa4/gFsFtg==, figureFileBig=QnfKymyzlnR8WDa9zxH+NA==, tableContent=null), ArticleFig(id=1329457780206973150, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Figure 7, caption=Impact of threshold $ d $ on the performance of DELTA model, figureFileSmall=J5YrJy+1hE7fVJl4LvWJYw==, figureFileBig=sADBZf0+JPJWFtm6oP8CxQ==, tableContent=null), ArticleFig(id=1329457780274082015, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=图7, caption=阈值$ d $对DELTA模型性能的影响, figureFileSmall=J5YrJy+1hE7fVJl4LvWJYw==, figureFileBig=sADBZf0+JPJWFtm6oP8CxQ==, tableContent=null), ArticleFig(id=1329457780345385184, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Figure 8, caption=Comparison of training time and inference time of different models, figureFileSmall=rvDKQlLULmycotqCPiBong==, figureFileBig=vsYEWs1pjQgJH2noteGHog==, tableContent=null), ArticleFig(id=1329457780404105441, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=图8, caption=不同模型的训练时间和推理时间对比, figureFileSmall=rvDKQlLULmycotqCPiBong==, figureFileBig=vsYEWs1pjQgJH2noteGHog==, tableContent=null), ArticleFig(id=1329457780462825698, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Table 1, caption=

List of the Major Parameters

, figureFileSmall=null, figureFileBig=null, tableContent=
参数取值
PPO学习率3E–4
A2C学习率7E–4
折扣因子$ \gamma $0.99
裁剪因子$ \varepsilon $0.2
批次大小64
最大仿真步数2048
参数更新轮数10
梯度裁剪最大值0.5
滑动窗口值12
), ArticleFig(id=1329457780538323171, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=表1, caption=

主要参数列表

, figureFileSmall=null, figureFileBig=null, tableContent=
参数取值
PPO学习率3E–4
A2C学习率7E–4
折扣因子$ \gamma $0.99
裁剪因子$ \varepsilon $0.2
批次大小64
最大仿真步数2048
参数更新轮数10
梯度裁剪最大值0.5
滑动窗口值12
), ArticleFig(id=1329457780630597860, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Table 2, caption=

Overall Performance Results %

, figureFileSmall=null, figureFileBig=null, tableContent=
模型类型模型准确率召回率F1值
无监督学习iForest67.3062.7964.97
DIF69.2359.5567.66
LODA66.6263.4765.00
VAE66.5563.4064.94
beta-VAE67.5364.3365.89
ECOD71.4268.0469.69
COPOD71.6268.2369.88
强化学习RLMSAD71.75±0.3968.31±0.0669.99±0.19
DELTA-A(本文)88.84±4.9363.07±3.5173.57±1.72
DELTA-P(本文)88.40±2.6446.68±6.3460.88±5.72
), ArticleFig(id=1329457780714483941, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=表2, caption=

总体性能结果

, figureFileSmall=null, figureFileBig=null, tableContent=
模型类型模型准确率召回率F1值
无监督学习iForest67.3062.7964.97
DIF69.2359.5567.66
LODA66.6263.4765.00
VAE66.5563.4064.94
beta-VAE67.5364.3365.89
ECOD71.4268.0469.69
COPOD71.6268.2369.88
强化学习RLMSAD71.75±0.3968.31±0.0669.99±0.19
DELTA-A(本文)88.84±4.9363.07±3.5173.57±1.72
DELTA-P(本文)88.40±2.6446.68±6.3460.88±5.72
), ArticleFig(id=1329457780806758630, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Table 3, caption=

Impact of Data Augmentation Module on the Performance of Baseline Model %

, figureFileSmall=null, figureFileBig=null, tableContent=
模型准确率召回率F1值
iForest w/o DA67.3062.7964.97
iForest69.6966.3968.00
DIF w/o DA69.2359.5567.66
DIF69.7066.4068.01
ECOD w/o DA71.4268.0469.69
ECOD72.2268.8170.47
COPOD w/o DA71.6268.2369.88
COPOD72.0368.6270.28
), ArticleFig(id=1329457780886450407, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=表3, caption=

数据增强模块对基准模型性能的影响

, figureFileSmall=null, figureFileBig=null, tableContent=
模型准确率召回率F1值
iForest w/o DA67.3062.7964.97
iForest69.6966.3968.00
DIF w/o DA69.2359.5567.66
DIF69.7066.4068.01
ECOD w/o DA71.4268.0469.69
ECOD72.2268.8170.47
COPOD w/o DA71.6268.2369.88
COPOD72.0368.6270.28
), ArticleFig(id=1329457780953559272, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=EN, label=Table 4, caption=

Impact of Data Augmentation Module on the Performance of DELTA Model %

, figureFileSmall=null, figureFileBig=null, tableContent=
模型准确率召回率F1值
DELTA-A88.84±4.9363.07±3.5173.57±1.72
DELTA-A w/o DA87.44±7.3455.04±18.7564.51±20.27
DELTA-P88.40±2.6446.68±6.3460.88±5.72
DELTA-P w/o DA85.30±3.5044.15±14.2856.93±13.62
), ArticleFig(id=1329457781016473833, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, language=CN, label=表4, caption=

数据增强模块对DELTA模型性能的影响

, figureFileSmall=null, figureFileBig=null, tableContent=
模型准确率召回率F1值
DELTA-A88.84±4.9363.07±3.5173.57±1.72
DELTA-A w/o DA87.44±7.3455.04±18.7564.51±20.27
DELTA-P88.40±2.6446.68±6.3460.88±5.72
DELTA-P w/o DA85.30±3.5044.15±14.2856.93±13.62
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440387, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440387, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202440387, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202440387, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1727712000000, fullTextJson=null, articleText=null, reference=null)
收藏切换
动态决策驱动的工控网络数据要素威胁检测方法
王泽鹏 , 马超 , 张壮壮 , 吴黎兵 , 石小川 *
计算机研究与发展 | 数据要素安全流通 2024,61(10): 2404-2416
收起
收藏切换
计算机研究与发展 |数据要素安全流通 2024 , 61 (10) : 2404 -2416
动态决策驱动的工控网络数据要素威胁检测方法
全屏
[Author(id=1329457777312903344, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=wangzepeng@whu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329457777384206513, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777312903344, language=EN, stringName=Zepeng Wang, firstName=Zepeng, middleName=null, lastName=Wang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1999. PhD candidate. Student member of CCF. His main research interests include reinforcement learning, anomaly detection, and big data analysis

"}, bioImg=null, bioContent=

born in 1999. PhD candidate. Student member of CCF. His main research interests include reinforcement learning, anomaly detection, and big data analysis

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457777442926770, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777312903344, language=CN, stringName=王泽鹏, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"pIc5lPdqZh2IjtlvIxBwbw==","content":"

1999年生. 博士研究生. CCF学生会员. 主要研究方向为强化学习、异常检测、大数据分析

"}, bioImg=pIc5lPdqZh2IjtlvIxBwbw==, bioContent=

1999年生. 博士研究生. CCF学生会员. 主要研究方向为强化学习、异常检测、大数据分析

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1329457777505841332, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329457777581338805, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777505841332, language=EN, stringName=Chao Ma, firstName=Chao, middleName=null, lastName=Ma, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1982. PhD, lecturer. Member of CCF. His main research interests include time series analytics, generative AI, explainable AI, and reinforcement learning

"}, bioImg=null, bioContent=

born in 1982. PhD, lecturer. Member of CCF. His main research interests include time series analytics, generative AI, explainable AI, and reinforcement learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457777644253366, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777505841332, language=CN, stringName=马超, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"B2F5N9Nw7/vraflvB/BgTg==","content":"

1982年生. 博士,讲师. CCF会员. 主要研究方向为时间序列分析、生成式人工智能、可解释性人工智能、强化学习

"}, bioImg=B2F5N9Nw7/vraflvB/BgTg==, bioContent=

1982年生. 博士,讲师. CCF会员. 主要研究方向为时间序列分析、生成式人工智能、可解释性人工智能、强化学习

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1329457777702973624, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329457777761693881, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777702973624, language=EN, stringName=Zhuangzhuang Zhang, firstName=Zhuangzhuang, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1994. PhD. Student member of CCF. His main research interests include federated learning security and IoV security

"}, bioImg=null, bioContent=

born in 1994. PhD. Student member of CCF. His main research interests include federated learning security and IoV security

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457777816219834, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777702973624, language=CN, stringName=张壮壮, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"yh8lNKeZZ52v2748fdZN4A==","content":"

1994年生. 博士. CCF学生会员. 主要研究方向为联邦学习安全、车联网安全

"}, bioImg=yh8lNKeZZ52v2748fdZN4A==, bioContent=

1994年生. 博士. CCF学生会员. 主要研究方向为联邦学习安全、车联网安全

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1329457777879134396, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329457777958826173, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777879134396, language=EN, stringName=Libing Wu, firstName=Libing, middleName=null, lastName=Wu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1972. PhD, professor, PhD supervisor. Distinguished member of CCF. His main research interests include network security, Internet of things, machine learning, and data security

"}, bioImg=null, bioContent=

born in 1972. PhD, professor, PhD supervisor. Distinguished member of CCF. His main research interests include network security, Internet of things, machine learning, and data security

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457778021740734, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457777879134396, language=CN, stringName=吴黎兵, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"Fbk3eERlOefqT7ZSqHOlHA==","content":"

1972年生. 博士,教授,博士生导师. CCF杰出会员. 主要研究方向为网络安全、物联网、机器学习、数据安全

"}, bioImg=Fbk3eERlOefqT7ZSqHOlHA==, bioContent=

1972年生. 博士,教授,博士生导师. CCF杰出会员. 主要研究方向为网络安全、物联网、机器学习、数据安全

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1329457778080460992, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=shixiaochuan@whu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329457778160152769, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457778080460992, language=EN, stringName=Xiaochuan Shi, firstName=Xiaochuan, middleName=null, lastName=Shi, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=null, bio={"content":"

born in 1984. PhD, associate professor, PhD supervisor. Member of CCF. His main research interests include big data analysis, reinforcement learning, deep learning, and wireless sensor networks

"}, bioImg=null, bioContent=

born in 1984. PhD, associate professor, PhD supervisor. Member of CCF. His main research interests include big data analysis, reinforcement learning, deep learning, and wireless sensor networks

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329457778244038851, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329457774238478473, authorId=1329457778080460992, language=CN, stringName=石小川, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=null, bio={"img":"E2MAcK0kb9xELx9IgnkZhA==","content":"

1984年生. 博士,副教授,博士生导师. CCF会员. 主要研究方向为大数据分析、强化学习、深度学习、无线传感器网络

"}, bioImg=E2MAcK0kb9xELx9IgnkZhA==, bioContent=

1984年生. 博士,副教授,博士生导师. CCF会员. 主要研究方向为大数据分析、强化学习、深度学习、无线传感器网络

, aboutCorrespAuthor=null)}, companyList=null)]
王泽鹏 , 马超, 张壮壮, 吴黎兵, 石小川*
作者信息
  • 武汉大学国家网络安全学院 武汉 430072
  • 空天信息安全与可信计算教育部重点实验室(武汉大学) 武汉 430072
通讯作者:
作者简介:

1999年生. 博士研究生. CCF学生会员. 主要研究方向为强化学习、异常检测、大数据分析

1982年生. 博士,讲师. CCF会员. 主要研究方向为时间序列分析、生成式人工智能、可解释性人工智能、强化学习

1994年生. 博士. CCF学生会员. 主要研究方向为联邦学习安全、车联网安全

1972年生. 博士,教授,博士生导师. CCF杰出会员. 主要研究方向为网络安全、物联网、机器学习、数据安全

1984年生. 博士,副教授,博士生导师. CCF会员. 主要研究方向为大数据分析、强化学习、深度学习、无线传感器网络

Dynamic Decision-Driven Threat Detection Method for Data Elements in Industrial Control Networks
Zepeng Wang , Chao Ma, Zhuangzhuang Zhang, Libing Wu, Xiaochuan Shi*
Affiliations
  • School of Cyber Science Engineering-WHU, Wuhan University 430072
  • Key Laboratory of Aerospace Information Security and Trusted Computing (Wuhan University), Ministry of Education, Wuhan 430072
  • born in 1999. PhD candidate. Student member of CCF. His main research interests include reinforcement learning, anomaly detection, and big data analysis

    born in 1982. PhD, lecturer. Member of CCF. His main research interests include time series analytics, generative AI, explainable AI, and reinforcement learning

    born in 1994. PhD. Student member of CCF. His main research interests include federated learning security and IoV security

    born in 1972. PhD, professor, PhD supervisor. Distinguished member of CCF. His main research interests include network security, Internet of things, machine learning, and data security

    born in 1984. PhD, associate professor, PhD supervisor. Member of CCF. His main research interests include big data analysis, reinforcement learning, deep learning, and wireless sensor networks

出版时间: 2024-10-01 doi: 10.7544/issn1000-1239.202440387
文章导航
收藏切换

近年来,工控网络发展势头迅猛. 其数字化、智能化、自动化的优势为工业带来巨大效益的同时,也面临着愈发复杂多变的攻击威胁. 在数据要素安全的背景下,及时发现和应对工控网络威胁成为一项迫切需要得到解决的任务. 通过对工控网络中的数据流进行连续监测和分析,工控网络威胁检测问题可以转化为时间序列异常检测问题. 然而现有时间序列异常检测方法受限于工控网络数据集的质量,且往往仅对单一类型异常敏感而忽略其他异常. 针对上述问题,提出了一种基于深度强化学习和数据增强的工控网络威胁检测方法(deep reinforcement learning and data augmentation based threat detection method in industrial control networks,DELTA). 该方法提出了一种新的时序数据集数据增强选择方法,可以针对不同的基准模型选择合适的数据增强操作集以提升工控网络时间序列数据集的质量;同时使用深度强化学习算法(A2C/PPO)在不同时间点从基线模型中动态选取候选模型,以利用多种类型的异常检测模型解决单一类型异常敏感问题. 与现有时间序列异常检测模型对比的实验结果表明,在付出可接受的额外时间消耗成本下,DELTA在准确率和F1值上比所有基线模型有明显的提升,验证了方法的有效性与实用性.

工控网络  /  数据要素安全  /  时间序列  /  异常检测  /  深度强化学习  /  数据增强  /  模型选择

In recent years, the industrial control network has been developing rapidly. The advantages of digitization, intelligence, and automation have brought significant benefits to the industry while also introducing increasingly complex and variable attack threats. In the context of data element security, timely detection and response to industrial control network threats have become an urgent task to be solved. By continuously monitoring and analyzing the data flow in industrial control networks, the problem of industrial control network threat detection can be transformed into a time series anomaly detection problem. However, the existing time-series anomaly detection methods are limited by the quality of industrial control network datasets and are often sensitive to only a single type of anomaly while ignoring other anomalies. Therefore, in this paper, we propose a deep reinforcement learning and data augmentation based threat detection method in industrial control networks (DELTA). DELTA introduces a novel data augmentation selection technique for time series datasets, which allows for the selection of appropriate data augmentation operations sets tailored to different baseline models to enhance the quality of the industrial control network time series datasets. Simultaneously, deep reinforcement learning algorithms (A2C/PPO) dynamically select candidate models from the baseline models at different time points, leveraging multiple types of anomaly detection models to address the issue of sensitivity to single-type anomalies. The experimental results compared with the existing time series anomaly detection models show that DELTA has a significant improvement in accuracy and F1 value over all baseline models at an acceptable cost of additional time consumption, which verifies the effectiveness and practicality of the method.

industrial control networks  /  data element security  /  time series  /  anomaly detection  /  deep reinforcement learning  /  data augmentation  /  model selection
王泽鹏, 马超, 张壮壮, 吴黎兵, 石小川. 动态决策驱动的工控网络数据要素威胁检测方法. 计算机研究与发展, 2024 , 61 (10) : 2404 -2416 . DOI: 10.7544/issn1000-1239.202440387
Zepeng Wang, Chao Ma, Zhuangzhuang Zhang, Libing Wu, Xiaochuan Shi. Dynamic Decision-Driven Threat Detection Method for Data Elements in Industrial Control Networks[J]. Journal of Computer Research and Development, 2024 , 61 (10) : 2404 -2416 . DOI: 10.7544/issn1000-1239.202440387
工控系统作为现代工业自动化的核心组成部分,广泛应用于能源、制造、交通等关键基础设施领域. 工控网络威胁检测是针对工业控制系统网络中的安全威胁进行监测和识别的重要领域. 随着工业物联网(IoT)和工业4.0的快速发展,工控系统的数字化、智能化和互联化程度的不断提高,工控系统的安全性成为了一个日益突出的问题. 特别是在数据要素安全的背景下,工控网络面临着多样化的威胁和越来越多的潜在威胁和攻击风险. 这些威胁不仅涉及传统的网络安全问题,而且包括针对特定数据要素的攻击,如数据篡改、非授权访问等. 网络攻击者可以利用漏洞、恶意软件或未经授权的访问,对工控网络进行入侵,从而导致设备故障、生产中断甚至人身安全风险. 确保工控网络的安全性和稳定性成为保护关键基础设施的重要任务. 因此,开发能够有效检测和应对这些威胁的技术至关重要.
时间序列数据是按时间顺序收集的观测值的序列,如股票价格、气象数据、传感器测量等,已成为各个领域中不可或缺的重要资源. 然而,这些数据中常常存在着异常,即与正常模式不符的异常值或行为. 异常可能是由故障、欺诈、异常事件或未知因素引起的,对于电网、通信网络等工控网络关键基础设施来说及早识别和处理这些异常非常关键. 如图1所示,图1(a)和1(b)分别为工控安全水处理系统的液位传感器与流量计实时收集的流量信息,当系统受到攻击时,流量数据时间序列会表现出与正常数据时间序列不同的异常模式,同时对系统的正常运行造成不同程度的破坏与影响. 因此,工控网络威胁检测可以自然地建模为时间序列异常检测问题,时间序列异常检测也成为了对于工控网络系统数据分析和预测中一个重要而具有挑战性的问题[1-3].
时间序列异常检测是一项旨在识别和定位时间序列数据中异常值的任务. 与传统的离群点检测方法不同,时间序列异常检测需要考虑数据的时序性质,即异常值与周围数据点的时间依赖性. 传统的基于阈值或统计方法的离群点检测技术在处理时间序列数据时往往效果不佳,因为它们无法捕捉到时间序列中的复杂模式和关联性. 而基于监督学习的算法可以通过从数据中提取特征并建立分类模型,使其在异常检测任务上表现良好,但其依赖于大量的标签来训练模型,这对于现实世界是很难满足的. 因此,在无监督环境下开发针对时间序列异常检测的专门方法和算法是至关重要的.
此外,由于不同类型的异常检测算法侧重点不同,单一类型的异常检测模型可能会对特定类型的异常敏感,而忽视其他异常. 而传统的集成学习方法难以确定基础模型的权重来发挥组合模型的最优性能. 如何设计一个通用的模型可以针对存在的不同类型异常的数据在检测性能上完全超越其他异常检测模型,成为一个困难且有研究价值的问题.
为解决上述问题,本文提出了一种基于深度强化学习和数据增强的工控网络威胁检测方法(deep reinforcement learning and data augmentation based threat detection method in industrial control networks,DELTA). 首先提出一种可靠的数据增强方法,以提升时序数据集异常数据的多样性. 然后利用增强数据集训练多个不同类型的异常检测基线模型. 最后利用深度强化学习算法在不同时间步数据点从基线模型中动态选取合适的模型用于最终的异常检测. 本文的主要贡献有3个方面:
1)提出了一种新的时序数据集数据增强选择方法,提升了时序数据集样本的质量,为后续异常检测模型和基于深度强化学习的模型筛选的训练提供了基础.
2)提出了使用深度强化学习在不同时间步数据点上筛选合适的基准模型进行组合预测的方案,结合不同类型异常检测模型的优势,提高了异常检测的总体性能.
3)在公共和真实的数据集上对DELTA方法进行了评估,与其他基线模型相比,DELTA方法在准确率和F1值2个关键指标上有显著的提升,在时序数据异常检测任务上展现了明显的优势.
近年来越来越多的研究开始关注异常检测领域并取得了一定的成果[4-5]. 尽管大部分的异常检测算法并非针对时间序列异常检测任务所提出的,但这些算法在时间序列任务上仍有良好的应用.
由于不需要大量的标签数据,且具有推断时间序列之间相关性的能力,基于无监督学习技术的算法在异常检测领域越来越受到研究者的关注[6]. 本节主要对近年来的基于无监督学习的多变量时间序列异常检测方法进行介绍.
Audibert等人[6]提出了一种基于反向训练自编码器的快速稳定的多变量时间序列异常检测方法USAD,对抗性训练的架构使得其能够进行快速训练的同时隔离异常,在多方面上提升了异常检测性能. Zhang等人[7]提出的MSCRED联合考虑了时序依赖性、噪声鲁棒性和异常严重程度的可解释性,有效地提升了多变量时间序列异常检测和诊断的能力. LSTM-VAE[8]使用LSTM替换VAE中的前馈网络,对时序数据的时间依赖性进行建模,得到了比传统方法更好的泛化能力. Su等人[9]提出了一种使用随机递归神经网络的多变量时间序列异常检测模型OmniAnomaly,通过随机变量连接和平面归一化流等关键技术来捕获鲁棒的多变量时间序列的正常表示以重构输入数据,并使用重建概率来确定异常.
集成学习通过组合多个基础模型,以减小单个模型带来的误差,提高最终模型对于任务决策的准确率,近年来也愈发受到研究者的关注[10-11].
在异常检测领域,最常用的方法是通过多个基础模型的最大值或平均值来计算异常得分. 一些研究关注基于机器学习和深度学习的集成学习方法[12]. Tama等人[13]使用3种分类器(随机森林、梯度提升器和XGBoost)作为基础模型,并使用一个广义线性模型作为组合器. Adeyemo 等人[14]结合深度学习和集成学习,将LSTM网络与随机森林等算法结合,采取投票集合的平均概率方法解决网络入侵检测问题. Ünlü等人[15]提供了一种基于内部聚类质量度量的加权策略,赋予单个聚类不同的重要性. 这些集合学习研究主要集中在权重计算或不同基础模型的线性组合上.
如图2所示,针对时间序列数据的数据增强方法可以分为2大类:基础方法和高级方法[16].
基础方法主要是针对时间序列在不同域(时域、频域、时频域)上进行多种变换. 其中,时域变换是应用最广泛的时间序列数据增强方法,该类方法直接对原始输入时间序列进行变换操作,如注入噪声、窗口裁剪、窗口变形、翻转等. 频域和时频域上的变换方法研究相对较少. Gao等人[17]同时利用时域上和频域上的数据增强策略,在基于卷积神经网络的时间序列异常检测任务中生成更多的人工标记数据,有效地提升了时间序列异常检测的性能. Steven等人[18]采用短傅里叶变换(STFT)对传感器时间序列生成时频特征并进行数据增强. 类似地,Park等人[19]在语音时间序列上通过屏蔽翻转IAAFT增强时间步长块,大大提高了语音识别神经网络的性能.
高级方法可以分为基于分解的方法、基于统计生成模型的方法和基于学习的方法. 基于分解的时间序列增强方法在时间序列异常检测和预测等相关任务上表现优秀. Park等人[20]将时间序列中的每一个数据点分解为趋势信号、周期信号和余数信号3种信号的叠加. Gao等人[17]提出使用鲁棒性分解方法,利用生成的分解残差对时间序列进行时域和频域上的增强,显著地提高了异常检测的性能. 基于统计生成模型的时间序列增强方法通常是使用统计模型对时间序列进行动态建模. Li等人[21] 提出了一个耦合扩散概率模型,以在不增加不确定性的情况下增强时间序列数据,并利用 BVAE 实现更简便的推理过程. Kang等人[22]使用混合自回归模型来模拟时间序列集,以探究生成的时间序列在特征空间上的多样性和覆盖度. 最近,基于学习的方法展现了对真实数据的特征进行模拟的潜力. Devries等人[23]提出在嵌入空间中应用编码输入以产生更可信的生成时序数据,Cheung等人[24]则利用自动数据增强来微调隐空间中的4种通用数据转换操作,以适应不同模态的数据转换,在时间序列分类问题上展现出了优异的性能. Yoon等人[25]提出了一种生成式时间序列模型TimeGAN,通过具有监督损失和无监督损失的嵌入空间进行对抗性地训练和学习. Fons等人[26]使用自动数据增强方法为时间序列数据设计了2种样本自适应加权策略,在多个时间序列数据集上的分类问题都展现出了良好的性能.
本文所提出的DELTA整体框架如图3所示.
整体流程主要分为3步:首先,在数据处理阶段,DELTA利用模型性能评分迭代评估生成数据增强操作组合,针对不同的模型使用不同的操作组合对时间序列数据集进行数据增强,以提升数据集质量,再使用滑动窗口和下采样技术处理得到最终用于训练的增强数据;其次,使用增强数据集对多种不同类型的异常检测基线模型进行训练,同时生成每个基线模型对于数据集每个测试点的预测分数、异常检测器自身的异常阈值以及基于异常阈值得到的预测标签;最后将多种信息集成到用于强化学习建模的马尔可夫决策过程中,利用强化学习算法(A2C/PPO)在不同时间步对数据集上的不同数据点从训练好的异常检测基线模型中动态选择候选模型进行最终的预测.
为保证时间序列数据集数据的多样性,提升数据集数据质量,在进行模型训练和筛选之前,本文提出先使用数据增强技术对时序数据集进行增强的思路.
参考已有典型工作[17,24],本文采取了多种经验证有效的时序数据增强方式,包括注入噪声、数据漂移、时间缩放、平滑处理、时间剪切等多种类型的增强手段,并对多种增强方式进行组合,对于每一个异常检测基准模型,根据增强前后的异常检测器的检测性能来迭代更新每轮选取的增强方式,最终为每一个异常检测基准模型生成一个最佳的增强方式组合,进而用于后续的数据处理与模型训练.
数据增强方式选择的流程如算法1所示.
算法1. 时序数据集数据增强选择方法.
输入:数据增强可选操作集DA,训练集$ {D}_{\rm train} $,测试集$ {D}_{\rm test} $,异常检测基准模型M,数据增强操作数量$ n $,迭代轮数T;
输出:数据增强操作方式组合DAC.
① $ M=T rainModel(M,{D}_{\rm train}) $;/*使用原始训练集 $ {D}_{\rm train} $训练模型$ M $*/
② $ {S}_{0}=T estModel(M,{D}_{\rm test}) $; /*使用测试集$ {D}_{\rm test} $测 试,得到初始性能得分$ {S}_{0} $*/
③ $ DA=DA\cup \left\{\rm None\right\}; $ /*为数据增强可选操作集 增加一个空操作*/
④ $ DAC=\{{da}_{1},{da}_{2},… ,{da}_{n}\} $;/*利用$ DA $中的可选操 作随机初始化数据增强方式组合*/
⑤ $ {DAC}_{\rm tmp}\leftarrow DAC; $ /*初始化一个临时数据增强 方式组合*/
⑥ for $ i=1 $ to $ T $
⑦  $ {D}_{\rm train}=Aug({D}_{\rm train},{DAC}_{\rm tmp}) $;/*使用$ {DAC}_{\rm tmp} $对  数据集进行数据增强处理*/
⑧  $ M=T rainModel(M,{D}_{\rm train}) $; /*利用增强后的  数据集$ {D}_{\rm train} $训练异常检测基准模型$ M $*/
⑨  $ {S}_{i}=T estModel(M,{D}_{\rm test}) $; /*在测试集$ {D}_{\rm test} $上  评估模型$ M $的性能,得到性能得分$ {S}_{i} $*/
⑩ if $ {S}_{0} < {S}_{i} $ then
⑪   $ DAC\leftarrow {DAC}_{\rm tmp} $; /*更新最优数据增强方   式组合*/
⑫   $ {S}_{0}\leftarrow {S}_{i} $;
⑬ else
⑭   $ {DAC}_{\rm tmp}\leftarrow DAC; $ /*回退为上一轮的数据   增强方式组合*/
⑮ end if
⑯  $ da'=RandomChoice1\left(DA\right) $; /*从$ DA $中随机  选择一个增强操作$ da' $*/
⑰  $ j=RandomChoice2\left({DAC}_{\rm tmp}\right); $ /*从$ {DAC}_{\rm tmp} $中  随机选择一个位置$ j $*/
⑱  $ {DAC}_{\rm tmp}\left[j\right]\leftarrow da' $;
⑲ end for
具体来说,首先使用原始训练集训练得到一个异常检测模型,并评估其在测试集上的初步性能. 然后引入一个包含空操作的数据增强操作集,并从中随机选择初始的数据增强组合. 在每次迭代中,算法使用当前的数据增强组合处理训练数据,并重新训练模型,然后在测试集上评估模型的性能. 如果新模型的性能超过了之前的最佳性能,算法将更新最优的数据增强组合;否则,保持上一轮的数据增强组合不变. 此外,算法在每轮迭代中随机更换一个数据增强操作,以探索更多可能的操作组合. 通过设定的迭代轮数,算法最终确定一组能够最大化模型性能的数据增强操作组合.
此外,在上述数据增强流程的基础上,还可以采取3种可选的措施来降低算法复杂度:
1)控制数据增强操作集中选取的数据增强操作数量n. 由于初始数据增强操作集的选取是经验性的,而针对不同的数据集及异常检测模型,不同数据增强操作的作用程度是不同的,往往只有少数的操作发挥效果明显,因此通过限制n的大小来得到多个最优操作的集合是更高效的操作.
2)控制数据集的采样率. 在对数据增强操作集评估时,由于需要训练并测试一个完整的异常检测模型,时间成本较大. 因此可以通过控制数据集的采样率,选取一定比例的数据及增强后的数据组成新的训练数据集用于训练. 尽管数据集样本数量的减少会导致最终性能的下降,但由于增强前后使用相同规模的数据集进行训练与检测,性能的优劣变化可以反映数据增强操作的有效与否,只需保证每次数据集的采样率保持一致,即可大大提升训练效率,降低算法复杂度.
3)子策略自适应调整. 根据性能评估的结果,可以动态地调整子操作的更新频率. 如果某一个子操作连续几轮表现较好,可以减少其更新频率,以节省计算资源. 相反,如果某一个子操作连续几轮表现不佳,可以增加其更新频率,以尝试更多的改进. 这样可以根据子操作的表现情况自适应地调整更新策略,提高效率.
同时,在数据预处理阶段,除了进行时序数据增强操作外,还进行了滑动窗口和下采样处理.
滑动窗口处理是将时序数据转换为以大小固定的窗口为单位的样本序列,帮助模型捕捉到时序数据中的局部模式和关联性. 假设训练集或测试集中的一个时间序列$ X=\{{x}_{1},{x}_{2},… ,{x}_{n}\} $,本文将其处理为一组新的样本集合:
$ \begin{split} {X}_{w}=&\{\left({x}_{1},{x}_{2},… ,{x}_{w}\right),\left({x}_{2},{x}_{3},… ,{x}_{w+1}\right), … ,\\&\left({x}_{n-w+1},{x}_{n-w+2},… ,{x}_{n}\right)\}\text{,}\end{split} $
其中,$ w $为滑动窗口的大小,每个样本的大小为$ w $,样本$ {X}_{i} $的标签由该样本最后一个时间点的数据确定,当该时间点的原始数据为异常样本时,则样本$ {X}_{i} $的标签设置为1(异常),否则设置为0(正常). 滑动窗口操作将更多的时间信息作为输入特征,以便于模型更好地学习和训练.
下采样操作是通过减少采样频率或样本数量来减小数据集的规模,由于时序数据集规模较大(包含大量时间步的多维信息). 本文采取和文献[6, 27]一致的下采样率,每5个时间步进行1次平均值计算,这样下采样处理后的数据集规模为原数据集的20%,可以显著减少计算和存储需求,减少模型训练和推理的计算负担,从而提高效率. 同时某些时序数据集可能存在长期依赖关系,即当前样本的预测可能与过去较远的样本相关. 在这种情况下,下采样可以通过降低采样频率,减少时间步长之间的间隔,以更好地捕捉长期依赖关系,从而提高模型的性能.
经过数据增强和滑动窗口、下采样处理后的时序数据集将用于多类别异常检测基准模型的训练. 一个多类别异常检测基准模型的训练流程为:
在无监督环境下,经过数据增强、滑动窗口处理及下采样处理后的训练样本集合表示为$ {X}_{\rm train}=\{({x}_{1}, {x}_{2},… ,{x}_{w}),… ,\left({x}_{n-w+1},… ,{x}_{n}\right)\} $,其中$ n $为训练样本数量,在无监督环境下,训练数据集全部由正常数据组成;经过滑动窗口处理的测试数据集可以表示为$ {X}_{\rm test}= \{\left({x}_{1},{x}_{2},… ,{x}_{w}\right),… ,\left({x}_{m-w+1},… ,{x}_{m}\right)\} $,其中$ m $为测试样本数量,测试数据集由正常数据和少量异常数据组成. 在训练阶段,使用增强后的训练集数据对异常检测模型进行训练,学习捕获正常时序数据的特征;在测试阶段,训练好的异常检测模型将对测试集数据的每个测试样本输出一个异常值分数,用于表征该样本为异常样本的可能性,异常值分数越高则该样本为异常的可能性越大,将该异常值分数与每一个基准模型预先设置的一个经验阈值来决定最终的预测标签为1(异常)或为0(正常).
为了后续利用深度强化学习进行模型筛选,DELTA尝试从异常检测基准模型中提取到更多深层次的信息以利于后续强化学习的训练. 受文献[27]的启发,DELTA使用2个分数来表征模型对预测结果的自信程度:距离阈值置信分数和预测共识置信分数.
距离阈值置信分数反映了模型预测输出异常值分数超出预设异常阈值的程度,置信分数越高则表示该模型对于该样本被认定为异常的自信程度越高,同时本文对该分数进行了最大最小值尺度缩放,将其归一化为[0,1]的范围以便于训练,表示为:
$ {s}_{\rm distance}=\dfrac{s-pt}{max\_s-min\_s}\text{,} $
其中,$ s $为模型预测输出异常值分数,$ pt $为预设异常阈值,$ max\_s $为模型预测输出异常值分数的最大值,$ min\_s $为模型预测输出异常值分数的最小值.
预测共识置信分数反映了所有基准模型对于样本预测结果所达成一致的程度,置信分数越高则表示该样本被所有基准模型认定为同一类型的共识程度越高,表示为:
$ s_{\rm{c}onsensus}=\left\{\begin{aligned} & \dfrac{\displaystyle\sum_{i=0}^{m-1}\left(pl_i==1\right)}{m},\; \; {\rm if}\; l_{\rm{o}utput}\ge d, \\ & \dfrac{\displaystyle\sum_{i=0}^{m-1}\left(pl_i==0\right)}{m},\; \; {\rm if}\; l_{\rm{o}utput} < d,\end{aligned}\right. $
其中,$ m $为基准模型的数量,$ {pl}_{i} $为第$ i+1 $个基准模型的预测标签,$ {l}_{\rm output} $为经过模型筛选后输出的加权标签,$ d\in \left[\mathrm{0,1}\right] $为一个超参数,作为控制模型最终输出为异常或正常的阈值,$ d $越大则模型输出为异常的条件更严格,当$ d=1 $时,意味着当且仅当全部基准模型输出的预测标签均为异常时,模型最终才输出为异常.
此外,利用后续经过使用强化学习模型筛选得到的结果,还可以提取到3种类型的关键信息:加权归一化预测分数、加权归一化异常阈值、加权预测标签.
加权归一化预测分数是经过模型筛选后对原基准模型预测分数进行加权处理得到的新的预测分数,反映了经过模型筛选后样本异常值的预测结果,表示为:
$ {s}_{ scaled}=\dfrac{\displaystyle\sum _{i=0}^{m-1}{scaled\_s}_{i}\times a\left[i\right]}{m}\text{,} $
其中,$ m $为异常检测基准模型的数量,$ {scaled\_s}_{i} $为第$ i+1 $个基准模型经过归一化处理得到的原始预测分数,$ a\left[i\right] $为模型筛选时是否选择该模型用于后续预测.
加权归一化异常阈值是经过模型筛选后对原基准模型的多个异常阈值pt进行加权处理得到的新的异常预测阈值,反映了经过模型筛选后对样本标签预测为异常与否的新的阈值,表示为:
$ {pt}_{scaled}=\dfrac{\displaystyle\sum _{i=0}^{m-1}{scaled\_pt}_{i}\times a\left[i\right]}{m}\text{,} $
其中,$ {scaled\_pt}_{i} $为第$ i+1 $个基准模型经过归一化处理得到的原始异常预测阈值.
加权预测标签是经过模型筛选后对测试集数据点标签预测进行加权处理得到的新的标签,表示为:
${l}_{\rm output}=\dfrac{\displaystyle\sum _{i=0}^{m-1}{l}_{i}\times a\left[i\right]}{m}\text{,} $
其中,$ {l}_{i} $为第$ i+1 $个基准模型的原始预测标签.
异常检测基准模型训练完毕后,DELTA进一步将模型选择问题建模为马尔可夫决策过程[28],对应的元素具体描述为:
1)状态. 状态空间由多个异常检测基准模型(具体使用的基准模型将在实验部分进行介绍)训练后提取得到的关键信息所组成,包括加权归一化预测分数、加权归一化异常阈值、加权预测标签、预测距离置信分数和预测共识置信分数共5项关键信息,具体信息的定义已在2.2节介绍.
2)动作. 动作空间设置为多维离散空间$ A=\{{a}_{1}, {a}_{2},… ,{a}_{n}\} $,维数$ n $为异常检测基准模型的数量,每一维的动作是离散的二值变量,表示当前状态下是否选择该模型用于后续的异常检测(“1”为选取,“0”为不选取).
3)状态转移矩阵. 在该问题环境下,状态按照数据集中的时间戳的顺序进行确定性的转移,即对于数据集中连续的2个时间步对应产生的状态$ {s}_{t} $和$ {s}_{t+1} $,对应的状态转移概率$ P({s}_{t+1}|{s}_{t})=1 $,而对于其他情况下$ P({s}_{t+1}|{s}_{t})=0 $.
4)奖励. 奖励函数是根据当前时间步模型的预测标签与真实标签的对比确定的,真阳性(true positive,$ TP $)表示预测标签和真实标签均为异常;真阴性(true negative,$ TN $)表示预测标签和真实标签均为正常;假阳性(false positive,$ FP $)表示预测标签为异常、真实标签为正常;假阴性(false negative,$ FN $)表示预测标签为正常、真实标签为异常,根据这4种情况构建奖励函数,对真阳性和真阴性给予正奖励反馈,对假阳性和假阴性给予负奖励反馈,具体表示为:
$ R=\left\{\begin{aligned}& {r}_{1},\;\;\quad TP,\\&{r}_{2},\;\;\quad TN,\\&-{r}_{3},\;\;FP,\\&-{r}_{4},\;\;FN,\end{aligned}\right. $
其中,$ {r}_{1} $,$ {r}_{2} $,$ {r}_{3} $,$ {r}_{4} $均为正值.
5)折扣因子. 在该问题环境下,无需对未来奖励进行衰减处理,因此折扣因子设置为1,对应的回报函数为无折扣累积回报.
马尔可夫决策过程构建完毕后,便可以选取合适的深度强化学习算法来进行学习训练. 由于动作空间为多维离散空间,本文选择使用了A2C(advantage Actor-Critic)和PPO(proximal policy optimization)[29]两种深度强化学习算法进行实验. 但值得强调的是,本文所提出的方法框架具有良好的算法兼容性,用于训练使用的强化学习算法类型并不受限,理论上任何适用于多维离散动作空间下的强化学习算法都可适用于本文所提出的方法框架.
为展现框架的通用性,选取的2种算法均是经过时间检验证实有效的经典强化学习算法,二者均属于基于策略梯度的强化学习算法. 其中,A2C使用基于价值函数的优势估计来更新策略,可以减少更新的方差,有良好的训练稳定性和收敛性;PPO则通过对策略更新的幅度进行限制,保证新旧策略之前的更新幅度不会过大,同样有助于提高训练的稳定性和收敛性.
依赖于神经网络强大的拟合能力,2种算法均使用神经网络对价值函数和策略函数进行拟合,分别用$ {V}_{\omega } $和$ {\pi }_{\theta } $表示. 对于价值函数,二者均使用均方误差作为价值函数的损失函数,使用梯度下降法进行更新,其损失函数表示为:
$ \begin{array}{c}Loss=MSE\left(r+\gamma {V}_{\omega }\left({s'}\right),{V}_{\omega }\left(s\right)\right). \end{array} $
对于策略函数,A2C采取最大化策略梯度的方式来进行策略网络参数的更新:
$ \theta =\theta +\alpha {\nabla }_{\theta }{\rm{log}}{\pi }_{\theta }\left({s}_{t},{a}_{t}\right){A}_{\omega }\left({s}_{t},{a}_{t}\right)\text{,} $
其中$ {A}_{\omega } $为优势函数,用于反映当前状态$ {s}_{t} $采取动作$ {a}_{t} $相对于平均程度的优势大小,$ \alpha $为学习率.
而PPO则采取重要性采样、目标函数裁剪等多种优化手段对策略网络参数进行更新,其损失函数表示为:
$ \begin{split} Loss=& \displaystyle\sum _{({s}_{t},{a}_{t})}(\mathrm{min}(ratio\cdot {A}_{\omega }\left({s}_{t},{a}_{t}\right),clip(ratio,1-\varepsilon ,1+\\& \varepsilon )\cdot {A}_{\omega }\left({s}_{t},{a}_{t}\right))) \text{,}\\[-1pt]\end{split} $
其中$ ratio=\dfrac{\pi\mathrm{_{\theta^{\rm{n}ew}}}\left(a_t\right|s_t)}{\pi_{\theta^{\mathrm{\rm{o}ld}}}\left(a_t\right|s_t)} $,$ clip\left(\right) $为裁剪函数,参数$ \varepsilon $为超参数,三者共同控制策略更新幅度不会过大或过小.
本文选取用于进行工控网络威胁检测实验评估的数据集为安全水处理数据集(SWaT)[30],该数据集由新加坡理工大学网络安全中心iTrust所采集并开源,其模拟了一个工业水处理厂的日常运行情况,捕获了试验台内51个传感器和执行器的运行数据,包含了7天的正常运行数据和4天的异常(被攻击)运行数据. 具体来说,SWaT是一个具有51个特征的多源时间序列数据集,包含495000个时间步的正常数据作为训练集,包含449919个时间步的异常数据作为测试集,其中测试集中异常数据占比为12.13%. 该数据集可以真实反映一个现实工控环境下的标准物理系统的运行过程.
本文选取了8种不同类型的时间序列异常检测模型作为对比模型用于实验性能评估. 其中7种模型均基于无监督学习算法,具体介绍为:
1)iForest(isolation forest)算法[31]. iForest算法是异常检测最常用的算法之一,其利用异常样本与正常样本的数量及特征值差异很大的特点,能够在更少的随机特征划分次数中有效地分离出异常样本. 主要优点是它不需要依赖复杂的统计模型和假设,且具有较低的计算复杂度,对于高维数据和大规模数据集也能够有效地进行异常检测.
2)DIF (deep isolation forest for anomaly detection)[32]. DIF在iForest算法的基础上,利用随意初始化的神经网络将原始数据映射到随机表示集合中,随后应用随机轴平行切割来执行数据分割. 这种表示方案有利于在原始数据空间中进行高自由度的分割,从而促进随机表示和基于随机分割的隔离之间的独特协同作用. 同时,作为一种集成学习方法,该模型还提出了一种高效计算表示集合方法,以减小计算开销.
3)LODA(lightweight on-line detector of anomalies) [33]. LODA是一种轻量级的基于集成学习的异常检测模型,主要通过一组非常弱的检测器组成的集合产生一个检测能力更高的检测器.
4)VAE(variational autoencoder)[34] . VAE是一种基于变分推理的生成模型,同时结合了深度学习和概率模型二者的优点,其通过最小化重构误差和最大化隐分布的似然函数学习得到数据的真实分布.
5)beta-VAE (beta variational autoencoder) [35] . beta-VAE作为VAE算法的一种变体,在VAE的基础上通过引入一个beta超参数,以控制损失项中KL散度的重要程度,从而控制模型表示能力与解纠缠能力之间的平衡,当beta为1时beta-VAE退化为原始的VAE.
6)ECOD(empirical cumulative distribution for outlier detection)[36]. ECOD是一种基于经验累积分布的异常值检测算法. ECOD首先计算数据每个维度上的经验累积分布,再利用经验累积分布来估计每个维度上的尾部概率,最后通过汇总在各维度上估计的尾部概率来计算每个数据点的异常评分.
7)COPOD(copula-based outlier detection)[37]. COPOD是一种基于Copula(一种用于建模多变量分布的函数)的多变量数据异常检测算法. COPOD使用Copula来描述多变量数据中的依赖结构,并计算每个样本在Copula模型下的概率密度函数,最后得到每个数据点的异常评分.
此外,本文同样选取了一种基于强化学习的异常检测模型RLMSAD[27]作为对比模型,RLMSAD同样是一种使用强化学习来进行异常检测模型选择的方法,但是不同于本文提出的方法,RLMSAD仅使用DQN算法来对一维动作空间的模型进行选择,且每次只选择一个异常检测模型进行异常评分的预测,没有将所有异常检测模型提供的关键信息完全利用.
本文使用准确率($ P $)、召回率($ R $)和$ F1 $值作为异常检测性能的评估指标:
$ \left\{\begin{aligned}& P=\dfrac{TP}{TP+FP}\text{,}\\& R=\dfrac{TP}{TP+FN}\text{,}\\&F1=\dfrac{2\times P\times R}{P+R}\text{,}\end{aligned}\right. $
其中,异常样本被视为正样本,正常样本被视为负样本. $ TP $表示异常样本被模型正确预测为异常的样本个数,$ FP $表示正常样本被模型错误预测为异常的样本个数,$ FN $表示异常样本被模型错误预测为正常的样本个数. 准确率$ P $表示被模型检测为异常的样本中实际异常样本的百分比,召回率$ R $表示实际异常样本中被模型预测为异常的样本的百分比,$ F1 $值则结合了准确率和召回率两者的优势.
本文实验环境为Python 3.8、PyTorch 2.0.1、cuda 12.1,服务器配置为Intel Xeon Gold 6138 CPU、RTX4090、128 GB内存.
对比模型中,iForest算法使用Scikit-Learn[38]库中的IsolationForest进行实现,DIF,LODA,VAE,beta-VAE,ECOD和COPOD采取PyOD[39]工具箱的官方实现. 其中,训练好的iForest,DIF,LODA,ECOD和COPOD模型构成DELTA模型的基线模型组,参与后续的训练. RLMSAD基于官方代码库[40]实现,所有算法的超参数均与论文或官方代码库中的默认配置保持一致.
为公平比较2种基于强化学习的模型,除部分参数外(将在后续实验部分进行讨论),本文提出的DELTA模型参数与RLMSAD模型保持一致,使用的基准强化学习算法均来自于Stable-baselines3[41](一个通用的标准强化学习算法库),DELTA模型的相关参数设置如表1所示,其中滑动窗口值过大或过小均会影响模型最终性能,为保障性能和对比公平性,参数值参考文献[27]的设置. 同时为了减少误差,以下的所有实验中强化学习模型都随机初始化运行5次,统计评估指标的平均值和标准差结果.
此外,根据使用的深度强化学习基础算法不同,本文基于PPO算法与A2C算法分别提出了2种基于深度强化学习和数据增强的异常检测方法DELTA-P和DELTA-A.
首先展示本文方法与其他对比模型在多变量时间序列数据集SWaT上的总体异常检测性能,结果如表2和图4所示.
由表2和图4结果,可以观察到3个结论:
1)不同类型模型之间的对比. 基于强化学习的异常检测模型总体性能明显优于基于无监督学习的异常检测模型,这得益于前者使用了强化学习算法对不同时间步数据点进行了不同的异常检测模型筛选,进而提升了检测准确率.
2)不同强化学习模型筛选框架之间的对比. 本文提出的模型DELTA-A相较于同样基于强化学习的RLMSAD模型在准确率和$ F1 $值上有明显的提升,但在召回率上有一定程度的下降,这得益于DELTA在不同时间步数据点进行异常检测模型选择时结合了更多数量的异常检测模型,提高了检测准确率,但同时会导致对部分异常过于谨慎,从而降低了召回率.
3)不同强化学习算法之间的对比. 基于A2C算法的DELTA-A模型在准确率、召回率和$ F1 $值上都优于基于PPO算法的DELTA-P模型,从召回率上比较大的差距可以得出,这可能是由于PPO的多次迭代更新会导致训练过程稳定性较差,相反在模型筛选问题上A2C则更容易收敛到更好的结果.
本节探究DELTA中数据增强模块对异常检测性能的作用,该实验分为2部分:数据增强模块对基准模型的影响实验,以及数据增强模块对最终的DELTA模型性能的影响实验.
首先,利用数据增强模块对不同类型的异常检测基准模型进行处理,以探究所提出数据增强模块的通用性与有效性. 实验结果如表3所示.
可以发现,在4种无监督异常检测模型上利用所提出的数据增强模块进行数据增强处理后,所有模型的性能都有不同程度的提高. 其中,基于树形结构的iForest和DIF经过数据增强后得到的数据可以更好地覆盖数据空间,有助于构建更加全面和稳健的隔离树模型;而ECOD和COPOD算法的性能提升相对较小,其原因是此类算法通过对数据分布进行建模,更关注于捕捉原始数据的整体分布特征,而对增强数据的依赖较小.
再利用数据增强处理后的基准模型作为用于DELTA进行模型选择的基准模型,得到最终数据增强模块对于DELTA模型性能的影响对比结果,结果如表4所示.
由表4可以发现,基于A2C算法的DELTA-A模型与基于PPO算法下的DELTA-P模型,在添加数据增强模块后都会使模型的异常检测性能有明显的提升,这得益于数据增强模块对于时序数据集增加样本多样性、提高模型鲁棒性上的贡献,证明了数据增强模块对于DELTA的重要性.
本节探究了强化学习算法中不同奖励函数的设置对于算法的影响,以DELTA-A为例,主要针对FP和FN两个部分的奖励函数进行对比实验,不同值对实验性能影响的结果如图5和图6所示.
由图5可以发现,当FP的惩罚值增大时,检测结果召回率下降,准确率普遍上升. 这是由于加大对FP的惩罚,会使模型在预测时更加保守,即模型更倾向于将待测样本预测为正常,而只会在对预测结果足够有信心时才会报告异常,从而使得预测准确率上升,但同时这会导致模型对于一些较小的异常情况过于谨慎,使得对异常情况的覆盖会变小,从而使得召回率降低.
由图6可以发现,当FN的惩罚值增大时,检测结果召回率上升,准确率普遍下降. 这是由于加大对FN的惩罚,错过实际异常的成本会变得很高,从而会使模型在预测时更加激进,即模型更倾向于将待测样本预测为异常,以避免错过大量的真实异常样本,进而由于覆盖到更多的异常样本,预测召回率随之上升,但同时这也会导致预测准确率的下降.
本节探究了模型中的一个关键超参数——阈值$ d $对模型最终性能的影响,以DELTA-A为例,不同阈值d对实验性能影响的结果如图7所示.
由图7可以发现,随着阈值$ d $的上升,异常检测结果的准确率会上升,召回率会下降. 这是由于当阈值$ d $越高,模型对于作为基线的异常检测模型的需求就越高,即只有当更多数量的基线异常检测模型均将当前时间步样本点预测为异常时,模型最终的预测输出结果才为异常,因此模型最终预测的准确率会提升,同时由于模型对异常样本的判定更为严格,使得轻微的异常很难被所有类型的基准模型判定为异常,进而使得模型最终预测的召回率降低.
本节通过对比不同异常检测模型的训练时间与推理时间来探究模型之间的效率差异,由于相较于传统方法,深度学习类方法(VAE,beta-VAE)训练代价十分大,故在模型效率研究实验中只以效率更高的传统方法作为对比方法进行比较. 以DELTA-A为例,与其他异常检测模型的时间消耗对比实验结果如图8所示.
由图8可以发现,强化学习类的算法模型(RLMSAD和DELTA)由于需要进行额外的模型筛选步骤,比传统的算法模型要花费更多的时间进行模型的训练和推理,但换来了十分明显的性能提升. 而相比于RLMSAD仅支持单维动作空间环境,DELTA由于需要在多维动作空间中进行筛选,会付出更大的时间代价,但在可接受的范围内(少于基于集成学习的DIF的时间花费)同样可以获得更高的性能(准确率提高17.09%,$ F1 $值提高3.58%).
本文提出了一种基于深度强化学习和数据增强的工控网络威胁检测模型选择方法DELTA,该方法首先使用一种时序数据集数据增强选择方案来选取一组合适的数据增强策略,进而对时序数据集进行数据增强,再利用增强数据集对多种类型的异常检测基准模型进行训练,最后利用深度强化学习算法在不同时间步数据点下选择利用合适的基准模型进行组合预测. 与其他异常检测模型相比,DELTA在可接受程度的额外时间成本消耗下,在准确率和F1值上有明显的提高,在时间序列异常检测问题上展现出更大的潜力.
此外,DELTA仍有可以改进的地方. 为降低算法复杂度,本文所使用的数据增强集是基于传统的数据增强策略所组成的,然而近年来基于对抗生成网络进行数据增强的思路得到了广泛应用,未来可以尝试使用对抗生成网络等先进技术进行时序数据增强的尝试. 此外,如何设计高效的数据增强选择方案和合适的强化学习算法以提升模型效率成为未来值得探索的方向.
作者贡献声明:王泽鹏负责设计方案、完成实验及撰写论文;马超对论文模型方法提出指导意见;张壮壮和吴黎兵对论文进行修改和完善;石小川提出选题和研究内容.

参考文献 引证文献
排序方式:
1
Wu H S. A survey of research on anomaly detection for time series[C]//Proc of 2016 13th Int Computer Conf on Wavelet Active Media Technology and Information Processing (ICCWAMTIP). Piscataway, NJ: IEEE, 2016: 426−431
2
Chalapathy R, Chawla S. Deep learning for anomaly detection: A survey[J]. arXiv preprint, arXiv: 1901.03407, 2019
3
徐丽娟,王佰玲,杨美红,等. 工业控制网络多模式攻击检测及异常状态评估方法[J]. 计算机研究与发展,2021,58(11):2333−2349
Xu Lijuan, Wang Bailing, Yang Meihong, et al. Multi-mode attack detection and evaluation of abnormal states for industrial control network[J]. Journal of Computer Research and Development, 2021, 58(11): 2333−2349 (in Chinese)
4
席亮,王勇,张凤斌. 基于自适应人工鱼群FCM的异常检测算法[J]. 计算机研究与发展,2019,56(5):1048−1059
Xi Liang, Wang Yong, Zhang Fengbin. Anomaly detection algorithm based on FCM with adaptive artificial Fish-Swarm[J]. Journal of Computer Research and Development, 2019, 56(5): 1048−1059 (in Chinese)
5
陈波冯,李靖东,卢兴见,等. 基于深度学习的图异常检测技术综述[J]. 计算机研究与发展,2021,58(7):1436−1455
Chen Bofeng, Li Jingdong, Lu Xingjian, et al. Survey of deep learning based graph anomaly detection methods[J]. Journal of Computer Research and Development, 2021, 58(7): 1436−1455 (in Chinese)
6
Audibert J, Michiardi P, Guyard F, et al. USAD: Unsupervised anomaly detection on multivariate time series[C]//Proc of the 26th ACM SIGKDD Int Conf on Knowledge Discovery &amp; Data Mining. New York: ACM, 2020: 3395−3404
7
Zhang Chuxu, Song Dongjin, Chen Yuncong, et al. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 1409−1416
8
Park D, Hoshi Y, Kemp C C. A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder[J]. IEEE Robotics and Automation Letters, 2018, 3(3): 1544−1551
9
Su Ya, Zhao Youjian, Niu Chenhao, et al. Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]//Proc of the 25th ACM SIGKDD Int Conf on Knowledge Discovery &amp; Data Mining. New York: ACM, 2019: 2828−2837
10
Xin Ruyue, Liu Hongyun, Chen Peng, et al. Robust and accurate performance anomaly detection and prediction for cloud applications: A novel ensemble learning-based framework[J]. Journal of Cloud Computing, 2023, 12(1): 1−16
11
Zhao Yubo, Guo Ni, Chen Wei, et al. Multi-step ahead forecasting for electric power load using an ensemble model[J]. Expert Systems with Applications, 2023, 211: 118649
12
Aggarwal C C, Sathe S. Theoretical foundations and algorithms for outlier ensembles[J]. ACM Sigkdd Explorations Newsletter, 2015, 17(1): 24−47
13
Tama B A, Nkenyereye L, Islam S M R, et al. An enhanced anomaly detection in web traffic using a stack of classifier ensemble[J]. IEEE Access, 2020, 8: 24120−24134
14
Adeyemo V E, Abdullah A, JhanJhi N Z, et al. Ensemble and deep-learning methods for two-class and multi-attack anomaly intrusion detection: An empirical study[J]. International Journal of Advanced Computer Science and Applications, 2019, 10(9): 520−528
15
Ünlü R, Xanthopoulos P. A weighted framework for unsupervised ensemble learning based on internal quality measures[J]. Annals of Operations Research, 2019, 276: 229−247
16
Wen Qingsong, Sun Liang, Yang Fan, et al. Time series data augmentation for deep learning: A survey[C]//Proc of the 30th Int Joint Conf on Artificial Intelligence. Freiburg, German: IJCAI, 2021: 4653−4660
17
Gao Jingkun, Song Xiaomin, Wen Qingsong, et al. Robusttad: Robust time series anomaly detection via decomposition and convolutional neural networks[C]//Proc of ACM SIGKDD Workshop on Mining and Learning from Time Series (KDD-MiLeTS 2020). New York: ACM, 2020: 1−9
18
Steven Eyobu O, Han D S. Feature representation and data augmentation for human activity classification based on wearable IMU sensor data using a deep LSTM neural network[J]. Sensors, 2018, 18(9): 2892−2917
19
Park D S, Chan W, Zhang Yu, et al. SpecAugment: A simple data augmentation method for automatic speech recognition[C]. Proc of. Interspeech New York: ACM, 2019: 2613−2617
20
Park D S, Chan W, Zhang Yu, et al. RobustSTL: A robust seasonal-trend decomposition algorithm for long time series[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 5409−5416
21
Li Yan, Lu Xinjiang, Wang Yaqing, et al. Generative time series forecasting with diffusion, denoise, and disentanglement[J]. Advances in Neural Information Processing Systems, 2022, 35: 23009−23022
22
Kang Yanfei, Hyndman R J, Li Feng. GRATIS: GeneRAting time series with diverse and controllable characteristics[J]. Statistical Analysis and Data Mining: The ASA Data Science Journal, 2020, 13(4): 354−376
23
Devries T, Taylor G W. Dataset augmentation in feature space[J]. arXiv preprint, arXiv: 1702.05538, 2017
24
Cheung T H, Yeung D Y. Modals: Modality-agnostic automated data augmentation in the latent space[C]//Proc of the 9th Int Conf on Learning Representations. Washington, DC: ICLR, 2021: 1−18
25
Yoon J, Jarrett D, Van der Schaar M. Time-series generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2019, 32: 5508−5518
26
Fons E, Dawson P, Zeng Xiaojun, et al. Adaptive weighting scheme for automatic time-series data augmentation[J]. arXiv preprint, arXiv: 2102.08310, 2021
27
Zhang J E, Wu Di, Boulet B. Time series anomaly detection via reinforcement learning-based model selection[C]//Proc of 2022 IEEE Canadian Conf on Electrical and Computer Engineering (CCECE). Piscataway, NJ: IEEE, 2022: 193−199
28
Sutton R S, Barto A G. Reinforcement Learning: An Introduction[M]. Cambridge, MA: MIT press, 2018
29
Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms[J]. arXiv preprint, arXiv: 1707.06347, 2017
30
Goh J, Adepu S, Junejo K N, et al. A dataset to support research in the design of secure water treatment systems[C]//Proc of the 11th Int Conf on Critical Information Infrastructures Security (CRITIS 2016). Berlin: Springer, 2016: 88−99
31
Liu F T, Ting Kaiming, Zhou Zhihua. Isolation forest[C]//Proc of 2008 8th IEEE Int Conf on Data Mining. Piscataway, NJ: IEEE, 2008: 413−422
32
Xu Hongzuo, Pang Guansong, Wang Yijie, et al. Deep isolation forest for anomaly detection[J]. IEEE Transactions on Knowledge and Data Engineering, 2023: 12591−12604
33
Pevný T. Loda: Lightweight on-line detector of anomalies[J]. Machine Learning, 2016, 102: 275−304
34
Kingma D P, Welling M. Auto-encoding variational Bayes[J]. arXiv preprint, arXiv: 1312.6114, 2013
35
Higgins I, Matthey L, Pal A, et al. Beta-vae: Learning basic visual concepts with a constrained variational framework[C]//Proc of the 5th Int Conf on Learning Representations. Washington, DC: ICLR, 2017: 1−22
36
Li Zheng, Zhao Yue, Hu Xiyang, et al. Ecod: Unsupervised outlier detection using empirical cumulative distribution functions[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 35(12): 12181−12193
37
Li Zheng, Zhao Yue, Botta N, et al. COPOD: Copula-based outlier detection[C]//Proc of 2020 IEEE Int Conf on Data Mining (ICDM). Piscataway, NJ: IEEE, 2020: 1118−1123
38
Scikit-learn. Scikit-Learn[EB/OL]. 2024 [2024-04-02]. https://github. com/scikit-learn/scikit-learn
39
Zhao Yue. PyOD[EB/OL]. 2024 [2024-04-02]. https://github.com/ yzhao062/pyod
40
Zhang J E. RLMSAD[EB/OL]. 2022 [2024-04-02]. https://github. com/elisejiuqizhang/RLMSAD
41
Raffin A, Hill A, Gleave A, et al. Stable-baselines3: Reliable reinforcement learning implementations[J]. Journal of Machine Learning Research, 2021, 22(268): 1−8
2024年第61卷第10期
61
29
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202440387
  • 接收时间:2024-05-31
  • 首发时间:2026-11-17
  • 出版时间:2024-10-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2024-05-31
  • 修回日期:2024-07-17
基金
作者信息
    武汉大学国家网络安全学院 武汉 430072
    空天信息安全与可信计算教育部重点实验室(武汉大学) 武汉 430072

通讯作者:

参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440387
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏