Article(id=1329419833877311562, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329419793389695520, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202550467, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1749398400000, receivedDateStr=2025-06-09, revisedDate=1768492800000, revisedDateStr=2026-01-16, acceptedDate=1769616000000, acceptedDateStr=2026-01-29, onlineDate=1794887987822, onlineDateStr=2026-11-17, pubDate=1780243200000, pubDateStr=2026-06-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794887987822, onlineIssueDateStr=2026-11-17, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794887987822, creator=13701087609, updateTime=1794887987822, updator=13701087609, issue=Issue{id=1329419793389695520, tenantId=1146029695717560320, journalId=1324759914146529478, year='2026', volume='63', issue='6', pageStart='1377', pageEnd='1638', issueExtLink='null', onlineDate='null', pubDate='1780243200000', pubDateStr='2026-06-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794887978168, creator='13701087609', updateTime=1794892139295, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1329437246815580885, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329419793389695520, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1329437246815580886, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329419793389695520, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=1534, endPage=1547, ext={EN=ArticleExt(id=1329419834066055243, articleId=1329419833877311562, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=GALENet: Neuro-Oriented Speaker Extraction Network Fusing Gated Attention and Local Feature Enhancement, columnId=1325489891789910943, journalTitle=Journal of Computer Research and Development, columnName=Artificial Intelligence, runingTitle=null, highlight=null, articleAbstract=

Neuro-oriented speaker extraction is an intelligent speech processing technique that emulates human auditory attention mechanisms, enabling target speaker extraction from mixed speech by decoding the attentional focus embedded in listeners’ EEG (electroencephalogram) signals. This technology offers novel insights into resolving the “cocktail party problem” and holds significant implications for developing intelligent hearing-assistive devices. However, existing approaches confront three critical challenges: inefficient multimodal feature fusion, inadequate local speech feature extraction, and high computational complexity, which collectively constrain practical application efficacy. We propose GALENet, a novel end-to-end time-domain speaker extraction model, featuring three innovative modules: 1) A gated cross-attention fusion mechanism that establishes cross-modal correlations between EEG and speech through bidirectional cross-attention, while adaptively modulating modal contributions via dynamic gating weights to overcome the information interaction limitations of conventional linear fusion; 2) An E-DPRNN network incorporating Conv-U modules within the dual-path architecture to reinforce local short-term feature capture, synergized with residual connections to enhance gradient propagation efficiency; 3) A single-path global modulation (SPGM) block that replaces traditional inter-block modeling with parameter-free pooling and feature modulation, preserves global semantic modeling capability while reducing parameters. These modules collectively achieve efficient multimodal fusion and precise speaker extraction. Experimental validation demonstrates that the proposed method attains average SI-SDR scores of 13.59 dB, 10.44 dB, and 10.17 dB on the Cocktail Party, AVED, and MM-AAD datasets respectively, outperforming the baseline model MSFNet by 5.4%−8.2% while reducing parameters by 49.7%−55.6%. These results substantiate the model’s application potential and effectiveness in neuro-oriented speaker extraction.

, authors=Cunhang Fan1, Junling Li1, Jingjing Zhang1, Youdian Gao1, Ying Chen1, Jiangyan Yi2, Zhao Lü1, *, authorsList=Cunhang Fan, Junling Li, Jingjing Zhang, Youdian Gao, Ying Chen, Jiangyan Yi, Zhao Lü, authorCompany=null, correspAuthors=Zhao Lü, authorNote=

born in 1991. PhD, associate professor, master supervisor. His main research interests include speech information processing, brain-computer interface technology, and brain-inspired auditory computing

born in 2002. Master candidate. His main research interests include speech separation and brain-controlled target speaker extraction

born in 2000. Master. Her main research interests include speech separation and brain-controlled target speaker extraction

born in 2001. Master candidate. His main research interests include audio-visual speech separation and target speaker extraction

born in 2001. Master candidate. Her main research interests include speech separation and brain-controlled target speaker extraction

born in 1984. PhD, assistant professor. Her main research interests include speech recognize, and speech synthesis and transfer learning

born in 1979. PhD, professor, PhD supervisor. His main research interests include emotion recognition, speech enhancement, and motor imagery

, correspAuthorsNote=null, copyrightStatement=Copyright © 2026 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1329419836427448422, articleId=1329419833877311562, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=GALENet:融合门控注意力与局部特征增强的神经导向说话人提取网络, columnId=1325489892104483745, journalTitle=计算机研究与发展, columnName=人工智能, runingTitle=null, highlight=null, articleAbstract=

神经导向说话人提取是一种模拟人类听觉注意机制的智能语音处理技术,通过解码听者脑电信号(electroencephalography,EEG)中的注意力指向,从混合语音中提取目标说话人语音。该技术为解决鸡尾酒会问题提供了新的解决思路,对开发智能助听设备具有重要意义。然而,现有方法面临多模态特征融合效率低、局部语音特征提取不足、模型计算复杂度高等挑战,制约了实际应用效果。提出新型端到端时域神经导向说话人提取模型GALENet,其创新性体现在3个核心模块:1)门控交叉注意力融合机制,通过双向交叉注意力建模EEG与语音的跨模态关联,引入动态门控权重自适应调节模态贡献,解决传统线性融合的信息交互不足问题;2)E-DPRNN网络,在双路径架构中嵌入Conv-U模块强化局部短时特征捕捉能力,配合残差连接提升梯度传递效率;3)单路径全局调制块(single-path global modulation,SPGM),采用无参数池化与特征调制替代传统块间建模,在减少参数量的同时保持全局语义建模能力。这3个模块协同实现了高效的多模态特征融合与精准的说话人提取。实验验证表明,该方法在Cocktail Party,AVED,MM-AAD这3个数据集上的平均SI-SDR分别达到13.59 dB,10.44 dB,10.17 dB,相较基线模型MSFNet性能提升5.4%~8.2%,参数量减少49.7%~55.6%,验证了该模型在神经导向说话人提取中的应用潜力与有效性。

, authors=范存航1, 李骏凌1, 张晶晶1, 高佑典1, 陈颖1, 易江燕2, 吕钊1, *, authorsList=范存航, 李骏凌, 张晶晶, 高佑典, 陈颖, 易江燕, 吕钊, authorCompany=null, correspAuthors=吕钊, authorNote=

1991年生。博士,教授,博士生导师。主要研究方向为语音信息处理、脑机接口技术、类脑听觉计算

2002年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

2000年生。硕士。主要研究方向为语音分离、脑控目标说话人提取

2001年生。硕士研究生。主要研究方向为视听语音分离、目标说话人提取

2001年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

1984年生。博士,助理教授。主要研究方向为语音识别、语音合成和迁移学习

1979年生。博士,教授,博士生导师。主要研究方向为情绪识别、语音增强、运动想象

, correspAuthorsNote=
吕钊()
, copyrightStatement=版权所有 © 2026 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=ppj9TJYR+u56niSAa1o57g==, magXml=pbR5JzixUt2M2Zl6eZJ7eg==, pdfUrl=null, pdf=5CEoEd3p9/3pTCa4ts5x4A==, pdfFileSize=1769419, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=ixfOdSstDPCi9OeKlwInpg==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=qtk/D6UjhJvLUUa+wjDTuw==, mapNumber=null, fund=null)}, authors=[Author(id=1329419836972707950, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=cunhang.fan@ahu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419837148868722, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419836972707950, language=EN, stringName=Cunhang Fan, firstName=Cunhang, middleName=null, lastName=Fan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 1991. PhD, associate professor, master supervisor. His main research interests include speech information processing, brain-computer interface technology, and brain-inspired auditory computing

"}, bioImg=null, bioContent=

born in 1991. PhD, associate professor, master supervisor. His main research interests include speech information processing, brain-computer interface technology, and brain-inspired auditory computing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419837236949107, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419836972707950, language=CN, stringName=范存航, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"nxBbKEIxEi7Q2TIFrl53Gg==","content":"

1991年生。博士,教授,博士生导师。主要研究方向为语音信息处理、脑机接口技术、类脑听觉计算

"}, bioImg=nxBbKEIxEi7Q2TIFrl53Gg==, bioContent=

1991年生。博士,教授,博士生导师。主要研究方向为语音信息处理、脑机接口技术、类脑听觉计算

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419837299863669, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419837366972535, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837299863669, language=EN, stringName=Junling Li, firstName=Junling, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 2002. Master candidate. His main research interests include speech separation and brain-controlled target speaker extraction

"}, bioImg=null, bioContent=

born in 2002. Master candidate. His main research interests include speech separation and brain-controlled target speaker extraction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419837434081400, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837299863669, language=CN, stringName=李骏凌, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"SC90zRwOOGRF1I6tHZRpdQ==","content":"

2002年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

"}, bioImg=SC90zRwOOGRF1I6tHZRpdQ==, bioContent=

2002年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419837517967482, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419837635407996, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837517967482, language=EN, stringName=Jingjing Zhang, firstName=Jingjing, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 2000. Master. Her main research interests include speech separation and brain-controlled target speaker extraction

"}, bioImg=null, bioContent=

born in 2000. Master. Her main research interests include speech separation and brain-controlled target speaker extraction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419837727682685, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837517967482, language=CN, stringName=张晶晶, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"wMxt/tYMPOYETvvJpVyq8w==","content":"

2000年生。硕士。主要研究方向为语音分离、脑控目标说话人提取

"}, bioImg=wMxt/tYMPOYETvvJpVyq8w==, bioContent=

2000年生。硕士。主要研究方向为语音分离、脑控目标说话人提取

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419837790597247, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419837878677633, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837790597247, language=EN, stringName=Youdian Gao, firstName=Youdian, middleName=null, lastName=Gao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 2001. Master candidate. His main research interests include audio-visual speech separation and target speaker extraction

"}, bioImg=null, bioContent=

born in 2001. Master candidate. His main research interests include audio-visual speech separation and target speaker extraction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419837975146626, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837790597247, language=CN, stringName=高佑典, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"JYfLH9CtpukZpR8V9lC4Hg==","content":"

2001年生。硕士研究生。主要研究方向为视听语音分离、目标说话人提取

"}, bioImg=JYfLH9CtpukZpR8V9lC4Hg==, bioContent=

2001年生。硕士研究生。主要研究方向为视听语音分离、目标说话人提取

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419838054838404, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419838130335878, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838054838404, language=EN, stringName=Ying Chen, firstName=Ying, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 2001. Master candidate. Her main research interests include speech separation and brain-controlled target speaker extraction

"}, bioImg=null, bioContent=

born in 2001. Master candidate. Her main research interests include speech separation and brain-controlled target speaker extraction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419838197444743, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838054838404, language=CN, stringName=陈颖, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"J+olV1r/Eu65Zq+c/5fNLg==","content":"

2001年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

"}, bioImg=J+olV1r/Eu65Zq+c/5fNLg==, bioContent=

2001年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419838256165001, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419838365216907, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838256165001, language=EN, stringName=Jiangyan Yi, firstName=Jiangyan, middleName=null, lastName=Yi, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2Department of Automation, Tsinghua University, Beijing 100084, bio={"content":"

born in 1984. PhD, assistant professor. Her main research interests include speech recognize, and speech synthesis and transfer learning

"}, bioImg=null, bioContent=

born in 1984. PhD, assistant professor. Her main research interests include speech recognize, and speech synthesis and transfer learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419838436520076, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838256165001, language=CN, stringName=易江燕, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2清华大学自动化系 北京 100084, bio={"img":"iW98g6Hv4hCoxhD7lkEMEA==","content":"

1984年生。博士,助理教授。主要研究方向为语音识别、语音合成和迁移学习

"}, bioImg=iW98g6Hv4hCoxhD7lkEMEA==, bioContent=

1984年生。博士,助理教授。主要研究方向为语音识别、语音合成和迁移学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836834295914, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=2, ext=[AuthorCompanyExt(id=1329419836851073131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836834295914, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Department of Automation, Tsinghua University, Beijing 100084), AuthorCompanyExt(id=1329419836859461740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836834295914, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2清华大学自动化系 北京 100084)])]), Author(id=1329419838608486542, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=6, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=kjlz@163.com, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329419838738509968, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838608486542, language=EN, stringName=Zhao Lü, firstName=Zhao, middleName=null, lastName=Lü, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 1979. PhD, professor, PhD supervisor. His main research interests include emotion recognition, speech enhancement, and motor imagery

"}, bioImg=null, bioContent=

born in 1979. PhD, professor, PhD supervisor. His main research interests include emotion recognition, speech enhancement, and motor imagery

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419838822396049, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838608486542, language=CN, stringName=吕钊, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"N5cDRVP7QDTYZVXlFeG49g==","content":"

1979年生。博士,教授,博士生导师。主要研究方向为情绪识别、语音增强、运动想象

"}, bioImg=N5cDRVP7QDTYZVXlFeG49g==, bioContent=

1979年生。博士,教授,博士生导师。主要研究方向为情绪识别、语音增强、运动想象

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])])], keywords=[Keyword(id=1329419838927253650, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, orderNo=1, keyword=neuro-oriented speaker extraction), Keyword(id=1329419839006945427, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, orderNo=2, keyword=EEG signals), Keyword(id=1329419839086637204, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, orderNo=3, keyword=multimodal fusion), Keyword(id=1329419839145357461, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, orderNo=4, keyword=gated cross-attention), Keyword(id=1329419839208272022, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, orderNo=5, keyword=speech separation), Keyword(id=1329419839266992279, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, orderNo=6, keyword=end-to-end learning), Keyword(id=1329419839405404312, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, orderNo=1, keyword=神经导向说话人提取), Keyword(id=1329419839464124569, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, orderNo=2, keyword=EEG信号), Keyword(id=1329419839531233434, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, orderNo=3, keyword=多模态融合), Keyword(id=1329419839598342299, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, orderNo=4, keyword=门控交叉注意力), Keyword(id=1329419839673839772, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, orderNo=5, keyword=语音分离), Keyword(id=1329419839749337245, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, orderNo=6, keyword=端到端学习)], refs=[Reference(id=1329419841984901308, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Bronkhorst A W. The cocktail party phenomenon: A review of research on speech intelligibility in multiple-talker conditions[J]. Acta Acustica United with Acustica, 2000, 86(1): 117−128, articleTitle=null, refAbstract=null), Reference(id=1329419842064593085, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Cherry E C. Some experiments on the recognition of speech, with one and with two ears[J]. Journal of the Acoustical Society of America, 1953, 25: 975−979, articleTitle=null, refAbstract=null), Reference(id=1329419842135896254, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Yan, Wang Deliang, Johnson E M, et al. A deep learning based segregation algorithm to increase speech intelligibility for hearing-impaired listeners in reverberant-noisy conditions[J]. The Journal of the Acoustical Society of America, 2018, 144(3): 1627−1637, articleTitle=null, refAbstract=null), Reference(id=1329419842203005119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=刘宇宸, 宗成庆. 跨模态信息融合的端到端语音翻译[J]. 软件学报, 2023, 34(4): 1837−1849, articleTitle=null, refAbstract=null), Reference(id=1329419842265919680, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Yuchen, Zong Chengqing. End-to-end speech translation by integrating cross-modal information[J]. Journal of Software, 2023, 34(4): 1837−1849 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419842324639937, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=韩松莘, 郭松辉, 徐开勇, 等. 基于帧结构的语音对抗样本重点区域扰动分析[J]. 计算机研究与发展, 2024, 61(3): 685−700, articleTitle=null, refAbstract=null), Reference(id=1329419842383360194, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Han Songshen, Guo Songhui, Xu Kaiyong, et al. Perturbation analysis of the vital region in speech adversarial example based on frame structure[J]. Journal of Computer Research and Development, 2024, 61(3): 685−700 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419842433691843, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Xiaoyu, Du Changde, Zhou Qiongyi, et al. Auditory attention decoding with task-related multi-view contrastive learning[C]//Proc of the 31st ACM Int Conf on Multimedia. New York: ACM, 2023: 6025−6033, articleTitle=null, refAbstract=null), Reference(id=1329419842500800708, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Fan Cunhang, Zhang Hongyu, Huang Wei, et al. DGSD: Dynamical graph self-distillation for EEG-based auditory spatial attention detection[J]. Neural Networks, 2024, 179: 106580, articleTitle=null, refAbstract=null), Reference(id=1329419842572103877, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=O’Sullivan J, Chen Zhuo, Herrero J, et al. Neural decoding of attentional selection in multi-speaker environments without access to clean sources[J]. Journal of Neural Engineering, 2017, 14(5): 056001, articleTitle=null, refAbstract=null), Reference(id=1329419842630824134, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Cai Siqi, Zhang Ran, Li Haizhou. Robust decoding of the auditory attention from EEG recordings through graph convolutional networks[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 2320−2324, articleTitle=null, refAbstract=null), Reference(id=1329419842702127303, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Jie, Xu Qingtian, Zhu Qiushi, et al. BASEN: Time-domain brain-assisted speech enhancement network with convolutional cross attention in multi-talker conditions[C]//Proc of the 24th Interspeech. Kos Island, GR: ISCA, 2023: 3117−3121, articleTitle=null, refAbstract=null), Reference(id=1329419842756653256, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Pan Zexu, Borsdorf M, Cai Siqi, et al. NeuroHeed: Neuro-steered speaker extraction using EEG signals[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 4456−4470, articleTitle=null, refAbstract=null), Reference(id=1329419842823762121, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Fan Cunhang, Zhang Jingjing, Zhang Hongyu, et al. MSFNet: Multi-scale fusion network for brain-controlled speaker extraction[C]//Proc of the 32nd ACM Int Conf on Multimedia. New York: ACM, 2024: 1652−1661, articleTitle=null, refAbstract=null), Reference(id=1329419842886676682, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Zeghidour N, Grangier D. Wavesplit: End-to-end speech separation by speaker clustering[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 2840−2849, articleTitle=null, refAbstract=null), Reference(id=1329419842957979851, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Luo Yi, Chen Zhuo, Yoshioka T. Dual-path RNN: Efficient long sequence modeling for time-domain single-channel speech separation[C]//Proc of the 45th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2020: 46−50, articleTitle=null, refAbstract=null), Reference(id=1329419843020894412, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Subakan C, Ravanelli M, Cornell S, et al. Exploring self-attention mechanisms for speech separation[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023, 31: 2169−2180, articleTitle=null, refAbstract=null), Reference(id=1329419843092197581, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=黄雅婷, 石晶, 许家铭, 等. 鸡尾酒会问题与相关听觉模型的研究现状与展望[J]. 自动化学报, 2019, 45(2): 234−251, articleTitle=null, refAbstract=null), Reference(id=1329419843176083662, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Huang Yating, Shi Jing, Xu Jiaming, et al. Research advances and perspectives on the cocktail party problem and related auditory models[J]. Acat Automatiac Sinica, 2019, 45(2): 234−251 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419843230609615, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=黄翔东, 陈红红, 甘霖. 基于频率-时间扩张密集网络的语音增强方法[J]. 计算机研究与发展, 2023, 60(7): 1628−1638, articleTitle=null, refAbstract=null), Reference(id=1329419843327078608, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Huang Xiangdong, Chen Honghong, Gan Lin. Speech enhancement method based on frequency-time dilated dense network[J]. Journal of Computer Research and Development, 2023, 60(7): 1628−1638 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419843415158993, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Zmolikova K, Delcroix M, Ochiai T, et al. Neural target speech extraction: An overview[J]. IEEE Signal Processing Magazine, 2023, 40(3): 8−29, articleTitle=null, refAbstract=null), Reference(id=1329419843486462162, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Kong Yonghui, Fan Cunhang, Chen Yujie, et al. Bilateral masking with prompt for knowledge graph completion[C]//Proc of Findings of the 19th Association for Computational Linguistics (NAACL). Stroudsburg, PA: ACL, 2024: 240−249, articleTitle=null, refAbstract=null), Reference(id=1329419843549376723, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=李泽宇, 张旭鸿, 蒲誉文, 等. 多模态深度伪造及检测技术综述[J]. 计算机研究与发展, 2023, 60(6): 1396−1416, articleTitle=null, refAbstract=null), Reference(id=1329419843620679892, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Zeyu, Zhang Xuhong, Pu Yuwen, et al. A survey on multimodal deepfake and detection techniques[J]. Journal of Computer Research and Development, 2023, 60(6): 1396−1416 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419843704565973, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Fan Cunhang, Gao Youdian, Pan Zexu, et al. Improved feature extraction network for neuro-oriented target speaker extraction[C]//Proc of the 50th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2025: 1−5, articleTitle=null, refAbstract=null), Reference(id=1329419843788452054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=邱泽林, 姚鼎鼎, 李军锋. 融合双视角特征的两阶段脑控语音增强[J]. 声学学报, 2025, 50(2): 362−372, articleTitle=null, refAbstract=null), Reference(id=1329419843872338135, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Qiu Zelin, Yao Dingding, Li Junfeng. Two-stage brain-controlled speech enhancement with integrated dual-view features[J]. Acat Acustica, 2025, 50(2): 362−372 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419843947835608, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Fan Cunhang, Zhang Sheng, Zhang Jingjing, et al. SSM2Mel: State space model to reconstruct mel spectrogram from the EEG[C]//Proc of the 50th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2025: 1−5, articleTitle=null, refAbstract=null), Reference(id=1329419844014944473, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Yujie, Yi Jiangyan, Fan Cunhang, et al. Region-based optimization in continual learning for audio deepfake detection[C]//Proc of the 39th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2025: 23651−23659, articleTitle=null, refAbstract=null), Reference(id=1329419844082053338, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Fan Cunhang, Chen Yujie, Xue Jun, et al. Progressive distillation based on masked generation feature method for knowledge graph completion[C]//Proc of the 38th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2024: 8380−8388, articleTitle=null, refAbstract=null), Reference(id=1329419844170133723, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Han Cong, O’Sullivan J, Luo Yi, et al. Speaker-independent auditory attention decoding without access to clean speech sources[J]. Science Advances, 2019, 5(5): eaav6134, articleTitle=null, refAbstract=null), Reference(id=1329419844245631196, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Ceolini E, Hjortkjær J, Wong D D E, et al. Brain-informed speech separation (BISS) for enhancement of target speaker in multitalker speech perception[J]. NeuroImage, 2020, 223: 117282, articleTitle=null, refAbstract=null), Reference(id=1329419844312740061, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Reju V G, Koh S N, Soon Y. Convolution using discrete sine and cosine transforms[J]. IEEE Signal Processing Letters, 2007, 14(7): 445−448, articleTitle=null, refAbstract=null), Reference(id=1329419844388237534, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Hosseini M, Celotti L, Plourde É. Speaker-independent brain enhanced speech denoising[C]//Proc of the 46th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2021: 1310−1314, articleTitle=null, refAbstract=null), Reference(id=1329419844459540703, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Hosseini M, Celotti L, Plourde E. End-to-end brain-driven speech enhancement in multi-talker conditions[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022, 30: 1718−1733, articleTitle=null, refAbstract=null), Reference(id=1329419844522455264, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Perez E, Strub F, De Vries H, et al. Film: Visual reasoning with a general conditioning layer[C]//Proc of the 32nd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2018: 1, articleTitle=null, refAbstract=null), Reference(id=1329419844597952737, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=Pan Z, Wichern G, Germain F G, et al. NeuroHeed+: Improving neuro-steered speaker extraction with joint auditory attention detection[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 11456−11460, articleTitle=null, refAbstract=null), Reference(id=1329419844669255906, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=祝传振, 王璇, 郑强. 基于元素分离与整体注意力的图卷积网络框架[J]. 计算机研究与发展, 2024, 61(8): 2008−2019, articleTitle=null, refAbstract=null), Reference(id=1329419844736364771, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhu Chuanzhen, Wang Xuan, Zheng Qiang. Element separation and holistic attention based graph convolutional network framework[J]. Journal of Computer Research and Development, 2024, 61(8): 2008−2019 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419844803473636, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Shengkui, Ma Bin. Mossformer: Pushing the performance limit of monaural speech separation using gated single-head transformer with convolution-augmented joint self-attentions[C]//Proc of the 48th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2023: 1−5, articleTitle=null, refAbstract=null), Reference(id=1329419844878971109, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=41, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Shengkui, Ma Yukun, Ni Chongjia, et al. Mossformer2: Combining transformer and rnn-free recurrent network for enhanced time-domain monaural speech separation[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 10356−10360, articleTitle=null, refAbstract=null), Reference(id=1329419845008994534, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=42, authorNames=null, journalName=null, refType=null, unstructuredReference=Subakan C, Ravanelli M, Cornell S, et al. Attention is all you need in speech separation[C]//Proc of the 46th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2021: 21−25, articleTitle=null, refAbstract=null), Reference(id=1329419845071909095, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=37, rfOrder=43, authorNames=null, journalName=null, refType=null, unstructuredReference=Yip J Q, Zhao Shengkui, Ma Yukun, et al. SPGM: Prioritizing local features for enhanced speech separation performance[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE: 326−330, articleTitle=null, refAbstract=null), Reference(id=1329419845126435048, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=44, authorNames=null, journalName=null, refType=null, unstructuredReference=Broderick M P, Anderson A J, Di Liberto G M, et al. Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narrative speech[J]. Current Biology, 2018, 28(5): 803−809, articleTitle=null, refAbstract=null), Reference(id=1329419845180961001, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=39, rfOrder=45, authorNames=null, journalName=null, refType=null, unstructuredReference=张宏玉, 张晶晶, 董兴广, 等. 基于视听诱发的听觉注意检测脑电数据集[J]. 清华大学学报: 自然科学版, 2024, 64(11): 1919−1926, articleTitle=null, refAbstract=null), Reference(id=1329419845239681258, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=39, rfOrder=46, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Hongyu, Zhang Jingjing, Dong Xingguang, et al. Based on audio-video evoked auditory attention detection electroencephalogram dataset[J]. Journal of Tsinghua University: Science and Technology, 2024, 64(11): 1919−1926 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419845306790123, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=40, rfOrder=47, authorNames=null, journalName=null, refType=null, unstructuredReference=Fan Cunhang, Zhang Hongyu, Ni Qinke, et al. Seeing helps hearing: A multi-modal dataset and a mamba-based dual branch parallel network for auditory attention decoding[J]. Information Fusion, 2025, 118: 102946, articleTitle=null, refAbstract=null), Reference(id=1329419845399064812, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=41, rfOrder=48, authorNames=null, journalName=null, refType=null, unstructuredReference=Delorme A, Makeig S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis[J]. Journal of Neuroscience Methods, 2004, 134(1): 9−21, articleTitle=null, refAbstract=null), Reference(id=1329419845478756589, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=42, rfOrder=49, authorNames=null, journalName=null, refType=null, unstructuredReference=Moinnereau M A, Rouat J, Whittingstall K, et al. A frequency-band coupling model of EEG signals can capture features from an input audio stimulus[J]. Hearing Research, 2020, 393: 107994, articleTitle=null, refAbstract=null), Reference(id=1329419845541671150, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=43, rfOrder=50, authorNames=null, journalName=null, refType=null, unstructuredReference=Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint, arXiv: 1412.6980, 2014, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)]), AuthorCompany(id=1329419836834295914, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=2, ext=[AuthorCompanyExt(id=1329419836851073131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836834295914, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Department of Automation, Tsinghua University, Beijing 100084), AuthorCompanyExt(id=1329419836859461740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836834295914, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2清华大学自动化系 北京 100084)])], figs=[ArticleFig(id=1329419839862583454, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Figure 1, caption=Overall block diagram of the proposed GALENet model, figureFileSmall=HvGsH2RiGlbJnrP+kwhTpA==, figureFileBig=ixfOdSstDPCi9OeKlwInpg==, tableContent=null), ArticleFig(id=1329419839925498015, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=图1, caption=所提出的 GALENet 模型整体框图, figureFileSmall=HvGsH2RiGlbJnrP+kwhTpA==, figureFileBig=ixfOdSstDPCi9OeKlwInpg==, tableContent=null), ArticleFig(id=1329419840122630304, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Figure 2, caption=Basic principles of multi-layer GCN, figureFileSmall=oqB8xwW/OrtT21Ps7XUzbw==, figureFileBig=PNEllfrb2xKa73mU8RYsoA==, tableContent=null), ArticleFig(id=1329419840227487906, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=图2, caption=多层 GCN 基本原理, figureFileSmall=oqB8xwW/OrtT21Ps7XUzbw==, figureFileBig=PNEllfrb2xKa73mU8RYsoA==, tableContent=null), ArticleFig(id=1329419840294596771, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Figure 3, caption=Composition structure of ResNet module, figureFileSmall=yNaAi7vgGERA/kat7vr6bg==, figureFileBig=g5wqTSkNHSNnuB53OTuoXw==, tableContent=null), ArticleFig(id=1329419840365899940, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=图3, caption=ResNet 模块组成结构, figureFileSmall=yNaAi7vgGERA/kat7vr6bg==, figureFileBig=g5wqTSkNHSNnuB53OTuoXw==, tableContent=null), ArticleFig(id=1329419840437203109, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Figure 4, caption=Composition structure of Conv-U module, figureFileSmall=ToVDFhJM9P41f/D+XcPjjg==, figureFileBig=neLJ+9zXMU+/sp7PA+WClQ==, tableContent=null), ArticleFig(id=1329419840500117670, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=图4, caption=Conv-U 模块组成结构, figureFileSmall=ToVDFhJM9P41f/D+XcPjjg==, figureFileBig=neLJ+9zXMU+/sp7PA+WClQ==, tableContent=null), ArticleFig(id=1329419840571420839, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Figure 5, caption=Composition structure of SPGM module, figureFileSmall=gz5SDjdwcLnLILrBzEu6qg==, figureFileBig=JPU88eeiltM2RcirR9VZng==, tableContent=null), ArticleFig(id=1329419840642724008, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=图5, caption=SPGM模块组成结构, figureFileSmall=gz5SDjdwcLnLILrBzEu6qg==, figureFileBig=JPU88eeiltM2RcirR9VZng==, tableContent=null), ArticleFig(id=1329419840705638569, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Figure 6, caption=Comparison of the proposed model and MSFNet method on Cocktail Party dataset, figureFileSmall=NVzpAKkJIA/np6GBrl2pmw==, figureFileBig=rMVE2ZZCxudovlASF1KhNQ==, tableContent=null), ArticleFig(id=1329419840772747434, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=图6, caption=本文模型在 Cocktail Party 数据集与 MSFNet 方法的比较, figureFileSmall=NVzpAKkJIA/np6GBrl2pmw==, figureFileBig=rMVE2ZZCxudovlASF1KhNQ==, tableContent=null), ArticleFig(id=1329419840831467691, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Table 1, caption=

Summary of Detailed Settings for Each Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
参数Cocktail Party
数据集
自采AVED
数据集
MM-AAD
数据集
受试者数量332050
受试者性别28名男性、
5名女性
14名男性、
6名女性
34名男性、
16名女性
演讲者性别不同男性男性和女性男性和女性
EEG采集通道数1283232
每个受试者参与试验次数301620
每个EEG试验持续时间/s60152165
演讲语言英语普通话普通话
), ArticleFig(id=1329419840898576556, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=表1, caption=

各个数据集的详细设置总结

, figureFileSmall=null, figureFileBig=null, tableContent=
参数Cocktail Party
数据集
自采AVED
数据集
MM-AAD
数据集
受试者数量332050
受试者性别28名男性、
5名女性
14名男性、
6名女性
34名男性、
16名女性
演讲者性别不同男性男性和女性男性和女性
EEG采集通道数1283232
每个受试者参与试验次数301620
每个EEG试验持续时间/s60152165
演讲语言英语普通话普通话
), ArticleFig(id=1329419840969879725, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Table 2, caption=

Overall Structure Parameters of GALENet Model

, figureFileSmall=null, figureFileBig=null, tableContent=
模块模型层输入尺寸超参数输出尺寸
语音编码器Conv1D(B,1,29184)$ {K}_{\mathrm{s}} $=16 $ {S}_{\mathrm{t}} $=8(B,128,3653)
EEG编码器GCN(B,128/32,29184)(B,128/32,29184)
Conv1D(B,128/32,29184)$ {K}_{\mathrm{s}} $=2 $ {S}_{\mathrm{t}} $=1(B,64,29183)
通道层归一化(B,64,29183)(B,64,29183)
Conv1D(B,64,29183)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,64,29183)
ResBlock×3(B,64,29183)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,128,1080)
Conv1D(B,128,1080)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,64,1080)
注意力融合语音Conv1D(B,128,3653)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,64,3653)
EEG线性插值(B,64,1080)(B,64,3653)
多层交叉注意力(B,3653,64)$ {A}_{\mathrm{s}} $=4(B,3653,64)
门控注意力融合(B,3653,64)(B,64,3653)
说话人提取
E-DPRNN×5
分段(B,64,3653)$ K $=250 $ S $=32(B,64,250,32)
块内BLSTM(B*32,64,250)(B,64,250,32)
块间SPGM(B*250,64,32)(B,64,250,32)
重叠相加(B,64,250,32)(B,64,3653)
Conv1D(B,64,3653)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,128,3653)
语音解码器Deconv1D(B,128,3653)$ {K}_{\mathrm{s}} $=16 $ {S}_{\mathrm{t}} $=8(B,1,29232)
切片操作(B,1,29232)(B,1,29184)
), ArticleFig(id=1329419841041182894, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=表2, caption=

GALENet模型整体结构参数

, figureFileSmall=null, figureFileBig=null, tableContent=
模块模型层输入尺寸超参数输出尺寸
语音编码器Conv1D(B,1,29184)$ {K}_{\mathrm{s}} $=16 $ {S}_{\mathrm{t}} $=8(B,128,3653)
EEG编码器GCN(B,128/32,29184)(B,128/32,29184)
Conv1D(B,128/32,29184)$ {K}_{\mathrm{s}} $=2 $ {S}_{\mathrm{t}} $=1(B,64,29183)
通道层归一化(B,64,29183)(B,64,29183)
Conv1D(B,64,29183)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,64,29183)
ResBlock×3(B,64,29183)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,128,1080)
Conv1D(B,128,1080)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,64,1080)
注意力融合语音Conv1D(B,128,3653)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,64,3653)
EEG线性插值(B,64,1080)(B,64,3653)
多层交叉注意力(B,3653,64)$ {A}_{\mathrm{s}} $=4(B,3653,64)
门控注意力融合(B,3653,64)(B,64,3653)
说话人提取
E-DPRNN×5
分段(B,64,3653)$ K $=250 $ S $=32(B,64,250,32)
块内BLSTM(B*32,64,250)(B,64,250,32)
块间SPGM(B*250,64,32)(B,64,250,32)
重叠相加(B,64,250,32)(B,64,3653)
Conv1D(B,64,3653)$ {K}_{\mathrm{s}} $=1 $ {S}_{\mathrm{t}} $=1(B,128,3653)
语音解码器Deconv1D(B,128,3653)$ {K}_{\mathrm{s}} $=16 $ {S}_{\mathrm{t}} $=8(B,1,29232)
切片操作(B,1,29232)(B,1,29184)
), ArticleFig(id=1329419841116680367, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Table 3, caption=

Comparison of Various Methods on Cocktail Party Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
方法SI-SDR/dB↑SDR/dB↑STOI↑ESTOI↑PESQ↑参数量
Mixture0.450.470.710.551.61
BESD‎[29]5.750.791.790.54×106
UBESD‎[30]8.540.831.971.84×106
BASEN*‎[10]11.5611.660.860.722.210.65×106
MSFNet‎[12]12.8913.030.880.772.511.53×106
GALENet
(本文)
13.5913.740.890.782.540.77×106
), ArticleFig(id=1329419841187983537, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=表3, caption=

在Cocktail Party数据集上各种方法的比较

, figureFileSmall=null, figureFileBig=null, tableContent=
方法SI-SDR/dB↑SDR/dB↑STOI↑ESTOI↑PESQ↑参数量
Mixture0.450.470.710.551.61
BESD‎[29]5.750.791.790.54×106
UBESD‎[30]8.540.831.971.84×106
BASEN*‎[10]11.5611.660.860.722.210.65×106
MSFNet‎[12]12.8913.030.880.772.511.53×106
GALENet
(本文)
13.5913.740.890.782.540.77×106
), ArticleFig(id=1329419841271869618, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Table 4, caption=

Comparison of Various Methods on AVED Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
方法SI-SDR/dB↑SDR/dB↑STOI↑ESTOI↑PESQ↑参数量
Mixture1.721.730.760.631.58
UBESD*‎[29]7.898.100.850.721.751.69×106
BASEN*‎[30]8.468.680.860.751.910.56×106
MSFNet‎[12]9.659.840.890.792.071.42×106
GALENet
(本文)
10.4410.600.900.802.080.63×106
), ArticleFig(id=1329419841347367091, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=表4, caption=

在AVED数据集上各种方法的比较

, figureFileSmall=null, figureFileBig=null, tableContent=
方法SI-SDR/dB↑SDR/dB↑STOI↑ESTOI↑PESQ↑参数量
Mixture1.721.730.760.631.58
UBESD*‎[29]7.898.100.850.721.751.69×106
BASEN*‎[30]8.468.680.860.751.910.56×106
MSFNet‎[12]9.659.840.890.792.071.42×106
GALENet
(本文)
10.4410.600.900.802.080.63×106
), ArticleFig(id=1329419841427058868, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Table 5, caption=

Comparison of Various Methods on MM-AAD Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
方法SI-SDR/dB↑SDR/dB↑STOI↑ESTOI↑PESQ↑参数量
Mixture2.122.140.800.641.54
UBESD*‎[29]6.296.600.840.701.661.69×106
BASEN*‎[30]8.048.220.880.742.010.56×106
MSFNet*‎[12]9.579.690.900.792.181.42×106
GALENet
(本文)
10.1710.320.910.812.190.63×106
), ArticleFig(id=1329419841502556341, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=表5, caption=

在MM-AAD数据集上各种方法的比较

, figureFileSmall=null, figureFileBig=null, tableContent=
方法SI-SDR/dB↑SDR/dB↑STOI↑ESTOI↑PESQ↑参数量
Mixture2.122.140.800.641.54
UBESD*‎[29]6.296.600.840.701.661.69×106
BASEN*‎[30]8.048.220.880.742.010.56×106
MSFNet*‎[12]9.579.690.900.792.181.42×106
GALENet
(本文)
10.1710.320.910.812.190.63×106
), ArticleFig(id=1329419841565470902, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Table 6, caption=

Comparison of Model Performance Under Different Modal Inputs

, figureFileSmall=null, figureFileBig=null, tableContent=
方法SI-SDR/dB↑SDR/dB↑STOI↑ESTOI↑PESQ↑
Speech-Only0.470.460.710.551.61
GALENet(本文)13.5913.740.890.782.54
), ArticleFig(id=1329419841619996855, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=表6, caption=

不同模态输入下的模型性能对比

, figureFileSmall=null, figureFileBig=null, tableContent=
方法SI-SDR/dB↑SDR/dB↑STOI↑ESTOI↑PESQ↑
Speech-Only0.470.460.710.551.61
GALENet(本文)13.5913.740.890.782.54
), ArticleFig(id=1329419841695494328, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Table 7, caption=

Ablation Experiments of E-DPRNN Network

, figureFileSmall=null, figureFileBig=null, tableContent=
Conv-U
模块
SPGM
模块
SI-SDR/
dB↑
SDR/
dB↑
STOI↑ESTOI↑PESQ↑参数量
××13.4913.630.890.782.521.03×106
×√13.3113.510.890.772.470.72×106
√√13.5913.740.890.782.540.77×106
), ArticleFig(id=1329419841766797497, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=表7, caption=

E-DPRNN网络的消融实验

, figureFileSmall=null, figureFileBig=null, tableContent=
Conv-U
模块
SPGM
模块
SI-SDR/
dB↑
SDR/
dB↑
STOI↑ESTOI↑PESQ↑参数量
××13.4913.630.890.782.521.03×106
×√13.3113.510.890.772.470.72×106
√√13.5913.740.890.782.540.77×106
), ArticleFig(id=1329419841821323450, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=EN, label=Table 8, caption=

Comparative Study of Different Fusion Methods

, figureFileSmall=null, figureFileBig=null, tableContent=
方法融合方式SI-SDR/
dB↑
SDR/
dB↑
STOI↑ESTOI↑PESQ↑参数量
Mixture0.450.470.710.551.61
GALENet#1Add12.8312.990.880.752.380.73×106
GALENet#2Concat13.0813.220.880.762.440.74×106
GALENet#3CMCA13.3413.490.890.772.471.25×106
GALENet
(本文)
MCA+GA13.5913.740.890.782.540.77×106
), ArticleFig(id=1329419841905209531, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, language=CN, label=表8, caption=

不同融合方式的对比研究

, figureFileSmall=null, figureFileBig=null, tableContent=
方法融合方式SI-SDR/
dB↑
SDR/
dB↑
STOI↑ESTOI↑PESQ↑参数量
Mixture0.450.470.710.551.61
GALENet#1Add12.8312.990.880.752.380.73×106
GALENet#2Concat13.0813.220.880.762.440.74×106
GALENet#3CMCA13.3413.490.890.772.471.25×106
GALENet
(本文)
MCA+GA13.5913.740.890.782.540.77×106
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550467, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550467, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202550467, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202550467, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1780243200000, fullTextJson=null, articleText=null, reference=null)
收藏切换
GALENet:融合门控注意力与局部特征增强的神经导向说话人提取网络
范存航 1 , 李骏凌 1 , 张晶晶 1 , 高佑典 1 , 陈颖 1 , 易江燕 2 , 吕钊 1, *
计算机研究与发展 | 人工智能 2026,63(6): 1534-1547
收起
收藏切换
计算机研究与发展 |人工智能 2026 , 63 (6) : 1534 -1547
GALENet:融合门控注意力与局部特征增强的神经导向说话人提取网络
全屏
[Author(id=1329419836972707950, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=cunhang.fan@ahu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419837148868722, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419836972707950, language=EN, stringName=Cunhang Fan, firstName=Cunhang, middleName=null, lastName=Fan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 1991. PhD, associate professor, master supervisor. His main research interests include speech information processing, brain-computer interface technology, and brain-inspired auditory computing

"}, bioImg=null, bioContent=

born in 1991. PhD, associate professor, master supervisor. His main research interests include speech information processing, brain-computer interface technology, and brain-inspired auditory computing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419837236949107, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419836972707950, language=CN, stringName=范存航, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"nxBbKEIxEi7Q2TIFrl53Gg==","content":"

1991年生。博士,教授,博士生导师。主要研究方向为语音信息处理、脑机接口技术、类脑听觉计算

"}, bioImg=nxBbKEIxEi7Q2TIFrl53Gg==, bioContent=

1991年生。博士,教授,博士生导师。主要研究方向为语音信息处理、脑机接口技术、类脑听觉计算

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419837299863669, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419837366972535, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837299863669, language=EN, stringName=Junling Li, firstName=Junling, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 2002. Master candidate. His main research interests include speech separation and brain-controlled target speaker extraction

"}, bioImg=null, bioContent=

born in 2002. Master candidate. His main research interests include speech separation and brain-controlled target speaker extraction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419837434081400, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837299863669, language=CN, stringName=李骏凌, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"SC90zRwOOGRF1I6tHZRpdQ==","content":"

2002年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

"}, bioImg=SC90zRwOOGRF1I6tHZRpdQ==, bioContent=

2002年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419837517967482, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419837635407996, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837517967482, language=EN, stringName=Jingjing Zhang, firstName=Jingjing, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 2000. Master. Her main research interests include speech separation and brain-controlled target speaker extraction

"}, bioImg=null, bioContent=

born in 2000. Master. Her main research interests include speech separation and brain-controlled target speaker extraction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419837727682685, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837517967482, language=CN, stringName=张晶晶, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"wMxt/tYMPOYETvvJpVyq8w==","content":"

2000年生。硕士。主要研究方向为语音分离、脑控目标说话人提取

"}, bioImg=wMxt/tYMPOYETvvJpVyq8w==, bioContent=

2000年生。硕士。主要研究方向为语音分离、脑控目标说话人提取

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419837790597247, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419837878677633, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837790597247, language=EN, stringName=Youdian Gao, firstName=Youdian, middleName=null, lastName=Gao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 2001. Master candidate. His main research interests include audio-visual speech separation and target speaker extraction

"}, bioImg=null, bioContent=

born in 2001. Master candidate. His main research interests include audio-visual speech separation and target speaker extraction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419837975146626, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419837790597247, language=CN, stringName=高佑典, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"JYfLH9CtpukZpR8V9lC4Hg==","content":"

2001年生。硕士研究生。主要研究方向为视听语音分离、目标说话人提取

"}, bioImg=JYfLH9CtpukZpR8V9lC4Hg==, bioContent=

2001年生。硕士研究生。主要研究方向为视听语音分离、目标说话人提取

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419838054838404, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419838130335878, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838054838404, language=EN, stringName=Ying Chen, firstName=Ying, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 2001. Master candidate. Her main research interests include speech separation and brain-controlled target speaker extraction

"}, bioImg=null, bioContent=

born in 2001. Master candidate. Her main research interests include speech separation and brain-controlled target speaker extraction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419838197444743, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838054838404, language=CN, stringName=陈颖, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"J+olV1r/Eu65Zq+c/5fNLg==","content":"

2001年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

"}, bioImg=J+olV1r/Eu65Zq+c/5fNLg==, bioContent=

2001年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])]), Author(id=1329419838256165001, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419838365216907, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838256165001, language=EN, stringName=Jiangyan Yi, firstName=Jiangyan, middleName=null, lastName=Yi, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2Department of Automation, Tsinghua University, Beijing 100084, bio={"content":"

born in 1984. PhD, assistant professor. Her main research interests include speech recognize, and speech synthesis and transfer learning

"}, bioImg=null, bioContent=

born in 1984. PhD, assistant professor. Her main research interests include speech recognize, and speech synthesis and transfer learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419838436520076, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838256165001, language=CN, stringName=易江燕, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2清华大学自动化系 北京 100084, bio={"img":"iW98g6Hv4hCoxhD7lkEMEA==","content":"

1984年生。博士,助理教授。主要研究方向为语音识别、语音合成和迁移学习

"}, bioImg=iW98g6Hv4hCoxhD7lkEMEA==, bioContent=

1984年生。博士,助理教授。主要研究方向为语音识别、语音合成和迁移学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836834295914, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=2, ext=[AuthorCompanyExt(id=1329419836851073131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836834295914, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Department of Automation, Tsinghua University, Beijing 100084), AuthorCompanyExt(id=1329419836859461740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836834295914, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2清华大学自动化系 北京 100084)])]), Author(id=1329419838608486542, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, orderNo=6, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=kjlz@163.com, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329419838738509968, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838608486542, language=EN, stringName=Zhao Lü, firstName=Zhao, middleName=null, lastName=Lü, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601, bio={"content":"

born in 1979. PhD, professor, PhD supervisor. His main research interests include emotion recognition, speech enhancement, and motor imagery

"}, bioImg=null, bioContent=

born in 1979. PhD, professor, PhD supervisor. His main research interests include emotion recognition, speech enhancement, and motor imagery

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419838822396049, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, authorId=1329419838608486542, language=CN, stringName=吕钊, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601, bio={"img":"N5cDRVP7QDTYZVXlFeG49g==","content":"

1979年生。博士,教授,博士生导师。主要研究方向为情绪识别、语音增强、运动想象

"}, bioImg=N5cDRVP7QDTYZVXlFeG49g==, bioContent=

1979年生。博士,教授,博士生导师。主要研究方向为情绪识别、语音增强、运动想象

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419836700078183, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, xref=1, ext=[AuthorCompanyExt(id=1329419836712661096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601), AuthorCompanyExt(id=1329419836721049705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419833877311562, companyId=1329419836700078183, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601)])])]
范存航1 , 李骏凌1, 张晶晶1, 高佑典1, 陈颖1, 易江燕2, 吕钊1, *
作者信息
  • 1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601
  • 2清华大学自动化系 北京 100084
通讯作者:
吕钊()
作者简介:

1991年生。博士,教授,博士生导师。主要研究方向为语音信息处理、脑机接口技术、类脑听觉计算

2002年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

2000年生。硕士。主要研究方向为语音分离、脑控目标说话人提取

2001年生。硕士研究生。主要研究方向为视听语音分离、目标说话人提取

2001年生。硕士研究生。主要研究方向为语音分离、脑控目标说话人提取

1984年生。博士,助理教授。主要研究方向为语音识别、语音合成和迁移学习

1979年生。博士,教授,博士生导师。主要研究方向为情绪识别、语音增强、运动想象

GALENet: Neuro-Oriented Speaker Extraction Network Fusing Gated Attention and Local Feature Enhancement
Cunhang Fan1 , Junling Li1, Jingjing Zhang1, Youdian Gao1, Ying Chen1, Jiangyan Yi2, Zhao Lü1, *
Affiliations
  • 1State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology (Anhui University), Hefei 230601
  • 2Department of Automation, Tsinghua University, Beijing 100084
  • born in 1991. PhD, associate professor, master supervisor. His main research interests include speech information processing, brain-computer interface technology, and brain-inspired auditory computing

    born in 2002. Master candidate. His main research interests include speech separation and brain-controlled target speaker extraction

    born in 2000. Master. Her main research interests include speech separation and brain-controlled target speaker extraction

    born in 2001. Master candidate. His main research interests include audio-visual speech separation and target speaker extraction

    born in 2001. Master candidate. Her main research interests include speech separation and brain-controlled target speaker extraction

    born in 1984. PhD, assistant professor. Her main research interests include speech recognize, and speech synthesis and transfer learning

    born in 1979. PhD, professor, PhD supervisor. His main research interests include emotion recognition, speech enhancement, and motor imagery

出版时间: 2026-06-01 doi: 10.7544/issn1000-1239.202550467
文章导航
收藏切换

神经导向说话人提取是一种模拟人类听觉注意机制的智能语音处理技术,通过解码听者脑电信号(electroencephalography,EEG)中的注意力指向,从混合语音中提取目标说话人语音。该技术为解决鸡尾酒会问题提供了新的解决思路,对开发智能助听设备具有重要意义。然而,现有方法面临多模态特征融合效率低、局部语音特征提取不足、模型计算复杂度高等挑战,制约了实际应用效果。提出新型端到端时域神经导向说话人提取模型GALENet,其创新性体现在3个核心模块:1)门控交叉注意力融合机制,通过双向交叉注意力建模EEG与语音的跨模态关联,引入动态门控权重自适应调节模态贡献,解决传统线性融合的信息交互不足问题;2)E-DPRNN网络,在双路径架构中嵌入Conv-U模块强化局部短时特征捕捉能力,配合残差连接提升梯度传递效率;3)单路径全局调制块(single-path global modulation,SPGM),采用无参数池化与特征调制替代传统块间建模,在减少参数量的同时保持全局语义建模能力。这3个模块协同实现了高效的多模态特征融合与精准的说话人提取。实验验证表明,该方法在Cocktail Party,AVED,MM-AAD这3个数据集上的平均SI-SDR分别达到13.59 dB,10.44 dB,10.17 dB,相较基线模型MSFNet性能提升5.4%~8.2%,参数量减少49.7%~55.6%,验证了该模型在神经导向说话人提取中的应用潜力与有效性。

神经导向说话人提取  /  EEG信号  /  多模态融合  /  门控交叉注意力  /  语音分离  /  端到端学习

Neuro-oriented speaker extraction is an intelligent speech processing technique that emulates human auditory attention mechanisms, enabling target speaker extraction from mixed speech by decoding the attentional focus embedded in listeners’ EEG (electroencephalogram) signals. This technology offers novel insights into resolving the “cocktail party problem” and holds significant implications for developing intelligent hearing-assistive devices. However, existing approaches confront three critical challenges: inefficient multimodal feature fusion, inadequate local speech feature extraction, and high computational complexity, which collectively constrain practical application efficacy. We propose GALENet, a novel end-to-end time-domain speaker extraction model, featuring three innovative modules: 1) A gated cross-attention fusion mechanism that establishes cross-modal correlations between EEG and speech through bidirectional cross-attention, while adaptively modulating modal contributions via dynamic gating weights to overcome the information interaction limitations of conventional linear fusion; 2) An E-DPRNN network incorporating Conv-U modules within the dual-path architecture to reinforce local short-term feature capture, synergized with residual connections to enhance gradient propagation efficiency; 3) A single-path global modulation (SPGM) block that replaces traditional inter-block modeling with parameter-free pooling and feature modulation, preserves global semantic modeling capability while reducing parameters. These modules collectively achieve efficient multimodal fusion and precise speaker extraction. Experimental validation demonstrates that the proposed method attains average SI-SDR scores of 13.59 dB, 10.44 dB, and 10.17 dB on the Cocktail Party, AVED, and MM-AAD datasets respectively, outperforming the baseline model MSFNet by 5.4%−8.2% while reducing parameters by 49.7%−55.6%. These results substantiate the model’s application potential and effectiveness in neuro-oriented speaker extraction.

neuro-oriented speaker extraction  /  EEG signals  /  multimodal fusion  /  gated cross-attention  /  speech separation  /  end-to-end learning
范存航, 李骏凌, 张晶晶, 高佑典, 陈颖, 易江燕, 吕钊. GALENet:融合门控注意力与局部特征增强的神经导向说话人提取网络. 计算机研究与发展, 2026 , 63 (6) : 1534 -1547 . DOI: 10.7544/issn1000-1239.202550467
Cunhang Fan, Junling Li, Jingjing Zhang, Youdian Gao, Ying Chen, Jiangyan Yi, Zhao Lü. GALENet: Neuro-Oriented Speaker Extraction Network Fusing Gated Attention and Local Feature Enhancement[J]. Journal of Computer Research and Development, 2026 , 63 (6) : 1534 -1547 . DOI: 10.7544/issn1000-1239.202550467
在复杂声学环境中,人类大脑会展现出卓越的听觉注意选择能力,即当多个说话人同时发声时,听者能通过选择性注意机制从混合语音中准确筛选出感兴趣的目标说话人,这一现象被称为“鸡尾酒会效应”[1]。其神经机制最早可追溯至Cherry[2]于1953年的双耳分听实验,该研究证实大脑会对专注侧的语音进行深度处理,而抑制非注意侧的干扰信号。然而,听力受损人群会因听觉系统功能障碍,难以在嘈杂环境中准确分辨出目标说话人,导致社交沟通受阻。为解决这一困难,说话人提取技术通过信号处理手段从混合声源中提取目标说话人语音,已广泛应用于助听设备和语音处理系统[3-5]。但传统的说话人提取方法往往缺乏对人类选择性听觉注意机制的建模能力,在多说话人环境中面临环境干扰,进而难以准确分辨并提取目标说话人的语音。而神经科学研究发现,听者的注意力指向可通过大脑活动的脑电信号(electroencephalography,EEG)解码[6-8],这为解决“鸡尾酒会”问题提供了新思路——通过对EEG中的听觉注意信息进行听觉注意解码(auditory attention detection,AAD)[9],建立混合语音与脑神经活动间的映射关系,实现基于主观注意指向的说话人提取。
当前目标说话人提取主流方法中比较高效的有BASEN模型[10]、NeuroHeed网络[11]和MSFNet模型[12],前两者采用时序卷积网络(temporal convolutional network,TCN)[13]和注意力机制的组合对语音序列进行有效建模,实现EEG与语音的时序对齐和特征交互,增强对目标说话人提取的针对性;后者采用双路径递归神经网络(dual-path recurrent neural network,DPRNN)[14]对语音序列进行降维,通过分块建模策略实现对时序特征的有效提取。但是,这些目标说话人提取方法仍受制于2个关键缺陷:首先,在局部精细特征提取方面,基于递归机制的建模过程易导致语音瞬时特征的过度平滑化,如瞬时变化、细节纹理等。原因在于时间步间的输出存在递归依赖机制,这会削弱高分辨率时间细节的保持能力,同时梯度传播过程中的衰减效应进一步限制了长时间上下文建模的有效性。其次,全局建模效率方面,Subakan等人[15]通过系统的实验研究发现,DPRNN用于建模全局关系的块间模块(inter-block)占模型整体近一半的参数量,但对整体性能提升的贡献非常有限。再者用于多模态特征交互的交叉注意力融合模块本身参数量也非常庞大,十分影响长序列语音处理的实时性。上述缺陷共同制约了语音波形端到端语音分离系统的性能,使得长序列语音信号处理成为当前技术的瓶颈。
为应对上述问题,本文在基线模型MSFNet基础上,提出了融合门控注意力与局部特征增强的神经导向说话人提取网络(gated attention and local feature enhancement neuro-orientedspeaker extraction network, GALENet),以解决当前研究方法面对长序列语音时无法在保证轻量化的前提下有效结合脑电信号进行建模提取特征的问题。该模型通过引入构建3个创新性模块来协同优化实现性能突破:门控机制确保跨模态信息的对齐融合,E-DPRNN提升对长序列语音的特征提取能力,SPGM模块优化全局信息建模的效率。为全面验证方法的有效性,本研究在3个多模态听觉注意解码数据集(cocktail-party,AVED,MM-AAD)上开展对比实验,并复现多个主流基线模型作为对照,系统地评估GALENet在神经导向说话人提取任务中的优势。实验结果表明,在这3个公开数据集上我们提出的模型在各项指标上均优于基线模型和其他方法,同时可训练参数量相较基线模型降低50%,为神经导向说话人提取技术的实用化提供了新的解决方案。
本文提出用于神经导向的目标说话人提取模型具体设计的3个核心创新点如下:1)门控交叉注意力机制,针对脑电信号与语音信号在时序动态和语义表征层面的异构性,提出双向跨模态交互范式。不同于传统线性投影的静态融合方式,该机制通过建立语音主导与脑电主导的双向注意力通路,实现跨模态特征的互补性重构。引入的动态权重分配机制突破固定比例融合的局限性,使网络能够依据语音场景复杂度自主调节模态贡献度,在抑制噪声干扰的同时强化目标说话人的声纹特征。2)E-DPRNN架构,针对语音信号短时瞬态特征易被递归网络平滑化的问题,创新性地将深度可分离卷积与扩展线性层相结合的Conv-U单元嵌入残差序列中。通过深度卷积路径捕获帧级别的声学细节,配合扩展线性层的非线性映射能力,形成局部-全局特征的双向增强机制,有效缓解梯度消失问题,使网络在长语音处理中仍能保持稳定的特征传播。3)单路径全局调制模块,针对DPRNN中对分离性能贡献较小的块间处理网络,提出使用单路径全局调制模块(single-path global modulation,SPGM)替代传统inter-BLSTM模块。SPGM模块由无参数的全局池化层与线性调制子模块构成,在确保整体语音分离性能提升的同时,也有效降低了块间信息在网络中建模传递的计算复杂度。
说话人提取技术[16]作为解决“鸡尾酒会”问题的关键第1步,旨在从复杂声学环境中的混合语音信号中分离出每个说话人的纯净语音,使每个声源都能够被独立处理[17]。目标说话人提取[18]方法通过引入辅助参考信息,仅提取目标说话人单一特定的语音通道,避免了传统语音分离任务——盲源分离(blind signal separation,BSS)中常见的通道排序歧义、需预先设定说话人数等问题,显著降低了系统复杂度,因此,成为近年来国内外语音处理领域的研究热点[19],受到广泛关注。而基于脑电信号,通过解码听者的EEG信号中的听觉注意模式、结合混合语音特征进行多模态融合[20-21]、提取听者关注的目标语音的此类方法则被称为神经导向说话人提取[22],融合了脑机接口(brain computer interface,BCI)与语音处理技术[23],是近年来兴起的前沿交叉研究方向,也是本文的研究内容。此类研究方向的理论依据源自于认知听觉神经科学的研究发现:当听者在特定环境中集中注意力时,其所关注的语音信号会在大脑中诱发出显著相关的神经活动模式[24-25]。
Han等人[26]最早提出将盲源分离与听觉注意解码相结合的策略,从脑电信号中估计目标语音包络,并与各分离声源进行匹配识别。然而,该方法需要先分离出全部声源,计算成本较高。为降低复杂度,Ceolini等人[27]提出了一种基于频域的脑启发说话人提取模型BISS,结合AAD进行说话人选择与语音提取,允许神经信号直接引导单通道语音分离过程。然而,BISS需分别训练指导网络与提取网络,并受限于频域方法固有的相位估计误差[28]。为克服上述不足,Hosseini等人[29-30]提出2种端到端时域脑控语音增强方法(BESD和UBESD),在EEG与混合语音之间引入特征层面线性调制(feature-wise linear modulation,FiLM)[31]进行多模态融合,从特征层面动态调整网络响应,从而更有效地提取目标说话人的特征表示。为进一步提升跨模态特征融合效果,Zhang等人[10]在BASEN模型中引入了卷积式多层交叉注意力模块(CMCA),基于TCN结构同时建模语音与EEG,显著改善分离语音的清晰度和可懂度。Pan等人[11]则提出了一种自回归说话人编码网络NeuroHeed,通过建模EEG与注意语音之间的时间关联,将历史语音动态纳入“听觉吸引子”,实现在线语音推理,模拟真实听觉过程中注意力的逐步累积。为解决神经导向语音增强中常见的说话人混淆问题,Pan等人[32]在后续工作中进一步优化了EEG编码器的判别能力,并增强了语音提取网络对EEG与混合语音中目标语音之间映射关系的建模能力,从而提升系统的准确性和实用性。为突破单尺度特征融合的局限性,Fan等人[12]提出的MSFNet创新性地构建多分支融合架构,在语音编码阶段并行采用不同时间窗长的卷积核,获取涵盖语音短时瞬态特征与长时全局模式的多尺度嵌入表示。通过多层交叉注意力模块将EEG空间特征动态注入各尺度语音表示中,并采用DPRNN时序建模增强跨模态对齐能力。相较于BASEN的固定尺度融合策略,该方法可以降低有效说话人混淆错误率的同时提高推理速度。
本文提出GALENet模型,对现有的神经导向说话人提取方法在多模态融合和捕捉语音的短时瞬态特征与局部声学细节存在的局限进行完善,通过实现EEG-语音的深度特征交互,优化网络架构在保证跨模态语义对齐与全局建模能力的同时,显著提升了特征表达效率与模型轻量化水平。
本文提出的GALENet模型在整体架构上借鉴了时域多模态说话人提取方法MSFNet的设计理念,保留了其高效的EEG编码器结构,并在单尺度网络框架下进行了进一步优化。GALENet主要由5个功能模块构成:语音编码器、EEG编码器、注意力融合模块、说话人提取网络和语音解码器。整体框架如图1所示。
给定一段多说话人混合语音信号$ \boldsymbol{x}(t) $及同步采集的EEG脑电信号$ \boldsymbol{e} $,GALENet的处理流程如下:
首先,语音编码器将时域混合语音信号$ \boldsymbol{x}(t) $编码为高维语音嵌入序列$ \boldsymbol{X}(t) $,以提取丰富的语音特征表示,可表示为
$ \boldsymbol{X}=SpeechEncoder\left(\boldsymbol{x}\right)\;。 $
EEG分支利用高效的EEG编码器,将所采集记录的EEG信号e编码为EEG嵌入向量E以提取脑信号模态的特征信息,将其表述为
$ \boldsymbol{E}=EEGEncoder\left(\boldsymbol{e}\right)\;。 $
分别在语音嵌入$ \boldsymbol{X} $与脑电嵌入$ \boldsymbol{E} $上应用交叉注意力(cross-attention)机制,实现模态间的信息交互与补充,构建潜在共享的表征空间。随后,引入门控注意力(gated attention)融合模块,根据语音与脑电模态的重要性而进行动态加权,最终得到融合特征$ \boldsymbol{Z} $,这个过程可表示为
$ \boldsymbol{Z}=\\ GatedAttention\left(CrossAttention\left(\boldsymbol{X},\boldsymbol{E}\right)\right)\;。 $
将融合特征$ \boldsymbol{Z} $输入所提出的E-DPRNN模块,并通过R次迭代,有效提取短时局部与长时全局的目标语音信息,学习到源特定掩码$ \boldsymbol{M} $,用于分离目标说话人的语音特征:
$ \boldsymbol{M}=Extractor\left(\boldsymbol{Z}\right)\;。 $
最后,通过源特定掩码$ \boldsymbol{M} $对原始语音嵌入$ \boldsymbol{X} $进行筛选,得到掩蔽的语音嵌入$ \hat{\boldsymbol{S}} $,以此重构时域语音波形,完成波形到波形预测任务,表示为
$ \hat{\boldsymbol{s}}=Decoder\left(\boldsymbol{X}\otimes \boldsymbol{M}\right)\;。 $
语音编码器由1个1维卷积层和1个ReLU激活函数构成,用于从原始时域语音信号中提取帧级别的高维特征。具体而言,对于输入的长时序语音样本$ \boldsymbol{x}(t)\in {\mathbb{R}}^{1\times T} $,编码器首先将其切分为长度为$ {T}_{\mathrm{t}} $的短时语音片段,并通过1维卷积操作将其转换为潜在空间中类似频谱的帧级嵌入序列表示$ \boldsymbol{X}(t) $,如式(6)所示。
$ \boldsymbol{X}\left(t\right)=\\ ReLU\left(conv1D\left(\boldsymbol{x}\left(t\right),1,{N}_{\mathrm{t}},L\right)\right)\in {\mathbb{R}}^{{{N}_{\mathrm{t}}}\times {{T}_{\mathrm{t}}}}\;, $
其中1维卷积的输入通道数量为1,输出通道数量为$ {N}_{\mathrm{t}} $,卷积核大小为$ L $,步长设为$ L/2 $,以保证帧间重叠。输出的$ \boldsymbol{X}(t) $为具有$ {N}_{\mathrm{t}} $维特征的$ {T}_{\mathrm{t}} $帧嵌入序列,用于表征语音的时频特性。为了降低计算复杂度并加深网络结构的深度,在后续处理过程中不直接使用$ \boldsymbol{X}\in {\mathbb{R}}^{{{N}_{\mathrm{t}}}\times {{T}_{\mathrm{t}}}} $,而是通过一个1×1的逐点卷积将通道数由$ {N}_{\mathrm{t}} $映射至更小的瓶颈特征维度$ C $,得到更紧凑的表征表示$ \boldsymbol{X}\in {\mathbb{R}}^{C\times {{T}_{\mathrm{t}}}} $,其中$ {N}_{\mathrm{t}}> L $。这种设计在保持语音重要特征的同时,有效降低了计算复杂度。
在说话人提取模块中,网络会生成1个与语音特征维度一致的掩码$ \boldsymbol{M}\in {\mathbb{R}}^{{{N}_{\mathrm{t}}}\times {{T}_{\mathrm{t}}}} $,用于从混合语音特征$ \boldsymbol{X} $中筛选出目标说话人的特征表示$ \hat{\boldsymbol{S}} $。最后,语音解码器执行编码器的逆操作,通过将与编码器结构对称的1维转置卷积层还原为时域上的目标语音波形信号$ \hat{\boldsymbol{s}}\in {\mathbb{R}}^{1\times T} $。该解码器的卷积核大小和步长仍设为$ L $和$ L/2 $,确保输出与原始输入的时间序列长度对齐:
$ \hat{\boldsymbol{s}}\left(t\right)=Deconv1D\left(\hat{\boldsymbol{S}}\left(t\right),{N}_{\mathrm{t}},1,L\right)\in {\mathbb{R}}^{1\times T}\;。 $
EEG编码器旨在从与目标语音时序对齐的多个通道EEG信号中提取高区分度嵌入表征。如图2所示,编码器前端采用3层图卷积网络(graph convolutional network,GCN)建模脑电通道间空间依赖关系[33]:将电极抽象为图节点,通过邻接矩阵定义通道拓扑连接关系,采用对称归一化拉普拉斯矩阵实现图卷积运算。GCN通过多层特征传播机制聚合相邻节点信息,利用切比雪夫多项式简化卷积核计算,有效捕获与听觉注意相关的空间模式。
经GCN提取的时空特征经通道层归一化后,通过1×1卷积进行初步维度压缩,随后输入到由3个残差块构成的深度特征提取模块。如图3所示,每个ResNet块包含2个1×1卷积层、1个卷积核大小为1×3的最大池化层、2个批归一化(batch normalization,BN)层和1个参数化修正线性单元PReLU(parametric ReLU)激活函数。2个1×1卷积层分别用于对输入特征进行升维和降维处理,用于匹配特征维度和减少模型参数,随后引入批归一化层进行归一化处理,稳定训练;1×3最大池化实现3倍时域下采样以强化关键信息并抑制冗余特征;跳跃连接通过元素相加融合浅层细节与深层语义,实现残差学习以缓解梯度消失问题。PReLU激活函数引入可学习参数提高网络的非线性变换的灵活性和特征表达能力。
最终,经过ResNet模块处理的特征将通过1个1×1卷积层映射到预设的固定维度,生成下采样后的EEG嵌入表示E。整个过程不仅实现了对脑电信号时序特征的有效压缩,也保留了重要的时域结构信息,为后续的目标说话人提取模块提供了高质量的脑电特征支撑。
在完成语音和EEG这2个单模态的独立特征提取之后,本文引入多头交叉注意力(multi-head cross attention,MCA)机制,以建模EEG模态和音频模态之间的语义依赖关系,并在跨模态的交互之后通过非线性变换进一步挖掘更具判别性的特征。多头交叉注意力结构由1个线性全连接层与多个注意力头组成,注意力头数量设为$ {A}_{\mathrm{S}} $。该模块的核心思想是以目标模态$ {{m}}_{1} $为主导,从源模态$ {{m}}_{2} $中引入需要对目标模态进行补充的信息,实现信息从$ {{m}}_{2} $到$ {{m}}_{1} $的跨模态流动与融合,得到信息增强的特征表示$ {\boldsymbol{Z}}_{{m_2}\rightarrow {m_1}} $,从而增强目标模态的特征表达能力。其具体如式(8)所示。
$ MCA\left({\boldsymbol{Z}}_{{m_1}},{\boldsymbol{Z}}_{{m_2}}\right)=\\ Concat\left({head}_{1},{head}_{2},… ,{head}_{{{A}_{\mathrm{S}}}}\right)\boldsymbol{W}\;。 $
每个头的注意力计算公式如式(9)(10)所示。
$ {head}_{i}=\\ S oftmax\left(\frac{\boldsymbol{Q}_{i}^{m_1}\times (\boldsymbol{K}_{i}^{{m}_{2}})^{\mathrm{T}}}{\sqrt{d}}\right)\boldsymbol{V}_{i}^{m_2},\;\;i=1,2,… ,{A}_{\mathrm{S}}\;, $
$ \begin{matrix}\left\{\begin{aligned}& \boldsymbol{Q}_{i}^{\mathrm{m}}={\boldsymbol{Z}}_{\mathrm{m}}\times \boldsymbol{W}_{i}^{\boldsymbol{Q}}\;,\\& \boldsymbol{K}_{i}^{\mathrm{m}}={\boldsymbol{Z}}_{\mathrm{m}}\times \boldsymbol{W}_{i}^{\boldsymbol{K}}\;,\\ &\boldsymbol{V}_{i}^{\mathrm{m}}={\boldsymbol{Z}}_{\mathrm{m}}\times \boldsymbol{W}_{i}^{\boldsymbol{V}}\;, \end{aligned}\right.\\ \end{matrix} $
其中$ \boldsymbol{W} $,$\boldsymbol{W}_{i}^{\boldsymbol{Q}} $,$\boldsymbol{W}_{i}^{\boldsymbol{K}} $,$\boldsymbol{W}_{i}^{\boldsymbol{V}} $均为可学习的参数矩阵。
本文中的MCA在2条特征路径上分别进行交互建模:
1)语音主导路径(EEG为辅助模态)。当目标模态为语音特征$ \boldsymbol{A} $且源模态为EEG特征$ \boldsymbol{E} $时,MCA模块的输出表示如式(11)所示。
$ {\boldsymbol{Z}}_{\boldsymbol{E}\rightarrow \boldsymbol{A}}=FC\left(MCA\left(\boldsymbol{A},\boldsymbol{E}\right)\right)+\boldsymbol{A}\;。 $
2)EEG主导路径(语音为辅助模态)。当目标模态为EEG特征$ \boldsymbol{E} $且源模态为语音特征$ \boldsymbol{A} $时,MCA模块的输出表示如式(12)所示。
$ {\boldsymbol{Z}}_{\boldsymbol{A}\rightarrow \boldsymbol{E}}=FC\left(MCA\left(\boldsymbol{A},\boldsymbol{E}\right)\right)+\boldsymbol{E}\;。 $
经过上述双路径交叉注意力机制后,分别得到2组初步融合的特征表示$ {\boldsymbol{Z}}_{\boldsymbol{E}\rightarrow \boldsymbol{A}} $与$ {\boldsymbol{Z}}_{\boldsymbol{A}\rightarrow \boldsymbol{E}} $,其中均融合了对方模态的上下文信息。
为了进一步提升模态融合的灵活性与有效性,本文创新型地引入了一种门控注意力(gated attention)机制,用于自适应地调整2个模态在最终融合中的权重。该机制首先将$ {\boldsymbol{Z}}_{\boldsymbol{E}\rightarrow \boldsymbol{A}} $和$ {\boldsymbol{Z}}_{\boldsymbol{A}\rightarrow \boldsymbol{E}} $进行拼接,并输入至线性变换层和Tanh激活函数,获得原始注意力得分,随后通过Softmax归一化生成融合权重,如式(13)所示。
$ \left[\alpha ,1-\alpha \right]=\\ S oftmax\left(Tanh\left(FC\left({\boldsymbol{Z}}_{\boldsymbol{E}\rightarrow \boldsymbol{A}},{\boldsymbol{Z}}_{\boldsymbol{A}\rightarrow \boldsymbol{E}}\right)\right)\right)\;。 $
最终,融合后的特征表示$ \boldsymbol{Z} $可由以下加权求和得到:
$ \boldsymbol{Z}=\alpha {\boldsymbol{Z}}_{\boldsymbol{E}\rightarrow \boldsymbol{A}}+\left(1-\alpha \right){\boldsymbol{Z}}_{\boldsymbol{A}\rightarrow \boldsymbol{E}}\;。 $
这种交叉注意力与门控机制的联合设计,突破了固定模态权重融合的限制,使网络能够根据任务需求动态调节各模态的贡献比例,增强关键信息的表达能力,有效抑制冗余特征,从而显著提升神经导向说话人提取的性能。
本文创新型地提出了一种基于E-DPRNN模块的新型说话人提取网络架构,旨在更高效地对语音信号进行时序建模与特征融合。该架构的核心思想是将长时间的语音序列划分为若干小块,并分别在块内(intra-chunk)和块间(inter-chunk)2个层面进行特征提取,从而兼顾局部信息的精细建模与全局语义的整合。
在块内特征提取阶段,采用了双分支结构以提升局部特征的表达能力。一条分支引入Conv-U模块,通过卷积操作捕捉每个块内的短时动态特征,增强局部结构信息的感知能力;另一条分支继承了DPRNN中的intra-BLSTM设计,利用双向LSTM模型挖掘更长时间跨度内的时序依赖,提升块内语义建模能力。2条路径的输出在特征维度上进行相加操作,以获得更精细化的块内特征表示。在块间特征提取阶段,网络同样采用双路径设计,以平衡建模能力与计算效率。一条路径继续使用Conv-U模块,用于捕捉语音信号在块与块之间的动态变化,强化全局时间结构的建模;另一条路径引入了轻量级的单路径全局调制块(single-path global modulation,SPGM)模块,通过低参数设计有效提取块间的全局语义信息,从而减少冗余计算并提升整体建模效率。通过上述设计,网络能够在局部与全局2个层次上实现对语音信号的多粒度建模,既保证了说话人特征的精细提取,又控制了模型的参数规模与计算复杂度,达成性能与效率的良好平衡。
为在保留BLSTM全局建模能力的基础上进一步增强局部特征的刻画能力,并缓解训练过程中的梯度消失或爆炸问题,本文在E-DPRNN框架中引入了Zhao等人[34-35]提出的Conv-U模块,如图4所示。该模块首先将输入序列经过层归一化(LayerNorm)层进行标准化处理,以提升特征分布的一致性;接着,通过带有扩展因子的线性变换层,结合SiLU(sigmoid linear unit)激活函数,实现对特征的非线性映射,从而提升模型的表达能力;随后,使用深度1维卷积(depthwise 1D convolution)提取序列中的短时局部模式,充分捕捉语音中的细粒度短时动态信息。同时为增强模型的鲁棒性与训练稳定性,模块引入跳跃连接机制并叠加Dropout层进行正则化,有效防止过拟合并加快收敛速度。
Subakan等人[36]针对Sepformer模型中跨块建模效率问题展开研究,发现inter-block结构虽占据模型近半参数量,但对性能提升贡献有限,表明全局信息建模需求可通过更高效机制实现。基于此,本文在GALENet模型中引入Yip等人[37]提出的SPGM模块,结构如图5所示,能通过全局池化与特征调制的方式有效建模块间信息,同时减少计算复杂度。该模块由全局池化模块和特征调制模块2个核心子模块组成。
全局池化模块采用层级压缩策略,首先通过最后元素选择法(last element selection,LE)进行块内池化(intra block pooling),直接选取各时间块末位时间步特征作为块级向量。由于E-DPRNN采用50%重叠分块策略,末位元素在相邻块间重复出现,可以将其重新用作全局向量的代表,无需额外的可训练参数即可实现有效建模。块池化将输入特征维度由$ [K,S,N] $压缩至$ [1,S,N] $,其中$ K $为块长,$ S $为块数,$ N $为特征维度。随后执行跨块池化(inter block pooling),沿块维度$ S $对各通道特征进行平均池化操作,最终生成一个维度大小为$ [1,1,N] $的全局向量。这个全局向量随后作为输入传递给SPGM的调制模块。该方法确保所有块在最终全局表示中具有相同的权重,使得全局向量能够综合所有块的信息,而不会偏向某个特定的块。
特征调制模块基于特征线性调制思想,将全局向量与局部特征动态融合。输入包含块内BLSTM提取的局部特征$ {\boldsymbol{x}}_{\mathrm{f}}\in {\mathbb{R}}^{K\times S\times N} $和通过全局池化模块生成的全局向量$ {\boldsymbol{x}}_{\text{ebm}}\in {\mathbb{R}}^{1\times 1\times N} $。模块设计双路径融合机制:1) 线性路径通过可训练矩阵$ {\boldsymbol{W}}_{\mathrm{g}}\in {\mathbb{R}}^{N\times N} $对$ {\boldsymbol{x}}_{\text{ebm}} $进行线性变换映射后,与$ {\boldsymbol{x}}_{\mathrm{f}} $执行逐元素相乘,实现特征通道的全局缩放;2)非线性路径利用$ {\boldsymbol{W}}_{\mathrm{s}}\in {\mathbb{R}}^{N\times N} $矩阵与Sigmoid函数生成门控权重,再与$ {\boldsymbol{x}}_{\mathrm{f}} $相乘,动态增强关键局部信息。2条路径输出经加和运算得到调制后特征$ {\boldsymbol{x}}_{0}\in {\mathbb{R}}^{R\times S\times N} $,作为后续处理的输入。
为了全面验证所提出的GALENet模型的有效性,本文在3个数据集Cocktail Party,AVED,MM-AAD上进行了广泛的对比实验。表1总结了所有实验中使用的数据集的设置和参数。
2018年Broderick等人[38]开展了一项基于鸡尾酒会问题的听觉注意机制研究。实验招募33名听力正常受试者(28名男性及5名女性,平均年龄27.3岁),采用128通道BioSemi ActiveTwo系统以512 Hz采样率采集EEG数据,每名完成30次60 s的试验。双耳听觉刺激为2名男性朗读的独立英文小说片段(Sennheiser HD650耳机播放,44.1 kHz采样率),要求受试者专注指定耳侧内容并忽略对侧语音。受试者分为2组,17人持续关注左耳,17人持续关注右耳(编号6被试因噪声干扰被剔除)。为确保试验连续性,各次试验的音频内容首尾衔接,避免故事重复。每次试验后通过4~6项选择题评估注意力集中程度。
实验采用严格参数控制,音频经幅度归一化处理至统一均方根(RMS)水平,静音间隔不超过0.5 s;要求受试者注视屏幕中央十字准线以抑制运动伪影。数据处理阶段,原始音频下采样至14.7 kHz并执行RMS归一化,混合信号作为神经网络输入。EEG信号经预处理后统一降采样至128 Hz。最终数据划分策略为:每位被试随机抽取5次试验构成测试集,其中2次作为验证集,其他用于训练集构建。
AVED数据集是由张宏玉等人[39]于2024年创建的新型脑电-音频数据集,该数据集专为听觉注意解码任务设计,所有受试者均签署知情同意书。实验招募20名听力正常受试者(14名男性和6名女性,平均年龄20岁),每名完成16次持续152 s的试验(含实验初始2 s静默时段)。听觉刺激选自中文短篇小说集,每次试验同步播放男女讲述者朗读的不同故事,要求受试者依据指令定向注意且确保关注故事无内容重叠。每次试验后通过3项多选题来验证注意力集中程度。音频信号统一设定为44.1 kHz采样率并保持相同响度水平。32通道脑电信号以1 kHz原始采样率采集后降采样至128 Hz。为避免注意力切换引起的实验不稳定性,仅选取关注男性讲述者的试验数据用于评估。数据划分采用75%训练集、12.5%验证集与12.5%测试集的标准化比例。
MM-AAD(multimodal auditory attention decoding)数据集是由Fan等人[40]于2025年提出的一项最新研究成果。该数据集包含来自50名正常听力、无神经系统疾病的在校大学生(34名男性和16名女性)的EEG数据。每名受试者参与了20次试验,每次试验持续约165 s,受试者平均年龄为22.4岁。所有实验均遵循伦理规范,确保受试者在知情同意的前提下自愿参与。
MM-AAD使用32通道EEG设备以2048 Hz的原始采样率进行记录,并将数据下采样至128 Hz,以提高处理效率。混合音频刺激信号的采样率为44.1 kHz,实验采用E-prime软件实现音频/音视频刺激与EEG采集设备之间的精确同步。与AVED数据集类似,MM-AAD同样设有音频刺激与音视频联合刺激2种试验条件。所有受试者均在这2种条件下完成了实验采集,而本研究中仅使用音频刺激条件下的数据进行分析与建模。
在实验过程中,受试者需在每个试验中交替将注意力集中于左右耳不同的说话人。考虑到注意目标频繁切换可能导致模型训练不稳定,为提升训练质量,本文仅选取受试者在注意男性说话人情境下的数据进行训练与评估,与AVED数据集的设置保持一致。最终,针对每名受试者选取的10个有效试验,按照如下比例划分数据集:2个试验用于测试集,1个用于验证集,其余7个用于训练集。
本研究为EEG信号构建了一套系统的预处理框架,旨在高效提取神经活动特征。针对Cocktail Party数据集,在进行0.1~45 Hz带通滤波并应用独立成分分析(ICA)技术[41]抑制眼动、肌电等生理伪影后,还引入了频带耦合模型[42]。该模型通过联合解析高频γ波段的功率谱与低频δ波段的相位动态,估算出与听觉皮层神经放电高度相关的多单元神经活动(MUA)特征,并以此作为模型的最终EEG输入,显著增强了信号与语音刺激的神经关联性。对于AVED数据集,预处理的重点是采用50 Hz陷波滤波器消除工频干扰,并结合ICA分离残留伪影。而对于MM-AAD数据集,则主要采用带通滤波和通道级Z-score归一化来提升数据的稳定性。
最后,为确保EEG与语音特征能在同一时间尺度上进行融合,研究对2类数据的采样率进行了调整:对语音信号进行降采样以降低计算复杂度,同时对EEG信号进行上采样,最终实现了多模态特征的精确时序对齐,为构建后续的神经解码模型奠定了坚实基础。
GALENet模型的实现基于PyTorch深度学习框架,构建环境为Python3.8.12,CUDA版本为11.3,并在NVIDIA 3090Ti GPU上完成训练任务。模型训练总共进行60个epoch,初始学习率设定为8E−4,批量大小为8。为提升训练效率与稳定性,采用线性预热与余弦退火策略对学习率进行动态调节。优化器选用Adam(adaptive moment estimation)优化算法[43],并使用负SI-SDR(scale-invariant signal-to-distortion ratio)作为损失函数,用于衡量语音重构质量并进行反向传播优化。在多数据集适配方面,不同EEG数据集对应的编码器输入通道数略有差异,Cocktail Party数据集使用128通道EEG输入;而AVED与MM-AAD数据集则使用32通道EEG输入。为缓解不同数据集带来的过拟合风险以及抑制训练过程中的梯度爆炸问题,实验设置了针对性权重衰减参数:AVED数据集训练时设定权重衰减为1E−2,MM-AAD数据集则设为5E−3,其余参数保持一致。
模型结构参数详见表2,其中,超参数包括卷积核大小$ {K}_{\mathrm{s}} $、卷积步长$ {S}_{\mathrm{t}} $、多头交叉注意力机制中的注意头数量$ {A}_{\mathrm{s}} $、E-DPRNN模块中的分段长度$ K $与块数量$ S $。最终模型通过合理的训练配置、有效的优化策略以及对不同数据集的适配性设计,在保障计算效率的同时,有效提升了神经导向说话人提取任务的整体性能表现。
为全面评估所提出的GALENet方法在神经导向说话人提取任务中的性能表现,本研究选取了目前神经导向说话人提取的主流基线模型进行对比实验,涵盖BESD,UBESD,BASEN,MSFNet。
所有模型均在Cocktail Party,AVED,MM-AAD这3个数据集上进行训练与测试,性能评估主要基于测试集中所有受试者语音片段的SI-SDR(中位数)指标。同时,为验证模型的轻量化效果,还对各方法的参数量进行了对比分析,以检验GALENet是否能在保持或提升性能的同时显著降低模型复杂度。特别说明,为了保证实验结果的公平性与可比性,表3~5中类似BASEN*的形式的结果均为本研究在统一配置下的复现结果。其中,Cocktail Party数据集采用未知目标说话人提取设定,以更贴近真实应用场景;而AVED与MM-AAD数据集由于实验范式相似,均在已知目标说话人提取条件下进行训练与推理,从而确保结果的可重复性与稳定性。
在神经导向说话人提取任务中,性能评估指标主要包括主观评价与客观评价两大类,其中客观评价可通过公式算法实现快速、可重复的定量分析,已广泛应用于本领域研究当中。常用的客观指标包括源信号失真比(signal-to-distortion ratio,SDR),其通过衡量目标语音与失真成分之间的能量比来评估分离质量,虽效果直观但对噪声较敏感。尺度不变信号失真比(scale-invariant SDR,SI-SDR)在SDR基础上引入尺度因子,增强了鲁棒性与稳定性,是当前语音增强任务中的主流评估方式。短时客观可懂度(short-time objective intelligibility,STOI)指标通过分析参考语音与预测语音之间短时频谱包络的相关性来量化语音可懂度,得分范围为[0, 1]。为进一步提升在噪声环境中的评估精度,扩展短时客观可懂度(extended short-time objective intelligibility,ESTOI)在建模过程中融入了听觉感知的非线性特性。语音质量感知评估指标(perceptual evaluation of speech quality,PESQ)则模拟人类听觉系统对语音质量的感知过程,输出与平均意见得分(mean opinion score,MOS)相关的量化结果,广泛用于评估语音增强与分离系统的感知质量表现。上述指标共同构成了神经导向语音分离系统性能评估的核心工具体系。
在模拟未知目标说话人提取的复杂语音环境下,如表3所示,GALENet在主要性能指标上均取得最佳结果,SI-SDR达到13.59 dB,STOI为0.89,PESQ为2.54,相较当前最优的MSFNet分别提升了0.7 dB,0.01 dB,0.03 dB,验证GALENet在多模态融合和目标语音建模方面的有效性。同时,在参数量方面,GALENet也展现出良好的轻量化优势。与BASEN模型相比,GALENet在SI-SDR上实现了2.03 dB的显著提升,而模型参数仅小幅增加0.12×106。图6总结了MSFNet模型和GALENet模型在测试受试者上的SI-SDR,STOI,PESQ评估结果。可以清晰地看到,所提出的GALENet在所有评估指标和全部受试者中均优于MSFNet方法。另外,这两者的性能在不同受试者之间表现出高度的一致性,这充分表明DPRNN对于长序列语音特征提取的有效性和在听觉动态变化下卓越的鲁棒能力。进一步对比MSFNet,尽管GALENet性能提升幅度较小,但模型参数量从1.53×106减少至0.77×106,降幅近50%,显示出在性能与复杂度之间达成了更优的平衡。
在已知目标说话人提取的实验设定下,本研究进一步在AVED和MM-AAD这2个数据集上进行了对比实验,以全面评估GALENet方法在不同任务条件下的泛化能力与鲁棒性。由于EEG输入通道数的差异,GALENet在Cocktail Party与AVED数据集之间的模型参数量存在一定差异,但整体性能表现和趋势保持一致。如表4所示,在AVED数据集上,GALENet在仅使用0.63×106参数量的条件下,分别在SI-SDR和SDR这2项指标上相较MSFNet模型取得了0.79 dB和0.76 dB的性能提升,进一步凸显了GALENet在提升语音质量和减少语音失真的能力。虽然BASEN模型在参数量上与GALENet相近,但GALENet在SI-SDR指标上实现了约23.4%的性能提升,验证了其更优的说话人提取能力。
在MM-AAD数据集上的评估结果也同样印证了GALENet的卓越表现。如表5所示,GALENet在主要评估指标上均取得当前最优值:SI-SDR达到10.17 dB,STOI为0.91,PESQ为2.19。特别是在SI-SDR指标上,相较于BASEN方法提升了约26.5%,并且相比MSFNet方法进一步提升了6.3%。同时,GALENet的模型参数量较MSFNet减少了0.79×106,且与BASEN模型保持在同一数量级,展现出在保持高性能的前提下显著压缩模型规模的能力。
为了进一步评估模型在实际部署场景中的可行性,本研究还使用PyTorch-OpCounter工具对训练过程中的乘积累加运算量(multiply-accumulate operations,MAC)进行了统计。结果表明,GALENet的计算量为47.89 GMAC,相比于MSFNet的89.58 GMAC,计算复杂度减少近一半,表明GALENet不仅在性能上优于现有方法,也具备更强的资源效率与实际应用潜力。
为系统评估GALENet模型中各关键组成部分(包括输入模态及关键模块)的有效性,本节设计了多组消融实验,并基于Cocktail Party数据集开展对比分析,实验结果详见表6和表7。
首先,我们分析了独立模态的作用,测试了仅使用语音(speech-only)的提取效果。实验结果显示,模型性能发生显著下降,实验结果详见表6,其SI-SDR指标仅为0.47 dB,PESQ评分也低至1.61。该结果与融合EEG信号后的GALENet性能形成鲜明对比,这有力地证实了EEG作为神经引导信号在本研究中的关键指导作用,凸显了多模态融合的绝对必要性。
其次,为验证SPGM和Conv-U这2个模块的有效性,我们构建了一个移除2者的基线模型(DPRNN, inter-BLSTM),其SI-SDR为13.49 dB,参数量大小为1.03×106。在此基础上,仅引入SPGM来替换Inter-BLSTM,参数量显著降至0.72×106,有效验证了SPGM模块通过全局池化和轻量调制机制在极低参数开销下仍能维持较强的块间建模能力。最后,在已引入SPGM的基础上,加入Conv-U模块构成完整GALENet,SI-SDR与SDR指标提升至整体最优,PESQ升至2.54,而模型参数量仅小幅增加至0.77×106。表明Conv-U模块能有效加强语音局部特征的建模能力,并在不显著增加模型复杂度的前提下,提升说话人提取的整体性能。
表8展示了在4种不同信息融合策略下,GALENet系统的性能对比结果,旨在评估不同模态融合机制对神经导向说话人提取效果的影响。所比较的方法包括:元素相加Add方法(GALENet#1)、特征拼接Concat方法(GALENet#2)、多层交叉注意力CMCA方法(GALENet#3)以及本文提出的门控交叉注意力机制(MCA+GA)。其中,Add的具体实现是通过线性变换调整EEG嵌入特征维度,使其与语音特征的维度匹配后进行加法融合,保留每种模态特征信息的同时,减少维度不一致带来的问题。Concat的具体实现是将语音特征与EEG特征直接拼接,形成更丰富的特征表示。从实验结果来看,Add和Concat方法虽结构简单、计算开销低,但未能有效建模语音与EEG模态间的复杂依赖关系,导致语音分离性能受限。相比之下,CMCA方法通过引入多层交叉注意力机制,能够更深入地挖掘跨模态特征之间的相互作用关系,显著提升系统性能。但这一方法的代价在于其引入了大量参数和计算负担,影响模型的整体效率与可部署性。为兼顾性能与效率,本研究提出的MCA+GA融合方式在结构上采用单层多头交叉注意力以实现特征交互,同时通过引入门控机制自适应地调节模态间的融合权重。实验结果表明,MCA+GA方法在参数量仅为0.77×106的情况下,超越其他方法取得最优性能,不仅有效压缩了模型规模,还充分保留了跨模态深层特征的表达能力。
在本文中我们提出了一种新型端到端时域神经导向说话人提取模型——GALENet,通过整合时域神经导向技术与EEG信号的深度解码,成功地解决了传统语音分离方法在复杂环境中面临的挑战。创新型地引入门控交叉注意力融合机制,模型能够在语音和EEG信号之间进行高效的信息交互,从而提升了目标语音的提取精度。同时,模型设计中创新性地引入和设计局部特征来增强模块和E-DPRNN网络结构,进一步提升了对说话人特征的建模能力。在多个基准数据集上的实验结果表明,GALENet在神经导向说话人提取任务中展现出强大的优势,特别是在处理多说话人场景时。实验结果也表明,GALENet在性能与效率之间找到了良好的平衡,其SI-SDR和PESQ等指标在多个数据集上均超越了基线MSFNet模型和其他各种先进的对比模型,验证了该模型的广泛适用性和潜在的工业应用价值。在未来的工作中我们将进一步优化模型结构,探索更高效的模态信息和跨模态特征融合方法,并考察不同语言、不同导联数等条件对模型性能的影响,并在实际应用中实现更低的延迟和更高的实时处理能力。这些改进有望为助听器、语音增强设备以及多模态智能助手等应用提供技术支持,从而推动个性化听觉系统的发展,并为听力受损群体提供更好的沟通支持。
作者贡献声明:范存航提出了模型思路和实验方案;李骏凌开展实验、实验分析并撰写和修改论文;张晶晶负责补充实验和实验分析;高佑典、陈颖、易江燕提出指导意见并修改论文;吕钊参与论文审阅并提出修改意见。

参考文献 引证文献
排序方式:
1
Bronkhorst A W. The cocktail party phenomenon: A review of research on speech intelligibility in multiple-talker conditions[J]. Acta Acustica United with Acustica, 2000, 86(1): 117−128
2
Cherry E C. Some experiments on the recognition of speech, with one and with two ears[J]. Journal of the Acoustical Society of America, 1953, 25: 975−979
3
Zhao Yan, Wang Deliang, Johnson E M, et al. A deep learning based segregation algorithm to increase speech intelligibility for hearing-impaired listeners in reverberant-noisy conditions[J]. The Journal of the Acoustical Society of America, 2018, 144(3): 1627−1637
4
刘宇宸, 宗成庆. 跨模态信息融合的端到端语音翻译[J]. 软件学报, 2023, 34(4): 1837−1849
Liu Yuchen, Zong Chengqing. End-to-end speech translation by integrating cross-modal information[J]. Journal of Software, 2023, 34(4): 1837−1849 (in Chinese)
5
韩松莘, 郭松辉, 徐开勇, 等. 基于帧结构的语音对抗样本重点区域扰动分析[J]. 计算机研究与发展, 2024, 61(3): 685−700
Han Songshen, Guo Songhui, Xu Kaiyong, et al. Perturbation analysis of the vital region in speech adversarial example based on frame structure[J]. Journal of Computer Research and Development, 2024, 61(3): 685−700 (in Chinese)
6
Chen Xiaoyu, Du Changde, Zhou Qiongyi, et al. Auditory attention decoding with task-related multi-view contrastive learning[C]//Proc of the 31st ACM Int Conf on Multimedia. New York: ACM, 2023: 6025−6033
7
Fan Cunhang, Zhang Hongyu, Huang Wei, et al. DGSD: Dynamical graph self-distillation for EEG-based auditory spatial attention detection[J]. Neural Networks, 2024, 179: 106580
8
O’Sullivan J, Chen Zhuo, Herrero J, et al. Neural decoding of attentional selection in multi-speaker environments without access to clean sources[J]. Journal of Neural Engineering, 2017, 14(5): 056001
9
Cai Siqi, Zhang Ran, Li Haizhou. Robust decoding of the auditory attention from EEG recordings through graph convolutional networks[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 2320−2324
10
Zhang Jie, Xu Qingtian, Zhu Qiushi, et al. BASEN: Time-domain brain-assisted speech enhancement network with convolutional cross attention in multi-talker conditions[C]//Proc of the 24th Interspeech. Kos Island, GR: ISCA, 2023: 3117−3121
11
Pan Zexu, Borsdorf M, Cai Siqi, et al. NeuroHeed: Neuro-steered speaker extraction using EEG signals[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 4456−4470
12
Fan Cunhang, Zhang Jingjing, Zhang Hongyu, et al. MSFNet: Multi-scale fusion network for brain-controlled speaker extraction[C]//Proc of the 32nd ACM Int Conf on Multimedia. New York: ACM, 2024: 1652−1661
13
Zeghidour N, Grangier D. Wavesplit: End-to-end speech separation by speaker clustering[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 2840−2849
14
Luo Yi, Chen Zhuo, Yoshioka T. Dual-path RNN: Efficient long sequence modeling for time-domain single-channel speech separation[C]//Proc of the 45th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2020: 46−50
15
Subakan C, Ravanelli M, Cornell S, et al. Exploring self-attention mechanisms for speech separation[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023, 31: 2169−2180
16
黄雅婷, 石晶, 许家铭, 等. 鸡尾酒会问题与相关听觉模型的研究现状与展望[J]. 自动化学报, 2019, 45(2): 234−251
Huang Yating, Shi Jing, Xu Jiaming, et al. Research advances and perspectives on the cocktail party problem and related auditory models[J]. Acat Automatiac Sinica, 2019, 45(2): 234−251 (in Chinese)
17
黄翔东, 陈红红, 甘霖. 基于频率-时间扩张密集网络的语音增强方法[J]. 计算机研究与发展, 2023, 60(7): 1628−1638
Huang Xiangdong, Chen Honghong, Gan Lin. Speech enhancement method based on frequency-time dilated dense network[J]. Journal of Computer Research and Development, 2023, 60(7): 1628−1638 (in Chinese)
18
Zmolikova K, Delcroix M, Ochiai T, et al. Neural target speech extraction: An overview[J]. IEEE Signal Processing Magazine, 2023, 40(3): 8−29
19
Kong Yonghui, Fan Cunhang, Chen Yujie, et al. Bilateral masking with prompt for knowledge graph completion[C]//Proc of Findings of the 19th Association for Computational Linguistics (NAACL). Stroudsburg, PA: ACL, 2024: 240−249
20
李泽宇, 张旭鸿, 蒲誉文, 等. 多模态深度伪造及检测技术综述[J]. 计算机研究与发展, 2023, 60(6): 1396−1416
Li Zeyu, Zhang Xuhong, Pu Yuwen, et al. A survey on multimodal deepfake and detection techniques[J]. Journal of Computer Research and Development, 2023, 60(6): 1396−1416 (in Chinese)
21
Fan Cunhang, Gao Youdian, Pan Zexu, et al. Improved feature extraction network for neuro-oriented target speaker extraction[C]//Proc of the 50th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2025: 1−5
22
邱泽林, 姚鼎鼎, 李军锋. 融合双视角特征的两阶段脑控语音增强[J]. 声学学报, 2025, 50(2): 362−372
Qiu Zelin, Yao Dingding, Li Junfeng. Two-stage brain-controlled speech enhancement with integrated dual-view features[J]. Acat Acustica, 2025, 50(2): 362−372 (in Chinese)
23
Fan Cunhang, Zhang Sheng, Zhang Jingjing, et al. SSM2Mel: State space model to reconstruct mel spectrogram from the EEG[C]//Proc of the 50th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2025: 1−5
24
Chen Yujie, Yi Jiangyan, Fan Cunhang, et al. Region-based optimization in continual learning for audio deepfake detection[C]//Proc of the 39th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2025: 23651−23659
25
Fan Cunhang, Chen Yujie, Xue Jun, et al. Progressive distillation based on masked generation feature method for knowledge graph completion[C]//Proc of the 38th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2024: 8380−8388
26
Han Cong, O’Sullivan J, Luo Yi, et al. Speaker-independent auditory attention decoding without access to clean speech sources[J]. Science Advances, 2019, 5(5): eaav6134
27
Ceolini E, Hjortkjær J, Wong D D E, et al. Brain-informed speech separation (BISS) for enhancement of target speaker in multitalker speech perception[J]. NeuroImage, 2020, 223: 117282
28
Reju V G, Koh S N, Soon Y. Convolution using discrete sine and cosine transforms[J]. IEEE Signal Processing Letters, 2007, 14(7): 445−448
29
Hosseini M, Celotti L, Plourde É. Speaker-independent brain enhanced speech denoising[C]//Proc of the 46th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2021: 1310−1314
30
Hosseini M, Celotti L, Plourde E. End-to-end brain-driven speech enhancement in multi-talker conditions[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022, 30: 1718−1733
31
Perez E, Strub F, De Vries H, et al. Film: Visual reasoning with a general conditioning layer[C]//Proc of the 32nd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2018: 1
32
Pan Z, Wichern G, Germain F G, et al. NeuroHeed+: Improving neuro-steered speaker extraction with joint auditory attention detection[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 11456−11460
33
祝传振, 王璇, 郑强. 基于元素分离与整体注意力的图卷积网络框架[J]. 计算机研究与发展, 2024, 61(8): 2008−2019
Zhu Chuanzhen, Wang Xuan, Zheng Qiang. Element separation and holistic attention based graph convolutional network framework[J]. Journal of Computer Research and Development, 2024, 61(8): 2008−2019 (in Chinese)
34
Zhao Shengkui, Ma Bin. Mossformer: Pushing the performance limit of monaural speech separation using gated single-head transformer with convolution-augmented joint self-attentions[C]//Proc of the 48th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2023: 1−5
35
Zhao Shengkui, Ma Yukun, Ni Chongjia, et al. Mossformer2: Combining transformer and rnn-free recurrent network for enhanced time-domain monaural speech separation[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 10356−10360
36
Subakan C, Ravanelli M, Cornell S, et al. Attention is all you need in speech separation[C]//Proc of the 46th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2021: 21−25
37
Yip J Q, Zhao Shengkui, Ma Yukun, et al. SPGM: Prioritizing local features for enhanced speech separation performance[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE: 326−330
38
Broderick M P, Anderson A J, Di Liberto G M, et al. Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narrative speech[J]. Current Biology, 2018, 28(5): 803−809
39
张宏玉, 张晶晶, 董兴广, 等. 基于视听诱发的听觉注意检测脑电数据集[J]. 清华大学学报: 自然科学版, 2024, 64(11): 1919−1926
Zhang Hongyu, Zhang Jingjing, Dong Xingguang, et al. Based on audio-video evoked auditory attention detection electroencephalogram dataset[J]. Journal of Tsinghua University: Science and Technology, 2024, 64(11): 1919−1926 (in Chinese)
40
Fan Cunhang, Zhang Hongyu, Ni Qinke, et al. Seeing helps hearing: A multi-modal dataset and a mamba-based dual branch parallel network for auditory attention decoding[J]. Information Fusion, 2025, 118: 102946
41
Delorme A, Makeig S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis[J]. Journal of Neuroscience Methods, 2004, 134(1): 9−21
42
Moinnereau M A, Rouat J, Whittingstall K, et al. A frequency-band coupling model of EEG signals can capture features from an input audio stimulus[J]. Hearing Research, 2020, 393: 107994
43
Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint, arXiv: 1412.6980, 2014
2026年第63卷第6期
84
38
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202550467
  • 接收时间:2025-06-09
  • 首发时间:2026-11-17
  • 出版时间:2026-06-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2025-06-09
  • 修回日期:2026-01-16
  • 录用日期:2026-01-29
基金
作者信息
    1光电信息获取与防护技术全国重点实验室(安徽大学) 合肥 230601
    2清华大学自动化系 北京 100084

通讯作者:

吕钊()
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550467
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏