Article(id=1329455905067872752, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329455653363495339, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202330715, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1694361600000, receivedDateStr=2023-09-11, revisedDate=1701446400000, revisedDateStr=2023-12-02, acceptedDate=null, acceptedDateStr=null, onlineDate=1794896587863, onlineDateStr=2026-11-17, pubDate=1706803200000, pubDateStr=2024-02-02, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794896587863, onlineIssueDateStr=2026-11-17, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794896587863, creator=13701087609, updateTime=1794896587863, updator=13701087609, issue=Issue{id=1329455653363495339, tenantId=1146029695717560320, journalId=1324759914146529478, year='2024', volume='61', issue='2', pageStart='273', pageEnd='538', issueExtLink='null', onlineDate='null', pubDate='1706803200000', pubDateStr='2024-02-02', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794896527853, creator='13701087609', updateTime=1794896527853, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext=null, issueFiles=null, downloadFileDto=null}, startPage=291, endPage=306, ext={EN=ArticleExt(id=1329455905315336689, articleId=1329455905067872752, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=CodeScore-R: An Automated Robustness Metric for Assessing the Functional Correctness of Code Synthesis, columnId=1329455654147830188, journalTitle=Journal of Computer Research and Development, columnName=Special Issue on Generative AI and Software Automation, runingTitle=null, highlight=null, articleAbstract=
Evaluation metrics are crucial in the field of code synthesis. Commonly used code evaluation metrics can be classified into three types: match-based, semantic-based, and execution-based. Among them, the execution-based Pass@k metric accurately assesses the functionality of predicted code by executing test cases. However, calculating this metric requires a significant amount of overhead, necessitating the design of an automated evaluation metric that can assess the functionality of predicted code without the need for test cases. Additionally, a good evaluation metric should be robust, that is the metric can maintain its accuracy even when the predicted code undergoes minor changes. To address these challenges, we propose an automated robust metric, called CodeScore-R, based on UniXcoder and contrastive learning, for evaluating the functionality of code synthesis. CodeScore-R employs techniques such as sketch-based processing, syntactic-equivalent transformations, and mutation testing to effectively mitigate the interference caused by identifiers, syntax structures, and operators on evaluation results. Experimental results demonstrate that in the tasks of code generation and migration in Java and Python, CodeScore-R outperforms other evaluation metrics and is more closely aligned with the Pass@k metric, while exhibiting stronger robustness.
, authors=Guang Yang
1, Yu Zhou
1, *, Xiang Chen
2, Xiangyu Zhang
1, authorsList=Guang Yang, Yu Zhou, Xiang Chen, Xiangyu Zhang, authorCompany=null, correspAuthors=Yu Zhou, authorNote=
born in 1997. PhD candidate. Student Member of CCF. His main research interests include code generation, code translation, and robustness research in code synthesis
born in 1981. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include software evolution analysis, mining software repositories, software architecture, and reliability analysis
born in 1980. PhD, associate professor. Member of CCF. His main research interests include software defect prediction, software defect location, regression testing, and combination testing
born in 1999. Master candidate. His main research interests include code generation and code pre-trained model
, correspAuthorsNote=null, copyrightStatement=Copyright © 2024 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1329455909304119814, articleId=1329455905067872752, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=CodeScore-R:用于评估代码合成功能准确性的自动化鲁棒指标, columnId=1329455654328185262, journalTitle=计算机研究与发展, columnName=生成式AI与软件自动化专题, runingTitle=null, highlight=null, articleAbstract=
评估指标在代码合成领域中至关重要. 常用的代码评估指标可以分为3种类型:基于匹配、基于语义和基于执行. 其中,基于执行的Pass@k指标通过执行测试用例,能够准确判断预测代码的功能准确性. 然而,该指标的计算需要大量开销,因此亟需设计一种自动化评估指标,在无需测试用例时仍可评估预测代码的功能准确性. 此外,好的评估指标应当具有鲁棒性,即预测代码发生微小改变时,评估指标仍能保持其准确性. 为此,提出了一种基于UniXcoder和对比学习的自动化鲁棒指标CodeScore-R,用于评估代码合成的功能准确性. CodeScore-R采用草图化处理、语法等价转换和变异测试等技术手段,有效减轻了标识符、语法结构和运算符对评估结果的干扰. 实验结果表明,在Java和Python语言上的代码生成和迁移任务中,CodeScore-R的表现优于其他无需测试用例的评估指标,且更接近Pass@k指标,并具有更强的鲁棒性.
, authors=杨光
1, 周宇
1, *, 陈翔
2, 张翔宇
1, authorsList=杨光, 周宇, 陈翔, 张翔宇, authorCompany=null, correspAuthors=周宇, authorNote=
 |
1997年生,博士研究生.CCF学生会员. 主要研究方向为代码生成、代码迁移、代码合成中的鲁棒性研究 |
 |
1981年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为软件演化分析、挖掘软件仓库、软件架构、可靠性分析 |
 |
1980年生. 博士,副教授. CCF会员. 主要研究方向为软件缺陷预测、软件缺陷定位、回归测试、组合测试 |
 |
1999年生. 硕士研究生. 主要研究方向为代码生成、代码预训练模型 |
, correspAuthorsNote=
, copyrightStatement=版权所有 © 2024 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=qB2UkE6AxCKY0zWOazKpxQ==, magXml=7HLM99xNGvrwR+qdYcUHxQ==, pdfUrl=null, pdf=HOe49R0qLKJzl4anv7Tq5A==, pdfFileSize=2494981, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=GxECN8L9k95Enwcj6f9pBw==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=77oHNYVoWjObYAmrBW2uxA==, mapNumber=null, fund=null)}, authors=[Author(id=1329455909694190094, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=yang.guang@nuaa.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329455909752910352, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, authorId=1329455909694190094, language=EN, stringName=Guang Yang, firstName=Guang, middleName=null, lastName=Yang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1College of Computer Science and Technology/College of Artificial Intelligence/College of Software, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, bio={"content":"
born in 1997. PhD candidate. Student Member of CCF. His main research interests include code generation, code translation, and robustness research in code synthesis
"}, bioImg=null, bioContent=
born in 1997. PhD candidate. Student Member of CCF. His main research interests include code generation, code translation, and robustness research in code synthesis
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329455909815824913, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, authorId=1329455909694190094, language=CN, stringName=杨光, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1南京航空航天大学计算机科学与技术学院/人工智能学院/软件学院 南京 211106, bio={"img":"hTaOpXwIaigQWB2WlwsylA==","content":"
1997年生,博士研究生.CCF学生会员. 主要研究方向为代码生成、代码迁移、代码合成中的鲁棒性研究
"}, bioImg=hTaOpXwIaigQWB2WlwsylA==, bioContent=
1997年生,博士研究生.CCF学生会员. 主要研究方向为代码生成、代码迁移、代码合成中的鲁棒性研究
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329455909505446407, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, xref=1, ext=[AuthorCompanyExt(id=1329455909518029320, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909505446407, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1College of Computer Science and Technology/College of Artificial Intelligence/College of Software, Nanjing University of Aeronautics and Astronautics, Nanjing 211106), AuthorCompanyExt(id=1329455909526417929, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909505446407, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1南京航空航天大学计算机科学与技术学院/人工智能学院/软件学院 南京 211106)])]), Author(id=1329455909870350867, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhouyu@nuaa.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329455909937459733, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, authorId=1329455909870350867, language=EN, stringName=Yu Zhou, firstName=Yu, middleName=null, lastName=Zhou, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, *, address=
1College of Computer Science and Technology/College of Artificial Intelligence/College of Software, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, bio={"content":"
born in 1981. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include software evolution analysis, mining software repositories, software architecture, and reliability analysis
"}, bioImg=null, bioContent=
born in 1981. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include software evolution analysis, mining software repositories, software architecture, and reliability analysis
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329455910000374294, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, authorId=1329455909870350867, language=CN, stringName=周宇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, *, address=
1南京航空航天大学计算机科学与技术学院/人工智能学院/软件学院 南京 211106, bio={"img":"ZTQ2scsjnI5M7Gx4hWh1YA==","content":"
1981年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为软件演化分析、挖掘软件仓库、软件架构、可靠性分析
"}, bioImg=ZTQ2scsjnI5M7Gx4hWh1YA==, bioContent=
1981年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为软件演化分析、挖掘软件仓库、软件架构、可靠性分析
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329455909505446407, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, xref=1, ext=[AuthorCompanyExt(id=1329455909518029320, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909505446407, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1College of Computer Science and Technology/College of Artificial Intelligence/College of Software, Nanjing University of Aeronautics and Astronautics, Nanjing 211106), AuthorCompanyExt(id=1329455909526417929, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909505446407, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1南京航空航天大学计算机科学与技术学院/人工智能学院/软件学院 南京 211106)])]), Author(id=1329455910067483160, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329455910147174938, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, authorId=1329455910067483160, language=EN, stringName=Xiang Chen, firstName=Xiang, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
2, address=
2School of Information Science and Technology, Nantong University, Nantong 226019, bio={"content":"
born in 1980. PhD, associate professor. Member of CCF. His main research interests include software defect prediction, software defect location, regression testing, and combination testing
"}, bioImg=null, bioContent=
born in 1980. PhD, associate professor. Member of CCF. His main research interests include software defect prediction, software defect location, regression testing, and combination testing
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329455910222672411, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, authorId=1329455910067483160, language=CN, stringName=陈翔, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
2, address=
2南通大学信息科学技术学院 江苏南通 226019, bio={"img":"h0uP8aCYKrwOP2ZxjzqIEg==","content":"
1980年生. 博士,副教授. CCF会员. 主要研究方向为软件缺陷预测、软件缺陷定位、回归测试、组合测试
"}, bioImg=h0uP8aCYKrwOP2ZxjzqIEg==, bioContent=
1980年生. 博士,副教授. CCF会员. 主要研究方向为软件缺陷预测、软件缺陷定位、回归测试、组合测试
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329455909601915402, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, xref=2, ext=[AuthorCompanyExt(id=1329455909610304011, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909601915402, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
2School of Information Science and Technology, Nantong University, Nantong 226019), AuthorCompanyExt(id=1329455909614498316, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909601915402, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
2南通大学信息科学技术学院 江苏南通 226019)])]), Author(id=1329455910293975581, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329455910369473055, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, authorId=1329455910293975581, language=EN, stringName=Xiangyu Zhang, firstName=Xiangyu, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1College of Computer Science and Technology/College of Artificial Intelligence/College of Software, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, bio={"content":"
born in 1999. Master candidate. His main research interests include code generation and code pre-trained model
"}, bioImg=null, bioContent=
born in 1999. Master candidate. His main research interests include code generation and code pre-trained model
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329455910428193312, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, authorId=1329455910293975581, language=CN, stringName=张翔宇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1南京航空航天大学计算机科学与技术学院/人工智能学院/软件学院 南京 211106, bio={"img":"Y10r9fosbnliLKdhZslTmw==","content":"
1999年生. 硕士研究生. 主要研究方向为代码生成、代码预训练模型
"}, bioImg=Y10r9fosbnliLKdhZslTmw==, bioContent=
1999年生. 硕士研究生. 主要研究方向为代码生成、代码预训练模型
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329455909505446407, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, xref=1, ext=[AuthorCompanyExt(id=1329455909518029320, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909505446407, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1College of Computer Science and Technology/College of Artificial Intelligence/College of Software, Nanjing University of Aeronautics and Astronautics, Nanjing 211106), AuthorCompanyExt(id=1329455909526417929, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909505446407, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1南京航空航天大学计算机科学与技术学院/人工智能学院/软件学院 南京 211106)])])], keywords=[Keyword(id=1329455910533050913, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, orderNo=1, keyword=code synthesis evaluation metric), Keyword(id=1329455910591771170, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, orderNo=2, keyword=functional correctness), Keyword(id=1329455910663074339, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, orderNo=3, keyword=robustness), Keyword(id=1329455910730183204, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, orderNo=4, keyword=code synthesis), Keyword(id=1329455910797292069, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, orderNo=5, keyword=neural network), Keyword(id=1329455912361767462, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, orderNo=1, keyword=代码合成评估指标), Keyword(id=1329455912445653543, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, orderNo=2, keyword=功能准确性), Keyword(id=1329455912550511144, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, orderNo=3, keyword=鲁棒性), Keyword(id=1329455912609231401, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, orderNo=4, keyword=代码合成), Keyword(id=1329455912688923178, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, orderNo=5, keyword=神经网络)], refs=[Reference(id=1329455917634007637, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Gulwani S, Polozov O, Singh R. Program synthesis[J]. Foundations and Trends in Programming Languages, 2017, 4(1-2): 1−119, articleTitle=null, refAbstract=null), Reference(id=1329455917692727894, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Ren Shuo, Guo Daya, Lu Shuai, et al. CodeBLEU: A method for automatic evaluation of code synthesis[J]. arXiv preprint, arXiv: 2009. 10297, 2020, articleTitle=null, refAbstract=null), Reference(id=1329455917755642455, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Lu Shuai, Guo Daya, Ren Shuo, et al. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation[J]. arXiv preprint, arXiv: 2102. 04664, 2021, articleTitle=null, refAbstract=null), Reference(id=1329455917822751320, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Yujia, Choi D, Chung J, et al. Competition-level code generation with alphacode[J]. Science, 2022, 378(6624): 1092−1097, articleTitle=null, refAbstract=null), Reference(id=1329455917881471577, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Weisz J D, Muller M, Houde S, et al. Perfection not required human-AI partnerships in code translation[C]//Proc of the 26th Int Conf on Intelligent User Interfaces. New YorK: ACM, 2021: 402−412, articleTitle=null, refAbstract=null), Reference(id=1329455917944386138, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Jain N, Vaidyanath S, Iyer A, et al. Jigsaw: Large language models meet program synthesis[C]//Proc of the 44th Int Conf on Software Engineering. New York: ACM, 2022: 1219−1231, articleTitle=null, refAbstract=null), Reference(id=1329455917998912091, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Chaoning, Zhang Chenshuang, Zheng Sheng, et al. A complete survey on generative AI: Is ChatGPT from GPT-4 to GPT-5 all you need?[J]. arXiv preprint, arXiv: 2303. 11717, 2023, articleTitle=null, refAbstract=null), Reference(id=1329455918049243740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Yue, Wang Weishi, Joty S, et al. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation[C]//Proc of the 2021 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 8696−8708, articleTitle=null, refAbstract=null), Reference(id=1329455918107963997, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Nijkamp E, Pang B, Hayashi H, et al. CodeGen: An open large language model for code with multi-turn program synthesis[J]. arXiv preprint, arXiv: 2203. 13474, 2022, articleTitle=null, refAbstract=null), Reference(id=1329455918166684254, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Zheng Qinkai, Xia Xiao, Zou Xu, et al. CodeGeeX: A pre-trained model for code generation with multilingual evaluations on humaneval-x[J]. arXiv preprint, arXiv: 2303. 17568, 2023, articleTitle=null, refAbstract=null), Reference(id=1329455918229598815, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Liang Qingyuan, Sun Zeyu, Zhu Qihao, et al. Lyra: A benchmark for Turducken-style code generation[C]// Proc of the Thirty-First Int Joint Conf on Artificial Intelligence, San Francisco, CA: Morgan Kaufmann. 2022: 4238−4244, articleTitle=null, refAbstract=null), Reference(id=1329455918296707680, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Austin J, Odena A, Nye M, et al. Program synthesis with large language models[J]. arXiv preprint, arXiv: 2108. 07732, 2021, articleTitle=null, refAbstract=null), Reference(id=1329455918359622241, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Iyer S, Konstas I, Cheung A, et al. Mapping language to code in programmatic context[C]//Proc of the 2018 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2018: 1643−1652, articleTitle=null, refAbstract=null), Reference(id=1329455918426731106, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Papineni K, Roukos S, Ward T, et al. BLEU: A method for automatic evaluation of machine translation[C]//Proc of the 40th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2002: 311−318, articleTitle=null, refAbstract=null), Reference(id=1329455918481257059, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Popović M. chrF: Character n-gram F-Score for automatic MT evaluation[C]//Proc of the 10th workshop on Statistical Machine Translation. Stroudsburg, PA: ACL, 2015: 392−395, articleTitle=null, refAbstract=null), Reference(id=1329455918544171620, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Kulal S, Pasupat P, Chandra K, et al. Spoc: Search-based pseudocode to code[J]. Advances in Neural Information Processing Systems, 2019, 32: 11906−11917, articleTitle=null, refAbstract=null), Reference(id=1329455918607086181, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Niu Changan, Li Chuanyi, Luo Bin, et al. Deep learning meets software engineering: A survey on pre-trained models of source code[C]// Proc of the 31st Int Joint Conf on Artificial Intelligence. San Francisco, CA: Morgan Kaufmann, 2022: 5546−5555, articleTitle=null, refAbstract=null), Reference(id=1329455918695166566, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhuo T Y. Large language models are state-of-the-art evaluators of code generation[J]. arXiv preprint, arXiv: 2304. 14317, 2023, articleTitle=null, refAbstract=null), Reference(id=1329455918774858343, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Guo Daya, Lu Shuai, Duan Nan, et al. UniXcoder: Unified cross-modal pre-training for code representation[C]//Proc of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL. 2022: 7212−7225, articleTitle=null, refAbstract=null), Reference(id=1329455918858744424, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen M, Tworek J, Jun H, et al. Evaluating large language models trained on code[J]. arXiv preprint, arXiv: 2107. 03374, 2021, articleTitle=null, refAbstract=null), Reference(id=1329455918959407721, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Ahmad W U, Tushar M G R, Chakraborty S, et al. AVATAR: A parallel corpus for Java-Python program translation[J]. arXiv preprint, arXiv: 2108. 11590, 2021, articleTitle=null, refAbstract=null), Reference(id=1329455919039099498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Liguori P, Improta C, Natella R, et al. Who evaluates the evaluators? On automatic metrics for assessing AI-based offensive code generators[J]. Expert Systems with Applications, 2023, 225: 120073, articleTitle=null, refAbstract=null), Reference(id=1329455919118791275, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin C Y. Rouge: A package for automatic evaluation of summaries[C]//Proc of the Text Summarization Branches Out. Stroudsburg, PA: ACL, 2004: 74−81, articleTitle=null, refAbstract=null), Reference(id=1329455919181705836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Evtikhiev M, Bogomolov E, Sokolov Y, et al. Out of the BLEU: How should we assess quality of the code generation models?[J]. Journal of Systems and Software, 2023, 203: 111741, articleTitle=null, refAbstract=null), Reference(id=1329455919248814701, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Eghbali A, Pradel M. CrystalBLEU: Precisely and efficiently measuring the similarity of code[C]//Proc of the 37th IEEE/ACM Int Conf on Automated Software Engineering. Piscataway, NJ: IEEE, 2022: 1−12, articleTitle=null, refAbstract=null), Reference(id=1329455919320117870, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Dong Yihong, Ding Jiazheng, Jiang Xue, et al. CodeScore: Evaluating code generation by learning code execution[J]. arXiv preprint, arXiv: 2301. 09043, 2023, articleTitle=null, refAbstract=null), Reference(id=1329455919408198255, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Shuyan, Alon U, Agarwal S, et al. CodeBERTScore: Evaluating code generation with pretrained models of code[J]. arXiv preprint, arXiv: 2302. 05527, 2023, articleTitle=null, refAbstract=null), Reference(id=1329455919483695728, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Feng Zhangyin, Guo Daya, Tang Duyu, et al. CodeBERT: A pre-trained model for programming and natural languages[C]//Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 1536−1547, articleTitle=null, refAbstract=null), Reference(id=1329455919550804593, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Guang, Zhou Yu, Chen Xiang, et al. ExploitGen: Template-augmented exploit code generation based on CodeBERT[J]. Journal of Systems and Software, 2023, 197: 111577, articleTitle=null, refAbstract=null), Reference(id=1329455919622107762, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 2017: 6000−6010, articleTitle=null, refAbstract=null), Reference(id=1329455919680828019, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Husain H, Wu H H, Gazit T, et al. CodeSearchNet challenge: Evaluating the state of semantic code search[J]. arXiv preprint, arXiv: 1909. 09436, 2019, articleTitle=null, refAbstract=null), Reference(id=1329455919752131188, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Gao Tianyu, Yao Xingcheng, Chen Danqi. SimCSE: Simple contrastive learning of sentence embeddings[C]//Proc of the 2021 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 6894−6910, articleTitle=null, refAbstract=null), Reference(id=1329455920964285045, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Chakraborty S, Ahmed T, Ding Yangruibo, et al. NatGen: Generative pre-training by “naturalizing” source code[C]//Proc of the 30th ACM Joint European Software Engineering Conf and Symp. on the Foundations of Software Engineering. New York: ACM, 2022: 18−30, articleTitle=null, refAbstract=null), Reference(id=1329455921031393910, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Jia Yue, Harman M. An analysis and survey of the development of mutation testing[J]. IEEE Transactions on Software Engineering, 2010, 37(5): 649−678, articleTitle=null, refAbstract=null), Reference(id=1329455921090114167, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=van den Heuvel E, Zhan Z. Myths about linear and monotonic associations: Pearson’s r, Spearman’s ρ, and Kendall’s τ[J]. The American Statistician, 2022, 76(1): 44−52, articleTitle=null, refAbstract=null), Reference(id=1329455921157223032, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Willmott C J, Matsuura K. Advantages of the mean absolute error over the root mean square error in assessing average model performance[J]. Climate Research, 2005, 30(1): 79−82, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1329455909505446407, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, xref=1, ext=[AuthorCompanyExt(id=1329455909518029320, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909505446407, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1College of Computer Science and Technology/College of Artificial Intelligence/College of Software, Nanjing University of Aeronautics and Astronautics, Nanjing 211106), AuthorCompanyExt(id=1329455909526417929, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909505446407, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1南京航空航天大学计算机科学与技术学院/人工智能学院/软件学院 南京 211106)]), AuthorCompany(id=1329455909601915402, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, xref=2, ext=[AuthorCompanyExt(id=1329455909610304011, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909601915402, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
2School of Information Science and Technology, Nantong University, Nantong 226019), AuthorCompanyExt(id=1329455909614498316, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, companyId=1329455909601915402, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
2南通大学信息科学技术学院 江苏南通 226019)])], figs=[ArticleFig(id=1329455912797975083, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Figure 1, caption=
Example of a robustness problem for code synthesis evaluation metrics, figureFileSmall=mSca0ykRYIwA/uyiVCy4Mw==, figureFileBig=GxECN8L9k95Enwcj6f9pBw==, tableContent=null), ArticleFig(id=1329455912865083948, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=图1, caption=
代码合成评估指标的一个鲁棒性问题示例, figureFileSmall=mSca0ykRYIwA/uyiVCy4Mw==, figureFileBig=GxECN8L9k95Enwcj6f9pBw==, tableContent=null), ArticleFig(id=1329455913074799149, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Figure 2, caption=
CodeScore-R framework, figureFileSmall=4uAExSiaqzpoRQiYPJF1VQ==, figureFileBig=Ak8KfcUmKCV+Llw8ATGg+Q==, tableContent=null), ArticleFig(id=1329455913137713710, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=图2, caption=
CodeScore-R框架, figureFileSmall=4uAExSiaqzpoRQiYPJF1VQ==, figureFileBig=Ak8KfcUmKCV+Llw8ATGg+Q==, tableContent=null), ArticleFig(id=1329455913200628271, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Figure 3, caption=
An example of code pre-processing, figureFileSmall=4DQaLr6H41Nq7sNbStnsKQ==, figureFileBig=hwyK4BydRD69KLTkqU9vEw==, tableContent=null), ArticleFig(id=1329455913263542832, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=图3, caption=
代码预处理示例, figureFileSmall=4DQaLr6H41Nq7sNbStnsKQ==, figureFileBig=hwyK4BydRD69KLTkqU9vEw==, tableContent=null), ArticleFig(id=1329455913330651697, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Figure 4, caption=
Illustration of code representation learning tasks, figureFileSmall=doL5R2tzkiF5UE5WGiUvXw==, figureFileBig=dwqn1X3PypMKGS6W6frr7Q==, tableContent=null), ArticleFig(id=1329455913389371954, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=图4, caption=
掩码表征学习任务示例图, figureFileSmall=doL5R2tzkiF5UE5WGiUvXw==, figureFileBig=dwqn1X3PypMKGS6W6frr7Q==, tableContent=null), ArticleFig(id=1329455913477452339, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Figure 5, caption=
Classification results of CodeScore-R and functional correctness, figureFileSmall=afniHUv3IHM/DGJTtRcCLw==, figureFileBig=wF+ATE6z/cFD4qAwrdH5xA==, tableContent=null), ArticleFig(id=1329455913565532724, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=图5, caption=
CodeScore-R与功能准确性的分类结果, figureFileSmall=afniHUv3IHM/DGJTtRcCLw==, figureFileBig=wF+ATE6z/cFD4qAwrdH5xA==, tableContent=null), ArticleFig(id=1329455913636835893, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Figure 6, caption=
Process diagram of CodeBERTScore calculation, figureFileSmall=OoSOqtuEOz4kd0CdfyHMJw==, figureFileBig=p9rqUkK7f15mdhYbKrBLXA==, tableContent=null), ArticleFig(id=1329455913695556150, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=图6, caption=
CodeBERTScore计算过程图, figureFileSmall=OoSOqtuEOz4kd0CdfyHMJw==, figureFileBig=p9rqUkK7f15mdhYbKrBLXA==, tableContent=null), ArticleFig(id=1329455913783636535, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 1, caption=
Table of Code Syntax Equivalence Transformation Rules
, figureFileSmall=null, figureFileBig=null, tableContent=
| 规则 | 代码示例 |
|---|
| 循环交换(loop exchange) | for ⇔ while |
| 表达式交换(expression exchange) | a+=b ⇔ a=a+b |
| 判断体交换(permute exchange) | if(a){A}else{B} ⇔ if(!a){B}else{A} |
| 判断条件交换(condition exchange) | a>b ⇔ b>a or True ⇔ !False |
), ArticleFig(id=1329455913850745400, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表1, caption=
代码语法等价转换规则表
, figureFileSmall=null, figureFileBig=null, tableContent=
| 规则 | 代码示例 |
|---|
| 循环交换(loop exchange) | for ⇔ while |
| 表达式交换(expression exchange) | a+=b ⇔ a=a+b |
| 判断体交换(permute exchange) | if(a){A}else{B} ⇔ if(!a){B}else{A} |
| 判断条件交换(condition exchange) | a>b ⇔ b>a or True ⇔ !False |
), ArticleFig(id=1329455913951408697, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 2, caption=
Table of Code Mutation Testing Rules
, figureFileSmall=null, figureFileBig=null, tableContent=
| 规则 | 运算符 |
|---|
| 算术运算符 | +, −, ×, /, ××, % |
| 关系运算符 | >, <, >=, <=, ==, ≠ |
| 条件运算符 | &&, $|| $, &, |, ^ |
| 移位运算符 | <<, >>, >>> |
| 逻辑运算符 | &, |, ^ |
| 赋值运算符 | =, +=, …−=, ×=, /=, %=, ××=, <<=, >>=, >>>= |
), ArticleFig(id=1329455914022711866, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表2, caption=
代码变异测试规则表
, figureFileSmall=null, figureFileBig=null, tableContent=
| 规则 | 运算符 |
|---|
| 算术运算符 | +, −, ×, /, ××, % |
| 关系运算符 | >, <, >=, <=, ==, ≠ |
| 条件运算符 | &&, $|| $, &, |, ^ |
| 移位运算符 | <<, >>, >>> |
| 逻辑运算符 | &, |, ^ |
| 赋值运算符 | =, +=, …−=, ×=, /=, %=, ××=, <<=, >>=, >>>= |
), ArticleFig(id=1329455914089820731, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 3, caption=
Comparison Results Between CodeScore-R and Baseline Metrics for MAE
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | 代码生成 | | 代码迁移 |
|---|
| Java | Python | | Java | Python |
|---|
| BLEU | 0.453 | 0.498 | | 0.268 | 0.347 |
| ROUGE-L | 0.391 | 0.409 | | 0.225 | 0.192 |
| ChrF | 0.395 | 0.424 | | 0.229 | 0.262 |
| ED | 0.392 | 0.401 | | 0.208 | 0.198 |
| WeightBLEU | 0.415 | 0.467 | | 0.252 | 0.329 |
| SyntaxMatch | 0.404 | 0.436 | | 0.283 | 0.420 |
| DataflowMatch | 0.398 | 0.415 | | 0.217 | 0.313 |
| CodeBLEU | 0.418 | 0.454 | | 0.255 | 0.352 |
| CrystalBLEU | 0.401 | 0.432 | | 0.235 | 0.220 |
| CodeBERTScore | 0.342 | 0.324 | | 0.202 | 0.151 |
| CodeScore-R | 0.317 | 0.287 | | 0.175 | 0.125 |
), ArticleFig(id=1329455914152735292, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表3, caption=
CodeScore-R与基准指标的MAE对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | 代码生成 | | 代码迁移 |
|---|
| Java | Python | | Java | Python |
|---|
| BLEU | 0.453 | 0.498 | | 0.268 | 0.347 |
| ROUGE-L | 0.391 | 0.409 | | 0.225 | 0.192 |
| ChrF | 0.395 | 0.424 | | 0.229 | 0.262 |
| ED | 0.392 | 0.401 | | 0.208 | 0.198 |
| WeightBLEU | 0.415 | 0.467 | | 0.252 | 0.329 |
| SyntaxMatch | 0.404 | 0.436 | | 0.283 | 0.420 |
| DataflowMatch | 0.398 | 0.415 | | 0.217 | 0.313 |
| CodeBLEU | 0.418 | 0.454 | | 0.255 | 0.352 |
| CrystalBLEU | 0.401 | 0.432 | | 0.235 | 0.220 |
| CodeBERTScore | 0.342 | 0.324 | | 0.202 | 0.151 |
| CodeScore-R | 0.317 | 0.287 | | 0.175 | 0.125 |
), ArticleFig(id=1329455914219844157, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 4, caption=
Comparison Results for Token Perturbation on Code Generation Tasks
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Java代码生成 | | Python代码生成 |
|---|
| O2S | S2S | | O2S | S2S |
|---|
| BLEU | 0.544 | 0.435 | | 0.593 | 0.465 |
| ROUGE-L | 0.501 | 0.390 | | 0.444 | 0.398 |
| ChrF | 0.483 | 0.376 | | 0.537 | 0.398 |
| ED | 0.429 | 0.381 | | 0.454 | 0.385 |
| WeightBLEU | 0.522 | 0.394 | | 0.574 | 0.432 |
| SyntaxMatch | 0.404 | 0.401 | | 0.450 | 0.439 |
| DataflowMatch | 0.402 | 0.395 | | 0.422 | 0.409 |
| CodeBLEU | 0.468 | 0.406 | | 0.510 | 0.436 |
| CrystalBLEU | 0.486 | 0.389 | | 0.508 | 0.405 |
| CodeBERTScore | 0.376 | 0.340 | | 0.371 | 0.316 |
| CodeScore-R | | 0.317 | | | 0.287 |
), ArticleFig(id=1329455914282758718, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表4, caption=
在代码生成任务上针对词素扰动的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Java代码生成 | | Python代码生成 |
|---|
| O2S | S2S | | O2S | S2S |
|---|
| BLEU | 0.544 | 0.435 | | 0.593 | 0.465 |
| ROUGE-L | 0.501 | 0.390 | | 0.444 | 0.398 |
| ChrF | 0.483 | 0.376 | | 0.537 | 0.398 |
| ED | 0.429 | 0.381 | | 0.454 | 0.385 |
| WeightBLEU | 0.522 | 0.394 | | 0.574 | 0.432 |
| SyntaxMatch | 0.404 | 0.401 | | 0.450 | 0.439 |
| DataflowMatch | 0.402 | 0.395 | | 0.422 | 0.409 |
| CodeBLEU | 0.468 | 0.406 | | 0.510 | 0.436 |
| CrystalBLEU | 0.486 | 0.389 | | 0.508 | 0.405 |
| CodeBERTScore | 0.376 | 0.340 | | 0.371 | 0.316 |
| CodeScore-R | | 0.317 | | | 0.287 |
), ArticleFig(id=1329455914362450495, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 5, caption=
Comparison Results for Token Perturbation on Code Migration Tasks
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Python-to-Java | | Java-to-Python |
|---|
| O2S | S2S | | O2S | S2S |
|---|
| BLEU | 0.590 | 0.265 | | 0.718 | 0.340 |
| ROUGE-L | 0.485 | 0.227 | | 0.310 | 0.191 |
| ChrF | 0.493 | 0.223 | | 0.598 | 0.242 |
| ED | 0.335 | 0.208 | | 0.380 | 0.198 |
| WeightBLEU | 0.579 | 0.250 | | 0.709 | 0.323 |
| SyntaxMatch | 0.283 | 0.283 | | 0.420 | 0.411 |
| DataflowMatch | 0.241 | 0.218 | | 0.350 | 0.304 |
| CodeBLEU | 0.423 | 0.254 | | 0.549 | 0.345 |
| CrystalBLEU | 0.474 | 0.213 | | 0.449 | 0.211 |
| CodeBERTScore | 0.282 | 0.201 | | 0.273 | 0.149 |
| CodeScore-R | | 0.175 | | | 0.125 |
), ArticleFig(id=1329455914425365056, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表5, caption=
在代码迁移任务上针对词素扰动的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Python-to-Java | | Java-to-Python |
|---|
| O2S | S2S | | O2S | S2S |
|---|
| BLEU | 0.590 | 0.265 | | 0.718 | 0.340 |
| ROUGE-L | 0.485 | 0.227 | | 0.310 | 0.191 |
| ChrF | 0.493 | 0.223 | | 0.598 | 0.242 |
| ED | 0.335 | 0.208 | | 0.380 | 0.198 |
| WeightBLEU | 0.579 | 0.250 | | 0.709 | 0.323 |
| SyntaxMatch | 0.283 | 0.283 | | 0.420 | 0.411 |
| DataflowMatch | 0.241 | 0.218 | | 0.350 | 0.304 |
| CodeBLEU | 0.423 | 0.254 | | 0.549 | 0.345 |
| CrystalBLEU | 0.474 | 0.213 | | 0.449 | 0.211 |
| CodeBERTScore | 0.282 | 0.201 | | 0.273 | 0.149 |
| CodeScore-R | | 0.175 | | | 0.125 |
), ArticleFig(id=1329455914488279617, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 6, caption=
Comparison Results of CodeScore-R and Baseline Metrics for Syntax Perturbation
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | 代码生成 | | 代码迁移 |
|---|
| Java | Python | | Java | Python |
|---|
| BLEU | 0.451 | 0.479 | | 0.318 | 0.388 |
| ROUGE-L | 0.453 | 0.493 | | 0.377 | 0.502 |
| ChrF | 0.385 | 0.421 | | 0.251 | 0.318 |
| ED | 0.391 | 0.393 | | 0.238 | 0.230 |
| WeightBLEU | 0.412 | 0.444 | | 0.300 | 0.362 |
| SyntaxMatch | 0.414 | 0.448 | | 0.304 | 0.444 |
| DataflowMatch | 0.396 | 0.414 | | 0.221 | 0.324 |
| CodeBLEU | 0.418 | 0.446 | | 0.286 | 0.379 |
| CrystalBLEU | 0.430 | 0.447 | | 0.316 | 0.369 |
| CodeBERTScore | 0.347 | 0.320 | | 0.219 | 0.167 |
| CodeScore-R | 0.311 | 0.278 | | 0.182 | 0.129 |
), ArticleFig(id=1329455914567971394, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表6, caption=
CodeScore-R与基准指标针对语法扰动的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | 代码生成 | | 代码迁移 |
|---|
| Java | Python | | Java | Python |
|---|
| BLEU | 0.451 | 0.479 | | 0.318 | 0.388 |
| ROUGE-L | 0.453 | 0.493 | | 0.377 | 0.502 |
| ChrF | 0.385 | 0.421 | | 0.251 | 0.318 |
| ED | 0.391 | 0.393 | | 0.238 | 0.230 |
| WeightBLEU | 0.412 | 0.444 | | 0.300 | 0.362 |
| SyntaxMatch | 0.414 | 0.448 | | 0.304 | 0.444 |
| DataflowMatch | 0.396 | 0.414 | | 0.221 | 0.324 |
| CodeBLEU | 0.418 | 0.446 | | 0.286 | 0.379 |
| CrystalBLEU | 0.430 | 0.447 | | 0.316 | 0.369 |
| CodeBERTScore | 0.347 | 0.320 | | 0.219 | 0.167 |
| CodeScore-R | 0.311 | 0.278 | | 0.182 | 0.129 |
), ArticleFig(id=1329455914643468867, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 7, caption=
Comparison Results for Semantic Perturbation on Java Code Generation Task
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Java代码生成 |
|---|
| Top25% | Top50% | Top75% | Top100% |
|---|
| BLEU | 0.460 | 0.471 | 0.473 | 0.450 |
| ROUGE-L | 0.441 | 0.476 | 0.499 | 0.498 |
| ChrF | 0.489 | 0.575 | 0.647 | 0.673 |
| ED | 0.491 | 0.572 | 0.644 | 0.675 |
| WeightBLEU | 0.473 | 0.526 | 0.565 | 0.559 |
| SyntaxMatch | 0.481 | 0.529 | 0.565 | 0.570 |
| DataflowMatch | 0.476 | 0.546 | 0.587 | 0.590 |
| CodeBLEU | 0.473 | 0.518 | 0.547 | 0.542 |
| CrystalBLEU | 0.461 | 0.517 | 0.560 | 0.582 |
| CodeBERTScore | 0.515 | 0.660 | 0.788 | 0.872 |
| CodeScore-R | 0.326 | 0.304 | 0.320 | 0.335 |
), ArticleFig(id=1329455914714772036, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表7, caption=
在Java代码生成任务上针对语义扰动的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Java代码生成 |
|---|
| Top25% | Top50% | Top75% | Top100% |
|---|
| BLEU | 0.460 | 0.471 | 0.473 | 0.450 |
| ROUGE-L | 0.441 | 0.476 | 0.499 | 0.498 |
| ChrF | 0.489 | 0.575 | 0.647 | 0.673 |
| ED | 0.491 | 0.572 | 0.644 | 0.675 |
| WeightBLEU | 0.473 | 0.526 | 0.565 | 0.559 |
| SyntaxMatch | 0.481 | 0.529 | 0.565 | 0.570 |
| DataflowMatch | 0.476 | 0.546 | 0.587 | 0.590 |
| CodeBLEU | 0.473 | 0.518 | 0.547 | 0.542 |
| CrystalBLEU | 0.461 | 0.517 | 0.560 | 0.582 |
| CodeBERTScore | 0.515 | 0.660 | 0.788 | 0.872 |
| CodeScore-R | 0.326 | 0.304 | 0.320 | 0.335 |
), ArticleFig(id=1329455914777686597, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 8, caption=
Comparison Results for Semantic Perturbation on Python Code Generation Task
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Python代码生成 |
|---|
| Top25% | Top50% | Top75% | Top100% |
|---|
| BLEU | 0.481 | 0.468 | 0.428 | 0.391 |
| ROUGE-L | 0.479 | 0.558 | 0.583 | 0.598 |
| ChrF | 0.484 | 0.550 | 0.575 | 0.592 |
| ED | 0.472 | 0.555 | 0.591 | 0.619 |
| WeightBLEU | 0.489 | 0.508 | 0.489 | 0.471 |
| SyntaxMatch | 0.501 | 0.510 | 0.504 | 0.492 |
| DataflowMatch | 0.482 | 0.576 | 0.611 | 0.637 |
| CodeBLEU | 0.488 | 0.515 | 0.508 | 0.498 |
| CrystalBLEU | 0.484 | 0.552 | 0.577 | 0.596 |
| CodeBERTScore | 0.470 | 0.645 | 0.759 | 0.862 |
| CodeScore-R | 0.301 | 0.317 | 0.320 | 0.312 |
), ArticleFig(id=1329455914840601158, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表8, caption=
在Python代码生成任务上针对语义扰动的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Python代码生成 |
|---|
| Top25% | Top50% | Top75% | Top100% |
|---|
| BLEU | 0.481 | 0.468 | 0.428 | 0.391 |
| ROUGE-L | 0.479 | 0.558 | 0.583 | 0.598 |
| ChrF | 0.484 | 0.550 | 0.575 | 0.592 |
| ED | 0.472 | 0.555 | 0.591 | 0.619 |
| WeightBLEU | 0.489 | 0.508 | 0.489 | 0.471 |
| SyntaxMatch | 0.501 | 0.510 | 0.504 | 0.492 |
| DataflowMatch | 0.482 | 0.576 | 0.611 | 0.637 |
| CodeBLEU | 0.488 | 0.515 | 0.508 | 0.498 |
| CrystalBLEU | 0.484 | 0.552 | 0.577 | 0.596 |
| CodeBERTScore | 0.470 | 0.645 | 0.759 | 0.862 |
| CodeScore-R | 0.301 | 0.317 | 0.320 | 0.312 |
), ArticleFig(id=1329455914911904327, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 9, caption=
Comparison Results for Semantic Perturbation on Python-to-Java Code Migration Task
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Python-to-Java代码迁移 |
|---|
| Top25% | Top50% | Top75% | Top100% |
|---|
| BLEU | 0.377 | 0.487 | 0.586 | 0.722 |
| ROUGE-L | 0.342 | 0.451 | 0.555 | 0.681 |
| ChrF | 0.378 | 0.531 | 0.678 | 0.851 |
| ED | 0.378 | 0.545 | 0.711 | 0.901 |
| WeightBLEU | 0.377 | 0.501 | 0.614 | 0.759 |
| SyntaxMatch | 0.395 | 0.499 | 0.606 | 0.738 |
| DataflowMatch | 0.359 | 0.480 | 0.615 | 0.769 |
| CodeBLEU | 0.377 | 0.492 | 0.605 | 0.747 |
| CrystalBLEU | 0.352 | 0.484 | 0.615 | 0.769 |
| CodeBERTScore | 0.387 | 0.567 | 0.747 | 0.947 |
| CodeScore-R | 0.243 | 0.307 | 0.381 | 0.456 |
), ArticleFig(id=1329455914995790408, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表9, caption=
在Python-to-Java代码迁移任务上针对语义扰动的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Python-to-Java代码迁移 |
|---|
| Top25% | Top50% | Top75% | Top100% |
|---|
| BLEU | 0.377 | 0.487 | 0.586 | 0.722 |
| ROUGE-L | 0.342 | 0.451 | 0.555 | 0.681 |
| ChrF | 0.378 | 0.531 | 0.678 | 0.851 |
| ED | 0.378 | 0.545 | 0.711 | 0.901 |
| WeightBLEU | 0.377 | 0.501 | 0.614 | 0.759 |
| SyntaxMatch | 0.395 | 0.499 | 0.606 | 0.738 |
| DataflowMatch | 0.359 | 0.480 | 0.615 | 0.769 |
| CodeBLEU | 0.377 | 0.492 | 0.605 | 0.747 |
| CrystalBLEU | 0.352 | 0.484 | 0.615 | 0.769 |
| CodeBERTScore | 0.387 | 0.567 | 0.747 | 0.947 |
| CodeScore-R | 0.243 | 0.307 | 0.381 | 0.456 |
), ArticleFig(id=1329455915079676489, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 10, caption=
Comparison Results for Semantic Perturbation on Java-to-Python Code Migration Task
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Java-to-Python代码迁移 |
|---|
| Top25% | Top50% | Top75% | Top100% |
|---|
| BLEU | 0.403 | 0.461 | 0.524 | 0.607 |
| ROUGE-L | 0.360 | 0.523 | 0.696 | 0.873 |
| ChrF | 0.378 | 0.504 | 0.641 | 0.787 |
| ED | 0.365 | 0.525 | 0.695 | 0.867 |
| WeightBLEU | 0.400 | 0.464 | 0.538 | 0.631 |
| SyntaxMatch | 0.449 | 0.475 | 0.518 | 0.581 |
| DataflowMatch | 0.395 | 0.471 | 0.586 | 0.715 |
| CodeBLEU | 0.412 | 0.468 | 0.541 | 0.634 |
| CrystalBLEU | 0.368 | 0.523 | 0.683 | 0.851 |
| CodeBERTScore | 0.355 | 0.552 | 0.757 | 0.961 |
| CodeScore-R | 0.197 | 0.257 | 0.303 | 0.371 |
), ArticleFig(id=1329455915150979658, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表10, caption=
在Java-to-Python代码迁移任务上针对语义扰动的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Java-to-Python代码迁移 |
|---|
| Top25% | Top50% | Top75% | Top100% |
|---|
| BLEU | 0.403 | 0.461 | 0.524 | 0.607 |
| ROUGE-L | 0.360 | 0.523 | 0.696 | 0.873 |
| ChrF | 0.378 | 0.504 | 0.641 | 0.787 |
| ED | 0.365 | 0.525 | 0.695 | 0.867 |
| WeightBLEU | 0.400 | 0.464 | 0.538 | 0.631 |
| SyntaxMatch | 0.449 | 0.475 | 0.518 | 0.581 |
| DataflowMatch | 0.395 | 0.471 | 0.586 | 0.715 |
| CodeBLEU | 0.412 | 0.468 | 0.541 | 0.634 |
| CrystalBLEU | 0.368 | 0.523 | 0.683 | 0.851 |
| CodeBERTScore | 0.355 | 0.552 | 0.757 | 0.961 |
| CodeScore-R | 0.197 | 0.257 | 0.303 | 0.371 |
), ArticleFig(id=1329455915213894219, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 11, caption=
Comparison Results of Different Semantic Extraction Methods for CodeScore-R on Code Generation Tasks
, figureFileSmall=null, figureFileBig=null, tableContent=
| 方法 | Java代码生成 | | Python代码生成 |
|---|
| MAE | 准确率 | 精度 | 召回率 | F1-Score | | MAE | 准确率 | 精度 | 召回率 | F1-Score |
|---|
| last-avg | 0.323 | 0.677 | 0.74 | 0.813 | 0.774 | | 0.341 | 0.659 | 0.761 | 0.761 | 0.761 |
| first-last-avg | 0.311 | 0.689 | 0.744 | 0.83 | 0.785 | | 0.323 | 0.677 | 0.762 | 0.795 | 0.778 |
| CLS | 0.323 | 0.677 | 0.748 | 0.795 | 0.771 | | 0.354 | 0.646 | 0.761 | 0.735 | 0.748 |
| CLS+ReLU | 0.317 | 0.683 | 0.717 | 0.884 | 0.792 | | 0.287 | 0.713 | 0.765 | 0.863 | 0.811 |
), ArticleFig(id=1329455915276808780, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表11, caption=
不同的语义提取方法在代码生成任务上对CodeScore-R的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 方法 | Java代码生成 | | Python代码生成 |
|---|
| MAE | 准确率 | 精度 | 召回率 | F1-Score | | MAE | 准确率 | 精度 | 召回率 | F1-Score |
|---|
| last-avg | 0.323 | 0.677 | 0.74 | 0.813 | 0.774 | | 0.341 | 0.659 | 0.761 | 0.761 | 0.761 |
| first-last-avg | 0.311 | 0.689 | 0.744 | 0.83 | 0.785 | | 0.323 | 0.677 | 0.762 | 0.795 | 0.778 |
| CLS | 0.323 | 0.677 | 0.748 | 0.795 | 0.771 | | 0.354 | 0.646 | 0.761 | 0.735 | 0.748 |
| CLS+ReLU | 0.317 | 0.683 | 0.717 | 0.884 | 0.792 | | 0.287 | 0.713 | 0.765 | 0.863 | 0.811 |
), ArticleFig(id=1329455915343917645, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 12, caption=
Comparison Results of Different Semantic Extraction Methods for CodeScore-R on Code Migration Tasks
, figureFileSmall=null, figureFileBig=null, tableContent=
| 方法 | Python-to-Java代码迁移 | | Java-to-Python代码迁移 |
|---|
| MAE | 准确率 | 精度 | 召回率 | F1-Score | | MAE | 准确率 | 精度 | 召回率 | F1-Score |
|---|
| last-avg | 0.175 | 0.825 | 0.828 | 0.988 | 0.901 | | 0.135 | 0.865 | 0.876 | 0.983 | 0.926 |
| first-last-avg | 0.170 | 0.830 | 0.829 | 0.994 | 0.904 | | 0.130 | 0.870 | 0.877 | 0.988 | 0.929 |
| CLS | 0.180 | 0.820 | 0.824 | 0.988 | 0.898 | | 0.140 | 0.860 | 0.876 | 0.977 | 0.923 |
| CLS+ReLU | 0.175 | 0.825 | 0.821 | 1.000 | 0.902 | | 0.125 | 0.875 | 0.878 | 0.994 | 0.932 |
), ArticleFig(id=1329455916946141774, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表12, caption=
不同的语义提取方法在代码迁移任务上对CodeScore-R的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 方法 | Python-to-Java代码迁移 | | Java-to-Python代码迁移 |
|---|
| MAE | 准确率 | 精度 | 召回率 | F1-Score | | MAE | 准确率 | 精度 | 召回率 | F1-Score |
|---|
| last-avg | 0.175 | 0.825 | 0.828 | 0.988 | 0.901 | | 0.135 | 0.865 | 0.876 | 0.983 | 0.926 |
| first-last-avg | 0.170 | 0.830 | 0.829 | 0.994 | 0.904 | | 0.130 | 0.870 | 0.877 | 0.988 | 0.929 |
| CLS | 0.180 | 0.820 | 0.824 | 0.988 | 0.898 | | 0.140 | 0.860 | 0.876 | 0.977 | 0.923 |
| CLS+ReLU | 0.175 | 0.825 | 0.821 | 1.000 | 0.902 | | 0.125 | 0.875 | 0.878 | 0.994 | 0.932 |
), ArticleFig(id=1329455917067776591, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 13, caption=
Comparison Results of Different Metrics Score on Code Generation Tasks
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Java代码生成 | | Python代码生成 |
|---|
| 原始 | 词素扰动 | 语法扰动 | 语义扰动 | | 原始 | 词素扰动 | 语法扰动 | 语义扰动 |
|---|
| BLEU | 0.459 | 0.271 | 0.457 | 0.456 | | 0.395 | 0.200 | 0.411 | 0.417 |
| ROUGE-L | 0.655 | 0.385 | 0.478 | 0.504 | | 0.618 | 0.550 | 0.416 | 0.626 |
| ChrF | 0.645 | 0.460 | 0.678 | 0.680 | | 0.589 | 0.353 | 0.582 | 0.621 |
| ED | 0.663 | 0.581 | 0.673 | 0.681 | | 0.624 | 0.521 | 0.641 | 0.649 |
| WeightBLEU | 0.569 | 0.338 | 0.567 | 0.566 | | 0.476 | 0.240 | 0.498 | 0.498 |
| SyntaxMatch | 0.639 | 0.639 | 0.620 | 0.580 | | 0.513 | 0.497 | 0.502 | 0.507 |
| DataflowMatch | 0.627 | 0.630 | 0.622 | 0.612 | | 0.622 | 0.621 | 0.618 | 0.627 |
| CodeBLEU | 0.573 | 0.470 | 0.567 | 0.553 | | 0.501 | 0.389 | 0.507 | 0.512 |
| CrystalBLEU | 0.626 | 0.436 | 0.571 | 0.589 | | 0.583 | 0.437 | 0.536 | 0.622 |
| CodeBERTScore | 0.879 | 0.778 | 0.875 | 0.878 | | 0.881 | 0.781 | 0.892 | 0.897 |
| CodeScore-R | 0.841 | 0.841 | 0.813 | 0.341 | | 0.805 | 0.805 | 0.806 | 0.349 |
), ArticleFig(id=1329455917193605712, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表13, caption=
代码生成任务上不同指标分数对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Java代码生成 | | Python代码生成 |
|---|
| 原始 | 词素扰动 | 语法扰动 | 语义扰动 | | 原始 | 词素扰动 | 语法扰动 | 语义扰动 |
|---|
| BLEU | 0.459 | 0.271 | 0.457 | 0.456 | | 0.395 | 0.200 | 0.411 | 0.417 |
| ROUGE-L | 0.655 | 0.385 | 0.478 | 0.504 | | 0.618 | 0.550 | 0.416 | 0.626 |
| ChrF | 0.645 | 0.460 | 0.678 | 0.680 | | 0.589 | 0.353 | 0.582 | 0.621 |
| ED | 0.663 | 0.581 | 0.673 | 0.681 | | 0.624 | 0.521 | 0.641 | 0.649 |
| WeightBLEU | 0.569 | 0.338 | 0.567 | 0.566 | | 0.476 | 0.240 | 0.498 | 0.498 |
| SyntaxMatch | 0.639 | 0.639 | 0.620 | 0.580 | | 0.513 | 0.497 | 0.502 | 0.507 |
| DataflowMatch | 0.627 | 0.630 | 0.622 | 0.612 | | 0.622 | 0.621 | 0.618 | 0.627 |
| CodeBLEU | 0.573 | 0.470 | 0.567 | 0.553 | | 0.501 | 0.389 | 0.507 | 0.512 |
| CrystalBLEU | 0.626 | 0.436 | 0.571 | 0.589 | | 0.583 | 0.437 | 0.536 | 0.622 |
| CodeBERTScore | 0.879 | 0.778 | 0.875 | 0.878 | | 0.881 | 0.781 | 0.892 | 0.897 |
| CodeScore-R | 0.841 | 0.841 | 0.813 | 0.341 | | 0.805 | 0.805 | 0.806 | 0.349 |
), ArticleFig(id=1329455917294269009, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 14, caption=
Comparison Results of Different Metrics Score on Code Migration Tasks
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Python-to-Java代码迁移 | | Java-to-Python代码迁移 |
|---|
| 原始 | 词素扰动 | 语法扰动 | 语义扰动 | | 原始 | 词素扰动 | 语法扰动 | 语义扰动 |
|---|
| BLEU | 0.779 | 0.320 | 0.712 | 0.717 | | 0.670 | 0.194 | 0.617 | 0.607 |
| ROUGE-L | 0.890 | 0.489 | 0.657 | 0.679 | | 0.891 | 0.740 | 0.481 | 0.873 |
| ChrF | 0.862 | 0.479 | 0.841 | 0.850 | | 0.795 | 0.356 | 0.725 | 0.786 |
| ED | 0.914 | 0.724 | 0.879 | 0.900 | | 0.882 | 0.646 | 0.847 | 0.867 |
| WeightBLEU | 0.815 | 0.343 | 0.750 | 0.754 | | 0.695 | 0.207 | 0.653 | 0.631 |
| SyntaxMatch | 0.791 | 0.791 | 0.767 | 0.734 | | 0.565 | 0.569 | 0.534 | 0.555 |
| DataflowMatch | 0.883 | 0.850 | 0.875 | 0.864 | | 0.694 | 0.670 | 0.682 | 0.693 |
| CodeBLEU | 0.817 | 0.576 | 0.776 | 0.767 | | 0.656 | 0.410 | 0.622 | 0.621 |
| CrystalBLEU | 0.867 | 0.507 | 0.753 | 0.768 | | 0.853 | 0.555 | 0.658 | 0.851 |
| CodeBERTScore | 0.966 | 0.844 | 0.943 | 0.946 | | 0.966 | 0.803 | 0.947 | 0.960 |
| CodeScore-R | 0.980 | 0.980 | 0.945 | 0.450 | | 0.980 | 0.980 | 0.968 | 0.371 |
), ArticleFig(id=1329455917378155090, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表14, caption=
代码迁移任务上不同指标分数对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 指标 | Python-to-Java代码迁移 | | Java-to-Python代码迁移 |
|---|
| 原始 | 词素扰动 | 语法扰动 | 语义扰动 | | 原始 | 词素扰动 | 语法扰动 | 语义扰动 |
|---|
| BLEU | 0.779 | 0.320 | 0.712 | 0.717 | | 0.670 | 0.194 | 0.617 | 0.607 |
| ROUGE-L | 0.890 | 0.489 | 0.657 | 0.679 | | 0.891 | 0.740 | 0.481 | 0.873 |
| ChrF | 0.862 | 0.479 | 0.841 | 0.850 | | 0.795 | 0.356 | 0.725 | 0.786 |
| ED | 0.914 | 0.724 | 0.879 | 0.900 | | 0.882 | 0.646 | 0.847 | 0.867 |
| WeightBLEU | 0.815 | 0.343 | 0.750 | 0.754 | | 0.695 | 0.207 | 0.653 | 0.631 |
| SyntaxMatch | 0.791 | 0.791 | 0.767 | 0.734 | | 0.565 | 0.569 | 0.534 | 0.555 |
| DataflowMatch | 0.883 | 0.850 | 0.875 | 0.864 | | 0.694 | 0.670 | 0.682 | 0.693 |
| CodeBLEU | 0.817 | 0.576 | 0.776 | 0.767 | | 0.656 | 0.410 | 0.622 | 0.621 |
| CrystalBLEU | 0.867 | 0.507 | 0.753 | 0.768 | | 0.853 | 0.555 | 0.658 | 0.851 |
| CodeBERTScore | 0.966 | 0.844 | 0.943 | 0.946 | | 0.966 | 0.803 | 0.947 | 0.960 |
| CodeScore-R | 0.980 | 0.980 | 0.945 | 0.450 | | 0.980 | 0.980 | 0.968 | 0.371 |
), ArticleFig(id=1329455917449458259, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=EN, label=Table 15, caption=
Comparison Results of Different Similarity Thresholds
, figureFileSmall=null, figureFileBig=null, tableContent=
| 阈值 | 代码生成 | | 代码迁移 |
|---|
| Java | Python | | Java | Python |
|---|
| 0.1 | 0.317 | 0.238 | | 0.175 | 0.125 |
| 0.2 | 0.317 | 0.238 | | 0.175 | 0.125 |
| 0.3 | 0.329 | 0.256 | | 0.175 | 0.125 |
| 0.4 | 0.323 | 0.268 | | 0.175 | 0.125 |
| 0.5 | 0.317 | 0.287 | | 0.175 | 0.125 |
| 0.6 | 0.323 | 0.348 | | 0.180 | 0.135 |
| 0.7 | 0.353 | 0.433 | | 0.175 | 0.140 |
| 0.8 | 0.396 | 0.433 | | 0.165 | 0.165 |
| 0.9 | 0.421 | 0.500 | | 0.200 | 0.260 |
), ArticleFig(id=1329455917529150036, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329455905067872752, language=CN, label=表15, caption=
不同相似度阈值的对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 阈值 | 代码生成 | | 代码迁移 |
|---|
| Java | Python | | Java | Python |
|---|
| 0.1 | 0.317 | 0.238 | | 0.175 | 0.125 |
| 0.2 | 0.317 | 0.238 | | 0.175 | 0.125 |
| 0.3 | 0.329 | 0.256 | | 0.175 | 0.125 |
| 0.4 | 0.323 | 0.268 | | 0.175 | 0.125 |
| 0.5 | 0.317 | 0.287 | | 0.175 | 0.125 |
| 0.6 | 0.323 | 0.348 | | 0.180 | 0.135 |
| 0.7 | 0.353 | 0.433 | | 0.175 | 0.140 |
| 0.8 | 0.396 | 0.433 | | 0.165 | 0.165 |
| 0.9 | 0.421 | 0.500 | | 0.200 | 0.260 |
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202330715, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330715, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202330715, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202330715, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1706803200000, fullTextJson=null, articleText=null, reference=null)