首页 期刊 论文 学科刊群 资讯 加盟 关于
EN
image
Article(id=1316695248374448234, tenantId=1146029695717560320, journalId=1308397381743640680, issueId=1316695224617914963, articleNumber=null, orderNo=null, doi=null, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1755187200000, receivedDateStr=2025-08-15, revisedDate=1763827200000, revisedDateStr=2025-11-23, acceptedDate=null, acceptedDateStr=null, onlineDate=1791854210227, onlineDateStr=2026-10-13, pubDate=1768406400000, pubDateStr=2026-01-15, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1791854210227, onlineIssueDateStr=2026-10-13, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1791854210227, creator=13701087609, updateTime=1791854210227, updator=13701087609, issue=Issue{id=1316695224617914963, tenantId=1146029695717560320, journalId=1308397381743640680, year='2026', volume='44', issue='1', pageStart='1', pageEnd='224', issueExtLink='null', onlineDate='null', pubDate='1768406400000', pubDateStr='2026-01-15', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1791854204563, creator='13701087609', updateTime=1791854713654, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1316697359963239159, tenantId=1146029695717560320, journalId=1308397381743640680, issueId=1316695224617914963, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1316697359963239160, tenantId=1146029695717560320, journalId=1308397381743640680, issueId=1316695224617914963, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=41, endPage=50, ext={EN=ArticleExt(id=1316695248764518507, articleId=1316695248374448234, tenantId=1146029695717560320, journalId=1308397381743640680, language=EN, title=The process of sequential AI-Human collaborative decision-making——Insights from initial manuscript screening in journal peer review, columnId=1316695225842651733, journalTitle=Studies in Science of Science, columnName=Forums of Science and Technology, runingTitle=null, highlight=null, articleAbstract=

The widespread application of the "AI-human" system in management decision-making has sparked discussions on enhancing the synergy of the "AI-human" system to improve management decision-making performance. Although existing research has proposed some methods to improve the synergy of the "AI-human" system in management decision-making, there is still a lack of validation in specific business scenarios. This article chooses the initial screening of journal articles as a scenario and delve into the specific process of sequential AI-Human collaboration to exploring the path to improving the synergy of the "AI-human" system in management decision-making.

This article proposes that we can use the scoring results of AI to eliminate some low-quality submitted papers before the editor starts the initial screening work. Then, the editor screens the remaining papers with higher scores. In this process, AI’s filtering effect will reduce the number of papers reviewed by editors. Therefore, we can use the sequential AI-Human collaborative mode to improve the overall efficiency of paper review work. That is to say, the key to this type decision-making lies in designing AI screening thresholds based on an evaluation of AI's screening effectiveness.

Firstly, this article introduces Spearman correlation and innovatively designs priority comparison coefficient to evaluate the overall decision-making efficiency of AI. Secondly, by defining the negative and positive states of the screening results, sensitivity, specificity, and accuracy indicators were introduced to evaluate the AI’s decision screening effectiveness; Then, the design of evaluating benchmarks is carried out by aggregating human collective judgments, and it is proposed that the judgments of multiple human experts can be combined, enhanced, and jointly determined to form different evaluation benchmarks. These benchmarks can avoid the limitation of using a single evaluation benchmark that cannot accurately reflect AI’s decision-making ability; Finally, by designing the minimum retention rate as an indicator, the threshold setting method for AI in screening decision was clarified. This indicator not only reflects AI’s screening effectiveness, but also enables the rapid implementation of AI applications in the specific scenario. The analysis of three groups of papers in a journal demonstrates that by retaining 75% of the papers during the initial screening phase using existing large language models, high-quality papers identified in relevant evaluation benchmarks can be effectively covered. This reduces editorial workload by approximately 25%, increases overall editorial efficiency by about 15%, and improves the efficiency of the entire editorial process by roughly 8%.

The research is not limited to the level of AI (or LLM) capabilities, but focuses on how to effectively introduce AI into management decision-making on the basis of existing AI capabilities, so as to accelerate the pace of AI implementation and application. The relevant evaluation indicators of AI decision-making capability and their usage logic proposed in this article can not only be applied to decision screening in other scenarios, but also adapt to the dynamic evolution of LLM capability. After significant changes in AI capability, the threshold for AI decision screening can be reanalyzed and calculated, achieving continuous improvement in the synergy level of the "AI-human" systems.

, authors=Hui-bin SHI1, Cheng-han LI2, authorsList=Hui-bin SHI, Cheng-han LI, authorCompany=null, correspAuthors=Cheng-han LI, authorNote=null, correspAuthorsNote=null, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1316695250656149612, articleId=1316695248374448234, tenantId=1146029695717560320, journalId=1308397381743640680, language=CN, title=AI和人类顺序性协同的管理决策过程——来自期刊论文初审筛选的启示, columnId=1316695226064949848, journalTitle=科学学研究, columnName=专稿, runingTitle=null, highlight=null, articleAbstract=

本文深入期刊论文初审筛选决策的具体过程,使用大语言模型,针对“先AI后人类”的顺序性管理决策,提出了提高“人类-AI”系统协同性的决策方法。本文指出“先AI后人类”的顺序性管理决策的关键是在对AI决策方案筛选效能评估基础上,设计AI的决策方案筛选阈值。在引入和设计了spearman相关系数、优序比较系数、灵敏性、特异性、准确性及AI最小保留率指标后,使用聚合多个人类主体的评价基准,完成了确立决策方案筛选阈值的过程。本文对一个期刊三组论文的分析表明,利用现有大语言模型的论文初审筛选决策中,通过保留75%的论文,就能够覆盖相关评估基准中的高质量论文,从而使编辑的初审工作量降低约25%,编辑总体工作效率提高约15%,编审全流程效率提高约8%。本文提出的决策方法能够适应不同业务场景以及大语言模型能力的动态演进,具有较强的适用性。

, authors=史会斌1, 李成汉2, authorsList=史会斌, 李成汉, authorCompany=null, correspAuthors=李成汉, authorNote=

史会斌(1978-),男,讲师,博士。

, correspAuthorsNote=
李成汉(1992-),男,博士研究生, E-mail:。
, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=8Z/DPSdIGs2s6Mca4YlbGg==, magXml=jko9pg6Dv2D7H9DxJK/1bQ==, pdfUrl=null, pdf=cwtc1fxHRkKOJyBvPGfnWg==, pdfFileSize=1003414, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=yHp7fbswFSBrfJmAtj4+2A==, mapNumber=null, fund=null)}, authors=[Author(id=1316695251352404084, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1316695251432095862, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, authorId=1316695251352404084, language=EN, stringName=Hui-bin SHI, firstName=Hui-bin, middleName=null, lastName=SHI, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1 School of Economics and Management, Xidian University, Xi’an 710126, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1316695251528564855, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, authorId=1316695251352404084, language=CN, stringName=史会斌, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1 西安电子科技大学经济与管理学院, 陕西西安 710126, bio={"content":"

史会斌(1978-),男,讲师,博士。

"}, bioImg=null, bioContent=

史会斌(1978-),男,讲师,博士。

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1316695251058802797, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, xref=1, ext=[AuthorCompanyExt(id=1316695251062997102, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251058802797, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 School of Economics and Management, Xidian University, Xi’an 710126, China), AuthorCompanyExt(id=1316695251071385711, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251058802797, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 西安电子科技大学经济与管理学院, 陕西西安 710126)])]), Author(id=1316695251633422457, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=lichenghan@tongji.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1316695251759251579, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, authorId=1316695251633422457, language=EN, stringName=Cheng-han LI, firstName=Cheng-han, middleName=null, lastName=LI, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2 School of Economics and Management, Tongji University, Shanghai 200092, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1316695251872497788, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, authorId=1316695251633422457, language=CN, stringName=李成汉, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2 同济大学经济与管理学院, 上海 200092, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1316695251264323696, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, xref=2, ext=[AuthorCompanyExt(id=1316695251276906609, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251264323696, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 School of Economics and Management, Tongji University, Shanghai 200092, China), AuthorCompanyExt(id=1316695251297878130, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251264323696, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 同济大学经济与管理学院, 上海 200092)])])], keywords=[Keyword(id=1316695251981549693, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, orderNo=1, keyword=AI-Human Collaboration), Keyword(id=1316695252057047166, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, orderNo=2, keyword=management decision-making), Keyword(id=1316695252128350335, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, orderNo=3, keyword=peer review), Keyword(id=1316695252229013632, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, orderNo=4, keyword=screening effectiveness), Keyword(id=1316695252308705409, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, orderNo=5, keyword=minimum retention rate), Keyword(id=1316695252396785794, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, orderNo=1, keyword=人机协同), Keyword(id=1316695252459700355, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, orderNo=2, keyword=管理决策), Keyword(id=1316695252539392132, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, orderNo=3, keyword=期刊论文评审), Keyword(id=1316695252635861125, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, orderNo=4, keyword=筛选效能), Keyword(id=1316695252736524422, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, orderNo=5, keyword=最小保留率)], refs=[Reference(id=1316695256532369562, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2021, volume=106, issue=null, pageStart=102312, pageEnd=null, url=null, language=null, rfNumber=[1], rfOrder=0, authorNames=Cao G, Duan Y, Edwards J S, journalName=Technovation, refType=null, unstructuredReference=Cao G, Duan Y, Edwards J S, et al. Understanding managers’ attitudes and behavioral intentions towards using artificial intelligence for organizational decision-making[J]. Technovation, 2021, 106: 102312., articleTitle=Understanding managers’ attitudes and behavioral intentions towards using artificial intelligence for organizational decision-making, refAbstract=null), Reference(id=1316695256612061339, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=46, issue=10, pageStart=18, pageEnd=38, url=null, language=null, rfNumber=[2], rfOrder=1, authorNames=张亚莉, 李辽辽, 丁振斌, journalName=外国经济与管理, refType=null, unstructuredReference=张亚莉, 李辽辽, 丁振斌. 组织管理中的人工智能决策:述评与展望[J]. 外国经济与管理, 2024, 46(10): 18-38., articleTitle=组织管理中的人工智能决策:述评与展望, refAbstract=null), Reference(id=1316695256691753116, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=46, issue=10, pageStart=18, pageEnd=38, url=null, language=null, rfNumber=[2], rfOrder=2, authorNames=Zhang Y, Li L, Ding Z, journalName=Foreign Economics & Management, refType=null, unstructuredReference=Zhang Y, Li L, Ding Z. AI decision-making in organizational management: A review and prospects[J]. Foreign Economics & Management, 2024, 46(10): 18-38., articleTitle=AI decision-making in organizational management: A review and prospects, refAbstract=null), Reference(id=1316695256758861981, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2025, volume=33, issue=5, pageStart=99, pageEnd=112, url=null, language=null, rfNumber=[3], rfOrder=3, authorNames=谷炜, 刘亚金, Lu F S, journalName=中国管理科学, refType=null, unstructuredReference=谷炜, 刘亚金, Lu F S, 等. 人工智能驱动管理决策:应用、感知与偏见[J]. 中国管理科学, 2025, 33(5): 99-112., articleTitle=人工智能驱动管理决策:应用、感知与偏见, refAbstract=null), Reference(id=1316695256863719582, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2025, volume=33, issue=5, pageStart=99, pageEnd=112, url=null, language=null, rfNumber=[3], rfOrder=4, authorNames=Gu W, Liu Y, Lu F S, journalName=Chinese Journal of Management Science, refType=null, unstructuredReference=Gu W, Liu Y, Lu F S, et al. AI-driven decision sciences: Application, perception and bias[J]. Chinese Journal of Management Science, 2025, 33(5): 99-112., articleTitle=AI-driven decision sciences: Application, perception and bias, refAbstract=null), Reference(id=1316695256943411359, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2025, volume=46, issue=3, pageStart=583, pageEnd=610, url=null, language=null, rfNumber=[4], rfOrder=5, authorNames=Doshi A R, Bell J J, Mirzayev E, journalName=Strategic Management Journal, refType=null, unstructuredReference=Doshi A R, Bell J J, Mirzayev E, et al. Generative artificial intelligence and evaluating strategic decisions[J]. Strategic Management Journal, 2025, 46(3): 583-610., articleTitle=Generative artificial intelligence and evaluating strategic decisions, refAbstract=null), Reference(id=1316695257039880352, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2021, volume=46, issue=1, pageStart=192, pageEnd=210, url=null, language=null, rfNumber=[5], rfOrder=6, authorNames=Raisch S, Krakowski S, journalName=Academy of Management Review, refType=null, unstructuredReference=Raisch S, Krakowski S. Artificial intelligence and management: The automation-augmentation paradox[J]. Academy of Management Review, 2021, 46(1): 192-210., articleTitle=Artificial intelligence and management: The automation-augmentation paradox, refAbstract=null), Reference(id=1316695257161515169, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2025, volume=50, issue=2, pageStart=441, pageEnd=464, url=null, language=null, rfNumber=[6], rfOrder=7, authorNames=Raisch S, Kateryna F, journalName=Academy of Management Review, refType=null, unstructuredReference=Raisch S, Kateryna F. Combining human and artificial intelligence: Hybrid problem-solving in organizations[J]. Academy of Management Review, 2025, 50(2): 441-464., articleTitle=Combining human and artificial intelligence: Hybrid problem-solving in organizations, refAbstract=null), Reference(id=1316695257237012642, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2019, volume=61, issue=4, pageStart=66, pageEnd=83, url=null, language=null, rfNumber=[7], rfOrder=8, authorNames=Shrestha Y R, Ben-Menahem S M, Von Krogh G, journalName=California Management Review, refType=null, unstructuredReference=Shrestha Y R, Ben-Menahem S M, Von Krogh G. Organizational decision-making structures in the age of artificial intelligence[J]. California Management Review, 2019, 61(4): 66-83., articleTitle=Organizational decision-making structures in the age of artificial intelligence, refAbstract=null), Reference(id=1316695257350258851, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=8, issue=12, pageStart=2293, pageEnd=2303, url=null, language=null, rfNumber=[8], rfOrder=9, authorNames=Vaccaro M, Almaatouq A, Malone T, journalName=Nature Human Behaviour, refType=null, unstructuredReference=Vaccaro M, Almaatouq A, Malone T. When combinations of humans and AI are useful: A systematic review and meta-analysis[J]. Nature Human Behaviour, 2024, 8(12): 2293-2303., articleTitle=When combinations of humans and AI are useful: A systematic review and meta-analysis, refAbstract=null), Reference(id=1316695257450922148, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=37, issue=1, pageStart=4, pageEnd=12, url=null, language=null, rfNumber=[9], rfOrder=10, authorNames=Kousha K, Thelwall M, journalName=Learned Publishing, refType=null, unstructuredReference=Kousha K, Thelwall M. Artificial intelligence to support publishing and peer review: A summary and review[J]. Learned Publishing, 2024, 37(1): 4-12., articleTitle=Artificial intelligence to support publishing and peer review: A summary and review, refAbstract=null), Reference(id=1316695257539002533, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=15, issue=4, pageStart=616, pageEnd=626, url=null, language=null, rfNumber=[10], rfOrder=11, authorNames=Khraisha Q, Put S, Kappenberg J, journalName=Research Synthesis Methods, refType=null, unstructuredReference=Khraisha Q, Put S, Kappenberg J, et al. Can large language models replace humans in systematic reviews? Evaluating GPT‐4's efficacy in screening and extracting data from peer‐reviewed and grey literature in multiple languages[J]. Research Synthesis Methods, 2024, 15(4): 616-626., articleTitle=Can large language models replace humans in systematic reviews? Evaluating GPT‐4's efficacy in screening and extracting data from peer‐reviewed and grey literature in multiple languages, refAbstract=null), Reference(id=1316695257669025958, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2025, volume=25, issue=3, pageStart=1, pageEnd=23, url=null, language=null, rfNumber=[11], rfOrder=12, authorNames=Thelwall M, Abdallah Y, journalName=Scientometrics, refType=null, unstructuredReference=Thelwall M, Abdallah Y. Evaluating the predictive capacity of ChatGPT for academic peer review outcomes across multiple platforms[J]. Scientometrics, 2025, 25(3): 1-23., articleTitle=Evaluating the predictive capacity of ChatGPT for academic peer review outcomes across multiple platforms, refAbstract=null), Reference(id=1316695259359330472, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2025, volume=32, issue=4, pageStart=616, pageEnd=625, url=null, language=null, rfNumber=[12], rfOrder=13, authorNames=Li Y, Datta S, Rastegar-Mojarad M, journalName=Journal of the American Medical Informatics Association, refType=null, unstructuredReference=Li Y, Datta S, Rastegar-Mojarad M, et al. Enhancing systematic literature reviews with generative artificial intelligence: Development, applications, and performance evaluation[J]. Journal of the American Medical Informatics Association, 2025, 32(4): 616-625., articleTitle=Enhancing systematic literature reviews with generative artificial intelligence: Development, applications, and performance evaluation, refAbstract=null), Reference(id=1316695259464188073, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=[13], rfOrder=14, authorNames=Latona G R, Ribeiro M H, Davidson T R, journalName=arXiv preprint arXiv, refType=null, unstructuredReference=Latona G R, Ribeiro M H, Davidson T R, et al. The AI review lottery: Widespread AI-assisted peer reviews boost paper scores and acceptance rates[J]. arXiv preprint arXiv, 2024, 2405.02150., articleTitle=The AI review lottery: Widespread AI-assisted peer reviews boost paper scores and acceptance rates, refAbstract=null), Reference(id=1316695259585822890, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=35, issue=5, pageStart=1589, pageEnd=1607, url=null, language=null, rfNumber=[14], rfOrder=15, authorNames=Boussioux L, Lane J N, Zhang M, journalName=Organization Science, refType=null, unstructuredReference=Boussioux L, Lane J N, Zhang M, et al. The crowdless future? Generative AI and creative problem-solving[J]. Organization Science, 2024, 35(5): 1589-1607., articleTitle=The crowdless future? Generative AI and creative problem-solving, refAbstract=null), Reference(id=1316695259682291883, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2023, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=[15], rfOrder=16, authorNames=Liang W, Zhang Y, Cao H, journalName=arXiv preprint arXiv, refType=null, unstructuredReference=Liang W, Zhang Y, Cao H, et al. Can Large Language Models provide useful feedback on research papers? A large-scale empirical analysis[J]. arXiv preprint arXiv, 2023, 2310.01783., articleTitle=Can Large Language Models provide useful feedback on research papers? A large-scale empirical analysis, refAbstract=null), Reference(id=1316695259753595052, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=null, issue=null, pageStart=9340, pageEnd=9351, url=null, language=null, rfNumber=[16], rfOrder=17, authorNames=Zhou R, Chen L, Yu K, journalName=Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), refType=null, unstructuredReference=Zhou R, Chen L, Yu K. Is LLM a Reliable Reviewer? A Comprehensive Evaluation of LLM on Automatic Paper Reviewing Tasks[C]. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 2024: 9340-9351., articleTitle=Is LLM a Reliable Reviewer? A Comprehensive Evaluation of LLM on Automatic Paper Reviewing Tasks, refAbstract=null), Reference(id=1316695259837481133, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=[17], rfOrder=18, authorNames=Koren M, journalName=Management Science, refType=null, unstructuredReference=Koren M. The gatekeeper effect: The implications of pre-screening, self-selection, and bias for hiring processes[J]. Management Science, 2024, Published Online., articleTitle=The gatekeeper effect: The implications of pre-screening, self-selection, and bias for hiring processes, refAbstract=null), Reference(id=1316695259929755822, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2024, volume=67, issue=1, pageStart=5, pageEnd=32, url=null, language=null, rfNumber=[18], rfOrder=19, authorNames=Jia N, Luo X, Fang Z, journalName=Academy of Management Journal, refType=null, unstructuredReference=Jia N, Luo X, Fang Z, et al. When and how artificial intelligence augments employee creativity[J]. Academy of Management Journal, 2024, 67(1): 5-32., articleTitle=When and how artificial intelligence augments employee creativity, refAbstract=null), Reference(id=1316695260009447599, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, doi=null, pmid=null, pmcid=null, year=2025, volume=71, issue=5, pageStart=3641, pageEnd=3666, url=null, language=null, rfNumber=[19], rfOrder=20, authorNames=Law K K, Shen M, journalName=Management Science, refType=null, unstructuredReference=Law K K, Shen M. How does artificial intelligence shape audit firms[J]. Management Science, 2025, 71(5): 3641-3666., articleTitle=How does artificial intelligence shape audit firms, refAbstract=null)], funds=[Fund(id=1316695256440094873, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, awardId=23YJC630150, language=CN, fundingSource=教育部人文社会科学基金青年基金项目(23YJC630150), fundOrder=null, country=null)], companyList=[AuthorCompany(id=1316695251058802797, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, xref=1, ext=[AuthorCompanyExt(id=1316695251062997102, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251058802797, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 School of Economics and Management, Xidian University, Xi’an 710126, China), AuthorCompanyExt(id=1316695251071385711, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251058802797, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 西安电子科技大学经济与管理学院, 陕西西安 710126)]), AuthorCompany(id=1316695251264323696, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, xref=2, ext=[AuthorCompanyExt(id=1316695251276906609, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251264323696, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 School of Economics and Management, Tongji University, Shanghai 200092, China), AuthorCompanyExt(id=1316695251297878130, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251264323696, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 同济大学经济与管理学院, 上海 200092)])], figs=[ArticleFig(id=1316695252950433927, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 1, caption=

Sample overview of this study

, figureFileSmall=null, figureFileBig=null, tableContent=
组别 论文总数 编辑初审拒稿数 编辑初审拒稿率 编辑初审保留率 外审/复审论文数 最终录用数
A组 13 4 31% 69% 9 1
B组 12 4 33% 67% 8 1
C组 13 5 38% 62% 8 1
), ArticleFig(id=1316695253021737096, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表1, caption=

本研究样本概况

, figureFileSmall=null, figureFileBig=null, tableContent=
组别 论文总数 编辑初审拒稿数 编辑初审拒稿率 编辑初审保留率 外审/复审论文数 最终录用数
A组 13 4 31% 69% 9 1
B组 12 4 33% 67% 8 1
C组 13 5 38% 62% 8 1
), ArticleFig(id=1316695253097234569, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 2, caption=

Determination of the proportion of papers screened by AI

, figureFileSmall=null, figureFileBig=null, tableContent=
序号 比例名称 比例解释
1 AI高质量论文比例 AI排序的前1/3论文
2 AI最小保留率比例 AI排序的前x%论文,如果刚好能够覆盖某一个人类基准所包含的全部论文,就说x%是这个基准的AI最小保留率
), ArticleFig(id=1316695253185314954, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表2, caption=

AI筛选论文比例的确定

, figureFileSmall=null, figureFileBig=null, tableContent=
序号 比例名称 比例解释
1 AI高质量论文比例 AI排序的前1/3论文
2 AI最小保留率比例 AI排序的前x%论文,如果刚好能够覆盖某一个人类基准所包含的全部论文,就说x%是这个基准的AI最小保留率
), ArticleFig(id=1316695254816899211, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 3, caption=

Benchmarks for evaluating the effectiveness of AI paper screening

, figureFileSmall=null, figureFileBig=null, tableContent=
序号 基准名称 基准解释
1 编辑部独立基准 包含编辑部排序的前1/3论文
2 编辑部与专家合并基准 编辑部、专家1、专家2各自排序前1/3论文的并集
3 专家增强编辑部的基准 专家1、专家2各自排序前1/3论文的交集,再与编辑部排序的前1/3论文合并
4 编辑部与专家共同决策的基准 包含被编辑部、专家1和专家2三个主体中至少两个主体排序前1/3的论文。
), ArticleFig(id=1316695254972088460, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表3, caption=

评价AI论文筛选效能的基准

, figureFileSmall=null, figureFileBig=null, tableContent=
序号 基准名称 基准解释
1 编辑部独立基准 包含编辑部排序的前1/3论文
2 编辑部与专家合并基准 编辑部、专家1、专家2各自排序前1/3论文的并集
3 专家增强编辑部的基准 专家1、专家2各自排序前1/3论文的交集,再与编辑部排序的前1/3论文合并
4 编辑部与专家共同决策的基准 包含被编辑部、专家1和专家2三个主体中至少两个主体排序前1/3的论文。
), ArticleFig(id=1316695255093723277, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 4, caption=

Spearman correlation of various subjects on group A papers

, figureFileSmall=null, figureFileBig=null, tableContent=
评价主体 编辑部 专家1 专家2 AI
编辑部 1.0
专家1 0.00 1.0
专家2 0.25 -0.13 1.0
AI 0.15 0.15 0.23 1.0
), ArticleFig(id=1316695255206969486, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表4, caption=

A组论文各主体排序的相关性

, figureFileSmall=null, figureFileBig=null, tableContent=
评价主体 编辑部 专家1 专家2 AI
编辑部 1.0
专家1 0.00 1.0
专家2 0.25 -0.13 1.0
AI 0.15 0.15 0.23 1.0
), ArticleFig(id=1316695255290855567, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 5, caption=

Spearman correlation of various subjects on group B papers

, figureFileSmall=null, figureFileBig=null, tableContent=
评价主体 编辑部 专家1 专家2 AI
编辑部 1.0
专家1 0.02 1.0
专家2 -0.49 -0.17 1.0
AI -0.25 -0.12 0.14 1.0
), ArticleFig(id=1316695255399907472, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表5, caption=

B组论文各主体排序的相关性

, figureFileSmall=null, figureFileBig=null, tableContent=
评价主体 编辑部 专家1 专家2 AI
编辑部 1.0
专家1 0.02 1.0
专家2 -0.49 -0.17 1.0
AI -0.25 -0.12 0.14 1.0
), ArticleFig(id=1316695255508959377, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 6, caption=

Priority comparison coefficient of various subjects on group A papers

, figureFileSmall=null, figureFileBig=null, tableContent=
编辑部 专家1 专家2 AI
编辑部 1.0
专家1 0.500 1.0
专家2 0.583 0.472 1.0
AI 0.500 0.556 0.583 1.0
), ArticleFig(id=1316695255727063186, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表6, caption=

A组论文各主体排序的优序比较系数

, figureFileSmall=null, figureFileBig=null, tableContent=
编辑部 专家1 专家2 AI
编辑部 1.0
专家1 0.500 1.0
专家2 0.583 0.472 1.0
AI 0.500 0.556 0.583 1.0
), ArticleFig(id=1316695255815143571, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 7, caption=

Priority comparison coefficient of various subjects on group B papers

, figureFileSmall=null, figureFileBig=null, tableContent=
编辑部 专家1 专家2 AI
编辑部 1.0
专家1 0.500 1.0
专家2 0.286 0.429 1.0
AI 0.429 0.464 0.536 1.0
), ArticleFig(id=1316695255932584084, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表7, caption=

B组论文各主体排序的优序比较系数

, figureFileSmall=null, figureFileBig=null, tableContent=
编辑部 专家1 专家2 AI
编辑部 1.0
专家1 0.500 1.0
专家2 0.286 0.429 1.0
AI 0.429 0.464 0.536 1.0
), ArticleFig(id=1316695256037441685, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 8, caption=

The AI screening efficiency of group A papers under different benchmarks

, figureFileSmall=null, figureFileBig=null, tableContent=
比较基准 保留AI排序的前1/3实现的筛选效能 AI最小保留率
灵敏性 特异性 准确性
编辑部独立基准 0.25 0.67 0.54 0.77
编辑部与专家合并基准 0.33 0.75 0.46 0.92
专家增强编辑部的基准 0.4 0.75 0.62 0.77
编辑部与专家共同决策基准 0.33 0.7 0.62 0.77
), ArticleFig(id=1316695256133910678, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表8, caption=

A组论文在不同基准下的AI筛选效能

, figureFileSmall=null, figureFileBig=null, tableContent=
比较基准 保留AI排序的前1/3实现的筛选效能 AI最小保留率
灵敏性 特异性 准确性
编辑部独立基准 0.25 0.67 0.54 0.77
编辑部与专家合并基准 0.33 0.75 0.46 0.92
专家增强编辑部的基准 0.4 0.75 0.62 0.77
编辑部与专家共同决策基准 0.33 0.7 0.62 0.77
), ArticleFig(id=1316695256221991063, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=EN, label=Table 9, caption=

The AI screening efficiency of group A papers under different benchmarks

, figureFileSmall=null, figureFileBig=null, tableContent=
比较基准 保留AI排序的前1/3实现的筛选效能 AI最小保留率
灵敏性 特异性 准确性
编辑部独立基准 0.25 0.63 0.5 1
编辑部与专家合并基准 0.44 1 0.58 1
专家增强编辑部的基准 0.5 0.83 0.67 1
编辑部与专家共同决策基准 0.67 0.78 0.75 0.75
), ArticleFig(id=1316695256326848664, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, language=CN, label=表9, caption=

B组论文在不同基准下的AI筛选效能

, figureFileSmall=null, figureFileBig=null, tableContent=
比较基准 保留AI排序的前1/3实现的筛选效能 AI最小保留率
灵敏性 特异性 准确性
编辑部独立基准 0.25 0.63 0.5 1
编辑部与专家合并基准 0.44 1 0.58 1
专家增强编辑部的基准 0.5 0.83 0.67 1
编辑部与专家共同决策基准 0.67 0.78 0.75 0.75
)], attaches=null, journal=Journal(id=1308396458388586594, delFlag=0, nameCn=科学学研究, nameEn=Studies in Science of Science, nameHistory1=null, nameHistory2=null, issn=1003-2053, eissn=null, cn=11-1805/G3, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=f9wd5Gr2r/Af7TJ15O/VGg==, journalPrice=null, startedYear=null, abbrevIsoEn=Studies in Science of Science, journalRemark=null, publicationField=null, createdTime=1789875624512, updatedTime=1789876606404, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=K, firstLetterEn=K, subjectCode=Multidisciplinary, subjectName=null, subjectCodeEn=Multidisciplinary, subjectNameEn=null, picCn=f9wd5Gr2r/Af7TJ15O/VGg==, picEn=XwgD5UmFTEVSmdnCgvlDZQ==, jcr=null, cjcr=null, exts=[JournalExt(id=1308400576836957147, language=CN, name=科学学研究, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1789876606425, updatedTime=1789876606425, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://kxxyj.magtechjournal.com/journalx_kxxyj/authorLogOn.action, submissionEditorUrl=https://kxxyj.magtechjournal.com/journalx_kxxyj/editorLogOn.action, submissionReviewUrl=https://kxxyj.magtechjournal.com/journalx_kxxyj/expertLogOn.action, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1308400576883094492, language=EN, name=Studies in Science of Science, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1789876606436, updatedTime=1789876606436, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://kxxyj.magtechjournal.com/journalx_kxxyj/authorLogOn.action, submissionEditorUrl=https://kxxyj.magtechjournal.com/journalx_kxxyj/editorLogOn.action, submissionReviewUrl=https://kxxyj.magtechjournal.com/journalx_kxxyj/expertLogOn.action, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1308397381743640680, websiteList=[Website(id=1308429426446848915, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1308397381743640680, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/kxxyj/CN, language=CN, createTime=1789883484708, createBy=18614031015, updateTime=1789902368384, updateBy=18614031015, name=科学学研究-中文, tplId=1146099689490845704, title=科学学研究, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1308508734821851572, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=articleTextType, value=kx, createTime=1789902393298, updateTime=1789902393298, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734784102833, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=banner, value=null, createTime=1789902393289, updateTime=1789902393289, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734872183223, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=grayFlag, value=0, createTime=1789902393310, updateTime=1789902393310, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734771519920, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=logo, value=https://castjournals.cast.org.cn/joweb/kxxyj/CN/file/pic?fileId=DFYjiBGwM+7bU4q3P0DayQ==, createTime=1789902393286, updateTime=1789902393286, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734888960441, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=minRunFlag, value=0, createTime=1789902393314, updateTime=1789902393314, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734809268659, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/kxxyj/CN/file/pic, createTime=1789902393295, updateTime=1789902393295, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734880571832, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=silenceFlag, value=0, createTime=1789902393312, updateTime=1789902393312, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734796685746, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1789902393292, updateTime=1789902393292, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734834434485, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=themeColor, value=null, createTime=1789902393301, updateTime=1789902393301, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508734851211702, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426446848915, code=themeStyle, value=null, createTime=1789902393305, updateTime=1789902393305, creator=18614031015, updator=18614031015)]), Website(id=1308429426513957785, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1308397381743640680, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/kxxyj/EN, language=EN, createTime=1789883484724, createBy=18614031015, updateTime=1789902358215, updateBy=18614031015, name=科学学研究-英文, tplId=1146101810881728533, title=Studies in Science of Science, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1308508708242547113, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=articleTextType, value=kx, createTime=1789902386961, updateTime=1789902386961, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708213186981, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=banner, value=null, createTime=1789902386954, updateTime=1789902386954, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708267712940, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=grayFlag, value=0, createTime=1789902386967, updateTime=1789902386967, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708200604068, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=logo, value=https://castjournals.cast.org.cn/joweb/kxxyj/EN/file/pic?fileId=DFYjiBGwM+7bU4q3P0DayQ==, createTime=1789902386951, updateTime=1789902386951, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708280295854, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=minRunFlag, value=0, createTime=1789902386970, updateTime=1789902386970, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708229964199, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/kxxyj/EN/file/pic, createTime=1789902386959, updateTime=1789902386959, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708271907245, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=silenceFlag, value=0, createTime=1789902386968, updateTime=1789902386968, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708217381286, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1789902386955, updateTime=1789902386955, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708246741418, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=themeColor, value=null, createTime=1789902386962, updateTime=1789902386962, creator=18614031015, updator=18614031015), WebsiteProps(id=1308508708259324331, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1308429426513957785, code=themeStyle, value=null, createTime=1789902386965, updateTime=1789902386965, creator=18614031015, updator=18614031015)])], journalTitle=科学学研究, weixinUrl=null, journalUrl=https://kxxyj.magtechjournal.com/kxxyj, iacademicId=null, status=1, seqNo=null, journalTitleEn=Studies in Science of Science, journalPhotoCn=f9wd5Gr2r/Af7TJ15O/VGg==, journalPhotoEn=XwgD5UmFTEVSmdnCgvlDZQ==, journalFirstLetter=K, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/kxxyj/CN/Y2026/V44/I1/41, detailUrlEn=https://castjournals.cast.org.cn/joweb/kxxyj/EN/Y2026/V44/I1/41, pdfUrlCn=https://castjournals.cast.org.cn/joweb/kxxyj/CN/PDF/Y2026/V44/I1/41, pdfUrlEn=https://castjournals.cast.org.cn/joweb/kxxyj/EN/PDF/Y2026/V44/I1/41, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1768406400000, fullTextJson=null, articleText=null, reference=null)
收藏切换
AI和人类顺序性协同的管理决策过程——来自期刊论文初审筛选的启示
收藏切换
PDF下载
史会斌 1 , 李成汉 2
科学学研究 | 专稿 2026,44(1): 41-50
收起
收藏切换
科学学研究 |专稿 2026 , 44 (1) : 41 -50
AI和人类顺序性协同的管理决策过程——来自期刊论文初审筛选的启示
全屏
[Author(id=1316695251352404084, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1316695251432095862, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, authorId=1316695251352404084, language=EN, stringName=Hui-bin SHI, firstName=Hui-bin, middleName=null, lastName=SHI, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1 School of Economics and Management, Xidian University, Xi’an 710126, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1316695251528564855, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, authorId=1316695251352404084, language=CN, stringName=史会斌, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1 西安电子科技大学经济与管理学院, 陕西西安 710126, bio={"content":"

史会斌(1978-),男,讲师,博士。

"}, bioImg=null, bioContent=

史会斌(1978-),男,讲师,博士。

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1316695251058802797, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, xref=1, ext=[AuthorCompanyExt(id=1316695251062997102, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251058802797, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 School of Economics and Management, Xidian University, Xi’an 710126, China), AuthorCompanyExt(id=1316695251071385711, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251058802797, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1 西安电子科技大学经济与管理学院, 陕西西安 710126)])]), Author(id=1316695251633422457, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=lichenghan@tongji.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1316695251759251579, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, authorId=1316695251633422457, language=EN, stringName=Cheng-han LI, firstName=Cheng-han, middleName=null, lastName=LI, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2 School of Economics and Management, Tongji University, Shanghai 200092, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1316695251872497788, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, authorId=1316695251633422457, language=CN, stringName=李成汉, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2 同济大学经济与管理学院, 上海 200092, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1316695251264323696, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, xref=2, ext=[AuthorCompanyExt(id=1316695251276906609, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251264323696, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 School of Economics and Management, Tongji University, Shanghai 200092, China), AuthorCompanyExt(id=1316695251297878130, tenantId=1146029695717560320, journalId=1308397381743640680, articleId=1316695248374448234, companyId=1316695251264323696, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2 同济大学经济与管理学院, 上海 200092)])])]
史会斌1, 李成汉2
作者信息
  • 1 西安电子科技大学经济与管理学院, 陕西西安 710126
  • 2 同济大学经济与管理学院, 上海 200092
通讯作者:
李成汉(1992-),男,博士研究生, E-mail:。
作者简介:

史会斌(1978-),男,讲师,博士。

The process of sequential AI-Human collaborative decision-making——Insights from initial manuscript screening in journal peer review
Hui-bin SHI1, Cheng-han LI2
Affiliations
  • 1 School of Economics and Management, Xidian University, Xi’an 710126, China
  • 2 School of Economics and Management, Tongji University, Shanghai 200092, China
出版时间: 2026-01-15
文章导航
收藏切换
摘要
收起

本文深入期刊论文初审筛选决策的具体过程,使用大语言模型,针对“先AI后人类”的顺序性管理决策,提出了提高“人类-AI”系统协同性的决策方法。本文指出“先AI后人类”的顺序性管理决策的关键是在对AI决策方案筛选效能评估基础上,设计AI的决策方案筛选阈值。在引入和设计了spearman相关系数、优序比较系数、灵敏性、特异性、准确性及AI最小保留率指标后,使用聚合多个人类主体的评价基准,完成了确立决策方案筛选阈值的过程。本文对一个期刊三组论文的分析表明,利用现有大语言模型的论文初审筛选决策中,通过保留75%的论文,就能够覆盖相关评估基准中的高质量论文,从而使编辑的初审工作量降低约25%,编辑总体工作效率提高约15%,编审全流程效率提高约8%。本文提出的决策方法能够适应不同业务场景以及大语言模型能力的动态演进,具有较强的适用性。

人机协同  /  管理决策  /  期刊论文评审  /  筛选效能  /  最小保留率

The widespread application of the "AI-human" system in management decision-making has sparked discussions on enhancing the synergy of the "AI-human" system to improve management decision-making performance. Although existing research has proposed some methods to improve the synergy of the "AI-human" system in management decision-making, there is still a lack of validation in specific business scenarios. This article chooses the initial screening of journal articles as a scenario and delve into the specific process of sequential AI-Human collaboration to exploring the path to improving the synergy of the "AI-human" system in management decision-making.

This article proposes that we can use the scoring results of AI to eliminate some low-quality submitted papers before the editor starts the initial screening work. Then, the editor screens the remaining papers with higher scores. In this process, AI’s filtering effect will reduce the number of papers reviewed by editors. Therefore, we can use the sequential AI-Human collaborative mode to improve the overall efficiency of paper review work. That is to say, the key to this type decision-making lies in designing AI screening thresholds based on an evaluation of AI's screening effectiveness.

Firstly, this article introduces Spearman correlation and innovatively designs priority comparison coefficient to evaluate the overall decision-making efficiency of AI. Secondly, by defining the negative and positive states of the screening results, sensitivity, specificity, and accuracy indicators were introduced to evaluate the AI’s decision screening effectiveness; Then, the design of evaluating benchmarks is carried out by aggregating human collective judgments, and it is proposed that the judgments of multiple human experts can be combined, enhanced, and jointly determined to form different evaluation benchmarks. These benchmarks can avoid the limitation of using a single evaluation benchmark that cannot accurately reflect AI’s decision-making ability; Finally, by designing the minimum retention rate as an indicator, the threshold setting method for AI in screening decision was clarified. This indicator not only reflects AI’s screening effectiveness, but also enables the rapid implementation of AI applications in the specific scenario. The analysis of three groups of papers in a journal demonstrates that by retaining 75% of the papers during the initial screening phase using existing large language models, high-quality papers identified in relevant evaluation benchmarks can be effectively covered. This reduces editorial workload by approximately 25%, increases overall editorial efficiency by about 15%, and improves the efficiency of the entire editorial process by roughly 8%.

The research is not limited to the level of AI (or LLM) capabilities, but focuses on how to effectively introduce AI into management decision-making on the basis of existing AI capabilities, so as to accelerate the pace of AI implementation and application. The relevant evaluation indicators of AI decision-making capability and their usage logic proposed in this article can not only be applied to decision screening in other scenarios, but also adapt to the dynamic evolution of LLM capability. After significant changes in AI capability, the threshold for AI decision screening can be reanalyzed and calculated, achieving continuous improvement in the synergy level of the "AI-human" systems.

AI-Human Collaboration  /  management decision-making  /  peer review  /  screening effectiveness  /  minimum retention rate
史会斌, 李成汉. AI和人类顺序性协同的管理决策过程——来自期刊论文初审筛选的启示. 科学学研究, 2026 , 44 (1) : 41 -50 .
Hui-bin SHI, Cheng-han LI. The process of sequential AI-Human collaborative decision-making——Insights from initial manuscript screening in journal peer review[J]. Studies in Science of Science, 2026 , 44 (1) : 41 -50 .
正文
收起
人工智能(Artificial Intelligence,AI),特别是大语言模型(Large Language Model,LLM),以其强大的信息搜集能力及分析处理能力,在组织管理决策中发挥着越来越重要的作用[1],现有研究一方面表明AI技术能够在人力资源管理、运营管理、市场营销、会计与金融、战略管理等领域的决策当中发挥重要作用[2-4],另一方面也对AI应用于管理决策的模式进行了提炼,Raisch和Krakowski[5]指出,对AI在管理决策中的自动化及增强化功能不能简单的强调某一个方面,而需要使用悖论视角从时间和空间上对自动化及增强化进行综合管理;Raich和Fomina[6]进一步把“人类-AI”的混合型解决问题过程分为三个阶段,解决方案搜索前的目标设计及数据提供阶段和搜索后的选择阶段是由人类完成的,解决方案的搜索过程是一个“人类-AI”的协同过程,可以分为自主搜索、顺序搜索(先AI后人类)和交互搜索三种协同方式;与Raich和Fomina的观点类似,Shrestha等[7] 认为人类和AI协同的管理决策结构可划分为三种类型:完全授权AI决策、AI与人类顺序决策(包括先AI后人类和先人类后AI两种顺序)以及AI与人类聚合决策。虽然这些研究为开展“人类-AI”协同的管理决策提供了落地场景的扩展及理论指引,但是Vaccaro和Malone[8]开展的一个跨场景元分析发现,现有的“人类-AI”系统在管理决策上并没有产生很好的协同效应,其决策绩效低于人类或AI中的最佳者,这就提出了如何增强“人类-AI”系统的协同性,以提高管理决策绩效的问题。
关于如何提高“人类-AI”系统在管理决策中的协同性问题,Doshi等人[4]指出通过聚合多个LLM的结果,可以抵消单个LLM的偏差,提升AI整体判断的准确性。Vaccaro和Malone[8] 指出可以把决策任务分成不同的子任务,并依据人类和AI的优势进行子任务的分配;也可以避免单一化的绩效指标设计,开发更稳健的“人类-AI”系统评估指标;还可以设定人类、AI及“人类-AI”系统的质量阈值,对决策的质量进行约束。这些改进“人类-AI”系统协同性的建议需要结合具体场景验证其可行性。
本文选择期刊论文初审这一业务场景,深入“先AI后人类”顺序性协同的具体过程,探讨管理决策中提高“人类-AI”系统协同性的路径。首先明确提出,“先AI后人类”顺序性协同的管理决策当中,关键是要了解AI的能力范围,并据此设置AI对决策方案的筛选比例,然后在期刊论文初审的场景下,对AI决策能力的评估指标、评估基准、决策方案筛选的阈值进行了设计、分析和应用,为开展“先AI后人类”顺序性协同的管理决策提供了可操作的程序及方法,同时也为寻求提升“人类-AI”系统的协同性提供了可行路径。
1 研究设计
收起
期刊论文评审是保持学术届活力的重要基础,但是面对论文评审的繁重任务,学者们一直在探讨如何使用人工智能降低评审工作量,提升评审的效率和质量。虽然在LLM出现之前,学者们就开始探讨自行设计的人工智能算法在论文评审中的应用范围及可行性[9],但是由于LLM在阅读理解,知识整合,甚至逻辑推理方面表现出很强的能力,以及其天然的使用方便性,这就使得利用LLM改进甚至替代人类进行论文评审的研究大量出现[10]。现有研究已经发现LLM在论文质量的总体排序方面与人类具有一定的一致性[11,12],但是在当前的一致性水平下,如何将LLM应用于论文评审的具体过程还未很好地解决。另外现有研究也发现LLM有扩大评分的倾向[13],而且单一LLM生成的结果往往存在多样性不足[14]以及偏见性[4]的问题,因此完全依赖单一LLM评分进行论文筛选决策可能会存在评分准确性及淘汰率过低的问题。
期刊论文的评审主要包括初审、外审和复审三个阶段,初审由编辑做出拒稿或进行外审的决策,外审则邀请外审专家对论文进行评价,并给出评审意见,然后编辑依据外审结果进行复审,并做出录用决策。在论文的初审阶段,需要编辑全面评估每一篇论文的质量,在投稿量较大的情形下,将形成很大的工作量。本文认为如果我们能够利用AI的评分结果,在编辑开始初审工作之前就淘汰一些质量较低的投稿论文,把剩余评分较高的论文让编辑进行初审筛选,就能够发挥AI对论文质量的初步把关作用,并降低编辑评审的论文数量,从而利用“先AI后人类”的顺序协同模式,从整体上提高论文评审的工作效率。在这个过程中对AI的决策方案筛选能力评估成为开展协同决策的关键。
因此本文基于现有LLM的能力,综合多个LLM的评分结果,设计相关方法,评估LLM的评分结果能够在多大程度上用于准确筛选出质量较低的论文,以降低需要编辑初审的论文数量。具体来说,本文针对某一个管理类期刊,搜集了其在三个不同时期,编辑对三组提交论文的最终排序及录用结果。针对这些提交的论文,本文同时让两位专家独立重新进行了质量排序,也使用主流的LLM(包括Deepseek-R1、通义千问、智谱清言、豆包、文心一言)对每篇论文进行了评分,并形成AI排序。基于第一和第二组数据,本文首先使用spearman相关系数和优序比较系数对AI综合排序的效能进行评价;然后综合编辑部和两位人类专家排序中的高质量论文(各自排序的前1/3),形成4个高质量论文集合,作为对AI筛选效能评估的基准,并使用灵敏性(sensitivity)、特异性(specificity)、准确性(accuracy)、AI最小保留率4个指标对AI的筛选效能进行评估。评估的结果表明AI最小保留率约为75%时,筛选出的结果能够覆盖编辑部和人类专家相关基准中的全部高质量论文,这也是应用AI进行论文筛选时一个阈值。按照这一筛选效能,把AI应用于论文的初审,大约能提高初审效率25%,提高编辑总体工作效率15%,提高编审全流程效率8%。最后,利用第三组论文的数据,对本文提出的筛选阈值进行了验证,基本确认了本文在AI筛选效能评估和AI筛选阈值确立方面所设计的方法及其指标的可行性。
2 数据来源
收起
2.1 编辑部评审数据
本研究的样本基于某管理类期刊对投稿论文的评审记录,包括A、B、C三组不同时期论文的评审及录用结果。该期刊的论文评审主要包括三个阶段:初审、外审和复审,编辑依据外审结果进行复审时,会使用评价量表给出复审评价,并做出录用决策。在复审过程中还会存在论文的退修及再评审过程,由于这一过程与本文研究关系不大,因此本文在复审环节只考虑最终录用决策结果,而对复审过程中的其他环节不做考虑。
如表1所示,本研究获得的A组论文数量为13篇,编辑部的评审结果为:4篇初审拒稿(31%),9篇进入外审,最终1篇录用。B组论文数量为12篇,评审结果为:4篇初审拒稿(33%),8篇进入外审,最终1篇录用;C组论文数量为13篇,评审结果为:5篇初审拒稿(38%),8篇进入外审,最终1篇录用。初审拒稿的论文没有评分,进入复审的论文编辑会按照一个评价量表对各维度进行等级性评估。本研究将使用A组和B组论文对AI的论文初审筛选效能进行评估,并确立使用AI进行论文初审筛选的阈值,然后应用于对C组论文进行初审筛选。
2.2 人类专家评审数据
对于A组和B组的论文,本文分别邀请了两位专家对每组论文的质量从高到低进行排序。考虑到编辑部的结果只是综合了编辑和两位外审专家意见形成的结论,决策人数较少,只有3个,而管理决策的准确性会随着决策主体规模的增加而增加[4],因此本文将编辑部的评审结果与两位人类专家的结果进行综合后,形成对AI筛选效能评估的基准。
2.3 AI评审数据
我国有多家综合性能领先的LLM,综合考虑国内主流LLM平台对输入输出内容的审查制度和API接口的便捷性,本文选择市场评价较高的五个LLM:Deepseek-R1、通义千问、智谱清言、豆包、文心一言。
为了确保学术论文的质量,并筛选出符合期刊要求的稿件,本文尽量使LLM以统一的标准对论文进行评分。按照本文所选样本期刊的评审原则,最终将论文质量评价的标准汇总为问题前沿性,理论创新性及研究严谨性3个主要维度。Doshi等[4]的研究结果表明,对LLM进行明确的角色设置以及思维链条的引导能够获得更准确的结果,因此本文设计了提示词,要求LLM作为论文评审专家从问题前沿性、理论创新性以及研究严谨性这3个维度对论文进行打分,并对3个维度的得分取平均值得到每篇论文最终的质量评价分数(0-100)。
本文使用的提示词如下:
“你是一位论文期刊的评审专家,请你阅读全文(不需要阅读末尾的参考文献部分),从问题前沿性(1到25分代表研究问题很多人讨论过,相关成果很多;26到50分代表研究问题有人关注过,但相关成果不多;51到75分代表文章的研究问题之前较少人关注,相关成果很少;76分到100分则代表文章的研究问题十分前沿,几乎没有人提出过相关研究成果)、理论创新性(1到25分代表文章理论部分几乎没有创新;26到50分代表文章的理论部分验证了前人的理论成果,但创新仍然不足;51分到75分代表文章扩展了现有理论的边界,提出了一些建设性的意见;76分到100分代表文章填补了现有理论的空白,做出了突破性的理论贡献)和研究严谨性(1到25分代表文章的研究方法有严重漏洞;26到50分代表文章的研究方法有部分漏洞,需要完善;51到75分代表文章的研究方法较为严谨,但仍有可以改进的地方;76分到100分代表文章的研究方法十分完备,结论可靠)这3个维度分别对论文的质量进行评分,每个维度满分为100分,分数越高代表文章的质量越高,最后将3个维度的分数取平均得到论文的综合得分,不需要列出打分依据。”
由于LLM运行过程的“黑箱”问题,导致同一个提示词多次运行可能会得到不同的评分,因此,本文使用每一个LLM对每一篇论文评分5次,在去除最高分和最低分后,对其余三次打分结果取平均值作为该LLM对特定论文的最终评分。为了避免单一LLM评分的偏见性[4],我们将5个模型对特定论文的单独评分结果去除最高分和最低分之后,剩余的3个分数取平均值,代表5个LLM对该论文的综合评分,这样就得到了每一篇论文的AI质量评分,再按照这个评分对A、B、C三组论文分别排序,最终得到每组论文的AI质量排名。
3 评估AI初审筛选效能的方法
收起
3.1 AI总体评审效能评估方法
本文通过比较AI评分产生的排序与人类主体(包括编辑部、两位独立人类专家)评价形成的排序,对AI的总体评审效能进行评估,AI评审的排序结果与人类主体实际排序结果的一致性程度越高,表明AI的评审效能越高。
编辑部对论文的排序是依据编辑在复审过程中形成的评价量表产生的。本文把复审中评价量表结果转化为具体的分数,将论文质量从高到低进行排序。如果两篇论文的排名相同,例如都处于第三名的位置,那么这两篇论文的排名均为3.5。初审被淘汰的论文由于编辑未进行打分,不参与排名,默认排在复审论文之后。
spearman相关系数可以用于评估两个等级变量之间的相关性大小,有了各主体(编辑部、两为独立人类专家、AI)对论文的评分,就可以分析他们的spearman相关系数,从中可以初步判断AI评审结果的准确性。另外针对特定主体的排序,通过将每组内的论文进行两两配对比较,可以形成优序比较集合,例如对于A组中的论文Ai,AI的排名若高于该组中的论文Aj的排名,则Rij=R(Ai,Aj) = 1,反之为0,如果两篇论文排名相同,则R(Ai,Aj)=0.5。
由于只有进入复审的论文才有编辑部排序结果,所以A、B、C组分别有9篇、8篇、8篇论文参与一致性指标分析。针对每一个主体,A、B、C各组内的论文可以分别形成由72对、56对以及56对论文的比较结果构成的优序比较集合。最后,通过比较AI与人类优序比较集合的一致性,可以评估AI评审结果与人类结果在论文优劣比较上的差异或者共识程度,计算优序比较系数(consistency)的公式如公式(1)所示:
$\mathit{c}\mathit{o}\mathit{n}\mathit{s}\mathit{i}\mathit{s}\mathit{t}\mathit{e}\mathit{n}\mathit{c}\mathit{y}=1-\frac{{\sum }_{\mathit{i}=1}^{\mathit{n}}\sum _{\mathit{j}=1}^{\mathit{n}-1}\left|{\mathit{R}}_{\mathit{i}\mathit{j}}-\mathit{R}{\mathit{\text{'}}}_{\mathit{i}\mathit{j}}\right|}{\mathit{n}(\mathit{n}-1)}$
其中,n为组内的论文数量(在这里三组论文数分别为9,8,8),Rij和R'ij分别代表不同主体针对同一组论文评审结果形成的优序比较集合中的一个元素(不同主体的优序比较集合的排序要一致),consistency的值介于0和1之间,AI评审与人类主体对任意两篇论文优劣的共识程度越高,则consistency值越趋近于1。
3.2 评估AI初审筛选效能的方法
本文的出发点是利用AI降低编辑初审的工作量。如果认为AI按照某一保留率(排名前x%)识别出来的论文为高质量论文,那么通过比较AI保留下来的高质量论文与编辑部识别的高质量论文的重合率,就能够判断AI的筛选效能,重合率越高就说明AI在保留率为x%情形下的筛选效能越高。
在比较AI筛选结果与人类主体结果的重合率时,参考Khraisha等[10]的指标设计,本文使用到了4个筛选状态:真阳性(TP True Positives)、真阴性(TN True Negative)、假阳性(FP False Positives)、假阴性(TN True Negative)。这里的阳性指的是按照人类的基准应该保留的样本(高质量论文),阴性指的是按照人类的基准应该删除的样本(其余论文)。
真阳性TP,若AI按照前x%的保留率保留的某一篇论文与人类基准保留的某一篇论文重合,则说明AI对该论文的筛选结果是真阳性,也就是说AI与人类同时判断该论文应该保留。
真阴性TN,若AI按照(1- x%)的淘汰率淘汰的某一篇论文与人类基准淘汰的某一篇论文重合,则说明AI对该论文的筛选结果是真阴性,也就是说AI与人类同时判断该论文应该淘汰(退稿)。
假阳性FP,若AI按照x%的保留率保留的某一篇论文与人类基准保留的任何一篇论文都不重合,则说明AI对该论文的筛选结果是假阳性,也就是说AI与人类基准对该论文的判断存在分歧,AI认为应该保留该论文以进入下一个评审环节,人类基准则认为应该淘汰该论文,该论文不应该进入下一个评审环节。
假阴性FN,若AI按照(1- x%)的淘汰率淘汰的某一篇论文与人类基准淘汰的任何一篇论文都不重合,则说明AI对该论文的筛选结果是假阴性,也就是说AI与人类基准对该论文的判断存在分歧,AI认为该论文应该淘汰,不应该进入下一个评审环节,而人类基准则认为该论文应该保留,并进入下一个评审环节。
基于这些状态,本文进一步使用三个结果性的指标评估AI的筛选效能,以判断AI筛选结果的有用性。这三个评估指标分别是灵敏性(sensitivity)、特异性(specificity)、准确性(accuracy)。
灵敏性(sensitivity)指AI正确识别阳性样本的能力,其计算如公式(2)所示。其中TP数量为AI识别出来的真实阳性的数量,FN数量为AI把真实阳性识别为阴性的数量,(TP数量+ FN数量)表示真实的阳性样本的总数量。sensitivity数值越大,表明AI识别出真实阳性样本的比例越大,当sensitivity=1时,FN数量=0,表明AI能够识别出全部的真实阳性样本。
$\mathit{s}\mathit{e}\mathit{n}\mathit{s}\mathit{i}\mathit{t}\mathit{i}\mathit{v}\mathit{i}\mathit{t}\mathit{y}=\frac{\mathit{T}\mathit{P}\mathrm{数}\mathrm{量}}{(\mathit{T}\mathit{P}\mathrm{数}\mathrm{量}+\mathit{F}\mathit{N}\mathrm{数}\mathrm{量})}$
特异性(specificity)指AI正确识别阴性样本的能力,其计算如公式(3)所示。其中TN数量为AI识别出来的真实阴性的数量,FP数量为AI把真实阴性识别为阳性的数量,(TN数量+ FP数量)表示真实的阴性样本的总数量。specificity数值越大,表明AI识别出真实阴性样本的比例越大,当specificity =1时,FP数量=0,表明AI能够识别出全部的真实阴性样本。
$\mathit{s}\mathit{p}\mathit{e}\mathit{c}\mathit{i}\mathit{f}\mathit{i}\mathit{c}\mathit{i}\mathit{t}\mathit{y}=\frac{\mathit{T}\mathit{N}\mathrm{数}\mathrm{量}}{(\mathit{T}\mathit{N}\mathrm{数}\mathrm{量}+\mathit{F}\mathit{P}\mathrm{数}\mathrm{量})}$
准确性(accuracy)指AI判断结果与真实情况一致的程度,其计算如公式(4)所示。(TP数量+ TN数量)为AI识别出来的真实阳性和真实阴性样本的总数量,分母为总的样本数量。
$\mathit{a}\mathit{c}\mathit{c}\mathit{u}\mathit{r}\mathit{a}\mathit{c}\mathit{y}=\frac{(\mathit{T}\mathit{P}\mathrm{数}\mathrm{量}+\mathit{T}\mathit{N}\mathrm{数}\mathrm{量})}{(\mathit{T}\mathit{P}\mathrm{数}\mathrm{量}+\mathit{F}\mathit{N}\mathrm{数}\mathrm{量}+\mathit{T}\mathit{N}\mathrm{数}\mathrm{量}+\mathit{F}\mathit{P}\mathrm{数}\mathrm{量})}$
3.3 AI初审筛选效能的评估基准设定
根据本文所选期刊的原则,编辑初审时基本是按照1/3的比例进行淘汰的,被淘汰的论文可以被认为是低质量的论文,而在经过外审后,编辑部综合外审意见从剩余论文中筛选出高质量论文并择优刊发。从编辑部评审结果来看,每组复审论文中排名前50%的论文可以保证涵盖该组所有最终录用论文,因此我们将这部分论文定义为高质量论文(论文总数的前1/3),其余论文定义为低质量论文,需要被淘汰。
在使用AI进行论文初审筛选时,我们的目标是使AI初审筛选的结果在包含高质量论文的情况下,尽可能降低AI初审筛选保留比率,这样就能够提高初审的效率。例如以编辑部的高质量论文(编辑部排序前1/3)作为基准,分析AI保留的高质量论文(AI排序前1/3)与编辑部高质量论文的一致性程度,一致性程度越高,说明AI的筛选效能越高。另外,通过分析AI需要保留多大比例的论文(AI最小保留率),才能使得编辑部保留的高质量论文都被AI保留下来,就可以得到使用AI进行论文筛选的阈值。相关定义如表2所示。
另外,虽然编辑部的排序结果是综合编辑和外审意见后得到的,但是对论文的质量还可能存在误判,特别是在编辑部初审淘汰的论文中可能存在高质量论文,因此我们进一步综合两个人类专家的判断结果,形成高质量论文的集合,作为判断AI筛选效能的基准。在综合两个人类专家判断结果时,使用了不同的投票决策原则,如表3所示,我们一共确立了4个由多个人类主体的判断聚合而成的用于评估AI论文筛选效能的基准:(1)编辑部独立基准,包含编辑部排序的前1/3论文;(2)编辑部与专家合并基准,包含编辑部排序的前1/3的论文、专家1排序的前1/3的论文、专家2排序的前1/3的论文所组成的集合,也就是编辑部、两个专家,只要有一个主体认为是高质量的论文都应该纳入基准当中。(3)专家增强编辑部的基准,包含编辑部排序的前1/3的论文,以及专家1和专家2各自排序前1/3论文中同时出现的论文。(4)编辑部与专家共同决策的基准,包含被编辑部、专家1和专家2中至少两个主体同时排序前1/3的论文。其中编辑部与专家共同决策基准对论文质量的判断最为严格,筛选出的高质量论文数量也最少。
4 主要结果
收起
4.1 利用A、B组数据评估AI总体评审效能
本文通过比较A组与B组论文的AI排序结果与人类专家及编辑部排序结果,进行AI总体评审效能的判断。由于只有进入复审的论文编辑部才有排序结果,所以在比较时,A、B组分别包括9篇和8篇论文。
首先我们通过spearman相关系数对各主体的判断一致性进行比较,如表4和表5所示,从表中可以看出,各主体排序在A组和B组上的相关系数都不高,而且都不显著,这与期刊论文评审这一决策的复杂性有关,现有研究也发现人类专家在论文评审意见上的重合率只有28.58%[15],这就导致人类专家之间对论文的排序会出现不一致。另外2024年的一项研究也指出[16],现有AI的能力在长论文的处理方面还比较有限,其在论文筛选决策中的准确率约为20%。虽然AI的能力在不断地快速演进,但是其提高程度可能还有限,导致其与人类主体判断的相关性也不会太高。虽然表4中显示A组中各主体相关系数不显著,但也给出一些提示:(1)各人类主体排序的相关系数不高,说明人类之间的排序一致性不强;(2)AI与人类主体的相关性系数大于0,排序结果与人类表现出一定的一致性。同样,表5中的相关系数也给出一些提示:(1)人类主体排序结果的相关系数大多为负,排序结果的一致性较差,对论文质量判断的分歧较大;(2)AI与人类主体的相关性出现负相关,说明在人类有较大分歧的论文组,AI也形成了有别于人类的独立判断。
其次,本文通过优序比较系数,对各主体的判断一致性进行比较,如表6和表7所示。从表6中可以看出:(1)人类主体对A组中任何两篇论文优劣的认识上,具有较高共识,最低的共识率也接近50%;(2)在A组论文中,AI与人类主体之间能够达成的共识大于50%,甚至接近60%,这就为使用AI进行A组论文筛选提供了基础。从表7中可以看出:(1)B组论文中人类主体间关于论文优劣比较的共识总体低于50%,特别是专家2与编辑部的优序比较系数较低;(2)B组中AI与人类主体关于论文优劣比较具有一定的共识,与编辑部和专家1的优序比较系数大于0.4,与专家2的优序比较系数更是达到了0.536。这就为使用AI进行B组论文筛选提供了基础。
4.2 利用A、B组数据评估AI的初审筛选效能
(1)A组论文初审筛选效能评估
A组论文一共13篇,编辑部独立基准包括4篇高质量论文,编辑部与专家合并基准包括9篇高质量论文,专家增强编辑部的基准包括5篇高质量论文,编辑部与专家共同决策的基准包括3篇高质量论文。本文把AI对A组论文排序结果的前1/3与四个基准分别进行比较,可以确定出每个基准下TP、FP、TN、FN的数量,从而计算出灵敏性、特异性、准确性三个指标。
如表8所示,从这些指标中我们可以看出保留AI排序前1/3的论文实现的筛选效果。在编辑部独立基准下的灵敏性和特异性指标均为最低,其中灵敏性只有0.25,表明AI保留前1/3论文的情况下,能够覆盖编辑部独立基准中25%的高质量论文。特异性指标为0.67,表明大概67%的低质量论文都被AI识别了出来。准确性指标为0.54,说明AI保留前1/3论文的情况下,正确保留和淘汰的论文总比率为54%。在考虑编辑部与专家的聚合基准时,灵敏性提高到了0.4,表明AI保留前1/3论文的情况下最多能够识别出40%的高质量论文。特异性提高到了0.75,表明AI能够识别出75%的低质量论文。准确性也提高到了0.62,意味着AI对62%的论文判断都与人类基准的筛选结果一致。
以上的数据表明,相比于编辑部独立基准,AI与编辑部和专家的聚合基准一致性更高,但AI保留1/3论文能够覆盖的高质量论文的比例并不是很高,要想达到对各基准中高质量论文的全覆盖,还需要进一步计算各基准对应的AI最小保留率。
AI的最小保留率,是指当AI的排序结果按照这个保留率保留时,保留下来的论文刚好能够覆盖基准中的全部高质量论文。对于编辑部独立基准来说,AI保留77%的论文(10篇)就可以覆盖基准中4篇高质量。对于编辑部与专家合并基准来说,AI保留92%的论文(11篇)就能够覆盖基准中的9篇高质量论文。对于专家增强编辑部的基准来说,AI保留77%的论文(10篇)就能够覆盖基准中的5篇高质量论文。对于编辑部与专家共同决策的基准来说,AI保留77%的论文(10篇)就能够覆盖基准中的3篇高质量论文。
如果我们以编辑部独立基准来看,当我们按照AI保留77%的比例进行论文筛选时,能够提高编审工作多大的效率呢?
在未使用AI之前,编辑需要先审核全部的论文(n篇次),然后淘汰其中的1/3送外审,外审专家处理的论文数为4n/3篇次(一个论文邀请两个外审专家进行评审)。对外审回来的论文,编辑复审要处理2n/3篇次。整个编审系统处理的论文篇次是(n+4n/3+2n/3)= 3n篇次,编辑的初审工作量是n篇次,编辑的总工作量是5n/3篇次。
在使用了AI之后,由AI先依据最小保留率x%进行初审,筛选掉n(1-x%)的论文,接着由编辑对n*x%篇的论文进行初审,工作量为n*x%篇次,并保留总论文数的2n/3篇论文送外审,外审专家处理论文4n/3篇次,对外审回来的论文,编辑复审要处理2n/3篇次,整个编审系统人工处理的论文篇次是(n*x%+2n/3+4n/3)= (2+x%)n篇次,编辑初审工作量是n*x%篇次,编辑的总工作量是n*x% +2n/3 = (2/3 +x%)n篇次。
由此对编辑初审工作量可以降低23%(计算公式为1-x%),对编辑总体工作量可以降低14%(计算公式为(3/5)*(1-x%)),对编辑部总体工作量可以降低约8%(计算公式为(1/3)*(1-x%))。
(2) B组论文初审筛选效能评估
B组论文一共12篇,编辑部独立基准一共包括4篇高质量论文,编辑部与专家合并基准包括9篇高质量论文,专家增强编辑部的基准包括6篇高质量论文,编辑部与专家共同决策的基准包括3篇高质量论文。本文把AI对B组论文排序结果的前1/3与四个基准分别进行比较,进行相关指标计算,如表9所示。通过这些指标的数值,我们发现AI保留前1/3论文实现的筛选效果,在B组与A组的特征基本相同。在编辑部独立基准下的灵敏性和特异性指标均为最低,其中灵敏性只有0.25,特异性只有0.63,准确性只有0.5。而在虑编辑部与专家共同决策基准下,灵敏性提高到了0.67,特异性提高到了0.78,准确性也提高到了0.75,意味着AI最高能够对75%的论文做出与人类基准筛选结果一致的判断。
但是在AI最小保留率上,对于前三个基准,AI保留下来的论文想要覆盖基准中的全部论文,就必须一篇论文也不淘汰,也就是基于AI的排序结果,在前三个基准下,不能实现筛选功能。这是由于AI把人类判别的某些高质量论文排在了最后一位,因此只有全部保留才能覆盖基准中的论文。
但是在最后一个编辑部与专家共同决策的基准中,AI的最小保留率为75%,也就是说按照AI的排序保留75%的论文就能够覆盖B组该基准下的全部高质量论文,此时对编辑初审工作量可以降低25%,对编辑总体工作量可以降低15%,对编辑部总体工作量可以降低约8%。
(3) A、B组论文筛选效能评估总结
综合A、B组的结果来看,AI在编辑部与专家综合判断的三个基准下的灵敏性、特异性和准确性指标都高于编辑部独立基准下的对应指标,在编辑部与专家共同决策基准下的指标最优(准确性最高,最小保留率最低),这说明相比于编辑部的独立判断,AI更能反应多个专家的综合意见。通过把人类主体确定的高质量论文作为AI筛选论文的目标,在最好的情况下AI必须保留75%的论文才能全部覆盖高质量的论文,能够发挥一定的初审筛选效能。在最差的情况下AI必须保留100%的论文才能全部覆盖高质量的论文,不能发挥初审筛选效能。
4.3 C组数据应用结果
以上AI筛选的结论表明,AI在编辑部与专家共同决策的基准下,通过保留75%左右的论文(A组10篇,B组9篇),就能够覆盖质量较高的论文,从而发挥初审筛选论文的效果。因此我们把这一结论应用于C组论文,验证是否能够达到较好的筛选效果。C组论文共13篇,其编辑部独立基准的高质量论文包含4篇论文,当AI的保留率为75%时(10篇论文),AI选择出来的论文可以覆盖编辑部确定的全部高质量论文,此保留率也为C组的最小保留率。
通过A、B、C三组数据分析,可以初步得出结论,AI的恰当运用在保留高质量论文的前提下,可以提高初审论文筛选的效率,但是目前提高的程度还比较有限,大约提高论文初审效率约25%,提高编辑总体工作效率约15%,提高编审全流程效率约8%。
5 总结与结论
收起
本文以期刊论文初审为场景,深入管理决策的具体过程,探索实现管理决策中“人类-AI”系统协同性的方法。针对“先AI后人类”的顺序性管理决策,需要在对AI的决策方案筛选效能评估基础上,设计AI的决策方案筛选阈值,使AI的筛选结果能够尽可能多地淘汰质量较低的方案,然后交由人类进行后续决策方案的选择。本文的主要贡献集中于对AI的决策方案筛选效能评估上:首先,通过引入spearman相关系数及创新性的设计优序比较系数,实现了对AI总体决策效能的评估;其次通过定义筛选结果的阴性和阳性状态,引入了灵敏性、特异性、准确性指标,实现了对AI决策方案筛选效能的评估;然后,通过聚合人类集体判断的方式进行基准设计,并提出多个人类专家之间的判断可以通过合并、增强、共决三种方式形成不同的评价基准,这就避免了使用单一评价基准不能准确反映AI决策能力的局限[8],使对AI决策方案筛选效能的评估更加全面;最后,通过设计最小保留率这一指标,明确了AI在决策方案筛选中的阈值设定方法,这一指标不仅能够反映AI的方案筛选效能,而且也使得对AI的应用不只是停留在AI与人类决策结果相关性的讨论层面,而是实现了把AI直接应用于管理决策的可能,也就是按照最小保留率这一阈值首先让AI进行决策方案的筛选,然后在此基础上让人类进一步筛选决策方案,就可以提高整体的决策效率。
同时本文的研究不拘泥于AI(或者说LLM)的能力水平,而是关注如何在现有AI能力水平的基础上,把AI切实引入到管理决策当中,这样就可以加快AI落地应用的步伐。本文提出的AI决策能力的相关评估指标及其使用逻辑,不仅能够应用于其他场景中的决策方案筛选,而且能够适应LLM能力的动态演进,在AI能力有显著变革后,重新分析计算AI决策方案筛选的阈值,实现“人类-AI”系统协同性水平的不断提升。
对于期刊论文初审场景来说,本文的分析表明,现有AI的能力在期刊论文的初审筛选决策中能够实现一定的筛选效应,从而实现“先AI后人类”的顺序性协同决策。本文的分析结论显示,以现有通用LLM的能力,通过保留75%的论文,就能够覆盖被评审论文中质量较高的论文,能够使编辑的初审工作量降低大约25%,提高编辑总体工作效率约15%,提高编审全流程效率约8%。如果随着未来LLM能力的提升,其在论文筛选中的最小保留率能够降低到67%,那对于本文研究的样本期刊来说(该期刊初审淘汰约1/3论文),AI将能够完全替代编辑的初审工作,实现期刊评审效率的大幅度提高。
虽然本研究从管理决策的过程角度,提出了提高“人类-AI”系统协同性的方法,但是还存在以下局限,需要未来的研究进行拓展。第一,由于受到样本的可获得性限制,本文分析中使用的样本量较小,使相关指标在对应的子样本量发生一个较小变化时,指标数值会发生较大的跳跃,从而影响了本文对AI提升方案决策效率的准确判断。但是这个不足并不影响本文提出的方法的可行性。未来的研究可以把本文的方法应用于决策方案数量较大的场景,例如一些具有较大论文投稿量的大型会议,以增加评估指标数值的稳定性,帮助我们更加准确地认识AI在决策方案筛选中的效能。第二,在对本文决策方法有效性进行验证时,使用的C组论文与A组和B组来自同一个期刊的不同时间,这虽然能够为本研究基于A、B组论文得出的结论提供一定的效度检验,但是与使用来自其他期刊的论文进行效度检验相比,对本文研究结论有效性的验证还是略显不足,未来的研究可以使用不同来源的样本,对本文的结论进行验证。第三,使用AI进行决策方案筛选,相当于改变了决策中“守门人”规则的实现方法,Korena[17]指出初步筛选的“守门人”规则会导致整个管理决策过程产生偏见,那么AI作为“守门人”会对管理决策结果产生哪些偏见及如何进行矫正,将是未来可以进一步研究的内容。第四,现有研究发现AI的引入会改变对人类的技能要求[18,19],在本文研究的期刊论文评审场景中,把AI应用于论文的初审筛选对编辑的能力要求会带来哪些变化,也是未来研究可以进一步关注的内容。
基金
收起
  • 教育部人文社会科学基金青年基金项目(23YJC630150)
文献
收起
参考文献 引证文献
排序方式：
[1]
Cao G, Duan Y, Edwards J S, et al. Understanding managers’ attitudes and behavioral intentions towards using artificial intelligence for organizational decision-making[J]. Technovation, 2021, 106: 102312.
[2]
张亚莉, 李辽辽, 丁振斌. 组织管理中的人工智能决策:述评与展望[J]. 外国经济与管理, 2024, 46(10): 18-38.
Zhang Y, Li L, Ding Z. AI decision-making in organizational management: A review and prospects[J]. Foreign Economics & Management, 2024, 46(10): 18-38.
[3]
谷炜, 刘亚金, Lu F S, 等. 人工智能驱动管理决策:应用、感知与偏见[J]. 中国管理科学, 2025, 33(5): 99-112.
Gu W, Liu Y, Lu F S, et al. AI-driven decision sciences: Application, perception and bias[J]. Chinese Journal of Management Science, 2025, 33(5): 99-112.
[4]
Doshi A R, Bell J J, Mirzayev E, et al. Generative artificial intelligence and evaluating strategic decisions[J]. Strategic Management Journal, 2025, 46(3): 583-610.
[5]
Raisch S, Krakowski S. Artificial intelligence and management: The automation-augmentation paradox[J]. Academy of Management Review, 2021, 46(1): 192-210.
[6]
Raisch S, Kateryna F. Combining human and artificial intelligence: Hybrid problem-solving in organizations[J]. Academy of Management Review, 2025, 50(2): 441-464.
[7]
Shrestha Y R, Ben-Menahem S M, Von Krogh G. Organizational decision-making structures in the age of artificial intelligence[J]. California Management Review, 2019, 61(4): 66-83.
[8]
Vaccaro M, Almaatouq A, Malone T. When combinations of humans and AI are useful: A systematic review and meta-analysis[J]. Nature Human Behaviour, 2024, 8(12): 2293-2303.
[9]
Kousha K, Thelwall M. Artificial intelligence to support publishing and peer review: A summary and review[J]. Learned Publishing, 2024, 37(1): 4-12.
[10]
Khraisha Q, Put S, Kappenberg J, et al. Can large language models replace humans in systematic reviews? Evaluating GPT‐4's efficacy in screening and extracting data from peer‐reviewed and grey literature in multiple languages[J]. Research Synthesis Methods, 2024, 15(4): 616-626.
[11]
Thelwall M, Abdallah Y. Evaluating the predictive capacity of ChatGPT for academic peer review outcomes across multiple platforms[J]. Scientometrics, 2025, 25(3): 1-23.
[12]
Li Y, Datta S, Rastegar-Mojarad M, et al. Enhancing systematic literature reviews with generative artificial intelligence: Development, applications, and performance evaluation[J]. Journal of the American Medical Informatics Association, 2025, 32(4): 616-625.
[13]
Latona G R, Ribeiro M H, Davidson T R, et al. The AI review lottery: Widespread AI-assisted peer reviews boost paper scores and acceptance rates[J]. arXiv preprint arXiv, 2024, 2405.02150.
[14]
Boussioux L, Lane J N, Zhang M, et al. The crowdless future? Generative AI and creative problem-solving[J]. Organization Science, 2024, 35(5): 1589-1607.
[15]
Liang W, Zhang Y, Cao H, et al. Can Large Language Models provide useful feedback on research papers? A large-scale empirical analysis[J]. arXiv preprint arXiv, 2023, 2310.01783.
[16]
Zhou R, Chen L, Yu K. Is LLM a Reliable Reviewer? A Comprehensive Evaluation of LLM on Automatic Paper Reviewing Tasks[C]. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 2024: 9340-9351.
[17]
Koren M. The gatekeeper effect: The implications of pre-screening, self-selection, and bias for hiring processes[J]. Management Science, 2024, Published Online.
[18]
Jia N, Luo X, Fang Z, et al. When and how artificial intelligence augments employee creativity[J]. Academy of Management Journal, 2024, 67(1): 5-32.
[19]
Law K K, Shen M. How does artificial intelligence shape audit firms[J]. Management Science, 2025, 71(5): 3641-3666.
2026年第44卷第1期
PDF下载
59
22
引用本文
BibTeX
文章信息
  • 接收时间：2025-08-15
  • 首发时间：2026-10-13
  • 出版时间：2026-01-15
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期：2025-08-15
  • 修回日期：2025-11-23
基金
教育部人文社会科学基金青年基金项目(23YJC630150)
作者信息
    1 西安电子科技大学经济与管理学院, 陕西西安 710126
    2 同济大学经济与管理学院, 上海 200092

通讯作者:

李成汉(1992-),男,博士研究生, E-mail:。
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/kxxyj/CN/1316695248374448234
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family		 属数
Number of
genus		 种数
Number of
species		 占总种数比例
Percentage of
total species (%)		 属
Genus		 种数
Number of
species		 占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
相关链接
论文
最新文章
热读文章
热点专题
内参
蓝皮书
产业发展报告
传播力报告
期刊分级目录
资讯
学术新闻
行业新闻
学术会议
行业会议
关于
关于我们
联系我们
北京市海淀区学院南路86号
100081
010-62662699（运营及技术支持）
qkjq@cast.org.cn

中国科学技术协会版权所有 京ICP 备10216604号-4 海淀分局备案 1101084647
科技导报社主办 中国科协信息中心技术支持