Article(id=1327013251445011390, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327013054811845282, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202330984, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1701705600000, receivedDateStr=2023-12-05, revisedDate=1735747200000, revisedDateStr=2025-01-02, acceptedDate=1737820800000, acceptedDateStr=2025-01-26, onlineDate=1794314213858, onlineDateStr=2026-11-10, pubDate=1751299200000, pubDateStr=2025-07-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794314213858, onlineIssueDateStr=2026-11-10, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794314213858, creator=13701087609, updateTime=1794314213858, updator=13701087609, issue=Issue{id=1327013054811845282, tenantId=1146029695717560320, journalId=1324759914146529478, year='2025', volume='62', issue='7', pageStart='1595', pageEnd='1832', issueExtLink='null', onlineDate='null', pubDate='1751299200000', pubDateStr='2025-07-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794314166977, creator='13701087609', updateTime=1794376853183, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1327275980453950013, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327013054811845282, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1327275980453950014, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327013054811845282, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=1771, endPage=1785, ext={EN=ArticleExt(id=1327013253051429825, articleId=1327013251445011390, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Multi-Scale Cost Aggregation Framework for 3D Shape Reconstruction from Multi-Focus Images, columnId=1327013252946572224, journalTitle=Journal of Computer Research and Development, columnName=Graphics and Image Processing, runingTitle=null, highlight=null, articleAbstract=

3D shape reconstruction aims to recover the 3D structure information of the scene by using image sequences with different focus levels. Most of the existing 3D shape reconstruction methods evaluate the focus level of the image sequence from a single scale, and guide the reconstruction process by introducing regularization or post-processing methods. Due to the limitation of the selection space of depth information, the reconstruction results often cannot converge effectively. To address this issue, we propose a multi-scale cost aggregation framework for shape from focus (MSCAS). Firstly, non-downsampling multi-scale transformation is introduced to increase the depth information selection space of the input image sequence, and then the cost aggregation is performed by combining the intra-scale sequence correlation and the inter-scale information constraint. Through this expansion-aggregation mode, the doubling of scene depth representation information and the effective fusion of cross-scale and cross-sequence representation information are realized. As a general framework, MSCAS framework can embed existing model design methods and deep learning methods to achieve performance improvement. The experimental results show that MSCAS framework in this paper reduces the root mean square error (RMSE) on average by 14.91% and improves the structural similarity (SSIM) by 56.69% in the four datasets after embedding the model design class SFF method. After embedding the deep learning class SFF method, RMSE in the four datasets decreases by an average of 1.55% and SSIM increases by an average of 1.61%. These results verify the effectiveness of MSCAS framework.

, authors=Tao Yan1, 2, Qihui Shang1, 2, Peng Wu1, Jiangfeng Zhang1, 2, Yuhua Qian1, *, Bin Chen3, 4, authorsList=Tao Yan, Qihui Shang, Peng Wu, Jiangfeng Zhang, Yuhua Qian, Bin Chen, authorCompany=null, correspAuthors=Yuhua Qian, authorNote=

born in 1987. PhD, associate professor. Member of CCF. His main research interests include 3D shape reconstruction and machine vision

born in 1998. Master candidate. His main research interests include 3D shape reconstruction and data mining

born in 1987. PhD, associate professor. Member of CCF. His main research interests include real time operating system and blockchain

born in 1998. PhD candidate. Student member of CCF. His main research interests include deep learning and 3D shape reconstruction

born in 1976. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence and machine learning

born in 1970. PhD, professor, PhD supervisor. Member of CCF. His main research interests include artificial intelligence and large language models

, correspAuthorsNote=null, copyrightStatement=Copyright © 2025 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1327013258361418723, articleId=1327013251445011390, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=多尺度代价聚合的多聚焦图像3维形貌重建框架, columnId=1327013253131121602, journalTitle=计算机研究与发展, columnName=图形图像处理, runingTitle=null, highlight=null, articleAbstract=

多聚焦图像3维形貌重建旨在利用不同聚焦水平的图像序列恢复场景的3维结构信息. 现有的3维形貌重建方法大多从单一尺度对图像序列的聚焦水平进行评价,通过引入正则化或后处理方法引导重建过程,由于深度信息选择空间的局限性往往导致重建结果无法有效收敛. 针对上述问题,提出一种多尺度代价聚合的多聚焦图像3维形貌重建框架(multi-scale cost aggregation framework for 3D shape reconstruction from multi-focus images,MSCAS),该框架首先引入非降采样的多尺度变换增加输入图像序列的深度信息选择空间,然后联合尺度内序列关联与尺度间信息约束进行代价聚合,通过这种扩张-聚合模式实现了场景深度表征信息的倍增与跨尺度和跨序列表征信息的有效融合. 作为一种通用框架,MSCAS框架可实现已有模型设计类方法和深度学习类方法的嵌入进而实现性能提升. 实验结果表明:MSCAS框架在嵌入模型设计类SFF方法后4组数据集中的均方根误差(root mean squared error,RMSE)平均下降14.91个百分点,结构相似度(structural similarity index measure,SSIM)平均提升56.69个百分点,嵌入深度学习类SFF方法后4组数据集中的RMSE平均下降1.55个百分点,SSIM平均提升1.61个百分点. 验证了MSCAS框架的有效性和通用性.

, authors=闫涛1, 2, 尚起慧1, 2, 吴鹏1, 张江峰1, 2, 钱宇华1, *, 陈斌3, 4, authorsList=闫涛, 尚起慧, 吴鹏, 张江峰, 钱宇华, 陈斌, authorCompany=null, correspAuthors=钱宇华, authorNote=

1987年生. 博士,副教授. CCF会员. 主要研究方向为3维形貌重建、机器视觉

1998年生. 硕士研究生. 主要研究方向为3维形貌重建、数据挖掘

1987年生. 博士,副教授. CCF会员. 主要研究方向为实时操作系统、区块链

1998年生. 博士研究生. CCF学生会员. 主要研究方向为深度学习、3维形貌重建

1976年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习

1970年生. 博士,教授,博士生导师. CCF会员. 主要研究方向为人工智能、大语言模型

, correspAuthorsNote=
钱宇华()
, copyrightStatement=版权所有 © 2025 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=aRgU7SjJOzUKYGe3GLim/A==, magXml=lJaiP8ZHrIIVuhGML1fbgQ==, pdfUrl=null, pdf=6q14VwZLU44BWZ2cVYApYA==, pdfFileSize=4772572, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=tp2IviYn+6c9Ihk9m5xz9g==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=KmaJkaoS3HIrS2nZ3dHL/A==, mapNumber=null, fund=null)}, authors=[Author(id=1327013259087033329, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=hongyanyutian@sxu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013259200279540, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259087033329, language=EN, stringName=Tao Yan, firstName=Tao, middleName=null, lastName=Yan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006
2School of Computer and Information Technology, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1987. PhD, associate professor. Member of CCF. His main research interests include 3D shape reconstruction and machine vision

"}, bioImg=null, bioContent=

born in 1987. PhD, associate professor. Member of CCF. His main research interests include 3D shape reconstruction and machine vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013259275777013, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259087033329, language=CN, stringName=闫涛, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1山西大学大数据科学与产业研究院 太原 030006
2山西大学计算机与信息技术学院 太原 030006, bio={"img":"U9jyB5+vwQMaIgO0kS2vtA==","content":"

1987年生. 博士,副教授. CCF会员. 主要研究方向为3维形貌重建、机器视觉

"}, bioImg=U9jyB5+vwQMaIgO0kS2vtA==, bioContent=

1987年生. 博士,副教授. CCF会员. 主要研究方向为3维形貌重建、机器视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)]), AuthorCompany(id=1327013258764071911, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=2, ext=[AuthorCompanyExt(id=1327013258776654824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer and Information Technology, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258793432041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山西大学计算机与信息技术学院 太原 030006)])]), Author(id=1327013259351274487, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013259460326394, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259351274487, language=EN, stringName=Qihui Shang, firstName=Qihui, middleName=null, lastName=Shang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006
2School of Computer and Information Technology, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1998. Master candidate. His main research interests include 3D shape reconstruction and data mining

"}, bioImg=null, bioContent=

born in 1998. Master candidate. His main research interests include 3D shape reconstruction and data mining

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013259556795387, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259351274487, language=CN, stringName=尚起慧, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1山西大学大数据科学与产业研究院 太原 030006
2山西大学计算机与信息技术学院 太原 030006, bio={"img":"Jn9mWr8YGSqUXauJIGul4A==","content":"

1998年生. 硕士研究生. 主要研究方向为3维形貌重建、数据挖掘

"}, bioImg=Jn9mWr8YGSqUXauJIGul4A==, bioContent=

1998年生. 硕士研究生. 主要研究方向为3维形貌重建、数据挖掘

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)]), AuthorCompany(id=1327013258764071911, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=2, ext=[AuthorCompanyExt(id=1327013258776654824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer and Information Technology, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258793432041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山西大学计算机与信息技术学院 太原 030006)])]), Author(id=1327013259644875773, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013259732956159, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259644875773, language=EN, stringName=Peng Wu, firstName=Peng, middleName=null, lastName=Wu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1987. PhD, associate professor. Member of CCF. His main research interests include real time operating system and blockchain

"}, bioImg=null, bioContent=

born in 1987. PhD, associate professor. Member of CCF. His main research interests include real time operating system and blockchain

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013259812647936, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259644875773, language=CN, stringName=吴鹏, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1山西大学大数据科学与产业研究院 太原 030006, bio={"img":"6rzU65zHLYGJOzT37Ujp2A==","content":"

1987年生. 博士,副教授. CCF会员. 主要研究方向为实时操作系统、区块链

"}, bioImg=6rzU65zHLYGJOzT37Ujp2A==, bioContent=

1987年生. 博士,副教授. CCF会员. 主要研究方向为实时操作系统、区块链

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)])]), Author(id=1327013259896532993, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013259976224772, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259896532993, language=EN, stringName=Jiangfeng Zhang, firstName=Jiangfeng, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006
2School of Computer and Information Technology, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1998. PhD candidate. Student member of CCF. His main research interests include deep learning and 3D shape reconstruction

"}, bioImg=null, bioContent=

born in 1998. PhD candidate. Student member of CCF. His main research interests include deep learning and 3D shape reconstruction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013261632974853, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259896532993, language=CN, stringName=张江峰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1山西大学大数据科学与产业研究院 太原 030006
2山西大学计算机与信息技术学院 太原 030006, bio={"img":"n0VFQ7ePVabitkH9rnxlfA==","content":"

1998年生. 博士研究生. CCF学生会员. 主要研究方向为深度学习、3维形貌重建

"}, bioImg=n0VFQ7ePVabitkH9rnxlfA==, bioContent=

1998年生. 博士研究生. CCF学生会员. 主要研究方向为深度学习、3维形貌重建

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)]), AuthorCompany(id=1327013258764071911, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=2, ext=[AuthorCompanyExt(id=1327013258776654824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer and Information Technology, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258793432041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山西大学计算机与信息技术学院 太原 030006)])]), Author(id=1327013261796552711, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=jinchengqyh@sxu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1327013261872050185, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013261796552711, language=EN, stringName=Yuhua Qian, firstName=Yuhua, middleName=null, lastName=Qian, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1976. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence and machine learning

"}, bioImg=null, bioContent=

born in 1976. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence and machine learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013261947547658, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013261796552711, language=CN, stringName=钱宇华, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1山西大学大数据科学与产业研究院 太原 030006, bio={"img":"VDv418oE1ouEPpe6PydM2w==","content":"

1976年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习

"}, bioImg=VDv418oE1ouEPpe6PydM2w==, bioContent=

1976年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)])]), Author(id=1327013262031433740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013262140485647, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013262031433740, language=EN, stringName=Bin Chen, firstName=Bin, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=3, 4, address=3Chongqing Research Institute of Harbin Institute of Technology, Chongqing 401151
4International Research Institute for Artificial Intelligence, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055, bio={"content":"

born in 1970. PhD, professor, PhD supervisor. Member of CCF. His main research interests include artificial intelligence and large language models

"}, bioImg=null, bioContent=

born in 1970. PhD, professor, PhD supervisor. Member of CCF. His main research interests include artificial intelligence and large language models

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013262211788816, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013262031433740, language=CN, stringName=陈斌, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=3, 4, address=3哈尔滨工业大学重庆研究院 重庆 401151
4哈尔滨工业大学(深圳)国际人工智能学院 广东深圳 518055, bio={"img":"oTPuW0n+sv3ZMR7EchdX4Q==","content":"

1970年生. 博士,教授,博士生导师. CCF会员. 主要研究方向为人工智能、大语言模型

"}, bioImg=oTPuW0n+sv3ZMR7EchdX4Q==, bioContent=

1970年生. 博士,教授,博士生导师. CCF会员. 主要研究方向为人工智能、大语言模型

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258889901034, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=3, ext=[AuthorCompanyExt(id=1327013258898289643, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258889901034, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Chongqing Research Institute of Harbin Institute of Technology, Chongqing 401151), AuthorCompanyExt(id=1327013258906678252, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258889901034, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3哈尔滨工业大学重庆研究院 重庆 401151)]), AuthorCompany(id=1327013258982175725, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=4, ext=[AuthorCompanyExt(id=1327013258994758638, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258982175725, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4International Research Institute for Artificial Intelligence, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055), AuthorCompanyExt(id=1327013259007341551, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258982175725, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4哈尔滨工业大学(深圳)国际人工智能学院 广东深圳 518055)])])], keywords=[Keyword(id=1327013262329229329, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, orderNo=1, keyword=multi-scale cost aggregation), Keyword(id=1327013262408921106, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, orderNo=2, keyword=depth estimation), Keyword(id=1327013262480224275, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, orderNo=3, keyword=focus measure), Keyword(id=1327013262543138836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, orderNo=4, keyword=multi-focus image sequence), Keyword(id=1327013262610247701, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, orderNo=5, keyword=3D shape reconstruction), Keyword(id=1327013262685745174, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, orderNo=1, keyword=多尺度代价聚合), Keyword(id=1327013262765436951, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, orderNo=2, keyword=深度估计), Keyword(id=1327013262832545816, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, orderNo=3, keyword=聚焦测量), Keyword(id=1327013262912237593, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, orderNo=4, keyword=多聚焦图像序列), Keyword(id=1327013262979346458, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, orderNo=5, keyword=3维形貌重建)], refs=[Reference(id=1327013266976518200, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Nayar S K, Nakagawa Y. Shape from focus[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(8): 824−831, articleTitle=null, refAbstract=null), Reference(id=1327013267052015673, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Tao M W, Srinivasan P P, Hadap S, et al. Shape estimation from shading, defocus, and correspondence using light-field angular coherence[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 546−560, articleTitle=null, refAbstract=null), Reference(id=1327013267123318842, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Clerc M, Mallat S. The texture gradient equation for recovering shape from texture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 536−549, articleTitle=null, refAbstract=null), Reference(id=1327013267215593531, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Lee J Y, Park R H. Complex-valued disparity: Unified depth model of depth from stereo, depth from focus, and depth from defocus based on the light field gradient[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(3): 830−841, articleTitle=null, refAbstract=null), Reference(id=1327013267278508092, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhan Yu, Guo Xinqing, Lin Haibing, et al. Line assisted light field triangulation and stereo matching[C]//Proc of the 14th IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2013: 2792−2799, articleTitle=null, refAbstract=null), Reference(id=1327013267349811261, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Jianqiao, Lu Mindong, Nian Ze. Continuous depth map reconstruction from light fields[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3257−3265, articleTitle=null, refAbstract=null), Reference(id=1327013267416920126, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=闫涛,陈斌,刘凤娴,等. 基于多景深融合模型的显微三维重建方法[J]. 计算机辅助设计与图形学学报,2017,29(9):1613−1623, articleTitle=null, refAbstract=null), Reference(id=1327013267513389119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Yan Tao, Chen Bin, Liu Fengxian, et al. Multi-focus image fusion model for micro 3D reconstruction[J]. Journal of Computer-Aided Design and Computer Graphics, 2017, 29(9): 1613−1623 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327013267576303680, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=闫涛,高浩轩,张江峰,等. 分组并行的实时微观三维形貌重建方法[J]. 软件学报,2024,35(4):1717−1731, articleTitle=null, refAbstract=null), Reference(id=1327013267639218241, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Yan Tao, Gao Haoxuan, Zhang Jiangfeng, et al. Grouping parallel lightweight real-time microscopic 3D shape reconstruction method[J]. Journal of Software, 2024, 35(4): 1717−1731 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327013267702132802, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=闫涛,钱宇华,李飞江,等. 三维时频变换视角的智能微观三维形貌重建方法[J]. 中国科学:信息科学,2023,53(2):282−308, articleTitle=null, refAbstract=null), Reference(id=1327013267777630275, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Yan Tao, Qian Yuhua, Li Feijiang, et al. Intelligent microscopic 3D shape reconstruction method based on 3D time-frequency transformation[J]. SCIENTIA SINICA Informationis, 2023, 53(2): 282−308 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327013267844739140, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Ali U, Mahmood M T. Robust focus volume regularization in shape from focus[J]. IEEE Transactions on Image Processing, 2021, 30: 7215−7227, articleTitle=null, refAbstract=null), Reference(id=1327013267920236613, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Fengting, Huang Xiaolei, Zhou Zihan. Deep depth from focus with differential focus volume[C]//Proc of the 35th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 12642−12651, articleTitle=null, refAbstract=null), Reference(id=1327013268004122694, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Muhammad M S, Choi T S. Sampling for shape from focus in optical microscopy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(3): 564−573, articleTitle=null, refAbstract=null), Reference(id=1327013268083814471, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Pertuz S, Puig D, Garcia M A. Analysis of focus measure operators for shape-from-focus[J]. Pattern Recognition, 2013, 46(5): 1415−1432, articleTitle=null, refAbstract=null), Reference(id=1327013268159311944, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Ma Z, Kim D, Shin Y G. Shape-from-focus reconstruction using nonlocal matting Laplacian prior followed by MRF based refinement[J]. Pattern Recognition, 2020, 103: 107302, articleTitle=null, refAbstract=null), Reference(id=1327013268243198025, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Thelen A, Frey S, Hirsch S, et al. Improvements in shape-from-focus for holographic reconstructions with regard to focus operators, neighborhood-size, and height value interpolation[J]. IEEE Transactions on Image Processing, 2009, 18(1): 151−157, articleTitle=null, refAbstract=null), Reference(id=1327013268339667018, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Jeon H G, Surh J, Im S, et al. Ring difference filter for fast and noise robust depth from focus[J]. IEEE Transactions on Image Processing, 2020, 29: 1045−1060, articleTitle=null, refAbstract=null), Reference(id=1327013268452913227, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Yan Tao, Wu Peng, Qian Yuhua, et al. Multiscale fusion and aggregation PCNN for 3D shape recovery[J]. Information Sciences, 2020, 536: 277−297, articleTitle=null, refAbstract=null), Reference(id=1327013268545187916, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Singh S, Gupta D, Anand R S, et al. Nonsubsampled shearlet based CT and MR medical image fusion using biologically inspired spiking neural network[J]. Biomedical Signal Processing and Control, 2015, 18: 91−101, articleTitle=null, refAbstract=null), Reference(id=1327013268641656909, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Minhas R, Mohammed A A, Wu Q M J. Shape from focus using fast discrete curvelet transform[J]. Pattern Recognition, 2011, 44(4): 839−853, articleTitle=null, refAbstract=null), Reference(id=1327013268721348686, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Wee C Y, Paramesran R. Measure of image sharpness using eigenvalues[J]. Information Sciences, 2007, 177(12): 2533−2552, articleTitle=null, refAbstract=null), Reference(id=1327013268822011983, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Yan Tao, Hu Zhiguo, Qian Yuhua, et al. 3D shape reconstruction from multifocus image fusion using a multidirectional modified Laplacian operator[J]. Pattern Recognition, 2020, 98: 107065, articleTitle=null, refAbstract=null), Reference(id=1327013268952035408, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Honauer K, Johannsen O, Kondermann D, et al. A dataset and evaluation methodology for depth estimation on 4D light fields[C]//Proc of the 13th Asian Conf on Computer Vision. Berlin: Springer, 2016: 19−34, articleTitle=null, refAbstract=null), Reference(id=1327013270675894355, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Won C, Jeon H. Learning depth from focus in the wild[J]. arXiv preprint, arXiv: 2207.09658, 2022, articleTitle=null, refAbstract=null), Reference(id=1327013270768169044, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Mayer N, Ilg E, Husser P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proc of the 30th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 4040−4048, articleTitle=null, refAbstract=null), Reference(id=1327013270839472213, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Maximov M, Galim K, Leal L. Focus on defocus: Bridging the synthetic to real domain gap for depth estimation[C]//Proc of the 35th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 1071−1080, articleTitle=null, refAbstract=null), Reference(id=1327013270927552598, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Hazirbas C, Soyer S G, Staab M C, et al. Deep depth from focus[C]//Proc of the 14th Asian Conf on Computer Vision. Berlin: Springer, 2018: 525−541, articleTitle=null, refAbstract=null), Reference(id=1327013271003050072, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Ninghsu, Wang Ren, Liu Yulun, et al. Bridging unsupervised and supervised depth from focus via all-in-focus supervision[C]//Proc of the 21st IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2021: 12621−12631, articleTitle=null, refAbstract=null), Reference(id=1327013271128879193, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=张江峰,闫涛,王克琪,等. 多景深图像聚焦信息的三维形貌重建:数据集与模型[J]. 计算机学报,2023,46(8):1734−1752, articleTitle=null, refAbstract=null), Reference(id=1327013271187599450, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Jiangfeng, Yan Tao, Wang Keqi, et al. 3D shape reconstruction from multi depth of field images: Datasets and models[J]. Chinese Journal of Computers, 2023, 46(8): 1734−1752 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327013271258902619, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu C, Yuen J, Torralba A. SIFT flow: Dense correspondence across scene and its application[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 978−994, articleTitle=null, refAbstract=null), Reference(id=1327013271355371612, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Hosni A, Rheman C, Bleyer M, et al. Fast cost-volume filtering for visual correspondence and beyond[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(2): 504−511, articleTitle=null, refAbstract=null), Reference(id=1327013271414091869, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=He Kaiming, Sun Jian, Tang Xiaoou. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397−1409, articleTitle=null, refAbstract=null), Reference(id=1327013271535726686, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Qinxiong. A non-local cost aggregation method for stereo matching[C]//Proc of the 25th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2012: 1402−1409, articleTitle=null, refAbstract=null), Reference(id=1327013271607029855, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Kang, Fang Yuqiang, Min Dongbo, et al. Cross-scale cost aggregation for stereo matching[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(5): 965−976, articleTitle=null, refAbstract=null), Reference(id=1327013271707693152, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=梁新彦,钱宇华,郭倩,等. 多粒度融合驱动的超多视图分类方法[J],计算机研究与发展,2022,59(8):1653−1667, articleTitle=null, refAbstract=null), Reference(id=1327013271770607713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=Liang Xinyan, Qian Yuhua, Guo Qian, et al. Multi-granulation fusion-driven method for many-view classification[J]. Journal of Computer Research and Development, 2022, 59(8): 1653−1667 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327013271879659618, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=王晓慧,贾珈,蔡莲红. 基于小波图像融合的表情细节合成[J],计算机研究与发展,2013,50(2):387−393, articleTitle=null, refAbstract=null), Reference(id=1327013271992905827, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Xiaohui, Jia Jia, Cai Lianhong. Expression detail synthesis based on wavelet-based image fusion[J]. Journal of Computer Research and Development, 2013, 50(2): 387−393 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327013272064208996, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=41, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Lei, Bao P, Wu Xiaolin. Multiscale LMMSE-based image denoising with optimal wavelet selection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2005, 15(4): 469−481, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)]), AuthorCompany(id=1327013258764071911, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=2, ext=[AuthorCompanyExt(id=1327013258776654824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer and Information Technology, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258793432041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山西大学计算机与信息技术学院 太原 030006)]), AuthorCompany(id=1327013258889901034, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=3, ext=[AuthorCompanyExt(id=1327013258898289643, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258889901034, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Chongqing Research Institute of Harbin Institute of Technology, Chongqing 401151), AuthorCompanyExt(id=1327013258906678252, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258889901034, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3哈尔滨工业大学重庆研究院 重庆 401151)]), AuthorCompany(id=1327013258982175725, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=4, ext=[AuthorCompanyExt(id=1327013258994758638, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258982175725, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4International Research Institute for Artificial Intelligence, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055), AuthorCompanyExt(id=1327013259007341551, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258982175725, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4哈尔滨工业大学(深圳)国际人工智能学院 广东深圳 518055)])], figs=[ArticleFig(id=1327013263096786971, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 1, caption=Impact of different scales and noise on the reconstruction results of typical SFF methods, figureFileSmall=ca99L3/6tDAEfcg4FXidnA==, figureFileBig=tp2IviYn+6c9Ihk9m5xz9g==, tableContent=null), ArticleFig(id=1327013263159701532, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图1, caption=不同尺度与噪声对典型SFF方法重建结果的影响, figureFileSmall=ca99L3/6tDAEfcg4FXidnA==, figureFileBig=tp2IviYn+6c9Ihk9m5xz9g==, tableContent=null), ArticleFig(id=1327013263419748381, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 2, caption=Schematic diagram of multi-scale cost aggregation in model-based SFF and deep learning-based SFF methods reconstruction framework, figureFileSmall=ZP0/CKRX6HI7fVdBdBFKbg==, figureFileBig=A9yAobtcfL+pK/4smA0nfQ==, tableContent=null), ArticleFig(id=1327013263499440158, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图2, caption=多尺度代价聚合在模型类SFF方法与深度学习类SFF方法的重建框架示意图, figureFileSmall=ZP0/CKRX6HI7fVdBdBFKbg==, figureFileBig=A9yAobtcfL+pK/4smA0nfQ==, tableContent=null), ArticleFig(id=1327013263579131935, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 3, caption=Differences in neighborhood design between existing cost aggregation methods and MACAS framework, figureFileSmall=nYRWnNEbXJmzRMc93loRng==, figureFileBig=bg/3F380z5PkVmPZ0J4VaQ==, tableContent=null), ArticleFig(id=1327013263642046496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图3, caption=现有代价聚合方法与MACAS框架在邻域设计的差异, figureFileSmall=nYRWnNEbXJmzRMc93loRng==, figureFileBig=bg/3F380z5PkVmPZ0J4VaQ==, tableContent=null), ArticleFig(id=1327013263717543969, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 4, caption=Experimental setting for different regularization parameters, figureFileSmall=rpyOmsdSQXiHEdiXoIMFpA==, figureFileBig=xIfJ6KcJrUrD9SO1dgh1PA==, tableContent=null), ArticleFig(id=1327013263793041442, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图4, caption=不同正则化参数设置实验, figureFileSmall=rpyOmsdSQXiHEdiXoIMFpA==, figureFileBig=xIfJ6KcJrUrD9SO1dgh1PA==, tableContent=null), ArticleFig(id=1327013263876927523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 5, caption=Comparison between RR method and RR-MSCAS method in sparse texture samples, figureFileSmall=+QawtGjRXJFZ2N+7fGZzFg==, figureFileBig=PiYHKr/4KpwLOGELdV7s+w==, tableContent=null), ArticleFig(id=1327013263939842084, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图5, caption=RR方法与RR-MSCAS方法在稀疏纹理样本的比较, figureFileSmall=+QawtGjRXJFZ2N+7fGZzFg==, figureFileBig=PiYHKr/4KpwLOGELdV7s+w==, tableContent=null), ArticleFig(id=1327013264006950949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 6, caption=Comparison between the original method and the embedded MSCAS framework at different scales, figureFileSmall=4eEo1Q+TpCO2h/XXJGiayg==, figureFileBig=vgSMcOSQaP/uhqPUrc2qdg==, tableContent=null), ArticleFig(id=1327013264086642726, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图6, caption=原方法与嵌入MSCAS框架后在不同尺度下的对比, figureFileSmall=4eEo1Q+TpCO2h/XXJGiayg==, figureFileBig=vgSMcOSQaP/uhqPUrc2qdg==, tableContent=null), ArticleFig(id=1327013264187306023, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 7, caption=Average SSIM of MSCAS framework components and original method under different noise levels, figureFileSmall=2/0ZRbz7pIK+PubvdrdXBw==, figureFileBig=CaiQzzD2Vy8Cfa5z92GAAQ==, tableContent=null), ArticleFig(id=1327013264254414888, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图7, caption=不同强度噪声下MSCAS框架各部分与原方法的平均SSIIM, figureFileSmall=2/0ZRbz7pIK+PubvdrdXBw==, figureFileBig=CaiQzzD2Vy8Cfa5z92GAAQ==, tableContent=null), ArticleFig(id=1327013264313135145, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 8, caption=Results comparison of different algorithms before and after embedding MSCAS framework in LF_4D_PSF dataset, figureFileSmall=dfWQHGIB3zgOmTmDq/JUzg==, figureFileBig=H0Pt/z8j8+gYGXTTLd4F1A==, tableContent=null), ArticleFig(id=1327013264376049707, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图8, caption=LF_4D_PSF数据集中不同算法在嵌入MSCAS框架前后的结果对比, figureFileSmall=dfWQHGIB3zgOmTmDq/JUzg==, figureFileBig=H0Pt/z8j8+gYGXTTLd4F1A==, tableContent=null), ArticleFig(id=1327013264459935788, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 9, caption=Results comparison of different algorithms before and after embedding MSCAS framework in Middlebury_FS dataset, figureFileSmall=2gHoXNAdu8yOTrc11aC7Eg==, figureFileBig=msLjckxSwfvF8KEQIZdO4g==, tableContent=null), ArticleFig(id=1327013264522850349, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图9, caption=Middlebury_FS数据集中不同算法在嵌入MSCAS框架前后的结果对比, figureFileSmall=2gHoXNAdu8yOTrc11aC7Eg==, figureFileBig=msLjckxSwfvF8KEQIZdO4g==, tableContent=null), ArticleFig(id=1327013266196377648, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Figure 10, caption=Results comparison of different algorithms before and after embedding MSCAS framework in FlyingThings3D dataset, figureFileSmall=VlAMjZovyt2S/aFyF8+Ucw==, figureFileBig=RgehybeXo+9tnhzFH0AfBQ==, tableContent=null), ArticleFig(id=1327013266288652337, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=图10, caption=FlyingThings3D数据集中不同算法在嵌入MSCAS框架前后的结果对比, figureFileSmall=VlAMjZovyt2S/aFyF8+Ucw==, figureFileBig=RgehybeXo+9tnhzFH0AfBQ==, tableContent=null), ArticleFig(id=1327013266380927026, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Table 1, caption=Statistic Information of Datasets, figureFileSmall=null, figureFileBig=null, tableContent=
数据集图像来源场景数图像大小
LF_4D_PSF合成场景24512×512×30
Middlebury_FS真实场景332 964×1 988×15
FlyingThings3D合成场景1 000960×540×15
FoD500合成场景500256×256×5
), ArticleFig(id=1327013266448035891, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=表1, caption=

数据集统计信息

, figureFileSmall=null, figureFileBig=null, tableContent=
数据集图像来源场景数图像大小
LF_4D_PSF合成场景24512×512×30
Middlebury_FS真实场景332 964×1 988×15
FlyingThings3D合成场景1 000960×540×15
FoD500合成场景500256×256×5
), ArticleFig(id=1327013266515144756, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Table 2, caption=Information Summary of Comparative SFF Methods, figureFileSmall=null, figureFileBig=null, tableContent=
对比
方法
归属类型期刊或会
议/年份
参数设置
SML模型设计类SFFPR/2013T1=7,step=1,N=2
GD模型设计类SFFPR/2013Sigma=2,hsize=5
RDF模型设计类SFFTIP/2020Tmad=0.1,Tbokeh=0.15,
Fm (r1=1,r2=2,r3=5)
RR模型设计类SFFTIP/2021lambda=0.3,alpha=0.1,beta=1.5,gamma=2.5,itr=8,nei=2
DFV深度学习类SFFCVPR/2022Adam (β1 = 0.9,β2 = 0.999),
alpha $\in $ {8/15,4/15,2/15,1/15}
FV深度学习类SFFCVPR/2022Adam(β1 = 0.9 ,β2 = 0.999),
alpha $\in $ {8/15,4/15,2/15,1/15}
DFW深度学习类SFFECCV/2022Adam( β1 = 0.9,β2 = 0.99)
AiF深度学习类SFFICCV/2021Adam( β1 = 0.9,β2 = 0.999)
), ArticleFig(id=1327013266594836533, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=表2, caption=

对比SFF方法信息汇总

, figureFileSmall=null, figureFileBig=null, tableContent=
对比
方法
归属类型期刊或会
议/年份
参数设置
SML模型设计类SFFPR/2013T1=7,step=1,N=2
GD模型设计类SFFPR/2013Sigma=2,hsize=5
RDF模型设计类SFFTIP/2020Tmad=0.1,Tbokeh=0.15,
Fm (r1=1,r2=2,r3=5)
RR模型设计类SFFTIP/2021lambda=0.3,alpha=0.1,beta=1.5,gamma=2.5,itr=8,nei=2
DFV深度学习类SFFCVPR/2022Adam (β1 = 0.9,β2 = 0.999),
alpha $\in $ {8/15,4/15,2/15,1/15}
FV深度学习类SFFCVPR/2022Adam(β1 = 0.9 ,β2 = 0.999),
alpha $\in $ {8/15,4/15,2/15,1/15}
DFW深度学习类SFFECCV/2022Adam( β1 = 0.9,β2 = 0.99)
AiF深度学习类SFFICCV/2021Adam( β1 = 0.9,β2 = 0.999)
), ArticleFig(id=1327013266750025782, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=EN, label=Table 3, caption=Performance Comparison of Eight SFF Methods Before and After Embedding MSCAS Framework Across Four Datasets, figureFileSmall=null, figureFileBig=null, tableContent=
数据集方法类型SFF方法RMSESSIMPSNRCorr
原方法MSCAS原方法MSCAS原方法MSCAS原方法MSCAS
注:黑体数值表示性能提升的指标.
LF_4D_PSF模型设计
类方法
SML0.14960.10900.44690.796618.366823.91980.86960.9485
GD0.22230.11650.28320.744214.545220.43000.63360.8989
RDF0.23160.21990.75600.784223.617624.29110.92770.9404
RR0.20320.20080.72020.732620.456421.66890.95280.9533
平均值0.20170.16160.55160.764419.246522.57750.84590.9353
MSCAS-原方法↓ 19.90%↑ 38.58%↑ 17.31%↑ 10.56%
LF_4D_PSF深度学习
类方法
DFV0.15370.15350.85500.856831.330631.33230.97240.9726
FV0.15490.15390.85700.857031.274631.27460.97220.9727
DFW0.12880.12670.90590.907522.838822.94070.98020.9807
AiF0.20140.20080.86090.861024.789724.80600.96100.9614
平均值0.15970.15870.86970.870627.55827.58840.97140.9753
MSCAS-原方法↓0.61%↑ 0.10%↑ 0.11%↑ 0.40%
Middlebury_FS模型设计
类方法
SML0.24270.15870.58220.835720.212028.51600.65370.7842
GD0.33230.23150.28320.744214.545220.43000.63360.8989
RDF0.21730.20010.75600.784223.617624.29110.92770.9404
RR0.23730.21770.69930.744320.419220.87740.73030.7617
平均值0.25740.20200.58020.777119.698523.52860.73630.8463
MSCAS-原方法↓ 21.52%↑ 33.94%↑ 19.44%↑14.94%
深度学习
类方法
DFV0.20720.17810.62590.783216.533220.35610.62770.7242
FV0.23150.22570.71950.742818.548019.11250.64780.6453
DFW0.16110.16040.90030.905624.323924.71800.94770.9573
AiF0.18630.18390.83160.834415.635115.75960.64700.6528
平均值0.19650.18700.76930.816518.760019.98660.71750.7449
MSCAS-原方法↓ 4.83%↑ 6.13%↑ 6.54%↑ 3.81%
FlyingThings3D模型设计
类方法
SML0.49320.42330.57120.767619.556222.23960.60810.7771
GD0.56740.49410.34210.685518.010821.03060.48600.6908
RDF0.42960.46880.80840.808822.551922.59930.80820.7995
RR0.35950.33320.67770.738021.314822.69250.66170.7336
平均值0.46240.42990.59990.750020.358422.14050.64100.7503
MSCAS-原方法↓ 7.04%↑ 25.03%↑ 8.75%↑ 17.04%
深度学习
类方法
DFV0.16710.1650.72110.724317.413617.56630.76370.7659
FV0.17850.17750.65890.659411.103711.2590.77830.7796
DFW0.09890.09820.89810.900521.296521.38390.97520.9775
AiF0.12210.12200.93030.930720.044120.04410.81570.8158
平均值0.14160.14060.80210.803717.464417.56330.83320.8346
MSCAS-原方法↓ 0.68%↑ 0.20%↑ 0.56%↑ 0.17%
FoD500模型设计
类方法
SML0.75680.65560.05880.29077.78047.42550.13230.4021
GD0.77250.76720.03820.45697.44209.02620.08630.2245
RDF0.65170.64920.41190.45148.03948.75210.46750.4990
RR0.75800.53910.21030.449313.915415.22500.20360.3114
平均值0.73480.65280.17980.41219.294310.10720.22240.3593
MSCAS-原方法↓ 11.16%↑ 129.19%↑ 8.75%↑ 61.52%
深度学习
类方法
DFV0.16670.16670.90640.906431.047231.04820.85480.8549
FV0.16670.16670.90640.906431.047531.04790.85480.8549
DFW0.08590.08570.92510.925224.253924.25410.92070.9213
AiF0.15090.15070.91040.910730.203630.20380.87420.8743
平均值0.14250.14240.91200.912129.138029.13850.87610.8764
MSCAS-原方法↓ 0.07%↑ 0.01%↑ 0.00%↑ 0.02%
), ArticleFig(id=1327013266859077687, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, language=CN, label=表3, caption=

8种SFF方法在嵌入MSCAS框架前后在4组数据集中的性能比较

, figureFileSmall=null, figureFileBig=null, tableContent=
数据集方法类型SFF方法RMSESSIMPSNRCorr
原方法MSCAS原方法MSCAS原方法MSCAS原方法MSCAS
注:黑体数值表示性能提升的指标.
LF_4D_PSF模型设计
类方法
SML0.14960.10900.44690.796618.366823.91980.86960.9485
GD0.22230.11650.28320.744214.545220.43000.63360.8989
RDF0.23160.21990.75600.784223.617624.29110.92770.9404
RR0.20320.20080.72020.732620.456421.66890.95280.9533
平均值0.20170.16160.55160.764419.246522.57750.84590.9353
MSCAS-原方法↓ 19.90%↑ 38.58%↑ 17.31%↑ 10.56%
LF_4D_PSF深度学习
类方法
DFV0.15370.15350.85500.856831.330631.33230.97240.9726
FV0.15490.15390.85700.857031.274631.27460.97220.9727
DFW0.12880.12670.90590.907522.838822.94070.98020.9807
AiF0.20140.20080.86090.861024.789724.80600.96100.9614
平均值0.15970.15870.86970.870627.55827.58840.97140.9753
MSCAS-原方法↓0.61%↑ 0.10%↑ 0.11%↑ 0.40%
Middlebury_FS模型设计
类方法
SML0.24270.15870.58220.835720.212028.51600.65370.7842
GD0.33230.23150.28320.744214.545220.43000.63360.8989
RDF0.21730.20010.75600.784223.617624.29110.92770.9404
RR0.23730.21770.69930.744320.419220.87740.73030.7617
平均值0.25740.20200.58020.777119.698523.52860.73630.8463
MSCAS-原方法↓ 21.52%↑ 33.94%↑ 19.44%↑14.94%
深度学习
类方法
DFV0.20720.17810.62590.783216.533220.35610.62770.7242
FV0.23150.22570.71950.742818.548019.11250.64780.6453
DFW0.16110.16040.90030.905624.323924.71800.94770.9573
AiF0.18630.18390.83160.834415.635115.75960.64700.6528
平均值0.19650.18700.76930.816518.760019.98660.71750.7449
MSCAS-原方法↓ 4.83%↑ 6.13%↑ 6.54%↑ 3.81%
FlyingThings3D模型设计
类方法
SML0.49320.42330.57120.767619.556222.23960.60810.7771
GD0.56740.49410.34210.685518.010821.03060.48600.6908
RDF0.42960.46880.80840.808822.551922.59930.80820.7995
RR0.35950.33320.67770.738021.314822.69250.66170.7336
平均值0.46240.42990.59990.750020.358422.14050.64100.7503
MSCAS-原方法↓ 7.04%↑ 25.03%↑ 8.75%↑ 17.04%
深度学习
类方法
DFV0.16710.1650.72110.724317.413617.56630.76370.7659
FV0.17850.17750.65890.659411.103711.2590.77830.7796
DFW0.09890.09820.89810.900521.296521.38390.97520.9775
AiF0.12210.12200.93030.930720.044120.04410.81570.8158
平均值0.14160.14060.80210.803717.464417.56330.83320.8346
MSCAS-原方法↓ 0.68%↑ 0.20%↑ 0.56%↑ 0.17%
FoD500模型设计
类方法
SML0.75680.65560.05880.29077.78047.42550.13230.4021
GD0.77250.76720.03820.45697.44209.02620.08630.2245
RDF0.65170.64920.41190.45148.03948.75210.46750.4990
RR0.75800.53910.21030.449313.915415.22500.20360.3114
平均值0.73480.65280.17980.41219.294310.10720.22240.3593
MSCAS-原方法↓ 11.16%↑ 129.19%↑ 8.75%↑ 61.52%
深度学习
类方法
DFV0.16670.16670.90640.906431.047231.04820.85480.8549
FV0.16670.16670.90640.906431.047531.04790.85480.8549
DFW0.08590.08570.92510.925224.253924.25410.92070.9213
AiF0.15090.15070.91040.910730.203630.20380.87420.8743
平均值0.14250.14240.91200.912129.138029.13850.87610.8764
MSCAS-原方法↓ 0.07%↑ 0.01%↑ 0.00%↑ 0.02%
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202330984, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330984, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202330984, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202330984, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1751299200000, fullTextJson=null, articleText=null, reference=null)
收藏切换
多尺度代价聚合的多聚焦图像3维形貌重建框架
闫涛 1, 2 , 尚起慧 1, 2 , 吴鹏 1 , 张江峰 1, 2 , 钱宇华 1, * , 陈斌 3, 4
计算机研究与发展 | 图形图像处理 2025,62(7): 1771-1785
收起
收藏切换
计算机研究与发展 |图形图像处理 2025 , 62 (7) : 1771 -1785
多尺度代价聚合的多聚焦图像3维形貌重建框架
全屏
[Author(id=1327013259087033329, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=hongyanyutian@sxu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013259200279540, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259087033329, language=EN, stringName=Tao Yan, firstName=Tao, middleName=null, lastName=Yan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006
2School of Computer and Information Technology, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1987. PhD, associate professor. Member of CCF. His main research interests include 3D shape reconstruction and machine vision

"}, bioImg=null, bioContent=

born in 1987. PhD, associate professor. Member of CCF. His main research interests include 3D shape reconstruction and machine vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013259275777013, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259087033329, language=CN, stringName=闫涛, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1山西大学大数据科学与产业研究院 太原 030006
2山西大学计算机与信息技术学院 太原 030006, bio={"img":"U9jyB5+vwQMaIgO0kS2vtA==","content":"

1987年生. 博士,副教授. CCF会员. 主要研究方向为3维形貌重建、机器视觉

"}, bioImg=U9jyB5+vwQMaIgO0kS2vtA==, bioContent=

1987年生. 博士,副教授. CCF会员. 主要研究方向为3维形貌重建、机器视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)]), AuthorCompany(id=1327013258764071911, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=2, ext=[AuthorCompanyExt(id=1327013258776654824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer and Information Technology, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258793432041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山西大学计算机与信息技术学院 太原 030006)])]), Author(id=1327013259351274487, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013259460326394, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259351274487, language=EN, stringName=Qihui Shang, firstName=Qihui, middleName=null, lastName=Shang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006
2School of Computer and Information Technology, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1998. Master candidate. His main research interests include 3D shape reconstruction and data mining

"}, bioImg=null, bioContent=

born in 1998. Master candidate. His main research interests include 3D shape reconstruction and data mining

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013259556795387, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259351274487, language=CN, stringName=尚起慧, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1山西大学大数据科学与产业研究院 太原 030006
2山西大学计算机与信息技术学院 太原 030006, bio={"img":"Jn9mWr8YGSqUXauJIGul4A==","content":"

1998年生. 硕士研究生. 主要研究方向为3维形貌重建、数据挖掘

"}, bioImg=Jn9mWr8YGSqUXauJIGul4A==, bioContent=

1998年生. 硕士研究生. 主要研究方向为3维形貌重建、数据挖掘

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)]), AuthorCompany(id=1327013258764071911, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=2, ext=[AuthorCompanyExt(id=1327013258776654824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer and Information Technology, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258793432041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山西大学计算机与信息技术学院 太原 030006)])]), Author(id=1327013259644875773, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013259732956159, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259644875773, language=EN, stringName=Peng Wu, firstName=Peng, middleName=null, lastName=Wu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1987. PhD, associate professor. Member of CCF. His main research interests include real time operating system and blockchain

"}, bioImg=null, bioContent=

born in 1987. PhD, associate professor. Member of CCF. His main research interests include real time operating system and blockchain

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013259812647936, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259644875773, language=CN, stringName=吴鹏, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1山西大学大数据科学与产业研究院 太原 030006, bio={"img":"6rzU65zHLYGJOzT37Ujp2A==","content":"

1987年生. 博士,副教授. CCF会员. 主要研究方向为实时操作系统、区块链

"}, bioImg=6rzU65zHLYGJOzT37Ujp2A==, bioContent=

1987年生. 博士,副教授. CCF会员. 主要研究方向为实时操作系统、区块链

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)])]), Author(id=1327013259896532993, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013259976224772, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259896532993, language=EN, stringName=Jiangfeng Zhang, firstName=Jiangfeng, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006
2School of Computer and Information Technology, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1998. PhD candidate. Student member of CCF. His main research interests include deep learning and 3D shape reconstruction

"}, bioImg=null, bioContent=

born in 1998. PhD candidate. Student member of CCF. His main research interests include deep learning and 3D shape reconstruction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013261632974853, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013259896532993, language=CN, stringName=张江峰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1山西大学大数据科学与产业研究院 太原 030006
2山西大学计算机与信息技术学院 太原 030006, bio={"img":"n0VFQ7ePVabitkH9rnxlfA==","content":"

1998年生. 博士研究生. CCF学生会员. 主要研究方向为深度学习、3维形貌重建

"}, bioImg=n0VFQ7ePVabitkH9rnxlfA==, bioContent=

1998年生. 博士研究生. CCF学生会员. 主要研究方向为深度学习、3维形貌重建

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)]), AuthorCompany(id=1327013258764071911, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=2, ext=[AuthorCompanyExt(id=1327013258776654824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer and Information Technology, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258793432041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258764071911, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山西大学计算机与信息技术学院 太原 030006)])]), Author(id=1327013261796552711, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=jinchengqyh@sxu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1327013261872050185, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013261796552711, language=EN, stringName=Yuhua Qian, firstName=Yuhua, middleName=null, lastName=Qian, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006, bio={"content":"

born in 1976. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence and machine learning

"}, bioImg=null, bioContent=

born in 1976. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence and machine learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013261947547658, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013261796552711, language=CN, stringName=钱宇华, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1山西大学大数据科学与产业研究院 太原 030006, bio={"img":"VDv418oE1ouEPpe6PydM2w==","content":"

1976年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习

"}, bioImg=VDv418oE1ouEPpe6PydM2w==, bioContent=

1976年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258629854180, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=1, ext=[AuthorCompanyExt(id=1327013258646631397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006), AuthorCompanyExt(id=1327013258667602918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258629854180, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山西大学大数据科学与产业研究院 太原 030006)])]), Author(id=1327013262031433740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327013262140485647, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013262031433740, language=EN, stringName=Bin Chen, firstName=Bin, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=3, 4, address=3Chongqing Research Institute of Harbin Institute of Technology, Chongqing 401151
4International Research Institute for Artificial Intelligence, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055, bio={"content":"

born in 1970. PhD, professor, PhD supervisor. Member of CCF. His main research interests include artificial intelligence and large language models

"}, bioImg=null, bioContent=

born in 1970. PhD, professor, PhD supervisor. Member of CCF. His main research interests include artificial intelligence and large language models

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327013262211788816, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, authorId=1327013262031433740, language=CN, stringName=陈斌, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=3, 4, address=3哈尔滨工业大学重庆研究院 重庆 401151
4哈尔滨工业大学(深圳)国际人工智能学院 广东深圳 518055, bio={"img":"oTPuW0n+sv3ZMR7EchdX4Q==","content":"

1970年生. 博士,教授,博士生导师. CCF会员. 主要研究方向为人工智能、大语言模型

"}, bioImg=oTPuW0n+sv3ZMR7EchdX4Q==, bioContent=

1970年生. 博士,教授,博士生导师. CCF会员. 主要研究方向为人工智能、大语言模型

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327013258889901034, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=3, ext=[AuthorCompanyExt(id=1327013258898289643, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258889901034, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Chongqing Research Institute of Harbin Institute of Technology, Chongqing 401151), AuthorCompanyExt(id=1327013258906678252, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258889901034, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3哈尔滨工业大学重庆研究院 重庆 401151)]), AuthorCompany(id=1327013258982175725, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, xref=4, ext=[AuthorCompanyExt(id=1327013258994758638, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258982175725, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4International Research Institute for Artificial Intelligence, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055), AuthorCompanyExt(id=1327013259007341551, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327013251445011390, companyId=1327013258982175725, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4哈尔滨工业大学(深圳)国际人工智能学院 广东深圳 518055)])])]
闫涛1, 2 , 尚起慧1, 2, 吴鹏1, 张江峰1, 2, 钱宇华1, * , 陈斌3, 4
作者信息
  • 1山西大学大数据科学与产业研究院 太原 030006
  • 2山西大学计算机与信息技术学院 太原 030006
  • 3哈尔滨工业大学重庆研究院 重庆 401151
  • 4哈尔滨工业大学(深圳)国际人工智能学院 广东深圳 518055
通讯作者:
钱宇华()
作者简介:

1987年生. 博士,副教授. CCF会员. 主要研究方向为3维形貌重建、机器视觉

1998年生. 硕士研究生. 主要研究方向为3维形貌重建、数据挖掘

1987年生. 博士,副教授. CCF会员. 主要研究方向为实时操作系统、区块链

1998年生. 博士研究生. CCF学生会员. 主要研究方向为深度学习、3维形貌重建

1976年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习

1970年生. 博士,教授,博士生导师. CCF会员. 主要研究方向为人工智能、大语言模型

Multi-Scale Cost Aggregation Framework for 3D Shape Reconstruction from Multi-Focus Images
Tao Yan1, 2 , Qihui Shang1, 2, Peng Wu1, Jiangfeng Zhang1, 2, Yuhua Qian1, * , Bin Chen3, 4
Affiliations
  • 1Institute of Big Data Science and Industry, Shanxi University, Taiyuan 030006
  • 2School of Computer and Information Technology, Shanxi University, Taiyuan 030006
  • 3Chongqing Research Institute of Harbin Institute of Technology, Chongqing 401151
  • 4International Research Institute for Artificial Intelligence, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055
  • born in 1987. PhD, associate professor. Member of CCF. His main research interests include 3D shape reconstruction and machine vision

    born in 1998. Master candidate. His main research interests include 3D shape reconstruction and data mining

    born in 1987. PhD, associate professor. Member of CCF. His main research interests include real time operating system and blockchain

    born in 1998. PhD candidate. Student member of CCF. His main research interests include deep learning and 3D shape reconstruction

    born in 1976. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence and machine learning

    born in 1970. PhD, professor, PhD supervisor. Member of CCF. His main research interests include artificial intelligence and large language models

出版时间: 2025-07-01 doi: 10.7544/issn1000-1239.202330984
文章导航
收藏切换

多聚焦图像3维形貌重建旨在利用不同聚焦水平的图像序列恢复场景的3维结构信息. 现有的3维形貌重建方法大多从单一尺度对图像序列的聚焦水平进行评价,通过引入正则化或后处理方法引导重建过程,由于深度信息选择空间的局限性往往导致重建结果无法有效收敛. 针对上述问题,提出一种多尺度代价聚合的多聚焦图像3维形貌重建框架(multi-scale cost aggregation framework for 3D shape reconstruction from multi-focus images,MSCAS),该框架首先引入非降采样的多尺度变换增加输入图像序列的深度信息选择空间,然后联合尺度内序列关联与尺度间信息约束进行代价聚合,通过这种扩张-聚合模式实现了场景深度表征信息的倍增与跨尺度和跨序列表征信息的有效融合. 作为一种通用框架,MSCAS框架可实现已有模型设计类方法和深度学习类方法的嵌入进而实现性能提升. 实验结果表明:MSCAS框架在嵌入模型设计类SFF方法后4组数据集中的均方根误差(root mean squared error,RMSE)平均下降14.91个百分点,结构相似度(structural similarity index measure,SSIM)平均提升56.69个百分点,嵌入深度学习类SFF方法后4组数据集中的RMSE平均下降1.55个百分点,SSIM平均提升1.61个百分点. 验证了MSCAS框架的有效性和通用性.

多尺度代价聚合  /  深度估计  /  聚焦测量  /  多聚焦图像序列  /  3维形貌重建

3D shape reconstruction aims to recover the 3D structure information of the scene by using image sequences with different focus levels. Most of the existing 3D shape reconstruction methods evaluate the focus level of the image sequence from a single scale, and guide the reconstruction process by introducing regularization or post-processing methods. Due to the limitation of the selection space of depth information, the reconstruction results often cannot converge effectively. To address this issue, we propose a multi-scale cost aggregation framework for shape from focus (MSCAS). Firstly, non-downsampling multi-scale transformation is introduced to increase the depth information selection space of the input image sequence, and then the cost aggregation is performed by combining the intra-scale sequence correlation and the inter-scale information constraint. Through this expansion-aggregation mode, the doubling of scene depth representation information and the effective fusion of cross-scale and cross-sequence representation information are realized. As a general framework, MSCAS framework can embed existing model design methods and deep learning methods to achieve performance improvement. The experimental results show that MSCAS framework in this paper reduces the root mean square error (RMSE) on average by 14.91% and improves the structural similarity (SSIM) by 56.69% in the four datasets after embedding the model design class SFF method. After embedding the deep learning class SFF method, RMSE in the four datasets decreases by an average of 1.55% and SSIM increases by an average of 1.61%. These results verify the effectiveness of MSCAS framework.

multi-scale cost aggregation  /  depth estimation  /  focus measure  /  multi-focus image sequence  /  3D shape reconstruction
闫涛, 尚起慧, 吴鹏, 张江峰, 钱宇华, 陈斌. 多尺度代价聚合的多聚焦图像3维形貌重建框架. 计算机研究与发展, 2025 , 62 (7) : 1771 -1785 . DOI: 10.7544/issn1000-1239.202330984
Tao Yan, Qihui Shang, Peng Wu, Jiangfeng Zhang, Yuhua Qian, Bin Chen. Multi-Scale Cost Aggregation Framework for 3D Shape Reconstruction from Multi-Focus Images[J]. Journal of Computer Research and Development, 2025 , 62 (7) : 1771 -1785 . DOI: 10.7544/issn1000-1239.202330984
深度估计是计算机视觉和3维形貌重建领域的重要任务之一,目前基于图像的深度估计主要包括单目与多目深度估计. 其中,单目深度估计主要包括基于聚焦信息深度估计(shape from focus,SFF)[1],还有基于阴影[2]、纹理[3]和虚焦[4]等线索的深度估计方法;多目深度估计主要包括双目视觉[5]和4D光场[6]等深度估计方法. 由于双目视觉深度估计的准确性依赖于配准算法的有效性,而4D光场方法则依赖于复杂的硬件设备,因此多目深度估计通常缺乏场景普适性. 单目深度估计方法因其数据采集端无差别化与数据易获取等特点,近年来广受学术界与工业界的关注[7]. 鉴于图像的阴影、纹理和轮廓等深度线索大多采用单幅图像估计场景深度信息,较少的先验信息使其无法适用于高精度的3维形貌重建场景. 而SFF重建则采用多幅不同聚焦水平的图像序列进行场景深度估计,可实现诸多场景深度更为精确的深度信息估计,广泛应用于精密制造产品质量控制、大气环境颗粒物形态学分析、生物医学分子结构观测等领域[8].
SFF重建首先通过等间隔调整相机与重建场景之间的距离,获得可以覆盖重建场景全部景深范围的不同聚焦水平的图像序列,然后采用聚焦测量方法计算图像序列的聚焦水平,最后利用后处理方法融合所有位置最清晰像素的索引,进而得到场景的3维形貌重建结果. 然而现有的聚焦测量方法具有典型的场景偏好性,使其无法有效应对复杂场景中的噪声和稀疏纹理的干扰,导致深度信息估计容易出现偏差;与此同时,现有的后处理方法难以有效联合图像序列中的关联关系与不同尺度间互补信息,无法捕捉不同场景的多尺度特征,导致深度误差被放大和积累,降低了场景的重建精度[9].
图1展示了不同尺度和噪声对场景深度估计结果造成的影响. 图1(a)为模型设计类RR算法[10](2021/TIP)在加入高斯噪声与尺度缩小为64×64时的重建结果;图1(b)为深度学习类FV[11]方法(2022/CVPR)在加入高斯噪声和尺度缩小为64×64时的重建结果. 由图1的结果可知,相比深度学习的FV方法,模型设计类RR算法在降低尺度后重建结果有一定优势,但无法有效克服噪声的干扰,而FV方法则有一定的噪声鲁棒性,但具有典型的尺度敏感性.
综上所述,为应对现有SFF方法出现的噪声和尺度鲁棒性等问题,本文提出了一种基于多尺度代价聚合的多聚焦图像3维形貌重建框架(multi-scale cost aggregation framework for 3D shape reconstruction from multi-focus images,MSCAS). 作为一种通用框架,MSCAS框架可对大多数模型设计类SFF方法与深度学习类SFF方法进行嵌入,进而实现方法在噪声鲁棒性、尺度敏感性方面的性能提升,本文主要贡献有2方面:
1)提出了框架MSCAS,该框架主要分为多尺度分解与跨尺度代价聚合2个关键核心步骤. 多尺度分解有助于增加场景的深度表征信息选择空间;跨尺度代价聚合不仅加入了尺度间与尺度内的双重深度信息约束,而且在整个聚合过程中充分考虑图像序列中各相邻图像之间的聚焦趋势变化关系,有效提升场景重建精度的同时加速重建结果的鲁棒性收敛.
2)MSCAS作为一种通用框架,现有的大多数模型设计类SFF方法与深度学习类SFF方法均可以嵌入MSCAS框架,实现不同类型SFF重建方法性能的提升.
多聚焦图像3维形貌重建SFF的原理是利用图像中聚焦区域反推相对深度的一种方法. 该方法通过调节成像端与场景之间的相对距离,获得场景不同焦距的图像序列;然后采用聚焦测量(focus measure,Fm)函数对图像序列进行聚焦水平评价,形成聚焦度量堆栈;最后,通过聚合所有聚焦度量堆栈最大值所在索引,即可得场景的3维重建结果. 其过程如式(1)所示.
$ {D_i} = \mathop {\arg \max }\limits_{1 \leqslant n \leqslant N} \{ {F{m_{i,n}}} \} \text{,} $
其中图像序列总数为$N$,单帧图像大小为$H \times W$,$ F{m_{i,n}} $表示第$n$张图像中第$ i$个像素索引位置的焦点测量结果,${D_i}$表示第$i$个像素索引位置对应的深度信息,最后对深度结果进行3维空间映射可得到待测场景的3维结构信息[12].
多聚焦图像3维形貌重建可分为模型设计重建与深度学习重建两大类方法.
模型设计重建方法按照处理流程可以分为聚焦测量与后处理2个关键步骤[13]. 其中,聚焦测量主要用来评价图像序列中的聚焦水平,进而获得场景的初始深度图,该过程根据聚焦信息判断的差异可以分为空间域、频率域、统计类3类. 空间域聚焦测量评价方法主要通过聚焦测量算子与图像的卷积操作判断图像的聚焦信息,代表性的聚焦测量算子包括Laplace算子及其变体[14]、梯度类算子[15]、可变形算子等[16];频率域聚焦测量方法主要将图像变换至频域空间,然后利用高频信息判断图像的聚焦水平,代表性方法包括非降采样小波变换[17]、非降采样剪切波变换[18]、Curvelet变换[19]等;统计类聚焦测量方法主要通过统计学指标计算图像的聚焦程度,代表性方法包括基于熵的聚焦测量算子和本征值等[20]. 由于聚焦测量过程形成的初始深度会包含较多的深度不连续信息,后处理主要针对初始深度图的修复展开,典型的方法包括罚函数法、正则化、迭代修复法[21]等. 由于模型设计类聚焦判别重建方法依赖于聚焦测量方法设计的鲁棒性与有效性,其对聚焦信息判断的误差可能会传播至后处理过程,导致重建结果无法收敛. 因此,深度学习类重建方法正逐渐引起研究者们的关注.
深度学习类重建方法首先需要构建多聚焦图像序列与深度真值数据集,然后通过训练端到端的深度网络模型对场景的深度信息进行预测. 因此,现有的深度学习类重建方法主要包括数据集构建与深度网络模型设计2类.
由于多聚焦图像3维形貌重建领域缺乏标准的真实数据集,现有的数据集大多通过其他任务的数据移植而来. 如LF_4D_PSF数据集[22]来自于4维光场重建领域,该数据集包括人为设计的多组室内场景,数据集包含场景的全聚焦图像与深度图像,通过深度图像的点扩散函数与全聚焦图像的卷积操作获得多幅不同聚焦水平的图像序列;Middlebury_FS数据集[23]来源于多目立体匹配领域,该数据集通过RGB相机与结构光扫描仪获取场景图与视差图;FlyingThings3D数据集[24]来源于光流估计领域,主要通过深度网络将光流估计扩展至视差估计;FoD500数据集[25]来自于虚焦深度估计领域,但该数据无法有效覆盖场景完整的深度范围. 基于上述高质量的合成数据集,借助相关网络模型可实现多聚焦图像的3维形貌结构的预测与推理.
在网络模型设计方面,DDFF模型主要提出一种单纯的端到端的卷积神经网络,并利用构建的DDFF-12数据集进行训练与预测[26];AiF在网络模型中设计了一个共享的中间注意力模块,可实现场景中全聚焦图像与深度图像的同时推理[27];FV与DFV网络模型重新审视了SFF重建问题的本质,构建了聚焦差分模块实现场景的深度信息预测[11]; DFW模型提出一种对齐网络实现了错位图像序列的配准,然后设计了聚焦信息检测模块获取场景的焦点信息[22];DSFFNet通过设计可变形卷积与局部-全局关系耦合模块提升模型的全局聚焦信息捕获能力[28]. 因此,深度学习类SFF重建方法致力于对网络模型进行有效设计,进而提升多聚焦图像序列中聚焦体积特征刻画的准确性.
代价聚合源于双目立体匹配问题[29],通常情况下双目立体匹配算法首先需要进行代价计算,即在一定的视差搜索范围内计算待匹配像素与候选像素之间的相关性,这种相关性通过代价值体现,由于代价计算过程通常仅考虑局部信息,极易受到噪声与弱纹理区域的影响. 因此代价聚合则是基于一些特定假设(如相邻像素的视差值一致)使得聚合后的代价值能够更为准确地反映像素之间的相关性,其原理是将一个位置的加权响应传播到附近位置,该权重由参考图像像素之间的先验知识确定[30]. 而在SFF重建中,对不同聚焦水平图像序列的聚焦测量函数Fm可以视为一种代价量变体,代价聚合过程亦可对Fm值进行优化.
代价聚合可分为局部方法和全局方法. 局部代价聚合方法的一个例子是边缘保持过滤器. 由于深度图中的异质过渡区域通常对应于场景图像中的边缘,因此可以使用双边滤波器和图像引导滤波器[31]作为代价聚合方法. 这些滤波器使用局部窗口优化代价量,同时保留引导图像中的边缘,由于局部代价聚合方法中仅仅利用了局部特征进行聚合,得到的视差对全局特征效果较差. 全局代价聚合方法将全局特征整合为一个能量函数,能量函数主要对整幅视差图进行能量评估,最小化能量函数可得到最终视差图,但其计算复杂度高且效率低下[32]. 上述方法[31-32]基本仅考虑单一尺度情况,由于单一尺度的代价聚合容易受到噪声的影响,特别是对弱纹理或无纹理区域的聚合,可能会导致视差计算过程中误差的传播. 而多尺度代价聚合[33]在捕获不同尺度的多层次特征时,能够对不同尺度的图像局部信息进行融合与筛选[34], 由于现有的多尺度代价聚合方法在分解过程存在降采样操作,且这种操作在特征融合过程中可能会出现信息损失,这无法直接应用于SFF重建过程,因为SFF重建严重依赖于图像的细节特征,任何降采样操作都可能损失一些关键深度表征信息. 因此,设计非降采样操作的多尺度代价聚合并结合SFF重建本身具有的序列关联特点,有望解决目前SFF重建方法面临的深度信息有效聚合收敛的问题.
研究表明无论是模型设计类SFF方法还是深度学习类SFF方法,均需通过聚焦体积(focus volume,Fv)导出场景最终的深度结果[28]. 因此Fv的准确性是决定重建结果的关键. 现有的Fv的计算方法和设计模式大多从单一尺度进行评价,无法有效利用多尺度之间的有效信息. 尽管一些研究也采用了多尺度聚焦评价的方式[19],但这类方法缺乏图像序列之间关联关系的约束,容易导致深度信息的不连续.
本文提出的MSCAS框架如图2所示. MSCAS框架引入了非降采样的多尺度分解与多尺度代价聚合操作,这种扩展-聚合的模式不仅考虑了单一尺度下图像序列之间的关联关系,而且同时加入了不同尺度之间的内在联系,有助于深度表征信息的扩增与跨尺度深度信息的融合收敛. 可适配于现有主流的模型设计类SFF方法与深度学习类SFF方法中.
多尺度分解的主要目的是为了得到多聚焦图像序列的多尺度表达,由于典型的多尺度分解方法大都涉及图像序列的降采样操作,这会导致后续聚合过程中深度表征信息的缺失. Atrous多尺度分解,因其滤波器上采样操作替代图像的下采样操作,可保证分解前后的信息在尺度上的一致性,广泛应用于图像融合[35]和图像去噪[36]等领域. 目前,已有研究利用Atrous多尺度分解算法应用于多聚焦图像序列融合[7],但该研究采用Atrous多尺度分解仅为获得质量较好的融合图像来引导深度信息,未考虑不同尺度对深度信息本身存在的影响. 除此之外,也有研究利用Atrous多尺度分解算法解决SFF问题[21],但该研究仅采用了其中单一尺度的信息进行深度回溯,没有充分利用多尺度信息蕴含的丰富的深度线索. 因此,本文首先对多聚焦图像序列进行Atrous多尺度分解,可保证分解后的图像序列与原始图像序列的大小一致,然后在此基础上充分挖掘不同尺度中的深度线索.
定义1. 给定输入图像序列中图像的大小为$H \times W$,则$N$幅图像序列组成的多聚焦图像序列可表示为
$ {\boldsymbol I} = ( {{I_i}\left( {x,y} \right)} )_{i = 1}^N\text{,} $
其中$ 1\le x,y\le N $.
定义2. 给定一个$S$级塔式分解滤波器组${{\boldsymbol H}_s}\left( z \right), 0 \leqslant s \leqslant S$,则多聚焦图像序列${\boldsymbol I}$在$S + 1$个尺度的低频子图${{\boldsymbol I}^s}$可表示为
$ {{\boldsymbol I}^s} = {{\boldsymbol{H}}_s}\left( z \right) * {\boldsymbol I},\;\;0 \leqslant s \leqslant S, 1\leqslant z\leqslant N, $
其中${{\boldsymbol H}_s}\left( z \right)$为滤波器组$\{ {{\boldsymbol H}_s}\} $的$z$变换,表达式为
$ {{\boldsymbol H}_s}\left( z \right) = \prod\limits_{j = 0}^s {{{\boldsymbol H}_0}({z^{{2^j}}})} ,\;\;0 \leqslant s \leqslant S, 1\leqslant z\leqslant N,$
其中${{\boldsymbol H}_0}(z)$为低通分解滤波器.
定义2中后一级滤波器${{\boldsymbol H}_0}({z^{{2^{j + 1}}}})$由前一级滤波器${{\boldsymbol H}_0}({z^{{2^j}}})$按照采样矩阵2E(E为2阶单位矩阵)进行上采样得到,因此经过塔式滤波器后的多尺度图像序列${{\boldsymbol I}^s}$与原图像序列${\boldsymbol I}$具有相同大小,这种不同尺度对齐的方式不仅可减少后续多尺度代价聚合过程中复杂的权重划分问题,而且可有效保留不同尺度之间同一位置的深度表征信息,进而有助于增强深度信息判断的准确性.
多尺度代价聚合是MSCAS框架的核心,该模块的主要目的是在不同尺度上对聚焦体积进行逐尺度逐层聚合,单一尺度聚合过程在原有单帧图像的邻域范围的基础上加入了图像序列之间的关联关系. 跨尺度聚合过程则考虑了不同尺度的权重划分问题.
定义3. 给定一个聚焦测量(focus measure,Fm)方法,该方法可建立图像序列${\boldsymbol I}$与聚焦体积Fv之间的映射关系:
$ {\boldsymbol {Fv}} = Fm\left( {\boldsymbol I} \right)\text{,} $
其中Fm可以是模型设计类方法的聚焦测量算子或者深度学习类方法的编码过程,对于单个像素$i $位于$( {{x_i},{y_i}}) $(${x_i},{y_i}$表示像素位置)而言,$ {Fv}\left( {i,l} \right)$ 表示第$l$幅图像中$\left( {{x_i},{y_i}} \right)$位置的聚焦测量结果.
由于SFF方法对多聚焦图像序列采用单帧图像独立聚焦评价的模式,而在实际场景中广泛存在稀疏纹理和低对比度区域的情形,这可能导致聚焦体积Fv中包含较多的噪声信息. 受加权最小二乘法去噪的启发,引入代价聚合可对Fv中的噪声进行有效去除,去噪后的聚焦体积$\widetilde {{\boldsymbol {Fv}}}$表示为
$ \widetilde { {Fv}}\left( {i,l} \right) = \mathop {\arg \min }\limits_z \dfrac{1}{{{Z_i}}}\sum\limits_{j \in {{{N}}_i}} {W\left( {i,j} \right){{\left\| {z - {Fv}\left( {j,l} \right)} \right\|}^2}} \text{,} $
其中${N_i}$表示像素$i $的邻域范围,$W\left( {i,j} \right)$表示像素$i$与像素$j$之间的相似性度量,${Z_i}$为归一化常量,其表达式为
$ {Z_i} = \sum\limits_{j \in {N_i}} {W\left( {i,j} \right)} . $
由于代价聚合过程本身来自于双目立体匹配,因此典型的代价聚合过程仅考虑单幅图像邻域之间的联系. 由于多聚焦图像序列之间存在较为密切的关联关系,因此本文提出的MSCAS框架中将相似性度量过程扩展至序列之间,有助于解决低对比度或者稀疏纹理区域的有效聚合问题. 具体邻域设计的差异如图3所示.
式(6)的解可通过式(8)得到:
$ \widetilde { {Fv}}\left( {i,l} \right) = \dfrac{1}{{{Z_i}}}\sum\limits_{j \in {N_i}} {W\left( {i,j} \right) {Fv}\left( {j,l} \right)} . $
研究表明2维图像中的颜色纹理信息通常与深度图中同质和异质区域呈现一定相关性. 受引导滤波过程启发,像素间的相似性度量$W\left( {i,j} \right)$采用式(9)计算.
$ W\left( {i,j} \right) = \dfrac{1}{{{{\left| \omega \right|}^2}}}\sum\limits_{k:\left( {i,j} \right) \in {\omega _k}} {\left( {1 + \dfrac{{\left( {{I_i} - {\mu _k}} \right)\left( {{I_j} - {\mu _k}} \right)}}{{\sigma _k^2 + \varepsilon }}} \right)} \text{,} $
其中${\mu _k}$和${\sigma _k}$是图像序列${{\boldsymbol{I}}}$以第$k$个像素为中心,以$r \times r \times r$立方体范围为窗口的均值和方差. ${I_i}$和${I_j}$表示图像序列中对应位置的像素值,$|\omega |$表示该窗口中的像素数,$\varepsilon $是平滑度参数,在本文中参考文献[30]设置$\varepsilon = {0.01^2}$.
当加入尺度信息后,对于任一尺度$s \in \{ 0,1, 2, … , S\}$,其可通过式(10)独立求解.
$ {\widetilde { {Fv}}^s}\left( {{i^s},{l^s}} \right) = \mathop {\arg \min }\limits_{\{ {z^s}\} _{s = 0}^S} \dfrac{1}{{Z_{^{{i^s}}}^s}}\sum\limits_{j \in {N_{{i^s}}}} {W\left( {{i^s},{j^s}} \right){{\left\| {{z^s} - { {Fv}}{^s}\left( {{j^s},{l^s}} \right)} \right\|}^2}}. $
根据式(8)可以计算得到式(10)的解为:
$ \forall s,{\widetilde { {Fv}}^s}\left( {{i^s},{l^s}} \right) = \dfrac{1}{{Z_{{i^s}}^s}}\sum\limits_{j \in {N_{{i^s}}}} {W\left( {{i^s},{j^s}} \right)C\left( {{j^s},{l^s}} \right)} . $
由于不同尺度之间聚焦体积存在信息互补的情形,因此为有效利用不同尺度之间的联系,通过引入正则化项可实现不同尺度之间的约束. 式(10)则变为如下优化问题:
$ \begin{split} {\widehat { {Fv}}^s}\left( {{i^s},{l^s}} \right) =& \mathop {\arg \min }\limits_{\{ {z^s}\} _{s = 0}^S} \Bigg( \dfrac{1}{{Z_{^{{i^s}}}^s}}\sum\limits_{j \in {N_{{i^s}}}} {W\left( {{i^s},{j^s}} \right){{\left\| {{z^s} - { {Fv}}{^s}\left( {{j^s},{l^s}} \right)} \right\|}^2}} +\\& \lambda \sum\limits_{s = 1}^S {{{\left\| {{z^s} - {z^{s - 1}}} \right\|}^2}} \Bigg).\\[-1pt]\end{split} $
通过对式(12)求导可得
$ \begin{split}& {\dfrac{{\partial {{\widehat {\boldsymbol {Fv}}}^s}}}{{\partial {z^s}}} = \dfrac{2}{{Z_{{i^s}}^s}}\sum\limits_{{j^s} \in {N_{{i^s}}}} {W\left( {{i^s},{j^s}} \right)\left( {{z^s} - { {Fv}}{^s}\left( {{j^s},{l^s}} \right)} \right)} } + 2\lambda \left( {{z^s} - {z^{s - 1}}} \right) -\\&2\lambda \left( {{z^{s + 1}} - {z^s}} \right) = 2\left( { - \lambda {z^{s - 1}} + \left( {1 + 2\lambda } \right){z^s} - \lambda {z^{s + 1}} - {{\widetilde { {Fv}}}^s}\left( {{i^s},{l^s}} \right)} \right). \end{split} $
令$\dfrac{{\partial {{\widehat {\boldsymbol {Fv}}}^s}}}{{\partial {z^s}}} = 0$,即有
$ {\widetilde { {Fv}}^s}\left( {{i^s},{l^s}} \right) = - \lambda {z^{s - 1}} + \left( {1 + 2\lambda } \right){z^s} - \lambda {z^{s + 1}}. $
同理上述结果适用于尺度范围0~$S$的情形,因此可以建立 ${\widehat {\boldsymbol {Fv}}^s}$与${\widetilde {{\boldsymbol {Fv}}}^s}$的联系,即
$ {\boldsymbol A}{\widehat {\boldsymbol {Fv}}^s} = {\widetilde {{\boldsymbol {Fv}}}^s}\text{,} $
其中矩阵${\boldsymbol A}$是由不同尺度构成的三角矩阵,当最大尺度$S = 2$时,其表达式为
$ {\boldsymbol A} = \left( {\begin{array}{*{20}{c}}{1 + \lambda }&{ - \lambda }&0 \\ { - \lambda }&{1 + 2\lambda }&{ - \lambda } \\ 0&{ - \lambda }&{1 + \lambda } \end{array}} \right)\text{,} $
且其必然存在逆矩阵${{\boldsymbol A}^{ - 1}}$,则可得到
$ {\widehat {\boldsymbol {Fv}}^s} = {{\boldsymbol A}^{ - 1}}{\widetilde {{\boldsymbol {Fv}}}^s}. $
最后将所有尺度汇总到最精细尺度${\widehat {\boldsymbol {Fv}}^0}$:
$ {\widehat { {Fv}}^0}\left( {{i^{\,0}},{l^{\,0}}} \right) = \sum\limits_{s = 0}^S {{{\boldsymbol A}^{ - 1}}\left( {0,s} \right){{\widetilde { {Fv}}}^s}\left( {{i^s},{l^s}} \right)} . $
通过式(19)得到多尺度代价聚合后聚焦体积的深度结果:
$ {{\boldsymbol D}_{{i^{\,0}}}} = \mathop {\arg \max }\limits_{{l^{\,0}}} \left\{ {P\left( {{{\widehat { {Fv}}}^0}\left( {{i^{\,0}},{l^{\,0}}} \right)} \right)} \right\} \text{,} $
其中$ {\boldsymbol D} $为3维形貌重建结果,由于$ {i^{\,0}} $为最精细尺度, 所以有$ {i^{\,0}} = i $,$ {l^{\,0}} = l $. $ P( \cdot ) $为模型设计类方法的后处理过程或者为深度学习类方法解码过程.
通过上述多尺度代价聚合过程,MSCAS框架不仅在特定尺度考虑了图像序列间的关联关系,还将跨尺度之间的信息约束考虑在内,进而从水平的单一尺度与垂直的跨多尺度2个层面提升深度信息判断的准确性与抗噪鲁棒性.
算法1. 多尺度代价聚合SFF重建框架MSCAS.
输入:多聚焦图像序列${\boldsymbol I}$;
输出:根据式(19)得到场景的3维形貌重建结果$ {\boldsymbol D} $.
① 图像序列${\boldsymbol I}$进行Atrous多尺度分解,得到${{\boldsymbol I}^s}$, $0 \leqslant s \leqslant S$;
② 对任意尺度使用聚焦评价得到聚焦体积${\boldsymbol F}{{\boldsymbol v}^s}$, $0 \leqslant s \leqslant S$;
③ 根据式(11)对所有尺度的${\boldsymbol F}{{\boldsymbol v}^s}$进行代价聚合 得到${\widetilde {{\boldsymbol {Fv}}}^s}$;
④ 根据式(18)计算多尺度代价聚合后的聚焦体 积${\widehat {\boldsymbol {Fv}}^0}$.
本文实验可分为2部分:模型设计类SFF重建方法与深度学习类SFF重建方法. 其中模型设计类SFF重建方法均使用Matlab中的图像处理与计算机视觉工具包实现,深度学习类SFF方法则使用PyTorch框架实现,运算环境为Ubuntu 18.04.6 LTS,62.5 GB DDR4 DDIM,Intel@ Xeon® W-2255 CPU @ 3.70 GHz×20,NVIDIA GeForce RTX 4090,24 GB显存. 模型设计类SFF重建各方法均采用原算法默认参数. 深度学习类SFF重建方法中所有模型的训练设置相同,使用Adam优化器,初始学习率设置为10−4,训练100轮次,批处理大小设置为4.
本文采用SFF重建方法常用的4种多聚焦图像3维形貌重建基准数据集LF_4D_PSF[22],Middlebury_FS[23],FlyingThings3D[24],FoD500[25]. 各数据集统计信息如表1所示.
LF_4D_PSF数据集来源于4维光场重建领域,包含24个模拟场景,每个场景提供了场景图与深度真值. 每个场景的深度真值通过点扩散函数映射至场景图以实现30幅不同聚焦水平的图像序列.
Middlebury_FS数据集来源于立体匹配3维重建领域,包含33个静态室内场景,提供了相机标定参数和结构光扫描仪获取的场景真值视差图. 每个场景包含15幅不同聚焦水平的堆栈,场景包括重复结构、遮挡、细线状物体以及无纹理区域.
FlyingThings3D数据集来源于光流估计领域,包含1 000幅随机生成的飞行物体及3D轨迹,该数据集将卷积网络光流估计的概念扩展到视差估计,每个场景包含15幅不同聚焦水平的堆栈.
FoD500数据集来源于单目深度估计领域,包含500个场景,由于该数据集采用散焦模式生成,因此每个场景包含的5幅不同聚焦水平的堆栈无法有效覆盖场景全部景深范围,因此本文实验仅比较其量化指标.
本文分别选择4种模型设计类SFF方法与4种深度学习类SFF方法进行实验分析. 各对比方法的参数设置见表2.
1)4种模型设计类SFF方法
①修正拉普拉斯和函数(sum-modified Laplacian,SML)[13]. SML作为聚焦度量算子,主要通过提取图像边缘判定图像的清晰区域,该方法对亮度变化不敏感,可得到明显聚焦峰值,表达式为
$ F{m}_{\text{SML}}(x,y)= {\displaystyle \sum _{u=-k}^{k}{\displaystyle \sum _{v=-k}^{k}\left|{I}_{xx}(x+u,y+v)\right|}+\left|{I}_{yy}(x+u,y+v)\right|\text{,}} $
其中${I_{xx}}$和${I_{yy}}$代表图像在x和y方向上的2阶偏导数,$k$表示模板的半径. $u$和$v$表示模板中的像素位置相对于当前像素$(x,y)$的偏移量.
②高斯导数(Gaussian derivative,GD)算子[13]. GD算子通过高斯平滑有效地抑制噪声,由于拥有不同大小的高斯核,具备较好的尺度不变性,表达式为
$ F{m_{{\text{GD}}}}(x,y) = - \dfrac{1}{{2{\text{π}}{\sigma ^2}}}{{\text{e}}^{ - \tfrac{{{x^2} + {y^2}}}{{2{\sigma ^2}}}}}\text{,} $
其中$ \sigma $为高斯函数的宽度标准差.
③环差滤波器(ring difference filter,RDF)[16]. 该算子通过设计不同半径的环形模板,将多个环形模板得到的局部聚焦度量值进行线性加权平均以获取图像的局部和全局信息,进而得到综合聚焦度量值. 其表达式为
$ F{m}_{\text{RDF}}=\left\{\begin{aligned}&\dfrac{2}{\text{π}{R}_{1}^{2}}\text{,}\quad\quad\quad\quad\,\left|x-y\right| < {R}_{1}\text{,}\\ &-\dfrac{2}{\text{π}\left({R}_{3}^{2}-{R}_{2}^{2}\right)}\text{,}\;\; {R}_{2}\le \left|x-y\right|\le {R}_{3}\text{,}\end{aligned}\right. $
其中${R_1},{R_2},{R_3}$分别为环半径.
④ 鲁棒正则化(robust regularization,RR)[10]. RR方法采用体积约束进行正则化,将3维形貌重建问题转化为一个罚函数优化问题,并通过最小化能量函数来实现深度估计,其表达式为:
$ F{m_{{\text{RR}}}} = \sum\limits_{p \in {\varOmega }} {{{({u_p} - F{m_{{\text{ML}}}})}^2} + \lambda {\varTheta }(u)} \text{,} $
其中${u_p}$为数据保真项,$F{m_{{\text{ML}}}}$为修正的拉普拉斯聚焦测量算子,${\varTheta }(u)$为正则化项,$\lambda $控制${u_p} $和${\varTheta }(u) $两项的相对重要性.
2)4种深度学习类SFF方法
① 深度差分聚焦体积(deep differential focus volume,DFV)网络[11]. DFV 旨在通过使用不同焦距上的堆叠特征计算一阶导数,进而捕获焦点和上下文信息辅助进行焦点分析.
②聚焦体积(focus volume,FV)网络[11]. FV将4维聚焦体积引入SFF任务,通过最大值自变量点集计算位置概率信息,具有高效实时等特点,可以快速响应图像中的轮廓和边缘信息.
③ DFW(depth from wild)[23] . DFW网络提出了可学习的基于上下文图像对齐方法,能够对聚焦图像进行有效对齐. 引入了锐化区域检测模块减少弱纹理区域中细微焦点变化引起的模糊歧义,同时设计了一种高效下采样模块提取焦点信息流.
④ AiF(all-in-focus)[27]. AiF方法连接了有监督和无监督学习,AiF提出了一种共享网络,可从不同聚焦图像序列中利用一个中间的注意力机制同时估计深度和全焦图像.
本文在重建结果评价方面采用4种评价指标:均方根误差(root mean squared error,RMSE)、结构相似度(structural similarity index measure,SSIM)、相关系数(correlation coefficient,Corr)、峰值信噪比(peak signal-to-noise ratio,PSNR),其公式为:
$ \left\{\begin{aligned}& RMS E=\sqrt{\dfrac{1}{M}{\displaystyle \sum _{i\in M}{(G{T}_{i}-{D}_{i})}^{2}}}\text{,}\\ & S S I M=\dfrac{(2{\mu }_{GT}{\mu }_{D}+{C}_{1})(2{\sigma }_{GT,D}+{C}_{2})}{({\mu }_{GT}^{2}+{\mu }_{D}^{2}+{C}_{1})({\sigma }_{GT}^{2}+{\sigma }_{D}^{2}+{C}_{2})}\text{,}\\ & Corr=\dfrac{{\displaystyle \sum _{i\in M}({D}_{i}-{\mu }_{D})(G{T}_{i}-{\mu }_{GT})}}{\sqrt{{\displaystyle \sum _{i\in M}{({D}_{i}-{\mu }_{D})}^{2}}}\sqrt{{\displaystyle \sum _{i\in M}(G{T}_{i}-{\mu }_{GT}){}^{2}}}}\text{,}\\& PS NR=10\times \mathrm{lg}\left(\dfrac{{max}\times M}{{\displaystyle \sum _{i\in M}{(G{T}_{i}-{D}_{i})}^{2}}}\right)\text{,}\end{aligned}\right. $
其中$GT$代表场景真值深度图,$D$代表预测深度图,$M$代表深度图的总像素数,${\mu}$代表深度均值,${\sigma }$代表深度标准差,${{max}}$代表图像中像素比较的最大范围.
由式(12)可知,正则化参数$\lambda $可调整不同尺度之间权重的变化,$\lambda $越小表明最精细尺度Fv起主要作用,随着$\lambda $的增加,其他尺度的权重逐渐增加. 为客观评价本文框架MSCAS对不同方法性能的提升,本节实验将正则化参数$\lambda $的范围设置为0~2,观察不同方法在LF_4D_PSF 数据集中平均RMSE指标的变化情况.
图4表明所有方法随着$\lambda $逐渐增大平均RMSE值出现降低,在$\lambda \geqslant 1$时RMSE下降幅度逐渐放缓. 因此, 针对不同方法设计不同的$\lambda $值会使得方法的性能得到有效提升,为兼顾不同方法间的性能,同时便于分析比较MSCAS框架对各方法的提升效果,本文推荐$\lambda {\text{ = }}1$作为各方法的正则化参数值.
本节选择具有稀疏区域的样本验证MSCAS框架的有效性. 由图5(a)(b)可知,场景中小矩形框中的白色瓶体因为光照原因导致了表面具有稀疏纹理的特性,而大矩形区域由于场景物体表面的金属高反光特性也产生了弱纹理区域. 由图5(c)(d)可以看出,RR方法因为未考虑图像序列间聚焦信息的变化趋势,导致其无法对这2部分区域进行有效重建,而嵌入本文框架后的RR-MSCAS方法则充分考虑了尺度内序列间关联关系与跨尺度间深度表征信息的互补特性,可对这2类区域实现有效重建.
为验证本文MSCAS框架在不同尺度下的鲁棒性,本节选择对场景的结构信息保留程度较好的平均SSIM值作为评价指标,并以LF_4D_PSF数据集为例,将该数据集扩展为512×512,256×256,128×128,64×64这4种尺度,分别在8种SFF重建方法进行 实验.
图6中不带网点的SxOA($x \in \{ 1,2,3,4\} $分别表示上述4种不同尺度)代表未嵌入MSCAS的原方法,带网点的SxMSCAS代表嵌入MSCAS后的SFF重建方法. 结果可以发现不同尺度中嵌入MSCAS框架的SxMSCAS方法的平均SSIM值有显著提升. 具体而言, 嵌入MSCAS框架后方法在512×512尺度上平均SSIM提升了15.04个百分点,在256×256尺度上平均提升了27.88个百分点,在128×128尺度上平均提升了47.14个百分点,在64×64尺度上平均提升了64.87个百分点. 实验结果呈现出尺度越低,性能提升越明显的趋势. 因此,引入MSCAS框架可有效提升不同类型SFF重建方法的尺度鲁棒性.
为验证本文MSCAS框架的噪声鲁棒性,本节以LF_4D_PSF数据集为例,分别对该数据集加入$\sigma = 0.005$,$\sigma = 0.01$,$\sigma = 0.02$,$\sigma = 0.04$这4种不同强度的高斯白噪声. 然后以平均SSIM测试8种不同算法的噪声鲁棒性. 具体从以下4个角度进行实验分析,分别是加入噪声的原方法(NxOA,$x \in \{ 1,2,3,4\} $分别表示上述4种不同强度噪声)、仅加入多尺度分解的方法(NxMS)、仅加入跨尺度代价聚合方法(NxCA)和嵌入本文MSCAS框架的方法(NxMSCAS).
由图7可知,嵌入MSCAS框架后的各SFF方法NxMSCAS在不同强度噪声干扰下性能均优于NxMS方法、NxCA方法和原方法NxOA. 其中NxCA方法的贡献显著优于NxMS方法. 具体结果为:当噪声强度$\sigma = 0.005$时(图7(a)),NxMSCAS方法、NxCA方法和NxMS方法相比于NxOA方法的平均SSIM分别提升了29.21个百分点、23.20个百分点、12.58个百分点,在$\sigma = 0.01$时(图7(b))分别提升了32.29个百分点、18.63个百分点、7.64个百分点,在$\sigma = 0.02$时(图7(c))分别提升了30.87个百分点、20.88个百分点、11.69个百分点,在$\sigma = 0.04$时(图7(d))分别提升了30.91个百分点、20.72个百分点、12.53个百分点. 上述实验表明了嵌入MSCAS框架后各SFF重建方法具有了一定的噪声鲁棒性.
本节主要从3.4节的4种客观评价指标和3维形貌重建的实际效果2个层面对本文提出的MSCAS框架进行定量与定性分析.
如表3所示,分别为将本文MSCAS框架嵌入8种不同类型的SFF重建方法中,并在LF_4D_PSF,Middlebury_FS,FlyingThings3D,FoD500这4种数据集中通过RMSE,SSIM,PSNR,Corr这4个评价指标的平均值进行统一量化分析. 由于深度学习类SFF重建方法将RMSE值做了归一化处理,为了便于统一分析比较,本节实验中将模型设计类SFF重建方法的RMSE指标也进行了归一化处理.
表3表明本文MSCAS框架对模型设计类SFF重建方法性能的提升显著优于深度学习类SFF重建方法. 这是由于深度学习类SFF重建方法基于上述数据集进行训练,可能会出现过拟合的情况,加之训练数据集场景样本多样性较少,如FoD500数据集中堆栈数仅为5,除此之外, RMSE指标均做归一化处理,所以体现在性能指标上的变化也最小.
图8选择了LF_4D_PSF数据集中的2个样本进行主观分析. 从矩形区域可以看出,本文MSCAS框架对模型设计类SFF重建方法的提升效果明显,特别是较好地聚合了GD方法原始稀疏的重建结果;除此之外,MSCAS框架可对RR方法的错误深度点进行有效聚合;最后本文MSCAS框架对深度学习类SFF算法在边缘细节方面有一定提升.
图9为Middlebury_FS数据集中选择的场景,且该数据集中的深度真值本身存在一些深度缺失区域. 从矩形区域可以看出,MSCAS框架不仅对深度信息缺失区域有一定的修复效果,而且对RDF和RR方法的细节保留和噪声去除方面有一定效果,特别是对深度学习类的FV方法的有显著提升,如场景1中的椅子与场景2中的桌子的深度层次更加趋于真值.
图10为FlyingThings3D数据集中的重建结果比较,由于该数据集通过软件合成,因此深度真值具有典型的层次结构,各算法均可以得到较为理想的结果. 从矩形框可以看出,模型设计类SFF在嵌入MSCAS框架后可以去除一些背景噪声,深度学习类SFF方法在嵌入MSCAS框架后可以得到较为清晰的边缘细节.
综上所述,本文提出的MSCAS框架无论是在细节保留还是背景去噪方面均具有一定的优势,而且可以对主流的大多数模型设计类SFF方法与深度学习类SFF重建方法进行嵌入且实现性能提升,尤其是对模型设计类SFF重建方法的性能有较大提升,对深度学习类SFF重建方法的边缘细节信息有较好的保留.
为了克服现有多聚焦图像3维形貌重建方法在重建过程中的深度信息缺乏和序列间趋势关联信息的缺失,本文提出了一种多尺度代价聚合的多聚焦图像3维形貌重建通用框架MSCAS,该框架引入了非降采样多尺度分解过程实现深度表征信息的倍增,通过多尺度代价聚合实现了尺度间与尺度内关联信息的快速收敛,可对主流的模型设计类SFF方法与深度学习类SFF方法进行嵌入并实现性能的提升. 在未来的工作中,需要进一步考虑如何根据场景内容实现不同尺度间自适应权重的划分,这将有助于算法的性能进一步提升.
作者贡献声明:闫涛提出算法框架并撰写论文;尙起慧进行实验分析;吴鹏和张江峰提出了论文修改意见;钱宇华指导了论文整体框架设计;陈斌提出了实验设计思路.

参考文献 引证文献
排序方式:
1
Nayar S K, Nakagawa Y. Shape from focus[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(8): 824−831
2
Tao M W, Srinivasan P P, Hadap S, et al. Shape estimation from shading, defocus, and correspondence using light-field angular coherence[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 546−560
3
Clerc M, Mallat S. The texture gradient equation for recovering shape from texture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 536−549
4
Lee J Y, Park R H. Complex-valued disparity: Unified depth model of depth from stereo, depth from focus, and depth from defocus based on the light field gradient[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(3): 830−841
5
Zhan Yu, Guo Xinqing, Lin Haibing, et al. Line assisted light field triangulation and stereo matching[C]//Proc of the 14th IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2013: 2792−2799
6
Li Jianqiao, Lu Mindong, Nian Ze. Continuous depth map reconstruction from light fields[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3257−3265
7
闫涛,陈斌,刘凤娴,等. 基于多景深融合模型的显微三维重建方法[J]. 计算机辅助设计与图形学学报,2017,29(9):1613−1623
Yan Tao, Chen Bin, Liu Fengxian, et al. Multi-focus image fusion model for micro 3D reconstruction[J]. Journal of Computer-Aided Design and Computer Graphics, 2017, 29(9): 1613−1623 (in Chinese)
8
闫涛,高浩轩,张江峰,等. 分组并行的实时微观三维形貌重建方法[J]. 软件学报,2024,35(4):1717−1731
Yan Tao, Gao Haoxuan, Zhang Jiangfeng, et al. Grouping parallel lightweight real-time microscopic 3D shape reconstruction method[J]. Journal of Software, 2024, 35(4): 1717−1731 (in Chinese)
9
闫涛,钱宇华,李飞江,等. 三维时频变换视角的智能微观三维形貌重建方法[J]. 中国科学:信息科学,2023,53(2):282−308
Yan Tao, Qian Yuhua, Li Feijiang, et al. Intelligent microscopic 3D shape reconstruction method based on 3D time-frequency transformation[J]. SCIENTIA SINICA Informationis, 2023, 53(2): 282−308 (in Chinese)
10
Ali U, Mahmood M T. Robust focus volume regularization in shape from focus[J]. IEEE Transactions on Image Processing, 2021, 30: 7215−7227
11
Yang Fengting, Huang Xiaolei, Zhou Zihan. Deep depth from focus with differential focus volume[C]//Proc of the 35th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 12642−12651
12
Muhammad M S, Choi T S. Sampling for shape from focus in optical microscopy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(3): 564−573
13
Pertuz S, Puig D, Garcia M A. Analysis of focus measure operators for shape-from-focus[J]. Pattern Recognition, 2013, 46(5): 1415−1432
14
Ma Z, Kim D, Shin Y G. Shape-from-focus reconstruction using nonlocal matting Laplacian prior followed by MRF based refinement[J]. Pattern Recognition, 2020, 103: 107302
15
Thelen A, Frey S, Hirsch S, et al. Improvements in shape-from-focus for holographic reconstructions with regard to focus operators, neighborhood-size, and height value interpolation[J]. IEEE Transactions on Image Processing, 2009, 18(1): 151−157
16
Jeon H G, Surh J, Im S, et al. Ring difference filter for fast and noise robust depth from focus[J]. IEEE Transactions on Image Processing, 2020, 29: 1045−1060
17
Yan Tao, Wu Peng, Qian Yuhua, et al. Multiscale fusion and aggregation PCNN for 3D shape recovery[J]. Information Sciences, 2020, 536: 277−297
18
Singh S, Gupta D, Anand R S, et al. Nonsubsampled shearlet based CT and MR medical image fusion using biologically inspired spiking neural network[J]. Biomedical Signal Processing and Control, 2015, 18: 91−101
19
Minhas R, Mohammed A A, Wu Q M J. Shape from focus using fast discrete curvelet transform[J]. Pattern Recognition, 2011, 44(4): 839−853
20
Wee C Y, Paramesran R. Measure of image sharpness using eigenvalues[J]. Information Sciences, 2007, 177(12): 2533−2552
21
Yan Tao, Hu Zhiguo, Qian Yuhua, et al. 3D shape reconstruction from multifocus image fusion using a multidirectional modified Laplacian operator[J]. Pattern Recognition, 2020, 98: 107065
22
Honauer K, Johannsen O, Kondermann D, et al. A dataset and evaluation methodology for depth estimation on 4D light fields[C]//Proc of the 13th Asian Conf on Computer Vision. Berlin: Springer, 2016: 19−34
23
Won C, Jeon H. Learning depth from focus in the wild[J]. arXiv preprint, arXiv: 2207.09658, 2022
24
Mayer N, Ilg E, Husser P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proc of the 30th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 4040−4048
25
Maximov M, Galim K, Leal L. Focus on defocus: Bridging the synthetic to real domain gap for depth estimation[C]//Proc of the 35th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 1071−1080
26
Hazirbas C, Soyer S G, Staab M C, et al. Deep depth from focus[C]//Proc of the 14th Asian Conf on Computer Vision. Berlin: Springer, 2018: 525−541
27
Wang Ninghsu, Wang Ren, Liu Yulun, et al. Bridging unsupervised and supervised depth from focus via all-in-focus supervision[C]//Proc of the 21st IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2021: 12621−12631
28
张江峰,闫涛,王克琪,等. 多景深图像聚焦信息的三维形貌重建:数据集与模型[J]. 计算机学报,2023,46(8):1734−1752
Zhang Jiangfeng, Yan Tao, Wang Keqi, et al. 3D shape reconstruction from multi depth of field images: Datasets and models[J]. Chinese Journal of Computers, 2023, 46(8): 1734−1752 (in Chinese)
29
Liu C, Yuen J, Torralba A. SIFT flow: Dense correspondence across scene and its application[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 978−994
30
Hosni A, Rheman C, Bleyer M, et al. Fast cost-volume filtering for visual correspondence and beyond[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(2): 504−511
31
He Kaiming, Sun Jian, Tang Xiaoou. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397−1409
32
Yang Qinxiong. A non-local cost aggregation method for stereo matching[C]//Proc of the 25th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2012: 1402−1409
33
Zhang Kang, Fang Yuqiang, Min Dongbo, et al. Cross-scale cost aggregation for stereo matching[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(5): 965−976
34
梁新彦,钱宇华,郭倩,等. 多粒度融合驱动的超多视图分类方法[J],计算机研究与发展,2022,59(8):1653−1667
Liang Xinyan, Qian Yuhua, Guo Qian, et al. Multi-granulation fusion-driven method for many-view classification[J]. Journal of Computer Research and Development, 2022, 59(8): 1653−1667 (in Chinese)
35
王晓慧,贾珈,蔡莲红. 基于小波图像融合的表情细节合成[J],计算机研究与发展,2013,50(2):387−393
Wang Xiaohui, Jia Jia, Cai Lianhong. Expression detail synthesis based on wavelet-based image fusion[J]. Journal of Computer Research and Development, 2013, 50(2): 387−393 (in Chinese)
36
Zhang Lei, Bao P, Wu Xiaolin. Multiscale LMMSE-based image denoising with optimal wavelet selection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2005, 15(4): 469−481
2025年第62卷第7期
105
42
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202330984
  • 接收时间:2023-12-05
  • 首发时间:2026-11-10
  • 出版时间:2025-07-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2023-12-05
  • 修回日期:2025-01-02
  • 录用日期:2025-01-26
基金
作者信息
    1山西大学大数据科学与产业研究院 太原 030006
    2山西大学计算机与信息技术学院 太原 030006
    3哈尔滨工业大学重庆研究院 重庆 401151
    4哈尔滨工业大学(深圳)国际人工智能学院 广东深圳 518055

通讯作者:

钱宇华()
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202330984
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏