Latest ArticlesIn recent years， large language models （LLMs） have achieved remarkable progress in natural language processing （NLP）， demonstrating exceptional capabilities in language understanding and generation. These advancements have driven widespread applications in tasks such as text generation， machine translation， question answering， text summarization， and text classification. However， despite their impressive performance in handling and generating text， LLMs face notable limitations when handling highly complex multimodal tasks， particularly in the domain of video question answering （Video QA）. Video QA is a particularly challenging task that requires models to comprehend and generate responses based on dynamic visual content， which often includes temporal and auditory information. Unlike static images or purely textual contents， video data contains inherent temporal dependencies， where the meaning of events and actions unfolds over time. This temporal dimension adds substantial complexity to the understanding process because models must not only interpret individual frames but also maintain coherent understanding across sequences of frames within the broader video context. Consequently， effective Video QA demands advanced temporal information processing capabilities that many LLMs， primarily designed for static text， often struggle to handle adequately. Moreover， the multimodal nature of video， which often involves the integration of visual， auditory， and occasionally textual cues， further complicates the task. Effective Video QA requires the model to seamlessly fuse information across these different modalities， ensuring accurate interpretation and response to questions regarding video content. This process involves understanding visual scenes， recognizing speech or background sounds， and correlating them with the corresponding textual information. The challenge lies not only in processing each modality independently but also in establishing meaningful connections between them to generate coherent and contextually appropriate responses. This paper presents a comprehensive review of the current state of research on Video QA models based on large language models. The technical characteristics， strengths， and weaknesses of non-real-time and real-time Video QA models are also investigated. Non-real-time Video QA models typically operate on pre-recorded video content， allowing them to access and analyze the entire video sequence before generating responses. These models can leverage global contextual information， making such models particularly effective for tasks that require video content analysis， such as video summarization or detailed scene interpretation. However， they may struggle with efficiency and scalability， particularly when handling long videos or large datasets. In contrast， real-time Video QA models are designed to process video streams as they are received， increasing their suitability for applications requiring immediate responses， such as live video monitoring or interactive video systems. However， these models must maintain a balance between processing speed and accuracy due to their frequently limited access to the full temporal context of the video. The paper discusses the challenges encountered by these models in maintaining performance under real-time constraints， including efficient computation and prediction capability based on partial information. Additionally， the paper explores the commonly used datasets in Video QA research， highlighting their features， limitations， and the types of tasks they are designed to address. The evaluation of Video QA models is also examined， focusing on the metrics and benchmarks used to assess their performance. Understanding the strengths and weaknesses of different datasets is crucial for advancing the field， helping in the identification of gaps in current research and guiding the development of robust and versatile models. Finally， the paper addresses the extensive challenges and bottlenecks in the field of Video QA， including the difficulties in scaling models to handle large and diverse video datasets， the need for efficient multimodal fusion techniques， and the computational demands associated with video data processing in real-time. The discussion is further extended to consider the potential future research directions in Video QA， with particular emphasis on improving the temporal reasoning capabilities of LLMs， enhancing their multimodal integration， and developing efficient model architectures that can operate effectively under resource constraints. Overall， while large language models have presented new possibilities in the field of video interpretation， considerable challenges remain in adapting these models to the specific demands of Video QA. Through the systematic review of the current advancements and the presentation of the key obstacles and future directions， this paper aims to contribute to the ongoing efforts to develop highly capable and intelligent multimodal AI systems. The field must continue innovations in the following areas： temporal modeling， where novel architectures that can effectively capture long-range dependencies in video sequences are needed； multimodal representation learning， where sophisticated approaches for integrating visual， auditory， and textual features could yield substantial improvements. Furthermore， the development of highly efficient training paradigms that can address the computational intensity of video processing while retaining model performance is essential for practical applications. Another critical area for future work focuses on the creation of highly comprehensive and challenging benchmark datasets that effectively reflect real-world scenarios， pushing the boundaries of what current models can achieve. As research in this area progresses， addressing these challenges will be crucial for realizing the full potential of LLMs in video interpretation applications. Achieving this goal will require AI systems that can interpret and reason about dynamic visual content with a level of proficiency comparable to human cognition. The integration of advanced techniques from computer vision， speech processing， and natural language understanding will be pivotal in developing truly multimodal systems capable of managing the complexity and variability in real-world video data. Through continued innovation and interdisciplinary collaboration， the field can overcome current limitations and drive the development of next-generation video understanding technologies with broad applicability across domains such as education， entertainment， surveillance， and human-computer interaction.
Weakly supervised semantic segmentation （WSSS） aims to reduce the cost associated with annotating “strong” pixel-level labels by using “weak” labels， such as points， bounding boxes， image-level class labels， and scribbles. Among these， image-level class labels are the most cost-effective and readily available； however， leveraging them for precise segmentation remains a considerable challenge. A widely used WSSS approach based on image-level class labels generally comprises the following steps： 1） training a neural network for image classification using the class labels； 2） using the trained network to generate class activation maps （CAMs）， which serve as seed regions for the segmentation task； and 3） refining these CAMs into pseudo-labels， which are then used as the ground truth to supervise a segmentation network. These steps can be integrated into a single collaborative stage； typically， single-stage frameworks are highly efficient due to their simplified training pipeline. However， the quality of pseudo-labels is crucial to the overall performance of semantic segmentation. High-quality pseudo-labels result in superior segmentation outcomes， whereas noisy or inaccurate pseudo-labels hinder the capability of the model to learn meaningful features. WSSS based on image-level labels faces considerable challenges due to the absence of precise positional and shape-related information， making it difficult to generate accurate segmentation maps. These challenges have led to the development of various approaches， which can be broadly categorized into two types： single-stage methods and multistage methods. Although single-stage methods offer greater efficiency and simplify the overall training process， they often produce less accurate pseudo-labels. This condition is due to the limited refinement of CAMs， resulting in imprecise supervision signals that ultimately degrade segmentation performance. Aiming to alleviate these limitations， a simple yet novel single-stage WSSS framework that incorporates knowledge distillation is introduced to enhance pseudo-label quality without relying on any additional external supervision. The framework enhances the feature learning process within the teacher-student network using a dual-stage knowledge distillation module. This module allows the student network to acquire more dynamic and informative knowledge from the teacher network while preserving key features， thereby enhancing the overall robustness of the student model. Moreover， to further improve segmentation accuracy， a pseudo-label correction module based on a Gaussian mixture model （GMM） is introduced. This module refines the pseudo-labels by modeling the distribution of the CAMs， resulting in highly accurate and reliable supervision signals. The combination of dual-stage knowledge distillation and the Gaussian correction module ensures accurate learning and improved segmentation results， even under weak supervision signals such as image-level labels. Ultimately， the proposed method effectively mitigates the impact of noise during training and enhances the accuracy of the generated pseudo-labels， resulting in superior semantic segmentation outcomes in WSSS tasks.
A novel weakly-supervised semantic segmentation method， aimed at addressing the challenges posed by noisy data points and weak supervision， is proposed. First， a dual-stage knowledge interaction module is introduced to enhance the feature learning process of the teacher and student networks. By enabling highly effective knowledge exchange between the two networks， the proposed approach notably reduces the impact of noise during training， leading to robust feature extraction. Additionally， a Gaussian correction module is proposed to enhance the quality of pseudo-labels. This module refines the pseudo-labels by modeling the distribution of class activation maps. By fitting the distribution more accurately， the module corrects potential errors in the pseudo-labels， ensuring that the model learns from high-quality， refined labels. Therefore， the method boosts the overall performance of weakly-supervised semantic segmentation， making it more robust to noise and improving segmentation accuracy. This method provides a promising solution for weakly-supervised segmentation tasks.
The mIoU values of this method on the PASCAL VOC 2012 and MS COCO 2014 datasets were 74.8% and 42.3%， respectively， surpassing other comparative methods. Specifically， on the PASCAL VOC 2012 dataset， the proposed method achieved a 3.7% improvement over ToCo， an 8.8% enhancement compared to AFA， a 7.5% increase relative to TSCD， and 1.1% compared to BECO. On the MS COCO 2014 dataset， the method improved performance by 2.2% compared to TSCD， 3.4% compared to AFA， and 5.3% compared to AuxSegNet+. Additionally， the mIoU values of different categories are compared on the PASCAL VOC 2012 validation set. The experimental results showed that the method outperformed the competing methods in 16 categories. Notably， for the background class， the method achieved an mIoU of 92.4%， the highest among all methods evaluated. This result indicates that the method effectively leverages the Gaussian correction module to reduce misclassification of background regions， thereby improving segmentation performance. Furthermore， the method achieved notable improvements in categories such as bird， bottle， car， chair， and cow， further demonstrating its effectiveness.
The proposed method effectively mitigates the impact of noise during training and address the issue of incomplete pseudo-label generation through the integration of a dual-stage knowledge distillation module and a Gaussian correction module. This approach achieves remarkable performance improvements compared to existing methods. Overall， the results demonstrate notable advantages in end-to-end weakly supervised semantic segmentation and holds considerable research value.
RGB-D salient object detection （SOD） combines complementary information from RGB and depth images， offering substantially enhanced performance in complex and challenging scenes compared to RGB-only models. This technique has gained considerable attention in the academic community due to its capability to effectively capture salient objects by leveraging visual and spatial information. However， existing RGB-D detection models face several key challenges. First， efficiently utilizing and fusing multi-modal information from RGB and depth inputs remains a difficult task due to the inherent differences between the two modalities. RGB images provide rich color and texture details but lack depth information， whereas depth maps offer spatial cues but are often noisy or of low quality. Second， achieving accurate boundary detection is particularly challenging in cluttered or noisy environments. Noisy depth maps and cluttered backgrounds can obscure object contours， making it difficult to predict sharp and precise boundaries. These challenges highlight the urgent need for a robust model that can effectively integrate RGB and depth information while simultaneously addressing noise and enhancing boundary precision.
Aiming to address these challenges， a novel method， the cross-modal feature fusion and detail-enhanced RGB-D salient object detection network （CFADNet）， is introduced. The proposed network incorporates two innovative modules： the cross-modal attention fusion enhancement module （CAFEM） and the boundary feature extraction module （BFEM）. The CAFEM is designed to enhance the integration of RGB and depth features by leveraging attention mechanisms that emphasize the most informative aspects of each modality. Specifically， channel attention is applied to the RGB features to suppress noise and enhance critical color and texture details. Similarly， spatial attention is applied to the depth features to emphasize spatial regions that are relevant for salient object detection. This attention-based fusion mechanism ensures that the model effectively retains global semantic information from the depth map while preserving fine-grained details from the RGB image. The fusion process is structured in multiple layers， progressively integrating features at different scales to fully utilize the complementary strengths of RGB and depth modalities. In contrast， the BFEM is specifically designed to improve the accuracy of salient object boundaries. Accurate contour detection is crucial for generating high-quality saliency maps； thus， BFEM leverages low-level CNN features， which are rich in edge and texture information. These features are refined through channel attention， which filters out noise and irrelevant details， enhancing the clarity of boundary-related cues. The refined features are then used to guide cross-modal feature decoding， ensuring that the final saliency maps exhibit sharp and accurate boundaries. By combining the edge-extraction capabilities of low-level CNN features with the semantic richness of cross-modal features， BFEM notably improves boundary precision in RGB-D salient object detection.
Aiming to evaluate the performance of CFADNet， extensive experiments are conducted on four widely used RGB-D salient object detection datasets： NJU2K， NLPR， STERE， and SIP. These datasets encompass a wide range of diverse and challenging scenes， making them ideal for evaluating the generalization capability of the proposed model. CFADNet is compared against 16 state-of-the-art RGB-D salient object detection methods， including DCF， CIRNet， and CAVER， using standard quantitative metrics such as mean absolute error （MAE）， F-measure（Fβ）， and structural similarity （Sα）. CFADNet demonstrated superior performance across all datasets， particularly excelling in the MAE metric. Specifically， this network outperformed the second-best method by 6.9%， 10.5%， 9.7%， and 2.4% on the NJU2K， NLPR， STERE， and SIP datasets， respectively. These substantial improvements highlight the effectiveness of the attention-based fusion strategy and edge refinement mechanisms. Furthermore， CFADNet consistently achieved higher F-measure and Sα scores， indicating that the model not only reduces pixel-level errors but also more accurately preserves the overall structure and shape of salient objects compared to competing methods. In addition to quantitative evaluations， qualitative comparisons are conducted to visually assess the performance of CFADNet in various challenging scenarios. Results show that the proposed method generates saliency maps with sharp and accurate boundaries， even in cases where salient objects exhibit complex edges or are embedded in cluttered and noisy backgrounds. This finding demonstrates the robustness of CFADNet in handling difficult scenes by effectively separating salient objects from their background while preserving fine boundary details. The visual results further confirm that CFADNet successfully captures global semantic information and local detail， ensuring accurate identification and clear isolation of salient objects from the background.
This paper presents CFADNet， a cross-modal feature fusion and detail-enhancement network for RGB-D SOD， designed to address the two major challenges： effective multimodal feature fusion and accurate boundary detection. CFADNet introduces two novel modules， the CAFEM and the BFEM. CFADNet effectively integrates RGB and depth information while notably enhancing the precision of salient object boundaries. The attention mechanisms used in the CAF0EM enable the network to fully leverage the complementary information from RGB and depth modalities. Simultaneously， the BFEM module focuses on refining edge details， resulting in sharper and more accurate saliency predictions. Extensive experiments conducted on four benchmark datasets demonstrate that CFADNet consistently outperforms existing state-of-the-art methods， achieving superior performance across key evaluation metric， including MAE， F-measure， and structural similarity index. These findings highlight the robustness and strong generalization capability of CFADNet in diverse and challenging environments. By combining attention-based feature fusion with effective edge refinement， CFADNet emerges as a powerful and reliable solution for RGB-D salient object detection into complex scenarios. Future research could explore extending this approach to other multi-modal tasks， such as RGB-Thermal or multi-spectral image processing， where challenges related to multi-modal fusion and boundary detection are also prevalent. Additionally， optimizing the computational efficiency of CFADNet for real-time deployment represents a potential research direction， enabling its application in time-sensitive applications such as autonomous driving and robotics.
Facial age estimation from images constitutes a prominent area of research within the field of computer vision， offering extensive potential applications in fields such as biometrics， digital marketing， healthcare， and human-computer interaction. Despite substantial efforts by numerous researchers in this field， achieving accurate facial age estimation remains a formidable challenge， primarily due to the lack of high-quality， large-scale labeled datasets for facial age estimation. The manual annotation of facial datasets necessitates considerable time and financial costs. Semi-supervised learning has emerged as a promising strategy for solving this problem because it enables the simultaneous utilization of labeled and unlabeled data. However， achieving satisfactory results in the domain of facial age estimation using semi-supervised learning methods is difficult. This difficulty arises from the limited accuracy of the pseudo-labels produced by these methods， as well as their susceptibility to the influence of outlier data. These factors hinder the effective utilization of unlabeled data， consequently limiting overall performance. Aiming to address these challenges， optimizing the capability of the model to extract features is essential. Such improvements will facilitate the effective acquisition of valuable representations from unlabeled data， thereby yielding highly precise pseudo-labels. Additionally， establishing a semi-supervised learning framework that can adeptly manage the challenges associated with outlier data while optimizing the utilization of the unlabeled dataset is crucial. Consequently， this study presents an open-set semi-supervised multi-task approach for facial age estimation.
This research presents the SwinLEDF model to optimize the capability of the model to extract local and global features from facial images. This model is based on the Swin Transformer architecture and integrates local enhanced feedforward （LEFF） modules along with dynamic filter networks （DFNs）. The Swin Transformer demonstrates proficient capabilities in capturing long-range dependencies and global characteristics， particularly in the analysis of age-related trends and the overall morphology of facial structures. The LEFF module incorporates non-linear transformations at the feature level， facilitating the identification of local patterns within images or feature representations. This capability is essential for differentiating age-related attributes， including intricate details such as wrinkles and skin texture. The DFN module implements a dynamic filtering operation within the spatial dimension of the model’s output， thereby enhancing model flexibility and adaptability. Furthermore， this research presents an open-set semi-supervised multitask learning algorithm to optimize the use of labeled and unlabeled data. In this algorithm， the model assesses the probability of unlabeled data being classified as outliers by integrating the outcomes of a closed-set classifier and a multi-class binary classifier. Subsequently， the model generates pseudo-labels for non-outlier data that meet a specified confidence threshold. Additionally， the model simultaneously learns to estimate sex， race， and age using labeled and unlabeled data. Through this process， the model learns not only the unique characteristics associated with each specific task but also the interrelationships among gender， race， and age， thereby enhancing the capability of the model to process diverse data and increases its expressive power and robustness. Furthermore， the process enables the effective utilization of unlabeled datasets， addressing the challenge of limited labeled data in the field of age estimation. This study employs an adaptive threshold mechanism and a negative learning strategy to optimize the use of unlabeled data. The adaptive threshold mechanism dynamically adjusts the confidence threshold for pseudo-labels based on the model’s training performance across different categories， effectively addressing category imbalance and improving the precision of pseudo-label production. The negative learning strategy enhances the handling of unlabeled data by identifying categories to which the input data does not belong， thereby mitigating the adverse effects of false pseudo-labels on model performance.
This study assesses the proposed methodology using the MORPH and UTKface datasets. On the MORPH dataset， the model exhibits a mean absolute error （MAE） of 1.908 when trained solely on labeled data. This error is further reduced to 1.885 with the inclusion of labeled and unlabeled datasets. Similarly， for the UTKface dataset， the initial MAE is recorded at 4.343 using only labeled datasets， which subsequently reduces to 4.246 following the integration of labeled and unlabeled datasets. Compared to current facial age estimation methods， the proposed approach exhibits superior performance and further optimizes its accuracy by leveraging unlabeled facial datasets.
This study introduces an open-set semi-supervised multi-task learning method for facial age estimation. The proposed method effectively extracts gender， race， and age attributes from facial images while leveraging unlabeled data and appropriately handling potential outliers. This approach addresses the challenges associated with limited labeled data， thereby enhancing the accuracy of facial age estimation. Furthermore， the methodology presents innovative strategies for achieving precise results and holds strong potential for practical applications.
Cultural relics embody the brilliance of civilization， preserve historical heritage， and uphold the national spirit， serving as vivid manifestations of the confidence and depth of Chinese cultural identity. These artifacts are not merely historical remnants； they are living testaments to a nation’s cultural consciousness and aesthetic achievements. In the Chinese context， such artifacts encompass a wide array of forms——ceramics， bronzes， calligraphy， painting， architecture， and intangible heritage such as folk music and traditional theater——which jointly form a rich， continuous narrative of human development and collective memory. Through their material and symbolic importance， these cultural elements reflect and reinforce a shared sense of belonging and historical continuity. With the rapid development of artificial intelligence （AI）， now empowering a broad spectrum of industries and becoming deeply integrated into everyday life， the field of cultural heritage is undergoing a fundamental transformation. This transformation is not only technical but also conceptual， redefining how we understand， protect， and interact with our cultural legacy. AI no longer functions solely as a tool for automation or computation； it now plays a central role in knowledge production， decision-making， and creative processes. These capacities are driving a paradigm shift in cultural heritage work——from reactive， static models to proactive， adaptive systems powered by data and learning. The cultural heritage sector， historically reliant on manual preservation， scholarly interpretation， and traditional dissemination methods， is currently being transformed by advanced algorithms， machine learning models， and intelligent data processing frameworks. The profound capabilities of AI——in areas such as image recognition， natural language processing （NLP）， data mining， semantic segmentation， and 3D reconstruction——are increasingly being leveraged to support the digitization， restoration， analysis， management， and public engagement of cultural heritage. These developments， driven by advancements in artificial intelligence， are poised to reshape the entire lifecycle of cultural relics—from their initial discovery and documentation to their long-term preservation and dynamic presentation to the public. The key challenge that currently arises—and forms the central concern of this paper—is the effective， responsible， and innovative application of AI within the cultural heritage field. While the potential of AI is undeniable， its implementation must be carefully aligned with the values， sensitivities， and interdisciplinary nature of cultural preservation. In particular， the complex materiality， symbolic importance， and contextual uniqueness of cultural relics demand AI approaches that are interpretable， ethical， and inclusive of human expertise. Ethical considerations， such as bias in training data， the risks of over-reliance on automated interpretations， and the protection of indigenous knowledge systems， must be at the forefront of AI deployment in cultural domains. This paper explores five critical dimensions of AI applications in the cultural heritage sector： prevention， preservation， research， management， and utilization. The five aspects collectively form a holistic framework for understanding how AI technologies can support the sustained vitality and accessibility of cultural resources. In terms of prevention， AI can play a crucial role in developing early warning systems to identify environmental changes and potential threats to cultural relics. By integrating sensor networks with AI-driven monitoring tools， institutions can proactively detect fluctuations in humidity， temperature， light exposure， and other environmental factors that may contribute to material deterioration. Additionally， predictive models based on historical data can be employed to forecast risks and guide strategic conservation efforts. For instance， machine learning algorithms have been employed in several European museums to predict mold outbreaks in organic cultural relics based on microclimatic data， enabling timely interventions. In terms of preservation， AI contributes to digital restoration， 3D reconstruction， and non-invasive diagnostics. For example， deep learning models can help reconstruct missing parts of fragmented murals or manuscripts by learning visual patterns from intact sections. Additionally， hyperspectral imaging combined with AI analysis can uncover faded texts or underdrawings that are invisible to the human eye. These technologies not only extend the physical lifespan of cultural relics but also introduce innovative approaches to virtual preservation. Some institutions are already using AI in the reconstruction of historical architecture through photogrammetry and simulation of ancient environments for educational use. In the domain of research， AI notably enhances the capabilities of scholars to extract knowledge from vast， heterogeneous datasets. Natural language processing facilitates the digitization and analysis of ancient texts， while computer vision supports the classification of cultural relics based on style， origin， and function. Semantic knowledge graphs and AI-assisted databases promote cross-referencing across disciplines and collections， fostering highly integrated and interdisciplinary research outcomes. These tools are proving essential in digital humanities projects that aim to map large cultural corpora or trace stylistic influences across time and geography. The management of cultural heritage institutions and resources also benefits substantially from AI. Intelligent information systems can optimize inventory tracking， automate metadata tagging， and streamline exhibition logistics. Recommendation systems can be tailored to guide curatorial decisions and enhance user interaction. AI can also help balance conservation needs with public access by dynamically regulating visitor flow in sensitive exhibition areas. Moreover， the integration of blockchain with AI for provenance tracking is emerging as a promising area， enhancing the security and transparency of cultural relic records. In terms of utilization， AI is reshaping how cultural heritage is accessed and experienced—particularly in education， tourism， and public engagement. Virtual museums， intelligent chatbots， augmented reality （AR）， and personalized content delivery are making cultural experiences highly interactive and accessible. AI-generated reconstructions and immersive simulations allow audiences to engage with history in immersive ways， expanding the reach of cultural heritage to new demographics and global audiences. Platforms such as Google Arts & Culture， along with various national museum initiatives， are increasingly leveraging AI for context-aware storytelling and multilingual access， making culture more inclusive and dynamic. Beyond practical applications， this paper also examines how traditional research methodologies are evolving in response to AI integration. The paper highlights the epistemological shifts occur as cultural interpretation moves from purely human-centered approaches to hybrid models that combine human expertise with computational inference. While AI presents powerful tools， it also raises critical questions about authenticity， authorship， and cultural sovereignty——especially when applied across diverse cultural contexts and communities. The co-construction of meaning between human curators and intelligent systems may enrich interpretations， but it also demands careful calibration of roles and responsibilities. Overall， the integration of AI into the field of cultural heritage presents an unprecedented opportunity and a profound responsibility. As we navigate this new landscape， balancing technological innovation and cultural sensitivity is essential， ensuring that AI serves as a tool for cultural empowerment， rather than erasure. This paper ultimately offers insights into the current landscape and future trajectory of AI in cultural heritage， advocating for collaborative， interdisciplinary efforts to harness the potential of AI while honoring the depth， diversity， and dignity of the world’s cultural legacies.
No. 86 Xueyuan South Road, Haidian District, Beijing
010-62199257
qkjq@cast.org.cn
Copyright © 2025 China Association for Science and Technology. All rights reserved. For all open access content, the relevant licensing terms apply.
Sponsored by the Office of the Leading Group for Cybersecurity and Informatization of CAST, and supported by Science and Technology Review Publishing House
