PDF
Full
Outline
In order to explore the connection between brain and vision and improve the clarity and accuracy of brain activity reconstruction video, a new method called high quality electroencephalogram video reconstruction (HQEEGVR) was proposed to reconstruct video from EEG (electroencephalogram) signals. Firstly, the masking spatio-temporal frequency fusion network (MSTFFNet), a three-branch EEG feature extraction network, was proposed to extract brain activity information from EEG signals and dig deeper into the semantics behind brain activity changes, spatio-temporal frequency information was extracted at the same time. Secondly, cross-modal contrast learning was introduced to align EEG, text and image features for use in the generation stage. Then, a cascade video diffusion model was proposed, specifically, the stable diffusion model was used to generate reference video frames based on EEG features, and then the video frames were used as references, motion vectors were integrated, and the video diffusion model was introduced to capture the video time features. High quality videos were ultimately generated. The results show that the model performs well in the reconstruction of the subject, motion, color and semantics of the video. It can be seen that the EEG signal can be used to capture the visual and semantic information of the brain activity, so as to reconstruct the video with high fidelity and visual authenticity.
PDF
271
116
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |
关闭全屏
No. 86 Xueyuan South Road, Haidian District, Beijing
010-62199257
qkjq@cast.org.cn
Copyright © 2025 China Association for Science and Technology. All rights reserved. For all open access content, the relevant licensing terms apply.
Sponsored by the Office of the Leading Group for Cybersecurity and Informatization of CAST, and supported by Science and Technology Review Publishing House
