检索结果-内蒙古大学图书馆

Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Hongsheng Ji, Zhong Liu, Jingren Pang, Yanwei Han, Jungong The School of Electrical and Information Engineering The Tianjin Key Laboratory of Brain-Inspired Intelligence Technology Tianjin University Tianjin300072 China The Shanghai Artificial Intelligence Laboratory Shanghai200232 China The Department of Computer Science The University of Sheffield Sheffield United Kingdom

Vision Language Models (VLMs), pre-trained on large-scale image-text datasets, enable zero-shot predictions for unseen data but may underperform on specific unseen tasks. Continual learning (CL) can help VLMs effectively adapt to new data distributions without joint training, but faces challenges of catastrophic forgetting and generalization forgetting. Although significant progress has been achieved by distillation-based methods, they exhibit two severe limitations. One is the popularly adopted single-teacher paradigm fails to impart comprehensive knowledge, The other is the existing methods inadequately leverage the multimodal information in the original training dataset, instead they rely on additional data for distillation, which increases computational and storage overhead. To mitigate both limitations, by drawing on Knowledge Integration Theory (KIT), we propose a Multi-Stage Knowledge Integration network (MulKI) to emulate the human learning process in distillation methods. MulKI achieves this through four stages, including Eliciting Ideas, Adding New Ideas, Distinguishing Ideas, and Making Connections. During the four stages, we first leverage prototypes to align across modalities, eliciting cross-modal knowledge, then adding new knowledge by constructing fine-grained intra- and inter-modality relationships with prototypes. After that, knowledge from two teacher models is adaptively distinguished and re-weighted. Finally, we connect between models from intra- and inter-task, integrating preceding and new knowledge. Our method demonstrates significant improvements in maintaining zero-shot capabilities while supporting continual learning across diverse downstream tasks, showcasing its potential in adapting VLMs to evolving data distributions. © 2024, CC BY.

关键词： Zero-shot learning

Innovative 3D CTF Correction Techniques for Cryo-ET of Individual Particles

学校读者我要写书评

暂无评论

Innovative 3D CTF Correction Techniques for Cryo-ET of Indiv...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Xuan Wang Shiwei Liu Xinsheng Wang Xiaohua Wan Fa Zhang School of Computer Science and Technology Donghua University Shanghai China Key Laboratory of Brain Health Intelligent Evaluation and Intervention Ministry of Education (Beijing Institute of Technology) Beijing P. R. China School of Medical Technology Beijing Institute of Technology Beijing China

ISBN: (数字)9798350386226

ISBN: (纸本)9798350386233

Cryo-electron tomography (Cryo-ET) and subtomogram averaging techniques are highly effective in revealing high-resolution molecular structures. In this technique, accurate Contrast Transfer Function (CTF) correction is essential, as it profoundly affects image quality. This impact arises from various factors, including electron beam characteristics, sample thickness, diffraction phenomena, and lens aberrations. Traditional two-dimensional CTF(2D-CTF) correction methods become inadequate when Cryo-ET is increasingly applied to thicker samples and multi-angle imaging scenarios, since variations in defocus along the tilt axis lead to resolution deterioration. To address this challenge, a novel methodology, termed the Focus Defocus Changes Contrast Transfer Function (FDC-CTF), is put forward as a new type of three-dimensional Contrast Transfer Function (3D-CTF) method. This method geometrically determines the degree of particle defocus based on their coordinates within tomographic slices, effectively tackling the defocus gradient issues induced by sample thickness and tilting. Furthermore, a new workflow is proposed where, after obtaining the particle coordinates, each particle undergoes individual 3D-CTF correction, thereby enhancing the overall accuracy and efficiency of the process. We use two datasets to demonstrate the effectiveness of our method, showcasing its superior performance in alignment accuracy compared to existing methods. It provides a powerful solution for tomographic reconstruction.

关键词： Image quality Accuracy Three-dimensional displays Image resolution Shape Transfer functions Estimation Reconstruction algorithms Image reconstruction Lenses

SNP2Vec: Scalable Self-Supervised Pre-Training for Genome-Wide Association Study

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cahyawijaya, Samuel Yu, Tiezheng Liu, Zihan Mak, Tiffany T.W. Zhou, Xiaopu Ip, Nancy Y. Fung, Pascale Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology Hong Kong Division of Life Science State Key Laboratory of Molecular Neuroscience Molecular Neuroscience Center The Hong Kong University of Science and Technology Clear Water Bay Kowloon Hong Kong Hong Kong Center for Neurodegenerative Diseases Hong Kong Science Park Hong Kong Guangdong Provincial Key Laboratory of Brain Science Disease and Drug Development Shenzhen–Hong Kong Institute of Brain Science HKUST Shenzhen Research Institute Shenzhen China

Self-supervised pre-training methods have brought remarkable breakthroughs in the understanding of text, image, and speech. Recent developments in genomics has also adopted these pre-training methods for genome understanding. However, they focus only on understanding haploid sequences, which hinders their applicability towards understanding genetic variations, also known as single nucleotide polymorphisms (SNPs), which is crucial for genome-wide association study. In this paper, we introduce SNP2Vec, a scalable self-supervised pre-training approach for understanding SNP. We apply SNP2Vec to perform long-sequence genomics modeling, and we evaluate the effectiveness of our approach on predicting Alzheimer’s disease risk in a Chinese cohort. Our approach significantly outperforms existing polygenic risk score methods and all other baselines, including the model that is trained entirely with haploid sequences. We release our code and dataset on https://***/HLTCHKUST/ snp2vec. © 2022, CC BY-SA.

关键词： Genes

Predictive coding can do exact backpropagation on convolutional and recurrent neural networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Salvatori, Tommaso Lukasiewicz, Thomas Song, Yuhang Bogacz, Rafal Xu, Zhenghua Department of Computer Science University of Oxford United Kingdom MRC Brain Network Dynamics Unit University of Oxford United Kingdom State Key Laboratory of Reliability and Intelligence of Electrical Equipment Hebei University of Technology Tianjin China

Predictive coding networks (PCNs) are an influential model for information processing in the brain. They have appealing theoretical interpretations and offer a single mechanism that accounts for diverse perceptual phenomena of the brain. On the other hand, backpropagation (BP) is commonly regarded to be the most successful learning method in modern machine learning. Thus, it is exciting that recent work formulates inference learning (IL) that trains PCNs to approximate BP. However, there are several remaining critical issues: (i) IL is an approximation to BP with unrealistic/non-trivial requirements, (ii) IL approximates BP in single-step weight updates;whether it leads to the same point as BP after the weight updates are conducted for more steps is unknown, and (iii) IL is computationally significantly more costly than BP. To solve these issues, a variant of IL that is strictly equivalent to BP in fully connected networks has been proposed. In this work, we build on this result by showing that it also holds for more complex architectures, namely, convolutional neural networks and (many-to-one) recurrent neural networks. To our knowledge, we are the first to show that a biologically plausible algorithm is able to exactly replicate the accuracy of BP on such complex architectures, bridging the existing gap between IL and BP, and setting an unprecedented performance for PCNs, which can now be considered as efficient alternatives to BP. Copyright © 2021, The Authors. All rights reserved.

关键词： Backpropagation

Reverse Differentiation via Predictive Coding

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Salvatori, Tommaso Song, Yuhang Xu, Zhenghua Lukasiewicz, Thomas Bogacz, Rafal Department of Computer Science University of Oxford United Kingdom Mrc Brain Network Dynamics Unit University of Oxford United Kingdom State Key Laboratory of Reliability and Intelligence of Electrical Equipment Hebei University of Technology Tianjin China

Deep learning has redefined AI thanks to the rise of artificial neural networks, which are inspired by neuronal networks in the brain. Through the years, these interactions between AI and neuroscience have brought immense benefits to both fields, allowing neural networks to be used in a plethora of applications. Neural networks use an efficient implementation of reverse differentiation, called backpropagation (BP). This algorithm, however, is often criticized for its biological implausibility (e.g., lack of local update rules for the parameters). Therefore, biologically plausible learning methods that rely on predictive coding (PC), a framework for describing information processing in the brain, are increasingly studied. Recent works prove that these methods can approximate BP up to a certain margin on multilayer perceptrons (MLPs), and asymptotically on any other complex model, and that zero-divergence inference learning (Z-IL), a variant of PC, is able to exactly implement BP on MLPs. However, the recent literature shows also that there is no biologically plausible method yet that can exactly replicate the weight update of BP on complex models. To fill this gap, in this paper, we generalize (PC and) Z-IL by directly defining it on computational graphs, and show that it can perform exact reverse differentiation. What results is the first PC (and so biologically plausible) algorithm that is equivalent to BP in the way of updating parameters on any neural network, providing a bridge between the interdisciplinary research of neuroscience and deep learning. Furthermore, the above results in particular also immediately provide a novel local and parallel implementation of BP. Copyright © 2021, The Authors. All rights reserved.

关键词： Complex networks

A Portable and Wearable Visual brain-computer interface for Assessing Patients With Disorders of Consciousness

学校读者我要写书评

暂无评论

A Portable and Wearable Visual Brain-Computer Interface for ...

Electronic Information Engineering and computer Communication (EIECC), International Conference on

作者： Huang Wu Jun Xiao Chenxin Xue Di Chen Jiahui Pan Yuanqing Li School of Automation Science and Engineering South China University of Technology Guangzhou China School of Electric Power Engineering South China University of Technology Guangzhou China School of Software South China Normal University Guangzhou China Pazhou Laboratory Research Center for Brain–Computer Interface Guangzhou China

The JFK Coma Recovery Scale-Revised (JFK CRSR) is a clinical behavioral method for assessing patients with disorders of consciousness (DOC). However, due to severe brain injury, the DOC patients may lack sufficient behavioral responses, leading to a relatively high clinical misdiagnosis rate of the JFK CRS-R. To address this limitation, this study developed a portable and wearable brain-computer interface (BCI) to evaluate the visual tracking abilities of the DOC patients and enhance the diagnostic accuracy of the JFK CRS-R. During the experiment, patients wore an Electroencephalogram headband, and the Graphical User interface of the BCI presented four human images with a background captured by a camera. A target human image was randomly selected, then the four images alternately flashed until the target moved to its initial position. The system guided patients to concentrate their attention on the moving target. In a case study, the BCI system was applied to a patient diagnosed with a minimally conscious state (MCS). In contrast to the lack of visual tracking behavior in CRS-R, our experiment result showed a significant accuracy of 60% achieved by the MCS patient, emphasizing the visual tracking abilities of the MCS patients and highlighting the effectiveness of the system.

关键词：

The applied principles of EEG analysis methods in neuroscience and clinical neurology

学校读者我要写书评

暂无评论

Military Medical Research 2024年第6期11卷 907-941页

作者： Hao Zhang Qing-Qi Zhou He Chen Xiao-Qing Hu Wei-Guang Li Yang Bai Jun-Xia Han Yao Wang Zhen-Hu Liang Dan Chen Feng-Yu Cong Jia-Qing Yan Xiao-Li Li School of Systems Science Beijing Normal UniversityBeijing 100875China College of Electrical and Control Engineering North China University of TechnologyBeijing 100041China School of Automation Science and Engineering South China University of TechnologyGuangzhou 510641China Department of Psychology the State Key Laboratory of Brain and Cognitive Sciencesthe University of Hong KongHong Kong SAR 999077China HKU-Shenzhen Institute of Research and Innovation Shenzhen 518057GuangdongChina Department of Health Technology and Informatics the Hong Kong Polytechnic UniversityHong Kong SAR 999077China Department of Rehabilitation Medicine the First Afliated Hospital of Nanchang UniversityNanchang 330006China Rehabilitation Medicine Clinical Research Center of Jiangxi Province Nanchang 330006China Beijing Key Laboratory of Learning and Cognition School of PsychologyCapital Normal UniversityBeijing 100048China School of Communication Science Beijing Language and Culture UniversityBeijing 100083China Institute of Electrical Engineering Yanshan UniversityQinhuangdao 066004HebeiChina School of Computer Science Wuhan UniversityWuhan 430072China School of Biomedical Engineering Faculty of Electronic Information and Electrical EngineeringDalian University of TechnologyDalian 116081LiaoningChina Guangdong Artifcial Intelligence and Digital Economy Laboratory(Guangzhou) Guangzhou 510335China.

Electroencephalography(EEG)is a non-invasive measurement method for brain *** to its safety,high resolution,and hypersensitivity to dynamic changes in brain neural signals,EEG has aroused much interest in scientific research and medical *** article reviews the types of EEG signals,multiple EEG signal analysis methods,and the application of relevant methods in the neuroscience feld and for diagnosing neurological ***,3 types of EEG signals,including time-invariant EEG,accurate event-related EEG,and random event-related EEG,are ***,5 main directions for the methods of EEG analysis,including power spectrum analysis,time-frequency analysis,connectivity analysis,source localization methods,and machine learning methods,are described in the main section,along with diferent sub-methods and effect evaluations for solving the same ***,the application scenarios of different EEG analysis methods are emphasized,and the advantages and disadvantages of similar methods are *** article is expected to assist researchers in selecting suitable EEG analysis methods based on their research objectives,provide references for subsequent research,and summarize current issues and prospects for the future.

关键词： Electroencephalogram analysis methods Applied principles Neuroscience Diagnosis Neurological diseases

Bi-Directional Deep Contextual Video Compression

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sheng, Xihua Li, Li Liu, Dong Wang, Shiqi MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China Hefei230027 China Institute of Artificial Intelligence Hefei Comprehensive National Science Center China Department of Computer Science City University of Hong Kong Hong Kong

Deep video compression has made impressive process in recent years, with the majority of advancements concentrated on P-frame coding. Although efforts to enhance B-frame coding are ongoing, their compression performance is still far behind that of traditional bi-directional video codecs. In this paper, we introduce a bi-directional deep contextual video compression scheme tailored for B-frames, termed DCVC-B, to improve the compression performance of deep B-frame coding. Our scheme mainly has three key innovations. First, we develop a bi-directional motion difference context propagation method for effective motion difference coding, which significantly reduces the bit cost of bi-directional motions. Second, we propose a bidirectional contextual compression model and a corresponding bi-directional temporal entropy model, to make better use of the multi-scale temporal contexts. Third, we propose a hierarchical quality structure-based training strategy, leading to an effective bit allocation across large groups of pictures (GOP). Experimental results show that our DCVC-B achieves an average reduction of 26.6% in BD-Rate compared to the reference software for H.265/HEVC under random access conditions. Remarkably, it surpasses the performance of the H.266/VVC reference software on certain test datasets under the same configuration. We anticipate our work can provide valuable insights and bring up deep B-frame coding to the next level. Copyright © 2024, The Authors. All rights reserved.

关键词： Software testing

A novel dual-stage algorithm for capacitated arc routing problems with time-dependent service costs

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Qingya Liu, Shengcai Zou, Juan Tang, Ke Guangdong Key Laboratory of Brain-Inspired Intelligent Computation Department of Computer Science and Engineering Southern University of Science and Technology Shenzhen518055 China The Key Laboratory of Hunan Province for Internet of Things and Information Security Xiangtan University Xiangtan411105 China The Key Laboratory of Intelligent Computing and Information Processing Ministry of Education Xiangtan University Xiangtan411105 China

This paper focuses on solving the capacitated arc routing problem with time-dependent service costs (CARPTDSC), which is motivated by winter gritting applications. In the current literature, exact algorithms designed for CARPTDSC can only handle small-scale instances, while heuristic algorithms fail to obtain high-quality solutions. To overcome these limitations, we propose a novel dual-stage algorithm, called MAENS-GN, that consists of a routing stage and a vehicle departure time optimization stage. The former obtains the routing plan, while the the latter determines the vehicle departure time. Importantly, existing literature often ignores the characteristic information contained in the relationship between the route cost and the vehicle departure time. The most significant innovation in this paper lies in the exploitation of this characteristic information during the vehicle departure time optimization stage. Specifically, we conduct a detailed analysis of this relationship under various scenarios and employ tailored methods to obtain the (approximately) optimal vehicle departure time. Furthermore, we propose an improved initialization strategy that considers time-dependent characteristics to achieve better solution quality. In addition to the modified benchmark test sets, we also experiment on a real-world test set. Experimental results demonstrate that MAENS-GN can obtain high-quality solutions on both small-scale and larger-scale instances of CARPTDSC. Copyright © 2024, The Authors. All rights reserved.

关键词： Combinatorial optimization