检索结果-内蒙古大学图书馆

RIFLE: Backpropagation in depth for deep transfer learning through re-initializing the fully-connected layer 37

学校读者我要写书评

暂无评论

RIFLE: Backpropagation in depth for deep transfer learning t...

37th International Conference on Machine Learning, ICML 2020

作者： Li, Xingjian Xiong, Haoyi An, Haozhe Xu, Chengzhong Dou, Dejing Big Data Lab Baidu Research Beijing China Faculty of Science and Technology University of Macau China State Key Lab of IOTSC Department of Computer Science University of Macau China

ISBN: (纸本)9781713821120

Fine-tuning the deep convolution neural network (CNN) using a pre-trained model helps transfer knowledge learned from larger datasets to the target task. While the accuracy could be largely improved even when the training dataset is small, the transfer learning outcome is usually constrained by the pre-trained model with close CNN weights (Liu et al., 2019), as the backpropagation here brings smaller updates to deeper CNN layers. In this work, we propose RI- FLE- a simple yet effective strategy that deepens backpropagation in transfer learning settings, through periodically Re-Initializing the Fullyconnected LayEr with random scratch during the fine-tuning procedure. RIFLE brings meaningful updates to the weights of deep CNN layers and improves low-level feature learning, while the effects of randomization can be easily converged throughout the overall learning procedure. The experiments show that the use of RI- FLE significantly improves deep transfer learning accuracy on a wide range of datasets, outperforming known tricks for the similar purpose, such as Dropout, DropConnect, Stochastic Depth, Disturb label and Cyclic Learning Rate, under the same settings with 0.5%-2% higher testing accuracy. Empirical cases and ablation studies further indicate RIFLE brings meaningful updates to deep CNN layers with accuracy improved. © International Conference on Machine Learning, ICML 2020. All rights reserved.

关键词： Backpropagation

SinCWIm: An imputation method for single-cell RNA sequence dropouts using weighted alternating least squares

学校读者我要写书评

暂无评论

Computers in Biology and Medicine 2024年 171卷 108225-108225页

作者： Gong, Lejun Cui, Xiong Liu, Yang Lin, Cai Gao, Zhihong Jiangsu Key Lab of Big Data Security & Intelligent Processing School of Computer Science Nanjing University of Posts and Telecommunications Nanjing China Department of Burn Wound Repair and Regenerative Medicine Center The First Affiliated Hospital of Wenzhou Medical University Zhejiang Wenzhou325000 China Zhejiang Engineering Research Center of Intelligent Medicine The First Affiliated Hospital of Wenzhou Medical University Wenzhou325000 China

Background and objectives: Single-cell RNA sequencing (scRNA-seq) provides a powerful tool for exploring cellular heterogeneity, discovering novel or rare cell types, distinguishing between tissue-specific cellular composition, and understanding cell differentiation during development. However, due to technological limitations, dropout events in scRNA-seq can mistakenly convert some entries in the real data to zero. This is equivalent to introducing noise into the data of cell gene expression entries. The data is contaminated, which affects the performance of downstream analyses, including clustering, cell annotation, differential gene expression analysis, and so on. Therefore, it is a crucial work to accurately determine which zeros are due to dropout events and perform imputation operations on them. Methods: Considering the different confidence levels of different zeros in the gene expression matrix, this paper proposes a SinCWIm method for dropout events in scRNA-seq based on weighted alternating least squares (WALS). The method utilizes Pearson correlation coefficient and hierarchical clustering to quantify the confidence of zero entries. It is then combined with WALS for matrix decomposition. And the imputation result is made close to the actual number by outlier removal and data correction operations. Results: A total of eight single-cell sequencing datasets were used for comparative experiments to demonstrate the overall superiority of SinCWIm over state-of-the-art models. SinCWIm was applied to cluster the data to obtain an adjusted RAND index evaluation, and the Usoskin, Pollen and Bladder datasets scored 94.46%, 96.48% and 76.74%, respectively. In addition, significant improvements were made in the retention of differential expression genes and visualization. Conclusions: SinCWIm provides a valuable imputation method for handling dropout events in single-cell sequencing data. In comparison to advanced methods, SinCWIm demonstrates excellent performance in

关键词： RNA

Attentive Symmetric Autoencoder for Brain MRI Segmentation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Huang, Junjia Li, Haofeng Li, Guanbin Wan, Xiang Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen China School of Computer Science and Engineering Sun Yat-Sen University Guangzhou China Pazhou Lab Guangzhou510330 China

Self-supervised learning methods based on image patch reconstruction have witnessed great success in training auto-encoders, whose pre-trained weights can be transferred to fine-tune other downstream tasks of image understanding. However, existing methods seldom study the various importance of reconstructed patches and the symmetry of anatomical structures, when they are applied to 3D medical images. In this paper we propose a novel Attentive Symmetric Auto-encoder (ASA) based on Vision Transformer (ViT) for 3D brain MRI segmentation tasks. We conjecture that forcing the auto-encoder to recover informative image regions can harvest more discriminative representations, than to recover smooth image patches. Then we adopt a gradient based metric to estimate the importance of each image patch. In the pre-training stage, the proposed auto-encoder pays more attention to reconstruct the informative patches according to the gradient metrics. Moreover, we resort to the prior of brain structures and develop a Symmetric Position Encoding (SPE) method to better exploit the correlations between long-range but spatially symmetric regions to obtain effective features. Experimental results show that our proposed attentive symmetric auto-encoder outperforms the state-of-the-art self-supervised learning methods and medical image segmentation models on three brain MRI segmentation benchmarks. Copyright © 2022, The Authors. All rights reserved.

关键词： Supervised learning

AEFNet: Adaptive Scale Feature Based on Elastic-and-Funnel Neural Network for Healthcare Representation

学校读者我要写书评

暂无评论

AEFNet: Adaptive Scale Feature Based on Elastic-and-Funnel N...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Yang Liu Jialun Wu Yuhua Wei Bing Mao Chen Li Tieliang Gong School of Computer Science and Technology National Engineering Lab for Big Data Analytics Xi’an Jiaotong University Xi’an Shaanxi China

ISBN: (纸本)9781665429825

Healthcare Representation learning has been a key element to achieving state-of-the-art performance on healthcare prediction. Recent advances based Electronic Healthcare Records(EHRs) are mostly devoted to extracting temporal progression patterns with temporal model and their variants. Although these works have shown excellent performances in healthcare prediction, the unified temporal pattern may not be suitable for individuals in all healthcare conditions. Moreover, some studies ususally introduce complex Deep Neural Networks models and medical prior knowledge to get compact representation, causing great computational burden. In this paper, we propose a general health care representation model, named AEFNet. We only leverage three simple convolution operations and a set of up and down sampling to ensure performance and model complexity equally, which achieves adaptively extract distinct individual key feature in a light manner. AEFNet can shrink and refine highly suitable scale information adaptively and comletely. Breaking traditional fixed convolution scale or multi-scale, AEFNet achieves scale adaptively to extract the most significant information and context relationship. Finally, We validate our method on the public dataset MIMIC-III, and the evaluation results indicate that our method can significantly outperform other remarkable baseline models.

关键词： Adaptation models Convolution Computational modeling Neural networks MIMICs Medical services Feature extraction

Hyperbolic Geometric Latent Diffusion Model for Graph Generation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fu, Xingcheng Gao, Yisen Wei, Yuecen Sun, Qingyun Peng, Hao Li, Jianxin Li, Xianxian Key Lab of Education Blockchain and Intelligent Technology Ministry of Education Guangxi Normal University Guilin China Institute of Artificial Intelligence Beihang University Beijing China School of Software Beihang University Beijing China Beijing Advanced Innovation Center for Big Data and Brain Computing School of Computer Science and Engineering Beihang University Beijing China

Diffusion models have made significant contributions to computer vision, sparking a growing interest in the community recently regarding the application of them to graph generation. Existing discrete graph diffusion models exhibit heightened computational complexity and diminished training efficiency. A preferable and natural way is to directly diffuse the graph within the latent space. However, due to the non-Euclidean structure of graphs is not isotropic in the latent space, the existing latent diffusion models effectively make it difficult to capture and preserve the topological information of graphs. To address the above challenges, we propose a novel geometrically latent diffusion framework HypDiff. Specifically, we first establish a geometrically latent space with interpretability measures based on hyperbolic geometry, to define anisotropic latent diffusion processes for graphs. Then, we propose a geometrically latent diffusion process that is constrained by both radial and angular geometric properties, thereby ensuring the preservation of the original topological properties in the generative graphs. Extensive experimental results demonstrate the superior effectiveness of HypDiff for graph generation with various topologies. Copyright © 2024, The Authors. All rights reserved.

关键词： Topology

Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xiao, Mingqing Meng, Qingyan Zhang, Zongpeng He, Di Lin, Zhouchen National Key Lab of General AI School of Intelligence Science and Technology Peking University China The Chinese University of Hong Kong Shenzhen China Shenzhen Research Institute of Big Data China Department of Biostatistics School of Public Health Peking University China Institute for Artificial Intelligence Peking University China Peng Cheng Laboratory China

Brain-inspired neuromorphic computing with spiking neural networks (SNNs) is a promising energy-efficient computational approach. However, successfully training SNNs in a more biologically plausible and neuromorphic-hardware-friendly way is still challenging. Most recent methods leverage spatial and temporal backpropagation (BP), not adhering to neuromorphic properties. Despite the efforts of some online training methods, tackling spatial credit assignments by alternatives with comparable performance as spatial BP remains a significant problem. In this work, we propose a novel method, online pseudo-zeroth-order (OPZO) training. Our method only requires a single forward propagation with noise injection and direct top-down signals for spatial credit assignment, avoiding spatial BP's problem of symmetric weights and separate phases for layer-by-layer forward-backward propagation. OPZO solves the large variance problem of zeroth-order methods by the pseudo-zeroth-order formulation and momentum feedback connections, while having more guarantees than random feedback. Combining online training, OPZO can pave paths to on-chip SNN training. Experiments on neuromorphic and static datasets with fully connected and convolutional networks demonstrate the effectiveness of OPZO with similar performance compared with spatial BP, as well as estimated low training costs. © 2024, CC BY-NC-SA.

关键词： Backpropagation

Motif-based graph self-supervised learning for molecular property prediction

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Zaixi Liu, Qi Wang, Hao Lu, Chengqiang Lee, Chee-Kong Anhui Province Key Lab of Big Data Analysis and Application School of Computer Science and Technology University of Science and Technology of China Tencent America

Predicting molecular properties with data-driven methods has drawn much attention in recent years. Particularly, Graph Neural Networks (GNNs) have demonstrated remarkable success in various molecular generation and prediction tasks. In cases where labeled data is scarce, GNNs can be pre-trained on unlabeled molecular data to first learn the general semantic and structural information before being finetuned for specific tasks. However, most existing self-supervised pre-training frameworks for GNNs only focus on node-level or graph-level tasks. These approaches cannot capture the rich information in subgraphs or graph motifs. For example, functional groups (frequently-occurred subgraphs in molecular graphs) often carry indicative information about the molecular properties. To bridge this gap, we propose Motif-based Graph Self-supervised Learning (MGSSL) by introducing a novel self-supervised motif generation framework for GNNs. First, for motif extraction from molecular graphs, we design a molecule fragmentation method that leverages a retrosynthesis-based algorithm BRICS and additional rules for controlling the size of motif vocabulary. Second, we design a general motif-based generative pre-training framework in which GNNs are asked to make topological and label predictions. This generative framework can be implemented in two different ways, i.e., breadth-first or depth-first. Finally, to take the multi-scale information in molecular graphs into consideration, we introduce a multi-level self-supervised pre-training. Extensive experiments on various downstream benchmark tasks show that our methods outperform all state-of-the-art baselines. © 2021, CC BY-SA.

关键词： Graph neural networks

Deconfound Semantic Shift and Incompleteness in Incremental Few-shot Semantic Segmentation 39

学校读者我要写书评

暂无评论

Deconfound Semantic Shift and Incompleteness in Incremental ...

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Wu, Yirui Xia, Yuhang Li, Hao Yuan, Lixin Chen, Junyang Liu, Jun Lu, Tong Wan, Shaohua Key Laboratory of Water Big Data Technology of Ministry of Water Resources Hohai University Nanjing China College of Computer Science and Software Engineering Shenzhen University Shenzhen China School of Computing and Communication Lancaster University Lancaster United Kingdom National Key Lab for Novel Software Technology Nanjing University Nanjing China Shenzhen Institute for Advanced Study University of Electronic Science and Technology of China Shenzhen China

ISBN: (纸本)157735897X

Incremental few-shot semantic segmentation (IFSS) expands segmentation capacity of the trained model to segment new-class images with few samples. However, semantic meanings may shift from background to object class or vice versa during incremental learning. Moreover, new-class samples often lack representative attribute features when the new class greatly differs from the pre-learned old class. In this paper, we propose a causal framework to discuss the cause of semantic shift and incompleteness in IFSS, and we deconfound the revealed causal effects from two aspects. First, we propose a Causal Intervention Module (CIM) to resist semantic shift. CIM progressively and adaptively updates prototypes of old class, and removes the confounder in an intervention manner. Second, a Prototype Refinement Module (PRM) is proposed to complete the missing semantics. In PRM, knowledge gained from the episode learning scheme assists in fusing features of new-class and old-class prototypes. Experiments on both PASCAL-VOC 2012 and ADE20k benchmarks demonstrate the outstanding performance of our method. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词： Semantic Segmentation

Mandari: Multi-Modal Temporal Knowledge Graph-aware Sub-graph Embedding for Next-POI Recommendation

学校读者我要写书评

暂无评论

Mandari: Multi-Modal Temporal Knowledge Graph-aware Sub-grap...

IEEE International Conference on Multimedia and Expo (ICME)

作者： Xiaoqian Liu Xiuyun Li Yuan Cao Fan Zhang Xiongnan Jin Jinpeng Chen School of Computer Science (National Pilot Software Engineering School) Beijing University of Posts and Telecommunications Beijing China Key Laboratory of Trustworthy Distributed Computing and Service (BUPT) Ministry of Education Beijing China The Technology Innovation Center of Cultural Tourism Big Data of Hebei Province Chengde China Hebei Normal University for Nationalities Chengde China Knowledge Discovery and Data Mining Research Center Zhejiang Lab Hangzhou China

Next-POI recommendation aims to explore from user check-in sequence to predict the next possible location to be visited. Existing methods are often difficult to model the implicit association of multi-modal data with user choices. Moreover, traditional methods struggle to fully explore the variation of user preferences at variable time intervals. To tackle these limitations, we propose a Multi-Modal Temporal Knowledge Graph-aware Sub-graph Embedding approach (Mandari). We first construct a novel Multi-Modal Temporal Knowledge Graph. Based on the proposed knowledge graph, we integrate multi-modal information and leverage the graph attention network to calculate sub-graph prediction probability. Next, we implement a temporal knowledge mining method to model the segmentation and periodicity of user check-in and obtain temporal prediction probability. Finally, we fuse temporal prediction probability with the previous sub-graph prediction probability to obtain the final result. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art methods.

关键词：