检索结果-内蒙古大学图书馆

No One Left Behind: Real-World Federated Class-Incremental Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Dong, Jiahua Li, Hongliu Cong, Yang Sun, Gan Zhang, Yulun Van Gool, Luc The State Key Laboratory of Robotics Shenyang Institute of Automation Chinese Academy of Sciences Shenyang110016 China The Institutes for Robotics and Intelligent Manufacturing Chinese Academy of Sciences Shenyang110169 China The University of Chinese Academy of Sciences Beijing100049 China The Department of Civil and Environmental Engineering Hong Kong Polytechnic University Hong Kong The College of Automation Science and Engineering South China University of Technology Guangzhou510640 China The Computer Vision Lab ETH Zürich Zürich8092 Switzerland

Federated learning (FL) is a hot collaborative training framework via aggregating model parameters of decentralized local clients. However, most FL methods unreasonably assume data categories of FL framework are known and fixed in advance. Moreover, some new local clients that collect novel categories unseen by other clients may be introduced to FL training irregularly. These issues render global model to undergo catastrophic forgetting on old categories, when local clients receive new categories consecutively under limited memory of storing old categories. To tackle the above issues, we propose a novel Local-Global Anti-forgetting (LGA) model. It ensures no local clients are left behind as they learn new classes continually, by addressing local and global catastrophic forgetting. Specifically, considering tackling class imbalance of local client to surmount local forgetting, we develop a category-balanced gradient-adaptive compensation loss and a category gradient-induced semantic distillation loss. They can balance heterogeneous forgetting speeds of hard-to-forget and easy-to-forget old categories, while ensure consistent class-relations within different tasks. Moreover, a proxy server is designed to tackle global forgetting caused by Non-IID class imbalance between different clients. It augments perturbed prototype images of new categories collected from local clients via self-supervised prototype augmentation, thus improving robustness to choose the best old global model for local-side semantic distillation loss. Experiments on representative datasets verify superior performance of our model against comparison methods. The code is available at https://***/JiahuaDong/LGA. Copyright © 2023, The Authors. All rights reserved.

关键词： Distillation

Learning Versatile Convolution Filters for Efficient Visual Recognition

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Han, Kai Wang, Yunhe Xu, Chang Xu, Chunjing Wu, Enhua Tao, Dacheng State Key Laboratory of Computer Science Institute of Software Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China Noah’s Ark Lab Huawei Technologies FST University of Macau Noah’s Ark Lab Huawei Technologies School of Computer Science the Faculty of Engineering The University of Sydney 6 Cleveland St DarlingtonNSW2008 Australia

—This paper introduces versatile filters to construct efficient convolutional neural networks that are widely used in various visual recognition tasks. Considering the demands of efficient deep learning techniques running on cost-effective hardware, a number of methods have been developed to learn compact neural networks. Most of these works aim to slim down filters in different ways, e.g., investigating small, sparse or quantized filters. In contrast, we treat filters from an additive perspective. A series of secondary filters can be derived from a primary filter with the help of binary masks. These secondary filters all inherit in the primary filter without occupying more storage, but once been unfolded in computation they could significantly enhance the capability of the filter by integrating information extracted from different receptive fields. Besides spatial versatile filters, we additionally investigate versatile filters from the channel perspective. Binary masks can be further customized for different primary filters under orthogonal constraints. We conduct theoretical analysis on network complexity and an efficient convolution scheme is introduced. Experimental results on benchmark datasets and neural networks demonstrate that our versatile filters are able to achieve comparable accuracy as that of original filters, but require less memory and computation cost. Copyright © 2021, The Authors. All rights reserved.

关键词： Convolutional neural networks

Asymptotically Optimal Circuit Depth for Quantum state Preparation and General Unitary Synthesis

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Sun, Xiaoming Tian, Guojing Yang, Shuai Yuan, Pei Zhang, Shengyu State Key Lab of Processors Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing100049 China Tencent Quantum Laboratory Tencent Guangdong Shenzhen518057 China

The Quantum state Preparation problem aims to prepare an n-qubit quantum state |ψvi = Σk=02n−1 vk|k〉 from the initial state |0i⊗n, for a given unit vector v = (v0, v1, v2, . . ., v2n−1)T ∈ ℂ2n with ||v||2 = 1. The problem is of fundamental importance in quantum algorithm design, Hamiltonian simulation and quantum machine learning, yet its circuit depth complexity remains open when ancillary qubits are available. In this paper, we study quantum circuits when there are m ancillary qubits available. We construct, for any m, circuits that can prepare |ψvi in depth Õ(2n/m+n) and size O(2n), achieving the optimal value for both measures simultaneously. These results also imply a depth complexity of Θ(4n/m+n) for quantum circuits implementing a general n-qubit unitary for any m ≤ O(2n/n) number of ancillary qubits. This resolves the depth complexity for circuits without ancillary qubits. And for circuits with exponentially many ancillary qubits, our result quadratically improves the currently best upper bound of O(4n) to Θ̰ (2n). Our circuits are deterministic, prepare the state and carry out the unitary precisely, utilize the ancillary qubits tightly and the depths are optimal in a wide parameter regime. The results can be viewed as (optimal) time-space trade-off bounds, which are not only theoretically interesting, but also practically relevant in the current trend that the number of qubits starts to take off, by showing a way to use a large number of qubits to compensate the short qubit lifetime. Copyright © 2021, The Authors. All rights reserved.

关键词： Qubits

DRAC: Diabetic Retinopathy Analysis Challenge with Ultra-Wide Optical Coherence Tomography Angiography Images

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Qian, Bo Chen, Hao Wang, Xiangning Che, Haoxuan Kwon, Gitaek Kim, Jaeyoung Choi, Sungjin Shin, Seoyoung Krause, Felix Unterdechler, Markus Hou, Junlin Feng, Rui Li, Yihao Daho, Mostafa El Habib Wu, Qiang Zhang, Ping Yang, Xiaokang Cai, Yiyu Li, Huating Jia, Weiping Sheng, Bin Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong Department of Chemical and Biological Engineering The Hong Kong University of Science and Technology Hong Kong Department of Ophthalmology Shanghai Jiao Tong University Affiliated Sixth People’s Hospital Shanghai China VUNO Inc. Seoul Korea Republic of AI/DX Convergence Business Group KT Korea Republic of Johannes Kepler University Linz Linz Austria School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University China Academy for Engineering and Technology Fudan University China LaTIM UMR 1101 Inserm Brest France University of Western Brittany Brest France Department of Computer Science and Engineering The Ohio State University Ohio United States Department of Biomedical Informatics The Ohio State University Ohio United States Translational Data Analytics Institute The Ohio State University Ohio United States MoE Key Lab of Artificial Intelligence Artificial Intelligence Institute Shanghai Jiao Tong University Shanghai China School of Mechanical and Aerospace Engineering Nanyang Technological University Singapore Department of Endocrinology and Metabolism Shanghai Jiao Tong University Affiliated Sixth People’s Hospital Shanghai China Shanghai Diabetes Institute Shanghai Clinical Center for Diabetes Shanghai China

computer-assisted automatic analysis of diabetic retinopathy (DR) is of great importance in reducing the risks of vision loss and even blindness. Ultra-wide optical coherence tomography angiography (UW-OCTA) is a non-invasive and safe imaging modality in DR diagnosis system, but there is a lack of publicly available benchmarks for model development and evaluation. To promote further research and scientific benchmarking for diabetic retinopathy analysis using UW-OCTA images, we organized a challenge named "DRAC - Diabetic Retinopathy Analysis Challenge" in conjunction with the 25th International Conference on Medical Image Computing and computer Assisted Intervention (MICCAI 2022). The challenge consists of three tasks: segmentation of DR lesions, image quality assessment and DR grading. The scientific community responded positively to the challenge, with 11, 12, and 13 teams from geographically diverse institutes submitting different solutions in these three tasks, respectively. This paper presents a summary and analysis of the top-performing solutions and results for each task of the challenge. The obtained results from top algorithms indicate the importance of data augmentation, model architecture and ensemble of networks in improving the performance of deep learning models. These findings have the potential to enable new developments in diabetic retinopathy analysis. The challenge remains open for post-challenge registrations and submissions for benchmarking future methodology developments. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical tomography

NI-UDA: Graph adversarial domain adaptation from non-shared-and-imbalanced big data to small imbalanced applications

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Xiao, Guangyi Xiang, Weiwei Liu, Huan Chen, Hao Peng, Shun Guo, Jingzhi Gong, Zhiguo College of Computer Science and Electronic Engineering Hunan University Chansha410082 China Key Lab. of Intelligent Contr. Technol. for Wuling-Mountain Ecological Agriculture in Hunan Province School of Electrical and Information Engineering Huaihua University Hunan Huaihua418008 China Department of Computer and Information Science University of Macau Macau SAR999078 China State Key Laboratory of Internet of Things for Smart City Department of Computer and Information Science University of Macau Macau SAR999078 China

We propose a new general Graph Adversarial Domain Adaptation (GADA) based on semantic knowledge reasoning of class structure for solving the problem of unsupervised domain adaptation (UDA) from the big data with non-shared and imbalanced classes to specified small and imbalanced applications (NI-UDA), where non-shared classes mean the label space out of the target domain. Our goal is to leverage priori hierarchy knowledge to enhance domain adversarial aligned feature representation with graph reasoning. In this paper, to address two challenges in NI-UDA, we equip adversarial domain adaptation with Hierarchy Graph Reasoning (HGR) layer and the Source Classifier Filter (SCF). For sparse classes transfer challenge, our HGR layer can aggregate local feature to hierarchy graph nodes by node prediction and enhance domain adversarial aligned feature with hierarchy graph reasoning for sparse classes. Our HGR contributes to learn direct semantic patterns for sparse classes by hierarchy attention in self-attention, non-linear mapping and graph normalization. our SCF is proposed for the challenge of knowledge sharing from non-shared data without negative transfer effect by filtering low-confidence non-shared data in HGR layer. Experiments on two benchmark datasets show our GADA methods consistently improve the state-of-theart adversarial UDA algorithms, e.g. GADA(HGR) can greatly improve f1 of the MDD by 7.19% and GVB-GD by 7.89% respectively on imbalanced source task in Meal300 dataset. The code is available at https://***/gada. Copyright © 2021, The Authors. All rights reserved.

关键词： Data transfer

Deep symmetric adaptation network for cross-modality medical image segmentation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Han, Xiaoting Qi, Lei Yu, Qian Zhou, Ziqi Zheng, Yefeng Shi, Yinghuan Gao, Yang The State Key Laboratory for Novel Software Technology The National Institute for Healthcare Data Science The Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing University China Southeast University China The School of Data and Computer Science Shandong Women’s University China The Tencent Jarvis Lab China

Unsupervised domain adaptation (UDA) methods have shown their promising performance in the cross-modality medical image segmentation tasks. These typical methods usually utilize a translation network to transform images from the source domain to target domain or train the pixel-level classifier merely using translated source images and original target images. However, when there exists a large domain shift between source and target domains, we argue that this asymmetric structure could not fully eliminate the domain gap. In this paper, we present a novel deep symmetric architecture of UDA for medical image segmentation, which consists of a segmentation sub-network, and two symmetric source and target domain translation sub-networks. To be specific, based on two translation sub-networks, we introduce a bidirectional alignment scheme via a shared encoder and private decoders to simultaneously align features 1) from source to target domain and 2) from target to source domain, which helps effectively mitigate the discrepancy between domains. Furthermore, for the segmentation sub-network, we train a pixel-level classifier using not only original target images and translated source images, but also original source images and translated target images, which helps sufficiently leverage the semantic information from the images with different styles. Extensive experiments demonstrate that our method has remarkable advantages compared to the state-of-the-art methods in both cross-modality Cardiac and BraTS segmentation tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： Semantics

Diffusive nonreciprocity and thermal diode

学校读者我要写书评

暂无评论

Physical Review B 2021年第1期103卷 014307-014307页

作者： Ying Li Jiaxin Li Minghong Qi Cheng-Wei Qiu Hongsheng Chen Interdisciplinary Center for Quantum Information State Key Laboratory of Modern Optical Instrumentation College of Information Science and Electronic Engineering Zhejiang University Hangzhou 310027 China ZJU-Hangzhou Global Science and Technology Innovation Center Key Lab. of Advanced Micro/Nano Electronic Devices & Smart Systems of Zhejiang Zhejiang University Hangzhou 310027 China International Joint Innovation Center ZJU-UIUC Institute The Electromagnetics Academy at Zhejiang University Zhejiang University Haining 314400 China Department of Electrical and Computer Engineering National University of Singapore Singapore 117583 Singapore State Key Laboratory of Robotics and System Harbin Institute of Technology Harbin 150001 China

Wave propagation and diffusion in linear materials preserve local reciprocity in terms of a symmetric Green's function. For wave propagations, the relation between the fields entering and leaving a system is more relevant than the detailed information about the fields inside it. In such cases, the global reciprocity of the scattering off a system through several ports is more important, which is defined as the symmetric transmission between the scattering channels. When a two-port system supports nonreciprocal (electromagnetic, acoustic) wave propagation, it is a (optical, phonon) diode directly following the definition. However, to date no concrete definition or discussion has been made on the global reciprocity of diffusive processes through a multiple-port system. It thus remains unclear what are the differences and relations between the three concepts, namely, local nonreciprocity, global nonreciprocity, and diode effect in diffusion. Here, we provide theoretical analysis on the frequency-domain Green's function and define the global reciprocity of heat diffusion through a two-port system, which has a different setup from that of a thermal diode. We further prove the equivalence between a heat transfer system with broken steady-state global reciprocity and a thermal diode, assuming no temperature-dependent heat generation. The validities of some typical mechanisms in breaking the diffusive reciprocity and making a thermal diode have been discussed. Our results set a general background for future studies on symmetric and asymmetric diffusive processes.

关键词： Diffusion Heat transfer Metamaterials Diodes Green's function methods

Computational neuroscience:a frontier of the 21st century

学校读者我要写书评

暂无评论

National science Review 2020年第9期7卷 1418-1422页

作者： Xiao-Jing Wang Hailan Hu Chengcheng Huang Henry Kennedy Chengyu Tony Li Nikos Logothetis Zhong-Lin Lu Qingming Luo Mu-ming Poo Doris Tsao Si Wu Zhaohui Wu Xu Zhang Douglas Zhou Center for Neural Science New York University Center for Neuroscience Key Laboratory of Medical Neurobiology of the Ministry of Health of China School of Medicine Zhejiang University Department of Neuroscience and Department of Mathematics Center for the Neural Basis of Cognition University of Pittsburgh Université Claude Bernard Lyon 1 Inserm Stem Cell and Brain Research Institute U1208 Institute of Neuroscience State Key Laboratory of Neuroscience Chinese Academy of SciencesCAS Center for Excellence in Brain Science and Intelligence Technology Shanghai Center for Brain Science and Brain-Inspired Technology Division of Arts and Sciences and NYU-ECNU Institute of Cognitive Neuroscience NYU Shanghai School of Biomedical Engineering Hainan University Huazhong University of Science and Technology-Suzhou Institute for Brainsmatics Division of Biology and Biological Engineering California Institute of Technology Howard Hughes Medical Institute School of Electronics Engineering and Computer Science IDG/Mc Govern Institute for Brain Research PKU-Tsinghua Center for Life SciencesPeking University College of Computer Science and Technology Zhejiang University Institute of Brain-Intelligence Science and Technology Zhangjiang Lab School of Mathematical Sciences MOE-LSCand Institute of Natural Sciences Shanghai Jiao Tong University

THEORY IN NEUROscience The human brain is a biological organ,weighing about three pounds or 1.4 kg,that determines our behaviors, thoughts,emotions and consciousness. Although comprising only 2%of the total body weigh...

关键词：

Resolution Switchable Networks for Runtime Efficient Image Recognition

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Yikai Sun, Fuchun Li, Duo Yao, Anbang State Key Lab on Intelligent Technology and Systems Department of Computer Science and Technology Tsinghua University Cognitive Computing Laboratory Intel Labs China

We propose a general method to train a single convolutional neural network which is capable of switching image resolutions at inference. Thus the running speed can be selected to meet various computational resource limits. Networks trained with the proposed method are named Resolution Switchable Networks (RS-Nets). The basic training framework shares network parameters for handling images which differ in resolution, yet keeps separate batch normalization layers. Though it is parameter-efficient in design, it leads to inconsistent accuracy variations at different resolutions, for which we provide a detailed analysis from the aspect of the train-test recognition discrepancy. A multi-resolution ensemble distillation is further designed, where a teacher is learnt on the fly as a weighted ensemble over resolutions. Thanks to the ensemble and knowledge distillation, RS-Nets enjoy accuracy improvements at a wide range of resolutions compared with individually trained models. Extensive experiments on the ImageNet dataset are provided, and we additionally consider quantization problems. Code and models are available at https://***/yikaiw/RS-Nets. Copyright © 2020, The Authors. All rights reserved.

关键词： Image recognition