检索结果-内蒙古大学图书馆

Hybrid Data-Free Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tang, Jialiang Chen, Shuo Gong, Chen School of Computer Science and Engineering Nanjing University of Science and Technology China Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education China Jiangsu Key Laboratory of Image and Video Understanding for Social Security China Center for Advanced Intelligence Project RIKEN Japan Department of Automation Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China

Data-free knowledge distillation aims to learn a compact student network from a pre-trained large teacher network without using the original training data of the teacher network. Existing collection-based and generation-based methods train student networks by collecting massive real examples and generating synthetic examples, respectively. However, they inevitably become weak in practical scenarios due to the difficulties in gathering or emulating sufficient real-world data. To solve this problem, we propose a novel method called Hybrid Data-Free Distillation (HiDFD), which leverages only a small amount of collected data as well as generates sufficient examples for training student networks. Our HiDFD comprises two primary modules, i.e., the teacher-guided generation and student distillation. The teacher-guided generation module guides a Generative Adversarial Network (GAN) by the teacher network to produce high-quality synthetic examples from very few real-world collected examples. Specifically, we design a feature integration mechanism to prevent the GAN from overfitting and facilitate the reliable representation learning from the teacher network. Meanwhile, we drive a category frequency smoothing technique via the teacher network to balance the generative training of each category. In the student distillation module, we explore a data inflation strategy to properly utilize a blend of real and synthetic data to train the student network via a classifier-sharing-based feature alignment technique. Intensive experiments across multiple benchmarks demonstrate that our HiDFD can achieve state-of-the-art performance using 120 times less collected data than existing methods. Code is available at https://***/tangjialiang97/HiDFD. Copyright © 2024, The Authors. All rights reserved.

关键词： Students

Elucidating Spatial Complex Structures from Mass Spectrometry Imaging with Deep Multimodal Model

学校读者我要写书评

暂无评论

Elucidating Spatial Complex Structures from Mass Spectrometr...

Medical Artificial Intelligence (MedAI), IEEE International Conference on

作者： Jiangnan Cui Junjie Xia Xuan Li Yan Wang Fufang Qiu Yaochen Xu Keren Xu Chunman Zuo School of Computer Science and Technology Institute of Artificial Intelligence Donghua University Shanghai China Institute of Artificial Intelligence Donghua University Shanghai China College of Computer Science and Technology Jilin University Changchun China Department of Neurosurgery Huashan Hospital Shanghai China CAS Key Laboratory of Systems Biology Shanghai Institute of Biochemistry and Cell Biology Center for Excellence in Molecular Cell Science Chinese Academy of Sciences Shanghai China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun China

Spatial mass spectrometry imaging (SMSI) technology has enabled the characterization of biomolecule patterns within the tissue microenvironment, yet its analyses suffer from substantial noise. The lack of effective methods for exploiting multi-view features (e.g., spatial location) has hindered their elucidation ability for tissue heterogeneity. Here, we propose MSG, a multimodal modal that integrates histology, spatial location, and MSI data, in deciphering tissue structure by graph neural network. Specifically, MSG learns low-dimensional representations by aggregating histomorphological features from neighboring tiles via graph attention autoencoder. MSG outperforms other methods in identifying spatial domains and denoising data. The subsequent domain-guided differential expression analysis of the denoised data enables the identification of ions with significantly enriched patterns within the recognized domains. MSG exhibits its versatility in handling SMSI data from various platforms.

关键词：

EEG-Eye Movements Cross-Modal Decision Confidence Measurement with Generative Adversarial Networks

学校读者我要写书评

暂无评论

EEG-Eye Movements Cross-Modal Decision Confidence Measuremen...

International IEEE/EMBS Conference on Neural Engineering, CNE

作者： Cheng Fei Rui Li Li-Ming Zhao Wei-Long Zheng Bao-Liang Lu Department of Computer Science and Engineering Center for Brain-Like Computing and Machine Intelligence Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering and Brain Science and Technology Research Center Shanghai Jiao Tong University Shanghai People's Republic of China RuiJin-Mihoyo Laboratory Clinical Neuroscience Center RuiJin Hospital Shanghai Jiao Tong University School of Medicine Shanghai People's Republic of China

Decision confidence is an individual's feeling of correctness or optimization when making a decision. Various physiological signals, including electroencephalography (EEG) and eye movements have been studied extensively in measuring levels of decision confidence in humans. While multimodal fusion generally performs better than single-modal approaches, it requires data from different modalities at a greater cost. In particular, collection of EEG data is more complicated and time consuming while eye movement signals are much easier to acquire. To tackle this problem, we propose a cross-modal method based on generative adversarial learning. In our method, the intrinsic relationship between eye movement and EEG features in a high-level feature space can be learned in the training phase, and then we can obtain multimodal information during the test phase when only eye movements are available as inputs. Experimental results on the SEED-VPDC dataset demonstrate that our proposed method outperforms single-modal methods trained and tested only on eye movement signals with an improvement of approximately 5.43% in accuracy, and maintains competitive performance in comparison with multimodal methods. Our cross-modal approach requires only eye movements as inputs and reduces reliance on EEG data, making the decision confidence measurement more applicable and practicable.

关键词： Training Costs Neural engineering Particle measurements Generative adversarial networks Electroencephalography Adversarial machine learning

Adaptive UAV-Assisted Hierarchical Federated Learning: Optimizing Energy, Latency, and Resilience for Dynamic Smart IoT Networks

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yang, Xiaohong Liwang, Minghui Fu, Liqun Su, Yuhan Hosseinalipour, Seyyedali Wang, Xianbin Hong, Yiguang School of Informatics Xiamen University Fujian China Department of Control Science and Engineering The National Key Laboratory of Autonomous Intelligent Unmanned Systems China Frontiers Science Center for Intelligent Autonomous Systems Ministry of Education Tongji University Shanghai China School of Electronic Science and Engineering Xiamen University Xiamen China Department of Electrical Engineering University at Buffalo-SUNY BuffaloNY United States Department of Electrical and Computer Engineering Western University ON Canada

Hierarchical Federated Learning (HFL) introduces intermediate aggregation layers, addressing the limitations of conventional Federated Learning (FL) in geographically dispersed environments with limited communication infrastructure. An application of HFL is in smart IoT systems, such as remote monitoring, disaster response, and battlefield operations, where cellular connectivity is often unreliable or unavailable. In these scenarios, UAVs serve as mobile aggregators, providing connectivity to the terrestrial IoT devices. This paper studies an HFL architecture for energy-constrained UAVs in smart IoT systems, pioneering a solution to minimize global training cost increased caused by UAV disconnection. In light of this, we formulate a joint optimization problem involving learning configuration, bandwidth allocation, and device-to-UAV association, and perform global aggregation in time before UAV drops disconnect and redeployment of UAVs. The problem explicitly accounts for the dynamic nature of IoT devices and their interruptible communications and is unveiled to be NP-hard. To address this, we decompose it into three subproblems. First, we optimize the learning configuration and bandwidth allocation using an augmented Lagrangian function to reduce training costs. Second, we propose a device fitness score, integrating data heterogeneity (via Kullback-Leibler divergence), device-to-UAV distances, and IoT device resources, and develop a twin-delayed deep deterministic policy gradient (TD3)-based algorithm for dynamic device-to-UAV assignment. Third, We introduce a low-complexity two-stage greedy strategy for finding the location of UAVs redeployment and selecting the appropriate global aggregator UAV. Experiments on real-world datasets demonstrate significant cost reductions and robust performance under communication interruptions. © 2025, CC BY-NC-ND.

关键词： Federated learning

Tagging Continuous Labels for EEG-based Emotion Classification

学校读者我要写书评

暂无评论

Tagging Continuous Labels for EEG-based Emotion Classificati...

Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

作者： Rong-Fei Gu Li-Ming Zhao Wei-Long Zheng Bao-Liang Lu Department of Computer Science and Engineering Center for Brain-Like Computing and Machine Intelligence the Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering and Brain Science and Technology Research Center Shanghai Jiao Tong University Shanghai People’s Republic of China RuiJin-Mihoyo Laboratory Clinical Neuroscience Center RuiJin Hospital Shanghai Jiao Tong University School of Medicine Shanghai People’s Republic of China

EEG-based emotion classification has long been a critical task in the field of affective brain-computer interface (aBCI). The majority of leading researches construct supervised learning models based on labeled datasets. Several datasets have been released, including different kinds of emotions while utilizing various forms of stimulus materials. However, they adopt discrete labeling methods, in which the EEG data collected during the same stimulus material are given a same label. These methods neglect the fact that emotion changes continuously, and mislabeled data possibly exist. The imprecision of discrete labels may hinder the progress of emotion classification in concerned works. Therefore, we develop an efficient system in this paper to support continuous labeling by giving each sample a unique label, and construct a continuously labeled EEG emotion dataset. Using our dataset with continuous labels, we demonstrate the superiority of continuous labeling in emotion classification through experiments on several classification models. We further utilize the continuous labels to identify the EEG features under induced and non-induced emotions in both our dataset and a public dataset. Our experimental results reveal the learnability and generality of the relation between the EEG features and their continuous labels.

关键词：

Objective Depression Detection Using EEG and Eye Movement Signals Induced by Oil Paintings

学校读者我要写书评

暂无评论

Objective Depression Detection Using EEG and Eye Movement Si...

Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

作者： Luyu Liu Dan Peng Wei-Long Zheng Bao-Liang Lu Department of Computer Science and Engineering Center for Brain-Like Computing and Machine Intelligence the Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering and Brain Science and Technology Research Center Shanghai Jiao Tong University Shanghai People’s Republic of China RuiJin-Mihoyo Laboratory Clinical Neuroscience Center RuiJin Hospital Shanghai Jiao Tong University School of Medicine Shanghai People’s Republic of China

Depression is a mental disorder characterized by persistent sadness and loss of interest, which has become one of the leading causes of disability worldwide. There are currently no objective diagnostic standards for depression in clinical practice. Previous studies have shown that depression causes both brain abnormalities and behavioral disorders. In this study, both electroencephalography (EEG) and eye movement signals were used to objectively detect depression. By presenting 40 carefully selected oil paintings—20 positive and 20 negative—as stimuli, we were able to successfully evoke emotions in 48 depressed patients (DPs) and 40 healthy controls (HCs) from three centers. We then used Transformer, a deep learning model, to conduct emotion recognition and depression detection. The experimental results demonstrate that: a) Transformer achieves the best accuracies of 89.21% and 92.19% in emotion recognition and depression detection, respectively; b) The HC group has higher accuracies than the DP group in emotion recognition for both subject-dependent and subject-independent experiments; c) The neural pattern differences do exist between DPs and HCs, and we find the consistent asymmetry of the neural patterns in DPs; d) For depression detection, using single oil painting achieves the best accuracies, and using negative oil paintings has higher accuracies than using positive oil paintings. These findings suggest that EEG and eye movement signals induced by oil paintings can be used to objectively identify depression.

关键词：

A Multi-Player Potential Game Approach for Sensor Network Localization with Noisy Measurements

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xu, Gehui Chen, Guanpu Fidan, Baris Hong, Yiguang Qi, Hongsheng Parisini, Thomas Johansson, Karl H. The Key Laboratory of Systems and Control Academy of Mathematics and Systems Science Beijing China The Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm100 44 Sweden The Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm100 44 Sweden The Department of Mechanical and Mechatronics Engineering University of Waterloo WaterlooONN2L 3G1 Canada Department of Control Science and Engineering Tongji University Shanghai201804 China Shanghai Research Institute for Intelligent Autonomous Systems Shanghai201210 China Key Laboratory of Systems and Control Academy of Mathematics and Systems Science Beijing China School of Mathematical Sciences University of Chinese Academy of Sciences Beijing China The Department of Electrical and Electronic Engineering Imperial College London LondonSW7 2AZ United Kingdom The Department of Engineering and Architecture University of Trieste Trieste34127 Italy

Sensor network localization (SNL) is a challenging problem due to its inherent non-convexity and the effects of noise in inter-node ranging measurements and anchor node position. We formulate a non-convex SNL problem as a multi-player non-convex potential game and investigate the existence and uniqueness of a Nash equilibrium (NE) in both the ideal setting without measurement noise and the practical setting with measurement noise. We first show that the NE exists and is unique in the noiseless case, and corresponds to the precise network localization. Then, we study the SNL for the case with errors affecting the anchor node position and the inter-node distance measurements. Specifically, we establish that in case these errors are sufficiently small, the NE exists and is unique. It is shown that the NE is an approximate solution to the SNL problem, and that the position errors can be quantified accordingly. Based on these findings, we apply the results to case studies involving only inter-node distance measurement errors and only anchor position information inaccuracies. Copyright © 2024, The Authors. All rights reserved.

关键词： Sensor nodes

SPP-CNN: An Efficient Framework for Network Robustness Prediction

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Chengpei Lou, Yang Wang, Lin Li, Junli Li, Xiang Chen, Guanrong The College of Computer Science Sichuan Normal University Chengdu610066 China The Key Laboratory of System Control and Information Processing Ministry of Education Shanghai200240 China The Department of Computer Science National Yang Ming Chiao Tung University Hsinchu300 Taiwan The Department of Automation Shanghai Jiao Tong University Shanghai200240 China The Institute of Complex Networks and Intelligent Systems Shanghai Research Institute for Intelligent Autonomous Systems Tongji University Shanghai201210 China The Department of Control Science and Engineering Tongji University Shanghai200240 China The Department of Electrical Engineering City University of Hong Kong Hong Kong

This paper addresses the robustness of a network to sustain its connectivity and controllability against malicious attacks. This kind of network robustness is typically measured by the time-consuming attack simulation, which returns a sequence of values that record the remaining connectivity and controllability after a sequence of node- or edge-removal attacks. For improvement, this paper develops an efficient framework for network robustness prediction, the spatial pyramid pooling convolutional neural network (SPP-CNN). The new framework installs a spatial pyramid pooling layer between the convolutional and fully-connected layers, overcoming the common mismatch issue in the CNN-based prediction approaches and extending its generalizability. Extensive experiments are carried out by comparing SPP-CNN with three state-of-the-art robustness predictors, namely a CNN-based and two graph neural networks-based frameworks. Synthetic and real-world networks, both directed and undirected, are investigated. Experimental results demonstrate that the proposed SPP-CNN achieves better prediction performances and better generalizability to unknown datasets, with significantly lower time-consumption, than its counterparts. © 2023, CC BY.

关键词： Complex networks

Distributed Semi-Supervised Sparse Statistical Inference

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Tu, Jiyuan Liu, Weidong Mao, Xiaojun Xu, Mingyue School of Mathematics Shanghai University of Finance and Economics Shanghai200433 China School of Mathematical Sciences MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University Shanghai200240 China School of Mathematical Sciences Ministry of Education Key Laboratory of Scientific and Engineering Computing Shanghai Jiao Tong University Shanghai200240 China Department of Computer Science Purdue University West LafayetteIN47906 United States

The debiased estimator is a crucial tool in statistical inference for high-dimensional model parameters. However, constructing such an estimator involves estimating the high-dimensional inverse Hessian matrix, incurring significant computational costs. This challenge becomes particularly acute in distributed setups, where traditional methods necessitate computing a debiased estimator on every machine. This becomes unwieldy, especially with a large number of machines. In this paper, we delve into semi-supervised sparse statistical inference in a distributed setup. An efficient multi-round distributed debiased estimator, which integrates both labeled and unlabelled data, is developed. We will show that the additional unlabeled data helps to improve the statistical rate of each round of iteration. Our approach offers tailored debiasing methods for M-estimation and generalized linear models according to the specific form of the loss function. Our method also applies to a non-smooth loss like absolute deviation loss. Furthermore, our algorithm is computationally efficient since it requires only one estimation of a high-dimensional inverse covariance matrix. We demonstrate the effectiveness of our method by presenting simulation studies and real data applications that highlight the benefits of incorporating unlabeled data. © 2023, CC BY.

关键词： Covariance matrix