检索结果-内蒙古大学图书馆

Cross Architecture Distillation for Face Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhao, Weisong Zhu, Xiangyu He, Zhixiang Zhang, Xiao-Yu Lei, Zhen Institute of Information Engineering Chinese Academy of Sciences Beijing China CBSR&NLPR Institute of Automation Chinese Academy of Sciences Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China China Telecom Corporation Ltd. Data & AI Technology Company China Centre for Artificial Intelligence and Robotics Hong Kong Institute of Science & Innovation Chinese Academy of Sciences Hong Kong

Transformers have emerged as the superior choice for face recognition tasks, but their insufficient platform acceleration hinders their application on mobile devices. In contrast, Convolutional Neural Networks (CNNs) capitalize on hardware-compatible acceleration libraries. Consequently, it has become indispensable to preserve the distillation efficacy when transferring knowledge from a Transformer-based teacher model to a CNN-based student model, known as Cross-Architecture Knowledge Distillation (CAKD). Despite its potential, the deployment of CAKD in face recognition encounters two challenges: 1) the teacher and student share disparate spatial information for each pixel, obstructing the alignment of feature space, and 2) the teacher network is not trained in the role of a teacher, lacking proficiency in handling distillation-specific knowledge. To surmount these two constraints, 1) we first introduce a Unified Receptive Fields Mapping module (URFM) that maps pixel features of the teacher and student into local features with unified receptive fields, thereby synchronizing the pixel-wise spatial information of teacher and student. Subsequently, 2) we develop an Adaptable Prompting Teacher network (APT) that integrates prompts into the teacher, enabling it to manage distillation-specific knowledge while preserving the model’s discriminative capacity. Extensive experiments on popular face benchmarks and two large-scale verification sets demonstrate the superiority of our method. Copyright © 2023, The Authors. All rights reserved.

关键词： Distillation

Privacy Preserving Discretized Spiking Neural Network with TFHE

学校读者我要写书评

暂无评论

Privacy Preserving Discretized Spiking Neural Network with T...

Neuromorphic Computing (ICNC), International Conference on

作者： Pengbo Li Ting Gao Huifang Huang Jiani Cheng Shuhong Gao Zhigang Zeng Jinqiao Duan Center for Mathematical Sciences Huazhong University of Science and Technology Wuhan China School of Mathematics and Statistics Huazhong University of Science and Technology Wuhan China School of Mathematical and Statistical Sciences Clemson University Clemson U.S.A School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan China Depatment of Mathematics and Department of Physics The Dongguan Key Laboratory for Data Science and Intelligent Medicine Great Bay University Dongguan China

artificial intelligence not only offers significant convenience but also raises privacy and security concerns, which pose challenges for Spiking Neural Networks (SNNs). Fully Homomorphic Encryption (FHE) is a crucial privacy-preserving computation technology, and the discrete nature of SNNs naturally aligns with FHE’s requirements. In this paper, we construct a framework to evaluate the homomorphic computation of SNN named FHE-DiSNN that enables SNN to achieve good prediction performance on encrypted data. First, benefitting from the discrete nature of spike signals, our proposed model avoids the errors introduced by discretizing activation functions. Second, by applying bootstrapping, we design new privacy-preserving functions FHE-Fire and FHE-Reset, through which noise can be refreshed, allowing us to evaluate SNN for an arbitrary number of operations. Furthermore, we improve the computational efficiency of FHE-DiSNN while maintaining a high level of accuracy. Finally, we evaluate our model on the MNIST dataset. The experiments show that FHE-DiSNN with 30 neurons in the hidden layer achieves a minimum prediction accuracy of 94.4%. Under optimal parameters, it achieves a 95.1% accuracy, with only a 0.6% decrease compared to the original SNN (95.7%). These results demonstrate the superiority of SNN over second-generation neural networks for homomorphic evaluation.

关键词：

Exploring the common principal subspace of deep features in neural networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Liu, Haoran Xiong, Haoyi Wang, Yaqing An, Haozhe Wu, Dongrui Dou, Dejing Big Data Lab Baidu Research Department of Computer Science University of Maryland College ParkMD United States School of Artificial Intelligence and Automation Huazhong University of Science and Technology

We find that different Deep Neural Networks (DNNs) trained with the same dataset share a common principal subspace in latent spaces, no matter in which architectures (e.g., Convolutional Neural Networks (CNNs), Multi-Layer Preceptors (MLPs) and Autoencoders (AEs)) the DNNs were built or even whether labels have been used in training (e.g., supervised, unsupervised, and self-supervised learning). Specifically, we design a new metric P-vector to represent the principal subspace of deep features learned in a DNN, and propose to measure angles between the principal subspaces using P- vectors. Small angles (with cosine close to 1.0) have been found in the comparisons between any two DNNs trained with different algorithms/architectures. Furthermore, during the training procedure from random scratch, the angle decrease from a larger one (70°-80° usually) to the small one, which coincides the progress of feature space learning from scratch to convergence. Then, we carry out case studies to measure the angle between the P-vector and the principal subspace of training dataset, and connect such angle with generalization performance. Extensive experiments with practically-used Multi-Layer Perceptron (MLPs), AEs and CNNs for classification, image reconstruction, and self-supervised learning tasks on MNIST, CIFAR-10 and CIFAR-100 datasets have been done to support our claims with solid evidences. © 2021, CC BY.

关键词： Deep neural networks

Spatiotemporal Dynamics in Networked GM System Under Fractional Pd Control

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Li, Hua Xiao, Min Wang, Zhengxin Xu, Fengyu Wang, Zhen Zheng, Weixing Rutkowski, Lezek College of Automation College of Artificial Intelligence Nanjing University of Posts and Telecommunications Nanjing210023 China College of Science Nanjing University of Posts and Telecommunications Nanjing210023 China College of Mathematics and Systems Science Shandong University of Science and Technology Qingdao266590 China College of Computer Data and Mathematical Sciences Western Sydney University SydneyNSW2751 Australia Systems Research Institute the Polish Academy of Sciences Warsaw01-447 Poland AGH University of Krakow Kraków30-059 Poland

This paper constructs a scale-free chemical network based on the Gierer-Meinhardt (GM) system and investigates its Turing instability. We establish a fractional-order single-node GM system with delay and design a fractional-order proportional derivative (PD) control strategy for the issue of bifurcation control. Using delay as bifurcation parameter, the existence of Hopf bifurcation is proven, and control over bifurcation threshold points is achieved through a fractional-order PD control strategy. For the scale-free chemical network based on the GM system, we obtain the condition of how the Turing instability occurs. We derive how the number of edges for the new nodes changes the stability of the network-organized system and investigate the relationship between degrees of nodes and eigenvalues of the network matrix. We give the instability condition caused by diffusion in the network-organized system. Finally, the numerical simulations verify analytical results. © 2024, The Authors. All rights reserved.

关键词： Hopf bifurcation

Investigating AI Explanations in Medical Diagnosis 6th

学校读者我要写书评

暂无评论

Investigating AI Explanations in Medical Diagnosis

6th International Conference on artificial intelligence and Speech Technology, AIST 2024

作者： Garg, Nikita Joshi, Manya Handa, Palak Goel, Nidhi Department of Electronics and Communication Engineering Indira Gandhi Delhi Technical University for Women New Delhi India Department of Artificial Intelligence and Data Science University School of Automation and Robotics GGSIPU EDC New Delhi India Research Centre for Medical Image Analysis and Artificial Intelligence Department of Medicine Danube Private University Krems an der Donau Austria

ISBN: (纸本)9783031913396

The lack of comprehensive explanations accompanying the results generated through automated diagnosis systems has encountered widespread mistrust among practitioners, patients, and stakeholders. This study aims to elucidate different Explainable artificial intelligence (XAI) strategies that can aid patients and practitioners in comprehending the results better. A set of biomedical images with different modalities served as the dataset for testing, predicting, and interpreting the predicted results. A pre-trained AI model was employed to generate predictions, which were subsequently explained using XAI techniques. Conventional criteria including accuracy, precision, recall, and F1 score were used to assess biological image analysis. Additionally, with the aid of visualization tools like feature maps and heatmaps produced by XAI techniques such as gradient-weighted class activation mapping, local interpretable model agnostic explanations, and shapley additive explanations, interpretability was observed. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Cardiography

Penetration trajectory optimization for the hypersonic gliding vehicle encountering two interceptors

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Shen, Zhipeng Yu, Jianglong Dong, Xiwang Hua, Yongzhao Ren, Zhang School of Automation Science and Electrical Engineering Science and Technology on Aircraft Control Laboratory Beihang University Beijing100191 China Institute of Artificial Intelligence Beihang University Beijing100191 China Beijing Advanced Innovation Center for Big Data and Brain Computing Beihang University Beijing100191 China

The penetration trajectory optimization problem for the hypersonic gliding vehicle (HGV) encountering two interceptors is investigated. The HGV penetration trajectory optimization problem considering the terminal target area is formulated as a nonconvex optimal control problem. The nonconvex optimal control problem is transformed into a second-order cone programming (SOCP) problem, which can be solved by state-of-the-art interior-point methods. In addition, a penetration strategy that only requires the initial line-of-sight angle information of the interceptors is proposed. The convergent trajectory obtained by the proposed method allows the HGV to evade two interceptors and reach the target area successfully. Furthermore, a successive SOCP method with a variable trust region is presented, which is critical to balancing the trade-off between time consumption and optimality. Finally, the effectiveness and performance of the proposed method are verified by numerical simulations. Copyright © 2022, The Authors. All rights reserved.

关键词： Numerical methods

Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Changwei Chen, Shunpeng Song, Yukun Xu, Rongtao Zhang, Zherui Zhang, Jiguang Yang, Haoran Zhang, Yu Fu, Kexue Du, Shide Xu, Zhiwei Gao, Longxiang Guo, Li Xu, Shibiao Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center Qilu University of Technology Shandong Academy of Sciences China School of Artificial Intelligence Beijing University of Posts and Telecommunications China Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing Shandong Fundamental Research Center for Computer Science China MAIS Institute of Automation Chinese Academy of Sciences China Tongji University China College of Computer and Data Science Fuzhou University China Shandong University China

Visual Place Recognition (VPR) is aimed at predicting the location of a query image by referencing a database of geotagged images. For VPR task, often fewer discriminative local regions in an image produce important effects while mundane background regions do not contribute or even cause perceptual aliasing because of easy overlap. However, existing methods lack precisely modeling and full exploitation of these discriminative regions. In this paper, we propose the Focus on Local (FoL) approach to stimulate the performance of image retrieval and re-ranking in VPR simultaneously by mining and exploiting reliable discriminative local regions in images and introducing pseudo-correlation supervision. First, we design two losses, Extraction-Aggregation Spatial Alignment Loss (SAL) and Foreground-Background Contrast Enhancement Loss (CEL), to explicitly model reliable discriminative local regions and use them to guide the generation of global representations and efficient re-ranking. Second, we introduce a weakly-supervised local feature training strategy based on pseudo-correspondences obtained from aggregating global features to alleviate the lack of local correspondences ground truth for the VPR task. Third, we suggest an efficient re-ranking pipeline that is efficiently and precisely based on discriminative region guidance. Finally, experimental results show that our FoL achieves the state-of-the-art on multiple VPR benchmarks in both image retrieval and re-ranking stages and also significantly outperforms existing two-stage VPR methods in terms of computational efficiency. Copyright © 2025, The Authors. All rights reserved.

关键词： Photointerpretation

Redundancy Reduction of Musculoskeletal Model for Robots with Group Sparse Neural Network

学校读者我要写书评

暂无评论

Redundancy Reduction of Musculoskeletal Model for Robots wit...

IEEE International Conference on Control and Robotics Engineering (ICCRE)

作者： Shanlin Zhong Jinhan Zhang Xiangli Nie Chinese Academy of Sciences Institute of Automation School of Artificial Intelligence University of Chinese Academy of Science Beijing China CETC Key Laboratory of Data Link Technology Xi’an China

ISBN: (纸本)9781665412391;9781665429542

Creating robots with similar behavior and appearance to humans has been a long-term dream for roboticists. However, the highly redundant characteristic of the muscle system brings tremendous difficulty for the control and manufacture of the musculoskeletal robot. In this paper, we propose a heuristic pruning-based recurrent neural network method with group sparse regularization to reduce redundancy of muscles in musculoskeletal model, which can maintain the motion accuracy of musculoskeletal robots without activating the redundant muscles. Experiments on an upper extremity musculoskeletal model validate the effectiveness of the proposed method.

关键词： Recurrent neural networks Redundancy Neurons Muscles Robots Extremities Biological neural networks

Initial Result of Design-to-Test Method for Developing Architecture of the Large-Scale Complex Industrial Software

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Chen, Ge Zhang, Junbo Cui, Wei Ma, Mengcheng School of Electric Power South China University of Technology Guangzhou510641 China School of Automation Science and Engineering South China University of Technology Guangzhou510641 China School of Big Data and Artificial Intelligence Guangdong Polytechnic of Water Resources and Electric Engineering Guangzhou510635 China

In the process of developing large-scale complex industrial software, defects may occur at any point. Cost of correction will increase exponentially if the defects are not detected until later stages, especially the architectural defects. To address this problem, an architecture design-to-test solution is proposed in this paper to find the architectural defects in the design stage, so it can effectively reduce the potential rework cost. First, we introduce a mathematical model for service-based software architecture testing, and determine that the testing content is to check the correctness of business processes the architecture can achieve. The paper then applies a pruning technique to cut down on the quantity of business processes, and proves its effectiveness of reducing test cost. Finally, an implementation method is proposed for architecture testing, including the process encoding, the process decoding, and a search algorithm considering some constraints, to search out all the business processes the architecture can achieve. We test the proposed method in an architecture design of typical industrial software, and the results show the method can indeed help us find architectural defects and reduce the potential rework cost. © 2022, The Authors. All rights reserved.

关键词： Defects