检索结果-内蒙古大学图书馆

Aware: Adaptive Distributed Training with Computation, Communication and Position Awareness for Deep Learning Model

学校读者我要写书评

暂无评论

Aware: Adaptive Distributed Training with Computation, Commu...

IEEE International Conference on High Performance computing and Communications (HPCC)

作者： Yan Zeng Guangzheng Yi Yuyu Yin Jiyang Wu Meiting Xue Jilin Zhang Jian Wan Yunquan Zhang School of Computer Science and Technology Hangzhou Dianzi University Hangzhou China School of Cyberspace Hangzhou Dianzi University Hangzhou China State Key Laboratory of Computer Architecture Institute of Computing Technology Chinese Academy of Sciences Beijing China

The accuracy of the neural networks can usually be improved by increasing the size of the dataset and the layers or operators of the network, as it has strong composability. But, it makes a challenge to train these models efficiently, due to the limited resources of a single device, such as memory, computational resource and so on. For this challenge, it has become an inevitable trend to automatically implement model parallelism across multiple devices. This paper proposes Aware, an adaptive distributed parallel training method, to search distributed parallel strategy automatically for deep learning model. It firstly proposes an operator fusion strategy with computation and communication awareness to simplify the computational graph of deep learning model. And then, it introduces position-aware graph embedding algorithm to extract the structural features of models, which can make the searched parallel strategies to transplant to other networks with similar structures. On this basis, we use reinforcement learning algorithm to search distributed parallel strategy automatically. This paper makes experiments with neural networks such as Inception V3, NMT, GNM, NasNet and ResNet, and uses CIFAR10 and PTB datasets to compare the two methods of Hierarchical and Placeto. The results show that compared with Placeto, Aware achieves up to 5% reductions in runtime. Aware achieves more than 8% reductions in runtime compared with Hierarchical. Moreover, it has better transplantation and generalization capabilities, and supports pre-training for large-scale network parallel strategy search and accelerates convergence.

关键词： Deep learning Training Adaptation models Runtime Computational modeling Neural networks Reinforcement learning

Knowledge Graph Enhanced Large Language Model Editing

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Mengqi Ye, Xiaotian Liu, Qiang Ren, Pengjie Wu, Shu Chen, Zhumin School of Computer Science and Technology Shandong University China School of Computer Science Beijing University of Posts and Telecommunications China Center for Research on Intelligent Perception and Computing State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences China

Large language models (LLMs) are pivotal in advancing natural language processing (NLP) tasks, yet their efficacy is hampered by inaccuracies and outdated knowledge. Model editing emerges as a promising solution to address these challenges. However, existing editing methods struggle to track and incorporate changes in knowledge associated with edits, which limits the generalization ability of post-edit LLMs in processing edited knowledge. To tackle these problems, we propose a novel model editing method that leverages knowledge graphs for enhancing LLM editing, namely GLAME. Specifically, we first utilize a knowledge graph augmentation module to uncover associated knowledge that has changed due to editing, obtaining its internal representations within LLMs. This approach allows knowledge alterations within LLMs to be reflected through an external graph structure. Subsequently, we design a graph-based knowledge edit module to integrate structured knowledge into the model editing. This ensures that the updated parameters reflect not only the modifications of the edited knowledge but also the changes in other associated knowledge resulting from the editing process. Comprehensive experiments conducted on GPT-J and GPT-2 XL demonstrate that GLAME significantly improves the generalization capabilities of post-edit LLMs in employing edited knowledge. Copyright © 2024, The Authors. All rights reserved.

关键词： Knowledge graph

SkinFormer: Learning Statistical Texture Representation with Transformer for Skin Lesion Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xu, Rongtao Wang, Changwei Zhang, Jiguang Xu, Shibiao Meng, Weiliang Zhang, Xiaopeng The State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences Beijing China The Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center Qilu University of Technology Jinan China Shandong Provincial Key Laboratory of Computer Networks Shandong Fundamental Research Center for Computer Science Jinan China CASIA Beijing China The State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences China School of Artificial Intelligence Beijing University of Posts and Telecommunications China

Accurate skin lesion segmentation from dermoscopic images is of great importance for skin cancer diagnosis. However, automatic segmentation of melanoma remains a challenging task because it is difficult to incorporate useful texture representations into the learning process. Texture representations are not only related to the local structural information learned by CNN, but also include the global statistical texture information of the input image. In this paper, we propose a transFormer network (SkinFormer) that efficiently extracts and fuses statistical texture representation for Skin lesion segmentation. Specifically, to quantify the statistical texture of input features, a Kurtosis-guided Statistical Counting Operator is designed. We propose Statistical Texture Fusion Transformer and Statistical Texture Enhance Transformer with the help of Kurtosis-guided Statistical Counting Operator by utilizing the transformer’s global attention mechanism. The former fuses structural texture information and statistical texture information, and the latter enhances the statistical texture of multi-scale features. Extensive experiments on three publicly available skin lesion datasets validate that our SkinFormer outperforms other SOAT methods, and our method achieves 93.2% Dice score on ISIC 2018. It can be easy to extend SkinFormer to segment 3D images in the future. Our code is available at https://***/RongtaoXu/SkinFormer. Copyright © 2024, The Authors. All rights reserved.

关键词： Image segmentation

On the Exploitation of Control Knowledge for Enhancing Automated Planning

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Lu, Xu Yu, Bin Chen, Chu Tian, Cong Duan, Zhenhua Institute of Computing Theory and Technology State Key Laboratory of Integrated Services Networks Xidian University Xi’an710071 China School of Computer Science Qufu Normal University Qufu273165 China

Heuristic search provides an efficient way to automatically explore the state space in planning, while Control Knowledge (CK) also has the potential to significantly increase the performance of planners. Currently, most of the state-of-the-art planners primarily rely on sophisticated heuristic mechanisms. However, these planners fail to scale up and to provide (high-quality) solutions in a range of problems. Appropriate CK may be capable of making hard problems easy to solve, thus leading planners to achieve much higher efficiency. In this paper, we propose a planning approach by incorporating CK with heuristic search in order to leverage the advantages of both. To achieve this, we encode CK specified by a novel variant of Linear Temporal Logic (LTL), referred to as LTL$_{\mathcal{P}}$, into standard planning models. Consequently, we can directly use existing heuristic planners to solve the augmented problem, and avoid tailoring the planners in order to deal with CK implicitly. Experiments demonstrate that our approach shows highly competitive results versus heuristic search and other techniques on many intractable benchmark problems, benefiting in improving the coverage and quality of plans. © 2023, The Authors. All rights reserved.

关键词： Temporal logic

Verifying Safety of Neural Networks from Topological Perspectives

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liang, Zhen Ren, Dejin Xue, Bai Wang, Ji Yang, Wenjing Liu, Wanwei National University of Defense Technology Institute for Quantum Information State Key Laboratory of High Performance Computing Hunan Changsha410073 China Chinese Academy of Sciences State Key Lab. of Computer Science BeijingBAI XUE China

Neural networks (NNs) are increasingly applied in safety-critical systems such as autonomous vehicles. However, they are fragile and are often ill-behaved. Consequently, their behaviors should undergo rigorous guarantees before deployment in practice. In this paper, we propose a set-boundary reachability method to investigate the safety verification problem of NNs from a topological perspective. Given an NN with an input set and a safe set, the safety verification problem is to determine whether all outputs of the NN resulting from the input set fall within the safe set. In our method, the homeomorphism property and the open map property of NNs are mainly exploited, which establish rigorous guarantees between the boundaries of the input set and the boundaries of the output set. The exploitation of these two properties facilitates reachability computations via extracting subsets of the input set rather than the entire input set, thus controlling the wrapping effect in reachability analysis and facilitating the reduction of computation burdens for safety verification. The homeomorphism property exists in some widely used NNs such as invertible residual networks (i-ResNets) and Neural ordinary differential equations (Neural ODEs), and the open map is a less strict property and easier to satisfy compared with the homeomorphism property. For NNs establishing either of these properties, our set-boundary reachability method only needs to perform reachability analysis on the boundary of the input set. Moreover, for NNs that do not feature these properties with respect to the input set, we explore subsets of the input set for establishing the local homeomorphism property and then abandon these subsets for reachability computations. Finally, some examples demonstrate the performance of the proposed *** Codes 68Q60, 68T07 Copyright © 2023, The Authors. All rights reserved.

关键词： Topology

Entirety of Quantum Uncertainty and Its Experimental Verification

学校读者我要写书评

暂无评论

Chinese Physics Letters 2021年第7期38卷 11-16页

作者： Jie Xie Li Zhou Aonan Zhang Huichao Xu Man-Hong Yung Ping Xu Nengkun Yu Lijian Zhang National Laboratory of Solid State Microstructures College of Engineering and Applied Sciences and School of PhysicsNanjing UniversityNanjing 210093China Collaborative Innovation Center of Advanced Microstructures Nanjing UniversityNanjing 210093China Department of Computer Science and Technology Tsinghua UniversityBeijing 100084China Shenzhen Institute for Quantum Science and Engineering and Department of Physics Southern University of Science and TechnologyShenzhen 518055China Shenzhen Key Laboratory of Quantum Science and Engineering Southern University of Science and TechnologyShenzhen 518055China Institute for Quantum Information&State Key Laboratory of High Performance Computing College of ComputerNational University of Defense TechnologyChangsha 410073China Centre for Quantum Software and Information School of SoftwareFaculty of Engineering and Information TechnologyUniversity of Technology SydneyNSWAustralia

As a foundation of quantum physics,uncertainty relations describe ultimate limit for the measurement uncertainty of incompatible ***,uncertainty relations are formulated by mathematical bounds for a specific *** we present a method for geometrically characterizing uncertainty relations as an entire area of variances of the observables,ranging over all possible input *** find that for the pair of position and momentum operators,Heisenberg's uncertainty principle points exactly to the attainable area of the variances of position and ***,for finite-dimensional systems,we prove that the corresponding area is necessarily semialgebraic;in other words,this set can be represented via finite polynomial equations and inequalities,or any finite union of such *** particular,we give the analytical characterization of the areas of variances of(a)a pair of one-qubit observables and(b)a pair of projective observables for arbitrary dimension,and give the first experimental observation of such areas in a photonic system.

关键词： observable union operators

Estimating Multiple Socioeconomic Attributes via Home Location-A Case Study in China

学校读者我要写书评

暂无评论

Journal of Social computing 2021年第1期2卷 71-88页

作者： Shichang Ding Xin Gao Yufan Dong Yiwei Tong Xiaoming Fu State Key Laboratory of Mathematical Engineering and Advanced Computing Zhengzhou 276800China Department of Sociology Tsinghua UniversityBeijing 100085China Institute of Computer Science University of GottingenGottingen 37077Germany Shanghai Hejin Information Technology Company Shanghai 200100China

Inferring people’s Socioeconomic Attributes(SEAs),including income,occupation,and education level,is an important problem for both social sciences and many networked applications like targeted advertising and personalized *** works mainly focus on estimating SEAs from peoples’cyberspace behaviors and relationships,such as the content of tweets or the social networks between online *** cyberspace data,alternative data sources about users’physical behavior,like their home location,may offer new *** specifically,in this paper,we study how to predict a person’s income level,family income level,occupation type,and education level from his/her home *** a case study,we collect people’s home locations and socioeconomic attributes through a survey involving 9 provinces and 85 cities in *** further enrich home location with the knowledge from real estate websites,government statistics websites,online map services,*** learn a shared representation from input features as well as attribute-specific representations for different SEAs,we propose H2SEA,a factorization machine-based multi-task learning method with attention *** experiment results show that:(1)Home location can clearly improve the estimation accuracy for all SEA prediction tasks(e.g.,80.2%improvement in terms of F1-score in estimating personal income level);(2)The proposed H2SEA model outperforms alternative models for SEA inference in terms of various evaluation metrics,such as Area Under Curve(AUC),F-measure,and specificity;(3)The performance of specific SEA prediction tasks(e.g.,personal income)can be further improved if H2SEA only focuses on cities or villages due to urban-rural gap in China;(4)Compared with online crawled housing price data,the area-level average income and Points Of Interest(POI)are more important features for SEA inferences in China.

关键词： personal income family income occupation education multi-task learning

Multi-Grained Feature Pruning for Video-Based Human Pose Estimation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Zhigang Fan, Shaojing Liu, Zhenguang Wu, Zheqi Wu, Sifan Jiao, Yingying College of Computer Science and Technology Zhejiang Gongshang University Hangzhou China School of Computing National University of Singapore Singapore The State Key Laboratory of Blockchain and Data Security Zhejiang University Hangzhou China Institute of Blockchain and Data Security Hangzhou China College of Computer Science and Technology Jilin University Changchun China College of Computer Science and Technology Zhejiang University of Technology Hangzhou China

Human pose estimation, with its broad applications in action recognition and motion capture, has experienced significant advancements. However, current Transformer-based methods for video pose estimation often face challenges in managing redundant temporal information and achieving fine-grained perception because they only focus on processing low-resolution features. To address these challenges, we propose a novel multi-scale resolution framework that encodes spatio-temporal representations at varying granularities and executes fine-grained perception compensation. Furthermore, we employ a density peaks clustering method to dynamically identify and prioritize tokens that offer important semantic information. This strategy effectively prunes redundant feature tokens, especially those arising from multi-frame features, thereby optimizing computational efficiency without sacrificing semantic richness. Empirically, it sets new benchmarks for both performance and efficiency on three large-scale datasets. Our method achieves a 93.8% improvement in inference speed compared to the baseline, while also enhancing pose estimation accuracy, reaching 87.4 mAP on the PoseTrack2017 dataset. © 2025, CC BY.

关键词： Spatio-temporal data

A Strong Privacy-Preserving and Efficient Fingerprint Authentication via Clustering

学校读者我要写书评

暂无评论

A Strong Privacy-Preserving and Efficient Fingerprint Authen...

IEEE Conference on Global Communications (GLOBECOM)

作者： Jingwei Liu Zihan Zhou Rong Sun Xiaojiang Du Mohsen Guizani Shaanxi Key Laboratory of Blockchain and Secure Computing Xidian University Xi'an China State Key Laboratory of ISN Xidian University Xi'an China Department of Electrical and Computer Engineering Stevens Institute of Technology Hoboken USA Department of Computer Science and Engineering Qatar University Doha Qatar

With the advancement of cloud technology, the storage and computing overhead in large-scale biometric authentication is mitigated by outsourcing data to the cloud. Since biometric features serve as a unique identifier bound to each individual, transmitting them directly to the cloud may bring about serious privacy disclosure risks. To guarantee users' biometric features, there are many solutions have been proposed. However, most of them neglect to protect identity security. In light of the challenges, this paper proposes a strong privacy-preserving fingerprint authentication via clustering. Besides safeguarding fingerprint features, the scheme also blurs the identities of users to enable anonymity of identity. Meanwhile, the authentication efficiency of the proposed scheme is improved by vector processing of fingerprints and fast retrieval of clustered identities. Furthermore, a dual-server matching architecture effectively reduces the communication overhead of the service provider. The security analysis and experimental results indicate that the proposed scheme provides strong privacy preservation while maintaining high efficiency.

关键词：