检索结果-内蒙古大学图书馆

Reveal training performance mystery between Tensor Flow and PyTorch in the single GPU environment

Science China(Information Sciences) 2022年第1期65卷 147-163页

作者： Hulin DAI Xuan PENG Xuanhua SHI Ligang HE Qian XIONG Hai JIN National Engineering Research Center for Big Data Technology and System Service Computing Technology and System LabSchool of Computer Science and Technology Huazhong University of Science and Technology Department of Computer Science University of Warwick

Deep learning has gained tremendous success in various fields while training deep neural networks(DNNs) is very compute-intensive, which results in numerous deep learning frameworks that aim to offer better usability and higher performance to deep learning practitioners. Tensor Flow and Py Torch are the two most popular frameworks. Tensor Flow is more promising within the industry context, while Py Torch is more appealing in academia. However, these two frameworks differ much owing to the opposite design philosophy:static vs dynamic computation graph. Tensor Flow is regarded as being more performance-friendly as it has more opportunities to perform optimizations with the full view of the computation graph. However, there are also claims that Py Torch is faster than Tensor Flow sometimes, which confuses the end-users on the choice between them. In this paper, we carry out the analytical and experimental analysis to unravel the mystery of comparison in training speed on single-GPU between Tensor Flow and Py Torch. To ensure that our investigation is as comprehensive as possible, we carefully select seven popular neural networks, which cover computer vision, speech recognition, and natural language processing(NLP). The contributions of this work are two-fold. First, we conduct the detailed benchmarking experiments on Tensor Flow and Py Torch and analyze the reasons for their performance difference. This work provides the guidance for the end-users to choose between these two frameworks. Second, we identify some key factors that affect the performance,which can direct the end-users to write their models more efficiently.

关键词： deep learning performance comparison TensorFlow PyTorch

来源：评论

学校读者我要写书评

暂无评论

A Graph Sequence Generator and Multi-head Self-attention Mechanism based Knowledge Graph Reasoning Architecture 26

A Graph Sequence Generator and Multi-head Self-attention Mec...

引用

26th International Conference on Computer Supported Cooperative Work in Design, CSCWD 2023

作者： Wu, Yuejia Zhou, Jian-Tao College of Computer Science Inner Mongolia University Hohhot China National & Local Joint Engineering Research Center Intelligent Info. Proc. Technology for Mongolian Engineering Research Center of Ecological Big Data Ministry of Education Hohhot China Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software Inner Mongolia Key Laboratory of Social Computing and Data Processing Hohhot China Inner Mongolia Key Laboratory of Discipline Inspection and Supervision Big Data Inner Mongolia Engineering Laboratory for Big Data Analysis Technology Hohhot China

ISBN: (纸本)9798350331684

Knowledge Graph (KG) is an essential research direction that involves storing and managing knowledge data, but its incompleteness and sparsity hinder its development in various applications. Knowledge Graph Reasoning (KGR) is an effective method to solve this limitation via reasoning missing knowledge based on existing knowledge. The graph Convolution Network (GCN) based method is one of the state-of-the-art approaches to this work. However, there are still some problems, such as the insufficient ability to perceive graph structure and the poor effect of learning data features which may limit the reasoning accuracy. This paper proposes a KGR architecture based on a graph sequence generator and multi-head self-attention mechanism, named GaM-KGR, to improve the above problems and enhance prediction accuracy. Specifically, the GaM-KGR first introduces the graph generation model into the field of KGR for graph representation learning to obtain the hidden features in the data so that enhancing the reasoning effect and then embeds the generated graph sequence into the multi-head self-attention mechanism for subsequent processing to improve the graph structure perception ability of the proposed architecture, so that it can process the graph structure data more appropriately. Extensive experimental results show that the GaM-KGR architecture can achieve the state-of-the-art prediction results of current GCN-based models. © 2023 IEEE.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

Large-scale Image Stitching Algorithm Integrating UAV Position Information and MGFT Model

Large-scale Image Stitching Algorithm Integrating UAV Positi...

引用

International Conference on Advanced Robotics and Mechatronics (ICARM)

作者： Zun Liu Junxin Liu Kaiwen Wang Jinxiao Tan Jianqiang Li National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen PR China

ISBN: (数字)9798350385724

ISBN: (纸本)9798350385731

We propose a large-scale image stitching algorithm integrating UAV position information and MGFT(Max Grad Feature with Transformer) model to deal with the issues of slow speed, low accuracy, and projection distortion in large-scale image stitching. First, the algorithm calculates the image splicing sequence through the special barrel structure of the containment vessel and the UAV’s position information, reducing the search space for large-scale image splicing and reducing ineffective image matching, thus greatly improving the efficiency of image splicing. Secondly, the algorithm implements a method of estimating the transformation matrix parameters of three types of images through UAV position information, and proposes a method for direct splicing using UAV position information. Finally, to estimate the translation matrix parameters more accurately, we propose the MGFT model. By stacking multiple layers of Transformer’s self-attention and cross-attention, the extracted maximum pixel gradient features can integrate the global information of two images. Experimental results show that the algorithm proposed in this article improves the efficiency of large-scale image stitching, improves the accuracy and quality of image stitching, and reduces cumulative errors and projection distortion.

关键词： Training Accuracy Splicing Stacking Inspection Autonomous aerial vehicles Feature extraction Distortion Transformers Image stitching

来源：评论

学校读者我要写书评

暂无评论

A nearly optimal distributed algorithm for computing the weighted girth

引用

Science China(Information Sciences) 2021年第11期64卷 80-94页

作者： Qiang-Sheng HUA Lixiang QIAN Dongxiao YU Xuanhua SHI Hai JIN National Engineering Research Center for Big Data Technology and System/Services Computing Technology and System Lab/Cluster and Grid Computing Lab School of Computer Science and TechnologyHuazhong University of Science and Technology School of Computer Science and Technology Shandong University

computing the weighted girth, which is the sum of weights of edges in the minimum weight cycle,is an important problem in network analysis. The problem for distributively computing girth in unweighted graphs has garnered lots of attention, but there are few studies in weighted graphs. In this paper, we propose a distributed randomized algorithm for computing the weighted girth in weighted graphs with integral edge weights in the range [1, nc], where n is the number of vertices and c is a constant. The algorithm is devised under the standard synchronous CON GE S T model, which limits each vertex can only transfer O（log n） bits information along each incident edge in a round. The upper bound of the algorithm is O（n log2n） rounds. We also prove the lower bound for computing the weighted girth is ?（D + n/log n） where D is the hop diameter of the weighted graph. This means our distributed algorithm is optimal within a factor of O（log3n）.

关键词： distributed algorithms weighted girth \({\cal C}{\cal O}{\cal N}{\cal G}{\cal E}{\cal S}{\cal T}\) model communication complexity round complexity

来源：评论

学校读者我要写书评

暂无评论

Dendritic Neural Regression Model Trained by Chicken Swarm Optimization Algorithm for Bank Customer Churn Prediction 30th

Dendritic Neural Regression Model Trained by Chicken Swarm O...

引用

30th International Conference on Neural Information Processing, ICONIP 2023

作者： Wang, Qi Zhang, Haiyan Ji, Junkai Tang, Cheng Tang, Yajiao College of Economics Central South University of Forestry and Technology Changsha410004 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen518060 China Faculty of Information Science and Electrical Engineering Kyushu University Fukuoka819-0395 Japan

ISBN: (纸本)9789819981830

Recently, banks are constantly facing the problem of customers churning. Customer churn not only leads to a decline in bank funds and profits but also reduces its credit capacity and affects the bank’s operational management. As an important component of Customer Relationship Management, predicting customer churn has been increasingly urgent. Inspired by biological neurons, we build up a dendritic neural regression model (DNRM) with four layers, namely the synaptic layer, the dendritic layer, the membrane layer, and the soma layer for bank customer churn prediction. To pursue better prediction performance in this experiment, the Chicken Swarm Optimization (CSO) algorithm is defined as the training algorithm of DNRM. With the ability to balance exploration and exploitation, CSO is implemented to optimize and improve the accuracy of the DNRM. In this paper, we propose a novel dendritic neural regression model called CSO-DNRM for churn prediction, and the experimental results are based on a benchmark dataset from Kaggle. Compared with other algorithms and models, our proposed model obtains the highest accuracy of 92.27% and convergence speed in customer churn prediction. Due to the novel bionic algorithms and the pruning function of the model, it is evident that our proposed model has advantages in accuracy and computational speed in the field of customer churn prediction and can be widely applied in commercial bank customer relationship management. © 2024, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

Last-X-Generation Archiving Strategy for Multi-Objective Evolutionary Algorithms 13

Last-X-Generation Archiving Strategy for Multi-Objective Evo...

引用

13th IEEE Congress on Evolutionary Computation, CEC 2024

作者： Shu, Tianye Nan, Yang Shang, Ke Ishibuchi, Hisao Southern University of Science and Technology Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation Department of Computer Science and Engineering Shenzhen518055 China Southern University of Science and Technology Shenzhen518055 China Shenzhen University National Engineering Laboratory for Big Data System Computing Technology Shenzhen518060 China

ISBN: (纸本)9798350308365

For evolutionary multi-objective optimization algorithms (EMOAs), an external archive can be utilized for saving good solutions found throughout the evolutionary process. Recent studies showed that a solution set selected from an external archive is usually superior to the final population. That is, the incorporation of an external archive improves the performance of EMOAs. However, the computation time for maintaining an external archive is long, especially when the archive size is large. To solve this issue, a simple archiving strategy is to save all solutions generated in the last several generations. In this paper, we examine this archiving strategy for three representative EMOAs on artificial test problems (Minus-DTLZ and WFG) and real-world problems (RE). Our results show that archiving the last several generations clearly improves the performance of EMOAs without severely increasing the computation time. © 2024 IEEE.

关键词： Optimization algorithms

来源：评论

学校读者我要写书评

暂无评论

Efficient distributed algorithms for holistic aggregation functions on random regular graphs

引用

Science China(Information Sciences) 2022年第5期65卷 32-50页

作者： Lin JIA Qiang-Sheng HUA Haoqiang FAN Qiuping WANG Hai JIN National Engineering Research Center for Big Data Technology and System/Services Computing Technology and System Lab/Cluster and Grid Computing Lab School of Computer Science and TechnologyHuazhong University of Science and Technology Institute for Interdisciplinary Information Science Tsinghua University

In this paper, we propose efficient distributed algorithms for three holistic aggregation functions on random regular graphs that are good candidates for network topology in next-generation data *** three holistic aggregation functions include SELECTION（select the k-th largest or smallest element）,DISTINCT（query the count of distinct elements）, MODE（query the most frequent element）. We design three basic techniques — Pre-order Network Partition, Pairwise-independent Random Walk, and Random Permutation Delivery, and devise the algorithms based on the techniques. The round complexity of the distributed SELECTION is Θ（log N） which meets the lower bound where N is the number of nodes and each node holds a numeric element. The round complexity of the distributed DISTINCT and MODE algorithms are O（log3N/log log N） and O（log2N log log N） respectively. All of our results break the lower bounds obtained on general graphs and our distributed algorithms are all based on the CON GE S T model, which restricts each node to send only O（log N） bits on each edge in one round under synchronous communications.

关键词： distributed algorithms holistic aggregation function random regular graph \({\cal C}{\cal O}{\cal N}{\cal G}{\cal E}{\cal S}{\cal T}\) model communication complexity round complexity

来源：评论

学校读者我要写书评

暂无评论

Mining Top-k Frequent Patterns over Streaming Graphs 28th

Mining Top-k Frequent Patterns over Streaming Graphs

引用

28th International Conference on database systems for Advanced Applications, DASFAA 2023

作者： Wang, Xi Zhang, Qianzhen Guo, Deke Zhao, Xiang Institute for Quantum Information and State Key Laboratory of High Performance Computing National University of Defense Technology Changsha China Science and Technology on Information Systems Engineering Laboratory National University of Defense Technology Changsha China Laboratory for Big Data and Decision National University of Defense Technology Changsha China

ISBN: (纸本)9783031306747

Mining top-k frequent patterns is an important operation on graphs, which is defined as finding k interesting subgraphs with the highest frequency. Most existing work assumes a static graph. However, graphs are dynamic in nature, which is described as streaming graphs. Mining top-k frequent patterns in streaming graphs is challenging due to the streaming nature of the input and the exponential time complexity of the problem. A naive solution is to calculate approximations of the frequent patterns in the streaming graph and then find the top-k answers, which is a memory- and time-consuming method. In this paper, we design a novel auxiliary data structure, FPC, to detect valid subgraph patterns and their frequency in real-time. We first convert each newly produced subgraph into a sequence and then map it into corresponding tracks in FPC based on hash functions. We theoretically prove that FPC can provide unbiased estimation and then give an error bound of our algorithm. In addition, we propose a vertical hashing and candidate buckets sampling technique to further improve FPC with higher space utilization and higher accuracy. Extensive experiments confirm that our approach generates high-quality results compared to the baseline method. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Hash functions

来源：评论

学校读者我要写书评

暂无评论

FedEdge: Accelerating Edge-Assisted Federated Learning 23

FedEdge: Accelerating Edge-Assisted Federated Learning

引用

2023 World Wide Web Conference, WWW 2023

作者： Wang, Kaibin He, Qiang Chen, Feifei Jin, Hai Yang, Yun School of Computer Science and Technology Huazhong University of Science and Technology China Department of Computing Technologies Swinburne University of Technology Australia School of Information Technology Deakin University Australia National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab Huazhong University of Science and Technology Wuhan430074 China

ISBN: (纸本)9781450394161

Federated learning (FL) has been widely acknowledged as a promising solution to training machine learning (ML) model training with privacy preservation. To reduce the traffic overheads incurred by FL systems, edge servers have been included between clients and the parameter server to aggregate clients' local models. Recent studies on this edge-assisted hierarchical FL scheme have focused on ensuring or accelerating model convergence by coping with various factors, e.g., uncertain network conditions, unreliable clients, heterogeneous compute resources, etc. This paper presents our three new discoveries of the edge-assisted hierarchical FL scheme: 1) it wastes significant time during its two-phase training rounds;2) it does not recognize or utilize model diversity when producing a global model;and 3) it is vulnerable to model poisoning attacks. To overcome these drawbacks, we propose FedEdge, a novel edge-assisted hierarchical FL scheme that accelerates model training with asynchronous local federated training and adaptive model aggregation. Extensive experiments are conducted on two widely-used public datasets. The results demonstrate that, compared with state-of-the-art FL schemes, FedEdge accelerates model convergence by 1.14 × -3.20 ×, and improves model accuracy by 2.14% - 6.63%. © 2023 ACM.

关键词： Edge computing

来源：评论

学校读者我要写书评

暂无评论

TCSloT: Text Guided 3D Context and Slope Aware Triple Network for Dental Implant Position Prediction

TCSloT: Text Guided 3D Context and Slope Aware Triple Networ...

引用

2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

作者： Yang, Xinquan Xie, Jinheng Li, Xuechen Li, Xuguang Shen, Linlin Deng, Yongqiang Shenzhen University College of Computer Science and Software Engineering Shenzhen China National University of Singapore Show Lab Singapore Shenzhen University National Engineering Laboratory for Big Data System Computing Technology Shenzhen China Shenzhen University General Hospital Department of Stomatology Shenzhen China

ISBN: (纸本)9798350337488

In implant prosthesis treatment, the surgical guide of implant is used to ensure accurate implantation. However, such design heavily relies on the manual location of the implant position. When deep neural network has been proposed to assist the dentist in locating the implant position, most of them take a single slice as input, which do not fully explore 3D contextual information and ignores the influence of implant slope. In this paper, we design a Text Guided 3D Context and Slope Aware Triple Network (TCSloT) to integrate the perception of contextual information from multiple adjacent slices and awareness of variation of implant slopes. A Texture Variation Perception (TVP) module is correspondingly design to process the multiple slices and capture the texture variation among slices and a Slope-Aware Loss (SAL) is proposed to dynamically assign adaptive weights for the regression head. Additionally, we design a conditional text guidance (CTG) module to integrate the text condition (i.e., left, middle and right) from the CLIP to assist the implant position prediction. Extensive experiments on a dental implant dataset through five-fold cross-validation, demonstrated that the proposed TCSloT achieves superior performance than existing methods. © 2023 IEEE.

关键词： Dental Implant Multi-module Detection Multi-slice Learning Text Guided Detection Transformer

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：