检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Wang, Wei Lai, Zhiquan Li, Shengwei Liu, Weijie Ge, Keshi Shen, Ao Su, Huayou Li, Dongsheng The National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Hunan Changsha China

he size of deep learning models has been increasing to enhance model quality. The linear increase in training computation budget with model size means that training an extremely large-scale model is exceedingly time-consuming. Recently, the Mixture of Expert (MoE) has drawn significant attention as it can scale models to extra-large sizes with a stable computation budget. However, inefficient distributed training of large-scale MoE models hinders their broader application. Specifically, a considerable dynamic load imbalance occurs among devices during training, significantly reducing throughput. Several load-balancing works have been proposed to address the challenge. System-level solutions draw more attention for their hardware affinity and non-disruption of model convergence compared to algorithm-level ones. However, they are troubled by high communication costs and poor communication-computation overlapping. To address these challenges, we propose a systematic load-balancing method, Pro-Prophet, which consists of a planner and a scheduler for efficient parallel training of large-scale MoE models. To adapt to the dynamic load imbalance, we profile training statistics and use them to design Pro-Prophet. For lower communication volume, Pro-Prophet planner determines a series of lightweight load-balancing strategies and efficiently searches for a communication-efficient one for training based on the statistics. For sufficient overlapping of communication and computation, Pro-Prophet scheduler schedules the data-dependent operations based on the statistics and operation features, further improving the training throughput. We conduct extensive experiments in four clusters and five MoE models. The results indicate that Pro-Prophet achieves up to 2.66x speedup compared to two popular MoE frameworks including Deepspeed-MoE and FasterMoE. Furthermore, Pro-Prophet has demonstrated a load-balancing improvement of up to 11.01x compared to a representative load-balancing work,

关键词： Budget control

来源：评论

学校读者我要写书评

暂无评论

Grid computing in China

引用

Journal of Grid computing 2004年第2期2卷 193-206页

作者： Yang, Guangwen Jin, Hai Li, Minglu Xiao, Nong Li, Wei Wu, Zhaohui Wu, Yongwei Tang, Feilong Department of Computer Science and Technology Tsinghua University Beijing China School of Computer Huazhong University of Science and Technology Wuhan China Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China Key Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha China Institute of Computing Technology Chinese Academy of Sciences Beijing 100080 China College of Computer Science Zhejiang University Hangzhou 310027 China

Grid computing presents a new trend to distributed computation and Internet applications, which can construct a virtual single image of heterogeneous resources, provide uniform application interface and integrate widespread computational resources into super, ubiquitous and transparent aggregation. In the adoption of Grid computing, China, who is facing more resource heterogeneity and other specific demands, has put much effort on both research and practical utilization. In this paper, we introduce the major China Grid research projects and their perspective applications. First we give the overview of the four government-sponsored programs in Grid, namely the China national Grid, ChinaGrid, NSFC Grid, and ShanghaiGrid. Then we present six representative ongoing Grid systems in details, which are categorized into Grid middleware and Grid application. This paper provides the general picture of Grid computing in China, and shows the great efforts, devotion and confidence in China to use Grid technology to boost the society, economics and scientific research. © Springer 2004.

关键词： Grid computing

来源：评论

学校读者我要写书评

暂无评论

Temporal Closing Path for PLM-based Temporal Knowledge Graph Completion

Temporal Closing Path for PLM-based Temporal Knowledge Graph...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Xin Zhou Yongxue Shan Zixuan Dong Haijiao Liu Xiaodong Wang National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Denfense Technology Changsha China

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

Temporal Knowledge Graph Completion (TKGC) aims to predict missing parts of quadruples, which is crucial for real-life knowledge graphs. Compared with methods that only use graph neural networks, the emergence of pre-trained model has introduced a trend of simultaneously leveraging text and graph structure information. However, most current methods based on pre-trained models struggle to effectively utilize both text and multi-hop graph structure information concurrently, resulting in insufficient association mining of relations. To address the challenge, we propose a novel model: Temporal Closing Path for Pre-trained Language Model-based TKGC (TCP-PLM). We obtain the temporal closing relation path of the target relation through sampling, and use the relation path as a bridge to simultaneously utilize text and multi-hop graph structure information. Moreover, the relation path serves as a tool for mining associations between relations. At the same time, due to the design of entity-independent relation paths, our model can also handle the inductive setting. Our experiments on three benchmarks, along with extensive analysis, demonstrate that our model not only achieves substantial performance enhancements across four metrics compared to other models but also adeptly handles inductive settings.

关键词： Training Measurement Knowledge engineering Bridges Analytical models Knowledge graphs Transforms

来源：评论

学校读者我要写书评

暂无评论

Fast query over encrypted data based on B+ tree

Fast query over encrypted data based on B+ tree

引用

2009 International Conference on Apperceiving computing and Intelligence Analysis, ICACIA 2009

作者： Wang, Zheng-Fei Tang, Ai-Guo Wang, Wei Department of Computer Hunan Business College Changsha 410205 China National Key Laboratory for Parallel and Distributed Processing School of Computer National University of Defense Technology Changsha 410073 China School of Computer Science Fudan University Shanghai 200433 China

ISBN: (纸本)9781424452040

There are a lot of important and sensitive data in databases, which need to be protected from attacks. To secure the data, Cryptography support is an effective mechanism. However, a tradeoff must be made between the performance and the security because encryption and decryption greatly degrade the query performance. In this paper, a framework is proposed that can quickly implement query over encrypted data based on B+ tree. When data are stored in the form of cipher, it not only stores the encrypted data themselves, but also creates a B+ tree index for the data before encrypting them. When querying the encrypted data, firstly, it locates the encrypted records related to the querying predicate based on the B+ tree index;secondly, it decrypts the encrypted records to accomplish the results. Meanwhile, it must encrypt the B+ tree itself to protect it from leaking confidential information. According to the structure of the B+ tree, it encrypts each node of the B+ tree separately. The results of experiments show that the query performance over the encrypted data decreases about 20 percent compared with the plaintext query performance, and validate the functionality and usability of our approach. ©2009 IEEE.

关键词： Cryptography

来源：评论

学校读者我要写书评

暂无评论

A Multi-Dimensional Assessment Model and Its Application in E-learning Courses of computer Science 20

A Multi-Dimensional Assessment Model and Its Application in ...

引用

21st Annual Conference of the Special Interest Group in Information Technology Education, SIGITE 2020

作者： Luo, Jiwen Lu, Feng Wang, Tao Key Lab. of Parallel and Distributed Computing College of Computer National University of Defense Technology Changsha China National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China

ISBN: (纸本)9781450370455

computer science is a practical discipline. It is always a great challenge to evaluate students' computer practice using computer-aided means for large scale students. We always need to address problems such as suspected plagiarism and deviation of the overall difficulty factor. In this paper, a multi-dimensional assessment model is designed for CS courses based on the detailed practice processing data in an E-learning system. The model comprehensively evaluates the students' learning process and results in three aspects of correctness, originality, and quality detection. Besides, the teacher can easily participate in the assessment according to their needs. The correctness is an essential requirement, and the originality is based on the clustering results of students' behaviors after clone detection to curb homework plagiarism. SonarQube is used to detect code quality and put forward higher requirements for codes. Manual participation intelligence has improved the flexibility and applicability of the model to a certain extent. We applied this model on the EduCoder online education platform and carried out a comprehensive analysis of 485 students in the parallel Programming Principles and Practice Class of Huazhong University of Science and Technology. Experiment results confirm the distinction, rationality, and fairness of the model in assessing student performance. It not only gives students a credible, comprehensive score in large-scale online practical programming courses but also gives teachers and students corresponding suggestions based on the evaluation results. Furthermore, the model can be extended to other online education platforms. © 2020 ACM.

关键词： Students

来源：评论

学校读者我要写书评

暂无评论

OLM2: Automatic Optimal Strategy Generating for Large-Scale Model Training with Limited-Memory

OLM2: Automatic Optimal Strategy Generating for Large-Scale ...

引用

IEEE International Conference on Joint Cloud computing (JCC)

作者： Zhilin Yang Yu Tang Linbo Qiao Xi Yang Zhen Huang National Key Laboratory of Parallel and Distributed Computing College of Computer Science National University of Defense Technology Changsha 410073 China

The scale of model parameters and the amount of training data is exponentially increasing. It requires more GPU memory with the exponential increasement of model parameters. Recomputation and swapping are two main memory optimization methods that have been extensively studied, and there are also optimization strategies that combine the two methods. However, most of them are based on heuristic search strategies, which do not explore the complete solution space and can’t guarantee the optimality of the solution results. An optimal search strategy with tensor-level recomputation and swapping is expected in large-scale model training. In this paper, we propose an optimal strategy searching algorithm combining tensor-based recomputation and swapping. Specifically, the memory swapping strategy is reformulated as an optimization problem, which converts the memory constraints into mixed integer programming, to find the optimal memory optimization strategy. By leveraging the advantages of both recomputation and swapping, this approach minimizes computation consumption without exceeding the available memory limitation. Experimental results show that our method exhibits about 60% reduction in memory requirements during the training process. Furthermore, our method can reduce the overall training time beyond the existing algorithms. Compared to Checkmate, our approach achieves about 0.3–0.9% reduction in computation cost per iteration.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Novel Evaluation Strategy to Artificial Neural Network Model Based on Bionics

引用

Journal of Bionic Engineering 2022年第1期19卷 224-239页

作者： Sen Tian Jin Zhang Xuanyu Shu Lingyu Chen Xin Niu You Wang School of Mathematics and Statistics Hunan Normal UniversityChangsha410081China College of Information Science and Engineering Hunan Normal UniversityChangsha410081China School of Computer and Communication Engineering Changsha University of Science and TechnologyChangsha410114China Key Laboratory of Industrial Control Technology Zhejiang UniversityHangzhou310058China Science and Technology on Parallel and Distributed Laboratory College of ComputerNational University of Defense TechnologyChangsha410199China Key Laboratory of Industrial Control Technology Institute of Cyber Systems and ControlZhejiang UniversityHangzhou310027China

With the continuous deepening of Artificial Neural Network(ANN)research,ANN model structure and function are improving towards diversification and ***,the model is more evaluated from the pros and cons of the problem-solving results and the lack of evaluation from the biomimetic aspect of imitating neural networks is not inclusive ***,a new ANN models evaluation strategy is proposed from the perspective of bionics in response to this problem in the ***,four classical neural network models are illustrated:Back Propagation(BP)network,Deep Belief Network(DBN),LeNet5 network,and olfactory bionic model(KIII model),and the neuron transmission mode and equation,network structure,and weight updating principle of the models are analyzed *** analysis results show that the KIII model comes closer to the actual biological nervous system compared with other models,and the LeNet5 network simulates the nervous system in ***,evaluation indexes of ANN are constructed from the perspective of bionics in this paper:small-world,synchronous,and chaotic ***,the network model is quantitatively analyzed by evaluation indexes from the perspective of *** experimental results show that the DBN network,LeNet5 network,and BP network have synchronous *** the DBN network and LeNet5 network have certain chaotic characteristics,but there is still a certain distance between the three classical neural networks and actual biological neural *** KIII model has certain small-world characteristics in structure,and its network also exhibits synchronization characteristics and chaotic *** with the DBN network,LeNet5 network,and the BP network,the KIII model is closer to the real biological neural network.

关键词： Artificial neural network(ANN) Back Propagation(BP)network Deep Belief Network(DBN) LeNet5 network Olfactory bionic model(KIII model) Small world Chaos Synchronous

来源：评论

学校读者我要写书评

暂无评论

Cooperative monitoring BGP among autonomous systems

引用

Security and Communication Networks 2015年第10期8卷 1943-1957页

作者： Hu, Ning Wang, BaoSheng Liu, Xin National Key Laboratory for Parallel and Distributed Processing College of Computer National University of Defense Technology Changsha Hunan China

As the de facto Internet inter-domain routing protocol, BGP protocol has a number of vulnerabilities and weakness. Monitoring BGP is an effective way to improve the security of inter-domain routing. This paper presents a cooperative BGP monitoring method, which is called cooperative information sharing model (CoISM). CoISM is based on self-organization and can be used to solve some problems during BGP monitoring, such as route validation and bogus route notification delivery. CoISM provides autonomous systems with a more comprehensive information view by introducing information diffuse reflection based on initiative inquiry and making use of the relativity of monitoring information. CoISM optimizes the information transmission by leveraging the data locality caused by BGP policy and implements ISP coordination with low communication and deployment cost. More specifically, CoISM provides a self-organizing and incentive mechanism, which drives autonomous systems to coordinate independently and shares information on-demand. CoISM supports incremental deployment and can also be applied to a wide range of inter-domain cooperative management applications such as inter-domain routing failure analysis and intrusion detection. © 2014 John Wiley & Sons, Ltd.

关键词： Monitoring

来源：评论

学校读者我要写书评

暂无评论

A Survey on Talking Head Generation: The Methods, Status and Challenges

SSRN

引用

SSRN 2023年

作者： Cai, Yali Qiao, Peng Li, Dongsheng National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha410073 China

The talking head generation aims to synthesize a speech video of the source identity from a driving video or audio or text data irrelevant to the source identity. It can not only be applied to games and virtual reality applications, but also provide data for fake data detection. In recent years, the research of talking head generation is widely popular, and the authenticity of the generated results has also been greatly improved. However, the synthetic results still have great room for progress. We summarize the existing researches in this paper, hoping to offer assistance for later researchers. Furthermore, we divide these methods into three categories according to the input data type, namely video, audio and text driven talking head generation methods, and analyze them in detail. In addition, we also summarize the data sets commonly used in this kind of research and explore the evaluation criteria for measuring the performance of the method. Finally, the shortcomings of the existing methods in this field and the future direction are presented in last section. © 2023, The Authors. All rights reserved.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

TeeSwap: Private Data Exchange using Smart Contract and Trusted Execution Environment 23

TeeSwap: Private Data Exchange using Smart Contract and Trus...

引用

23rd IEEE International Conference on High Performance computing and Communications, 7th IEEE International Conference on Data Science and Systems, 19th IEEE International Conference on Smart City and 7th IEEE International Conference on Dependability in Sensor, Cloud and Big Data Systems and Applications, HPCC-DSS-SmartCity-DependSys 2021

作者： Chen, Peng Shi, Peichang Xu, Jie Fu, Xiang Li, Linhui Zhong, Tao Xiang, Liangliang Kong, Jinzhu College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Processing Changsha410073 China Zhejiang Lab Hangzhou311100 China Kylin Software Co. Ltd Changsha410073 China

ISBN: (纸本)9781665494571

With importance of data value is approved, data-sharing will create more and greater value has become consensus. However, data exchange has to use a trusted third party(TTP) as an intermediary in an untrusted network environment, which may lead to leakage of data privacy. In this paper, we design a system named TeeSwap, which uses smart contract to complete data exchange, and at the same time uses cryptographic algorithms and trusted execution environment (TEE) to ensure privacy in process of data transmission, verification and contract execution. Meanwhile, TeeSwap can ensure the credibility, fairness, integrity and privacy while data are shared in our system. In the end, we implement contracts about data market and data exchange. When confirming the data transaction, data market contract can automatically generate and deploy exchange contract, which can complete the data exchange authoritatively. © 2021 IEEE.

关键词： Data privacy Smart contracts Data communication Cryptography

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：