检索结果-内蒙古大学图书馆

Achieving Peak Performance for Large Language Models: A Systematic Review

IEEE ACCESS 2024年 12卷 96017-96050页

作者： Rostam, Zhyar Rzgar K. Szenasi, Sandor Kertesz, Gabor Obuda Univ Doctoral Sch Appl Informat & Appl Math H-1034 Budapest Hungary Obuda Univ John von Neumann Fac Informat H-1034 Budapest Hungary J Selye Univ Fac Econ & Informat Komarno 94501 Slovakia Inst Comp Sci & Control SZTAK Hungarian Res Network HUN REN Lab Parallel & Distributed Syst H-1111 Budapest Hungary

In recent years, large language models (LLMs) have achieved remarkable success in natural language processing (NLP). LLMs require an extreme amount of parameters to attain high performance. As models grow into the trillion-parameter range, computational and memory costs increase significantly. This makes it difficult for many researchers to access the resources needed to train or apply these models. Optimizing LLM performance involves two main approaches: fine-tuning pre-trained models for specific tasks to achieve state-of-the-art performance, and reducing costs or improving training time while maintaining similar performance. This paper presents a systematic literature review (SLR) following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. We reviewed 65 publications out of 983 from 2017 to December 2023, retrieved from 5 databases. The study presents methods to optimize and accelerate LLMs while achieving cutting-edge results without sacrificing accuracy. We begin with an overview of the development of language modeling, followed by a detailed explanation of commonly used frameworks and libraries, and a taxonomy for improving and speeding up LLMs based on three classes: LLM training, LLM inference, and system serving. We then delve into recent optimization and acceleration strategies such as training optimization, hardware optimization, scalability and reliability, accompanied by the taxonomy and categorization of these strategies. Finally, we provide an in-depth comparison of each class and strategy, with two case studies on optimizing model training and enhancing inference efficiency. These case studies showcase practical approaches to address LLM resource limitations while maintaining performance.

关键词： Optimization methods Training Parallel processing Computational modeling Scalability Libraries Taxonomy distributed computing Graphics processing units distributed training GPU acceleration large language model LLM LLM acceleration LLM frameworks LLM optimization LLM optimization

来源：评论

学校读者我要写书评

暂无评论

Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks

引用

EGYPTIAN INFORMATICS JOURNAL 2024年 28卷

作者： Krstic, Lazar Ivanovic, Milos Simic, Visnja Stojanovic, Boban Univ Kragujevac Fac Sci 12 Radoja Domanovica St Kragujevac 34000 Serbia

The paper presents the GeNNsem (Genetic algorithm ANNs ensemble) software framework for the simultaneous optimization of individual neural networks and building their optimal ensemble. The proposed framework employs a genetic algorithm to search for suitable architectures and hyperparameters of the individual neural networks to maximize the weighted sum of accuracy and diversity in their predictions. The optimal ensemble consists of networks with low errors but diverse predictions, resulting in a more generalized model. The scalability of the proposed framework is ensured by utilizing micro-services and Kubernetes batching orchestration. GeNNsem has been evaluated on two regression benchmark problems and compared with related machine learning techniques. The proposed approach exhibited supremacy over other ensemble approaches and individual neural networks in all common regression modeling metrics. Real-world use-case experiments in the domain of hydro-informatics have further demonstrated the main advantages of GeNNsem: requires the least training sessions for individual models when optimizing an ensemble;networks in an ensemble are generally simple due to the regularization provided by a trivial initial population and custom genetic operators;execution times are reduced by two orders of magnitude as a result of parallelization.

关键词： Ensemble modeling Regression ANN Genetic algorithm distributed computing

来源：评论

学校读者我要写书评

暂无评论

A Scalable Federated Learning Approach for Collaborative Smart Healthcare Systems With Intermittent Clients Using Medical Imaging

引用

IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 2024年第6期28卷 3293-3304页

作者： Ullah, Farhan Srivastava, Gautam Xiao, Heng Ullah, Shamsher Lin, Jerry Chun-Wei Zhao, Yue Northwestern Polytech Univ Sch Software Xian 710072 Peoples R China Brandon Univ Dept Math & Comp Sci Brandon MB R7A 6A9 Canada China Med Univ Res Ctr Interneural Comp Taichung Taiwan Lebanese Amer Univ Dept Comp Sci & Math Beirut 11022801 Lebanon Shenzhen Univ Coll Comp Sci & Software Engn Shenzhen 518060 Peoples R China Silesian Tech Univ Fac Automatic Control Elect & Comp Sci PL-44100 Gliwice Poland

The healthcare industry is one of the most vulnerable to cybercrime and privacy violations because health data is very sensitive and spread out in many places. Recent confidentiality trends and a rising number of infringements in different sectors make it crucial to implement new methods that protect data privacy while maintaining accuracy and sustainability. Moreover, the intermittent nature of remote clients with imbalanced datasets poses a significant obstacle for decentralized healthcare systems. Federated learning (FL) is a decentralized and privacy-protecting approach to deep learning and machine learning models. In this article, we implement a scalable FL framework for interactive smart healthcare systems with intermittent clients using chest X-ray images. Remote hospitals may have imbalanced datasets with intermittent clients communicating with the FL global server. The data augmentation method is used to balance datasets for local model training. In practice, some clients may leave the training process while others join due to technical or connectivity issues. The proposed method is tested with five to eighteen clients and different testing data sizes to evaluate performance in various situations. The experiments show that the proposed FL approach produces competitive results when dealing with two distinct problems, such as intermittent clients and imbalanced data. These findings would encourage medical institutions to collaborate and use rich private data to quickly develop a powerful patient diagnostic model.

关键词： Servers Medical services Data models Training Federated learning Machine learning Hospitals CNN distributed computing data privacy deep learning Federated Learning healthcare

来源：评论

学校读者我要写书评

暂无评论

Heterogeneous Workload-Based Consumer Resource Recommendation Model for Smart Cities: eHealth Edge-Cloud Connectivity Using Federated Split Learning

引用

IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 2024年第1期70卷 4187-4196页

作者： Ahmed, Syed Thouheed Kumar, V. Vinoth Jeong, Junho Indian Inst Technol Hyderabad Dept Elect Engn Hyderabad 502205 India Vellore Inst Technol Sch Comp Sci Engn & Informat Syst Vellore 632014 India Dongguk Univ Dept Comp Sci & Engn Seoul 04620 South Korea

Over the past decade, there has been a significant surge in consumer application services and server connectivity, and this trend is expected to double in 2030. The primary contributors to the increased demand for network resources are devices connected through third-party service providers and mobile operators. Many prominent consumer services rely on a client-server architecture, which can introduce latency delays in the communication channel. Additionally, peer-to-peer (P2P) communication places a substantial load on eHealth servers, leading to service delays. In this research paper, we propose a model for scheduling heterogeneous workloads and recommending resources for eHealth edge-cloud connectivity using Federated Split Learning (FSL) model for smart cities. distributed FSL offers a robust solution for handling both direct and indirect user requests through a distributed mobile core operator stack. This technique empowers eHealth administrators to locally learn optimal policies and make informed decisions by prioritizing resource allocation and scheduling. We demonstrate the effectiveness of this technique through an active simulation server designed for track-driven caching policy and local policy scheduling, ultimately enhancing resource recommendation in eHealth applications. The proposed technique is focused on the development of a heterogeneous workload recommendation system and obtained accuracy of 89.63% over 200 users trails.

关键词： Federated split learning smart cities eHealth server distributed computing edge computing resource recommendation

来源：评论

学校读者我要写书评

暂无评论

USING A FEDERATED APPROACH TO SYNTHESISE IMAGES OF CONFIDENTIAL SCENE MODELS

LIGHT & ENGINEERING

引用

LIGHT & ENGINEERING 2024年第4期32卷 89-102页

作者： Zhdanov, Dmitry D. Guskov, Kirill S. Zhdanov, Andrei D. Potemin, Igor S. Kulbako, Artemy Yu Alexandrov, Yuri, V Lopatin, Alexei, V Sokolov, Vadim G. ITMO Univ Fac Software Engn & Comp Engn St Petersburg Russia ITMO Univ St Petersburg Russia

In some cases, a problem arises related to the synthesis of images of scenes whose data cannot be transferred to the rendering system. This may be due to the secrecy regime and fear of possible data leakage. On the other hand, the rendering system developer may be wary of unlicensed use of the software, resulting in reluctance to supply the software for installation on the customer's computers. The paper proposes a solution to this problem by isolating the rendering system from the scene data. For this purpose, an object- oriented organization of scene data and the necessary basic scene interfaces are used to organize path tracing methods and light rays. The rendering system itself is divided into two components: the customer's side and the owner's side of the rendering system. Only that part of the rendering system is supplied to the customer's side, which is responsible for the basic methods of the scene model, implemented in the form of services used by the owner of the rendering system. Client-side services, in accordance with scene interfaces, perform basic operations with scene data and rays without transferring the entire result to the rendering system, which, in turn, manages the rendering process using metrics and the minimum necessary intermediate data received from the client side. Thus, the scene data and the rendered image remain with the client, and the rendering process is managed by the developer's server. Naturally, the interaction between the client and the rendering system slows down the calculation process, but the image quality does not decrease, and the data is not transferred anywhere. The article presents the rendering results for a few scenes using a federated approach to image synthesis.

关键词： rendering federated rendering ray tracing scene programming interface abstract colour distributed computing server-client model data privacy

来源：评论

学校读者我要写书评

暂无评论

DisCOV: distributed COVID-19 Detection on X-Ray Images With Edge-Cloud Collaboration

引用

IEEE TRANSACTIONS ON SERVICES computing 2022年第3期15卷 1206-1219页

作者： Xu, Xiaolong Tian, Hao Zhang, Xuyun Qi, Lianyong He, Qiang Dou, Wanchun Nanjing Univ Informat Sci & Technol Sch Comp & Software Nanjing 210044 Peoples R China Nanjing Univ Informat Sci & Technol Engn Res Ctr Digital Forens Minist Educ Nanjing 211544 Peoples R China Soochow Univ Prov Key Lab Comp Informat Proc Technol Suzhou 215006 Jiangsu Peoples R China Macquarie Univ Dept Comp Sydney NSW 2109 Australia Qufu Normal Univ Sch Informat Sci & Engn Rizhao 276826 Peoples R China Nanjing Univ State Key Lab Novel Software Technol Nanjing 210023 Peoples R China Swinburne Univ Technol Sch Software & Elect Engn Melbourne Vic 3122 Australia

Currently, the world is experiencing the rapid spread of Coronavirus Disease 2019 (COVID-19). Since the epidemic continues to take a devastating impact on the society, economy, and healthcare, the real-time detection of COVID-19 is essential for fast and cost-effective diagnosis services. Fortunately, deep learning (DL), as a promising technology, enables the COVID-19 diagnosis services on chest X-ray (CXR) images. The training task of DL model is generally implemented at the centralized cloud. However, due to the geo-distributed data sources and the transmission of large amounts of raw data to the centralized cloud, the transmission latency becomes a bottleneck of the COVID-19 diagnosis model training. In this paper, we propose a distributed COVID-19 detection model training method on CXR images with edge-cloud collaboration, named DisCOV. Specifically, to improve the training efficiency and guarantee the model accuracy, a distributed lightweight model-based training algorithm is designed with the cooperation of edge computing and cloud computing. In addition, a resource allocation algorithm is developed during the training to jointly minimize the time cost and energy consumption. Extensive experiments based on real-world CXR image datasets demonstrate that DisCOV is better performed and more promising than the existing baselines.

关键词： X-ray imaging COVID-19 Computational modeling Image edge detection Data models Task analysis Collaboration Pandemica Cloud computing Edge computing distributed computing Biomedical imaging Training data COVID-19 deep learning CXR image classification edge computing edge-cloud collaboration

来源：评论

学校读者我要写书评

暂无评论

Enhancing Fairness in Autonomous Vehicles: Proof of Confidence With Blockchain Technology

引用

IEEE ACCESS 2024年 12卷 180125-180134页

作者： Ryu, Jimin Yoon, Yong Ik Sookmyung Womens Univ Dept Informat Technol Engn Seoul 04310 South Korea Sookmyung Womens Univ Dept Artificial Intelligence Engn Seoul 04310 South Korea

This paper introduces a novel framework for data management in network systems that emphasizes fairness and transparency by integrating Proof of Confidence (PoC) consensus method into blockchain-based autonomous driving systems. Unlike existing mechanisms, such as Proof of Work and Proof of Stake, PoC is designed to improve transaction efficiency while maintaining strong data integrity and supporting fair decision-making processes in dynamic driving environments. This framework enhances the management of autonomous driving systems by selecting a Lead-Node based on fairness-centered criteria and efficiently processing large data streams essential for autonomous driving. It is expected that this will be further optimized by utilizing distributed and edge computing technologies. The workflow for performing this algorithm is presented in a four-step design, which is a proposal that surpasses existing methods by adopting a process in which PoC selects a Lead-Node among nodes in the area and reaches an agreement by priority in managing a large transaction volume. The temporal efficiency of PoC is tested by comparing the consensus time with several consensus algorithms, and it could contribute as a promising alternative technology to improve the efficiency and stability of autonomous driving systems.

关键词： Autonomous vehicles Blockchains Accidents Consensus protocol Security Edge computing Synchronization Meteorology Lead Emergency services Autonomous driving data management distributed computing fairness Proof of Confidence consensus algorithm blockchain system transparency

来源：评论

学校读者我要写书评

暂无评论

Coded Matrix Computation in Wireless Network

引用

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS 2024年第6期23卷 6394-6410页

作者： Son, Kyungrak Choi, Wan HUFS Dept Informat & Commun Engn Yongin 17035 South Korea SNU Dept Elect & Comp Engn Seoul 08826 South Korea SNU Inst New Media & Commun Seoul 08826 South Korea

Despite its vital role in intelligent IoT networks, distributed computing was not studied well in wireless networks. Notably, the distortions introduced by wireless channels and noise can significantly undermine the accuracy of distributed computations. Unfortunately, existing distributed computing schemes inadequately address this critical issue. In this paper, we aim at resolving the numerical stability issue associated within wireless networks and building a new computationally accurate design framework which concretely integrates distributed computing and communication. To this end, we first explore the previous group algebra based coded matrix computation scheme, suitable for numerically stable computation in noisy network, in orthogonal multiple access channel and perform a thorough analysis of computational errors. Furthermore, to leverage the inherent communication structure constructed in distributed computing and thus more efficiently utilize limited wireless resources, we propose to combine the Compute-and-Forward scheme with coded matrix computation. Furthermore, we devise a novel transmission scheme for distributed computing called the Broadcast-and-Compute scheme, in which a master node broadcasts the same information, not transmitting differently encoded submatrices to worker nodes. We analyze the computation errors of Compute-and-Forward and Broadcast-and-Compute based coded matrix computation. The superiority of the proposed schemes is validated by various simulations.

关键词： distributed computing Task analysis Multiuser channels Wireless networks Algebra Numerical stability Encoding representation theory compute-and-forward broadcast-and-compute computation error analysis

来源：评论

学校读者我要写书评

暂无评论

A distributed Attribute Reduction Algorithm for High-Dimensional Data under the Spark Framework

引用

INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE SYSTEMS 2022年第1期15卷 1-14页

作者： Wu, Zhengjiang Mei, Qiuyu Zhang, Yaning Yang, Tian Luo, Junwei Henan Polytech Univ Coll Comp Sci & Technol 2001 Century Ave Jiaozuo 454000 Henan Peoples R China

Attribute reduction is an important issue in rough set theory. However, the rough set theory-based attribute reduction algorithms need to be improved to deal with high-dimensional data. A distributed version of the attribute reduction algorithm is necessary to enable it to effectively handle big data. The partition of attribute space is an important research direction. In this paper, a distributed attribution reduction algorithm based on cosine similarity (DARCS) for high-dimensional data pre-processing under the Spark framework is proposed. First, to avoid the repeated calculation of similar attributes, the algorithm gathers similar attributes based on similarity measure to form multiple clusters. And then one attribute is selected randomly as a representative from each cluster to form a candidate attribute subset to participate in the subsequent reduction operation. At the same time, to improve computing efficiency, an improved method is introduced to calculate the attribute dependency in the divided sub-attribute space. Experiments on eight datasets show that, on the premise of avoiding critical information loss, the reduction ability and computing efficiency of DARCS have been improved by 0.32 to 39.61% and 31.32 to 93.79% respectively compared to the distributed version of attribute reduction algorithm based on a random partitioning of the attributes space.

关键词： Rough set theory distributed computing Attribute reduction High-dimensional data

来源：评论

学校读者我要写书评

暂无评论

distributed Application Architecture and LinkNet Topology Processor for Distribution Networks Using the Common Information Model

引用

IEEE ACCESS 2022年 10卷 120765-120780页

作者： Anderson, Alexander A. A. Podmore, Robin Sharma, Poorva Reiman, Andrew P. P. Jinsiwale, Rohit A. A. Allwardt, Craig H. H. Black, Gary D. D. Pacific Northwest Natl Lab Energy Infrastructure & Bldg Div Richland WA 99352 USA Incremental Syst Corp Issaquah 98027 WA USA Pacific Northwest Natl Lab Adv Comp Math & Data Div Richland WA 99352 USA

Ongoing decarbonization efforts and increasing penetrations of distributed energy resources (DERs) may soon disrupt current operational paradigms, requiring a shift from centralized control and optimization to hierarchical and distributed applications. To support this transition, a distributed application architecture for communication and control of distribution network assets is introduced. The architecture defines a structure of communication and coordination layers based on the Laminar Coordination Framework. The challenge of defining distributed control areas is resolved through definition of switch-delimited topological areas based on the physical topology of the feeder instead of communications network layers. Implementation of the architecture in standards-based platforms is enabled by a robust graph-based topology processor leveraging the class structure of the Common Information Model (CIM), which differs significantly from traditional bus-branch representations by explicitly modeling the nodes and terminals of every type of power system equipment. The topology processor creates a set of LinkNet linked list structures for indexing the nodes and terminals of all CIM class object instances and mapping of the feeder topology in real-time. The data structures and algorithm are computationally lightweight, with performance tests presented for several IEEE distribution test systems.

关键词： Active distribution networks common information model distributed computing microgrids network topology smart grids

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：