检索结果-内蒙古大学图书馆

ISC High Performance 2024 Research Paper Proceedings (39th international conference)

作者： Sajal Dash Isaac R Lyngaas Junqi Yin Xiao Wang Romain Egele J. Austin Ellis Matthias Maiterth Guojing Cong Feiyi Wang Prasanna Balaprakash Oak Ridge National Laboratory Université Paris-Saclay AMD

Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Loss scaling studies have demonstrated the superior performance of larger LLMs compared to their smaller counterparts. Nevertheless, training LLMs with billions of parameters poses significant challenges and requires considerable computational resources. For example, training a one trillion parameter GPT-style model on 20 trillion tokens requires a staggering 120 million exaflops. This research explores efficient distributed training strategies to extract this computation from Frontier, the world's first exascale supercomputer. We enable and investigate various model and data parallel training techniques, such as tensor parallelism, pipeline parallelism, and sharded data parallelism, to facilitate training a trillion-parameter model on Frontier. We empirically assess these techniques and their associated parameters to determine their impact on memory footprint, communication latency, and GPU's computational efficiency. We analyze the complex interplay among these techniques and find a strategy to combine them to achieve high throughput through hyperparameter tuning. We have identified efficient strategies for training large LLMs of varying sizes through empirical analysis and hyperparameter tuning. For 22 Billion, 175 Billion, and 1 Trillion parameters, we achieved GPU throughputs of 38.38%, 36.14%, and 31.96%, respectively. For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved 100% weak scaling efficiency on 1024 and 3072 Mi250X GPUs, respectively. We also achieved strong scaling efficiencies of 89% and 87% for these two models. We trained these models only tens of iterations instead of training till completion.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Efficient processing of Streaming Data using Multiple Abstractions 14

Efficient Processing of Streaming Data using Multiple Abstra...

引用

IEEE 14th international conference on Cloud Computing (CLOUD)

作者： Qadeer, Abdul Heidemann, John Univ Southern Calif Informat Sci Inst Los Angeles CA 90007 USA

ISBN: (纸本)9781665400602

Large websites and distributed systems employ sophisticated analytics to evaluate successes to celebrate and problems to be addressed. As analytics grow, different teams often require different frameworks, with dozens of packages supporting with streaming and batch processing, SQL and no-SQL. Bringing multiple frameworks to bear on a large, changing dataset often create challenges where data transitions-these impedance mismatches can create brittle glue logic and performance problems that consume developer time. We propose Plumb, a meta-framework that can bridge three different abstractions to meet the needs of a large class of applications in a common workflow. Large-block streaming (Block-Streaming) is suitable for single-pass applications that care about the temporal and spatial locality. Windowed-Streaming allows applications to process a group of data and many reductions. Stateful-Streaming enables applications to keep a long-term state and always-on behavior. We show that it is possible to bridge abstractions, with a common, high-level workflow specification, while the system transitions data batch processing and block- and record-level streaming as required. The challenge in bridging abstractions is to minimize latency while allowing applications to select between sequential and parallel operation, while handling out-of-order data delivery, component failures, and providing clear semantics in the face of missing data. We demonstrate these abstractions evaluating a 10-stage workflow of DNS analytics that has been in production use with Plumb for 2 years, comparing to a brittle hand-built system that has run for more than 3 years.

关键词： programming model abstraction big-data abstraction large-block streaming programmer productivity

来源：评论

学校读者我要写书评

暂无评论

Efficient Asynchronous GCN Training on a GPU Cluster 27

Efficient Asynchronous GCN Training on a GPU Cluster

引用

27th IEEE international conference on parallel and distributed Systems, ICPADS 2021

作者： Zhang, Yi Goswami, Dhrubajyoti Concordia University Department of Computer Science and Software Engineering Montreal Canada

ISBN: (纸本)9781665408783

Research on Graph Convolutional Networks (GCNs) has increasingly gained popularity in recent years due to the powerful representational capacity of graphs. A common assumption in traditional synchronous parallel training of GCNs using multiple GPUs is that load is perfectly balanced. However, this assumption may not hold in a real-world scenario where there can be imbalances in workloads among GPUs for various reasons. In a synchronous parallel implementation, a straggler in the system can limit the overall speed up of parallel training. To address these performance issues, this research investigates approaches for asynchronous decentralized parallel training of GCNs on a GPU cluster. The techniques investigated are based on graph clustering and the Gossip protocol. The research specifically adapts the approach of Cluster GCN, which uses graph partitioning for SGD based training, and combines with a gossip algorithm specifically designed for a GPU cluster to periodically exchange gradients among randomly chosen partners (GPUs). In addition, it incorporates a work pool mechanism for load balancing among GPUs. The gossip algorithm is proven to be deadlock free. The implementation is performed on a deep learning cluster with 8 Tesla V100 GPUs per compute node, and PyTorch and DGL as the software platforms. Experiments are conducted on different benchmark datasets. The results demonstrate superior performance with similar accuracy scores, as compared to traditional synchronous training which uses 'all reduce' to synchronously accumulate parallel training results. © 2021 IEEE.

关键词： Graphics processing unit

来源：评论

学校读者我要写书评

暂无评论

A p2p environment to validate ensemble-based approaches in the cybersecurity domain 28

A p2p environment to validate ensemble-based approaches in t...

引用

28th Euromicro international conference on parallel, distributed and Network-Based processing (PDP)

作者： Folino, Francesco Folino, Gianluigi Pontieri, Luigi CNR Inst High Performance Comp & Networking ICAR Arcavacata Di Rende CS Italy

ISBN: (数字)9781728165820

ISBN: (纸本)9781728165820

The main data-driven techniques for detecting cybersecurity attacks are based on the analysis of network traffic data and/or of application/system logs (stored in a host or in some other kind of device). A wide range of machine-learning techniques (and possible alternative configurations of them) have been proposed in the literature so far, for this purpose, but none of them has been proven to consistently overcome the others across different datasets. In order to ensure better accuracy and stability, the ensemble paradigm can be exploited as an effective solution for combining such techniques. However, as attack detection problems are hard to cope with and, usually, entail the analysis of large and fast streams of data, different types of ensemble (and of base algorithms composing the ensemble) should have experimented, exploiting distributed architecture to suitably reduce the high-execution times necessary to run them. In order to handle all these issues, a p2p environment to validate ensemble-based approaches in the cybersecurity domain is proposed in this paper. Two case studies are analyzed by using this framework, which concern the detection of intrusions in network-traffic data and of deviant process instances. Preliminary scalability results demonstrate that the framework is a viable solution for these challenging kind of problems.

关键词： Ensemble-based Systems p2p frameworks cybersecurity applications

来源：评论

学校读者我要写书评

暂无评论

Advanced techniques in Data Ingestion and Pipelining for Scalable Big Data Platforms: A Comprehensive Review

Advanced Techniques in Data Ingestion and Pipelining for Sca...

引用

international conference on ICT in Business Industry & Government (ICTBIG)

作者： Sachin Murarka Anshuj Jain Laxmi Singh Department of Electronics & Communication Engineering Rabindranath Tagore University Bhopal India Department of Electronics & Communication Engineering Scope Global University Bhopal India

ISBN: (数字)9798331518981

ISBN: (纸本)9798331518998

In the era of big data, effective data ingestion and pipelining are critical for organizations to harness the power of vast and diverse data sources, such as IoT devices, social media, and transactional systems. Data ingestion refers to the process of systematically collecting and importing data for processing, while data pipelining ensures the continuous flow of this data through various transformation stages, preparing it for analysis. As data volumes grow, platforms like Hadoop, with its HDFS (Hadoop distributed File System) and MapReduce, have become essential in managing large-scale datasets by providing distributed storage and parallel processing capabilities. Additionally, tools such as Apache Kafka and Apache Flume facilitate both batch and real-time data ingestion, ensuring flexibility in handling different types of data streams. However, challenges remain, especially with the increasing complexity of datasets and the growing need for real-time processing. Emerging technologies such as cloud-native ingestion frameworks, edge computing, and AI-enhanced pipelines offer solutions to these challenges by improving scalability, performance, and cost efficiency. Machine learning is also playing an evolving role in optimizing pipeline workflows, enabling more intelligent and adaptive data processing. This paper reviews key technologies for data ingestion and pipelining, discusses their limitations, and explores cutting-edge trends aimed at enhancing the efficiency of big data platforms. These advancements promise to reshape how organizations handle big data, offering improved agility and operational effectiveness. In addition to established technologies, the rise of cloud-native architectures and AI-driven solutions is revolutionizing data ingestion and pipelining. Cloud platforms offer scalable, on-demand resources, while AI and machine learning enhance automation, anomaly detection, and predictive analytics within pipelines.

关键词： Technological innovation Data ingestion Reviews Scalability Machine learning Big Data Data processing Real-time systems Streams Pipeline processing

来源：评论

学校读者我要写书评

暂无评论

Product collaborative filtering based recommendation systems for large-scale E-commerce

引用

international Journal of Information Management Data Insights 2025年第1期5卷

作者： Trinh, Trang Nguyen, Van-Ho Nguyen, Nghia Nguyen, Duy-Nghia University of Economics and Law Ho Chi Minh City Viet Nam Vietnam National University Ho Chi Minh City Viet Nam

The rapid growth in e-commerce and the increasing diversity of customer preferences necessitates the development of an effective recommender system for a business offering a wide range of products. This paper introduces a product-based collaborative filtering approach utilizing Apache Spark, a powerful parallel processing framework to address the scalability issues of recommender systems in the cloud computing environment. Using Spark's distributed computing ability, our model attains a surprising 7.6 times speedup on the training time compared to traditional single-machine methods while preserving accuracy with a Root Mean Square Error (RMSE) 0.9. These results demonstrate the effectiveness of parallel and distributed techniques in developing efficient and accurate recommender systems for large-scale e-commerce applications. Future work will focus on applying multi-model to enhance the accuracy of prediction and configuration to optimize the cost of cluster operations. © 2025 The Author(s)

关键词： Apache spark Collaborative filtering E-commerce Large-scale parallel and distributed computing Recommendation systems

来源：评论

学校读者我要写书评

暂无评论

A Novel Approach to Braun Multiplier Design Utilizing High-Speed parallel Prefix Adder for Low Power applications

A Novel Approach to Braun Multiplier Design Utilizing High-S...

引用

international conference on Computation System and Information Technology for Sustainable Solutions (CSITSS)

作者： Kiran V E M Savi Teja Department of ECE RV College of Engineering Bengaluru India

This project develops an innovative Braun Multiplier design to address power consumption and chip area challenges. Integrating a high-speed parallel prefix adder enhances computational speed by leveraging parallel processing capabilities. The synergy between the Braun Multiplier and the adder yields a solution optimizing performance, power efficiency, and chip area usage. Meticulous design and engineering yield notable improvements in power efficiency and chip area utilization. The project drives advancements in digital circuit design and optimization methodologies, achieving a 16% area reduction and 18.2% lower power consumption via the FS-GDI technique, while achieving full-swing output voltage, distinguishing it from GDI and M-GDI techniques.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Comprehensive Analysis of Dithering Algorithms and GPU Implementations

A Comprehensive Analysis of Dithering Algorithms and GPU Imp...

引用

Applied Artificial Intelligence and Computing (ICAAIC), international conference on

作者： Radhika V. Kulkarni Aaditya Deshpande Pratik Dagale Devyani Manmode Department of Computer Engineering Vishwakarma Institute of Technology Pune India

ISBN: (数字)9798350375190

ISBN: (纸本)9798350375206

This research paper conducts a thorough comparative analysis of various dithering algorithms, prioritizing the GPU-optimized Floyd-Steinberg algorithm. Dithering plays a crucial role in improving image quality in constrained display scenarios. The study evaluates popular algorithms, including Bayer matrix, Ordered dithering, and Error Diffusion, while specifically exploring enhancements in the Floyd-Steinberg algorithm through GPU optimization. Quantitative metrics such as MSE, PSNR, and SSI are employed, alongside subjective assessments, to gauge algorithmic performance. The research delves into adaptability across color spaces and image types, providing insights for algorithm selection. Emphasizing real-time applications, the study offers developers valuable guidance in choosing efficient dithering techniques. The GPU-optimized Floyd-Steinberg algorithm emerges as a promising solution, showcasing its potential for accelerated computing environments and resource-intensive tasks.

关键词： Image quality Measurement Image color analysis Graphics processing units Focusing parallel processing Real-time systems

来源：评论

学校读者我要写书评

暂无评论

Balancing Energy Efficiency and Portability: Assessing Domain-Specific Languages in Edge Platforms

Balancing Energy Efficiency and Portability: Assessing Domai...

引用

international conference on distributed Computing in Sensor Systems (DCOSS)

作者： Youssef Faqir-Rhazoui Luis Costero Carlos García Computer Science Faculty Complutense University of Madrid Spain

ISBN: (数字)9798350369441

ISBN: (纸本)9798350369458

Edge computing stands as a crucial component of the IoT revolution. IoT benefits significantly from reduced latency, enhancing real-time applications, and time-sensitive processes. However, since edge platforms must be energy-efficient while maintaining high performance in real-time environments, exploiting accelerators becomes pivotal to reduce energy consumption while increasing performance. CUDA or HIP languages are widely used to program GPUs; however, they are limited to specific vendor platforms. In contrast, solutions such as SYCL or Kokkos could benefit from their ability to run across parallel *** paper explores the trade-offs of CUDA, SYCL, and Kokkos in three Edge platforms: Jetson AGX Orin, Jetson Orin Nano, and UP Squared Pro 7000 (Intel). The work conducts an analysis of the power modes of each platform and compares CUDA, SYCL, and Kokkos on those *** experimental results revealed that, on average, there is an increased level of energy consumption compared to native solutions. Specifically, Kokkos demonstrated a 25% higher demand, whereas SYCL exhibited an average increase of 11%. These figures underscore the trade-off that programmers face in pursuit of achieving code portability when most performant solutions are non-crucial.

关键词： Energy consumption Graphics processing units Real-time systems Energy efficiency Internet of Things distributed computing Hip

来源：评论

学校读者我要写书评

暂无评论

A Bidirectional Encoder Representations from Transformers – Multi Head Attention for Predicting the Stock Market Volatility Based on Textual Sentiment Analysis

A Bidirectional Encoder Representations from Transformers –...

引用

Integrated Intelligence and Communication Systems (ICIICS), international conference on

作者： G. Anjali Muntather Almusawi N. Zahira Jahan M. Mumjitha S Meenakshi Sundaram School of Computer Science and Engineering Malla Reddy Engineering College for Women (UGC-Autonomous) Maisammaguda Hyderabad India Department of Computers Techniques Engineering College of Technical Engineering The Islamic University Najaf Iraq Department of Computer Applications Nandha Engineering College (Autonomous) Erode India Department of Chemistry New Prince Shri Bhavani College of Engineering and Technology Chennai India Department of Computer Science and Engineering Nitte Meenakshi Institute of Technology Bengaluru India

ISBN: (数字)9798331504960

ISBN: (纸本)9798331504977

Recent days, stock market plays an important role in country's economic landscape, serving as a vital conduit for companies and investors to facilitate investments. As a cornerstone of economic growth. The stock market offers unparalleled opportunities for businesses to access capital and expand their operations. The existing approaches for stock market prediction based on textual sentiment analysis has gradient vanishing problems and limited parallelization. Therefore, this paper proposes Bidirectional Encoder Representations from Transformers (BERT)- Multi Head Attention which effectively enabled parallel processing and the transformer minimizes the gradient vanishing problem. Initially, the data is taken from Xueqiu investors community and the processing of text is done in three steps they are web crawling, data cleaning and data transforming. Then, this data is used for measuring volatility which helps the investors assess market risk and potential losses. Finally, proposed BERT- Multi Head Attention is used for predicting and effectively enabled the parallel processing and minimizes the problem of vanishing gradient. The proposed BERT- Multi Head Attention achieved better results in terms of accuracy of 90.65% and Fl-Score of 88.76% when compared with existing NB.

关键词： Support vector machines Sentiment analysis Head Accuracy parallel processing Streaming media Transformers Loss measurement Cleaning Stock markets

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：