检索结果-内蒙古大学图书馆

International Conference on data engineering

作者： Zhipeng Zhang Wentao Wu Jiawei Jiang Lele Yu Bin Cui Ce Zhang School of EECS & Key Laboratory of High Confidence Software Technologies (MOE) Peking University Microsoft Research Redmond USA ETH Zurich Switzerland Tencent Inc Center for Data Science Peking University & National Engineering Laboratory for Big Data Analysis and Applications

ISBN: (数字)9781728129037

ISBN: (纸本)9781728129044

Distributed machine learning (ML) has triggered tremendous research interest in recent years. Stochastic gradient descent (SGD) is one of the most popular algorithms for training ML models, and has been implemented in almost all distributed ML systems, such as Spark MLlib, Petuum, MXNet, and TensorFlow. However, current implementations often incur huge communication and memory overheads when it comes to large models. One important reason for this inefficiency is the row-oriented scheme (RowSGD) that existing systems use to partition the training data, which forces them to adopt a centralized model management strategy that leads to vast amount of data exchange over the network. We propose a novel, column-oriented scheme (ColumnSGD) that partitions training data by columns rather than by rows. As a result, ML model can be partitioned by columns as well, leading to a distributed configuration where individual data and model partitions can be collocated on the same machine. Following this locality property, we develop a simple yet powerful computation framework that significantly reduces communication overheads and memory footprints compared to RowSGD, for large-scale ML models such as generalized linear models (GLMs) and factorization machines (FMs). We implement ColumnSGD on top of Apache Spark, and study its performance both analytically and experimentally. Experimental results on both public and real-world datasets show that ColumnSGD is up to 930× faster than MLlib, 63× faster than Petuum, and 14× faster than MXNet.

关键词： Conferences data engineering Radio frequency

来源：评论

学校读者我要写书评

暂无评论

TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training

arXiv

引用

arXiv 2023年

作者： Chen, Chang Li, Min Wu, Zhihua Yu, Dianhai Yang, Chao Center for Data Science Peking University China School of Mathematical Sciences Peking University China National Engineering Laboratory for Big Data Analysis and Applications Peking University China Baidu Inc China Institute for Computing and Digital Economy Peking University China

Sparsely gated Mixture-of-Expert (MoE) has demonstrated its effectiveness in scaling up deep neural networks to an extreme scale. Despite that numerous efforts have been made to improve the performance of MoE from the model design or system optimization perspective, existing MoE dispatch patterns are still not able to fully exploit the underlying heterogeneous network environments. In this paper, we propose TA-MoE, a topology-aware routing strategy for large-scale MoE trainging, from a model-system co-design perspective, which can dynamically adjust the MoE dispatch pattern according to the network topology. Based on communication modeling, we abstract the dispatch problem into an optimization objective and obtain the approximate dispatch pattern under different topologies. On top of that, we design a topology-aware auxiliary loss, which can adaptively route the data to fit in the underlying topology without sacrificing the model accuracy. Experiments show that TA-MoE can substantially outperform its counterparts on various hardware and model configurations, with roughly 1.01x-1.61x, 1.01x-4.77x, 1.25x-1.54x improvements over the popular DeepSpeed-MoE, FastMoE and FasterMoE systems. Copyright © 2023, The Authors. All rights reserved.

关键词： Network topology

来源：评论

学校读者我要写书评

暂无评论

An efficient aggregate signature scheme for healthcare wireless sensor networks

Journal of Network Intelligence

引用

Journal of Network Intelligence 2021年第1期6卷 1-15页

作者： Chen, Jian-Neng Zhou, Yu-Ping Huang, Zhen-Jie Wu, Tsu-Yang Zou, Fu-Min Tso, Raylin Minnan Normal University Zhangzhou363000 China Key Laboratory of Data Science and Intelligence Application Zhangzhou363000 China Lab of Granular Computing Zhangzhou363000 China College of Computer Science and Engineering Shandong University of Science and Technology Qingdao266590 China Fujian Key Laboratory of Automotive Electronics and Electric Drive Fujian Provincial Key Laboratory of Big Data Mining and Applications Fujian University of Technology National Demonstration Center for Experimental Electronic Information and Electrical Technology Education Fujian University of Technology Fuzhou350008 China Department of Computer Science National Chengchi University Taipei Taiwan

In healthcare wireless sensor networks, there are a large number of sensors, which need to transmit a lot of information in real time. The aggregate signature scheme combines a great deal of signatures signed by different signers on different messages into one short signature, which greatly improves the efficiency of data transmission. In this paper, we present a scheme of certificate based aggregate signature, and prove its security in the random oracle model. The performance analysis shows that the scheme is efficient in sign and verify phase, and suitable for healthcare wireless sensor networks. © 2021, Taiwan Ubiquitous Information. All rights reserved.

关键词： Wireless sensor networks

来源：评论

学校读者我要写书评

暂无评论

Accelerating Scalable Graph Neural Network Inference with Node-Adaptive Propagation

Accelerating Scalable Graph Neural Network Inference with No...

引用

International Conference on data engineering

作者： Xinyi Gao Wentao Zhang Junliang Yu Yingxia Shao Quoc Viet Hung Nguyen Bin Cui Hongzhi Yin The University of Queensland Brisbane Australia Peking University Beijing China National Engineering Laboratory for Big Data Analysis and Applications Beijing China Beijing University of Posts and Telecommunications Beijing China Griffith University Gold Coast Australia

ISBN: (数字)9798350317152

ISBN: (纸本)9798350317169

Graph neural networks (GNNs) have exhibited exceptional efficacy in a diverse array of applications. However, the sheer size of large-scale graphs presents a significant challenge to real-time inference with GNNs. Although existing Scalable GNNs leverage linear propagation to preprocess the features and accelerate the training and inference procedure, these methods still suffer from scalability issues when making inferences on unseen nodes, as the feature preprocessing requires the graph to be known and fixed. To further accelerate Scalable GNNs inference in this inductive setting, we propose an online propagation framework and two novel node-adaptive propagation methods that can customize the optimal propagation depth for each node based on its topological information and thereby avoid redundant feature propagation. The trade-off between accuracy and latency can be flexibly managed through simple hyper-parameters to accommodate various latency constraints. Moreover, to compensate for the inference accuracy loss caused by the potential early termination of propagation, we further propose Inception Distillation to exploit the multi-scale receptive field information within graphs. The rigorous and comprehensive experimental study on public datasets with varying scales and characteristics demonstrates that the proposed inference acceleration framework outperforms existing state-of-the-art graph inference acceleration methods in terms of accuracy and efficiency. Particularly, the superiority of our approach is notable on datasets with larger scales, yielding a $75\times$ inference speedup on the largest Ogbn-products dataset.

关键词： Training Accuracy Scalability Redundancy Propagation losses data engineering Real-time systems

来源：评论

学校读者我要写书评

暂无评论

Accelerating Scalable Graph Neural Network Inference with Node-Adaptive Propagation

arXiv

引用

arXiv 2023年

作者： Gao, Xinyi Zhang, Wentao Yu, Junliang Shao, Yingxia Nguyen, Quoc Viet Hung Cui, Bin Yin, Hongzhi The University of Queensland Brisbane Australia Peking University Beijing China National Engineering Laboratory for Big Data Analysis and Applications Beijing China Beijing University of Posts and Telecommunications Beijing China Griffith University Gold Coast Australia

Graph neural networks (GNNs) have exhibited exceptional efficacy in a diverse array of applications. However, the sheer size of large-scale graphs presents a significant challenge to real-time inference with GNNs. Although existing Scalable GNNs leverage linear propagation to preprocess the features and accelerate the training and inference procedure, these methods still suffer from scalability issues when making inferences on unseen nodes, as the feature preprocessing requires the graph to be known and fixed. To further accelerate Scalable GNNs inference in this inductive setting, we propose an online propagation framework and two novel node-adaptive propagation methods that can customize the optimal propagation depth for each node based on its topological information and thereby avoid redundant feature propagation. The trade-off between accuracy and latency can be flexibly managed through simple hyper-parameters to accommodate various latency constraints. Moreover, to compensate for the inference accuracy loss caused by the potential early termination of propagation, we further propose Inception Distillation to exploit the multi-scale receptive field information within graphs. The rigorous and comprehensive experimental study on public datasets with varying scales and characteristics demonstrates that the proposed inference acceleration framework outperforms existing state-of-the-art graph inference acceleration methods in terms of accuracy and efficiency. Particularly, the superiority of our approach is notable on datasets with larger scales, yielding a 75× inference speedup on the largest Ogbn-products dataset. Copyright © 2023, The Authors. All rights reserved.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Towards Dynamic and Safe Configuration Tuning for Cloud databases

arXiv

引用

arXiv 2022年

作者： Zhang, Xinyi Wu, Hong Li, Yang Tan, Jian Li, Feifei Cui, Bin School of CS Key Laboratory of High Confidence Software Technologies Peking University China Database and Storage Laboratory Damo Academy Alibaba Group China Center for Data Science Peking University National Engineering Laboratory for Big Data Analysis and Applications China China

Configuration knobs of database systems are essential to achieve high throughput and low latency. Recently, automatic tuning systems using machine learning methods (ML) have shown to find better configurations compared to experienced database administrators (DBAs). However, there are still gaps to apply the existing systems in production environments, especially in the cloud. First, they conduct tuning for a given workload within a limited time window and ignore the dynamicity of workloads and data. Second, they rely on a copied instance and do not consider the availability of the database when sampling configurations, making the tuning expensive, delayed, and unsafe. To fill these gaps, we propose OnlineTune, which tunes the online databases safely in changing cloud environments. To accommodate the dynamicity, OnlineTune embeds the environmental factors as context feature and adopts contextual Bayesian Optimization with context space partition to optimize the database adaptively and scalably. To pursue safety during tuning, we leverage the black-box and the white-box knowledge to evaluate the safety of configurations and propose a safe exploration strategy via subspace adaptation. We conduct evaluations on dynamic workloads from benchmarks and real-world workloads. Compared with the state-of-the-art methods, OnlineTune achieves 14.4%∼165.3% improvement on cumulative performance while reducing 91.0%∼99.5% unsafe configuration recommendations. Copyright © 2022, The Authors. All rights reserved.

关键词： database systems

来源：评论

学校读者我要写书评

暂无评论

ZOOMER: Boosting Retrieval on Web-scale Graphs by Regions of Interest

arXiv

引用

arXiv 2022年

作者： Jiang, Yuezihan Cheng, Yu Zhao, Hanyu Zhang, Wentao Miao, Xupeng He, Yu Wang, Liang Yang, Zhi Cui, Bin School of Software and Microelectronics Peking University China School of CS Key Laboratory of High Confidence Software Technologies Peking University China Center for Data Science Peking University National Engineering Laboratory for Big Data Analysis and Applications China Alibaba Group China

We introduce ZOOMER, a system deployed at Taobao, the largest e-commerce platform in China, for training and serving GNN-based recommendations over web-scale graphs. ZOOMER is designed for tackling two challenges presented by the massive user data at Taobao: low training/serving efficiency due to the huge scale of the graphs, and low recommendation quality due to the information overload which distracts the recommendation model from specific user intentions. ZOOMER achieves this by introducing a key concept, Region of Interests (ROI) in GNNs for recommendations, i.e., a neighborhood region in the graph with significant relevance to a strong user intention. ZOOMER narrows the focus from the whole graph and "zooms in" on the more relevant ROIs, thereby reducing the training/serving cost and mitigating the information overload at the same time. With carefully designed mechanisms, ZOOMER identifies the interest expressed by each recommendation request, constructs an ROI subgraph by sampling with respect to the interest, and guides the GNN to reweigh different parts of the ROI towards the interest by a multi-level attention module. Deployed as a large-scale distributed system, ZOOMER supports graphs with billions of nodes for training and thousands of requests per second for serving. ZOOMER achieves up to 14x speedup when downsizing sampling scales with comparable (even better) AUC performance than baseline methods. Besides, both the offline evaluation and online A/B test demonstrate the effectiveness of ZOOMER. Copyright © 2022, The Authors. All rights reserved.

关键词： Graphic methods

来源：评论

学校读者我要写书评

暂无评论

Enabling and Scaling the HPCG Benchmark on the Newest Generation Sunway Supercomputer with 42 Million Heterogeneous Cores

Enabling and Scaling the HPCG Benchmark on the Newest Genera...

引用

Supercomputing Conference

作者： Qianchao Zhu Hao Luo Chao Yang Mingshuo Ding Wanwang Yin Xinhui Yuan Center for Data Science Peking University Beijing China School of Mathematical Sciences Peking University Beijing China National Engineering Laboratory for Big Data Analysis and Applications Peking University Beijing China School of Electronic Engineering and Computer Science Peking University Beijing China National Research Center of Parallel Computer Engineering & Technology Beijing China

ISBN: (数字)9781450384421

ISBN: (纸本)9781665483902

We study and evaluate performance optimization techniques for the HPCG benchmark on the newest generation Sunway super-computer. Specifically, a two-level blocking scheme is proposed to expose adequate parallelism in the symmetric Gauss-Seidel kernel while keeping a fast convergence rate, a fine-grained kernel fusion technique is developed to alleviate the bandwidth load on local storage with small capacity, and a low overhead thread collaboration method is presented to efficiently move data between threads and hide its cost with data transfer operations. Test results show that the optimized HPCG code is able to exploit 73.0% of the theoretical memory bandwidth, and scale to over 42 million heterogeneous cores with 95.5% weak-scaling efficiency and 5.91 Pflop/s performance. We also study how the performance can be improved if the specific rules of HPCG are not fully obeyed, and design dependency preserving parallelization and vectorization methods, further boosting performance to 27.6 Pflop/ s.

关键词： Instruction sets High performance computing Memory management Bandwidth Benchmark testing Parallel processing data transfer

来源：评论

学校读者我要写书评

暂无评论

Real-Time Fine-Grained Air Quality Sensing Networks in Smart City: Design, Implementation and Optimization

arXiv

引用

arXiv 2018年

作者： Hu, Zhiwen Bai, Zixuan Bian, Kaigui Wang, Tao Song, Lingyang National Engineering Laboratory for Big Data Analysis and Applications School of Electronics Engineering and Computer Science Peking University Beijing China State Key Laboratory of Advanced Optical Communication Systems and Networks School of Electronics Engineering and Computer Science Peking University Beijing China

—Driven by the increasingly serious air pollution problem, the monitoring of air quality has gained much attention in both theoretical studies and practical implementations. In this paper, we present the architecture, implementation and optimization of our own air quality sensing system, which provides real-time and fine-grained air quality map of the monitored area. As the major component, the optimization problem of our system is studied in detail. Our objective is to minimize the average joint error of the established real-time air quality map, which involves data inference for the unmeasured data values. A deep Q-learning solution has been proposed for the power control problem to reasonably plan the sensing tasks of the power-limited sensing devices online. A genetic algorithm has been designed for the location selection problem to efficiently find the suitable locations to deploy limited number of sensing devices. The performance of the proposed solutions are evaluated by simulations, showing a significant performance gain when adopting both strategies. Copyright © 2018, The Authors. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Aonn-2: An Adjoint-Oriented Neural Network Method for Pde-Constrained Shape Optimization

SSRN

引用

SSRN 2023年

作者： Wang, Xili Yin, Pengfei Zhang, Bo Yang, Chao School of Mathematical Sciences Peking University Beijing100871 China National Engineering Laboratory for Big Data Analysis and Applications Peking University Beijing100871 China PKU-Changsha Institute for Computing and Digital Economy Hunan410006 China

Shape optimization has been playing an important role in a large variety of engineering applications. Existing shape optimization methods are generally mesh-dependent and therefore encounter challenges due to mesh deformation. To overcome this limitation, we present a new adjoint-oriented neural network method, AONN-2, for PDE-constrained shape optimization problems. This method extends the capabilities of the original AONN method [1], which is developed for efficiently solving parametric optimal control problems. AONN-2 inherits the direct-adjoint looping (DAL) framework for computing the extremum of an objective functional and the neural network methods for solving complicated PDEs from AONN. Furthermore, AONN-2 expands the application scope to shape optimization by taking advantage of the shape derivatives to optimize the shape represented by discrete boundary points. AONN-2 is a fully mesh-free shape optimization approach, naturally sidestepping issues related to mesh deformation, with no needs for maintaining mesh quality and additional mesh corrections. A series of experimental results are presented, highlighting the flexibility, robustness, and accuracy of AONN-2. © 2023, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：