检索结果-内蒙古大学图书馆

40th IEEE International Conference on data engineering, ICDE 2024

作者： Li, Xunkai Wu, Zhengyu Zhang, Wentao Sun, Henan Li, Rong-Hua Wang, Guoren Beijing Institute of Technology Beijing China Peking University China National Engineering Labratory for Big Data Analytics and Applications Beijing China

ISBN: (纸本)9798350317152

Recently, Federated Graph Learning (FGL) has attracted significant attention as a distributed framework based on graph neural networks, primarily due to its capability to break data silos. Existing FGL studies employ community split on the homophilous global graph by default to simulate federated semisupervised node classification settings. Such a strategy assumes the consistency of topology between the multi-client subgraphs and the global graph, where connected nodes are highly likely to possess similar feature distributions and the same label. However, in real-world implementations, the varying perspectives of local data engineering result in various subgraph topologies, posing unique heterogeneity challenges in FGL. Unlike the well-known label Non-independent identical distribution (Non-iid) problems in federated learning, FGL heterogeneity essentially reveals the topological divergence among multiple clients, namely homophily or heterophily. To simulate and handle this unique challenge, we introduce the concept of structure Non-iid split and then present a new paradigm called Adaptive Federated Graph Learning (AdaFGL), a decoupled two-step personalized approach. To begin with, AdaFGL employs standard multi-client federated collaborative training to acquire the federated knowledge extractor by aggregating uploaded models in the final round at the server. Then, each client conducts personalized training based on the local subgraph and the federated knowledge extractor. Extensive experiments on the 12 graph benchmark datasets validate the superior performance of AdaFGL over state-of-the-art baselines. Specifically, in terms of test accuracy, our proposed AdaFGL outperforms baselines by significant margins of 3.24 % and 5.57 % on community split and structure Non-iid split, respectively. © 2024 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

MOMENTUM BENEFITS NON-IID FEDERATED LEARNING SIMPLY AND PROVABLY 12

MOMENTUM BENEFITS NON-IID FEDERATED LEARNING SIMPLY AND PROV...

引用

12th International Conference on Learning Representations, ICLR 2024

作者： Cheng, Ziheng Huang, Xinmeng Wu, Pengfei Yuan, Kun Peking University China University of Pennsylvania United States National Engineering Labratory for Big Data Analytics and Applications AI for Science Institute Beijing China

Federated learning is a powerful paradigm for large-scale machine learning, but it faces significant challenges due to unreliable network connections, slow communication, and substantial data heterogeneity across clients. FEDAVG and SCAFFOLD are two prominent algorithms to address these challenges. In particular, FEDAVG employs multiple local updates before communicating with a central server, while SCAFFOLD maintains a control variable on each client to compensate for "client drift" in its local updates. Various methods have been proposed to enhance the convergence of these two algorithms, but they either make impractical adjustments to the algorithmic structure or rely on the assumption of bounded data heterogeneity. This paper explores the utilization of momentum to enhance the performance of FEDAVG and SCAFFOLD. When all clients participate in the training process, we demonstrate that incorporating momentum allows FEDAVG to converge without relying on the assumption of bounded data heterogeneity even using a constant local learning rate. This is novel and fairly surprising as existing analyses for FEDAVG require bounded data heterogeneity even with diminishing local learning rates. In partial client participation, we show that momentum enables SCAFFOLD to converge provably faster without imposing any additional assumptions. Furthermore, we use momentum to develop new variance-reduced extensions of FEDAVG and SCAFFOLD, which exhibit state-of-the-art convergence rates. Our experimental results support all theoretical findings. © 2024 12th International Conference on Learning Representations, ICLR 2024. All rights reserved.

关键词： Momentum

来源：评论

学校读者我要写书评

暂无评论

LightDiC: A Simple yet Effective Approach for Large-scale Digraph Representation Learning 50th

LightDiC: A Simple yet Effective Approach for Large-scale Di...

引用

50th International Conference on Very Large data Bases, VLDB 2024

作者： Li, Xunkai Liao, Meihao Wu, Zhengyu Su, Daohan Zhang, Wentao Li, Rong-Hua Wang, Guoren Beijing Institute of Technology China Peking University National Engineering Laboratory for Big Data Analytics and Applications China

Most existing graph neural networks (GNNs) are limited to undirected graphs, whose restricted scope of the captured relational information hinders their expressive capabilities and deployment. Compared with undirected graphs, directed graphs (digraphs) fit the demand for modeling more complex topological systems by capturing more intricate relationships between nodes. While some directed GNNs have been introduced, their inspiration mainly comes from deep learning architectures, which lead to redundant complexity and computation, making them inapplicable to large-scale databases. To address these issues, we propose LightDiC, a scalable variant of the digraph convolution based on the magnetic Laplacian. Since topology-related computations are conducted solely during offline pre-processing, LightDiC achieves exceptional scalability, enabling downstream predictions to be trained separately without incurring recursive computational costs. Theoretical analysis shows that LightDiC achieves message passing based on the complex field, which corresponds to the proximal gradient descent process of the Dirichlet energy optimization function from the perspective of digraph signal denoising, ensuring its expressiveness. Experimental results demonstrate that LightDiC performs comparably well or even outperforms other SOTA methods in various downstream tasks, with fewer learnable parameters and higher efficiency. © 2024, VLDB Endowment. All rights reserved.

关键词： Directed graphs

来源：评论

学校读者我要写书评

暂无评论

Rethinking Node-wise Propagation for Large-scale Graph Learning 24

Rethinking Node-wise Propagation for Large-scale Graph Learn...

引用

33rd ACM Web Conference, WWW 2024

作者： Li, Xunkai Ma, Jingyuan Wu, Zhengyu Su, Daohan Zhang, Wentao Li, Rong-Hua Wang, Guoren Beijing Institute of Technology Beijing China Peking University National Engineering Laboratory for Big Data Analytics and Applications Beijing China

ISBN: (纸本)9798400701719

Scalable graph neural networks (GNNs) have emerged as a promising technique, which exhibits superior predictive performance and high running efficiency across numerous large-scale graph-based web applications. However, (i) Most scalable GNNs tend to treat all nodes with the same propagation rules, neglecting their topological uniqueness;(ii) Existing node-wise propagation optimization strategies are insufficient on web-scale graphs with intricate topology, where a full portrayal of nodes' local properties is required. Intuitively, different nodes in web-scale graphs possess distinct topological roles, and therefore propagating them indiscriminately or neglecting local contexts may compromise the quality of node representations. To address the above issues, we propose Adaptive Topology-aware Propagation (ATP), which reduces potential high-bias propagation and extracts structural patterns of each node in a scalable manner to improve running efficiency and predictive performance. Remarkably, ATP is crafted to be a plug-and-play node-wise propagation optimization strategy, allowing for offline execution independent of the graph learning process in a new perspective. Therefore, this approach can be seamlessly integrated into most scalable GNNs while remaining orthogonal to existing node-wise propagation optimization strategies. Extensive experiments on 12 datasets have demonstrated the effectiveness of ATP. © 2024 ACM.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Distributed Bilevel Optimization with Communication Compression 41

Distributed Bilevel Optimization with Communication Compress...

引用

41st International Conference on Machine Learning, ICML 2024

作者： He, Yutong Hu, Jie Huang, Xinmeng Lu, Songtao Wang, Bin Yuan, Kun Peking University China University of Pennsylvania United States IBM Research United States Zhejiang University China National Engineering Labratory for Big Data Analytics and Applications China AI for Science Institute Beijing China

Stochastic bilevel optimization tackles challenges involving nested optimization structures. Its fast-growing scale nowadays necessitates efficient distributed algorithms. In conventional distributed bilevel methods, each worker must transmit full-dimensional stochastic gradients to the server every iteration, leading to significant communication overhead and thus hindering efficiency and scalability. To resolve this issue, we introduce the first family of distributed bilevel algorithms with communication compression. The primary challenge in algorithmic development is mitigating bias in hypergradient estimation caused by the nested structure. We first propose C-SOBA, a simple yet effective approach with unbiased compression and provable linear speedup convergence. However, it relies on strong assumptions on bounded gradients. To address this limitation, we explore the use of moving average, error feedback, and multi-step compression in bilevel optimization, resulting in a series of advanced algorithms with relaxed assumptions and improved convergence properties. Numerical experiments show that our compressed bilevel algorithms can achieve 10× reduction in communication overhead without severe performance degradation. Copyright 2024 by the author(s)

关键词： Stochastic systems

来源：评论

学校读者我要写书评

暂无评论

AdaFGL: A New Paradigm for Federated Node Classification with Topology Heterogeneity

AdaFGL: A New Paradigm for Federated Node Classification wit...

引用

International Conference on data engineering

作者： Xunkai Li Zhengyu Wu Wentao Zhang Henan Sun Rong-Hua Li Guoren Wang Beijing Institute of Technology Beijing China Peking University National Engineering Labratory for Big Data Analytics and Applications Beijing China

ISBN: (数字)9798350317152

ISBN: (纸本)9798350317169

关键词： Training Network topology Federated learning Collaboration Benchmark testing data engineering Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

AdaFGL: A New Paradigm for Federated Node Classification with Topology Heterogeneity

arXiv

引用

arXiv 2024年

Recently, Federated Graph Learning (FGL) has attracted significant attention as a distributed framework based on graph neural networks, primarily due to its capability to break data silos. Existing FGL studies employ community split on the homophilous global graph by default to simulate federated semi-supervised node classification settings. Such a strategy assumes the consistency of topology between the multi-client subgraphs and the global graph, where connected nodes are highly likely to possess similar feature distributions and the same label. However, in real-world implementations, the varying perspectives of local data engineering result in various subgraph topologies, posing unique heterogeneity challenges in FGL. Unlike the well-known label Non-independent identical distribution (Non-iid) problems in federated learning, FGL heterogeneity essentially reveals the topological divergence among multiple clients, namely homophily or heterophily. To simulate and handle this unique challenge, we introduce the concept of structure Non-iid split and then present a new paradigm called Adaptive Federated Graph Learning (AdaFGL), a decoupled two-step personalized approach. To begin with, AdaFGL employs standard multi-client federated collaborative training to acquire the federated knowledge extractor by aggregating uploaded models in the final round at the server. Then, each client conducts personalized training based on the local subgraph and the federated knowledge extractor. Extensive experiments on the 12 graph benchmark datasets validate the superior performance of AdaFGL over state-of-the-art baselines. Specifically, in terms of test accuracy, our proposed AdaFGL outperforms baselines by significant margins of 3.24% and 5.57% on community split and structure Non-iid split, respectively. Copyright © 2024, The Authors. All rights reserved.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

MOMENTUM BENEFITS NON-IID FEDERATED LEARNING SIMPLY AND PROVABLY

arXiv

引用

arXiv 2023年

Federated learning is a powerful paradigm for large-scale machine learning, but it faces significant challenges due to unreliable network connections, slow communication, and substantial data heterogeneity across clients. FEDAVG and SCAFFOLD are two prominent algorithms to address these challenges. In particular, FEDAVG employs multiple local updates before communicating with a central server, while SCAFFOLD maintains a control variable on each client to compensate for "client drift" in its local updates. Various methods have been proposed to enhance the convergence of these two algorithms, but they either make impractical adjustments to the algorithmic structure or rely on the assumption of bounded data heterogeneity. This paper explores the utilization of momentum to enhance the performance of FEDAVG and SCAFFOLD. When all clients participate in the training process, we demonstrate that incorporating momentum allows FEDAVG to converge without relying on the assumption of bounded data heterogeneity even using a constant local learning rate. This is novel and fairly surprising as existing analyses for FEDAVG require bounded data heterogeneity even with diminishing local learning rates. In partial client participation, we show that momentum enables SCAFFOLD to converge provably faster without imposing any additional assumptions. Furthermore, we use momentum to develop new variance-reduced extensions of FEDAVG and SCAFFOLD, which exhibit state-of-the-art convergence rates. Our experimental results support all theoretical findings. Copyright © 2023, The Authors. All rights reserved.

关键词： Momentum

来源：评论

学校读者我要写书评

暂无评论

BIM: Improving Graph Neural Networks with Balanced Influence Maximization 40

BIM: Improving Graph Neural Networks with Balanced Influence...

引用

40th IEEE International Conference on data engineering, ICDE 2024

作者： Zhang, Wentao Gao, Xinyi Yang, Ling Cao, Meng Huang, Ping Shan, Jiulong Yin, Hongzhi Cui, Bin Peking University Center for Machine Learning Research China Institute of Advanced Algorithms Research Shanghai China National Engineering Labratory for Big Data Analytics and Applications The University of Queensland Australia Peking University Key Lab of High Confidence Software Technologies China Apple Inc. Institute of Computational Social Science Peking University Qingdao China

ISBN: (纸本)9798350317152

The imbalanced data classification problem has aroused lots of concerns from both academia and industry since data imbalance is a widespread phenomenon in many real-world scenarios. Although this problem has been well researched from the view of imbalanced class samples, we further argue that graph neural networks (GNNs) expose a unique source of imbalance from the influenced nodes of different classes of labeled nodes, i.e., labeled nodes are imbalanced in terms of the number of nodes they influenced during the influence propagation in GNNs. To tackle this previously unexplored influence-imbalance issue, we connect social influence maximization with the imbalanced node classification problem and propose balanced influence maximization (BIM). Specifically, BIM greedily assigns the pseudo label to the node which can maximize the number of influenced nodes in GNN training while making the influence of each class more balance. Experimental results on five public datasets demonstrate the effectiveness of our method in relieving the influence-imbalance issue. For example, when training a GCN with an imbalance ratio of 0.1, BIM significantly outperforms the most competitive baseline by 0.6% -9.8% in five public datasets in terms of the F1 score. © 2024 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Distributed bilevel optimization with communication compression 24

Distributed bilevel optimization with communication compress...

引用

Proceedings of the 41st International Conference on Machine Learning

作者： Yutong He Jie Hu Xinmeng Huang Songtao Lu Bin Wang Kun Yuan Peking University University of Pennsylvania IBM Research Zhejiang University Peking University and National Engineering Labratory for Big Data Analytics and Applications and AI for Science Institute Beijing China

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：