检索结果-内蒙古大学图书馆

IEEE Annual Joint Conference: INFOCOM, IEEE Computer and Communications Societies

作者： Bingqian Du Jun Liu Ziyue Luo Chuan Wu Qiankun Zhang Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China Department of Electrical and Computer Engineering The Ohio State University USA Department of Computer Science The University of Hong Kong Hong Kong School of Cyber Science and Engineering Huazhong University of Science and Technology Wuhan China

ISBN: (数字)9798350383508

ISBN: (纸本)9798350383515

Feature-only partition of large graph data in distributed Graph Neural Network (GNN) training offers advantages over commonly adopted graph structure partition, such as minimal graph preprocessing cost and elimination of cross-worker subgraph sampling burdens. Nonetheless, performance bottleneck of GNN training with feature-only partitions still largely lies in the substantial communication overhead due to cross-worker feature fetching. To reduce the communication overhead and expedite distributed training, we first investigate and answer two key questions on convergence behaviors of GNN model in feature-partition based distribute GNN training: 1) As no worker holds a complete copy of each feature, can gradient exchange among workers compensate for the information loss due to incomplete local features? 2) If the answer to the first question is negative, is feature fetching in every training iteration of the GNN model necessary to ensure model convergence? Based on our theoretical findings on these questions, we derive an optimal communication plan that decides the frequency for feature fetching during the training process, taking into account bandwidth levels among workers and striking a balance between model loss and training time. Extensive evaluation demonstrates consistent results with our theoretical analysis, and the effectiveness of our proposed design.

关键词： Training Time-frequency analysis Costs Computational modeling Distributed databases Bandwidth Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Snapshot boosting: a fast ensemble framework for deep neural networks

引用

science China(Information sciences) 2020年第1期63卷 77-88页

作者： Wentao ZHANG Jiawei JIANG Yingxia SHAO Bin CUI Center for Data Science Peking University National Engineering Laboratory for Big Data Analysis and Applications Department of Computer Science Beijing Key Lab of Intelligent Telecommunications Software and Multimedia School of Computer ScienceBeijing University of Posts and Telecommunications Key Lab of High Confidence Software Technologies Department of Computer SciencePeking University

Boosting has been proven to be effective in improving the generalization of machine learning models in many fields. It is capable of getting high-diversity base learners and getting an accurate ensemble model by combining a sufficient number of weak learners. However, it is rarely used in deep learning due to the high training budget of the neural network. Another method named snapshot ensemble can significantly reduce the training budget, but it is hard to balance the tradeoff between training costs and diversity. Inspired by the ideas of snapshot ensemble and boosting, we propose a method named snapshot boosting. A series of operations are performed to get many base models with high diversity and accuracy, such as the use of the validation set, the boosting-based training framework, and the effective ensemble strategy. Last, we evaluate our method on the computer vision(CV) and the natural language processing(NLP) tasks, and the results show that snapshot boosting can get a more balanced trade-off between training expenses and ensemble accuracy than other well-known ensemble methods.

关键词： ensemble learning deep learning boosting neural network snapshot ensemble

来源：评论

学校读者我要写书评

暂无评论

PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs

arXiv

引用

arXiv 2023年

作者： Han, Jiuzhou Collier, Nigel Buntine, Wray Shareghi, Ehsan Department of Data Science & AI Monash University Australia College of Engineering and Computer Science VinUniversity Viet Nam Language Technology Lab University of Cambridge United Kingdom

Large language models (LLMs) have shown great abilities of solving various natural language tasks in different domains. Due to the training objective of LLMs and their pre-training data, LLMs are not very well equipped for tasks involving structured data generation. We propose a framework, Prompting with Iterative Verification (PiVe), to improve graph-based generative capability of LLMs. We show how a small language model could be trained to act as a verifier module for the output of an LLM (i.e., ChatGPT, GPT-4), and to iteratively improve its performance via fine-grained corrective instructions. We also show how the verifier module could apply iterative corrections offline for a more cost-effective solution to the text-to-graph generation task. Experiments on three graph-based datasets show consistent improvement gained via PiVe. Additionally, we create GenWiki-HIQ and highlight that the verifier module can be used as a data augmentation tool to help improve the quality of automatically generated parallel text-graph datasets.1 © 2023, CC BY.

关键词： Cost effectiveness

来源：评论

学校读者我要写书评

暂无评论

Hyperbolic Geometric Latent Diffusion Model for Graph Generation 41

Hyperbolic Geometric Latent Diffusion Model for Graph Genera...

引用

41st International Conference on Machine Learning, ICML 2024

作者： Fu, Xingcheng Gao, Yisen Wei, Yuecen Sun, Qingyun Peng, Hao Li, Jianxin Li, Xianxian Key Lab of Education Blockchain and Intelligent Technology Ministry of Education Guangxi Normal University Guilin China Institute of Artificial Intelligence Beihang University Beijing China School of Software Beihang University Beijing China Beijing Advanced Innovation Center for Big Data and Brain Computing School of Computer Science and Engineering Beihang University Beijing China

Diffusion models have made significant contributions to computer vision, sparking a growing interest in the community recently regarding the application of them to graph generation. Existing discrete graph diffusion models exhibit heightened computational complexity and diminished training efficiency. A preferable and natural way is to directly diffuse the graph within the latent space. However, due to the non-Euclidean structure of graphs is not isotropic in the latent space, the existing latent diffusion models effectively make it difficult to capture and preserve the topological information of graphs. To address the above challenges, we propose a novel geometrically latent diffusion framework HypDiff. Specifically, we first establish a geometrically latent space with interpretability measures based on hyperbolic geometry, to define anisotropic latent diffusion processes for graphs. Then, we propose a geometrically latent diffusion process that is constrained by both radial and angular geometric properties, thereby ensuring the preservation of the original topological properties in the generative graphs. Extensive experimental results demonstrate the superior effectiveness of HypDiff for graph generation with various topologies. Copyright 2024 by the author(s)

关键词： Graph theory

来源：评论

学校读者我要写书评

暂无评论

Utility-Based Route Choice Behavior Modeling Using Deep Sequential Models

引用

Journal of Big data Analytics in Transportation 2022年第2期4卷 119-133页

作者： Dong, Guimin Kweon, Yonghyeon Park, B. Brian Boukhechba, Mehdi Link Lab and Engineering Systems and Environment University of Virginia Charlottesville USA School of Data Science University of Virginia Charlottesville USA

GPS-based navigation systems have played crucial roles to improve transportation system performances. A limitation of such route guidance systems is that their route recommendation algorithms are mostly based on the shortest travel time and/or toll avoidance. To enhance the performance of route recommendation systems, it is critical to model drivers’ route choice behaviors, such that personalized route recommendations can be delivered to individual drivers or groups of drivers who share similar route choice behaviors. Previous studies have focused on game theory-based discrete choice models (e.g., Multinomial Logit) or shallow machine learning models (e.g., support vector machine) to generate the interpretations of route choice behavior by using the estimated parameters. Despite the good interpretability of these models, such interpretation could be often biased due to a small number of observations, restricted model complexity, and incapacity to capture the rich variations in route choice behaviors. In this paper, by leveraging both route level and route segment level features, we apply deep sequential models for the route data , and propose a hybrid sequence-to-one deep neural network model, named hybrid transformer-LSTM (HTL), to model the complex route choice behaviors. We utilize a large real-world route choice dataset to evaluate HTL and demonstrate its performance as compared to other state-of-the-art sequential neural network models. A model agnostic interpretability method, called SHAP, is applied to explain the route choice rationals. Finally, a benchmark clustering of route choice behavior is demonstrated to unveil the latent heterogeneous partitions in route selection decision making, based on the shapely values provided by SHAP.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Generating Action-conditioned Prompts for Open-vocabulary Video Action Recognition 24

Generating Action-conditioned Prompts for Open-vocabulary Vi...

引用

32nd ACM International Conference on Multimedia, MM 2024

作者： Jia, Chengyou Luo, Minnan Chang, Xiaojun Dang, Zhuohang Han, Mingfei Wang, Mengmeng Dai, Guang Dang, Sizhe Wang, Jingdong School of Computer Science and Technology MOEKLINNS Lab Xi'an Jiaotong University Shaanxi Xi'an China University of Science and Technology of China Anhui Hefei China School of Computer Science and Technology Xi'an Jiaotong University Shaanxi Xi'an China ReLER Lab AAII University of Technology Sydney SydneyNSW Australia Zhejiang University of Technology College of Computer Science and Technology China SGIT AI Lab State Grid Corporation of China Beijing China Baidu Inc Beijing China United Arab Emirates Shaanxi Province Key Laboratory of Big Data Knowledge Engineering Xi'an Jiaotong University Xi'an710049 China SGIT AI Lab State Grid Corporation of China China School of Computer Science and Technology Ministry of Education Key Laboratory of Intelligent Networks and Network Security Xi'an Jiaotong University Xi'an710049 China

ISBN: (纸本)9798400706868

Exploring open-vocabulary video action recognition is a promising venture, which aims to recognize previously unseen actions within any arbitrary set of categories. Existing methods typically adapt pretrained image-text models to the video domain, capitalizing on their inherent strengths in generalization. A common thread among such methods is the augmentation of visual embeddings with temporal information to improve the recognition of seen actions. Yet, they compromise with standard less-informative action descriptions, thus faltering when confronted with novel actions. Drawing inspiration from human cognitive processes, we argue that augmenting text embeddings with human prior knowledge is pivotal for open-vocabulary video action recognition. To realize this, we innovatively blend video models with Large Language Models (LLMs) to devise Action-conditioned Prompts. Specifically, we harness the knowledge in LLMs to produce a set of descriptive sentences that contain distinctive features for identifying given actions. Building upon this foundation, we further introduce a multi-modal action knowledge alignment mechanism to align concepts in video and textual knowledge encapsulated within the prompts. Extensive experiments on various video benchmarks, including zero-shot, few-shot, and base-to-novel generalization settings, demonstrate that our method not only sets new SOTA performance but also possesses excellent interpretability. © 2024 ACM.

关键词： Embeddings

来源：评论

学校读者我要写书评

暂无评论

BIM: Improving Graph Neural Networks with Balanced Influence Maximization 40

BIM: Improving Graph Neural Networks with Balanced Influence...

引用

40th IEEE International Conference on data engineering, ICDE 2024

作者： Zhang, Wentao Gao, Xinyi Yang, Ling Cao, Meng Huang, Ping Shan, Jiulong Yin, Hongzhi Cui, Bin Peking University Center for Machine Learning Research China Institute of Advanced Algorithms Research Shanghai China National Engineering Labratory for Big Data Analytics and Applications The University of Queensland Australia Peking University Key Lab of High Confidence Software Technologies China Apple Inc. Institute of Computational Social Science Peking University Qingdao China

ISBN: (纸本)9798350317152

The imbalanced data classification problem has aroused lots of concerns from both academia and industry since data imbalance is a widespread phenomenon in many real-world scenarios. Although this problem has been well researched from the view of imbalanced class samples, we further argue that graph neural networks (GNNs) expose a unique source of imbalance from the influenced nodes of different classes of labeled nodes, i.e., labeled nodes are imbalanced in terms of the number of nodes they influenced during the influence propagation in GNNs. To tackle this previously unexplored influence-imbalance issue, we connect social influence maximization with the imbalanced node classification problem and propose balanced influence maximization (BIM). Specifically, BIM greedily assigns the pseudo label to the node which can maximize the number of influenced nodes in GNN training while making the influence of each class more balance. Experimental results on five public datasets demonstrate the effectiveness of our method in relieving the influence-imbalance issue. For example, when training a GCN with an imbalance ratio of 0.1, BIM significantly outperforms the most competitive baseline by 0.6% -9.8% in five public datasets in terms of the F1 score. © 2024 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

DeepGATGO: A Hierarchical Pretraining-Based Graph-Attention Model for Automatic Protein Function Prediction

arXiv

引用

arXiv 2023年

作者： Li, Zihao Jiang, Changkun Li, Jianqiang College of Computer Science and Software Engineering Shenzhen University Shenzhen China National Engineering Lab for Big Data System Computing Technology Shenzhen University Shenzhen China

Automatic protein function prediction (AFP) is classified as a large-scale multi-label classification problem aimed at automating protein enrichment analysis to eliminate the current reliance on labor-intensive wet-lab methods. Currently, popular methods primarily combine protein-related information and Gene Ontology (GO) terms to generate final functional predictions. For example, protein sequences, structural information, and protein-protein interaction networks are integrated as prior knowledge to fuse with GO term embeddings and generate the ultimate prediction results. However, these methods are limited by the difficulty in obtaining structural information or network topology information, as well as the accuracy of such data. Therefore, more and more methods that only use protein sequences for protein function prediction have been proposed, which is a more reliable and computationally cheaper approach. However, the existing methods fail to fully extract feature information from protein sequences or label data because they do not adequately consider the intrinsic characteristics of the data itself. Therefore, we propose a sequence-based hierarchical prediction method, DeepGATGO, which processes protein sequences and GO term labels hierarchically, and utilizes graph attention networks (GATs) and contrastive learning for protein function prediction. Specifically, we compute embeddings of the sequence and label data using pre-trained models to reduce computational costs and improve the embedding accuracy. Then, we use GATs to dynamically extract the structural information of non-Euclidean data, and learn general features of the label dataset with contrastive learning by constructing positive and negative example samples. Experimental results demonstrate that our proposed model is no longer limited to specific biological populations and exhibits better scalability in GO term enrichment analysis on large-scale datasets. Copyright © 2023, The Authors. All rights reser

关键词： Gene Ontology

来源：评论

学校读者我要写书评

暂无评论

A Case Study on Context-Aware Neural Machine Translation with Multi-Task Learning

arXiv

引用

arXiv 2024年

作者： Appicharla, Ramakrishna Gain, Baban Pal, Santanu Ekbal, Asif Bhattacharyya, Pushpak Department of Computer Science and Engineering Indian Institute of Technology Patna India Wipro AI Lab45 London United Kingdom School of AI and Data Science Indian Institute of Technology Jodhpur India Department of Computer Science and Engineering Indian Institute of Technology Bombay India

In document-level neural machine translation (DocNMT), multi-encoder approaches are common in encoding context and source sentences. Recent studies (Li et al., 2020) have shown that the context encoder generates noise and makes the model robust to the choice of context. This paper further investigates this observation by explicitly modelling context encoding through multi-task learning (MTL) to make the model sensitive to the choice of context. We conduct experiments on cascade MTL architecture, which consists of one encoder and two decoders. Generation of the source from the context is considered an auxiliary task, and generation of the target from the source is the main task. We experimented with German–English language pairs on News, TED, and Europarl corpora. Evaluation results show that the proposed MTL approach performs better than concatenation-based and multi-encoder DocNMT models in low-resource settings and is sensitive to the choice of context. However, we observe that the MTL models are failing to generate the source from the context. These observations align with the previous studies, and this might suggest that the available document-level parallel corpora are not context-aware, and a robust sentence-level model can outperform the context-aware models. © 2024, CC0.

关键词： Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

FlexiFed: Personalized Federated Learning for Edge Clients with Heterogeneous Model Architectures 23

FlexiFed: Personalized Federated Learning for Edge Clients w...

引用

2023 World Wide Web Conference, WWW 2023

作者： Wang, Kaibin He, Qiang Chen, Feifei Chen, Chunyang Huang, Faliang Jin, Hai Yang, Yun School of Computer Science and Technology Huazhong University of Science and Technology China Department of Computing Technologies Swinburne University of Technology Australia School of Information Technology Deakin University Australia Faculty of Information Technology Monash University Australia Guangxi Key Lab of Human-machine Interaction and Intelligent Decision Nanning Normal University China National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab Huazhong University of Science and Technology Wuhan430074 China

ISBN: (纸本)9781450394161

Mobile and Web-of-Things (WoT) devices at the network edge account for more than half of the world's web traffic, making a great data source for various machine learning (ML) applications, particularly federated learning (FL) which offers a promising solution to privacy-preserving ML feeding on these data. FL allows edge mobile and WoT devices to train a shared global ML model under the orchestration of a central parameter server. In the real world, due to resource heterogeneity, these edge devices often train different versions of models (e.g., VGG-16 and VGG-19) or different ML models (e.g., VGG and ResNet) for the same ML task (e.g., computer vision and speech recognition). Existing FL schemes have assumed that participating edge devices share a common model architecture, and thus cannot facilitate FL across edge devices with heterogeneous ML model architectures. We explored this architecture heterogeneity challenge and found that FL can and should accommodate these edge devices to improve model accuracy and accelerate model training. This paper presents our findings and FlexiFed, a novel scheme for FL across edge devices with heterogeneous model architectures, and three model aggregation strategies for accommodating architecture heterogeneity under FlexiFed. Experiments with four widely-used ML models on four public datasets demonstrate 1) the usefulness of FlexiFed;and 2) that compared with the state-of-the-art FL scheme, FlexiFed improves model accuracy by 2.6%-9.7% and accelerates model convergence by 1.24 × -4.04 ×. © 2023 ACM.

关键词： Speech recognition

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：