检索结果-内蒙古大学图书馆

25th International Conference on Parallel and distributed Computing, Applications and Technologies, PDCAT 2024

作者： Zhang, Chunying Dong, Ziao Wang, Liya Liu, Lu Ren, Jing Ma, Jiang Liu, Bin College of Science North China University of Science and Technology 21 Bohai Road Caofeidian Xincheng Hebei Tangshan063210 China Big data and Social Computing Research Center Hebei University of Science and Technology Hebei Shijiazhuang0500198 China

ISBN: (纸本)9789819642069

Capsule network is a new type of neural network encoding features into capsules and constructing the part-whole relationships, which demonstrated good performance in image classification. However, it has some issues such as weak feature extraction ability and large number of training parameters, leading to performance decline when processing images with complex background. To address these issues, this article proposes a novel capsule network structure called STB-GraCapsNet, with key improvements as follows: 1) It uses Swin Transformer Block with the shift window attention mechanism for data preprocessing to obtain richer features;2) the features are then processed through Linear Embedding and passed into the Primary Capsule Layer to generate high-dimension feature capsules, which reduces primary capsules number and improves feature extraction ability of the network. Ablation and comparative experiments based on multiple datasets show that STB-GraCapsNet performs better on complex datasets: 77.8% and 36.2% reduction in parameters number of capsule structure, and 12.07% and 5.22% improvement in model accuracy, as compared to CapsNet and GraCapsNet, respectively. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Signal encoding

来源：评论

学校读者我要写书评

暂无评论

STRec: Social-Augmented Time-Aware Cross-Domain Sequential Recommendation 22

STRec: Social-Augmented Time-Aware Cross-Domain Sequential R...

引用

22nd IEEE International Symposium on Parallel and distributed processing with Applications, ISPA 2024

作者： Gao, Xiangyun Meng, Dan Sun, Xianlan Wang, Wei Zhu, Haibei Kong, Chao Anhui Polytechnic University School of Information and Computer Wuhu China Oppo Research Institute Shenzhen China Georgia Institute of Technology School of Electrical and Computer Engineering Atlanta United States

ISBN: (纸本)9798331509712

Cross-domain sequential recommendation leverages a user's historical interaction behavior across multiple domains to forecast the subsequent item with which they are likely to engage. Despite the efforts made by previous methods in representing users across domains, we believe they still have three inherent limitations: 1) Ignoring the importance of the relations between items among various domains, leading to suboptimal performance. 2) Most existing cross-domain sequential recommendation models are incapable of making full use of timestamp information, merely representing the sequence of user-item interaction sequence through positional embeddings. 3) Existing cross-domain recommendation methods consider only individual users, failing to recognize that social relations may influence user preferences. To overcome these problems, we propose an innovative approach called Social-Augmented Time-Aware Cross-Domain Sequential Recommendation (STRec). The method leverages the integration of multi-domain data with temporal information, further enriched by social influence, to afford a holistic understanding of user preferences. STRec consists of three main components: Initially, embedding modules are employed to convert timestamps and interaction information into embedding vectors respectively. Subsequently, graph neural network modules to uncover collaborative characteristics among inter-sequence items, while sequence attention encoders extract sequential features within item sequences and uncover the potential dependencies of temporal information in different contexts. Conclusively, a social aware attention network is designed to consider users' social neighbor information and capture the dynamic patterns of social relationships. Through extensive experimentation on real-world datasets across multiple scenarios, we have demonstrated that STRec surpasses several state-of-the-art approaches. © 2024 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Extracting Interpretable Hierarchical Rules from Deep neural networks' Latent Space 7th

Extracting Interpretable Hierarchical Rules from Deep Neural...

引用

7th International Joint Conference on Rules and Reasoning (RuleML+RR)

作者： Wang, Ya Paschke, Adrian Fraunhofer Inst Open Commun Syst Kaiserin Augusta Allee 31 D-10589 Berlin Germany Free Univ Berlin Kaiserswerther Str 16-18 D-14195 Berlin Germany

ISBN: (纸本)9783031450716;9783031450723

Deep neural networks, known for their superior learning capabilities, excel in identifying complex relationships between inputs and outputs, leveraging hierarchical, distributed data processing. Despite their impressive performance, these networks often resemble 'black boxes' due to their highly intricate internal structure and representation, raising challenges in terms of safety, ethical standards, and social norms. Decompositional rule extraction techniques have sought to address these issues by delving into the latent space and retrieving a broad set of symbolic rules. However, the interpretability of these rules is often hampered by their size and complexity. In this paper, we introduce EDICT (Extracting Deep Interpretable Concepts using Trees), a novel approach for rule extraction which employs a hierarchy of decision trees to mine concepts learned in a neural network, thereby generating highly interpretable rules. Evaluations across multiple datasets reveal that our method extracts rules with greater speed and interpretability compared to existing decompositional rule extraction techniques. Simultaneously, our approach demonstrates competitive performance in classification accuracy and model fidelity.

关键词： neural network interpretability Rule-based explanations Decompositional rule extraction

来源：评论

学校读者我要写书评

暂无评论

Large-Scale Machine Learning Cluster Scheduling via Multi-Agent Graph Reinforcement Learning

引用

IEEE TRANSACTIONS ON network AND SERVICE MANAGEMENT 2022年第4期19卷 4962-4974页

作者： Zhao, Xiaoyang Wu, Chuan Univ Hong Kong Dept Comp Sci Hong Kong Peoples R China

Efficient scheduling of distributed deep learning (DL) jobs in large GPU clusters is crucial for resource efficiency and job performance. While server sharing among jobs improves resource utilization, interference among co-located DL jobs occurs due to resource contention. Interference-aware job placement has been studied, with white-box approaches based on explicit interference modeling and black-box schedulers with reinforcement learning. In today's clusters containing thousands of GPU servers, running a single scheduler to manage all arrival jobs in a timely and effective manner is challenging, due to the large workload scale. We adopt multiple schedulers in a large-scale cluster/data center, and propose a multi-agent reinforcement learning (MARL) scheduling framework to cooperatively learn fine-grained job placement policies, towards the objective of minimizing job completion time (JCT). To achieve topology-aware placements, our proposed framework uses hierarchical graph neural networks to encode the data center topology and server architecture. In view of a common lack of precise reward samples corresponding to different placements, a job interference model is further devised to predict interference levels in face of various co-locations, for training of the MARL schedulers. Testbed and trace-driven evaluations show that our scheduler framework outperforms representative scheduling schemes by more than 20% in terms of average JCT, and is adaptive to various machine learning cluster topologies.

关键词： Servers Interference Training Graphics processing units Task analysis Topology Reinforcement learning AI for management cluster scheduling distributed machine learning multi-agent reinforcement learning graph neural network

来源：评论

学校读者我要写书评

暂无评论

NASFLY: On-Device Split Federated Learning with neural Architecture Search 22

NASFLY: On-Device Split Federated Learning with Neural Archi...

引用

22nd IEEE International Symposium on Parallel and distributed processing with Applications, ISPA 2024

作者： Huo, Chao Jia, Juncheng Deng, Tao Dong, Mianxiong Yu, Zhanwei Yuan, Di Soochow University School of Computer Science and Technology Suzhou215006 China Muroran Institute of Technology Muroran050-8585 Japan Uppsala University Department of Information Technology Sweden

ISBN: (纸本)9798331509712

The integration of Artificial Intelligence (AI) and Internet of Things (IoT) devices has given rise to IoAT, promising transformative applications across various domains. Federated Learning (FL) and Split Learning (SL) are pivotal in harnessing the potential of IoAT, enabling decentralized model training while preserving data privacy. However, the heterogeneity and scalability challenges in IoAT environments necessitate advanced frameworks. In this paper, we introduce an integration of block-wise neural Architecture Search (NAS) with a multi-partition SFL framework, called NASFLY. This approach uses only devices for actual model training and offers flexible scaling of model fragments to accommodate a wide range of device capabilities. In particular, NASFLY allows devices to utilize idle periods during the lengthy SFL forward and backward propagation phases. This is achieved by employing auxiliary model components dispatched from the server to conduct local supernet elastification using the device's local dataset. Our method alternates between SFL for backbone network optimization and the local supernet elastification within NAS, where knowledge from the backbone network is transferred to the local supernet branches using distillation techniques. We also propose a device clustering algorithm to further improve training efficiency. Our experimental results demonstrate that this methodology significantly enhances device utilization and improves training efficiency compared with the conventional SFL. © 2024 IEEE.

关键词： Federated learning

来源：评论

学校读者我要写书评

暂无评论

AtRec: Accelerating Recommendation Model Training on CPUs

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed SYSTEMS 2024年第6期35卷 750-763页

作者： Wang, Siqi Feng, Tianyu Yang, Hailong You, Xin Chen, Bangduo Liu, Tongxuan Luan, Zhongzhi Qian, Depei Beihang Univ State Key Lab Software Dev Environm Beijing 100191 Peoples R China Beihang Univ Sch Comp Sci & Engn Beijing 100191 Peoples R China Alibaba Corp Beijing 100020 Peoples R China Univ Sci & Technol China Hefei 230026 Peoples R China

The popularity of recommendation models and the enhanced AI processing capability of CPUs have provided massive performance opportunities to deliver satisfactory experiences to a large number of users. Unfortunately, existing recommendation model training methods fail to achieve high efficiency due to unique challenges such as dynamic shape and high parallelism. To address the above limitations, we comprehensively study the distinctive characteristics of recommendation models and discover several unexploited optimization opportunities. To exploit such opportunities, we propose AtRec, a high-performant recommendation model training engine that significantly accelerates the training process on CPUs. Specifically, AtRec presents comprehensive approach of training that employs operator-level and graph-level joint optimizations and runtime optimization. At the operator-level, AtRec identifies and optimizes the time-consuming operators, which enables further efficient graph-level optimizations. At the graph-level, AtRec conducts an in-depth analysis of the inefficiencies in several frequently used subgraphs, enables further performance improvement via eliminating redundant computations and memory accesses. In addition, to achieve better runtime performance, AtRec also identifies inefficiencies prevalent in the current scheduling and proposes runtime batching. The experiment results demonstrate that AtRec can significantly outperform state-of-the-art recommendation model training engines. We have open sourced the implementation and corresponding data of AtRec to boost research in this direction.

关键词： Training Computational modeling Optimization Runtime Recommender systems Engines Libraries Recommendation system deep neural network model training performance optimization

来源：评论

学校读者我要写书评

暂无评论

Theoretically Better and Numerically Faster distributed Optimization with Smoothness-Aware Quantization Techniques 36

Theoretically Better and Numerically Faster Distributed Opti...

引用

36th Conference on neural Information processing Systems (NeurIPS)

作者： Wang, Bokun Safaryan, Mher Richtarik, Peter Texas A&M Univ College Stn TX 77843 USA KAUST Thuwal Saudi Arabia

ISBN: (纸本)9781713871088

To address the high communication costs of distributed machine learning, a large body of work has been devoted in recent years to designing various compression strategies, such as sparsification and quantization, and optimization algorithms capable of using them. Recently, Safaryan et al. [2021] pioneered a dramatically different compression design approach: they first use the local training data to form local smoothness matrices and then propose to design a compressor capable of exploiting the smoothness information contained therein. While this novel approach leads to substantial savings in communication, it is limited to sparsification as it crucially depends on the linearity of the compression operator. In this work, we generalize their smoothness-aware compression strategy to arbitrary unbiased compression operators, which also include sparsification. Specializing our results to stochastic quantization, we guarantee significant savings in communication complexity compared to standard quantization. In particular, we prove that block quantization with n blocks theoretically outperforms single block quantization, leading to a reduction in communication complexity by an O(n) factor, where n is the number of nodes in the distributed system. Finally, we provide extensive numerical evidence with convex optimization problems that our smoothness-aware quantization strategies outperform existing quantization schemes as well as the aforementioned smoothness-aware sparsification strategies with respect to three evaluation metrics: the number of iterations, the total amount of bits communicated, and wall-clock time.

关键词： Convex optimization

来源：评论

学校读者我要写书评

暂无评论

Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex distributed Learning 36

Escaping Saddle Points with Bias-Variance Reduced Local Pert...

引用

36th Conference on neural Information processing Systems (NeurIPS)

作者： Murata, Tomoya Suzuki, Taiji NTT DATA Math Syst Inc Tokyo Japan Univ Tokyo Grad Sch Informat Sci & Technol Tokyo Japan RIKEN Ctr Adv Intelligence Project Tokyo Japan

ISBN: (纸本)9781713871088

In recent centralized nonconvex distributed learning and federated learning, local methods are one of the promising approaches to reduce communication time. However, existing work has mainly focused on studying first-order optimality guarantees. On the other side, second-order optimality guaranteed algorithms, i.e., algorithms escaping saddle points, have been extensively studied in the nondistributed optimization literature. In this paper, we study a new local algorithm called Bias-Variance Reduced Local Perturbed SGD (BVR-L-PSGD), that combines the existing bias-variance reduced gradient estimator with parameter perturbation to find second-order optimal points in centralized nonconvex distributed optimization. BVR-L-PSGD enjoys second-order optimality with nearly the same communication complexity as the best known one of BVR-L-SGD to find first-order optimality. Particularly, the communication complexity is better than non-local methods when the local datasets heterogeneity is smaller than the smoothness of the local loss. In an extreme case, the communication complexity approaches to (Theta) over tilde (1) when the local datasets heterogeneity goes to zero. Numerical results validate our theoretical findings.

关键词： Computational complexity

来源：评论

学校读者我要写书评

暂无评论

Application of a novel artificial neural network model in flood forecasting

引用

ENVIRONMENTAL MONITORING AND ASSESSMENT 2022年第2期194卷 1-13页

作者： Wang, Guangsheng Yang, Jianqing Hu, Yuzhong Li, Jingbing Yin, Zhijie Minist Water Resources Hydrol & Water Resources Monitoring & Forecasting Beijing 100053 Peoples R China Minist Water Resources Dept Hydrol Beijing 100053 Peoples R China Hydrol Bur Anhui Prov Hefei 230000 Anhui Peoples R China

In this paper, a novel ANN flood forecasting model is proposed. The ANN model is combined with traditional hydrological concepts and methods, taking the initial Antecedent Precipitation Index (API), rainfall, upstream inflow and initial flow at the forecast river section as input of model, and flood flow forecast of the next time steps as output of the model. The distributed rainfall is realized as the input of the model. The simulation is processed by dividing the watershed into several rainfall-runoff processing units. Two hidden layers are used in the ANN, and the topology of ANN is optimized by connecting the hidden layer neurons only with the input which has physical conceptual causes. The topological structure of the proposed ANN model and its information transmission process are more consistent with the physical conception of rainfall-runoff, and the weight parameters of the model are reduced. The arithmetic moving-average algorithm is added to the output of the model to simulate the pondage action of the watershed. Satisfactory results have been achieved in the Mozitan and Xianghongdian reservoirs in the upper reaches of Pi river in Huaihe Basin, and the Fengman reservoir in the upper reach of Second Songhua river in Songhua basin in China.

关键词： Flood forecasting Artificial neural network Multi-hidden layer Topology of ANN API model distributed rainfall

来源：评论

学校读者我要写书评

暂无评论

RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent neural networks 36

RNNs of RNNs: Recursive Construction of Stable Assemblies of...

引用

36th Conference on neural Information processing Systems (NeurIPS)

作者： Kozachkov, Leo Ennis, Michaela Slotine, Jean-Jacques MIT Dept Brain & Cognit Sci Cambridge MA 02139 USA Harvard Univ Div Med Sci Cambridge MA 02138 USA MIT Dept Mech Engn Cambridge MA 02139 USA Google AI Mountain View CA USA

ISBN: (纸本)9781713871088

Recurrent neural networks (RNNs) are widely used throughout neuroscience as models of local neural activity. Many properties of single RNNs are well characterized theoretically, but experimental neuroscience has moved in the direction of studying multiple interacting areas, and RNN theory needs to be likewise extended. We take a constructive approach towards this problem, leveraging tools from nonlinear control theory and machine learning to characterize when combinations of stable RNNs will themselves be stable. Importantly, we derive conditions which allow for massive feedback connections between interacting RNNs. We parameterize these conditions for easy optimization using gradient-based techniques, and show that stability-constrained 'networks of networks' can perform well on challenging sequential-processing benchmark tasks. Altogether, our results provide a principled approach towards understanding distributed, modular function in the brain.

关键词： Recurrent neural networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：