检索结果-内蒙古大学图书馆

Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Ji Jia, Juncheng Zhang, Hong Yun, Yuhui Wang, Leye Zhou, Yang Dai, Huaiyu Dou, Dejing Hithink RoyalFlush Information Network Co. Ltd. Hangzhou China School of Computer Science and Technology Soochow University Collaborative Innovation Center of Novel Software Technology and Industrialization Suzhou China School of Computer Science and Technology Soochow University Suzhou China Baidu Research Beijing China Key Lab of High Confidence Software Technologies Ministry of Education Software Institute Peking University Beijing China Department of Computer Science and Software Engineering Auburn University Auburn United States Department of Electrical and Computer Engineering North Carolina State University NC United States BEDI Cloud and School of Computer Science Fudan University Beijing China

Despite achieving remarkable performance, Federated Learning (FL) encounters two important problems, i.e., low training efficiency and limited computational resources. In this paper, we propose a new FL framework, i.e., FedDUMAP, with three original contributions, to leverage the shared insensitive data on the server in addition to the distributed data in edge devices so as to efficiently train a global model. First, we propose a simple dynamic server update algorithm, which takes advantage of the shared insensitive data on the server while dynamically adjusting the update steps on the server in order to speed up the convergence and improve the accuracy. Second, we propose an adaptive optimization method with the dynamic server update algorithm to exploit the global momentum on the server and each local device for superior accuracy. Third, we develop a layer-adaptive model pruning method to carry out specific pruning operations, which is adapted to the diverse features of each layer so as to attain an excellent trade-off between effectiveness and efficiency. Our proposed FL model, FedDUMAP, combines the three original techniques and has a significantly better performance compared with baseline approaches in terms of efficiency (up to 16.9 times faster), accuracy (up to 20.4% higher), and computational cost (up to 62.6% smaller). © 2024, CC BY.

关键词： Federated learning

Enabling Real World Scale Structural Superlubricity All-Atom Simulation on the Next-Generation Sunway Supercomputer 23

学校读者我要写书评

暂无评论

Enabling Real World Scale Structural Superlubricity All-Atom...

2023 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023

作者： Duan, Xiaohui Wang, Jin Gao, Ping Ma, Ming Gan, Lin Liu, Xin Fu, Haohuan Xue, Wei Chen, Dexun Yang, Guangwen Liu, Weiguo School of Software Shandong University China International School for Advanced Studies Trieste Italy Tsinghua University Department of Computer Science and Technology Beijing China Department of Engineering Mechanics Center for Nano and Micro Mechanics Tsinghua University Beijing China National Supercomputing Center in Wuxi Wuxi China Ministry of Education Key Lab for Earth System Modeling Department of Earth System Science Tsinghua University Beijing China

ISBN: (纸本)9798400701092

Molecular dynamics (MD) simulation can provide an affordable way for inspecting microscopic phenomena, which is a powerful complement to real-world experiments. But the spatial scale of MD simulations is usually magnitudes smaller than experiment systems. In this paper, we present our work, redesigning the widely used inter-layer potential in structural superlubricity. By carrying out a specialized neighbor list for inter-layer potential computation, the total memory access amount is reduced significantly. Besides, a simple but efficient vectorization strategy is implemented based on the new neighbor list. In the extreme case, our work can scale to 38 million cores to achieve a sustainable performance of 61 PFLOPS, enabling a simulation of a superlubricity system of 32 μm2 with 7.2 billion atoms at 4.75 ns/day, which is 11,834 times of reported largest scale simulation in superlubricity systems in contact area and almost ten times faster in time-to-solution. Furthermore, we have done a simulation at 9 μm2 which results in consistency with real-world experiments and verified some theoretical predictions in the mesoscopic scale. © 2023 ACM.

关键词： Molecular dynamics

Service offloading oriented edge server placement in smart farming

学校读者我要写书评

暂无评论

Service offloading oriented edge server placement in smart f...

作者： Zhang, Jie Li, Xiang Zhang, Xuyun Xue, Yuan Srivastava, Gautam Dou, Wanchun State Key Laboratory for Novel Software Technology Nanjing University Nanjing China School of Computer and Software Nanjing University of Information Science and Technology Nanjing China Department of Computing Macquarie University SydneyNSW Australia Department of Mathematics and Computer Science Brandon University BrandonMB Canada

Currently, smart farming has been established to realize agriculture automation by leveraging sensors to gather the growth and environmental data for crops, and realizing multiple intelligent controls, such as irrigation, fertilization, and so on, to increase the crop yields. To support real-time intelligent controls, edge computing is introduced to smart farming by endowing computing and storage capacities to edge devices nearby the geographically distributed sensors. However, the farmers are relatively willing to purchase and deploy a small quantity of edge servers (ESs) in the farm from the perspective of expenditure saving, thereby leading to a key challenge to guarantee the performance of the real-time controls and the overall edge services. In view of this challenge, a service offloading-oriented ES placement method for supporting smart farming, called SOP, is proposed to optimize the data transmission delay from sensors to ESs and the load balance among ESs. More precisely, the corresponding service range of a certain ES is ascertained according to the specific analysis of the farming service requirements. Subsequently, the layout policies for the trade-offs of the ES performance and service efficiency are acquired. Then the most balanced policy is determined as the final ES placement strategy. Eventually, we evaluate the performance of the whole ES system and the service execution efficiency with SOP. © 2020 John Wiley & Sons, Ltd.

关键词： Edge computing

Validating SMT Solvers via Skeleton Enumeration Empowered by Historical Bug-Triggering Inputs

学校读者我要写书评

暂无评论

Validating SMT Solvers via Skeleton Enumeration Empowered by...

International Conference on software Engineering (ICSE)

作者： Maolin Sun Yibiao Yang Ming Wen Yongcong Wang Yuming Zhou Hai Jin School of Cyber Science and Engineering Huazhong University of Science and Technology China Hubei Key Laboratory of Distributed System Security Services Computing Technology and System Lab Cluster and Grid Computing Lab Hubei Engineering Research Center on Big Data Security National Engineering Research Center for Big Data Technology and System State Key Laboratory for Novel Software Technology Nanjing University China School of Computer Science and Technology Huazhong University of Science and Technology China

SMT solvers check the satisfiability of logic formulas over first-order theories, which have been utilized in a rich number of critical applications, such as software verification, test case generation, and program synthesis. Bugs hidden in SMT solvers would severely mislead those applications and further cause severe consequences. Therefore, ensuring the reliability and robustness of SMT solvers is of critical importance. Although many approaches have been proposed to test SMT solvers, it is still a challenge to discover bugs effectively. To tackle such a challenge, we conduct an empirical study on the historical bug-triggering formulas in SMT solvers' bug tracking systems. We observe that the historical bug-triggering formulas contain valuable skeletons (i.e., core structures of formulas) as well as associated atomic formulas which can cast significant impacts on formulas' ability in triggering bugs. Therefore, we propose a novel approach that utilizes the skeletons extracted from the historical bug-triggering formulas and enumerates atomic formulas under the guidance of association rules derived from historical formulas. In this study, we realized our approach as a practical fuzzing tool HistFuzz and conducted extensive testing on the well-known SMT solvers Z3 and cvc5. To date, HistFuzz has found 111 confirmed new bugs for Z3 and cvc5, of which 108 have been fixed by the developers. More notably, out of the confirmed bugs, 23 are soundness bugs and invalid model bugs found in the solvers' default mode, which are essential for SMT solvers. In addition, our experiments also demonstrate that HistFuzz outperforms the state-of-the-art SMT solver fuzzers in terms of achieved code coverage and effectiveness.

关键词：

A Sensor Network-Based Data Stream Clustering Algorithm for Pervasive Computing

学校读者我要写书评

暂无评论

Chinese Journal of Electronics 2023年第2期18卷 255-258页

作者： Ning Ye Ruchuan Wang Institute of Computer Science Nanjing University of Posts and Telecommunications Nanjing China Department of Information Science Nanjing College for Population Programme Management Nanjing China State Key Laboratory for Novel Software Technology Nanjing University Nanjing China

Pervasive computing is characterized by the integration with communication and digital media technology embedded to the people's living space. People can transparently access the digital service anywhere. Wireless sensor networks are a novel technology and have broad application prospects. With the maturity of the wireless sensor networks technology, pervasive computing is becoming a reality. It is become a new technology challenge to process the data streams of sensor networks for pervasive environment efficiently and to find useful knowledge in these data streams. A k-means data stream clustering algorithm based on sensor networks is presented. The main idea of this algorithm is to select the initial centroids according to the aggregation gain of the node, then to cluster the data stream using the average square error. The experimental results are showed that this algorithm is effective and efficient.

关键词： Pervasive computing Performance evaluation Knowledge engineering Wireless sensor networks Sociology Clustering algorithms Media

A Simple Geometric-Aware Indoor Positioning Interpolation Algorithm Based on Manifold Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yang, Suorong Zhang, Geng Zhao, Jian Shen, Furao State Key Laboratory for Novel Software Technology Nanjing University China Department of Computer Science and Technology Nanjing University China School of Artificial Intelligence Nanjing University China School of Electronic Science and Engineering Nanjing University China

Interpolation methodologies have been widely used within the domain of indoor positioning systems. However, existing indoor positioning interpolation algorithms exhibit several inherent limitations, including reliance on complex mathematical models, limited flexibility, and relatively low precision. To enhance the accuracy and efficiency of indoor positioning interpolation techniques, this paper proposes a simple yet powerful geometric-aware interpolation algorithm for indoor positioning tasks. The key to our algorithm is to exploit the geometric attributes of the local topological manifold using manifold learning principles. Therefore, instead of constructing complicated mathematical models, the proposed algorithm facilitates the more precise and efficient estimation of points grounded in the local topological manifold. Moreover, our proposed method can be effortlessly integrated into any indoor positioning system, thereby bolstering its adaptability. Through a systematic array of experiments and comprehensive performance analyses conducted on both simulated and real-world datasets, we demonstrate that the proposed algorithm consistently outperforms the most commonly used and representative interpolation approaches regarding interpolation accuracy and efficiency. Furthermore, the experimental results also underscore the substantial practical utility of our method and its potential applicability in real-time indoor positioning scenarios. Copyright © 2023, The Authors. All rights reserved.

关键词： Interpolation

An exploratory study on automatic identification of assumptions in the development of deep learning frameworks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Chen Liang, Peng Ma, Zinan School of Artificial Intelligence Shenzhen Polytechnic University Shenzhen518055 China State Key Laboratory for Novel Software Technology Nanjing University Nanjing China School of Computer Science Wuhan University Wuhan430072 China Department of Computer Science and Engineering Chalmers University of Technology GothenburgS-412 96 Sweden

Context: Stakeholders constantly make assumptions in the development of deep learning (DL) frameworks. These assumptions are related to various types of software artifacts (e.g., requirements, design decisions, and technical debt) and can turn out to be invalid, leading to system failures. Existing approaches and tools for assumption management usually depend on manual identification of assumptions. However, assumptions are scattered in various sources (e.g., code comments, commits, pull requests, and issues) of DL framework development, and manually identifying assumptions has high costs (e.g., time and resources). Objective: The objective of the study is to evaluate different classification models for the purpose of identification with respect to assumptions from the point of view of developers and users in the context of DL framework projects (i.e., issues, pull requests, and commits) on GitHub. Method: First, we constructed a new and largest dataset (i.e., the AssuEval dataset) of assumptions collected from the TensorFlow and Keras repositories on GitHub. Then we explored the performance of seven non-transformers based models (e.g., Support Vector Machine, Classification and Regression Trees), the ALBERT model, and three decoder-only models (i.e., ChatGPT, Claude, and Gemini) for identifying assumptions on the AssuEval dataset. Results: The study results show that ALBERT achieves the best performance (f1-score: 0.9584) for identifying assumptions on the AssuEval dataset, which is much better than the other models (the 2nd best f1-score is 0.8858, achieved by the Claude 3.5 Sonnet model). Though ChatGPT, Claude, and Gemini are popular models, we do not recommend using them to identify assumptions in DL framework development because of their low performance. Fine-tuning ChatGPT, Claude, Gemini, or other language models (e.g., Llama3, Falcon, and BLOOM) specifically for assumptions might improve their performance for assumption identification. Conclusions: This stu

关键词： Chatbots

Stochastic Approximation Approaches to Group Distributionally Robust Optimization and Beyond

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Lijun Bai, Haomin Zhao, Peng Yang, Tianbao Zhou, Zhi-Hua National Key Laboratory for Novel Software Technology Nanjing University China School of Artificial Intelligence Nanjing University China Department of Computer Science and Engineering Texas A&M University College Station United States

This paper investigates group distributionally robust optimization (GDRO) with the goal of learning a model that performs well over m different distributions. First, we formulate GDRO as a stochastic convex-concave saddle-point problem, which is then solved by stochastic mirror descent (SMD) with m samples in each iteration, and attain a nearly optimal sample complexity. To reduce the number of samples required in each round from m to 1, we cast GDRO as a two-player game, where one player conducts SMD and the other executes an online algorithm for non-oblivious multi-armed bandits, maintaining the same sample complexity. Next, we extend GDRO to address scenarios involving imbalanced data and heterogeneous distributions. In the first scenario, we introduce a weighted variant of GDRO, enabling distribution-dependent convergence rates that rely on the number of samples from each distribution. We design two strategies to meet the sample budget: one integrates non-uniform sampling into SMD, and the other employs the stochastic mirror-prox algorithm with mini-batches, both of which deliver faster rates for distributions with more samples. In the second scenario, we propose to optimize the average top-k risk instead of the maximum risk, thereby mitigating the impact of outlier distributions. Similar to the case of vanilla GDRO, we develop two stochastic approaches: one uses m samples per iteration via SMD, and the other consumes k samples per iteration through an online algorithm for non-oblivious combinatorial semi-bandits. Copyright © 2023, The Authors. All rights reserved.

关键词： Budget control

RADAP: A Robust and Adaptive Defense Against Diverse Adversarial Patches on Face Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Xiaoliang Shen, Furao Zhao, Jian Nie, Changhai State Key Laboratory for Novel Software Technology Nanjing University China Department of Computer Science and Technology Nanjing University China School of Artificial Intelligence Nanjing University China School of Electronic Science and Engineering Nanjing University China

Face recognition (FR) systems powered by deep learning have become widely used in various applications. However, they are vulnerable to adversarial attacks, especially those based on local adversarial patches that can be physically applied to real-world objects. In this paper, we propose RADAP, a robust and adaptive defense mechanism against diverse adversarial patches in both closed-set and open-set FR systems. RADAP employs innovative techniques, such as FCutout and F-patch, which use Fourier space sampling masks to improve the occlusion robustness of the FR model and the performance of the patch segmenter. Moreover, we introduce an edge-aware binary cross-entropy (EBCE) loss function to enhance the accuracy of patch detection. We also present the split and fill (SAF) strategy, which is designed to counter the vulnerability of the patch segmenter to complete white-box adaptive attacks. We conduct comprehensive experiments to validate the effectiveness of RADAP, which shows significant improvements in defense performance against various adversarial patches, while maintaining clean accuracy higher than that of the undefended Vanilla model. Copyright © 2023, The Authors. All rights reserved.

关键词： Face recognition