检索结果-内蒙古大学图书馆

funcX: Federated Function as a Service for science

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Zhuozhao Chard, Ryan Babuji, Yadu Galewsky, Ben Skluzacek, Tyler J. Nagaitsev, Kirill Woodard, Anna Blaiszik, Ben Bryan, Josh Katz, Daniel S. Foster, Ian T. Chard, Kyle Department of Computer Science and Engineering Research Institute of Trustworthy Autonomous Systems Southern University of Science and Technology Shenzhen China The University of Chicago ChicagoIL60637 United States Argonne National Laboratory LemontIL60439 United States NCSA CS ECE and the iSchool University of Illinois UrbanaIL61801 United States NCSA University of Illinois UrbanaIL61801 United States Northwestern University EvanstonIL60208 United States

funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX decouples the cloud-hosted management functionality from the edge-hosted execution functionality. funcX’s endpoint software can be deployed, by users or administrators, on arbitrary laptops, clouds, clusters, and supercomputers, in effect turning them into function serving systems. funcX’s cloud-hosted service provides a single location for registering, sharing, and managing both functions and endpoints. It allows for transparent, secure, and reliable function execution across the federated ecosystem of endpoints—enabling users to route functions to endpoints based on specific needs. funcX uses containers (e.g., Docker, Singularity, and Shifter) to provide common execution environments across endpoints. funcX implements various container management strategies to execute functions with high performance and efficiency on diverse funcX endpoints. funcX also integrates with an in-memory data store and Globus for managing data that may span endpoints. We motivate the need for funcX, present our prototype design and implementation, and demonstrate, via experiments on two supercomputers, that funcX can scale to more than 130 000 concurrent workers. We show that funcX’s container warming-aware routing algorithm can reduce the completion time for 3000 functions by up to 61% compared to a randomized algorithm and the in-memory data store can speed up data transfers by up to 3x compared to a shared file system. Copyright © 2022, The Authors. All rights reserved.

关键词： Supercomputers

Robust multimodal federated learning for non-IID multimodal data with incompleteness

学校读者我要写书评

暂无评论

Future Generation computer systems 2026年 174卷

作者： Songcan Yu Kaiming Zhu Feiyuan Liang Junbo Wang Krishna Kant Ling Yin Department of Computer Science and Technology Sichuan Police College Luzhou 646000 PR China Police Integration Computing Key Laboratory of Sichuan Province Chengdu 610206 PR China School of Intelligent Systems Engineering Sun Yat-Sen University Shenzhen 518107 PR China Department of Computer and Information Sciences Temple University Philadelphia USA Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen 518055 PR China

Federated learning (FL) provides a privacy-aware distributed learning framework using private data belonging to multiple users (or clients). Often the data is generated continuously by the sensors deployed by different clients and is increasingly multimodal. However, the data is invariably imbalanced across different modalities, and certain modalities may even be missing from some clients for certain periods due to sensor failure or unavailability. Furthermore, the available data across clients may not be identically distributed (non-IID). Non-IID data is already known to be a problem with FL, and incomplete modalities could worsen the results even further. Effectively fusing multimodal data with incompleteness and training a high-accuracy global model is still a challenging problem. To tackle this problem, we propose a novel framework called MFedNI that extracts cross-modal temporal features and adaptively adjusts the weights of each modality to mitigate the effect of modal incompleteness. The prototype contrastive learning (PCL)-based feature alignment method further improves the consistency of feature representations across clients. We evaluated MFedNI using two multimodal datasets, HAR and DEAP with different levels of incompleteness. The results demonstrate that MFedNI outperforms other approaches and can train highly accurate models on datasets comprising non-IID multimodal data with incompleteness.

关键词：

Improving unsupervised domain adaptation by reducing bi-level feature redundancy

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Mengzhu Zhang, Xiang Lan, Long Wang, Wei Tan, Huibin Luo, Zhigang Science and Technology on Parallel and Distributed Laboratory College of Computer National University of Defense Technology Changsha China Institute for Quantum State Key Laboratory of High Performance Computing National University of Defense Technology Changsha China DUT-RU International School of Information Science & Engineering Dalian University of Technology DalianLiaoning116000 China Department of Science and Technology on Parallel and Distributed Processing National University of Defense Technology Changsha China

Reducing feature redundancy has shown beneficial effects for improving the accuracy of deep learning models, thus it is also indispensable for the models of unsupervised domain adaptation (UDA). Nevertheless, most recent efforts in the field of UDA ignore this point. Moreover, main schemes realizing this in general independent of UDA purely involve a single domain, thus might not be effective for cross-domain tasks. In this paper, we emphasize the significance of reducing feature redundancy for improving UDA in a bi-level way. For the first level, we try to ensure compact domain-specific features with a transferable decorrelated normalization module, which preserves specific domain information whilst easing the side effect of feature redundancy on the sequel domain-invariance. In the second level, domain-invariant feature redundancy caused by domain-shared representation is further mitigated via an alternative brand orthogonality for better generalization. These two novel aspects can be easily plugged into any BN-based backbone neural networks. Specifically, simply applying them to ResNet50 has achieved competitive performance to the state-of-the-arts on five popular benchmarks. Our code will be available at https://***/dreamkily/gUDA. © 2020, CC BY.

关键词： Deep learning

Thermosim: Deep learning based framework for modeling and simulation of thermal-aware resource management for cloud computing environments

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Gill, Sukhpal Singh Tuli, Shreshth Toosi, Adel Nadjaran Cuadrado, Felix Garraghan, Peter Bahsoon, Rami Lutfiyya, Hanan Sakellariou, Rizos Rana, Omer Dustdar, Schahram Buyya, Rajkumar School of Electronic Engineering and Computer Science Queen Mary University of London United Kingdom Delhi India Faculty of Information Technology Monash University ClaytonVIC Australia Spain School of Computing and Communications Lancaster University United Kingdom School of Computer Science University of Birmingham Birmingham United Kingdom Department of Computer Science University of Western Ontario LondonON Canada School of Computer Science University of Manchester Oxford Road ManchesterUK United Kingdom School of Computer Science and Informatics Cardiff University CardiffUK United Kingdom Distributed Systems Group Vienna University of Technology Vienna Austria Laboratory School of Computing and Information Systems University of Melbourne Australia

Current cloud computing frameworks host millions of physical servers that utilize cloud computing resources in the form of different virtual machines. Cloud Data Center (CDC) infrastructures require significant amounts of energy to deliver large scale computational services. Moreover, computing nodes generate large volumes of heat, requiring cooling units in turn to eliminate the effect of this heat. Thus, overall energy consumption of the CDC increases tremendously for servers as well as for cooling units. However, current workload allocation policies do not take into account effect on temperature and it is challenging to simulate the thermal behaviour of CDCs. There is a need for a thermal-aware framework to simulate and model the behaviour of nodes and measure the important performance parameters which can be affected by its temperature. In this paper, we propose a lightweight framework, ThermoSim, for modelling and simulation of thermal-aware resource management for cloud computing environments. This work presents a Recurrent Neural Network based deep learning temperature predictor for CDCs which is utilized by ThermoSim for lightweight resource management in constrained cloud environments. ThermoSim extends the CloudSim toolkit helping to analyse the performance of various key parameters such as energy consumption, service level agreement violation rate, number of virtual machine migrations and temperature during the management of cloud resources for execution of workloads. Further, different energy-aware and thermal-aware resource management techniques are tested using the proposed ThermoSim framework in order to validate it against the existing framework (Thas). The experimental results demonstrate the proposed framework is capable of modelling and simulating the thermal behaviour of a CDC and ThermoSim framework is better than Thas in terms of energy consumption, cost, time, memory usage and prediction accuracy. Copyright © 2020, The Authors. All rights rese

关键词： Cloud computing

Interdependence between transportation system and power distribution system: a comprehensive review on models and applications

学校读者我要写书评

暂无评论

Journal of Modern Power systems and Clean Energy 2019年第3期7卷 433-448页

作者： Wei WEI Danman WU Qiuwei WU Miadreza SHAFIE-KHAH Joao PSCATALAO State Key Laboratory of Power System Department ofElectrical EngineeringTsinghua UniversityBeijingChina Department of Electrical Engineering Technical Universityof DenmarkCopenhagenDenmark School of Technology and Innovations University of VaasaVaasaFinland Institute for Systems and Computer Engineering Technologyand Science(INESC TEC)PortoPortugal Faculty of Engineering University of PortoPortoPortugal

The rapidly increasing penetration of electric vehicles(EVs) in modern metropolises has been witnessed during the past decade, inspired by financial subsidies as well as public awareness of climate change and environment pro-tection. Integrating charging facilities, especially highpower chargers in fast charging stations, into power distribution systems remarkably alters the traditional load flow pattern, and thus imposes great challenges on the operation of distribution network in which controllable resources are rare. On the other hand, provided with appropriate incentives, the energy storage capability of electric vehicle offers a unique opportunity to facilitate the integration of distributed wind and solar power generation into power distribution system. The above trends call for thorough investigation and research on the interdependence between transportation system and power distribution system. This paper conducts a comprehensive survey on this line of research. The basic models of transportation system and power distribution system are introduced,especially the user equilibrium model, which describes the vehicular flow on each road segment and is not familiar to the readers in power system community. The modelling of interdependence across the two systems is *** into account such interdependence, applications ranging from long-term planning to short-term operation are reviewed with emphasis on comparing the description of traffic-power interdependence. Finally, an outlook of prospective directions and key technologies in future research is summarized.

关键词： Charging station Electric vehicle(EV) Interdependence Power flow Power distribution system Transportation system User equilibrium

distributed Attack-Robust Submodular Maximization for Multi-Robot Planning

学校读者我要写书评

暂无评论

Distributed Attack-Robust Submodular Maximization for Multi-...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Lifeng Zhou Vasileios Tzoumas George J. Pappas Pratap Tokekar Department of Electrical and Computer Engineering Virginia Tech Blacksburg VA USA Laboratory of Information and Decision Systems Massachusetts Institute of Technology Cambridge MA USA Department of Electrical and Systems Engineering University of Pennsylvania Philadelphia PA USA Department of Computer Science University of Maryland College Park MD USA

ISBN: (数字)9781728173955

ISBN: (纸本)9781728173962

We aim to guard swarm-robotics applications against denial-of-service (DoS) attacks that result in withdrawals of robots. We focus on applications requiring the selection of actions for each robot, among a set of available ones, e.g., which trajectory to follow. Such applications are central in large-scale robotic applications, e.g., multi-robot motion planning for target tracking. But the current attack-robust algorithms are centralized, and scale quadratically with the problem size (e.g., number of robots). In this paper, we propose a general-purpose distributed algorithm towards robust optimization at scale, with local communications only. We name it distributed robust maximization (DRM). DRM proposes a divide-and-conquer approach that distributively partitions the problem among K cliques of robots. The cliques optimize in parallel, independently of each other. That way, DRM also offers computational speed-ups up to 1/K 2 the running time of its centralized counterparts. K depends on the robots' communication range, which is given as input to DRM. DRM also achieves a close-to-optimal performance. We demonstrate DRM's performance in Gazebo and MATLAB simulations, in scenarios of active target tracking with multiple robots. We observe DRM achieves significant computational speed-ups (it is 3 to 4 orders faster) and, yet, nearly matches the tracking performance of its centralized counterparts.

关键词： Planning Target tracking Robot kinematics Partitioning algorithms Robot sensing systems Robustness

SWRR:The Link Scheduling Algorithm Based on Weighted Round-Robin

学校读者我要写书评

暂无评论

computers, Materials & Continua 2019年第6期59卷 965-982页

作者： Tianbo Lu Ru Yan Chao Li Lihua Yin Hao Chen Xieyu Zou School of Software Engineering Beijing University of Posts and TelecommunicationsBeijing100876China Key Laboratory of Trustworthy Distributed Computing and Service(Beijing University of Posts and Telecommunications) Ministry of EducationBeijing100876China Cyberspace Institute of Advanced Technology Guangzhou UniversityGuangzhou510006China Department Computer Science and Engineering The University of Texas at ArlingtonArlington76091USA

With the rapid development of the Internet,people pay more and more attention to the protection of *** second-generation onion routing system Tor is the most commonly used among anonymous communication systems,which can be used to protect user privacy *** recent years,Tor’s congestion problem has become the focus of attention,and it can affect Tor’s performance even user ***,we investigate the causes of Tor network congestion and summarize some link scheduling algorithms proposed in recent *** we propose the link scheduling algorithm SWRR based on WRR(Weighted Round Robin).In this process,we design multiple weight functions and compare the performance of these weight functions under different congestion conditions,and the appropriate weight function is selected to be used in our algorithms based on the experiment ***,we also compare the performance of SWRR with other link scheduling algorithms under different congestion conditions by experiments,and verify the effectiveness of the algorithm SWRR.

关键词： Weighted Round-Robin Tor congestion control link scheduling anonymous communication

Communication-Efficient distributed Online Nonconvex Optimization with Time-Varying Constraints

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Kunpeng Xu, Lei Yi, Xinlei Wen, Guanghui Cao, Ming Johansson, Karl H. Chai, Tianyou Yang, Tao State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China Department of Mechanical Engineering University of Victoria VictoriaBCV8W 2Y2 Canada Shanghai Institute of Intelligent Science and Technology National Key Laboratory of Autonomous Intelligent Unmanned Systems Frontiers Science Center for Intelligent Autonomous Systems Ministry of Education Shanghai201210 China School of Automation Southeast University Nanjing210096 China Engineering and Technology Institute Groningen Faculty of Science and Engineering University of Groningen GroningenAG 9747 Netherlands Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm10044 Sweden

This paper considers distributed online nonconvex optimization with time-varying inequality constraints over a network of agents, where the nonconvex local loss and convex local constraint functions can vary arbitrarily across iterations, and the information of them is privately revealed to each agent at each iteration. For a uniformly jointly strongly connected time-varying directed graph, we propose two distributed bandit online primal–dual algorithm with compressed communication to efficiently utilize communication resources in the one-point and two-point bandit feedback settings, respectively. In nonconvex optimization, finding a globally optimal decision is often NP-hard. As a result, the standard regret metric used in online convex optimization becomes inapplicable. To measure the performance of the proposed algorithms, we use a network regret metric grounded in the first-order optimality condition associated with the variational inequality. We show that the compressed algorithm with one-point bandit feedback establishes an O(Tθ1) network regret bound and an O(T7/4−θ1) network cumulative constraint violation bound, where T is the number of iterations and θ1 ∈ (3/4,5/6] is a user-defined trade-off parameter. When Slater’s condition holds (i.e, there is a point that strictly satisfies the inequality constraints at all iterations), the network cumulative constraint violation bound is reduced to O(T5/2−2θ1). In addition, we show that the compressed algorithm with two-point bandit feedback establishes an O(Tmax{1−θ1,θ1}) network regret and an O(T1−θ1/2) network cumulative constraint violation bounds, where θ1 ∈ (0,1). Moreover, the network cumulative constraint violation bound is reduced to O(T1−θ1) under Slater’s condition. The bounds are comparable to the state-of-the-art results established by existing distributed online algorithms with perfect communication for distributed online convex optimization with inequality constraints. To the best of our knowledge, thi

关键词： Convex optimization

Foundational Competencies and Responsibilities of a Research Software Engineer

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Goth, Florian Alves, Renato Braun, Matthias Castro, Leyla Jael Chourdakis, Gerasimos Christ, Simon Cohen, Jeremy Druskat, Stephan Erxleben, Fredo Grad, Jean-Noël Hagdorn, Magnus Hodges, Toby Juckeland, Guido Kempf, Dominic Lamprecht, Anna-Lena Linxweiler, Jan Löffler, Frank Martone, Michele Schwarzmeier, Moritz Seibold, Heidi Thiele, Jan Philipp Waldow, Harald von Wittke, Samantha Institute for Theoretical Physics and Astrophysics University of Würzburg Germany European Molecular Biology Laboratory Heidelberg Germany Cluster of Excellence IntCDC University of Stuttgart Germany ZB MED Information Centre for Life Sciences Cologne Germany School of Computation Information and Technology Technical University of Munich Garching Germany Institute for Parallel and Distributed Systems University of Stuttgart Stuttgart Germany Leibniz University Hannover Department of Cell Biology and Biophysics Computational Biology Germany Imperial College London London United Kingdom Institute for Software Technology Berlin Germany Humboldt-Universität zu Berlin Department of Computer Science Berlin Germany Helmholtz-Zentrum Dresden-Rossendorf Germany Institute for Computational Physics University of Stuttgart Germany Geschäftsbereich IT Charité Universitätsmedizin Berlin Germany The Carpentries United States Heidelberg University Scientific Software Center Germany Institute of Computer Science University of Potsdam Germany Technische Universität Braunschweig Germany Michael Stifel Center Jena Friedrich Schiller University Jena Germany Leibniz Supercomputing Centre Garching Germany Mathematical Modeling and Analysis TU Darmstadt Germany IGDORE Munich Germany Weierstrass Institute Berlin Germany Leibniz University Hannover Institute of Applied Mathematics Scientific Computing Hannover Germany Johann Heinrich von Thünen Institute Centre for Information Management Germany

The term Research Software Engineer, or RSE, emerged a little over 10 years ago as a way to represent individuals working in the research community but focusing on software development. The term has been widely adopted and there are a number of high-level definitions of what an RSE is. However, the roles of RSEs vary depending on the institutional context they work in. At one end of the spectrum, RSE roles may look similar to a traditional research role. At the other extreme, they resemble that of a software engineer in industry. Most RSE roles inhabit the space between these two extremes. Therefore, providing a straightforward, comprehensive definition of what an RSE does and what experience, skills and competencies are required to become one is challenging. In this community paper we define the broad notion of what an RSE is, explore the different types of work they undertake, and define a list of foundational competencies as well as values that outline the general profile of an RSE. Further research and training can build upon this foundation of skills and focus on various aspects in greater detail. We expect that graduates and practitioners will have a larger and more diverse set of skills than outlined here. On this basis, we elaborate on the progression of these skills along different dimensions. We look at specific types of RSE roles, propose recommendations for organisations, and give examples of future specialisations. An appendix details how existing curricula fit into this framework. © 2023, CC BY.

关键词： computer aided software engineering