检索结果-内蒙古大学图书馆

DP-REC: PRIVATE & COMMUNICATION-EFFICIENT FEDERATED learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Triastcyn, Aleksei Reisser, Matthias Louizos, Christos Qualcomm AI Research

Privacy and communication efficiency are important challenges in federated training of neural networks, and combining them is still an open problem. In this work, we develop a method that unifies highly compressed communication and differential privacy (DP). We introduce a compression technique based on Relative Entropy Coding (REC) to the federated setting. With a minor modification to REC, we obtain a provably differentially private learning algorithm, DP-REC, and show how to compute its privacy guarantees. Our experiments demonstrate that DP-REC drastically reduces communication costs while providing privacy guarantees comparable to the state-of-the-art. © 2021, CC BY.

关键词： learning algorithms

PMFL: Partial Meta-Federated learning for heterogeneous tasks and its applications on real-world medical records

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Tianyi Zhang, Shirui Chen, Ziwei Bengio, Yoshua Liu, Dianbo Fulton Schools of Engineering Arizona State University AZ United States Singapore Beijing Jiaotong University Beijing China Mila - Quebec AI Institute CIFAR AI Chair QC Canada Broad Institute of MIT and Harvard MA United States Mila - Quebec AI Institute QC Canada

Federated machine learning is a versatile and flexible tool to utilize distributed data from different sources, especially when communication technology develops rapidly and an unprecedented amount of data could be collected on mobile devices nowadays. Federated learning method exploits not only the data but the computational power of all devices in the network to achieve more efficient model training. Nevertheless, while most traditional federated learning methods work well for homogeneous data and tasks, adapting the method to heterogeneous data and task distribution is challenging. This limitation has constrained the applications of federated learning in real-world contexts, especially in healthcare settings. Inspired by the fundamental idea of meta-learning, in this study we propose a new algorithm, which is an integration of federated learning and meta-learning, to tackle this issue. In addition, owing to the advantage of transfer learning for model generalization, we further improve our algorithm by introducing partial parameter sharing to balance global and local learning. We name this method partial meta-federated learning (PMFL). Finally, we apply the algorithms to two medical datasets. We show that our algorithm could obtain the fastest training speed and achieve the best performance when dealing with heterogeneous medical datasets. The source code is available at https://***/destiny301/PMFL. © 2021, CC BY.

关键词： learning algorithms

Frugal machine learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Evchenko, Mikhail Vanschoren, Joaquin Hoos, Holger H. Schoenauer, Marc Sebag, Michèle

Machine learning, already at the core of increasingly many IT systems and applications, is set to become even more ubiquitous with the rapid rise of wearable devices and the Internet of Things. In most machine learning applications, the main focus is on the quality of the results achieved (e.g., prediction accuracy), and hence vast amounts of data are being collected, requiring significant computational resources to build models. In many scenarios however, it is infeasible or impractical to set up large centralized data repositories. In personal health, for instance, privacy issues may inhibit the sharing of detailed personal data. In such cases, machine learning should ideally be performed on wearable devices themselves, which raises major computational limitations such as the battery capacity of smartwatches. This paper thus investigates frugal learning, aimed to build the most accurate possible models using the least amount of resources. A wide range of learning algorithms is examined through a ‘frugal lens’, analysing their accuracy/runtime performance on a wide range of data sets. The most promising algorithms are thereafter assessed in a real-world scenario by implementing them in a smartwatch and letting them learn activity recognition models on the watch itself. © 2021, CC BY.

关键词： learning algorithms

Semi-supervised learning for marked temporal point processes

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Reddy, Shivshankar Chauhan, Anand Vir Singh Singh, Maneet Singh, Karamjit AI Garage Mastercard India

Temporal Point Processes (TPPs) are often used to represent the sequence of events ordered as per the time of occurrence. Owing to their flexible nature, TPPs have been used to model different scenarios and have shown applicability in various real-world applications. While TPPs focus on modeling the event occurrence, Marked Temporal Point Process (MTPP) focuses on modeling the category/class of the event as well (termed as the marker). Research in MTPP has garnered substantial attention over the past few years, with an extensive focus on supervised algorithms. Despite the research focus, limited attention has been given to the challenging problem of developing solutions in semi-supervised settings, where algorithms have access to a mix of labeled and unlabeled data. This research proposes a novel algorithm for Semi-supervised learning for Marked Temporal Point Processes (SSL-MTPP) applicable in such scenarios. The proposed SSL-MTPP algorithm utilizes a combination of labeled and unlabeled data for learning a robust marker prediction model. The proposed algorithm utilizes an RNN-based Encoder-Decoder module for learning effective representations of the time sequence. The efficacy of the proposed algorithm has been demonstrated via multiple protocols on the Retweet dataset, where the proposed SSL-MTPP demonstrates improved performance in comparison to the traditional supervised learning approach. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

learning rate optimization for federated learning exploiting over-the-air computation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Xu, Chunmei Liu, Shengheng Yang, Zhaohui Huang, Yongming Wong, Kai-Kit School of Information Science and Engineering Southeast University Nanjing210096 China Purple Mountain Laboratories Nanjing211111 China Centre for Telecommunications Research Department of Engineering King’s College London WC2R 2LS United Kingdom Department of Electronic and Electrical Engineering University College London LondonWC1E 6BT United Kingdom

Federated learning (FL) as a promising edge-learning framework can effectively address the latency and privacy issues by featuring distributed learning at the devices and model aggregation in the central server. In order to enable efficient wireless data aggregation, over-the-air computation (AirComp) has recently been proposed and attracted immediate attention. However, fading of wireless channels can produce aggregate distortions in an AirComp-based FL scheme. To combat this effect, the concept of dynamic learning rate (DLR) is proposed in this work. We begin our discussion by considering multiple-input-single-output (MISO) scenario, since the underlying optimization problem is convex and has closed-form solution. We then extend our studies to more general multiple-input-multiple-output (MIMO) case and an iterative method is derived. Extensive simulation results demonstrate the effectiveness of the proposed scheme in reducing the aggregate distortion and guaranteeing the testing accuracy using the MNIST and CIFAR10 datasets. In addition, we present the asymptotic analysis and give a near-optimal receive beamforming design solution in closed form, which is verified by numerical simulations. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Quantum vs. classical: A comprehensive benchmark study for predicting time series with variational quantum machine learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Fellner, Tobias Kreplin, David Tovey, Samuel Holm, Christian Institute for Computational Physics University of Stuttgart Stuttgart70569 Germany Fraunhofer Institute for Manufacturing Engineering and Automation Stuttgart70569 Germany

Variational quantum machine learning algorithms have been proposed as promising tools for time series prediction, with the potential to handle complex sequential data more effectively than classical approaches. However, their practical advantage over established classical methods remains uncertain. In this work, we present a comprehensive benchmark study comparing a range of variational quantum algorithms and classical machine learning models for time series forecasting. We evaluate their predictive performance on three chaotic systems across 27 time series prediction tasks of varying complexity, and ensure a fair comparison through extensive hyperparameter optimization. Our results indicate that, in many cases, quantum models struggle to match the accuracy of simple classical counterparts of comparable complexity. Furthermore, we analyze the predictive performance relative to the model complexity and discuss the practical limitations of variational quantum algorithms for time series forecasting. © 2025, CC BY.

关键词： learning algorithms

OscNet: Machine learning on CMOS Oscillator Networks

学校读者我要写书评

暂无评论

IEEE Transactions on Circuits and Systems II: Express Briefs 2025年

作者： Cai, Wenxiao Li, Zongru Lee, Thomas H. Stanford University Department of Electrical Engineering StanfordCA94305 United States

In this brief, we propose a pipeline for machine learning on CMOS Oscillator Networks (OscNet), inspired by human visual system development. By relying solely on forward propagation, OscNet is energy efficient and preserves biological plausibility. Our simulations confirm the effectiveness of the proposed architectures, and experimental results demonstrate that the Hebbian-learning pipeline on OscNet matches traditional machine learning algorithms, underscoring its promise as an energy efficient and high performance new computational paradigm. © 2004-2012 IEEE.

关键词： learning algorithms

learning where to learn: Gradient sparsity in Meta and continual learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： von Oswald, Johannes Zhao, Dominic Kobayashi, Seijin Schug, Simon Caccia, Massimo Zucchet, Nicolas Sacramento, João Institute of Neuroinformatics University of Zürich ETH Zürich Mila University of Montreal ServiceNow

Finding neural network weights that generalize well from small datasets is difficult. A promising approach is to learn a weight initialization such that a small number of weight changes results in low generalization error. We show that this form of meta-learning can be improved by letting the learning algorithm decide which weights to change, i.e., by learning where to learn. We find that patterned sparsity emerges from this process, with the pattern of sparsity varying on a problem-by-problem basis. This selective sparsity results in better generalization and less interference in a range of few-shot and continual learning problems. Moreover, we find that sparse learning also emerges in a more expressive model where learning rates are meta-learned. Our results shed light on an ongoing debate on whether meta-learning can discover adaptable features and suggest that learning by sparse gradient descent is a powerful inductive bias for meta-learning systems. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Generalization in dexterous manipulation via geometry-aware multi-task learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Huang, Wenlong Mordatch, Igor Abbeel, Pieter Pathak, Deepak UC Berkeley Google Brain Carnegie Mellon University

Dexterous manipulation of arbitrary objects, a fundamental daily task for humans, has been a grand challenge for autonomous robotic systems. Although data-driven approaches using reinforcement learning can develop specialist policies that discover behaviors to control a single object, they often exhibit poor generalization to unseen ones. In this work, we show that policies learned by existing reinforcement learning algorithms can in fact be generalist when combined with multi-task learning and a well-chosen object representation. We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects and generalize to new objects with unseen shape or size. Interestingly, we find that multi-task learning with object point cloud representations not only generalizes better but even outperforms the single-object specialist policies on both training as well as held-out test objects. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms