检索结果-内蒙古大学图书馆

Networked federated multi-task learning

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2021年

作者： SarcheshmehPour, Yasmin Tian, Yu Zhang, Linli Jung, Alexander Department of Computer Science Aalto University Finland

Many important application domains generate distributed collections of heterogeneous local datasets. These local datasets are often related via an intrinsic network structure that arises from domain-specific notions of similarity between local datasets. Different notions of similarity are induced by spatio-temporal proximity, statistical dependencies or functional relations. We use this network structure to adaptively pool similar local datasets into nearly homogenous training sets for learning tailored models. Our main conceptual contribution is to formulate networked federated learning using the concept of generalized total variation (GTV) minimization as a regularizer. This formulation is highly flexible and can be combined with almost any parametric model including Lasso or deep neural networks. We unify and considerably extend some well-known approaches to federated multi-task learning. Our main algorithmic contribution is a novel federated learning algorithm which is well suited for distributed computing environments such as edge computing over wireless networks. This algorithm is robust against model misspecification and numerical errors arising from limited computational resources including processing time or wireless channel bandwidth. As our main technical contribution, we offer precise conditions on the local models as well on their network structure such that our algorithm learns nearly optimal local models. Our analysis reveals an interesting interplay between the (information-) geometry of local models and the (cluster-) geometry of their network. © 2021, CC BY.

关键词： learning algorithms

Calibrating and Improving Graph Contrastive learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ma, Kaili Yang, Haochen Yang, Han Chen, Yongqiang Cheng, James Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong

Graph contrastive learning algorithms have demonstrated remarkable success in various applications such as node classification, link prediction, and graph clustering. However, in unsupervised graph contrastive learning, some contrastive pairs may contradict the truths in downstream tasks and thus the decrease of losses on these pairs undesirably harms the performance in the downstream tasks. To assess the discrepancy between the prediction and the ground-truth in the downstream tasks for these contrastive pairs, we adapt expected calibration error (ECE) to graph contrastive learning. The analysis of ECE motivates us to propose a novel regularization method, Contrast-Reg, to ensure that decreasing the contrastive loss leads to better performance in the downstream tasks. As a plug-in regularizer, Contrast-Reg effectively improves the performance of existing graph contrastive learning algorithms. We provide both theoretical and empirical results to demonstrate the effectiveness of Contrast-Reg in enhancing the generalizability of the Graph Neural Network (GNN) model and improving the performance of graph contrastive algorithms with different similarity definitions and encoder backbones across various downstream tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Quantum vs. classical: A comprehensive benchmark study for predicting time series with variational quantum machine learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Fellner, Tobias Kreplin, David Tovey, Samuel Holm, Christian Institute for Computational Physics University of Stuttgart Stuttgart70569 Germany Fraunhofer Institute for Manufacturing Engineering and Automation Stuttgart70569 Germany

Variational quantum machine learning algorithms have been proposed as promising tools for time series prediction, with the potential to handle complex sequential data more effectively than classical approaches. However, their practical advantage over established classical methods remains uncertain. In this work, we present a comprehensive benchmark study comparing a range of variational quantum algorithms and classical machine learning models for time series forecasting. We evaluate their predictive performance on three chaotic systems across 27 time series prediction tasks of varying complexity, and ensure a fair comparison through extensive hyperparameter optimization. Our results indicate that, in many cases, quantum models struggle to match the accuracy of simple classical counterparts of comparable complexity. Furthermore, we analyze the predictive performance relative to the model complexity and discuss the practical limitations of variational quantum algorithms for time series forecasting. © 2025, CC BY.

关键词： learning algorithms

Doubly adaptive scaled algorithm for machine learning using second-order information

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Jahani, Majid Rusakov, Sergey Shi, Zheng Richtárik, Peter Mahoney, Michael W. Takáč, Martin Lehigh University United States KAUST Saudi Arabia University of California Berkeley United States MBZUAI United Arab Emirates

We present a novel adaptive optimization algorithm for large-scale machine learning problems. Equipped with a low-cost estimate of local curvature and Lipschitz smoothness, our method dynamically adapts the search direction and step-size. The search direction contains gradient information preconditioned by a well-scaled diagonal preconditioning matrix that captures the local curvature information. Our methodology does not require the tedious task of learning rate tuning, as the learning rate is updated automatically without adding an extra hyperparameter. We provide convergence guarantees on a comprehensive collection of optimization problems, including convex, strongly convex, and nonconvex problems, in both deterministic and stochastic regimes. We also conduct an extensive empirical evaluation on standard machine learning problems, justifying our algorithm's versatility and demonstrating its strong performance compared to other start-of-the-art first-order and second-order methods. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Evidence-based prescriptive analytics, causal digital twin and a learning estimation algorithm

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Madhavan, P.G. Jin Innovation Seattle United States

Evidence-based Prescriptive Analytics (EbPA) is necessary to determine optimal operational set-points that will improve business productivity. EbPA results from "what-if" analysis and counterfactual experimentation on CAUSAL Digital Twins (CDTs) that quantify cause-effect relationships in the DYNAMICS of a system of connected assets. We describe the basics of Causality and Causal Graphs and develop a learning Causal Digital Twin (LCDT) solution;our algorithm uses a simple recurrent neural network with some innovative modifications incorporating Causal Graph simulation. Since LCDT is a "learning" digital twin where parameters are learned online in real-time with minimal pre-configuration, the work of deploying digital twins will be significantly *** Codes 93-05 Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Co2L: Contrastive continual learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Cha, Hyuntak Lee, Jaeho Shin, Jinwoo Daejeon Korea Republic of

Recent breakthroughs in self-supervised learning show that such algorithms learn visual representations that can be transferred better to unseen tasks than joint-training methods relying on task-specific supervision. In this paper, we found that the similar holds in the continual learning context: contrastively learned representations are more robust against the catastrophic forgetting than jointly trained representations. Based on this novel observation, we propose a rehearsal-based continual learning algorithm that focuses on continually learning and maintaining transferable representations. More specifically, the proposed scheme (1) learns representations using the contrastive learning objective, and (2) preserves learned representations using a self-supervised distillation step. We conduct extensive experimental validations under popular benchmark image classification datasets, where our method sets the new state-of-the-art performance. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Regret analysis in deterministic reinforcement learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Tranos, Damianos Proutiere, Alexandre Stockholm Sweden

We consider Markov Decision Processes (MDPs) with deterministic transitions and study the problem of regret minimization, which is central to the analysis and design of optimal learning algorithms. We present logarithmic problem-specific regret lower bounds that explicitly depend on the system parameter (in contrast to previous minimax approaches) and thus, truly quantify the fundamental limit of performance achievable by any learning algorithm. Deterministic MDPs can be interpreted as graphs and analyzed in terms of their cycles, a fact which we leverage in order to identify a class of deterministic MDPs whose regret lower bound can be determined numerically. We further exemplify this result on a deterministic line search problem, and a deterministic MDP with state-dependent rewards, whose regret lower bounds we can state explicitly. These bounds share similarities with the known problem-specific bound of the multi-armed bandit problem and suggest that navigation on a deterministic MDP need not have an effect on the performance of a learning algorithm. © 2021, CC BY.

关键词： learning algorithms

N-Omniglot, a large-scale neuromorphic dataset for spatio-temporal sparse few-shot learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Li, Yang Dong, Yiting Zhao, Dongcheng Zeng, Yi Brain-inspired Cognitive Intelligence Lab Institute of Automation Chinese Academy of Sciences Beijing China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China School of Future Technology University of Chinese Academy of Sciences Beijing China National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing China Center for Excellence in Brain Science and Intelligence Technology Chinese Academy of Sciences Shanghai China

Few-shot learning (learning with a few samples) is one of the most important cognitive abilities of the human brain. However, the current artificial intelligence systems meet difficulties in achieving this ability. Similar challenges also exist for biologically plausible spiking neural networks (SNNs). Datasets for traditional few-shot learning domains provide few amounts of temporal information. and the absence of neuromorphic datasets has hindered the development of few-shot learning for SNNs. Here, to the best of our knowledge, we provide the first neuromorphic dataset for few-shot learning using SNNs: N-Omniglot, based on the Dynamic Vision Sensor. It contains 1,623 categories of handwritten characters, with only 20 samples per class. N-Omniglot eliminates the need for a neuromorphic dataset for SNNs with high spareness and tremendous temporal coherence. Additionally, the dataset provides a powerful challenge and a suitable benchmark for developing SNNs algorithms in the few-shot learning domain due to the chronological information of strokes. We also provide the improved nearest neighbor, convolutional network, SiameseNet, and meta-learning algorithm in the spiking version for verification. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Deep learning-Based Breast Cancer Diagnosis at Ultrasound: Initial Application of Weakly-Supervised Algorithm Without Image Annotation Original Research

学校读者我要写书评

暂无评论

Research Square

Research Square 2021年

作者： Kim, Jaeil Kim, Hye Jung Kim, Chanho Lee, Jin Hwa Kim, Keum Won Park, Young Mi Ki, So Yeon Kim, You Me Kim, Won Hwa Kim, Hye Won School of Computer Science and Engineering Kyungpook National University Daegu Korea Republic of Departments of Radiology Kyungpook National University Chilgok Hospital School of Medicine Kyungpook National University Daegu Korea Republic of Departments of Radiology Dong-A University College of Medicine Busan Korea Republic of Departments of Radiology School of Medicine Konyang University Konyang Univeristy Hospital Daejeon Korea Republic of Department of Radiology School of Medicine Inje University Busan Paik Hospital Busan Korea Republic of Department of Radiology Wonkwang University Hospital Wonkwang University School of Medicine Iksan Korea Republic of Department of Radiology School of Medicine Chonnam National University Chonnam National University Hwasun Hospital Hwasun Korea Republic of Department of Radiology School of Medicine Dankook University Dankook University Hospital Cheonan Korea Republic of

Conventional deep learning (DL) algorithm requires full supervision of annotating the region of interest (ROI) that is laborious and often biased. We aimed to develop a weakly-supervised DL algorithm that diagnosis breast cancer at ultrasound without image annotation. Weakly-supervised DL algorithms were implemented with three networks (VGG16, ResNet34, and GoogLeNet) and trained using 1000 unannotated US images (500 benign and 500 malignant masses). Two sets of 200 images (100 benign and 100 malignant masses) were used for internal and external validation sets. For comparison with fully-supervised algorithms, ROI annotation was performed manually and automatically. Diagnostic performances were calculated as the area under the receiver operating characteristic curve (AUC). Using the class activation map, we determined how accurately the weakly-supervised DL algorithms localized the breast masses. For internal validation sets, the weakly-supervised DL algorithms achieved excellent diagnostic performances, with AUC values of 0.92–0.96, which were not statistically different (all Ps > 0.05) from those of fully-supervised DL algorithms with either manual or automated ROI annotation (AUC, 0.92–0.96). For external validation sets, the weakly-supervised DL algorithms achieved AUC values of 0.86–0.90, which were not statistically different (Ps > 0.05) or higher (P = 0.04, VGG16 with automated ROI annotation) from those of fully-supervised DL algorithms (AUC, 0.84–0.92). In internal and external validation sets, weakly-supervised algorithms could localize 100% of malignant masses, except for ResNet34 (98%). The weakly-supervised DL algorithms developed in the present study were feasible for US diagnosis of breast cancer with well-performing localization and differential diagnosis. © 2021, CC BY.

关键词： learning algorithms