检索结果-内蒙古大学图书馆

Network-GIANT: Fully distributed Newton-type optimization via harmonic Hessian consensus

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Maritan, Alessio Sharma, Ganesh Schenato, Luca Dey, Subhrakanti Department of Information Engineering University of Padova Italy Hamilton Institute Maynooth University Ireland Department of Electrical Engineeringe Uppsala University Sweden

This paper considers the problem of distributed multi-agent learning, where the global aim is to minimize a sum of local objective (empirical loss) functions through local optimization and information exchange between neighboring nodes. We introduce a Newton-type fully distributed optimization algorithm, Network-GIANT, which is based on GIANT, a Federated learning algorithm that relies on a centralized parameter server. The Network-GIANT algorithm is designed via a combination of gradient-tracking and a Newton-type iterative algorithm at each node with consensus-based averaging of local gradient and Newton updates. The resulting algorithm is efficient in terms of both communication cost and run-time, making it suitable for wireless networks. We prove that our algorithm guarantees semi-global and exponential convergence to the exact solution over the network assuming strongly convex and smooth loss functions. We provide empirical evidence of the superior convergence performance of Network-GIANT over other state-of-art distributed learning algorithms such as Network-DANE and Newton-Raphson Consensus. Copyright © 2023, The Authors. All rights reserved.

关键词： learning algorithms

Defense Against Adversarial Attacks Based on Color Space Transformation

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Wu, Chunhua Zheng, Kangfeng Wang, Haoyu School of Cyberspace Security Beijing University of Posts and Telecommunications China

Deep learning algorithms have achieved state-of-the-art performances on a variety of important tasks. However recent works have found that an elaborate perturbation can cause the network to misclassify, which is called adversarial attack. In this paper, we implemented a defense method by proposing a new training strategy that applies an invertible transformation to training data. By this transformation, we changed the color space distribution and increased the color difference between neighboring pixel values, thus amplifying the visual effect of adversarial perturbations. Compared to traditional pre-processing defenses, our transformation of the image is more daring and results in better defense efficiency. Our experiments on Cifar-10 demonstrated that our defense method can effectively resist black-box transferable adversarial attacks, with the attack success rate of migration attacks under different models being less than 5%. We experiment our defense against four white-box,with a particular focus on the state-of-the-art BPDA attack. In white-box attack experiments, we increased the average pixel value of the perturbation from 1 to 52, and it was observed that adversarial examples could be detected by the human eye because slight adversarial perturbations were significantly amplified. Our model is the first defense model based on amplifying adversarial perturbations and is model-agnostic;thus, it can be applied to any deep learning model to improve robustness. © 2023, The Authors. All rights reserved.

关键词： learning algorithms

Best-Effort Adaptation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Awasthi, Pranjal Cortes, Corinna Mohri, Mehryar Google Research Mountain View United States Google Research Courant Institute of Mathematical Sciences New York United States

We study a problem of best-effort adaptation motivated by several applications and considerations, which consists of determining an accurate predictor for a target domain, for which a moderate amount of labeled samples are available, while leveraging information from another domain for which substantially more labeled samples are at one’s disposal. We present a new and general discrepancy-based theoretical analysis of sample reweighting methods, including bounds holding uniformly over the weights. We show how these bounds can guide the design of learning algorithms that we discuss in detail. We further show that our learning guarantees and algorithms provide improved solutions for standard domain adaptation problems, for which few labeled data or none are available from the target domain. We finally report the results of a series of experiments demonstrating the effectiveness of our best-effort adaptation and domain adaptation algorithms, as well as comparisons with several baselines. We also discuss how our analysis can benefit the design of principled solutions for fine-tuning. © 2023, CC BY.

关键词： learning algorithms

ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yan, Junjun Chen, Xinhai Wang, Zhichao Zhoui, Enqiang Liu, Jie Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha410073 China Laboratory of Digitizing Software for Frontier Equipment National University of Defense Technology Changsha410073 China

Partial differential equations (PDEs) are an essential computational kernel in physics and engineering. With the advance of deep learning, physics-informed neural networks (PINNs), as a mesh-free method, have shown great potential for fast PDE solving in various applications. To address the issue of low accuracy and convergence problems of existing PINNs, we propose a self-training physics-informed neural network, ST-PINN. Specifically, ST-PINN introduces a pseudo label based self-learning algorithm during training. It employs governing equation as the pseudo-labeled evaluation index and selects the highest confidence examples from the sample points to attach the pseudo labels. To our best knowledge, we are the first to incorporate a self-training mechanism into physics-informed learning. We conduct experiments on five PDE problems in different fields and scenarios. The results demonstrate that the proposed method allows the network to learn more physical information and benefit convergence. The ST-PINN outperforms existing physics-informed neural network methods and improves the accuracy by a factor of 1.33x-2.54x. The code of ST-PINN is available at GitHub: https://***/junjun-yan/ST-PINN. Copyright © 2023, The Authors. All rights reserved.

关键词： learning algorithms

Spiking Transformer Networks: A Rate Coded Approach for Processing Sequential Data 7

学校读者我要写书评

暂无评论

Spiking Transformer Networks: A Rate Coded Approach for Proc...

7th International Conference on Systems and Informatics, ICSAI 2021

作者： Mueller, Etienne Studenyak, Viktor Auge, Daniel Knoll, Alois Technical University of Munich Munich Germany

ISBN: (纸本)9781665426244

Machine learning applications are steadily increasing in performance, while also being deployed on a growing number of devices with limited energy resources. To minimize this trade-off, researchers are continually looking for more energy efficient solutions. A promising field involves the use of spiking neural networks in combination with neuromorphic hardware, significantly reducing energy consumption since energy is only consumed as information is being processed. However, as their learning algorithms lag behind conventional neural networks trained with backpropagation, not many applications can be found today. The highest levels of accuracy can be achieved by converting networks that are trained with backpropagation to spiking networks. Spiking neural networks can show nearly the same performance in fully connected and convolutional networks. The conversion of recurrent networks has been shown to be challenging. However, recent progress with transformer networks could change this. This type of network not only consists of modules that can easily be converted, but also shows the best accuracy levels for different machine learning tasks. In this work, we present a method to convert the transformer architecture to networks of spiking neurons. With only minimal conversion loss, our approach can be used for processing sequential data with very high accuracy while offering the possibility of reductions in energy consumption. © 2021 IEEE.

关键词： learning algorithms

Using simulation to calibrate real data acquisition in veterinary medicine

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Strzalka, Krystian Mazurek, Szymon Wielgosz, Maciej Russek, Pawel Caputa, Jakub Lukasik, Daria Krupiński, Jan Grzeszczyk, Jakub Karwatowski, Michal Fraczek, Rafal Jamro, Ernest Pietron, Marcin Koryciak, Sebastian Dabrowska-Boruch, Agnieszka Wiatr, Kazimierz ACK Cyfronet AGH ul. Nawojki 11 Krakow30-950 Poland AGH University of Science and Technology 30 Mickiewicza Ave. Krakow30-059 Poland

This paper explores the innovative use of simulation environments to enhance data acquisition and diagnostics in veterinary medicine, focusing specifically on gait analysis in dogs. The study harnesses the power of Blender and the Blenderproc library to generate synthetic datasets that reflect diverse anatomical, environmental, and behavioral conditions. The generated data, represented in graph form and standardized for optimal analysis, is utilized to train machine learning algorithms for identifying normal and abnormal gaits. Two distinct datasets with varying degrees of camera angle granularity are created to further investigate the influence of camera perspective on model accuracy. Preliminary results suggest that this simulation-based approach holds promise for advancing veterinary diagnostics by enabling more precise data acquisition and more effective machine learning models. By integrating synthetic and real-world patient data, the study lays a robust foundation for improving overall effectiveness and efficiency in veterinary medicine. Copyright © 2023, The Authors. All rights reserved.

关键词： learning algorithms

A Log-linear Gradient Descent Algorithm for Unbalanced Binary Classification using the All Pairs Squared Hinge Loss

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Rust, Kyle R. Hocking, Toby D. Northern Arizona University United States

Receiver Operating Characteristic (ROC) curves are plots of true positive rate versus false positive rate which are used to evaluate binary classification algorithms. Because the Area Under the Curve (AUC) is a constant function of the predicted values, learning algorithms instead optimize convex relaxations which involve a sum over all pairs of labeled positive and negative examples. Naïve learning algorithms compute the gradient in quadratic time, which is too slow for learning using large batch sizes. We propose a new functional representation of the square loss and squared hinge loss, which results in algorithms that compute the gradient in either linear or log-linear time, and makes it possible to use gradient descent learning with large batch sizes. In our empirical study of supervised binary classification problems, we show that our new algorithm can achieve higher test AUC values on imbalanced data sets than previous algorithms, and make use of larger batch sizes than were previously feasible. Copyright © 2023, The Authors. All rights reserved.

关键词： learning algorithms

Conjunctive queries: Unique characterizations and exact learnability 24

学校读者我要写书评

暂无评论

Conjunctive queries: Unique characterizations and exact lear...

24th International Conference on Database Theory, ICDT 2021

作者： Ten Cate, Balder Dalmau, Victor Google Mountain ViewCA United States Universitat Pompeu Fabra Barcelona Spain

ISBN: (纸本)9783959771795

We answer the question of which conjunctive queries are uniquely characterized by polynomially many positive and negative examples, and how to construct such examples efficiently. As a consequence, we obtain a new efficient exact learning algorithm for a class of conjunctive queries. At the core of our contributions lie two new polynomial-time algorithms for constructing frontiers in the homomorphism lattice of finite structures. We also discuss implications for the unique characterizability and learnability of schema mappings and of description logic concepts. © Balder ten Cate and Victor Dalmau.

关键词： learning algorithms

Guided-LORE: Improving LORE with a Focused Search of Neighbours 1

学校读者我要写书评

暂无评论

1st International Workshop on Trustworthy AI – Integrating learning, Optimization and Reasoning, TAILOR 2020 held as a part of European Conference on Artificial Intelligence, ECAI 2020

作者： Maaroof, Najlaa Moreno, Antonio Valls, Aida Jabreel, Mohammed ITAKA-Intelligent Technologies for Advanced Knowledge Acquisition Universitat Rovira i Virgili Tarragona Spain Department of Computer Science Hodeidah University Hodeidah Yemen

ISBN: (数字)9783030739591

ISBN: (纸本)9783030739584

In the last years many Machine learning-based systems have achieved an extraordinary performance in a wide range of fields, especially Natural Language Processing and Computer Vision. The construction of appropriate and meaningful explanations for the decisions of those systems, which are in may cases considered black boxes, is a hot research topic. Most of the state-of-the-art explanation systems in the field rely on generating synthetic instances to test them on the black box, analyse the results and build an interpretable model, which is then used to provide an understandable explanation. In this paper we focus on this generation process and propose Guided-LORE, a variant of the well-known LORE method. We conceptualise the problem of neighbourhood generation as a search and solve it using the Uniform Cost Search algorithm. The experimental results on four datasets reveal the effectiveness of our method when compared with the most relevant related works. © 2021, Springer Nature Switzerland AG.

关键词： learning algorithms