检索结果-内蒙古大学图书馆

Accelerated Methods with Compressed Communications for Distributed Optimization Problems under Data Similarity

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bylinkin, Dmitry Beznosikov, Aleksandr Moscow Institute of Physics and Technology Russia Ivannikov Institute for System Programming of the Russian Academy of Sciences Russia Sber AI Lab Skoltech Russia

In recent years, as data and problem sizes have increased, distributed learning has become an essential tool for training highperformance models. However, the communication bottleneck, especially for high-dimensional data, is a challenge. Several techniques have been developed to overcome this problem. These include communication compression and implementation of local steps, which work particularly well when there is similarity of local data samples. In this paper, we study the synergy of these approaches for efficient distributed optimization. We propose the first theoretically grounded accelerated algorithms utilizing unbiased and biased compression under data similarity, leveraging variance reduction and error feedback frameworks. In terms of communication time our theory gives O ( 1 + h M-1/4 + ω-1/2 ip δ/μ ) complexity for unbiased compressors and O ( 1 + β1/4 √ δ/μ ) for biased ones, where M is the number of computational nodes, β is the compression power, δ is the similarity measure and μ is the parameter of strong convexity of the objective. Our theoretical results are of record and confirmed by experiments on different average losses and datasets. Copyright © 2024, The Authors. All rights reserved.

关键词： Data compression

Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity 39

学校读者我要写书评

暂无评论

Accelerated Methods with Compressed Communications for Distr...

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Bylinkin, Dmitry Beznosikov, Aleksandr Moscow Institute of Physics and Technology Russian Federation Ivannikov Institute for System Programming The Russian Academy of Sciences Russian Federation Sber AI Lab Russian Federation Skoltech Russian Federation

ISBN: (纸本)157735897X

In recent years, as data and problem sizes have increased, distributed learning has become an essential tool for training high-performance models. However, the communication bottleneck, especially for high-dimensional data, is a challenge. Several techniques have been developed to overcome this problem. These include communication compression and implementation of local steps, which work particularly well when there is similarity of local data samples. In this paper, we study the synergy of these approaches for efficient distributed optimization. We propose the first theoretically grounded accelerated algorithms utilizing unbiased and biased compression under data similarity, leveraging variance reduction and error feedback frameworks. In terms of communication time our theory gives Õ (1 + [M⁻^1/4 + ω⁻^1/2i]√δ/µ ) complexity for unbiased compressors and Õ( 1 + β^1/4√δ/µ ) for biased ones, where M is the number of computational nodes, β is the compression power, δ is the similarity measure and µ is the parameter of strong convexity of the objective. Our theoretical results are of record and confirmed by experiments on different average losses and datasets. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词：

Local SGD for Near-Quadratic Problems: Improving Convergence under Unconstrained Noise Conditions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sadchikov, Andrey Chezhegov, Savelii Beznosikov, Aleksandr Gasnikov, Alexander Moscow Institute of Physics and Technology Moscow Russia Ivannikov Institute for System Programming RAS Moscow Russia Sber AI Lab Moscow Russia Innopolis University Innopolis Russia

Distributed optimization plays an important role in modern large-scale machine learning and data processing systems by optimizing the utilization of computational resources. One of the classical and popular approaches is Local Stochastic Gradient Descent (Local SGD), characterized by multiple local updates before averaging, which is particularly useful in distributed environments to reduce communication bottlenecks and improve scalability. A typical feature of this method is the dependence on the frequency of communications. But in the case of a quadratic target function with homogeneous data distribution over all devices, the influence of frequency of communications vanishes. As a natural consequence, subsequent studies include the assumption of a Lipschitz Hessian, as this indicates the similarity of the optimized function to a quadratic one to some extent. However, in order to extend the completeness of the Local SGD theory and unlock its potential, in this paper we abandon the Lipschitz Hessian assumption by introducing a new concept of approximate quadraticity. This assumption gives a new perspective on problems that have near quadratic properties. In addition, existing theoretical analyses of Local SGD often assume bounded variance. We, in turn, consider the unbounded noise condition, which allows us to broaden the class of studied problems. Copyright © 2024, The Authors. All rights reserved.

关键词： Stochastic systems

Searching by Code: a New SearchBySnippet Dataset and SnippeR Retrieval Model for Searching by Code Snippets

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Sedykh, Ivan Abulkhanov, Dmitry Sorokin, Nikita Nikolenko, Sergey Malykh, Valentin Huawei Noah's Ark lab Russia St. Petersburg Department of the Steklov Institute of Mathematics Russia Ivannikov Institute for System Programming Russia

Code search is an important and well-studied task, but it usually means searching for code by a text query. We argue that using a code snippet (and possibly an error traceback) as a query while looking for bugfixing instructions and code samples is a natural use case not covered by prior art. Moreover, existing datasets use code comments rather than full-text descriptions as text, making them unsuitable for this use case. We present a new SearchBySnippet dataset implementing the search-by-code use case based on StackOverflow data;we show that on SearchBySnippet, existing architectures fall short of a simple BM25 baseline even after fine-tuning. We present a new single encoder model SnippeR that outperforms several strong baselines on SearchBySnippet with a result of 0.451 Recall@10;we propose the SearchBySnippet dataset and SnippeR as a new important benchmark for code search evaluation. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

Sinkhorn Transformations for Single-Query Postprocessing in Text-Video Retrieval

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yakovlev, Konstantin Polyakov, Gregory Alimova, Ilseyar Podolskiy, Alexander Bout, Andrey Nikolenko, Sergey Piontkovskaya, Irina Huawei Noah’s Ark Lab Moscow Russia Ivannikov Institute for System Programming of the RAS Moscow Russia St. Petersburg Department the Steklov Institute of Mathematics St. Petersburg Russia

A recent trend in multimodal retrieval is related to postprocessing test set results via the dual-softmax loss (DSL). While this approach can bring significant improvements, it usually presumes that an entire matrix of test samples is available as DSL input. This work introduces a new postprocessing approach based on Sinkhorn transformations that outperforms DSL. Further, we propose a new postprocessing setting that does not require access to multiple test queries. We show that our approach can significantly improve the results of state of the art models such as CLIP4Clip, BLIP, X-CLIP, and DRL, thus achieving a new state-of-the-art on several standard text-video retrieval datasets both with access to the entire test set and in the single-query setting. Copyright © 2023, The Authors. All rights reserved.

关键词： Information retrieval

Epistemic Multiagent Reasoning with Collaborative Robots 37

学校读者我要写书评

暂无评论

Epistemic Multiagent Reasoning with Collaborative Robots

37th Italian Conference on Computational Logic, CILC 2022

作者： Soldà, Davide Fabiano, Francesco Dovier, Agostino Constraint and Logic Programming Lab University of Udine Udine33100 Italy Embedding System Unit of Fondazione Bruno Kessler Trento38123 Italy Department of Mathematical Physical and Computer Sciences University of Parma Parma43124 Italy Department of Mathematics Computer Science and Physics University of Udine Udine33100 Italy

Over the last few years, the fields of Artificial Intelligence, Robotics and IoT have gained a lot of attention. This increasing interest has brought, among other things, to the development of autonomous multi-agent systems where robotic entities may interact with each other. As for all the other autonomous settings, also these systems require arbitration. Our work tries to address this problem by presenting a framework that embeds both a classical and a multi-agent epistemic (epistemic, for brevity) planner in a robotic control architecture. The idea is to combine the (i) classical and the (ii) epistemic solvers to model efficiently the interaction with: the (i) physical world and the (ii) information flows, respectively. In particular, the presented architecture starts by planning on the "epistemic level" refining then single-agent world-altering actions thanks to the classical planner. To further optimize the solving process, we also introduce the concept of macros in epistemic planning. Macros, in fact, have been successfully employed in classical planning as they allow for opportune aggregations of actions that may lead to a reduction of plans' length. Finally, the overall framework is exemplified and validated with two Franka Emika manipulators. This allowed us to empirically justify how the combination of the two planning approaches (classical and epistemic), and the introduction of macros, reduce the computational time required by the orchestrating phase. © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0)

关键词： Robot programming

Design and Implementation of a QoL Improvement system for Visually Impaired Patients 5

学校读者我要写书评

暂无评论

Design and Implementation of a QoL Improvement System for Vi...

5th International Conference on Advances in Computational Tools for Engineering Applications, ACTEA 2023

作者： Abi Zeid Daou, Roy Beaino, Jad Hawat, Johnny Ibrahim, Ramy Hayek, Ali Boercsoek, Josef Olmedo, Jose Javier Serrano Université la Sagesse Faculty of Engineering Mechatronics Department Beirut Lebanon MART Learning Education and Research Center Chananir Lebanon University of Applied Science Hamm-Lippstadt Department Lippstadt 2 Hardware Engineering Hamm59063 Germany University of Kassel Faculty of Electrical Engineering/Computer Science Department of Computer Architecture and System Programming Germany Centro de Tecnología Biomédica Universidad Politécnica de Madrid Lab of Bioinstrumentation and Nanomedicine Spain

ISBN: (纸本)9798350309959

Visually impaired people encounter several challenges in their mobility and navigation. Their daily activities are obstructed due to their inability to adapt or recognize accurately their surroundings, especially outside their house which they are more familiar with. Thus, it becomes the main reason of accidents, falling off, getting lost in unknown areas, etc. Furthermore, one of the sensory systems that helps the body to process data about the external environment is the visual system. Blind people also lose touch with the outside world, develops poor motor habits, which results in postural problems as a result. This project will assist visually impaired people in their daily life and simplify normal tasks through a system combining two previously designed projects, 'Smart Shoes for Blind and Visually Impaired People', and 'Human posture monitoring device'. The multifunctional system is developed with the goal of securing safe movements for visually impaired people as well as maintaining a good back posture by detecting leaning postures (LP). The purpose of the smart shoe is to identify obstacles and protect the user from unintended accidents. A compatible Android application has been developed to alert the user when there is an obstruction or when he is walking on a wet surface. Voice alarms will be used to acoustically alert the user. If the user collapses, a message with their position will be sent right away to a relative. On the other hand, the smart vest will identify the position of the user's back and alert him to maintain a straight posture through the same application as well. As the system is dealing with human health, some safety measurements would be taken into consideration to implement a safe electrical system in order to reduce error and to increase accuracy. © 2023 IEEE.

关键词： Accidents

Design and Implementation of a Smart Human Posture Monitoring Device Using a Wearable Vest 7

学校读者我要写书评

暂无评论

Design and Implementation of a Smart Human Posture Monitorin...

7th International Conference on Advances in Biomedical Engineering, ICABME 2023

作者： Daou, Roy Abi Zeid Geha, Elio Aoun, Youssef Hayek, Ali Boercsoek, Josef Olmedo, Jose Javier Serrano Université la Sagesse Biomedical Engineering Department Furn El Chebbak Beirut Lebanon Mart Learning Education and Research Center Chananir Lebanon Lebanese German University Faculty of Public Health Biomedical Technologies Department Jounieh Lebanon University of Applied Science Hamm-Lippstadt Department Lippstadt 2 Hardware Engineering Marker Allee 76-78 Hamm59063 Germany University of Kassel Faculty of Electrical Engineering /Computer Science Department of Computer Architecture and System Programming Germany Universidad Politecnica de Madrid Lab of Bioinstrumentation and Nanomedicine Centro de Tecnologia Biomedica Spain

ISBN: (纸本)9798350325850

Chronic back pain can present a serious health concern, with symptoms that can significantly affect an individual's well-being, mobility, and overall quality of life over an extended period. While chronic back pain may manifest suddenly in some cases, it often develops gradually and persists for weeks, and in untreated cases, it can linger for years. Hence, the utilization of assistive devices such as wearable posture-monitoring vests can offer valuable assistance and guidance to users. This research paper is dedicated to the development of a system for detecting, diagnosing, and correcting poor posture, specifically leaning posture. The vest is designed to provide users with visual, auditory, and tactile cues to help them address this issue, thereby reducing the risk associated with leaning. Additionally, an integrated electrical box has been designed to consolidate all components directly onto the main board in a secure enclosure. This box also displays the daily count of instances where the user has leaned. This system is characterized by its electrical safety, portability, compactness, comfort, and affordability. A comprehensive analysis of the system's performance has been conducted with a meticulous evaluation of accuracy. Each component of the system has undergone successful testing, and the system as a whole is currently in the testing phase. The results of these tests have indicated a lack of faulty errors and have demonstrated outstanding accuracy and detection rates. Over 100 individuals of varying ages, genders, and BMI categories were involved in testing, with each person wearing the device for an average of six hours. The accuracy rate achieved was 98.85%, with an average of 54.35 instances of poor posture detected per participant. © 2023 IEEE.

关键词： Wearable technology

Design and Implementation of a QoL Improvement system for Visually Impaired Patients

学校读者我要写书评

暂无评论

Design and Implementation of a QoL Improvement System for Vi...

International Conference on Advances in Computational Tools for Engineering Applications, ACTEA

作者： Roy Abi Zeid Daou Jad Beaino Johnny Hawat Ramy Ibrahim Ali Hayek Josef Boercsoek Jose Javier Serrano Olmedo Mechatronics department Faculty of Engineering Université La Sagesse Beirut Lebanon MART Learning Education and Research Center Chananir Lebanon Department Lippstadt 2 Hardware Engineering University of Applied Science Hamm-Lippstadt Hamm Department of Computer Architecture and System Programming Faculty of Electrical Engineering/Computer Science University of Kassel Germany Lab of Bioinstrumentation and Nanomedicine Centro de Tecnología Biomédica Universidad Politécnica de Madrid Spain

Visually impaired people encounter several challenges in their mobility and navigation. Their daily activities are obstructed due to their inability to adapt or recognize accurately their surroundings, especially outside their house which they are more familiar with. Thus, it becomes the main reason of accidents, falling off, getting lost in unknown areas, etc. Furthermore, one of the sensory systems that helps the body to process data about the external environment is the visual system. Blind people also lose touch with the outside world, develops poor motor habits, which results in postural problems as a result. This project will assist visually impaired people in their daily life and simplify normal tasks through a system combining two previously designed projects, “Smart Shoes for Blind and Visually Impaired People”, and “Human posture monitoring device”. The multifunctional system is developed with the goal of securing safe movements for visually impaired people as well as maintaining a good back posture by detecting leaning postures (LP). The purpose of the smart shoe is to identify obstacles and protect the user from unintended accidents. A compatible Android application has been developed to alert the user when there is an obstruction or when he is walking on a wet surface. Voice alarms will be used to acoustically alert the user. If the user collapses, a message with their position will be sent right away to a relative. On the other hand, the smart vest will identify the position of the user's back and alert him to maintain a straight posture through the same application as well. As the system is dealing with human health, some safety measurements would be taken into consideration to implement a safe electrical system in order to reduce error and to increase accuracy.

关键词：