检索结果-内蒙古大学图书馆

Evolution and Recent Trends for the SGD Algorithm: Key Takeaways from an Educational Short Course

学校读者我要写书评

暂无评论

Evolution and Recent Trends for the SGD Algorithm: Key Takea...

European Signal Processing Conference (EUSIPCO)

作者： Paul Rodriguez Department of Electrical Engineering Pontificia Universidad Católica del Perú Lima Peru

ISBN: (数字)9789464593617

ISBN: (纸本)9798331519773

One of the key challenges when designing a ten (10) hours educational short course, entitled “A Hands-on Approach for Implementing Stochastic Optimization algorithms from Scratch”, which was accepted for inclusion at ICASSP'23, was related to addressing “how to introduce the Stochastic Gradient Descent (SGD) algorithm and variants in a consistent, accessible fashion”? From a simplistic perspective, the SGD algorithm is nothing else than the classical gradient descent (GD) algorithm along with a (very) noisy gradient. Nonetheless, arguably, SGD's most influential variants, e.g. AdaGrad, RMSprop and Adam, nor more recent ones (LookAhead, E-Adam, MadGrad, among several others) may not be explained in such superficial terms. Moreover, such variants are usually given as as black-boxes by most deep-learning (DL) libraries (e.g. TensorFlow, PyTorch, etc.). In this article, based on the experience of the aforementioned short-course, I propose to link the SGD algorithm and variants via an “evolutionary path”, in which each SGD variant may be understood as a set of add-on features over the vanilla SGD, resulting in a generalized algorithm along with a “family tree” graph which are both intuitive and useful when implementing a given SGD variant.

关键词： Geometry Signal processing algorithms Europe Closed box Signal processing Market research approximation algorithms Libraries Noise measurement Optimization

Filtering and smoothing estimation algorithms from uncertain nonlinear observations with time-correlated additive noise and random deception attacks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Caballero-Águila, R. Hu, J. Linares-Pérez, J. Departamento de Estadística Universidad de Jaén Paraje Las Lagunillas. Jaén23071 Spain Key Laboratory of Advanced Manufacturing and Intelligent Technology Ministry of Education Harbin University of Science and Technology Harbin150080 China Departamento de Estadística Universidad de Granada Avenida Fuentenueva. Granada18071 Spain

This paper discusses the problem of estimating a stochastic signal from nonlinear uncertain observations with time-correlated additive noise described by a first-order Markov process. Random deception attacks are assumed to be launched by an adversary, and both this phenomenon and the uncertainty in the observations are modelled by two sets of Bernoulli random variables. Under the assumption that the evolution model generating the signal to be estimated is unknown and only the mean and covariance functions of the processes involved in the observation equation are available, recursive algorithms based on linear approximations of the real observations are proposed for the least-squares filtering and fixed-point smoothing problems. Finally, the feasibility and effectiveness of the developed estimation algorithms are verified by a numerical simulation example, where the impact of uncertain observation and deception attack probabilities on estimation accuracy is evaluated. Copyright © 2024, The Authors. All rights reserved.

关键词： approximation algorithms

In-Depth Examination of Student Attrition in Higher Education Through Decision Tree, Random Forest, and XGBoost algorithms

学校读者我要写书评

暂无评论

In-Depth Examination of Student Attrition in Higher Educatio...

International Conference on Electrical Engineering, Computer Science and Informatics (EECSI)

作者： Clement Aurelio Johan Setiawan Faculty of Engineering and Informatics Information Systems Study Universitas Multimedia Nusantara Tangerang Indonesia

ISBN: (数字)9798350355314

ISBN: (纸本)9798350355321

Customer Customer churn, a critical metric for businesses reliant on customer loyalty, involves the loss of customers from a company. This study examines churn among college and university students, leading to financial losses, decreased accreditation, and diminished reputations for educational institutions, ultimately reducing their appeal to prospective students. To identify factors contributing to student churn and classify students as likely to churn or not, this research utilizes Decision Tree, Random Forest, and XGBoost algorithms. These classification algorithms, with XGBoost and Random Forest enhancing Decision Trees through boosting and bagging techniques, respectively, are well-suited for the task. The data was sourced from the Academic Information Bureau and processed before model development. The model with the best performance will predict student churn on new, unseen data. Findings indicate varying performance metrics for each model, including accuracy, precision, recall, and F1-score. The XGBoost model excelled, achieving an accuracy of 98%, precision of 99%, recall of 96%, and an F1-score of 97%, significantly improving over the base Decision Tree model across all metrics. Notably, all models showed better performance in predicting the majority class (not churn) compared to the minority class (churn).

关键词： Measurement Accuracy Computational modeling Predictive models approximation algorithms Prediction algorithms Data models Classification algorithms Decision trees Random forests

Proportionally Fair Matching via Randomized Roundin

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Duppala, Sharmila Grammel, Nathaniel Luque, Juan MacRury, Calum Srinivasan, Aravind University of Maryland College Park College ParkMD United States Columbia University New YorkNY United States

Given an edge-colored graph, the goal of the proportional fair matching problem is to find a maximum weight matching while ensuring proportional representation (with respect to the number of edges) of each color. The colors may correspond to demographic groups or other protected traits where we seek to ensure roughly equal representation from each group. It is known that, assuming ETH, it is impossible to approximate the problem with colors in time 2o()nO(1) (i.e., subexponential in ) even on unweighted path graphs. Further, even determining the existence of a non-empty matching satisfying proportionality is NP-Hard. To overcome this hardness, we relax the stringent proportional fairness constraints to a probabilistic notion. We introduce a notion we call δ-ProbablyFair, where we ensure proportionality up to a factor of at most (1 ± δ) for some small δ > 0 with high probability. The violation δ can be brought arbitrarily close to 0 for some good instances with large values of matching size. We propose and analyze simple and fast algorithms for bipartite graphs that achieve constant-factor approximation guarantees, and return a δ-ProbablyFair matching. Copyright © 2024, The Authors. All rights reserved.

关键词： approximation algorithms

A sublinear-time randomized algorithm for column and row subset selection based on strong rank-revealing QR factorizations

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cortinovis, Alice Ying, Lexing Department of Mathematics Stanford University CA United States Department of Mathematics and ICME Stanford University CA United States

In this work, we analyze a sublinear-time algorithm for selecting a few rows and columns of a matrix for low-rank approximation purposes. The algorithm is based on an initial uniformly random selection of rows and columns, followed by a refinement of this choice using a strong rank-revealing QR factorization. We prove bounds on the error of the corresponding low-rank approximation (more precisely, the CUR approximation error) when the matrix is a perturbation of a low-rank matrix that can be factorized into the product of matrices with suitable incoherence and/or sparsity *** Codes 65F30 (Primary) 15A23 (Secondary) Copyright © 2024, The Authors. All rights reserved.

关键词： approximation algorithms

Exact Minimum Weight Spanners via Column Generation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bökler, Fritz Chimani, Markus Jasper, Henning Wagner, Mirko H. Institute of Computer Science Osnabrück University Germany

Given a weighted graph G, a minimum weight α-spanner is a least-weight subgraph H ⊆ G that preserves minimum distances between all node pairs up to a factor of α. There are many results on heuristics and approximation algorithms, including a recent investigation of their practical performance [20]. Exact approaches, in contrast, have long been denounced as impractical: The first exact ILP (integer linear program) method [48] from 2004 is based on a model with exponentially many path variables, solved via column generation. A second approach [2], modeling via arc-based multicommodity flow, was presented in 2019. In both cases, only graphs with 40-100 nodes were reported to be solvable. In this paper, we briefly report on a theoretical comparison between these two models from a polyhedral point of view, and then concentrate on improvements and engineering aspects. We evaluate their performance in a large-scale empirical study. We report that our tuned column generation approach, based on multicriteria shortest path computations, is able to solve instances with over 16 000 nodes within 13 min. Furthermore, now knowing optimal solutions for larger graphs, we are able to investigate the quality of the strongest known heuristic on reasonably sized instances for the first *** Codes 68R10 (Primary) 05C85, 90C11 (Secondary) © 2024, CC BY.

关键词： approximation algorithms

InGo: In-Network Aggregation Routing with Batch Size Adjustment for Distributed Training

学校读者我要写书评

暂无评论

InGo: In-Network Aggregation Routing with Batch Size Adjustm...

International Workshop on Quality of Service

作者： Jianfeng Bao Gongming Zhao Hongli Xu Haibo Wang Peng Yang School of Computer Science and Technology University of Science and Technology of China Suzhou Institute for Advanced Research University of Science and Technology of China University of Kentucky USA

ISBN: (数字)9798350350128

ISBN: (纸本)9798350350135

Distributed training has emerged as a critical application in clusters due to the widespread adoption of AI technology across various domains. However, as distributed training continues to advance, it has become increasingly time-consuming. To address this challenge, researchers have explored leveraging In-Network Aggregation (INA) to expedite distributed model training. Specifically, by harnessing programmable hardware, such as Intel Tofino switches, INA can aggregate gradients within the network, thereby reducing the amount of gradient transmission and accelerating distributed training. However, previous works assume fixed routing selection and batch size, ignoring their impact on model convergence and resulting in extended completion time. To bridge this gap, we propose InGo, a pioneering approach that considers both in-network aggregation routing and batch size adjustment, and provide the rigorous convergence analysis. Then, we formally define the problem of in-network aggregation routing with batch size adjustment, and present an efficient algorithm with bounded approximation factors to solve this problem. Through extensive experiments on both physical platforms and simulated environments, we demonstrate that InGo significantly reduces the completion time by 25.2%-74.7% compared to state-of-the-art solutions.

关键词： Training Bridges Analytical models Aggregates Quality of service Routing approximation algorithms

Estimation of Time-Varying Underwater Acoustic Channel Via Dynamic Proximal Gradient Algorithm

学校读者我要写书评

暂无评论

Estimation of Time-Varying Underwater Acoustic Channel Via D...

OCEANS

作者： Yunfeng Hu Jun Tao Key Laboratory of Underwater Acoustic Signal Processing of the Ministry of Education School of Information Science and Engineering Southeast University Nanjing China Pengcheng Lab Shenzhen China

ISBN: (数字)9798350362077

ISBN: (纸本)9798350362084

Dynamic compressed sensing (DCS) techniques have been applied to enhance the performance of channel estimation (CE) in underwater acoustic (UWA) communications. Existing DCS-CE schemes are mainly applicable to slow-varying channels and solutions working under rapid channel conditions are highly desirable. In this paper, tracking of time-varying sparse UWA channel is formulated as a $\ell_{p}$ -norm regularized recursive least square (RLS) problem, which is then solved via a proximal gradient (PG) algorithm. The resulting CE scheme is named the dynamic PG (DPG) CE method. Experimental results verified the advantage of the proposed DPG CE scheme over existing sparse CE schemes.

关键词： Acoustic communication (telecommunication) Heuristic algorithms Oceans Channel estimation Estimation approximation algorithms Sensors

Solving Influence Maximization Problem Under Deterministic Linear Threshold Model Using Metaheuristic optimization

学校读者我要写书评

暂无评论

Solving Influence Maximization Problem Under Deterministic L...

Proceedings of the International Convention MIPRO

作者： A. Andreev K. Chukharev S. Kochemazov A. Semenov ITMO University St. Petersburg Russia

ISBN: (数字)9798350382501

ISBN: (纸本)9798350382518

In the paper we consider the discrete variant of the well-known Influence Maximization Problem (IMP). Given some influence model, it consists in finding a so-called seed set of influential users of fixed size, that maximizes the total spread of influence over the network. We limit our study to the influence model called Deterministic Linear Threshold Model (DLTM). It is well known that IMP under DLTM is computationally hard and there are no approximate algorithms for its solving with a constant approximation ratio if $P\neq NP$. Therefore, it makes sense to apply metaheuristic algorithms to this problem. In the present research we propose new algorithms for solving IMP under DLTM, which are based on a technique that combines evolutionary and genetic strategies for pseudo-Boolean optimization with a greedy algorithm which is used to find some initial approximation. We use the proposed strategy to solve another well-known combinatorial problem for networks called Target Set Selection (TSS). We propose to solve TSS as a sequence of IMPs with gradually decreasing of target set size. In the experimental part of the paper we demonstrate that our new strategy outperforms the previous ways to solve TSS, yielding smaller target sets of good quality.

关键词： Greedy algorithms Computational modeling Metaheuristics Evolutionary computation approximation algorithms Hypercubes Genetics