检索结果-内蒙古大学图书馆

Proceedings of the 41st International Conference on machine learning

作者： Anton Rodomanov Ali Kavis Yongtao Wu Kimon Antonakopoulos Volkan Cevher CISPA Helmholtz Center for Information Security Saarbrücken Germany Institute for Foundations of Machine Learning (IFML) UT Austin Texas Laboratory for Information and Inference Systems (LIONS) EPFL Lausanne Switzerland

We develop universal gradient methods for Stochastic Convex Optimization (SCO). Our algorithms automatically adapt not only to the oracle's noise but also to the Hölder smoothness of the objective function without a priori knowledge of the particular setting. The key ingredient is a novel strategy for adjusting step-size coefficients in the Stochastic Gradient Method (SGD). Unlike AdaGrad, which accumulates gradient norms, our Universal Gradient Method accumulates appropriate combinations of gradient- and iterate differences. The resulting algorithm has state-of-the-art worst-case convergence rate guarantees for the entire Hölder class including, in particular, both nonsmooth functions and those with Lipschitz continuous gradient. We also present the Universal Fast Gradient Method for SCO enjoying optimal efficiency estimates.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Test-Time Certifiable Self-Supervision to Bridge the Sim2Real Gap in Event-Based Satellite Pose Estimation

Test-Time Certifiable Self-Supervision to Bridge the Sim2Rea...

引用

IEEE/RSJ International Conference on Intelligent Robots and systems (IROS)

作者： Mohsi Jawaid Rajat Talak Yasir Latif Luca Carlone Tat-Jun Chin Sentient Satellites Laboratory (SSL) Australian Institute for Machine Learning (AIML) The University of Adelaide SA Australia Laboratory for Information and Decision Systems (LIDS) Massachusetts Institute of Technology Cambridge MA USA

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Deep learning plays a critical role in vision-based satellite pose estimation. However, the scarcity of real data from the space environment means that deep models need to be trained using synthetic data, which raises the Sim2Real domain gap problem. A major cause of the Sim2Real gap are novel lighting conditions encountered during test time. Event sensors have been shown to provide some robustness against lighting variations in vision-based pose estimation. However, challenging lighting conditions due to strong directional light can still cause undesirable effects in the output of commercial off-the-shelf event sensors, such as noisy/spurious events and inhomogeneous event densities on the object. Such effects are non-trivial to simulate in software, thus leading to Sim2Real gap in the event domain. To close the Sim2Real gap in event-based satellite pose estimation, the paper proposes a test-time self-supervision scheme with a certifier module. Self-supervision is enabled by an optimisation routine that aligns a dense point cloud of the predicted satellite pose with the event data to attempt to rectify the inaccurately estimated pose. The certifier attempts to verify the corrected pose, and only certified test-time inputs are backpropagated via implicit differentiation to refine the predicted landmarks, thus improving the pose estimates and closing the Sim2Real gap. Results show that the our method outperforms established test-time adaptation schemes.

关键词： Training Satellites Event detection Pose estimation Pipelines Lighting Software Robustness Intelligent sensors Synthetic data

来源：评论

学校读者我要写书评

暂无评论

On the generalization of stochastic gradient descent with momentum

The Journal of Machine Learning Research

引用

The Journal of machine learning Research 2024年第1期25卷 1132-1187页

作者： Ali Ramezani-Kebrya Kimon Antonakopoulos Volkan Cevher Ashish Khisti Ben Liang Department of Informatics University of Oslo and Visual Intelligence Centre Integreat Norwegian Centre for Knowledge-driven Machine Learning Oslo Norway Laboratory for Information and Inference Systems (LIONS) EPFL EPFL STI IEL LIONS Lausanne Switzerland Department of Electrical and Computer Engineering University of Toronto Toronto ON Canada

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that there exists a convex loss function for which the stability gap for multiple epochs of SGD with standard heavy-ball momentum (SGDM) becomes unbounded. Then, for smooth Lipschitz loss functions, we analyze a profiles momentum-based update rule, i.e., SGD with early momentum (SGDEM) under a broad range of step-sizes, and show that it can train machine learning models for multiple epochs with a guarantee for generalization. Finally, for the special case of strongly convex loss functions, we find a range of momentum such that multiple epochs of standard SGDM, as a special form of SGDEM, also generalizes. Extending our results on generalization, we also develop an upper bound on the expected true risk, in terms of the number of training steps, sample size, and momentum. Our experimental evaluations verify the consistency between the numerical results and our theoretical bounds. SGDEM improves the generalization error of SGDM when training ResNet-18 on ImageNet in practical distributed settings.

关键词： uniform stability generalization error heavy-ball momentum stochastic gradient descent non-convex

来源：评论

学校读者我要写书评

暂无评论

Change-Point Detection in Industrial Data Streams based on Online Dynamic Mode Decomposition with Control

arXiv

引用

arXiv 2024年

作者： Wadinger, Marek Kvasnica, Michal Kawahara, Yoshinobu Institute of Information Engineering Automation and Mathematics Slovak University of Technology in Bratislava Radlinskeho 9 Bratislava812 37 Slovakia Machine Learning & Systems Laboratory Osaka University 1–5 Yamadaoka Osaka Suita565–0871 Japan

We propose a novel change-point detection method based on online Dynamic Mode Decomposition with control (ODMDwC). Leveraging ODMDwC’s ability to find and track linear approximation of a non-linear system while incorporating control effects, the proposed method dynamically adapts to its changing behavior due to aging and seasonality. This approach enables the detection of changes in spatial, temporal, and spectral patterns, providing a robust solution that preserves correspondence between the score and the extent of change in the system dynamics. We formulate a truncated version of ODMDwC and utilize higher-order time-delay embeddings to mitigate noise and extract broad-band features. Our method addresses the challenges faced in industrial settings where safety-critical systems generate non-uniform data streams while requiring timely and accurate change-point detection to protect profit and life. Our results demonstrate that this method yields intuitive and improved detection results compared to the Singular-Value-Decomposition-based method. We validate our approach using synthetic and real-world data, showing its competitiveness to other approaches on complex systems’ benchmark datasets. Provided guidelines for hyperparameters selection enhance our method’s practical applicability. © 2024, CC BY.

关键词： Dynamic mode decomposition

来源：评论

学校读者我要写书评

暂无评论

Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent 37

Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Sto...

引用

37th Conference on Neural information Processing systems, NeurIPS 2023

作者： Zhu, Lingjiong Gürbüzbalaban, Mert Raj, Anant Şimşekli, Umut Dept. of Mathematics Florida State University United States Dept. of Management Science and Information Systems Rutgers Business School United States Center for Statistics and Machine Learning Princeton University United States Coordinated Science Laboratory University of Illinois Urbana-Champaign United States Inria Paris CNRS Ecole Normale Supérieure PSL Research University France

ISBN: (纸本)9781713899921

Algorithmic stability is an important notion that has proven powerful for deriving generalization bounds for practical algorithms. The last decade has witnessed an increasing number of stability bounds for different algorithms applied on different classes of loss functions. While these bounds have illuminated various properties of optimization algorithms, the analysis of each case typically required a different proof technique with significantly different mathematical tools. In this study, we make a novel connection between learning theory and applied probability and introduce a unified guideline for proving Wasserstein stability bounds for stochastic optimization algorithms. We illustrate our approach on stochastic gradient descent (SGD) and we obtain time-uniform stability bounds (i.e., the bound does not increase with the number of iterations) for strongly convex losses and non-convex losses with additive noise, where we recover similar results to the prior art or extend them to more general cases by using a single proof technique. Our approach is flexible and can be generalizable to other popular optimizers, as it mainly requires developing Lyapunov functions, which are often readily available in the literature. It also illustrates that ergodicity is an important component for obtaining time-uniform bounds - which might not be achieved for convex or non-convex losses unless additional noise is injected to the iterates. Finally, we slightly stretch our analysis technique and prove time-uniform bounds for SGD under convex and non-convex losses (without additional additive noise), which, to our knowledge, is novel. © 2023 Neural information processing systems foundation. All rights reserved.

关键词： Additive noise

来源：评论

学校读者我要写书评

暂无评论

Analysis of the Research Status of information Technology Education Literature in China and Abroad 24

Analysis of the Research Status of Information Technology Ed...

引用

24th IEEE International Conference on High Performance Computing and Communications, 8th IEEE International Conference on Data Science and systems, 20th IEEE International Conference on Smart City and 8th IEEE International Conference on Dependability in Sensor, Cloud and Big Data systems and Application, HPCC/DSS/SmartCity/DependSys 2022

作者： Zhao, Yulin Liu, Kai Li, Junke School of Information Engineering Suqian University Jiangsu Suqian223800 China School of Computer and Information Qiannan Normal University for Nationalities Duyun558000 China Jiangsu Province Engineering Research Center of Smart Poultry Farming and Intelligent Equipment Jiangsu 223800 China Key Laboratory of Machine Learning and Unstructured Data Processing of Qiannan Duyun558000 China Key Laboratory of Complex Systems and Intelligent Optimization of Guizhou Duyun558000 China

ISBN: (纸本)9798350319934

information technology education contributes to the development of national education. Understanding the research status of information technology education at home and abroad is helpful to the implementation of education informatization. This paper takes the information technology education research literature collected by CNKI and the Web of Science database as the object and uses the bibliometric method to analyze the literature characteristics of information technology education at home and abroad. Specifically, the paper mainly carries out statistical analysis from the time distribution of literature, journal distribution, and hot keywords, and visually describes the literature features of the information technology education discipline, providing references for the research of information technology education discipline at home and abroad. © 2022 IEEE.

关键词： Informatization

来源：评论

学校读者我要写书评

暂无评论

The Spatial-Temporal Evolution and Influencing Factors of Public Attention to 5G: Empirical Analysis Based on Baidu Index from 2011 to 2021

引用

Mathematical Problems in Engineering 2023年第1期2023卷

作者： Liu, Pan Wang, Guanyu Liu, Kai Li, Junke Chen, Yanlin School of Information Engineering Suqian University Jiangsu Suqian223800 China School of Computer and Information Qiannan Normal University for Nationalities Guizhou Duyun558000 China Key Laboratory of Machine Learning and Unstructured Data Processing of Qiannan Guizhou Duyun558000 China Key Laboratory of Complex Systems and Intelligent Optimization of Guizhou Duyun558000 China

The development of 5G (fifth-generation wireless systems) determines the future direction of technology and economy and has received extensive public attention. Studying the changing rules of public attention to 5G can provide an important guiding significance for the sustainable development of 5G. This paper takes Baidu Index as the measurement index of 5G public attention and analyzes the spatial and temporal evolution characteristics and influencing factors of public attention to 5G from 2011 to 2021 by using the elasticity coefficient, Gini coefficient, geographical concentration index, and panel data model. The results of the study show the following. (1) The public concern to 5G is generally on the rise, but the heat has declined in the past two years. (2) The public’s 5G attention shows a seasonal effect, with the highest attention in March and June. (3) The spatial difference of 5G public attention is obvious. The eastern region has a high degree of attention, the internal differences between the eastern and western regions are obvious, and the central region is relatively balanced. (4) The factors such as local economic level, education level, Internet development, and media attention have significantly affected the public focus on 5G. Also, some suggestions are made for the sustainable development of 5G and the planning of 6G (sixth-generation wireless systems). Copyright © 2023 Pan Liu et al.

关键词： Sustainable development

来源：评论

学校读者我要写书评

暂无评论

GSLB: The Graph Structure learning Benchmark 37

GSLB: The Graph Structure Learning Benchmark

引用

37th Conference on Neural information Processing systems, NeurIPS 2023

作者： Li, Zhixun Wang, Liang Sun, Xin Luo, Yifan Zhu, Yanqiao Chen, Dingshuo Luo, Yingtao Zhou, Xiangxin Liu, Qiang Wu, Shu Yu, Jeffrey Xu Department of Systems Engineering and Engineering Management The Chinese University of Hong Kong Hong Kong Center for Research on Intelligent Perception and Computing State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences China School of Artificial Intelligence University of Chinese Academy of Sciences China Department of Automation University of Science and Technology of China China School of Cyberspace Security Beijing University of Posts and Telecommunications China Department of Computer Science University of California Los Angeles United States Heinz College of Information Systems and Public Policy Machine Learning Department School of Computer Science Carnegie Mellon University United States

ISBN: (纸本)9781713899921

Graph Structure learning (GSL) has recently garnered considerable attention due to its ability to optimize both the parameters of Graph Neural Networks (GNNs) and the computation graph structure simultaneously. Despite the proliferation of GSL methods developed in recent years, there is no standard experimental setting or fair comparison for performance evaluation, which creates a great obstacle to understanding the progress in this field. To fill this gap, we systematically analyze the performance of GSL in different scenarios and develop a comprehensive Graph Structure learning Benchmark (GSLB) curated from 20 diverse graph datasets and 16 distinct GSL algorithms. Specifically, GSLB systematically investigates the characteristics of GSL in terms of three dimensions: effectiveness, robustness, and complexity. We comprehensively evaluate state-of-the-art GSL algorithms in node- and graph-level tasks, and analyze their performance in robust learning and model complexity. Further, to facilitate reproducible research, we have developed an easy-to-use library for training, evaluating, and visualizing different GSL methods. Empirical results of our extensive experiments demonstrate the ability of GSL and reveal its potential benefits on various downstream tasks, offering insights and opportunities for future research. The code of GSLB is available at: https://***/GSL-Benchmark/GSLB. © 2023 Neural information processing systems foundation. All rights reserved.

关键词： Graphic methods

来源：评论

学校读者我要写书评

暂无评论

Applying recurrence time entropy to identify changes in event-related potentials

引用

The European Physical Journal Special Topics 2022年第1期232卷 161-168页

作者： Frolov, Nikita Pitsik, Elena Maksimenko, Vladimir Hramov, Alexander Laboratory of Dynamics in Biological Systems KU Leuven Leuven Belgium Center for Neurotechnology and Machine Learning Immanuel Kant Baltic Federal University Kaliningrad Russia Neuroscience and Cognitive Technology Laboratory Innopolis University Innopolis Russia Institute of Information Technology Mathematics and Mechanics Lobachevsky State University of Nizhny Novgorod Nizhny Novgorod Russia

The event-related potentials (ERPs) are an essential response of the human brain to environmental changes that correlate with behavior. They are thus widely used as indicators of brain activity in fundamental research and response sources in brain communication devices. The problem of their robust identification from single-trial EEG recordings or limited data sets is timely and challenging. The current study addresses this issue by evaluating the ERP-associated variations of EEG signals using the measures of complexity based on the recurrence quantification analysis (RQA). Specifically, we demonstrate that the recurrence time entropy (RTE) is a good indicator of ERP-associated changes in the course of successive discrimination of ambiguous visual stimuli. Using the distribution of recurrence times, we conclude why exactly this measure is sensitive to ERP-associated variations of EEG signal.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Prognostic Prediction for Non-small-Cell Lung Cancer Based on Deep Neural Network and Multimodal Data 17th

Prognostic Prediction for Non-small-Cell Lung Cancer Based o...

引用

17th International Conference on Intelligent Computing, ICIC 2021

作者： Zhang, Zhong-Si Xu, Fei Jiang, Han-Jing Chen, Zhan-Heng Key Laboratory of Image Information Processing and Intelligent Control of Education Ministry of China Institute of Artificial Intelligence School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Institute of Machine Learning and Systems Biology School of Electronics and Information Engineering Tongji University No. 4800 Caoan Road Shanghai201804 China

ISBN: (纸本)9783030845315

Non-small-cell lung cancer (NSCLC) is the most common lung cancer with poor prognosis. Prognostic prediction is significant in improving the prognosis of NSCLC patients. Clinical information and multi-omics data including gene expression, miRNA, copy number variations, and DNA methylation are closely related to NSCLC prognosis. In this study, we propose a deep neural network to conduct prognostic prediction for NSCLC patients based on all five types of data. Given the high dimensional features of the omics data, past works reduce the feature dimension by regression analysis or correlation sorting algorithms. A shortcoming of these methods is that only a small number of features are considered. To overcome it, we propose a convolution neural network-based feature transformation method, which considers all features and extracts the abstract representations of the omics data. Based on the representations, we predict the five-year survival status of NSCLC patients. The results show that our method achieves a more precise prediction than previous work, with the areas under the curve improved from 0.8163 to 0.8508 and the accuracy improved from 0.7544 to 0.8096. © 2021, Springer Nature Switzerland AG.

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：