检索结果-内蒙古大学图书馆

The Journal of machine learning Research 2021年第1期22卷 9024-9090页

作者： Jinshan Zeng Shao-Bo Lin Yuan Yao Ding-Xuan Zhou School of Computer and Information Engineering Jiangxi Normal University Nanchang China and Liu Bie Ju Centre for Mathematical Sciences City University of Hong Kong Hong Kong and Department of Mathematics Hong Kong University of Science and Technology Hong Kong Center of Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University Xi'an China Department of Mathematics Hong Kong University of Science and Technology Hong Kong School of Data Science and Department of Mathematics City University of Hong Kong Hong Kong

In this paper, we develop an alternating direction method of multipliers (ADMM) for deep neural networks training with sigmoid-type activation functions (called sigmoid-ADMM pair), mainly motivated by the gradient-free nature of ADMM in avoiding the saturation of sigmoid-type activations and the advantages of deep neural networks with sigmoid-type activations (called deep sigmoid nets) over their rectified linear unit (ReLU) counterparts (called deep ReLU nets) in terms of approximation. In particular, we prove that the approximation capability of deep sigmoid nets is not worse than that of deep ReLU nets by showing that ReLU activation function can be well approximated by deep sigmoid nets with two hidden layers and finitely many free parameters but not vice-verse. We also establish the global convergence of the proposed ADMM for the nonlinearly constrained formulation of the deep sigmoid nets training from arbitrary initial points to a Karush-Kuhn-Tucker (KKT) point at a rate of order O(1/k). Besides sigmoid activation, such a convergence theorem holds for a general class of smooth activations. Compared with the widely used stochastic gradient descent (SGD) algorithm for the deep ReLU nets training (called ReLU-SGD pair), the proposed sigmoid-ADMM pair is practically stable with respect to the algorithmic hyperparameters including the learning rate, initial schemes and the pro-processing of the input data. Moreover, we find that to approximate and learn simple but important functions the proposed sigmoid-ADMM pair numerically outperforms the ReLU-SGD pair.

关键词： deep learning ADMM sigmoid global convergence saturation avoidance

来源：评论

学校读者我要写书评

暂无评论

Homophily outlier detection in non-IID categorical data

arXiv

引用

arXiv 2021年

作者： Pang, Guansong Cao, Longbing Chen, Ling Australian Institute for Machine Learning University of Adelaide AdelaideSA5000 Australia University of Technology Sydney Australia Data Science Lab University of Technology Sydney SydneyNSW2007 Australia Center of Artificial Intelligence University of Technology Sydney SydneyNSW2007 Australia

Most of existing outlier detection methods assume that the outlier factors (i.e., outlierness scoring measures) of data entities (e.g., feature values and data objects) are Independent and Identically Distributed (IID). This assumption does not hold in real-world applications where the outlierness of different entities is dependent on each other and/or taken from different probability distributions (non-IID). This may lead to the failure of detecting important outliers that are too subtle to be identified without considering the non-IID nature. The issue is even intensified in more challenging contexts, e.g., high-dimensional data with many noisy features. This work introduces a novel outlier detection framework and its two instances to identify outliers in categorical data by capturing non-IID outlier factors. Our approach first defines and incorporates distribution-sensitive outlier factors and their interdependence into a value-value graph-based representation. It then models an outlierness propagation process in the value graph to learn the outlierness of feature values. The learned value outlierness allows for either direct outlier detection or outlying feature selection. The graph representation and mining approach is employed here to well capture the rich non-IID characteristics. Our empirical results on 15 real-world data sets with different levels of data complexities show that (i) the proposed outlier detection methods significantly outperform five state-of-the-art methods at the 95%/99% confidence level, achieving 10%-28% AUC improvement on the 10 most complex data sets;and (ii) the proposed feature selection methods significantly outperform three competing methods in enabling subsequent outlier detection of two different existing detectors. Copyright © 2021, The Authors. All rights reserved.

关键词： Clustering algorithms

来源：评论

学校读者我要写书评

暂无评论

Towards DMC accuracy across chemical space with scalable ∆-QML

arXiv

引用

arXiv 2022年

作者： Huang, Bing von Lilienfeld, O. Anatole Krogel, Jaron T. Benali, Anouar University of Vienna Faculty of Physics Kolingasse 14-16 Vienna1090 Austria Departments of Chemistry Materials Science and Engineering and Physics University of Toronto St. George Campus TorontoON Canada Vector Institute for Artificial Intelligence TorontoONM5S 1M1 Canada Machine Learning Group Technische Universität Berlin Institute for the Foundations of Learning and Data Berlin10587 Germany Materials Science and Technology Division Oak Ridge National Laboratory Oak RidgeTN37831 United States Computational Sciences Division Argonne National Laboratory ArgonneIL60439 United States

In the past decade, quantum diffusion Monte Carlo (DMC) has been demonstrated to successfully predict the energetics and properties of a wide range of molecules and solids by numerically solving the electronic many-body Schrödinger equation. We show that when coupled with quantum machine learning (QML) based surrogate methods the computational burden can be alleviated such that QMC shows clear potential to under-gird the formation of high quality descriptions across chemical space. We discuss three crucial approximations necessary to accomplish this: The fixed node approximation, universal and accurate references for chemical bond dissociation energies, and scalable minimal amons set based QML (AQML) models. Numerical evidence presented includes converged DMC results for over one thousand small organic molecules with up to 5 heavy atoms used as amons, and 50 medium sized organic molecules with 9 heavy atoms to validate the AQML predictions. Numerical evidence collected for ∆-AQML models suggests that already modestly sized QMC training data sets of amons suffice to predict total energies with near chemical accuracy throughout chemical space. Copyright © 2022, The Authors. All rights reserved.

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformers

arXiv

引用

arXiv 2021年

作者： Zhu, Tianyu Hiller, Markus Ehsanpour, Mahsa Ma, Rongkai Drummond, Tom Reid, Ian Rezatofighi, Hamid The Department of Electrical and Computer Systems Engineering Monash University Australia The School of Computing and Information Systems The University of Melbourne Australia The Australian Institute for Machine Learning The University of Adelaide Australia The Department of Data Science and AI Monash University Australia The Australian Centre for Robotic Vision Australia

Tracking a time-varying indefinite number of objects in a video sequence over time remains a challenge despite recent advances in the field. Most existing approaches are not able to properly handle multi-object tracking challenges such as occlusion, in part because they ignore long-term temporal information. To address these shortcomings, we present MO3TR: a truly end-to-end Transformer-based online multi-object tracking (MOT) framework that learns to handle occlusions, track initiation and termination without the need for an explicit data association module or any heuristics. MO3TR encodes object interactions into long-term temporal embeddings using a combination of spatial and temporal Transformers, and recursively uses the information jointly with the input data to estimate the states of all tracked objects over time. The spatial attention mechanism enables our framework to learn implicit representations between all the objects and the objects to the measurements, while the temporal attention mechanism focuses on specific parts of past information, allowing our approach to resolve occlusions over multiple frames. Our experiments demonstrate the potential of this new approach, achieving results on par with or better than the current state-of-the-art on multiple MOT metrics for several popular multi-object tracking benchmarks. © 2021, CC BY.

关键词： Tracking (position)

来源：评论

学校读者我要写书评

暂无评论

PAC-Bayes meta-learning with implicit task-specific posteriors

arXiv

引用

arXiv 2020年

作者： Nguyen, Cuong Do, Thanh-Toan Carneiro, Gustavo The Australian Institute for Machine Learning University of Adelaide SA5000 Australia The Department of Data Science and AI Faculty of Information Technology Monash University Australia

We introduce a new and rigorously-formulated PAC-Bayes meta-learning algorithm that solves few-shot learning. Our proposed method extends the PAC-Bayes framework from a single task setting to the meta-learning multiple task setting to upper-bound the error evaluated on any, even unseen, tasks and samples. We also propose a generative-based approach to estimate the posterior of task-specific model parameters more expressively compared to the usual assumption based on a multivariate normal distribution with a diagonal covariance matrix. We show that the models trained with our proposed meta-learning algorithm are well calibrated and accurate, with state-of-the-art calibration and classification results on few-shot classification (mini-ImageNet and tiered-ImageNet) and regression (multi-modal task-distribution regression) benchmarks. Copyright © 2020, The Authors. All rights reserved.

关键词： learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Transformer-based normative modelling for anomaly detection of early schizophrenia

arXiv

引用

arXiv 2022年

作者： Da Costa, Pedro F. Dafflon, Jessica Mendes, Sergio Leonardo Sato, João Ricardo Jorge Cardoso, M. Leech, Robert Jones, Emily J.H. Pinaya, Walter H.L. Institute of Psychiatry Psychology & Neuroscience King’s College London United Kingdom Centre for Brain and Cognitive Development Birkbeck College London United Kingdom Data Science and Sharing Team National Institute of Mental Health BethesdaMD United States Machine Learning Team National Institute of Mental Health BethesdaMD United States Center of Mathematics Computing and Cognition Universidade Federal do ABC Brazil School of Biomedical Engineering & Imaging Sciences King’s College London United Kingdom

Despite the impact of psychiatric disorders on clinical health, early-stage diagnosis remains a challenge. machine learning studies have shown that classifiers tend to be overly narrow in the diagnosis prediction task. The overlap between conditions leads to high heterogeneity among participants that is not adequately captured by classification models. To address this issue, normative approaches have surged as an alternative method. By using a generative model to learn the distribution of healthy brain data patterns, we can identify the presence of pathologies as deviations or outliers from the distribution learned by the model. In particular, deep generative models showed great results as normative models to identify neurological lesions in the brain. However, unlike most neurological lesions, psychiatric disorders present subtle changes widespread in several brain regions, making these alterations challenging to identify. In this work, we evaluate the performance of transformer-based normative models to detect subtle brain changes expressed in adolescents and young adults. We trained our model on 3D MRI scans of neurotypical individuals (N=1,765). Then, we obtained the likelihood of neurotypical controls and psychiatric patients with early-stage schizophrenia from an independent dataset (N=93) from the Human Connectome Project. Using the predicted likelihood of the scans as a proxy for a normative score, we obtained an AUROC of 0.82 when assessing the difference between controls and individuals with early-stage schizophrenia. Our approach surpassed recent normative methods based on brain age and Gaussian Process, showing the promising use of deep generative models to help in individualised analyses. © 2022, CC BY.

关键词： Brain

来源：评论

学校读者我要写书评

暂无评论

Accurate global machine learning force fields for molecules with hundreds of atoms

arXiv

引用

arXiv 2022年

作者： Chmiela, Stefan Vassilev-Galindo, Valentin Unke, Oliver T. Kabylda, Adil Sauceda, Huziel E. Tkatchenko, Alexandre Müller, Klaus-Robert Machine Learning Group Technische Universität Berlin Berlin10587 Germany Berlin Institute for the Foundations of Learning and Data – BIFOLD Germany Department of Physics and Materials Science University of Luxembourg Luxembourg CityL-1511 Luxembourg Google Research Brain Team Berlin Germany Departamento de Materia Condensada Instituto de Física Universidad Nacional Autónoma de México Cd. de MéxicoC.P. 04510 Mexico BASLEARN - TU Berlin BASF Joint Lab for Machine Learning Technische Universität Berlin Berlin10587 Germany Max Planck Institute for Informatics Stuhlsatzenhausweg Saarbrücken66123 Germany Department of Artificial Intelligence Korea University Anam-dong Seongbuk-gu Seoul02841 Korea Republic of

Global machine learning force fields (MLFFs), that have the capacity to capture collective many-atom interactions in molecular systems, currently only scale up to a few dozen atoms due a considerable growth of the model complexity with system size. For larger molecules, locality assumptions are typically introduced, with the consequence that non-local interactions are poorly or not at all described, even if those interactions are contained within the reference ab initio data. Here, we approach this challenge and develop an exact iterative parameter-free approach to train global symmetric gradient domain machine learning (sGDML) force fields for systems with up to several hundred atoms, without resorting to any localization of atomic interactions or other potentially uncontrolled approximations. This means that all atomic degrees of freedom remain fully correlated in the global sGDML FF, allowing the accurate description of complex molecules and materials that present phenomena with far-reaching characteristic correlation lengths. We assess the accuracy and efficiency of our MLFFs on a newly developed MD22 benchmark dataset containing molecules from 42 to 370 atoms. The robustness of our approach is demonstrated in nanosecond long path-integral molecular dynamics simulations for the supramolecular complexes in the MD22 dataset. Copyright © 2022, The Authors. All rights reserved.

关键词： machine learning

来源：评论

学校读者我要写书评

暂无评论

Alchemical harmonic approximation based potential for iso-electronic diatomics: Foundational baseline for ∆-machine learning

arXiv

引用

arXiv 2024年

作者： Krug, Simon León Khan, Danish von Lilienfeld, O. Anatole Machine Learning Group Technische Universität Berlin Berlin Charlottenburg 10587 Germany Vector Institute for Artificial Intelligence TorontoONM5S 1M1 Canada Department of Chemistry University of Toronto St. George campus TorontoONM5S 3H6 Canada Berlin Institute for the Foundations of Learning and Data Charlottenburg Berlin10587 Germany Acceleration Consortium University of Toronto 80 St George St TorontoONM5S 3H6 Canada Department of Materials Science and Engineering University of Toronto St. George campus TorontoONM5S 3E4 Canada Department of Physics University of Toronto St. George campus TorontoONM5S 1A7 Canada

We introduce the alchemical harmonic approximation (AHA) of the absolute electronic energy for charge-neutral iso-electronic diatomics at fixed interatomic distance d0. To account for variations in distance, we combine AHA with this Ansatz for the electronic binding potential, E(d) = (Eu − Es)(formula presented), where Eu, Ec, Es correspond to the energies of united atom, calibration at d0, and sum of infinitely separated atoms, respectively. Our model covers the two-dimensional electronic potential energy surface spanned by distances of 0.7 to 2.5 Åand differences in nuclear charge from which only one single point (with elements of nuclear charge Z1, Z2 and distance d0) is drawn to calibrate Ec. Using reference data from pbe0/cc-pVDZ, we present numerical evidence for the electronic ground-state of all neutral diatomics with 8, 10, 12, 14 electrons. We assess the validity of our model by comparison to legacy interatomic potentials (Harmonic oscillator, Lennard-Jones, and Morse) within the most relevant range of binding (0.7-2.5 Å), and find comparable accuracy if restricted to single diatomics, and significantly better predictive power when extrapolating to the entire iso-electronic series. We also investigated ∆-learning of the electronic absolute energy using our model as baseline. This baseline model results in a systematic improvement, effectively reducing training data needs for reaching chemical accuracy by up to an order of magnitude from ∼1000 to ∼100. By contrast, using AHA+Morse as a baseline hardly leads to any improvement, and sometimes even deteriorates the predictive power. Inferring the energy of unseen CO converges to a prediction error of ∼0.1 Ha in direct learning, and ∼0.04 Ha with our baseline. © 2024, CC BY.

关键词： Harmonic analysis

来源：评论

学校读者我要写书评

暂无评论

Reinforcement learning in continuous time and space: a stochastic control approach

The Journal of Machine Learning Research

引用

The Journal of machine learning Research 2020年第1期21卷 8145-8178页

作者： Haoran Wang Thaleia Zariphopoulou Xun Yu Zhou CAI Data Science and Machine Learning The Vanguard Group Inc. Malvern PA Department of Mathematics and IROM The University of Texas at Austin Austin TX and Oxford-Man Institute University of Oxford Oxford UK Department of Industrial Engineering and Operations Research The Data Science Institute Columbia University New York NY

We consider reinforcement learning (RL) in continuous time with continuous feature and action spaces. We motivate and devise an exploratory formulation for the feature dynamics that captures learning under exploration, with the resulting optimization problem being a revitalization of the classical relaxed stochastic control. We then study the problem of achieving the best trade-off between exploration and exploitation by considering an entropy-regularized reward function. We carry out a complete analysis of the problem in the linear-quadratic (LQ) setting and deduce that the optimal feedback control distribution for balancing exploitation and exploration is Gaussian. This in turn interprets the widely adopted Gaussian exploration in RL, beyond its simplicity for sampling. Moreover, the exploitation and exploration are captured respectively by the mean and variance of the Gaussian distribution. We characterize the cost of exploration, which, for the LQ case, is shown to be proportional to the entropy regularization weight and inversely proportional to the discount rate. Finally, as the weight of exploration decays to zero, we prove the convergence of the solution of the entropy-regularized LQ problem to the one of the classical LQ problem.

关键词： reinforcement learning entropy regularization stochastic control relaxed control linear-quadratic Gaussian distribution

来源：评论

学校读者我要写书评

暂无评论

Self-supervised sparse to dense motion segmentation

arXiv

引用

arXiv 2020年

作者： Kardoost, Amirhossein Ho, Kalun Ochs, Peter Keuper, Margret Data and Web Science Group University of Mannheim Germany Fraunhofer Center Machine Learning Germany Fraunhofer ITWM Competence Center HPC Kaiserslautern Germany Mathematical Optimization Group Saarland University Germany

Observable motion in videos can give rise to the definition of objects moving with respect to the scene. The task of segmenting such moving objects is referred to as motion segmentation and is usually tackled either by aggregating motion information in long, sparse point trajectories, or by directly producing per frame dense segmentations relying on large amounts of training data. In this paper, we propose a self supervised method to learn the densification of sparse motion segmentations from single video frames. While previous approaches towards motion segmentation build upon pre-training on large surrogate datasets and use dense motion information as an essential cue for the pixelwise segmentation, our model does not require pre-training and operates at test time on single frames. It can be trained in a sequence specific way to produce high quality dense segmentations from sparse and noisy input. We evaluate our method on the well-known motion segmentation datasets FBMS59 and DAVIS16. Copyright © 2020, The Authors. All rights reserved.

关键词： Motion analysis

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：