检索结果-内蒙古大学图书馆

DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus Algorithm

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ding, Lisang Jin, Kexin Ying, Bicheng Yuan, Kun Yin, Wotao Department of Mathematics University of California Los AngelesCA United States Department of Mathematics Princeton University PrincetonNJ United States Google Inc. Los AngelesCA United States Center for Machine Learning Research Peking University Beijing China AI for Science Institute Beijing China National Engineering Labratory for Big Data Analytics and Applications Beijing China Decision Intelligence Lab. Alibaba US BellevueWA United States

Decentralized Stochastic Gradient Descent (SGD) is an emerging neural network training approach that enables multiple agents to train a model collaboratively and simultaneously. Rather than using a central parameter server to collect gradients from all the agents, each agent keeps a copy of the model parameters and communicates with a small number of other agents to exchange model updates. Their communication, governed by the communication topology and gossip weight matrices, facilitates the exchange of model updates. The state-of-the-art approach uses the dynamic one-peer exponential-2 topology, achieving faster training times and improved scalability than the ring, grid, torus, and hypercube topologies. However, this approach requires a power-of-2 number of agents, which is impractical at scale. In this paper, we remove this restriction and propose Decentralized SGD with Communication-optimal Exact Consensus Algorithm (DSGD-CECA), which works for any number of agents while still achieving state-of-the-art properties. In particular, DSGD-CECA incurs a unit per-iteration communication overhead and an Õ(n3) transient iteration complexity. Our proof is based on newly discovered properties of gossip weight matrices and a novel approach to combine them with DSGD’s convergence analysis. Numerical experiments show the efficiency of DSGD-CECA. Copyright © 2023, The Authors. All rights reserved.

关键词： Stochastic systems

Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ghojogh, Benyamin Ghodsi, Ali Karray, Fakhri Crowley, Mark Department of Electrical and Computer Engineering Machine Learning Laboratory University of Waterloo WaterlooON Canada Department of Statistics and Actuarial Science David R. Cheriton School of Computer Science Data Analytics Laboratory University of Waterloo WaterlooON Canada Department of Electrical and Computer Engineering Centre for Pattern Analysis and Machine Intelligence University of Waterloo WaterlooON Canada

This is a tutorial and survey paper on factor analysis, probabilistic Principal Component Analysis (PCA), variational inference, and Variational Autoencoder (VAE). These methods, which are tightly related, are dimensionality reduction and generative models. They assume that every data point is generated from or caused by a lowdimensional latent factor. By learning the parameters of distribution of latent space, the corresponding low-dimensional factors are found for the sake of dimensionality reduction. For their stochastic and generative behaviour, these models can also be used for generation of new data points in the data space. In this paper, we first start with variational inference where we derive the Evidence Lower Bound (ELBO) and Expectation Maximization (EM) for learning the parameters. Then, we introduce factor analysis, derive its joint and marginal distributions, and work out its EM steps. Probabilistic PCA is then explained, as a special case of factor analysis, and its closed-form solutions are derived. Finally, VAE is explained where the encoder, decoder and sampling from the latent space are introduced. Training VAE using both EM and backpropagation are explained. Copyright © 2021, The Authors. All rights reserved.

关键词： Maximum principle

Towards self-driving laboratories: The central role of density functional theory in the AI age

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Bing von Rudorff, Guido Falk Anatole von Lilienfeld, O. University of Vienna Faculty of Physics Kolingasse 14-16 WienAT1090 Austria University Kassel Department of Chemistry Heinrich-Plett-Str.40 Kassel34132 Germany Heinrich-Plett-Straße 40 Kassel34132 Germany Vector Institute for Artificial Intelligence TorontoONM5S 1M1 Canada Department of Chemistry Materials Science and Engineering and Physics University of Toronto St. George Campus TorontoON Canada Machine Learning Group Technische Universität Berlin Berlin Institute for the Foundations of Learning and Data Berlin10587 Germany

Density functional theory (DFT) plays a pivotal role for the chemical and materials science due to its relatively high predictive power, applicability, versatility and computational efficiency. We review recent progress in machine learning model developments which has relied heavily on density functional theory for synthetic data generation and for the design of model architectures. The general relevance of these developments is placed in some broader context for the chemical and materials sciences. Resulting in DFT based machine learning models with high efficiency, accuracy, scalability, and transferability (EAST), recent progress indicates probable ways for the routine use of successful experimental planning software within self-driving laboratories. © 2023, CC0.

关键词： Density functional theory

Effective resampling approach for skewed distribution on imbalanced data set

学校读者我要写书评

暂无评论

IAENG International Journal of Computer Science

IAENG International Journal of Computer science 2020年第2期47卷 234-249页

作者： Nwe, Mar Mar Lynn, Khin Thidar Data Mining and Machine Learning Lab University of Computer Studies Mandalay Myanmar Faculty of Information Science Department University of Computer Studies Mandalay Myanmar

Accurate classification of unknown input data for imbalanced data sets is difficult, because the predictions of learning classifiers tend to be biased towards the majority class and ignore the minority class. Moreover, the class distribution of imbalanced data has a significant impact on the misclassification rate of the learning classifier. So, this paper introduces an effective data pre-processing approach to improve the efficiency of imbalanced data classification, focusing on the skewed distribution of data points in the imbalanced data set. This proposed approach involves over-sampling and under-sampling techniques based on k-means clustering to overcome the problems associated with imbalanced learning of small disjuncts and small sample size. And, Tomek Link-based under-sampling method is also incorporated into the proposed cluster-based resampling methods to solve the class overlapping problem by eliminating the majority samples in overlapping regions. Experiments are performed on the 25 standard imbalanced data sets by applying four learning classifiers, and validated with the three popular metrics (i.e. Area Under the Curve (AUC), Geometric Mean (G-mean) and Balanced Accuracy (BA)). Specially, we show that the proposed approach has outperformed the other state-of-the-art resampling methods using performance metrics, probabilistic estimation, statistical analysis and multicriteria decision-making methodology (MCDM). © 2020, International Association of Engineers.

关键词： Classification (of information)

Latent space conditioning on generative adversarial networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Lopez, Ricard Durall Ho, Kalun Pfreundt, Franz-Josef Keuper, Janis Fraunhofer ITWM Germany IWR University of Heidelberg Germany Fraunhofer Center Machine Learning Germany Data and Web Science Group University of Mannheim Germany Institute for Machine Learning and Analytics Offenburg University Germany

Generative adversarial networks are the state of the art approach towards learned synthetic image generation. Although early successes were mostly unsupervised, bit by bit, this trend has been superseded by approaches based on labelled data. These supervised methods allow a much finer-grained control of the output image, offering more flexibility and stability. Nevertheless, the main drawback of such models is the necessity of annotated data. In this work, we introduce an novel framework that benefits from two popular learning techniques, adversarial training and representation learning, and takes a step towards unsupervised conditional GANs. In particular, our approach exploits the structure of a latent space (learned by the representation learning) and employs it to condition the generative model. In this way, we break the traditional dependency between condition and label, substituting the latter by unsupervised features coming from the latent space. Finally, we show that this new technique is able to produce samples on demand keeping the quality of its supervised counterpart. © 2020, CC BY.

关键词： Generative adversarial networks

Artificial Intelligence-Driven Protein Folding System Employing Alpha Fold

学校读者我要写书评

暂无评论

Artificial Intelligence-Driven Protein Folding System Employ...

Multi-Agent Systems for Collaborative Intelligence (ICMSCI), International Conference on

作者： Sowmiya R M. Pravin Kumar T. M. Sathish Kumar Rasika P M. Poornima Devi S. Priya Dharshini Department of Artificial Intelligence and Data Science SNS College of Engineering Coimbatore Tamil Nadu India Department of Medical Electronics Velalar College of Engineering and Technology (Autonomous) Erode Tamil Nadu India Department of Electronics and Communication Engineering K.S.R. College of Engineering Namakkal Tamil Nadu India Department of Pharmaceutics SNS College of Pharmacy and Health Sciences Coimbatore Tamil Nadu India Department of Artificial Intelligence and Machine Learning SNS College of Technology Coimbatore Tamil Nadu India Department of Pharmaceutics Periyar College of Pharmaceutical Sciences Tiruchirappalli Tamil Nadu India

ISBN: (数字)9798331509828

ISBN: (纸本)9798331509835

Artificial Intelligence (AI) is a main role to solve the lot of real-live issues. One of the significant challenges in the bioinformatics that researchers have been identified recently is understand the protein folding. First of all, people want to give a big hand off to the advancements because, day to day activities is performed in 3-dimensional basis that is formed based on a sequence of amino acids using protein folds. The folding of proteins is mostly related to three major issues. The first issue is the lack of a specific folding code. There is a folding mechanism in the second one. The ability to ascertain the 3-dimensional structure of proteins presents the last challenge. The structural components and microscopic machinery that make up human cells are called proteins. They provide vital tasks include storing oxygen, breaking down food, and creating scaffolding to support cells in maintaining their structure. Here, Amino acids is formed based on a single chain mechanism because amino acids folds safely and it performs a unique structure that is incorporated here. All living things, including viruses, are composed of proteins, including every element of human body and every other organism. The computational approach can be used to predict the structure of minuscule proteins. In this research work explained about how artificial intelligence is incorporated in predictions of protein structure and artificial intelligence give the solution for the problem of problem folding.

关键词： Proteins Hands Computer viruses Microscopy Amino acids Organisms Artificial intelligence machinery Collaborative intelligence Periodic structures

Overview of the Tenth Dialog System Technology Challenge: DSTC10

学校读者我要写书评

暂无评论

IEEE/ACM Transactions on Audio Speech and Language Processing 2024年 32卷 765-778页

作者： Yoshino, Koichiro Chen, Yun-Nung Crook, Paul Kottur, Satwik Li, Jinchao Hedayatnia, Behnam Moon, Seungwhan Fei, Zhengcong Li, Zekang Zhang, Jinchao Feng, Yang Zhou, Jie Kim, Seokhwan Liu, Yang Jin, Di Papangelis, Alexandros Gopalakrishnan, Karthik Hakkani-Tur, Dilek Damavandi, Babak Geramifard, Alborz Hori, Chiori Shah, Ankit Zhang, Chen Li, Haizhou Sedoc, Joao D'haro, Luis F. Banchs, Rafael Rudnicky, Alexander Guardian Robot Project R-IH RIKEN 2-2-2 Hikaridai Seika Shoraku619-0288 Japan Information Science Nara Institute of Science and Technology Ikoma630-0101 Japan Computer Science and Information Engineering National Taiwan University Taipei10617 Taiwan Inc. Palo AltoCA95054 United States Alexa AI *** Inc. SunnyvaleCA94089 United States Meta Seattle RedmondWA98052 United States Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China Tencent AI Lab Beijing Beijing China Kexueyuan South Road Zhongguancun Beijing100190 China Beijing 100190 China Alexa AI *** Inc. SunnyvaleCA United States 1120 Enterprise way Sunnyvale94089 United States *** Inc. SeattleWA United States Menlo Park CA United States Audio and Speech Group Mitsubishi Electric Research Laboratories CambridgeMA02139-1955 United States Carnegie Mellon University Department of Language and Information Technologies or just Carnegie Mellon University Pittsburgh United States National University of Singapore Singapore Singapore Department of Electrical and Computer Engineering National University of Singapore Singapore Singapore Shenzhen Research Institute of Big Data School of Data Science Chinese University of Hong Kong Shenzhen518172 China New York University New YorkNY United States ETSI de Telecomunicacion - Speech Technology and Machine Learning Group Universidad Politecnica de Madrid Ciudad Universitaria Madrid28040 Spain Nanyang Technological University Singapore Singapore Carnegie Mellon University PittsburghPA United States

This article introduces the Tenth Dialog System Technology Challenge (DSTC-10). This edition of the DSTC focuses on applying end-to-end dialog technologies for five distinct tasks in dialog systems, namely 1. Incorporation of Meme images into open domain dialogs, 2. Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations, 3. Situated Interactive Multimodal dialogs, 4. Reasoning for Audio Visual Scene-Aware Dialog, and 5. Automatic Evaluation and Moderation of Open-domainDialogue Systems. This article describes the task definition, provided datasets, baselines, and evaluation setup for each track. We also summarize the results of the submitted systems to highlight the general trends of the state-of-the-art technologies for the tasks. © 2023 The Authors.

关键词： Job analysis

Improving Generative Model-based Unfolding with Schrödinger Bridges

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Diefenbacher, Sascha Liu, Guan-Horng Mikuni, Vinicius Nachman, Benjamin Nie, Weili Physics Division Lawrence Berkeley National Laboratory BerkeleyCA94720 United States Autonomous Control and Decision Systems Laboratory Georgia Institute of Technology AtlantaGA30332 United States National Energy Research Scientific Computing Center Berkeley Lab BerkeleyCA94720 United States Berkeley Institute for Data Science University of California BerkeleyCA94720 United States Machine Learning Research Group NVIDIA Research United States

machine learning-based unfolding has enabled unbinned and high-dimensional differential cross section measurements. Two main approaches have emerged in this research area: one based on discriminative models and one based on generative models. The main advantage of discriminative models is that they learn a small correction to a starting simulation while generative models scale better to regions of phase space with little data. We propose to use Schrödinger Bridges and diffusion models to create SBUnfold, an unfolding approach that combines the strengths of both discriminative and generative models. The key feature of SBUnfold is that its generative model maps one set of events into another without having to go through a known probability density as is the case for normalizing flows and standard diffusion models. We show that SBUnfold achieves excellent performance compared to state of the art methods on a synthetic Z+jets dataset. © 2023, CC BY.

关键词： Phase space methods

Robust and Fast Measure of Information via Low-rank Representation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Dong, Yuxin Gong, Tieliang Yu, Shujian Chen, Hong Li, Chen School of Computer Science and Technology Xi’an Jiaotong University Xi’an710049 China Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Ministry of Education Xi’an710049 China Machine Learning Group UiT - The Arctic University of Norway Norway College of Science Huazhong Agriculture University Wuhan430070 China Engineering Research Center of Intelligent Technology for Agriculture Ministry of Education Wuhan430070 China

The matrix-based Rényi’s entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution. This intriguing property makes it widely applied in statistical inference and machine learning tasks. However, this information theoretical quantity is not robust against noise in the data, and is computationally prohibitive in large-scale applications. To address these issues, we propose a novel measure of information, termed low-rank matrix-based Rényi’s entropy, based on low-rank representations of infinitely divisible kernel matrices. The proposed entropy functional inherits the specialty of of the original definition to directly quantify information from data, but enjoys additional advantages including robustness and effective calculation. Specifically, our low-rank variant is more sensitive to informative perturbations induced by changes in underlying distributions, while being insensitive to uninformative ones caused by noises. Moreover, low-rank Rényi’s entropy can be efficiently approximated by random projection and Lanczos iteration techniques, reducing the overall complexity from O(n3) to O(n2s) or even O(ns2), where n is the number of data samples and s n. We conduct large-scale experiments to evaluate the effectiveness of this new information measure, demonstrating superior results compared to matrix-based Rényi’s entropy in terms of both performance and computational efficiency. Copyright © 2022, The Authors. All rights reserved.

关键词： Entropy