检索结果-内蒙古大学图书馆

DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus Algorithm

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ding, Lisang Jin, Kexin Ying, Bicheng Yuan, Kun Yin, Wotao Department of Mathematics University of California Los AngelesCA United States Department of Mathematics Princeton University PrincetonNJ United States Google Inc. Los AngelesCA United States Center for Machine Learning Research Peking University Beijing China AI for Science Institute Beijing China National Engineering Labratory for Big Data Analytics and Applications Beijing China Decision Intelligence Lab. Alibaba US BellevueWA United States

Decentralized Stochastic Gradient Descent (SGD) is an emerging neural network training approach that enables multiple agents to train a model collab.ratively and simultaneously. Rather than using a central parameter server to collect gradients from all the agents, each agent keeps a copy of the model parameters and communicates with a small number of other agents to exchange model updates. Their communication, governed by the communication topology and gossip weight matrices, facilitates the exchange of model updates. The state-of-the-art approach uses the dynamic one-peer exponential-2 topology, achieving faster training times and improved scalab.lity than the ring, grid, torus, and hypercube topologies. However, this approach requires a power-of-2 number of agents, which is impractical at scale. In this paper, we remove this restriction and propose Decentralized SGD with Communication-optimal Exact Consensus Algorithm (DSGD-CECA), which works for any number of agents while still achieving state-of-the-art properties. In particular, DSGD-CECA incurs a unit per-iteration communication overhead and an Õ(n3) transient iteration complexity. Our proof is based on newly discovered properties of gossip weight matrices and a novel approach to combine them with DSGD’s convergence analysis. Numerical experiments show the efficiency of DSGD-CECA. Copyright © 2023, The Authors. All rights reserved.

关键词： Stochastic systems

A study on the uncertainty of convolutional layers in deep neural networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Shen, Haojing Chen, Sihong Wang, Ran Big Data Institute College of Computer Science and Software Engineering Guangdong Key Lab. of Intelligent Information Processing Shenzhen University ShenzhenGuangdong518060 China College of Mathematics and Statistics Shenzhen University Shenzhen518060 China Shenzhen Key Laboratory of Advanced Machine Learning and Applications Shenzhen University Shenzhen518060 China

This paper shows a Min-Max property existing in the connection weights of the convolutional layers in a neural network structure, i.e., the LeNet. Specifically, the Min-Max property means that, during the back propagation-based training for LeNet, the weights of the convolutional layers will become far away from their centers of intervals, i.e., decreasing to their minimum or increasing to their maximum. From the perspective of uncertainty, we demonstrate that the Min-Max property corresponds to minimizing the fuzziness of the model parameters through a simplified formulation of convolution. It is experimentally confirmed that the model with the Min-Max property has a stronger adversarial robustness, thus this property can be incorporated into the design of loss function. This paper points out a changing tendency of uncertainty in the convolutional layers of LeNet structure, and gives some insights to the interpretability of convolution. © 2020, CC-BY.

关键词： Convolution

Fetal Re-Identification in Multiple Pregnancy Ultrasound Images Using Deep learning

学校读者我要写书评

暂无评论

Fetal Re-Identification in Multiple Pregnancy Ultrasound Ima...

Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

作者： Elisabeth Gabler Michael Nissen Thomas R. Altstidl Adriana Titzmann Kai Packhäuser Andreas Maier Peter A. Fasching Bjoern M. Eskofier Heike Leutheuser Department Artificial Intelligence in Biomedical Engineering Machine Learning and Data Analytics (MaD) Lab Friedrich-Alexander-Universität Erlangen-Nurnberg (FAU) Erlangen Germany Department of Gynecology and Obstetrics Erlangen University Hospital Friedrich-Alexander-Universität Erlangen-Nurnberg (FAU) Erlangen Germany Department of Computer Science Pattern Recognition Lab Friedrich-Alexander-Universität Erlangen-Nurnberg (FAU) Erlangen Germany

Ultrasound examinations during pregnancy can detect abnormal fetal development, which is a leading cause of perinatal mortality. In multiple pregnancies, the position of the fetuses may change between examinations. The individual fetus cannot be clearly identified. Fetal re-identification may improve diagnostic capabilities by tracing individual fetal changes. This work evaluates the feasibility of fetal re-identification on FETAL_PLANES_DB, a publicly availab.e dataset of singleton pregnancy ultrasound images. Five dataset subsets with 6,491 images from 1,088 pregnant women and two re-identification frameworks (Torchreid, FastReID) are evaluated. FastReID achieves a mean average precision of 68.77% (68.42%) and mean precision at rank 10 score of 89.60% (95.55%) when trained on images showing the fetal brain (abdomen). Visualization with gradient-weighted class activation mapping shows that the classifiers appear to rely on anatomical features. We conclude that fetal re-identification in ultrasound images may be feasible. However, more work on additional datasets, including images from multiple pregnancies and several subsequent examinations, is required to ensure and investigate performance stability and *** relevance— To date, fetuses in multiple pregnancies cannot be distinguished between ultrasound examinations. This work provides the first evidence for feasibility of fetal re-identification in pregnancy ultrasound images. This may improve diagnostic capabilities in clinical practice in the future, such as longitudinal analysis of fetal changes or abnormalities.

关键词：

Adversarial learning with Cost-Sensitive Classes

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Shen, Haojing Chen, Sihong Wang, Ran Wang, Xizhao Big Data Institute College of Computer Science and Software Engineering Guangdong Key Lab. of Intelligent Information Processing Shenzhen University Guangdong Shenzhen518060 China The College of Mathematics and Statistics Shenzhen University Shenzhen518060 China The Shenzhen Key Laboratory of Advanced Machine Learning and Applications Shenzhen University Shenzhen518060 China

It is necessary to improve the performance of some special classes or to particularly protect them from attacks in adversarial learning. This paper proposes a framework combining cost-sensitive classification and adversarial learning together to train a model that can distinguish between protected and unprotected classes, such that the protected classes are less vulnerable to adversarial examples. We find in this framework an interesting phenomenon during the training of deep neural networks, called Min-Max property, that is, the absolute values of most parameters in the convolutional layer approach zero while the absolute values of a few parameters are significantly larger becoming bigger. Based on this Min-Max property which is formulated and analyzed in a view of random distribution, we further build a new defense model against adversarial examples for adversarial robustness improvement. An advantage of the built model is that it performs better than the standard one and can combine with adversarial training to achieve an improved performance. It is experimentally confirmed that, regarding the average accuracy of all classes, our model is almost as same as the existing models when an attack does not occur and is better than the existing models when an attack occurs. Specifically, regarding the accuracy of protected classes, the proposed model is much better than the existing models when an attack occurs. © 2021, CC BY.

关键词： Deep neural networks

Incorporating Hidden Layer representation into Adversarial Attacks and Defences

学校读者我要写书评

暂无评论

arXiv 2020年

In this paper, we propose a defence strategy to improves adversarial robustness incorporating hidden layer representation. The key of this defence strategy aims to compress or filter input’s information including adversarial perturbation. And this defence strategy can be regarded as an activation function which can be applied any kind of neural networks. We also prove theoretically the effectiveness of this defense strategy under certain conditions. Besides, incorporating hidden layer representation we propose three types of adversarial attacks to generate three types of adversarial examples, respectively. The experiments show that our defence method can significantly improve the adversarial robustness of deep neural networks which achieves the state-of-the-art performance even though we do not adopt adversarial training. © 2020, CC BY.

关键词： Deep neural networks

Facial Expression Recognition Using Multi-Block Deep CNN

学校读者我要写书评

暂无评论

Facial Expression Recognition Using Multi-Block Deep CNN

International Conference on Circuits, Communication, Control and Computing

作者： Priyadarshini V Srinivasulu Reddy U Venkata Rami Reddy Chirra Mrudula M Suneetha M Dept. of Computer Science & Engineering National Institute of Technology Andhra Pradesh India Dept. of Computer Applications Machine Learning & Data Analytics lab National Institute of Technology Tiruchirappalli India School of Computer Science & Engineering VIT-AP University Amaravathi India Department of CSE Malla Reddy College of Engineering Telangana India Independent Researcher Amaravati India

ISBN: (数字)9798331528539

ISBN: (纸本)9798331528546

Emotions play a crucial role in decision-making, moral judgments, and other cognitive processes. The goal of this work is to identify people's facial emotions from face images. Facial expression recognition (FER), a highly diverse field with many commonalities between classes, is a challenging problem. The capabilities of current FER systems are limited in this regard. In this work, a multi-block Deep Convolutional Neural Networks (DCNN) model was proposed to identify facial expressions. Five blocks with different computational components were defined in a multi-block DCNN to extract the unique characteristics from input images. The second model (DCNN-SVM) is developed with an ensemble of SVM classifier to improve predictability and stability. The third model (DCNN-MV) is based on the majority voting with 4 classifiers SVM, KNN, Random Forest, and Decision Tree. The CK+ dataset was utilized and expanded via image data augmentation to enhance model performance and generalization. By adjusting hyperparameters, the suggested DCNN model's accuracy was investigated. The DCNN model reports an accuracy score of 97.71%. Both DCNN-SVM and DCNN-MV models achieved 97.79% accuracy. On the CK+ dataset, the proposed models demonstrated good accuracy in identifying facial expressions. When compared to existing models, the proposed model outperformed existing models in terms of accuracy.

关键词： Support vector machines Accuracy Computational modeling Face recognition Stability analysis Circuit stability Convolutional neural networks Integrated circuit modeling Thermal stability Random forests

Inconsistency Distillation For Consistency:Enhancing Multi-View Clustering via Mutual Contrastive Teacher-Student Leaning

学校读者我要写书评

暂无评论

Inconsistency Distillation For Consistency:Enhancing Multi-V...

IEEE International Conference on data Mining (ICDM)

作者： Dunqiang Liu Shu-Juan Peng Xin Liu Lei Zhu Zhen Cui Taihao Li Dept. of Comput. Sci. & Fujian Key Lab. of Big Data Intelligence and Security Huaqiao University Xiamen China Zhejiang Lab Hangzhou China Xiamen Key Lab. of Computer Vision and Pattern Recognition Huaqiao University Xiamen China Key Lab. of Computer Vision and Machine Learning (Huaqiao University) Fujian Province University Xiamen China School of Information Sci. and Eng. Shandong Normal University Jinan China School of Computer Sci. and Eng. Nanjing University of Science and Technology Nanjing China

Multi-view clustering has attracted more attention recently since many real-world data are comprised of different representations or views. Recent multi-view clustering works mainly exploit the instance consistency to obtain the shared representations across different views, and apply a single-view clustering method to perform data partitions. However, these existing methods often ignore the inconsistency of instance associations within the views, which may enlarge the intra-class diversity among the views and therefore degrade the clustering performance. To address this issue, this paper proposes an efficient mutual contrastive teacher-student leaning (MC-TSL) model to enhance the multi-view clustering, which is the first attempt to study the inconsistency distillation for consistency learning. First, the proposed MC-TSL approach exploits a view-specific encoder with two heads, an instance encoding head and a semantic distillation head, respectively, for capturing the consistent and discriminative feature representations. To be specific, the former head exploits a cross-view contrastive learning method to obtain a redundancy-free consistent representation at the instance level, while the latter head designs a mutual teacher-student learning module to capture the intra-view information at semantic level. By training these two heads in an end-to-end manner, the discriminative multi-view embeddings are efficiently obtained and refined by minimizing the weighted sum of the reconstruction loss, contrastive loss and contrast distillation loss. Extensive experiments verify the superiorities of the proposed MC-TSL framework and show its competitive clustering performances.

关键词： Training learning systems Clustering methods Semantics Encoding data mining

: Quantum Artificial Intelligence for the Automotive Industry

学校读者我要写书评

暂无评论

KI - Künstliche Intelligenz 2024年第4期38卷 351-359页

作者： Stollenwerk, Tobias Bhattacharya, Somtapa Cattelan, Michele Ciani, Alessandro Compostella, Gabriele Headley, David Klepsch, Johannes Klusch, Matthias Leder, Markus Macaluso, Antonio Michielsen, Kristel Nabok, Dmytro Papanikolaou, Anestis Rausch, Alexander Schumann, Marco Skolik, Andrea Yarkoni, Sheir Wilhelm, Frank K. Institute for Quantum Computing Analytics (PGI-12) Forschungszentrum Jülich Jülich Germany Volkswagen Data:Lab Volkswagen Group Munich Germany Machine Learning Research Lab Volkswagen Group Munich Germany Mercedes-Benz AG Stuttgart Germany BMW AG München Germany German Research Center for Artificial Intelligence (DFKI) Saarbrüecken Germany Jülich Supercomputing Centre (JSC) Forschungszentrum Jülich Jülich Germany Robert Bosch GmbH Corporate Research Renningen Germany

MaxHiC: A robust background correction model to identify biologically relevant chromatin interactions in Hi-C and capture Hi-C experiments (vol 18, e1010241, 2022)

学校读者我要写书评

暂无评论

PLOS COMPUTATIONAL BIOLOGY 2022年第9期18卷 e1010241页

作者： Alinejad-Rokny, Hamid Modegh, Rassa Ghavami Rabiee, Hamid R. Sarbandi, Ehsan Ramezani Rezaie, Narges Tam, Kin Tung Forrest, Alistair R. R. Harry Perkins Institute of Medical Research QEII Medical Centre and Centre for Medical Research The University of Western Australia Perth Australia Bio Medical Machine Learning Lab (BML) The Graduate School of Biomedical Engineering UNSW Sydney Sydney Australia Health Data Analytics Program AI-enabled Processes (AIP) Research Centre Macquarie University Sydney Australia Bioinformatics and Computational Biology Lab Department of Computer Engineering Sharif University of Technology Tehran Iran Center for Complex Biological Systems University of California Irvine Irvine California United States of America

Hi-C is a genome-wide chromosome conformation capture technology that detects interactions between pairs of genomic regions and exploits higher order chromatin structures. Conceptually Hi-C data counts interaction frequencies between every position in the genome and every other position. Biologically functional interactions are expected to occur more frequently than transient background and artefactual interactions. To identify biologically relevant interactions, several background models that take biases such as distance, GC content and mappability into account have been proposed. Here we introduce MaxHiC, a background correction tool that deals with these complex biases and robustly identifies statistically significant interactions in both Hi-C and capture Hi-C experiments. MaxHiC uses a negative binomial distribution model and a maximum likelihood technique to correct biases in both Hi-C and capture Hi-C libraries. We systematically benchmark MaxHiC against major Hi-C background correction tools including Hi-C significant interaction callers (SIC) and Hi-C loop callers using published Hi-C, capture Hi-C, and Micro-C datasets. Our results demonstrate that 1) Interacting regions identified by MaxHiC have significantly greater levels of overlap with known regulatory features (e.g. active chromatin histone marks, CTCF binding sites, DNase sensitivity) and also disease-associated genome-wide association SNPs than those identified by currently existing models, 2) the pairs of interacting regions are more likely to be linked by eQTL pairs and 3) more likely to link known regulatory features including known functional enhancer-promoter pairs validated by CRISPRi than any of the existing methods. We also demonstrate that interactions between different genomic region types have distinct distance distributions only revealed by MaxHiC. MaxHiC is publicly availab.e as a python package for the analysis of Hi-C, capture Hi-C and Micro-C data.

关键词： Genomics Chromatin Genome-wide association studies Single nucleotide polymorphisms Histones Gene regulation Deoxyribonucleases Insulators