检索结果-内蒙古大学图书馆

Statistical Dependency Guided Contrastive learning for Multiple Labeling in Prenatal Ultrasound

学校读者我要写书评

暂无评论

arXiv 2021年

作者： He, Shuangchi Lin, Zehui Yang, Xin Chen, Chaoyu Wang, Jian Shuang, Xue Deng, Ziwei Liu, Qin Cao, Yan Lu, Xiduo Huang, Ruobing Ravikumar, Nishant Frangi, Alejandro Zhang, Yuanji Xiong, Yi Ni, Dong National-Regional Key Technology Engineering Laboratory for Medical Ultrasound School of Biomedical Engineering Health Science Center Shenzhen University China Lab. Shenzhen University China Marshall Laboratory of Biomedical Engineering Shenzhen University China University of Leeds United Kingdom Leeds Institute of Cardiovascular and Metabolic Medicine University of Leeds United Kingdom KU Leuven Leuven Belgium Department of Ultrasound Luohu People’s Hospital Shenzhen China

Standard plane recognition plays an important role in prenatal ultrasound (US) screening. Automatically recognizing the standard plane along with the corresponding anatomical structures in US image can not only facilitate US image interpretation but also improve diagnostic efficiency. In this study, we build a novel multi-label learning (MLL) scheme to identify multiple standard planes and corresponding anatomical structures of fetus simultaneously. Our contribution is three-fold. First, we represent the class correlation by word embeddings to capture the fine-grained semantic and latent statistical concurrency. Second, we equip the MLL with a graph convolutional network to explore the inner and outer relationship among categories. Third, we propose a novel cluster relabel-based contrastive learning algorithm to encourage the divergence among ambiguous classes. Extensive validation was performed on our large in-house dataset. Our approach reports the highest accuracy as 90.25% for standard planes labeling, 85.59% for planes and structures labeling and mAP as 94.63%. The proposed MLL scheme provides a novel perspective for standard plane recognition and can be easily extended to other medical image classification tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

A deep learning algorithm for optimal investment strategies

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Gim, Daeyung Park, Hyungbin Department of Mathematical Sciences Research Institute of Mathematics Seoul National University 1 Gwanak-ro Gwanak-gu Seoul Korea Republic of

This paper treats the Merton problem how to invest in safe assets and risky assets to maximize an investor’s utility, given by investment opportunities modeled by a d-dimensional state process. The problem is represented by a partial differential equation with optimizing term: the Hamilton–Jacobi–Bellman equation. The main purpose of this paper is to solve partial differential equations derived from the Hamilton–Jacobi–Bellman equations with a deep learning algorithm: the Deep Galerkin method, first suggested by Sirignano and Spiliopoulos (2018). We then apply the algorithm to get the solution of the PDE based on some model settings and compare with the one from the finite difference method. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Meta-learning with neural tangent kernels

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhou, Yufan Wang, Zhenyi Xian, Jiayi Chen, Changyou Xu, Jinhui Department of Computer Science and Engineering State University of New York Buffalo United States

Model Agnostic Meta-learning (MAML) has emerged as a standard framework for meta-learning, where a meta-model is learned with the ability of fast adapting to new tasks. However, as a double-looped optimization problem, MAML needs to differentiate through the whole inner-loop optimization path for every outer-loop training step, which may lead to both computational inefficiency and sub-optimal solutions. In this paper, we generalize MAML to allow meta-learning to be defined in function spaces, and propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK). Within this paradigm, we introduce two meta-learning algorithms in the RKHS, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS;and 2) solving the adaptation analytically based on the NTK theory. Extensive experimental studies demonstrate advantages of our paradigm in both efficiency and quality of solutions compared to related meta-learning algorithms. Another interesting feature of our proposed methods is that they are demonstrated to be more robust to adversarial attacks and out-of-distribution adaptation than popular baselines, as demonstrated in our experiments. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Optimal online learning using potential functions

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Freund, Yoav

We study regret-minimizing online algorithms based on potential functions. First, we show that any algorithm with a regret bound that holds for any ǫ is equivalent to a potential minimizing algorithm and vice versa. Second we should a min-max learning algorithm for known horizon. We show a regret bound that is close to optimal when the horizon is not known. Finally we give an algorithm with second order bounds that characterize easy sequences. © 2021, CC BY.

关键词： learning algorithms

CSNAS: Contrastive self-supervised learning neural architecture search via sequential model-based optimization

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Nguyen, Nam Chang, J. Morris The Electrical Engineering Department University of South Florida TampaFL United States

This paper proposes a novel contrastive self-supervised neural architecture search algorithm (NAS), which completely alleviates the expensive costs of data labeling inherited from supervised learning. Our algorithm capitalizes on the effectiveness of self-supervised learning for image representations, which is an increasingly crucial topic of computer vision. First, using only a small amount of unlabeled train data under contrastive self-supervised learning (C-SSL) allow us to search on a more extensive search space, discovering better neural architectures without surging the computational resources. Second, we entirely relieve the cost for labeled data (by contrastive loss) in the search stage without compromising architectures' final performance in the evaluation phase. Finally, we tackle the inherent discrete search space of the NAS problem by sequential model-based optimization via the tree-parzen estimator (SMBO-TPE), enabling us to significantly reduce the computational expense response surface. An extensive number of experiments empirically show that our search algorithm can achieve state-of-the-art results with better efficiency in data labeling cost, searching time, and accuracy in final validation. © 2021, CC BY.

关键词： learning algorithms

Cross-Gradient Aggregation for Decentralized learning from Non-IID Data

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Esfandiari, Yasaman Tan, Sin Yong Jiang, Zhanhong Balu, Aditya Herron, Ethan Hegde, Chinmay Sarkar, Soumik Department of Mechanical Engineering Iowa State University AmesIA United States Johnson Controls MilwaukeeWI United States Computer Science and Engineering Department New York University New York CityNY United States

Decentralized learning enables a group of collaborative agents to learn models using a distributed dataset without the need for a central parameter server. Recently, decentralized learning algorithms have demonstrated state-of-the-art results on benchmark data sets, comparable with centralized algorithms. However, the key assumption to achieve competitive performance is that the data is independently and identically distributed (IID) among the agents which, in real-life applications, is often not applicable. Inspired by ideas from continual learning, we propose Cross-Gradient Aggregation (CGA), a novel decentralized learning algorithm where (i) each agent aggregates cross-gradient information, i.e., derivatives of its model with respect to its neighbors’ datasets, and (ii) updates its model using a projected gradient based on quadratic programming (QP). We theoretically analyze the convergence characteristics of CGA and demonstrate its efficiency on non-IID data distributions sampled from the MNIST and CIFAR-10 datasets. Our empirical comparisons show superior learning performance of CGA over existing state-of-the-art decentralized learning algorithms, as well as maintaining the improved performance under information compression to reduce peer-to-peer communication overhead. The code is available here on GitHub. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

MANDERA: Malicious Node Detection in Federated learning via Ranking

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhu, Wanchuang Zhao, Benjamin Zi Hao Luo, Simon Liu, Tongliang Deng, Ke Data61 CSIRO Sydney Australia Department of Computing Macquarie University Australia. Optus Macquarie Cyber Security Hub Sydney Australia School of Computer Science The University of New South Wales Sydney Australia School of Computer Science The University of Sydney Sydney Australia Center for Statistical Science Department of Industrial Engineering Tsinghua University Beijing China

Byzantine attacks hinder the deployment of federated learning algorithms. Although we know that the benign gradients and Byzantine attacked gradients are distributed differently, to detect the malicious gradients is challenging due to (1) the gradient is high-dimensional and each dimension has its unique distribution and (2) the benign gradients and the attacked gradients are always mixed (two-sample test methods cannot apply directly). To address the above, for the first time, we propose MANDERA which is theoretically guaranteed to efficiently detect all malicious gradients under Byzantine attacks with no prior knowledge or history about the number of attacked nodes. More specifically, we transfer the original updating gradient space into a ranking matrix. By such an operation, the scales of different dimensions of the gradients in the ranking space become identical. The high-dimensional benign gradients and the malicious gradients can be easily separated. The effectiveness of MANDERA is further confirmed by experimentation on four Byzantine attack implementations (Gaussian, Zero Gradient, Sign Flipping, Shifted Mean), comparing with state-of-the-art defenses. The experiments cover both IID and Non-IID datasets. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Risky Driving Behavior Identification and its Research Trends:A Review Focusing on Chinese Studies

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Xiong, Rui Deng, Yuanchang School of Intelligent Systems Engineering Sun Yat-sen University Shenzhen518107 China Guangdong Key Laboratory of Intelligent Transportation Systems Guangzhou510006 China

Risky driving behavior is an important cause of traffic accidents, identifying drivers' risky behavior accurately and reminding them in time can effectively reduce the occurrence of traffic accidents. The definition of risky driving behavior is ambiguous and the source data related to risky driving behavior identification are various, so there is still no unified framework for identifying risky driving behavior. Based on this, through CNKI (China National Knowledge Infrastructure) database, this paper obtained 514 articles related to risky driving behavior identification, which were written by Chinses scholars and published between 2010 to 2022 (up to 13rd April, 2022), and used VOSviewer to statistically analyze them. In addition, the risky driving behaviors which Chinese scholars mostly focused on and the identification methods they commonly used were reviewed, and the current research characteristics and future research trends were discussed. Results showed that the number of articles in the field of risky driving behavior identification published by Chinses scholars began to rise in 2018. Universities that contributed more to the number of theses included in 514 articles were mainly from eastern China, and the most popular source journal was Accident Analysis and Prevention. Besides, Chinese scholars commonly focused on 8 kinds of risky driving behaviors including speeding, rapid shifting, sharp turning, lane departure, overclose vehicle following, dangerous lane changing, fatigue driving and distracted driving. Rule-based identification methods and machine learning based identification methods were mainly used to identify risky driving behavior. Rule-based identification methods were relatively simple, but the thresholds set in various studies were not consistent. Machine learning algorithms performed better, but their generalization ability was poor. Finally, studies on real-time risky driving behavior identification based on multi-source data and portable dev

关键词： learning algorithms

CodeNet: A large-scale AI for code dataset for learning a diversity of coding tasks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Puri, Ruchir Kung, David S. Janssen, Geert Zhang, Wei Domeniconi, Giacomo Zolotov, Vladimir Dolby, Julian Chen, Jie Choudhury, Mihir Decker, Lindsey Thost, Veronika Buratti, Luca Pujar, Saurabh Ramji, Shyam Finkler, Ulrich Malaika, Susan Reiss, Frederick IBM Research MIT-IBM Watson AI Lab IBM Worldwide Ecosystems

Over the last several decades, software has been woven into the fabric of every aspect of our society. As software development surges and code infrastructure of enterprise applications ages, it is now more critical than ever to increase software development productivity and modernize legacy applications. Advances in deep learning and machine learning algorithms have enabled breakthroughs in computer vision, speech recognition, natural language processing and beyond, motivating researchers to leverage AI techniques to improve software development efficiency. Thus, the fast-emerging research area of "AI for Code" has garnered new interest and gathered momentum. In this paper, we present a large-scale dataset CodeNet, consisting of over 14 million code samples and about 500 million lines of code in 55 different programming languages, which is aimed at teaching AI to code. In addition to its large scale, CodeNet has a rich set of high-quality annotations to benchmark and help accelerate research in AI techniques for a variety of critical coding tasks, including code similarity and classification, code translation between a large variety of programming languages, and code performance (runtime and memory) improvement techniques. Additionally, CodeNet provides sample input and output test sets for 98.5% of the code samples, which can be used as an oracle for determining code correctness and potentially guide reinforcement learning for code quality improvements. As a usability feature, we provide several pre-processing tools in CodeNet to transform source code into representations that can be readily used as inputs into machine learning models. Results of code classification and code similarity experiments using the CodeNet dataset are provided as a reference. We hope that the scale, diversity and rich, high-quality annotations of CodeNet will offer unprecedented research opportunities at the intersection of AI and Software Engineering. Copyright © 2021, The Authors. All ri

关键词： learning algorithms