检索结果-内蒙古大学图书馆

A Survey of Learning on Small Data: Generalization, Optimization, and Challenge

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cao, Xiaofeng Bu, Weixin Huang, Shengjun Zhang, Minling Tsang, Ivor W. Ong, Yew Soon Kwok, James T. The School of Artificial Intelligence Jilin University Changchun130012 China The MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing211106 China The School of Computer Science and Engineering Southeast University Nanjing210096 China Ministry of Education China The Australian Artificial Intelligence Institute University of Technology SydneyNSW2008 Australia Singapore The School of Computer Science and Engineering Nanyang Technological University Nanyang Avenue Singapore639798 Singapore Artificial Intelligence Scientist of A∗STAR Singapore The Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong

Learning on big data brings success for artificial intelligence (AI), but the annotation and training costs are expensive. In future, learning on small data that approximates the generalization ability of big data is one of the ultimate purposes of AI, which requires machines to recognize objectives and scenarios relying on small data as humans. A series of learning topics is going on this way such as active learning and few-shot learning. However, there are few theoretical guarantees for their generalization performance. Moreover, most of their settings are passive, that is, the label distribution is explicitly controlled by finite training resources from known distributions. This survey follows the agnostic active sampling theory under a PAC (Probably Approximately Correct) framework to analyze the generalization error and label complexity of learning on small data in model-agnostic supervised and unsupervised fashion. Considering multiple learning communities could produce small data representation and related topics have been well surveyed, we thus subjoin novel geometric representation perspectives for small data: the Euclidean and non-Euclidean (hyperbolic) mean, where the optimization solutions including the Euclidean gradients, non-Euclidean gradients, and Stein gradient are presented and discussed. Later, multiple learning communities that may be improved by learning on small data are summarized, which yield data-efficient representations, such as transfer learning, contrastive learning, graph representation learning. Meanwhile, we find that the meta-learning may provide effective parameter update policies for learning on small data. Then, we explore multiple challenging scenarios for small data, such as the weak supervision and multi-label. Finally, multiple data applications that may benefit from efficient small data representation are surveyed. Copyright © 2022, The Authors. All rights reserved.

关键词： Big data

Stabilizing Q learning via soft Mellowmax operator

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Gan, Yaozhong Zhang, Zhe Tan, Xiaoyang College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Collaborative Innovation Center of Novel Software Technology and Industrialization China

Learning complicated value functions in high dimensional state space by function approximation is a challenging task, partially due to that the max-operator used in temporal difference updates can theoretically cause instability for most linear or non-linear approximation schemes. Mellowmax is a recently proposed differentiable and non-expansion softmax operator that allows a convergent behavior in learning and planning. Unfortunately, the performance bound for the fixed point it converges to remains unclear, and in practice, its parameter is sensitive to various domains and has to be tuned case by case. Finally, the Mellowmax operator may suffer from oversmoothing as it ignores the probability being taken for each action when aggregating them. In this paper, we address all the above issues with an enhanced Mellowmax operator, named SM2 (Soft Mellowmax). Particularly, the proposed operator is reliable, easy to implement, and has provable performance guarantee, while preserving all the advantages of Mellowmax. Furthermore, we show that our SM2 operator can be applied to the challenging multi-agent reinforcement learning scenarios, leading to stable value function approximation and state of the art performance. Copyright © 2020, The Authors. All rights reserved.

关键词： Reinforcement learning

AADL and Modelica model combination and model conversion based on CPS 20

学校读者我要写书评

暂无评论

AADL and Modelica model combination and model conversion bas...

Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer Engineering

作者： Yifeng Zhu Zining Cao Fujun Wang Weiwei Lu College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing China College of Computer Science and Technology Collaborative Innovation Center of Novel Software Technology and Industrialization MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing University of Aeronautics and Astronautics and Science and Technology on Electro-optic Control Laboratory Luoyang China Science and Technology on Electro-optic Control Laboratory Luoyang China

ISBN: (纸本)9781450387811

Cyber-Physical System (CPS), which realizes the close integration of physical resources and information resources, is a distributed and asynchronous dynamic hybrid system running in different time and space. In this paper, we use AADL to model information system and use Modelica to model physical system. However, they are all semi-formal models such that we cannot do future work about formal verification. To overcome this defect, we convert them to formal models like automaton through conversion algorithms and rules. Due to the fact of probabilistic events and resource consumption in CPS, this paper adopts the weighted probabilistic hybrid automaton as formal model, and adds the properties of probability and weight to the behavior annex of AADL. Then we conduct the rule and algorithm of the model conversion. In order to ensure consistency of further work about model checking, the paper prove the equivalence of the two models by bisimulation. Finally, the capability of modeling of the extended modeling language of AADL and the significance of conversion to automaton is illustrated by the aircraft system.

关键词： Cyber-Physical System model conversion weighted probabilistic hybrid automaton AADL behavior annex bisimulation

A Multi-Layer Random Walk Method for Local Dynamic Community Detection in Brain Functional Network

学校读者我要写书评

暂无评论

A Multi-Layer Random Walk Method for Local Dynamic Community...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Xuyun Wen Daoqiang Zhang College of Computer Science and Technology MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing University of Aeronautics and Astronautics Nanjing Jiangsu China College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing Jiangsu China Engineering Research Center of Traditional Chinese Medicine Intelligent Rehabilitation Ministry of Education Shanghai China

ISBN: (纸本)9781665429825

Detecting the time-varying community structure of brain functional network is very important to reveal dynamic properties of the human brain. Although several community detection methods have been proposed, they are limited in real application due to their poor performance in large dynamic network and difficulty in parameter setting without prior knowledge. To address these problems, this paper proposes a novel dynamic community detection method for the brain network based on random walk, named as ML-RW. This method uses local community discovery instead of global community detection to improve its ability to deal with large dynamic networks. Specifically, ML-RW first selects a query brain region and sends out multiple random walkers starting from this region to explore local community structures of all networks in dynamic network simultaneously. It updates the visiting probability vector of each walker by aggregating the transition probabilities from itself and two temporally adjacent networks. Since the influence strength from one network to another is adaptively tuned according to the relevance of visiting histories of two networks, ML-RW could guarantee the temporal smoothness of the detected modular structure without introducing the hyper-parameter and thus avoids the problem of parameter setting in existing methods. Experiments on two public real neuroimaging datasets demonstrate that our proposed method has more potential to capture subtle community variations in the brain region, stronger ability to discover biomarkers for brain diseases, and higher test-retest reliability than the conventional method.

关键词： Neuroimaging Knowledge engineering Conferences Biomarkers History Reliability Bioinformatics

Detail-recovery Image Deraining via Context Aggregation Networks

学校读者我要写书评

暂无评论

Detail-recovery Image Deraining via Context Aggregation Netw...

Conference on Computer Vision and pattern Recognition (CVPR)

作者： Sen Deng Mingqiang Wei Jun Wang Yidan Feng Luming Liang Haoran Xie Fu Lee Wang Meng Wang Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Microsoft Applied Sciences Group Lingnan University The Open University of Hong Kong Hefei University of Technology

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

This paper looks at this intriguing question: are single images with their details lost during deraining, reversible to their artifact-free status? We propose an end-to-end detail-recovery image deraining network (termed a DRDNet) to solve the problem. Unlike existing image deraining approaches that attempt to meet the conflicting goal of simultaneously deraining and preserving details in a unified framework, we propose to view rain removal and detail recovery as two seperate tasks, so that each part could specialize rather than trade-off between two conflicting goals. Specifically, we introduce two parallel sub-networks with a comprehensive loss function which synergize to derain and recover the lost details caused by deraining. For complete rain removal, we present a rain residual network with the squeeze-and-excitation (SE) operation to remove rain streaks from the rainy images. For detail recovery, we construct a specialized detail repair network consisting of welldesigned blocks, named structure detail context aggregation block (SDCAB), to encourage the lost details to return for eliminating image degradations. Moreover, the detail recovery branch of our proposed detail repair framework is detachable and can be incorporated into existing deraining methods to boost their performances. DRD-Net has been validated on several well-known benchmark datasets in terms of deraining robustness and detail accuracy. Comparisons show clear visual and numerical improvements of our method over the state-of-the-arts.

关键词： Rain Maintenance engineering Task analysis Degradation Training Feature extraction Visualization

Semi-Supervised Partial Multi-label Learning

学校读者我要写书评

暂无评论

Semi-Supervised Partial Multi-label Learning

IEEE International Conference on Data Mining (ICDM)

作者： Ming-Kun Xie Sheng-Jun Huang College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics MIlT Key Laboratory of Pattern Analysis and Machine Intelligence Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing China

ISBN: (数字)9781728183169

ISBN: (纸本)9781728183176

Partial multi-label learning (PML) deals with problems where each instance is associated with a candidate label set, which contains multiple relevant labels and some noisy labels. In many real-world scenarios, it is impractical to annotate all examples for a huge-size dataset. Instead, a more common case is that only a small set of the data are annotated with partial labels, while most data are unlabeled. In this paper, we formalize such problems as a new learning framework called Semi-Supervised Partial Multi-label Learning (SSPML). To solve the SSPML problem, a latent label variable is introduced for each example as the low-dimensional embedding of the feature space. On one hand, label variables are recovered by encouraging consistent similarity measurement between the feature space and the label space; on the other hand, the similarities are adaptively updated based on the feedback from the label space. Meanwhile, the multi-label classifier is jointly trained under the supervision of label variables. Extensive experiments on multiple datasets from various real-world tasks validate the effectiveness of the proposed approach.

关键词： Weight measurement Measurement uncertainty Prediction algorithms Extraterrestrial measurements Noise measurement Data mining Task analysis

With false friends like these, who can notice mistakes?

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Tao, Lue Feng, Lei Yi, Jinfeng Chen, Songcan College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science Chongqing University China JD AI Research

Adversarial examples crafted by an explicit adversary have attracted significant attention in machine learning. However, the security risk posed by a potential false friend has been largely overlooked. In this paper, we unveil the threat of hypocritical examples-inputs that are originally misclassified yet perturbed by a false friend to force correct predictions. While such perturbed examples seem harmless, we point out for the first time that they could be maliciously used to conceal the mistakes of a substandard (i.e., not as good as required) model during an evaluation. Once a deployer trusts the hypocritical performance and applies the "well-performed" model in realworld applications, unexpected failures may happen even in benign environments. More seriously, this security risk seems to be pervasive: we find that many types of substandard models are vulnerable to hypocritical examples across multiple datasets. Furthermore, we provide the first attempt to characterize the threat with a metric called hypocritical risk and try to circumvent it via several countermeasures. Results demonstrate the effectiveness of the countermeasures, while the risk remains non-negligible even after adaptive robust training. Copyright © 2020, The Authors. All rights reserved.

关键词： machine learning

Sampling algorithms, from survey sampling to Monte Carlo methods: Tutorial and literature review

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Ghojogh, Benyamin Nekoei, Hadi Ghojogh, Aydin Karray, Fakhri Crowley, Mark Department of Electrical and Computer Engineering Machine Learning Laboratory University of Waterloo WaterlooON Canada Quebec AI Institute MontrealQC Canada Department of Electrical and Computer Engineering Centre for Pattern Analysis and Machine Intelligence University of Waterloo WaterlooON Canada

This paper is a tutorial and literature review on sampling algorithms. We have two main types of sampling in statistics. The first type is survey sampling which draws samples from a set or population. The second type is sampling from probability distribution where we have a probability density or mass function. In this paper, we cover both types of sampling. First, we review some required background on mean squared error, variance, bias, maximum likelihood estimation, Bernoulli, Binomial, and Hypergeometric distributions, the Horvitz–Thompson estimator, and the Markov property. Then, we explain the theory of simple random sampling, bootstrapping, stratified sampling, and cluster sampling. We also briefly introduce multistage sampling, network sampling, and snowball sampling. Afterwards, we switch to sampling from distribution. We explain sampling from cumulative distribution function, Monte Carlo approximation, simple Monte Carlo methods, and Markov Chain Monte Carlo (MCMC) methods. For simple Monte Carlo methods, whose iterations are independent, we cover importance sampling and rejection sampling. For MCMC methods, we cover Metropolis algorithm, Metropolis-Hastings algorithm, Gibbs sampling, and slice sampling. Then, we explain the random walk behaviour of Monte Carlo methods and more efficient Monte Carlo methods, including Hamiltonian (or hybrid) Monte Carlo, Adler’s overrelaxation, and ordered overrelaxation. Finally, we summarize the characteristics, pros, and cons of sampling methods compared to each other. This paper can be useful for different fields of statistics, machine learning, reinforcement learning, and computational physics. Copyright © 2020, The Authors. All rights reserved.

关键词： Learning algorithms

Deep robust multilevel semantic cross-modal hashing

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Song, Ge Zhao, Jun Tan, Xiaoyang College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Miit Key Laboratory of Pattern Analysis and Machine Intelligence Collaborative Innovation Center of Novel Software Technology and Industrialization Nanyang Technological University

Hashing based cross-modal retrieval has rec made significant progress. But straightfor embedding data from different modalities in joint Hamming space will inevitably produce codes due to the intrinsic modality discrep and noises. We present a novel Robust M level Semantic Hashing (RMSH) for more a rate cross-modal retrieval. It seeks to pres fine-grained similarity among data with rich mantics, while explicitly require distances bet dissimilar points to be larger than a specific v for strong robustness. For this, we give an fective bound of this value based on the information coding-theoretic analysis, and the above goals are embodied into a margin-adaptive triplet loss. Furthermore, we introduce pseudo-codes via fusing multiple hash codes to explore seldom-seen semantics, alleviating the sparsity problem of similarity information. Experiments on three benchmarks show the validity of the derived bounds, and our method achieves state-of-the-art performance. Copyright © 2020, The Authors. All rights reserved.

关键词： Semantics