检索结果-内蒙古大学图书馆

Calibrating and Improving Graph Contrastive learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ma, Kaili Yang, Haochen Yang, Han Chen, Yongqiang Cheng, James Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong

Graph contrastive learning algorithms have demonstrated remarkable success in various applications such as node classification, link prediction, and graph clustering. However, in unsupervised graph contrastive learning, some contrastive pairs may contradict the truths in downstream tasks and thus the decrease of losses on these pairs undesirably harms the performance in the downstream tasks. To assess the discrepancy between the prediction and the ground-truth in the downstream tasks for these contrastive pairs, we adapt expected calibration error (ECE) to graph contrastive learning. The analysis of ECE motivates us to propose a novel regularization method, Contrast-Reg, to ensure that decreasing the contrastive loss leads to better performance in the downstream tasks. As a plug-in regularizer, Contrast-Reg effectively improves the performance of existing graph contrastive learning algorithms. We provide both theoretical and empirical results to demonstrate the effectiveness of Contrast-Reg in enhancing the generalizability of the Graph Neural Network (GNN) model and improving the performance of graph contrastive algorithms with different similarity definitions and encoder backbones across various downstream tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

QK iteration: A self-supervised representation learning algorithm for image similarity

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wu, David Wu, Yunnan

Self-supervised representation learning is a fundamental problem in computer vision with many useful applications (e.g., image search, instance level recognition, copy detection). In this paper we present a new contrastive self-supervised representation learning algorithm in the context of Copy Detection in the 2021 Image Similarity Challenge hosted by Facebook AI Research. Previous work in contrastive self-supervised learning has identified the importance of being able to optimize representations while "pushing" against a large number of negative examples. Representative previous solutions either use large batches enabled by modern distributed training systems or maintain queues or memory banks holding recently evaluated representations while relaxing some consistency properties. We approach this problem from a new angle: We directly learn a query model and a key model jointly and push representations against a very large number (e.g., 1 million) of negative representations in each SGD step. We achieve this by freezing the backbone on one side and by alternating between a Q-optimization step and a K-optimization step. During the competition timeframe, our algorithms achieved a micro-AP score of 0.3401 on the Phase 1 leaderboard, significantly improving over the baseline μAP of 0.1556. On the final Phase 2 leaderboard, our model scored 0.1919, while the baseline scored 0.0526. Continued training yielded further improvement. We conducted an empirical study to compare the proposed approach with a SimCLR style strategy where the negative examples are taken from the batch only. We found that our method (μAP of 0.3403) significantly outperforms this SimCLR-style baseline (μAP of 0.2001). Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

learning to detect few-shot-few-clue misinformation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Qiang Huang, Hongbin Liang, Shangsong Meng, Zaiqiao Yilmaz, Emine University College London London United Kingdom Sun Yat-sen University Guangzhou China University of Cambridge Cambridge United Kingdom University College London & Amazon London United Kingdom

A large volume of false textual information has been disseminating for a long time since the prevalence of social media. The potential negative influence of misinformation on the public is a growing concern. Therefore, it is strongly motivated to detect online misinformation as early as possible. Few-shot-few-clue learning applies in this misinformation detection task when the number of annotated statements is quite few (called few shots) and the corresponding evidence is also quite limited in each shot (called few clues). Within the few-shot-few-clue framework, we propose a task-aware Bayesian meta-learning algorithm to extract shared patterns among different topics (i.e. different tasks) of misinformation. Moreover, we derive a scalable method, i.e., amortized variational inference, to optimize the Bayesian meta-learning algorithm. Empirical results on three benchmark datasets demonstrate the superiority of our algorithm. This work focuses more on optimizing parameters than deigning detection models, and will generate fresh insights into data efficient detection of online misinformation at early stages. © 2021, CC BY.

关键词： learning algorithms

An empirical analysis of image-based learning techniques for malware classification

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Prajapati, Pratikkumar Stamp, Mark Department of Computer Science San Jose State University San JoseCA United States

In this paper, we consider malware classification using deep learning techniques and image-based features. We employ a wide variety of deep learning techniques, including multilayer perceptrons (MLP), convolutional neural networks (CNN), long short-term memory (LSTM), and gated recurrent units (GRU). Amongst our CNN experiments, transfer learning plays a prominent role-specifically, we test the VGG-19 and ResNet152 models. As compared to previous work, the results presented in this paper are based on a larger and more diverse malware dataset, we consider a wider array of features, and we experiment with a much greater variety of learning techniques. Consequently, our results are the most comprehensive and complete that have yet been published. © 2021, CC BY.

关键词： learning algorithms

S-Cyc: A learning rate schedule for iterative pruning of ReLU-based networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Liu, Shiyu Tan, Chong Min John Motani, Mehul Department of Electrical and Computer Engineering School of Engineering National University of Singapore

We explore a new perspective on adapting the learning rate (LR) schedule to improve the performance of the ReLU-based network as it is iteratively pruned. Our work and contribution consist of four parts: (i) We find that, as the ReLU-based network is iteratively pruned, the distribution of weight gradients tends to become narrower. This leads to the finding that as the network becomes more sparse, a larger value of LR should be used to train the pruned network. (ii) Motivated by this finding, we propose a novel LR schedule, called S-Cyclical (S-Cyc) which adapts the conventional cyclical LR schedule by gradually increasing the LR upper bound (max lr) in an S-shape as the network is iteratively pruned. We highlight that S-Cyc is a method agnostic LR schedule that applies to many iterative pruning methods. (iii) We evaluate the performance of the proposed S-Cyc and compare it to four LR schedule benchmarks. Our experimental results on three state-of-the-art networks (e.g., VGG-19, ResNet-20, ResNet-50) and two popular datasets (e.g., CIFAR-10, ImageNet-200) demonstrate that S-Cyc consistently outperforms the best performing benchmark with an improvement of 2.1% - 3.4%, without substantial increase in complexity. (iv) We evaluate S-Cyc against an oracle and show that S-Cyc achieves comparable performance to the oracle, which carefully tunes max lr via grid search. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

PropMix: Hard sample filtering and proportional mixup for learning with noisy labels

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Cordeiro, Filipe R. Belagiannis, Vasileios Reid, Ian Carneiro, Gustavo Universidade Federal Rural de Pernambuco Recife Brazil Universität Ulm Ulm Germany University of Adelaide Adelaide Australia

The most competitive noisy label learning methods rely on an unsupervised classification of clean and noisy samples, where samples classified as noisy are re-labelled and "MixMatched" with the clean samples. These methods have two issues in large noise rate problems: 1) the noisy set is more likely to contain hard samples that are incorrectly re-labelled, and 2) the number of samples produced by MixMatch tends to be reduced because it is constrained by the small clean set size. In this paper, we introduce the learning algorithm PropMix to handle the issues above. PropMix filters out hard noisy samples, with the goal of increasing the likelihood of correctly re-labelling the easy noisy samples. Also, PropMix places clean and re-labelled easy noisy samples in a training set that is augmented with MixUp, removing the clean set size constraint and including a large proportion of correctly re-labelled easy noisy samples. We also include self-supervised pre-training to improve robustness to high noisy label scenarios. Our experiments show that PropMix has state-of-the-art (SOTA) results on CIFAR-10/-100 (with symmetric, asymmetric and semantic label noise), Red Mini-ImageNet (from the Controlled Noisy Web Labels), Clothing1M and WebVision. In severe label noise benchmarks, our results are substantially better than other methods. The code is available at https://***/filipe-research/PropMix. © 2021, CC BY.

关键词： learning algorithms

A mechanistic account of serotonin's impact on mood

学校读者我要写书评

暂无评论

NATURE COMMUNICATIONS 2020年第1期11卷 2335-2335页

作者： Michely, Jochen Eldar, Eran Martin, Ingrid M. Dolan, Raymond J. UCL Max Planck UCL Ctr Computat Psychiat & Ageing Res London England UCL Wellcome Ctr Human Neuroimaging London England Charite Univ Med Berlin Dept Psychiat & Psychotherapy Berlin Germany Hebrew Univ Jerusalem Psychol & Cognit Sci Dept Jerusalem Israel

Selective serotonin reuptake inhibitors (SSRIs) constitute a first-line antidepressant intervention, though the precise cognitive and computational mechanisms that explain treatment response remain elusive. Using week-long SSRI treatment in healthy volunteer participants, we show serotonin enhances the impact of experimentally induced positive affect on learning of novel, and reconsolidation of previously learned, reward associations. Computational modelling indicated these effects are best accounted for by a boost in subjective reward perception during learning, following a positive, but not negative, mood induction. Thus, instead of influencing affect or reward sensitivity directly, SSRIs might amplify an interaction between the two, giving rise to a delayed mood response. We suggest this modulation of affect-learning dynamics may explain the evolution of a gradual mood improvement seen with these agents and provides a novel candidate mechanism for the unfolding of serotonin's antidepressant effects over time. The cognitive computational mechanisms underlying the antidepressant treatment response of SSRIs is not well understood. Here the authors show that SSRI treatment in healthy subjects for a week manifests as an amplification of the perception of positive outcomes when learning occurs in a positive mood setting.

关键词： Computational neuroscience learning algorithms Cognitive neuroscience Decision Computational neuroscience

A learning Paradigm for Selecting Few Discriminative Stimuli in Eye-Tracking Research

学校读者我要写书评

暂无评论

IEEE Transactions on Pattern Analysis and Machine Intelligence 2025年 PP卷 PP页

作者： Zhong, Wenqi Xia, Chen Yu, Linzhi Li, Kuan Li, Zhongyu Zhang, Dingwen Han, Junwei Northwestern Polytechnical University School of Automation Xi'an710072 China Hospital of Northwest University Xi'an710069 China Xi'an Jiaotong University School of Software Engineering Xi'an710049 China

Eye-tracking is a reliable method for quantifying visual information processing and holds significant potential for group recognition, such as identifying autism spectrum disorder (ASD). However, eye-tracking research typically faces the heterogeneity of stimuli and is time-consuming due to the large number of observed stimuli. To address these issues, we first mathematically define the stimulus selection problem and introduce the concept of stimulus discrimination ability to reduce the computational complexity of the solution. Then, we construct a scanpath-based recognition model to mine the stimulus discrimination ability. Specifically, we propose cross-subject entropy and cross-subject divergence scores for quantitatively evaluating stimulus discrimination ability, effectively capturing differences in intra-group collective trends and inter-subject consistency within a group. Furthermore, we propose an iterative learning mechanism that employs stimulus-wise attention to focus on discriminative stimuli for discrimination purification. In the experiment, we construct an ASD eye-tracking dataset with diverse stimulus types and conduct extensive tests on three representative models to validate our approach. Remarkably, our method demonstrates superior performance using only 10 selected stimuli compared to models utilizing 220 stimuli. Additionally, we perform experiments on another eye-tracking task, gender prediction, to further validate our method. We believe that our approach is both simple and flexible for integration into existing models, promoting large-scale ASD screening and extending to other eye-tracking research domains. © 1979-2012 IEEE.

关键词： learning algorithms

CDMA: A Practical Cross-Device Federated learning Algorithm for General Minimax Problems

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Xie, Jiahao Zhang, Chao Shen, Zebang Liu, Weijie Qian, Hui College of Computer Science and Technology Zhejiang University China Advanced Technology Institute Zhejiang University China ETH Zürich Switzerland Qiushi Academy for Advanced Studies Zhejiang University China State Key Lab of CAD&CG Zhejiang University China

Minimax problems arise in a wide range of important applications including robust adversarial learning and Generative Adversarial Network (GAN) training. Recently, algorithms for minimax problems in the Federated learning (FL) paradigm have received considerable interest. Existing federated algorithms for general minimax problems require the full aggregation (i.e., aggregation of local model information from all clients) in each training round. Thus, they are inapplicable to an important setting of FL known as the cross-device setting, which involves numerous unreliable mobile/IoT devices. In this paper, we develop the first practical algorithm named CDMA for general minimax problems in the cross-device FL setting. CDMA is based on a Start-Immediately-With-Enough-Responses mechanism, in which the server first signals a subset of clients to perform local computation and then starts to aggregate the local results reported by clients once it receives responses from enough clients in each round. With this mechanism, CDMA is resilient to the low client availability. In addition, CDMA is incorporated with a lightweight global correction in the local update steps of clients, which mitigates the impact of slow network connections. We establish theoretical guarantees of CDMA under different choices of hyperparameters and conduct experiments on AUC maximization, robust adversarial network training, and GAN training tasks. Theoretical and experimental results demonstrate the efficiency of CDMA. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms