检索结果-内蒙古大学图书馆

Adaptive gradient methods with dynamic bound of learning rate 7

学校读者我要写书评

暂无评论

Adaptive gradient methods with dynamic bound of learning rat...

7th International Conference on Learning Representations, ICLR 2019

作者： Luo, Liangchen Xiong, Yuanhao Liu, Yan Sun, Xu MOE Key Lab of Computational Linguistics School of EECS Peking University China College of Information Science and Electronic Engineering Zhejiang University China Department of Computer Science University of Southern California United States Center for Data Science Beijing Institute of Big Data Research Peking University China

Adaptive optimization methods such as ADAGRAD, RMSPROP and ADAM have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates. Though prevailing, they are observed to generalize poorly compared with SGD or even fail to converge due to unstable and extreme learning rates. Recent work has put forward some algorithms such as AMSGRAD to tackle this issue but they failed to achieve considerable improvement over existing methods. In our paper, we demonstrate that extreme learning rates can lead to poor performance. We provide new variants of ADAM and AMSGRAD, called ADABOUND and AMSBOUND respectively, which employ dynamic bounds on learning rates to achieve a gradual and smooth transition from adaptive methods to SGD and give a theoretical proof of convergence. We further conduct experiments on various popular tasks and models, which is often insufficient in previous work. Experimental results show that new variants can eliminate the generalization gap between adaptive methods and SGD and maintain higher learning speed early in training at the same time. Moreover, they can bring significant improvement over their prototypes, especially on complex deep networks. The implementation of the algorithm can be found at https://***/Luolc/AdaBound. © 7th International Conference on Learning Representations, ICLR 2019. All Rights Reserved.

关键词： Gradient methods

DRAG: Dynamic Region-Aware GCN for Privacy-Leaking Image Detection

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Guang Cao, Juan Sheng, Qiang Qi, Peng Li, Xirong Li, Jintao Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Key Lab of Data Engineering and Knowledge Engineering Renmin University of China China

The daily practice of sharing images on social media raises a severe issue about privacy leakage. To address the issue, privacy-leaking image detection is studied recently, with the goal to automatically identify images that may leak privacy. Recent advance on this task benefits from focusing on crucial objects via pretrained object detectors and modeling their correlation. However, these methods have two limitations: 1) they neglect other important elements like scenes, textures, and objects beyond the capacity of pretrained object detectors;2) the correlation among objects is fixed, but a fixed correlation is not appropriate for all the images. To overcome the limitations, we propose the Dynamic Region-Aware Graph Convolutional Network (DRAG) that dynamically finds out crucial regions including objects and other important elements, and models their correlation adaptively for each input image. To find out crucial regions, we cluster spatially-correlated feature channels into several region-aware feature maps. Further, we dynamically model the correlation with the self-attention mechanism and explore the interaction among the regions with a graph convolutional network. The DRAG achieved an accuracy of 87% on the largest dataset for privacy-leaking image detection, which is 10 percentage points higher than the state of the art. The further case study demonstrates that it found out crucial regions containing not only objects but other important elements like textures. The code and more details are in https://***/guang-yanng/DRAG. © 2022, CC BY.

关键词： Object detection

Personalizing search results using hierarchical RNN with query-aware attention

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Ge, Songwei Dou, Zhicheng Jiang, Zhengbao Nie, Jian-Yun Wen, Ji-Rong School of Information Renmin University of China DIRO Université de Montréal Beijing Key Laboratory of Big Data Management and Analysis Methods National Engineering Laboratory of Big Data System Software Beijing Institute of Technology Key Laboratory of Data Engineering and Knowledge Engineering MOE

Search results personalization has become an effective way to improve the quality of search engines. Previous studies extracted information such as past clicks, user topical interests, query click entropy and so on to tailor the original ranking. However, few studies have taken into account the sequential information underlying previous queries and sessions. Intuitively, the order of issued queries is important in inferring the real user interests. And more recent sessions should provide more reliable personal signals than older sessions. In addition, the previous search history and user behaviors should influence the personalization of the current query depending on their relatedness. To implement these intuitions, in this paper we employ a hierarchical recurrent neural network to exploit such sequential information and automatically generate user profile from historical data. We propose a query-aware attention model to generate a dynamic user profile based on the input query. Significant improvement is observed in the experiment with data from a commercial search engine when compared with several traditional personalization models. Our analysis reveals that the attention model is able to attribute higher weights to more related past sessions after fine training. Copyright © 2019, The Authors. All rights reserved.

关键词： Search engines

Article Reranking by Memory-Enhanced key Sentence Matching for Detecting Previously Fact-Checked Claims

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Sheng, Qiang Cao, Juan Zhang, Xueyao Li, Xirong Zhong, Lei Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Key Lab of Data Engineering and Knowledge Engineering Renmin University of China China

False claims that have been previously fact-checked can still spread on social media. To mitigate their continual spread, detecting previously fact-checked claims is indispensable. Given a claim, existing works retrieve fact-checking articles (FC-articles) for detection and focus on reranking candidate articles in the typical two-stage retrieval framework. However, their performance may be limited as they ignore the following characteristics of FC-articles: (1) claims are often quoted to describe the checked events, providing lexical information besides semantics;and (2) sentence templates to introduce or debunk claims are common across articles, providing pattern information. In this paper, we propose a novel reranker, MTM (Memoryenhanced Transformers for Matching), to rank FC-articles using key sentences selected using event (lexical and semantic) and pattern information. For event information, we propose to finetune the Transformer with regression of ROUGE. For pattern information, we generate pattern vectors as a memory bank to match with the parts containing patterns. By fusing event and pattern information, we select key sentences to represent an article and then predict if the article fact-checks the given claim using the claim, key sentences, and patterns. Experiments on two real-world datasets show that MTM outperforms existing methods. Human evaluation proves that MTM can capture key sentences for explanations. The code and the dataset are at https://***/ICTMCG/MTM. © 2021, CC BY-SA.

关键词： Semantics

RPR-Net: A Point Cloud-based Rotation-aware Large Scale Place Recognition Network

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Fan, Zhaoxin Song, Zhenbo Zhang, Wenping Liu, Hongyan He, Jun Du, Xiaoyong Key Laboratory of Data Engineering and Knowledge Engineering of MOE School of Information Renmin University of China Beijing100872 China Department of Management Science and Engineering Tsinghua University Beijing100084 China School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing210094 China

Point cloud-based large scale place recognition is an important but challenging task for many applications such as Simultaneous Localization and Mapping (SLAM). Taking the task as a point cloud retrieval problem, previous methods have made delightful achievements. However, how to deal with catastrophic collapse caused by rotation problems is still under-explored. In this paper, to tackle the issue, we propose a novel Point Cloud-based Rotation-aware Large Scale Place Recognition Network (RPR-Net). In particular, to solve the problem, we propose to learn rotation-invariant features in three steps. First, we design three kinds of novel Rotation-Invariant Features (RIFs), which are low-level features that can hold the rotation-invariant property. Second, using these RIFs, we design an attentive module to learn rotation-invariant kernels. Third, we apply these kernels to previous point cloud features to generate new features, which is the well-known SO(3) mapping process. By doing so, high-level scene-specific rotation-invariant features can be learned. We call the above process an Attentive Rotation-Invariant Convolution (ARIConv). To achieve the place recognition goal, we build RPR-Net, which takes ARIConv as a basic unit to construct a dense network architecture. Then, powerful global descriptors used for retrieval-based place recognition can be sufficiently extracted from RPR-Net. Experimental results on prevalent datasets show that our method achieves comparable results to existing state-of-the-art place recognition models and significantly outperforms other rotation-invariant baseline models when solving rotation problems. Copyright © 2021, The Authors. All rights reserved.

关键词： Network architecture

A two-stage dynamic credit risk assessment system 20

学校读者我要写书评

暂无评论

A two-stage dynamic credit risk assessment system

4th International Conference on Deep Learning Technologies, ICDLT 2020

作者： Li, Rui Deng, Shizhe Zhang, Jianquan He, Hao Jin, Yaohui Duan, Jiangang Department of Electronic Engineering Shanghai Jiao Tong University Shanghai China Shanghai Institute for Advanced Communication and Data Science Department of Electronic Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artifcial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China Seassoon Technology Shanghai China

ISBN: (纸本)9781450375481

Credit risk assessment has been thought of as a critical factor in financial companies and banks in the history of development economics. Recently, there has been renewed interest in credit risk assessment using deep learning methods. However, previous studies have not fine-grained dealt with static and dynamic features, which limits their effectiveness. Thus, in this paper, we present a two-stage model using FeedForward Neural Network(FNN) and Recurrent Neural Network(RNN). First, we design the aggregation layer to extract representative information from the static feature at time T. Second, the distinct moment representation constructs the dynamic features of a client. The dynamic features could be learned by the RNN layer. Experimental results on the real-world dataset show its superiority over various baselines. © 2020 ACM.

关键词： Risk assessment

SHLE: Devices Tracking and Depth Filtering for Stereo-based Height Limit Estimation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fan, Zhaoxin Yang, Kaixing Zhang, Min Song, Zhenbo Liu, Hongyan He, Jun Key Laboratory of Data Engineering and Knowledge Engineering of MOE School of Information Renmin University of China Beijing100872 China School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing210094 China Department of Management Science and Engineering Tsinghua University Beijing100084 China

Recently, over-height vehicle strike frequently occurs, causing great economic cost and serious safety problems. Hence, an alert system which can accurately discover any possible height limiting devices in advance is necessary to be employed in modern large or medium sized cars, such as touring cars. Detecting and estimating the height limiting devices act as the key point of a successful height limit alert system. Though there are some works research height limit estimation, existing methods are either too computational expensive or not accurate enough. In this paper, we propose a novel stereo-based pipeline named SHLE for height limit estimation. Our SHLE pipeline consists of two stages. In stage 1, a novel devices detection and tracking scheme is introduced, which accurately locate the height limit devices in the left or right image. Then, in stage 2, the depth is temporally measured, extracted and filtered to calculate the height limit device. To benchmark the height limit estimation task, we build a large-scale dataset named "Disparity Height", where stereo images, pre-computed disparities and ground-truth height limit annotations are provided. We conducted extensive experiments on "Disparity Height" and the results show that SHLE achieves an average error below than 10cm though the car is 70m away from the devices. Our method also outperforms all compared baselines and achieves state-of-the-art performance. Code is available at https://***/Yang-Kaixing/SHLE. Copyright © 2022, The Authors. All rights reserved.

关键词： Pipelines

Deep knn for medical image classification 23rd

学校读者我要写书评

暂无评论

Deep knn for medical image classification

23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020

作者： Zhuang, Jiaxin Cai, Jiabin Wang, Ruixuan Zhang, Jianguo Zheng, Wei-Shi School of Data and Computer Science Sun Yat-sen University Guangzhou China Department of Computer Science and Engineering Southern University of Science and Technology Shenzhen China Key Laboratory of Machine Intelligence and Advanced Computing MOE Guangzhou China Pazhou Lab Guangzhou China

ISBN: (纸本)9783030597092

Human-level diagnostic performance from intelligent systems often depends on large set of training data. However, the amount of available data for model training may be limited for part of diseases, which would cause the widely adopted deep learning models not generalizing well. One alternative simple approach to small class prediction is the traditional k-nearest neighbor (kNN). However, due to the non-parametric characteristics of kNN, it is difficult to combine the kNN classification into the learning of feature extractor. This paper proposes an end-to-end learning strategy to unify the kNN classification and the feature extraction procedure. The basic idea is to enforce that each training sample and its K nearest neighbors belong to the same class during learning the feature extractor. Experiments on multiple small-class and class-imbalanced medical image datasets showed that the proposed deep kNN outperforms both kNN and other strong classifiers. © Springer Nature Switzerland AG 2020.

关键词： Nearest neighbor search

An MRC Framework for Semantic Role labeling

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wang, Nan Li, Jiwei Meng, Yuxian Sun, Xiaofei Qiu, Han Wang, Ziyao Wang, Guoyin He, Jun Key Laboratory of Data Engineering and Knowledge Engineering of MOE School of Information Renmin University of China China Shannon.AI China Zhejiang University China Tsinghua University China University of Leeds United Kingdom Amazon Alexa AI United States

Semantic Role labeling (SRL) aims at recognizing the predicate-argument structure of a sentence and can be decomposed into two subtasks: predicate disambiguation and argument labeling. Prior work deals with these two tasks independently, which ignores the semantic connection between the two tasks. In this paper, we propose to use the machine reading comprehension (MRC) framework to bridge this gap. We formalize predicate disambiguation as multiple-choice machine reading comprehension, where the descriptions of candidate senses of a given predicate are used as options to select the correct sense. The chosen predicate sense is then used to determine the semantic roles for that predicate, and these semantic roles are used to construct the query for another MRC model for argument labeling. In this way, we are able to leverage both the predicate semantics and the semantic role semantics for argument labeling. We also propose to select a subset of all the possible semantic roles for computational efficiency. Experiments show that the proposed framework achieves state-of-the-art or comparable results to previous work. Code is available at https://***/ShannonAI/MRC-SRL. © 2021, CC BY.

关键词： Semantics