检索结果-内蒙古大学图书馆

Adaptive gradient methods with dynamic bound of learning rate

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Luo, Liangchen Xiong, Yuanhao Liu, Yan Sun, Xu MOE Key Lab of Computational Linguistics School of EECS Peking University College of Information Science and Electronic Engineering Zhejiang University Department of Computer Science University of Southern California Center for Data Science Beijing Institute of Big Data Research Peking University

Adaptive optimization methods such as ADAGRAD, RMSPROP and ADAM have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates. Though prevailing, they are observed to generalize poorly compared with SGD or even fail to converge due to unstable and extreme learning rates. Recent work has put forward some algorithms such as AMSGRAD to tackle this issue but they failed to achieve considerable improvement over existing methods. In our paper, we demonstrate that extreme learning rates can lead to poor performance. We provide new variants of ADAM and AMSGRAD, called ADABOUND and AMSBOUND respectively, which employ dynamic bounds on learning rates to achieve a gradual and smooth transition from adaptive methods to SGD and give a theoretical proof of convergence. We further conduct experiments on various popular tasks and models, which is often insufficient in previous work. Experimental results show that new variants can eliminate the generalization gap between adaptive methods and SGD and maintain higher learning speed early in training at the same time. Moreover, they can bring significant improvement over their prototypes, especially on complex deep networks. The implementation of the algorithm can be found at https://***/Luolc/AdaBound. Copyright © 2019, The Authors. All rights reserved.

关键词： Gradient methods

COCO-CN for cross-lingual image tagging, captioning and retrieval

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Li, Xirong Xu, Chaoxi Wang, Xiaoxu Lan, Weiyu Jia, Zhengxiong Yang, Gang Xu, Jieping Key Lab of Data Engineering and Knowledge Engineering Renmin University of China AI & Media Computing Lab School of Information Renmin University of China Beijing100872 China

This paper contributes to cross-lingual image annotation and retrieval in terms of data and baseline methods. We propose COCO-CN, a novel dataset enriching MS-COCO with manually written Chinese sentences and tags. For more effective annotation acquisition, we develop a recommendation-assisted collective annotation system, automatically providing an annotator with several tags and sentences deemed to be relevant with respect to the pictorial content. Having 20,342 images annotated with 27,218 Chinese sentences and 70,993 tags, COCO-CN is currently the largest Chinese-English dataset that provides a unified and challenging platform for cross-lingual image tagging, captioning and retrieval. We develop conceptually simple yet effective methods per task for learning from cross-lingual resources. Extensive experiments on the three tasks justify the viability of the proposed dataset and methods. data and code are publicly available at https://***/li-xirong/coco-cn. Copyright © 2018, The Authors. All rights reserved.

关键词： Image annotation

QUANTUM LOVÁSZ LOCAL LEMMA: SHEARER’S BOUND IS TIGHT

学校读者我要写书评

暂无评论

arXiv 2018年

作者： He, Kun Li, Qian Sun, Xiaoming Zhang, Jiapeng The Key Lab of Data Engineering and Knowledge Engineering MOE Renmin University of China Beijing China Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences Beijing China University of Southern California United States

The Lovász Local Lemma (LLL) is a very powerful tool in combinatorics and probability theory to show the possibility of avoiding all bad events under some weakly dependent conditions. In a seminal paper, Ambainis, Kempe, and Sattath (JACM 2012) introduced a quantum version LLL (QLLL) which shows the possibility of avoiding all "bad" Hamiltonians under some weakly dependent condition, and applied QLLL to the random k-QSAT problem. Sattath, Morampudi, Laumann, and moessner (PNAS 2015) extended Ambainis, Kempe, and Sattath’s result and showed that Shearer’s bound is a sufficient condition for QLLL, and conjectured that Shearer’s bound is indeed the tight condition for QLLL. In this paper, we affirm this conjecture. Precisely, we prove that Shearer’s bound is tight for QLLL, i.e., the relative dimension of the smallest satisfying subspace is completely characterized by the independent set polynomial. Our result implies the tightness of Gilyén and Sattath’s algorithm (FOCS 2017), and also implies that the lattice gas partition function fully characterizes quantum satisfiability for almost all Hamiltonians with large enough qudits (Sattath, Morampudi, Laumann and moessner, PNAS 2015). The commuting LLL (CLLL), which focuses on commuting local Hamiltonians, is also investigated here. We prove that the tight regions of CLLL and QLLL are different in general. This result indicates that it is possible to design an algorithm for CLLL which is still efficient beyond Shearer’s bound. Copyright © 2018, The Authors. All rights reserved.

关键词： Hamiltonians

Impact of Prior knowledge and data Correlation on Privacy Leakage: A Unified Analysis

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Li, Yanan Ren, Xuebin Yang, Shusen Yang, Xinyu National Engineering Laboratory for Big Data Analytics [NEL-BDA Xi’an Jiaotong University Xi’an Shaanxi710049 China School of Mathematics and Statistics Xi’an Jiaotong University Xi’an Shaanxi710049 China School of Electronic and Information Engineering Xi’an Jiaotong University Xi’an Shaanxi710049 China Ministry of Education Key Lab for Intelligent Networks and Network Security [MOE KLINNS Lab Xi’an Jiaotong University Xi’an Shaanxi710049 China

It has been widely understood that differential privacy (DP) can guarantee rigorous privacy against adversaries with arbitrary prior knowledge. However, recent studies demonstrate that this may not be true for correlated data, and indicate that three factors could influence privacy leakage: the data correlation pattern, prior knowledge of adversaries, and sensitivity of the query function. This poses a fundamental problem: what is the mathematical relationship between the three factors and privacy leakage? In this paper, we present a unified analysis of this problem. A new privacy definition, named prior differential privacy (PDP), is proposed to evaluate privacy leakage considering the exact prior knowledge possessed by the adversary. We use two models, the weighted hierarchical graph (WHG) and the multivariate Gaussian model to analyze discrete and continuous data, respectively. We demonstrate that positive, negative, and hybrid correlations have distinct impacts on privacy leakage. Considering general correlations, a closed-form expression of privacy leakage is derived for continuous data, and a chain rule is presented for discrete data. Our results are valid for general linear queries, including count, sum, mean, and histogram. Numerical experiments are presented to verify our theoretical analysis. Copyright © 2019, The Authors. All rights reserved.

关键词： Machine learning

Bridging explicit and implicit deep generative models via neural Stein estimators

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Wu, Qitian Gao, Rui Zha, Hongyuan Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University University of Texas Austin United States School of Data Science Shenzhen Institute of Artificial Intelligence and Robotics for Society The Chinese University of Hong Kong Shenzhen Hong Kong The Chinese University of Hong Kong Shenzhen China

There are two types of deep generative models: explicit and implicit. The former defines an explicit density form that allows likelihood inference;while the latter targets a flexible transformation from random noise to generated samples. While the two classes of generative models have shown great power in many applications, both of them, when used alone, suffer from respective limitations and drawbacks. To take full advantages of both models and enable mutual compensation, we propose a novel joint training framework that bridges an explicit (unnormalized) density estimator and an implicit sample generator via Stein discrepancy. We show that our method 1) induces novel mutual regularization via kernel Sobolev norm penalization and Moreau-Yosida regularization, and 2) stabilizes the training dynamics. Empirically, we demonstrate that proposed method can facilitate the density estimator to more accurately identify data modes and guide the generator to output higher-quality samples, comparing with training a single counterpart. The new approach also shows promising results when the training samples are contaminated or limited. © 2019, CC BY.

关键词：

Learning personalized attribute preference via multi-task auc optimization

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Yang, Zhiyong Xu, Qianqian Cao, Xiaochun Huang, Qingming SKLOIS Institute of Information Engineering Chinese Academy of Sciences Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China University of Chinese Academy of Sciences Beijing China Key Laboratory of Big Data Mining and Knowledge Management CAS Beijing China

Traditionally, most of the existing attribute learning methods are trained based on the consensus of annotations aggregated from a limited number of annotators. However, the consensus might fail in settings, especially when a wide spectrum of annotators with different interests and comprehension about the attribute words are involved. In this paper, we develop a novel multi-task method to understand and predict personalized attribute annotations. Regarding the attribute preference learning for each annotator as a specific task, we first propose a multi-level task parameter decomposition to capture the evolution from a highly popular opinion of the mass to highly personalized choices that are special for each person. Meanwhile, for personalized learning methods, ranking prediction is much more important than accurate classification. This motivates us to employ an Area Under ROC Curve (AUC) based loss function to improve our model. On top of the AUC-based loss, we propose an efficient method to evaluate the loss and gradients. Theoretically, we propose a novel closed-form solution for one of our non-convex subproblem, which leads to provable convergence behaviors. Furthermore, we also provide a generalization bound to guarantee a reasonable performance. Finally, empirical analysis consistently speaks to the efficacy of our proposed method. Copyright © 2019, The Authors. All rights reserved.

关键词： Machine learning

Τ-FPL: Tolerance-constrained learning in linear time 32

学校读者我要写书评

暂无评论

Τ-FPL: Tolerance-constrained learning in linear time

32nd AAAI Conference on Artificial Intelligence, AAAI 2018

作者： Zhang, Ao Li, Nan Pu, Jian Wang, Jun Yan, Junchi Zha, Hongyuan Shanghai Key Laboratory of Trustworthy Computing MOE International Joint Lab of Trustworthy Software School of Computer Science and Software Engineering East China Normal University Shanghai China Institute of Data Science and Technologies Alibaba Group Hangzhou China IBM Research China Georgia Institute of Technology Atlante United States

ISBN: (纸本)9781577358008

In many real-world applications, learning a classifier with false-positive rate under a specified tolerance is appealing. Existing approaches either introduce prior knowledge dependent label cost or tune parameters based on traditional classifiers, which are of limitation in methodology since they do not directly incorporate the false-positive rate tolerance. In this paper, we propose a novel scoring-thresholding approach, τ-False Positive Learning (τ-FPL) to address this problem. We show that the scoring problem which takes the false-positive rate tolerance into accounts can be efficiently solved in linear time, also an out-of-bootstrap thresholding method can transform the learned ranking function into a low false-positive classifier. Both theoretical analysis and experimental results show superior performance of the proposed τ-FPL over the existing approaches. Copyright © 2018, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词： Mathematical transformations

iSplit LBI: Individualized partial ranking with ties via split LBI

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Xu, Qianqian Sun, Xinwei Yang, Zhiyong Cao, Xiaochun Huang, Qingming Yao, Yuan Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS Microsoft Research Asia State Key Laboratory of Information Security Institute of Information Engineering CAS School of Cyber Security University of Chinese Academy of Sciences School of Computer Science and Tech. University of Chinese Academy of Sciences Key Laboratory of Big Data Mining and Knowledge Management CAS Peng Cheng Laboratory Department of Mathematics Hong Kong University of Science and Technology Hong Kong

Due to the inherent uncertainty of data, the problem of predicting partial ranking from pairwise comparison data with ties has attracted increasing interest in recent years. However, in real-world scenarios, different individuals often hold distinct preferences. It might be misleading to merely look at a global partial ranking while ignoring personal diversity. In this paper, instead of learning a global ranking which is agreed with the consensus, we pursue the tie-aware partial ranking from an individualized perspective. Particularly, we formulate a unified framework which not only can be used for individualized partial ranking prediction, but also be helpful for abnormal user selection. This is realized by a variable splitting-based algorithm called iSplitLBI. Specifically, our algorithm generates a sequence of estimations with a regularization path, where both the hyperparameters and model parameters are updated. At each step of the path, the parameters can be decomposed into three orthogonal parts, namely, abnormal signals, personalized signals and random noise. The abnormal signals can serve the purpose of abnormal user selection, while the abnormal signals and personalized signals together are mainly responsible for individual partial ranking prediction. Extensive experiments on simulated and real-world datasets demonstrate that our new approach significantly outperforms state-of-the-art alternatives. The code is now availiable at https://***/qianqianxu010/ NeurIPS2019-iSplitLBI. Copyright © 2019, The Authors. All rights reserved.

关键词： Forecasting

SEED: Entity oriented information search and exploration 22

学校读者我要写书评

暂无评论

SEED: Entity oriented information search and exploration

22nd International Conference on Intelligent User Interfaces, IUI 2017

作者： Chen, Jun Jacucci, Giulio Chen, Yueguo Ruotsalo, Tuukka School of Information Renmin University of China China Key Laboratory of Data Engineering and Knowledge Engineering MOE China Helsinki Institute for Information Technology HIIT Department of Computer Science University of Helsinki Finland

ISBN: (纸本)9781450348935

Entity search and exploration can enrich search user interfaces by presenting relevant information instantly and offering relevant exploration pointers to users. Previous research has demonstrated that large knowledge Graphs allow exploitation and recommendation of explicit links between the entities and other information to improve information access and ranking. However, less attention has been devoted to user interfaces for effectively presenting results, recommending related entities and explaining relations between entities. We introduce a system called SEED which is designed to support entity search and exploration in large knowledge Graphs. We demonstrate SEED using a dataset of hundreds of thousands of movie related entities from the DBpedia knowledge Graph. The system utilizes a graph embedding model for ranking entities and their relations, recommending related entities, and explaining their interrelations. Copyright is held by the author/owner(s).

关键词： knowledge graph