检索结果-内蒙古大学图书馆

A REDUCTION-BASED FRAMEWORK FOR CONSERVATIVE BANDITS AND REINFORCEMENT learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Yang, Yunchang Wu, Tianhao Zhong, Han Garcelon, Evrard Pirotta, Matteo Lazaric, Alessandro Wang, Liwei Du, Simon S. Center for Data Science Peking University China University of California Berkeley United States Center for Data Sience Peking University China Facebook AI Research Key Laboratory of Machine Perception MOE School of Artificial Intelligence Peking University International Center for Machine Learning Research China University of Washington United States

We study bandits and reinforcement learning (RL) subject to a conservative constraint where the agent is asked to perform at least as well as a given baseline policy. This setting is particular relevant in real-world domains including digital marketing, healthcare, production, finance, etc. In this paper, we present a reduction-based framework for conservative bandits and RL, in which our core technique is to calculate the necessary and sufficient budget obtained from running the baseline policy. For lower bounds, we improve the existing lower bound for conservative multi-armed bandits and obtain new lower bounds for conservative linear bandits, tabular RL and low-rank MDP, through a black-box reduction that turns a certain lower bound in the nonconservative setting into a new lower bound in the conservative setting. For upper bounds, in multi-armed bandits, linear bandits and tabular RL, our new upper bounds tighten or match existing ones with significantly simpler analyses. We also obtain a new upper bound for conservative low-rank MDP. © 2021, CC BY-NC-SA.

关键词： Budget control

Tackling Noisy Labels with Network Parameter Additive Decomposition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Jingyi Xia, Xiaobo Lan, Long Wu, Xinghao Yu, Jun Yang, Wenjing Han, Bo Liu, Tongliang The Department of Intelligent Data Science College of Computer Science and Technology National University of Defense Technology Changsha410073 China The Trustworthy Machine Learning Lab School of Computer Science Faculty of Engineering University of Sydney DarlingtonNSW2008 Australia The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China The Department of Automation University of Science and Technology of China Hefei230026 China The Department of Computer Science Hong Kong Baptist University Hong Kong

Given data with noisy labels, over-parameterized deep networks suffer overfitting mislabeled data, resulting in poor generalization. The memorization effect of deep networks shows that although the networks have the ability to memorize all noisy data, they would first memorize clean training data, and then gradually memorize mislabeled training data. A simple and effective method that exploits the memorization effect to combat noisy labels is early stopping. However, early stopping cannot distinguish the memorization of clean data and mislabeled data, resulting in the network still inevitably overfitting mislabeled data in the early training stage. In this paper, to decouple the memorization of clean data and mislabeled data, and further reduce the side effect of mislabeled data, we perform additive decomposition on network parameters. Namely, all parameters are additively decomposed into two groups, i.e. , parameters w are decomposed as w = σ + γ. Afterward, the parameters σ are considered to memorize clean data, while the parameters γ are considered to memorize mislabeled data. Benefiting from the memorization effect, the updates of the parameters σ are encouraged to fully memorize clean data in early training, and then discouraged with the increase of training epochs to reduce interference of mislabeled data. The updates of the parameters γ are the opposite. In testing, only the parameters σ are employed to enhance generalization. Extensive experiments on both simulated and real-world benchmarks confirm the superior performance of our method. The implementation is available at: https://***/wangjingyi9924/TNLPAD. Copyright © 2024, The Authors. All rights reserved.

关键词： Benchmarking

learning referee evaluation and assessing action quality from coarse to fine in diving sport

学校读者我要写书评

暂无评论

Neurocomputing 2025年 648卷

作者： Hong-Ming Qiu Hong-Bo Zhang Qing Lei Jing-Hua Liu Ji-Xiang Du Department of Computer Science and Technology Huaqiao University Xiamen 361021 China Fujian Key Laboratory of Big Data Intelligence and Security Huaqiao University Xiamen 361021 China Key Laboratory of Computer Vision and Machine Learning (Huaqiao University) Fujian Province University Xiamen 361021 China Xiamen Key Laboratory of Computer Vision and Pattern Recognition Huaqiao University Xiamen 361021 China

Intelligently assessing the quality of athletic performances in sports scenarios remains a fascinating challenge in computer vision. However, unraveling the subtle distinctions between two similar actions in videos and mapping those video representations to quality scores remain significant obstacles. To address these challenges, this work redefines the paradigm of quality score estimation from traditional relative quality score prediction to relative referee score prediction. To make this shift, a cross-feature fusion module rooted in Transformer-based video representation is introduced, to improve pairwise video feature learning in the realm of action quality assessment. Then, a novel contrastive action parsing decoder module generates mid-level representations to effectively connect visual features with detailed quality scores. Both modules utilize cross-attention mechanisms; the former refines the pairwise video features to represent the differences between video pairs, while the latter updates the input queries corresponding to each referee’s evaluation. Finally, to achieve precise quality score estimation, we introduce a meticulous coarse-to-fine decision process, integrating a score classifier and offset regressor. After validation on challenging diving datasets, including MTL-AQA, FineDiving, and TASD-2, the experimental results show that the proposed approach demonstrates effectiveness and feasibility when compared with state-of-the-art methods.

关键词： Athlete performance quality assessment Coarse-to-fine decision Contrastive action parsing decoder Cross feature fusion Relative referee score prediction

Voting-Based Multiagent Reinforcement learning for Intelligent IoT

学校读者我要写书评

暂无评论

IEEE INTERNET OF THINGS JOURNAL 2021年第4期8卷 2681-2693页

作者： Xu, Yue Deng, Zengde Wang, Mengdi Xu, Wenjun So, Anthony Man-Cho Cui, Shuguang Beijing Univ Posts & Telecommun Minist Educ Key Lab Universal Wireless Commun Beijing 100876 Peoples R China Chinese Univ Hong Kong Shenzhen Res Inst Big Data Shenzhen 518172 Peoples R China Chinese Univ Hong Kong Sch Sci & Engn Shenzhen 518172 Peoples R China Chinese Univ Hong Kong Dept Syst Engn & Engn Management Hong Kong Peoples R China Princeton Univ Dept Elect Engn Ctr Stat & Machine Learning Dept Operat Res & Financial Engn Princeton NJ 08544 USA Princeton Univ Dept Comp Sci Princeton NJ 08544 USA Chinese Univ Hong Kong Future Network Intelligence Inst Shenzhen 518172 Peoples R China

The recent success of single-agent reinforcement learning (RL) in Internet of Things (IoT) systems motivates the study of multiagent RL (MARL), which is more challenging but more useful in large-scale IoT. In this article, we consider a voting-based MARL problem, in which the agents vote to make group decisions and the goal is to maximize the globally averaged returns. To this end, we formulate the MARL problem based on the linear programming form of the policy optimization problem and propose a primal-dual algorithm to obtain the optimal solution. We also propose a voting mechanism through which the distributed learning achieves the same sublinear convergence rate as centralized learning. In other words, the distributed decision making does not slow down the process of achieving global consensus on optimality. Finally, we verify the convergence of our proposed algorithm with numerical simulations and conduct case studies in practical multiagent IoT systems.

关键词： Convergence Internet of Things Optimization Collaboration Task analysis learning (artificial intelligence) Games Multiagent reinforcement learning (MARL) primal– dual algorithm voting mechanism

Computationally Efficient Approximations for Matrix-based Rényi's Entropy

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Gong, Tieliang Dong, Yuxin Yu, Shujian Dong, Bo The School of Computer Science and Technology Xi'an Jiaotong University Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Xi’an710049 China The Machine Learning Group UiT - The Arctic University of Norway Department of Computer Science Vrije University Amsterdam Amsterdam Netherlands

The recently developed matrix-based Rényi's αorder entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi-definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the underlying data distribution. This intriguing property makes this new information measurement widely adopted in multiple statistical inference and learning tasks. However, the computation of such quantity involves the trace operator on a PSD matrix G to power α (i.e., tr(Gα)), with a normal complexity of nearly O(n3), which severely hampers its practical usage when the number of samples (i.e., n) is large. In this work, we present computationally efficient approximations to this new entropy functional that can reduce its complexity to even significantly less than O(n2). To this end, we leverage the recent progress on Randomized Numerical Linear Algebra, developing Taylor, Chebyshev and Lanczos approximations to tr(Gα) for arbitrary values of α by converting it into a matrix-vector multiplication problem. We also establish the connection between the matrix-based Rényi's entropy and PSD matrix approximation, which enables exploiting both clustering and block low-rank structure of G to further reduce the computational cost. We theoretically provide approximation accuracy guarantees and illustrate the properties for different approximations. Large-scale experimental evaluations on both synthetic and real-world data corroborate our theoretical findings, showing promising speedup with negligible loss in accuracy. Copyright © 2021, The Authors. All rights reserved.

关键词： Matrix algebra

Robust and Fast Measure of Information via Low-rank Representation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Dong, Yuxin Gong, Tieliang Yu, Shujian Chen, Hong Li, Chen School of Computer Science and Technology Xi’an Jiaotong University Xi’an710049 China Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Ministry of Education Xi’an710049 China Machine Learning Group UiT - The Arctic University of Norway Norway College of Science Huazhong Agriculture University Wuhan430070 China Engineering Research Center of Intelligent Technology for Agriculture Ministry of Education Wuhan430070 China

The matrix-based Rényi’s entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution. This intriguing property makes it widely applied in statistical inference and machine learning tasks. However, this information theoretical quantity is not robust against noise in the data, and is computationally prohibitive in large-scale applications. To address these issues, we propose a novel measure of information, termed low-rank matrix-based Rényi’s entropy, based on low-rank representations of infinitely divisible kernel matrices. The proposed entropy functional inherits the specialty of of the original definition to directly quantify information from data, but enjoys additional advantages including robustness and effective calculation. Specifically, our low-rank variant is more sensitive to informative perturbations induced by changes in underlying distributions, while being insensitive to uninformative ones caused by noises. Moreover, low-rank Rényi’s entropy can be efficiently approximated by random projection and Lanczos iteration techniques, reducing the overall complexity from O(n3) to O(n2s) or even O(ns2), where n is the number of data samples and s n. We conduct large-scale experiments to evaluate the effectiveness of this new information measure, demonstrating superior results compared to matrix-based Rényi’s entropy in terms of both performance and computational efficiency. Copyright © 2022, The Authors. All rights reserved.

关键词： Entropy

Adversarial learning with Cost-Sensitive Classes

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Shen, Haojing Chen, Sihong Wang, Ran Wang, Xizhao Big Data Institute College of Computer Science and Software Engineering Guangdong Key Lab. of Intelligent Information Processing Shenzhen University Guangdong Shenzhen518060 China The College of Mathematics and Statistics Shenzhen University Shenzhen518060 China The Shenzhen Key Laboratory of Advanced Machine Learning and Applications Shenzhen University Shenzhen518060 China

It is necessary to improve the performance of some special classes or to particularly protect them from attacks in adversarial learning. This paper proposes a framework combining cost-sensitive classification and adversarial learning together to train a model that can distinguish between protected and unprotected classes, such that the protected classes are less vulnerable to adversarial examples. We find in this framework an interesting phenomenon during the training of deep neural networks, called Min-Max property, that is, the absolute values of most parameters in the convolutional layer approach zero while the absolute values of a few parameters are significantly larger becoming bigger. Based on this Min-Max property which is formulated and analyzed in a view of random distribution, we further build a new defense model against adversarial examples for adversarial robustness improvement. An advantage of the built model is that it performs better than the standard one and can combine with adversarial training to achieve an improved performance. It is experimentally confirmed that, regarding the average accuracy of all classes, our model is almost as same as the existing models when an attack does not occur and is better than the existing models when an attack occurs. Specifically, regarding the accuracy of protected classes, the proposed model is much better than the existing models when an attack occurs. © 2021, CC BY.

关键词： Deep neural networks

The effect of deterministic noise on a quasi-subgradient method for quasi-convex feasibility problems

学校读者我要写书评

暂无评论

Journal of Applied and Numerical Optimization

Journal of Applied and Numerical Optimization 2020年第2期2卷 235-247页

作者： Hu, Yaohua Liu, Yuping Li, Minghua Shenzhen Key Laboratory of Advanced Machine Learning and Applications College of Mathematics and Statistics Shenzhen University Shenzhen518060 China School of Mathematics and Big Data Chongqing University of Arts and Sciences Yongchuan Chongqing402160 China

The quasi-convex feasibility problem (QFP), in which the involved functions are quasi-convex, is at the core of the modeling of many problems in various areas such as economics, finance and management science. In this paper, we consider an inexact incremental quasi-subgradient method to solve the QFP, in which an incremental control of component functions in the QFP is employed and the inexactness stems from computation error and noise arising from practical considerations and physical circumstances. Under the assumptions that the computation error and noise are deterministic and bounded and a Hölder condition on component functions in the QFP, we study the convergence property of the proposed inexact incremental quasi-subgradient method, and particularly, investigate the effect of the inexact terms on the incremental quasi-subgradient method when using the constant, diminishing and dynamic stepsize rules. © 2020 Journal of Applied and Numerical Optimization.

关键词： Numerical methods

Overview of the Tenth Dialog System Technology Challenge: DSTC10

学校读者我要写书评

暂无评论

IEEE/ACM Transactions on Audio Speech and Language Processing 2024年 32卷 765-778页

作者： Yoshino, Koichiro Chen, Yun-Nung Crook, Paul Kottur, Satwik Li, Jinchao Hedayatnia, Behnam Moon, Seungwhan Fei, Zhengcong Li, Zekang Zhang, Jinchao Feng, Yang Zhou, Jie Kim, Seokhwan Liu, Yang Jin, Di Papangelis, Alexandros Gopalakrishnan, Karthik Hakkani-Tur, Dilek Damavandi, Babak Geramifard, Alborz Hori, Chiori Shah, Ankit Zhang, Chen Li, Haizhou Sedoc, Joao D'haro, Luis F. Banchs, Rafael Rudnicky, Alexander Guardian Robot Project R-IH RIKEN 2-2-2 Hikaridai Seika Shoraku619-0288 Japan Information Science Nara Institute of Science and Technology Ikoma630-0101 Japan Computer Science and Information Engineering National Taiwan University Taipei10617 Taiwan Inc. Palo AltoCA95054 United States Alexa AI *** Inc. SunnyvaleCA94089 United States Meta Seattle RedmondWA98052 United States Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China Tencent AI Lab Beijing Beijing China Kexueyuan South Road Zhongguancun Beijing100190 China Beijing 100190 China Alexa AI *** Inc. SunnyvaleCA United States 1120 Enterprise way Sunnyvale94089 United States *** Inc. SeattleWA United States Menlo Park CA United States Audio and Speech Group Mitsubishi Electric Research Laboratories CambridgeMA02139-1955 United States Carnegie Mellon University Department of Language and Information Technologies or just Carnegie Mellon University Pittsburgh United States National University of Singapore Singapore Singapore Department of Electrical and Computer Engineering National University of Singapore Singapore Singapore Shenzhen Research Institute of Big Data School of Data Science Chinese University of Hong Kong Shenzhen518172 China New York University New YorkNY United States ETSI de Telecomunicacion - Speech Technology and Machine Learning Group Universidad Politecnica de Madrid Ciudad Universitaria Madrid28040 Spain Nanyang Technological University Singapore Singapore Carnegie Mellon University PittsburghPA United States

This article introduces the Tenth Dialog System Technology Challenge (DSTC-10). This edition of the DSTC focuses on applying end-to-end dialog technologies for five distinct tasks in dialog systems, namely 1. Incorporation of Meme images into open domain dialogs, 2. Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations, 3. Situated Interactive Multimodal dialogs, 4. Reasoning for Audio Visual Scene-Aware Dialog, and 5. Automatic Evaluation and Moderation of Open-domainDialogue Systems. This article describes the task definition, provided datasets, baselines, and evaluation setup for each track. We also summarize the results of the submitted systems to highlight the general trends of the state-of-the-art technologies for the tasks. © 2023 The Authors.

关键词： Job analysis