检索结果-内蒙古大学图书馆

REDUCR: robust data downsampling using class priority reweighting 24

学校读者我要写书评

暂无评论

REDUCR: robust data downsampling using class priority reweig...

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： William Bankes George Hughes Ilija Bogunovic Zi Wang Department of Computer Science University College London Department of Electrical Engineering University College London Google DeepMind

ISBN: (纸本)9798331314385

Modern machine learning models are becoming increasingly expensive to train for real-world image and text classification tasks, where massive web-scale data is collected in a streaming fashion. To reduce the training cost, online batch selection techniques have been developed to choose the most informative datapoints. However, many existing techniques are not robust to class imbalance and distributional shifts, and can suffer from poor worst-class generalization performance. This work introduces REDUCR, a robust and efficient data downsampling method that uses class priority reweighting. REDUCR reduces the training data while preserving worst-class generalization performance. REDUCR assigns priority weights to datapoints in a class-aware manner using an online learning algorithm. We demonstrate the data efficiency and robust performance of REDUCR on vision and text classification tasks. On web-scraped datasets with imbalanced class distributions, REDUCR significantly improves worst-class test accuracy (and average accuracy), surpassing state-of-the-art methods by around 15%.

关键词：

Value Profiles for Encoding Human Variation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Sorensen, Taylor Mishra, Pushkar Patel, Roma Tessler, Michael Henry Bakker, Michiel Evans, Georgina Gabriel, Iason Goodman, Noah Rieser, Verena Department of Computer Science University of Washington SeattleWA United States Google DeepMind London United Kingdom

Modelling human variation in rating tasks is crucial for enabling AI systems for personalization, pluralistic model alignment, and computational social science. We propose representing individuals using value profiles – natural language descriptions of underlying values compressed from in-context demonstrations – along with a steerable decoder model to estimate ratings conditioned on a value profile or other rater information. To measure the predictive information in rater representations, we introduce an information-theoretic methodology. We find that demonstrations contain the most information, followed by value profiles and then demographics. However, value profiles offer advantages in terms of scrutability, interpretability, and steerability due to their compressed natural language format. Value profiles effectively compress the useful information from demonstrations (>70% information preservation). Furthermore, clustering value profiles to identify similarly behaving individuals better explains rater variation than the most predictive demographic groupings. Going beyond test set performance, we show that the decoder models interpretably change ratings according to semantic profile differences, are well-calibrated, and can help explain instance-level disagreement by simulating an annotator population. These results demonstrate that value profiles offer novel, predictive ways to describe individual variation beyond demographics or group information. © 2025, CC BY.

关键词： Decoding

Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections

学校读者我要写书评

暂无评论

Distilling and Retrieving Generalizable Knowledge for Robot ...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Lihan Zha Yuchen Cui Li-Heng Lin Minae Kwon Montserrat Gonzalez Arenas Andy Zeng Fei Xia Dorsa Sadigh Computer Science Department Stanford University Stanford CA USA Google Deepmind Moutain View CA

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

Today’s robot policies exhibit subpar performance when faced with the challenge of generalizing to novel environments. Human corrective feedback is a crucial form of guidance to enable such generalization. However, adapting to and learning from online human corrections is a non-trivial endeavor: not only do robots need to remember human feedback over time to retrieve the right information in new settings and reduce the intervention rate, but also they would need to be able to respond to feedback that can be arbitrary corrections about high-level human preferences to low-level adjustments to skill parameters. In this work, we present Distillation and Retrieval of Online Corrections (DROC), a large language model (LLM)-based system that can respond to arbitrary forms of language feedback, distill generalizable knowledge from corrections, and retrieve relevant past experiences based on textual and visual similarity for improving performance in novel settings. DROC is able to respond to a sequence of online language corrections that address failures in both high-level task plans and low-level skill primitives. We demonstrate that DROC effectively distills the relevant information from the sequence of online corrections in a knowledge base and retrieves that knowledge in settings with new task or object instances. DROC outperforms other techniques that directly generate robot code via LLMs [1] by using only half of the total number of corrections needed in the first round and requires little to no corrections after two iterations. We show further results and videos on our project website: https://***/***/droc.

关键词： Visualization Codes Large language models Knowledge based systems Task analysis Robots Videos

Would I have gotten that reward? long-term credit assignment by counterfactual contribution analysis 23

学校读者我要写书评

暂无评论

Would I have gotten that reward? long-term credit assignment...

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Alexander Meulemans Simon Schug Seijin Kobayashi Nathaniel D. Daw Gregory Wayne Department of Computer Science ETH Zürich Google DeepMind and Princeton Neuroscience Institute Princeton University and Department of Psychology Princeton University Google DeepMind

To make reinforcement learning more sample efficient, we need better credit assignment methods that measure an action's influence on future rewards. Building upon Hindsight Credit Assignment (HCA) [1], we introduce Counterfactual Contribution Analysis (COCOA), a new family of model-based credit assignment algorithms. Our algorithms achieve precise credit assignment by measuring the contribution of actions upon obtaining subsequent rewards, by quantifying a counterfactual query: 'Would the agent still have reached this reward if it had taken another action?'. We show that measuring contributions w.r.t. rewarding states, as is done in HCA, results in spurious estimates of contributions, causing HCA to degrade towards the high-variance REINFORCE estimator in many relevant environments. Instead, we measure contributions w.r.t. rewards or learned representations of the rewarding objects, resulting in gradient estimates with lower variance. We run experiments on a suite of problems specifically designed to evaluate long-term credit assignment capabilities. By using dynamic programming, we measure ground-truth policy gradients and show that the improved performance of our new model-based credit assignment methods is due to lower bias and variance compared to HCA and common baselines. Our results demonstrate how modeling action contributions towards rewarding outcomes can be leveraged for credit assignment, opening a new path towards sample-efficient reinforcement learning.2

关键词：

FRAPPÉ: A Group Fairness Framework for Post-Processing Everything

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ţifrea, Alexandru Lahoti, Preethi Packer, Ben Halpern, Yoni Beirami, Ahmad Prost, Flavien Department of Computer Science ETH Zurich Switzerland Google DeepMind United Kingdom

Despite achieving promising fairness-error tradeoffs, in-processing mitigation techniques for group fairness cannot be employed in numerous practical applications with limited computation resources or no access to the training pipeline of the prediction model. In these situations, post-processing is a viable alternative. However, current methods are tailored to specific problem settings and fairness definitions and hence, are not as broadly applicable as in-processing. In this work, we propose a framework that turns any regularized in-processing method into a post-processing approach. This procedure prescribes a way to obtain post-processing techniques for a much broader range of problem settings than the prior post-processing literature. We show theoretically and through extensive experiments that our framework preserves the good fairness-error trade-offs achieved with in-processing and can improve over the effectiveness of prior post-processing methods. Finally, we demonstrate several advantages of a modular mitigation strategy that disentangles the training of the prediction model from the fairness mitigation, including better performance on tasks with partial group labels.1 Copyright © 2023, The Authors. All rights reserved.

关键词： Economic and social effects

Adapting to Evolving Adversaries with Regularized Continual Robust Training

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Dai, Sihui Cianfarani, Christian Bhagoji, Arjun Nitin Sehwag, Vikash Mittal, Prateek Department of Electrical and Computer Engineering Princeton University United States Department of Computer Science University of Chicago United States Google Deepmind United Kingdom

Robust training methods typically defend against specific attack types, such as p attacks with fixed budgets, and rarely account for the fact that defenders may encounter new attacks over time. A natural solution is to adapt the defended model to new adversaries as they arise via fine-tuning, a method which we call continual robust training (CRT). However, when implemented naively, fine-tuning on new attacks degrades robustness on previous attacks. This raises the question: how can we improve the initial training and fine-tuning of the model to simultaneously achieve robustness against previous and new attacks? We present theoretical results which show that the gap in a model’s robustness against different attacks is bounded by how far each attack perturbs a sample in the model’s logit space, suggesting that regularizing with respect to this logit space distance can help maintain robustness against previous attacks. Extensive experiments on 3 datasets (CIFAR-10, CIFAR-100, and ImageNette) and over 100 attack combinations demonstrate that the proposed regularization improves robust accuracy with little overhead in training time. Our findings and open-source code1 lay the groundwork for the deployment of models robust to evolving attacks. Copyright © 2025, The Authors. All rights reserved.

关键词： Budget control

SocialJax: An Evaluation Suite for Multi-agent Reinforcement Learning in Sequential Social Dilemmas

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Guo, Zihao Willis, Richard Shi, Shuqing Tomilin, Tristan Leibo, Joel Z. Du, Yali Department of Informatics King’s College London United Kingdom Mathematics and Computer Science Eindhoven University of Technology Netherlands Google DeepMind United Kingdom

Social dilemmas pose a significant challenge in the field of multi-agent reinforcement learning (MARL). Melting Pot is an extensive framework designed to evaluate social dilemma environments, providing an evaluation protocol that measures generalization to new social partners across various test scenarios. However, running reinforcement learning algorithms in the official Melting Pot environments demands substantial computational resources. In this paper, we introduce SocialJax, a suite of sequential social dilemma environments implemented in JAX. JAX is a high-performance numerical computing library for Python that enables significant improvements in the operational efficiency of SocialJax on GPUs and TPUs. Our experiments demonstrate that the training pipeline of SocialJax achieves a 50× speedup in real-time performance compared to Melting Pot’s RLlib baselines. Additionally, we validate the effectiveness of baseline algorithms within the SocialJax environments. Finally, we use Schelling diagrams to verify the social dilemma properties of these environments, ensuring they accurately capture the dynamics of social dilemmas. Our project is available at https://***/view/socialjax. Copyright © 2025, The Authors. All rights reserved.

关键词： Reinforcement learning

Feature Aggregation with Latent Generative Replay for Federated Continual Learning of Socially Appropriate Robot Behaviours

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Churamani, Nikhil Checker, Saksham Dogan, Fethiye Irmak Chiang, Hao-Tien Lewis Gunes, Hatice Department of Computer Science and Technology University of Cambridge United Kingdom King’s College London United Kingdom Google DeepMind United Kingdom

It is critical for robots to explore Federated Learning (FL) settings where several robots, deployed in parallel, can learn independently while also sharing their learning with each other. This collaborative learning in real-world environments requires social robots to adapt dynamically to changing and unpredictable situations and varying task settings. Our work contributes to addressing these challenges by exploring a simulated living room environment where robots need to learn the social appropriateness of their actions. First, we propose Federated Root (FedRoot) averaging, a novel weight aggregation strategy which disentangles feature learning across clients from individual task-based learning. Second, to adapt to challenging environments, we extend FedRoot to Federated Latent Generative Replay (FedLGR), a novel Federated Continual Learning (FCL) strategy that uses FedRoot-based weight aggregation and embeds each client with a generator model for pseudo-rehearsal of learnt feature embeddings to mitigate forgetting in a resource-efficient manner. Our results show that FedRoot-based methods offer competitive performance while also resulting in a sizeable reduction in resource consumption (up to 86% for CPU usage and up to 72% for GPU usage). Additionally, our results demonstrate that FedRoot-based FCL methods outperform other methods while also offering an efficient solution (up to 84% CPU and 92% GPU usage reduction), with FedLGR providing the best results across evaluations. Copyright © 2024, The Authors. All rights reserved.

关键词： Federated learning

AugInsert: Learning Robust Visual-Force Policies via Data Augmentation for Object Assembly Tasks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Diaz, Ryan Imdieke, Adam Veeriah, Vivek Desingh, Karthik Department of Computer Science and Engineering The University of Minnesota Twin Cities United States Google DeepMind United Kingdom

This paper primarily focuses on learning robust visual-force policies in the context of high-precision object assembly tasks. Specifically, we focus on the contact phase of the assembly task where both objects (peg and hole) have made contact and the objective lies in maneuvering the objects to complete the assembly. Moreover, we aim to learn contact-rich manipulation policies with multisensory inputs on limited expert data by expanding human demonstrations via online data augmentation. We develop a simulation environment with a dual-arm robot manipulator to evaluate the effect of augmented expert demonstration data. Our focus is on evaluating the robustness of our model with respect to certain task variations: grasp pose, peg/hole shape, object body shape, scene appearance, camera pose, and force-torque/proprioception noise. We show that our proposed data augmentation method helps in learning a multisensory manipulation policy that is robust to unseen instances of these variations, particularly physical variations such as grasp pose. Additionally, our ablative studies show the significant contribution of force-torque data to the robustness of our model. For additional experiments and qualitative results, we refer to the project webpage https://***/47skWXH. © 2024, CC BY.

关键词： Manipulators