检索结果-内蒙古大学图书馆

StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS

学校读者我要写书评

暂无评论

StereoPose: Category-Level 6D Transparent Object Pose Estima...

IEEE International Conference on robotics and Automation (ICRA)

作者： Kai Chen Stephen James Congying Sui Yun-Hui Liu Pieter Abbeel Qi Dou The Chinese University of Hong Kong University of California Berkeley Dyson Robot Learning Lab. Hong Kong Centre for Logistics Robotics

Most existing methods for category-level pose estimation rely on object point clouds. However, when considering transparent objects, depth cameras are usually not able to capture high-quality data, resulting in point clouds with severe artifacts. Without a complete point cloud, existing methods are not applicable to challenging transparent objects. To tackle this problem, we present StereoPose, a novel stereo image based framework for category-level object pose estimation, ideally suited for transparent objects. For a robust estimation from pure stereo images, we develop a pipeline that decouples category-level pose estimation into object size estimation, initial pose estimation, and pose refinement. StereoPose then estimates object pose based on representation in the normalized object coordinate space (NOCS). To address the issue of image content aliasing, we further define a back-view NOCS map for the transparent object. The back-view NOCS aims to reduce the network learning ambiguity caused by content aliasing, and leverage informative cues on the back of the transparent object for more accurate pose estimation. To further improve the performance of the stereo framework, StereoPose is equipped with a parallax attention module for stereo feature fusion and an epipolar loss for improving the stereo-view consistency of network predictions. Extensive experiments on the public TOD dataset demonstrate the superiority of the proposed StereoPose framework for category-level 6D transparent object pose estimation. Code and demos will be available on the project homepage: ***/~kaichen/***.

关键词：

Auto-λ: Disentangling Dynamic Task Relationships

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Shikun James, Stephen Davison, Andrew J. Johns, Edward Dyson Robotics Lab Imperial College London United Kingdom Robot Learning Lab Imperial College London United Kingdom UC Berkeley United States

Understanding the structure of multiple related tasks allows for multi-task learning to improve the generalisation ability of one or all of them. However, it usually requires training each pairwise combination of tasks together in order to capture task relationships, at an extremely high computational cost. In this work, we learn task relationships via an automated weighting framework, named Auto-λ. Unlike previous methods where task relationships are assumed to be fixed, Auto-λ is a gradient-based meta learning framework which explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss;where the validation loss automatically influences task weightings throughout training. We apply the proposed framework to both multi-task and auxiliary learning problems in computer vision and robotics, and show that Auto-λ achieves state-of-the-art performance, even when compared to optimisation strategies designed specifically for each problem and data domain. Finally, we observe that Auto-λ can discover interesting learning behaviors, leading to new insights in multi-task learning. Code is available at https://***/lorenmt/auto-lambda. Copyright © 2022, The Authors. All rights reserved.

关键词： learning systems

learning MULTI-AGENT COMMUNICATION WITH CONTRASTIVE learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lo, Yat Long Sengupta, Biswa Foerster, Jakob Noukhovitch, Michael Dyson Robot Learning Lab. United Kingdom JPMorgan Chase FLAIR University of Oxford United Kingdom Mila Université de Montréal Canada

Communication is a powerful tool for coordination in multi-agent RL. But inducing an effective, common language is a difficult challenge, particularly in the decentralized setting. In this work, we introduce an alternative perspective where communicative messages sent between agents are considered as different incomplete views of the environment state. By examining the relationship between messages sent and received, we propose to learn to communicate using contrastive learning to maximize the mutual information between messages of a given trajectory. In communication-essential environments, our method outperforms previous work in both performance and learning speed. Using qualitative metrics and representation probing, we show that our method induces more symmetric communication and captures global state information from the environment. Overall, we show the power of contrastive learning and the importance of leveraging messages as encodings for effective communication. © 2023, CC BY.

关键词： Multi agent systems

CHEAP TALK DISCOVERY AND UTILIZATION IN MULTI-AGENT REINFORCEMENT learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lo, Yat Long de Witt, Christian Schroeder Sokota, Samuel Foerster, Jakob Whiteson, Shimon University of Oxford Dyson Robot Learning Lab United Kingdom FLAIR University of Oxford United Kingdom Carnegie Mellon University United States University of Oxford United Kingdom

By enabling agents to communicate, recent cooperative multi-agent reinforcement learning (MARL) methods have demonstrated better task performance and more coordinated behavior. Most existing approaches facilitate inter-agent communication by allowing agents to send messages to each other through free communication channels, i.e., cheap talk channels. Current methods require these channels to be constantly accessible and known to the agents a priori. In this work, we lift these requirements such that the agents must discover the cheap talk channels and learn how to use them. Hence, the problem has two main parts: cheap talk discovery (CTD) and cheap talk utilization (CTU). We introduce a novel conceptual framework for both parts and develop a new algorithm based on mutual information maximization that outperforms existing algorithms in CTD/CTU settings. We also release a novel benchmark suite to stimulate future research in CTD/CTU. © 2023, CC BY.

关键词： Reinforcement learning

StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Chen, Kai James, Stephen Sui, Congying Liu, Yun-Hui Abbeel, Pieter Dou, Qi The Chinese University of Hong Kong Hong Kong University of California Berkeley United States Dyson Robot Learning Lab Hong Kong Centre for Logistics Robotics Hong Kong

Most existing methods for category-level pose estimation rely on object point clouds. However, when considering transparent objects, depth cameras are usually not able to capture meaningful data, resulting in point clouds with severe artifacts. Without a high-quality point cloud, existing methods are not applicable to challenging transparent objects. To tackle this problem, we present StereoPose, a novel stereo image framework for category-level object pose estimation, ideally suited for transparent objects. For a robust estimation from pure stereo images, we develop a pipeline that decouples category-level pose estimation into object size estimation, initial pose estimation, and pose refinement. StereoPose then estimates object pose based on representation in the normalized object coordinate space (NOCS). To address the issue of image content aliasing, we further define a back-view NOCS map for the transparent object. The back-view NOCS aims to reduce the network learning ambiguity caused by content aliasing, and leverage informative cues on the back of the transparent object for more accurate pose estimation. To further improve the performance of the stereo framework, StereoPose is equipped with a parallax attention module for stereo feature fusion and an epipolar loss for improving the stereo-view consistency of network predictions. Extensive experiments on the public TOD dataset demonstrate the superiority of the proposed StereoPose framework for category-level 6D transparent object pose estimation. Project homepage: ***/-kaichen/***. Copyright © 2022, The Authors. All rights reserved.

关键词： Stereo image processing

Guide Your Agent with Adaptive Multimodal Rewards

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Kim, Changyeon Seo, Younggyo Liu, Hao Lee, Lisa Shin, Jinwoo Lee, Honglak Lee, Kimin KAIST Korea Republic of Dyson Robot Learning Lab United Kingdom UC Berkeley United States Google DeepMind United Kingdom University of Michigan United States LG AI Research

Developing an agent capable of adapting to unseen environments remains a difficult challenge in imitation learning. This work presents Adaptive Return-conditioned Policy (ARP), an efficient framework designed to enhance the agent’s generalization ability using natural language task descriptions and pre-trained multimodal encoders. Our key idea is to calculate a similarity between visual observations and natural language instructions in the pre-trained multimodal embedding space (such as CLIP) and use it as a reward signal. We then train a return-conditioned policy using expert demonstrations labeled with multimodal rewards. Because the multimodal rewards provide adaptive signals at each timestep, our ARP effectively mitigates the goal misgeneralization. This results in superior generalization performances even when faced with unseen text instructions, compared to existing text-conditioned policies. To improve the quality of rewards, we also introduce a fine-tuning method for pre-trained multimodal encoders, further enhancing the performance. Video demonstrations and source code are available on the project website: https://***/view/2023arp. Copyright © 2023, The Authors. All rights reserved.

关键词： Visual languages

Speed Co-Augmentation for Unsupervised Audio-Visual Pre-training

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Jiangliu Jiao, Jianbo Song, Yibing James, Stephen Tong, Zhan Ge, Chongjian Abbeel, Pieter Liu, Yun-Hui The Chinese University of Hong Kong Hong Kong University of Birmingham United Kingdom Fudan University China Dyson Robot Learning Lab United Kingdom Tencent AI Lab China The University of Hong Kong Hong Kong UC Berkeley United States

This work aims to improve unsupervised audio-visual pre-training. Inspired by the efficacy of data augmentation in visual contrastive learning, we propose a novel speed co-augmentation method that randomly changes the playback speeds of both audio and video data. Despite its simplicity, the speed co-augmentation method possesses two compelling attributes: (1) it increases the diversity of audio-visual pairs and doubles the size of negative pairs, resulting in a significant enhancement in the learned representations, and (2) it changes the strict correlation between audio-visual pairs but introduces a partial relationship between the augmented pairs, which is modeled by our proposed SoftInfoNCE loss1 to further boost the performance. Experimental results show that the proposed method significantly improves the learned representations when compared to vanilla audio-visual contrastive learning. © 2023, CC BY.

关键词：

Improving International Climate Policy via Mutually Conditional Binding Commitments (Track 3)

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Heitzig, Jobst Oechssler, Jörg Pröschel, Christoph Ragavan, Niranjana Long Lo, Richie Yat FutureLab on Game Theory and Networks of Interacting Agents Complexity Science Department Potsdam Institute for Climate Impact Research Germany Alfred Weber Institute for Economics University of Heidelberg Germany Technical University of Berlin Germany Dyson Robot Learning Lab United Kingdom

This paper proposes enhancements to the RICE-N simulation and multi-agent reinforcement learning framework to improve the realism of international climate policy negotiations. Acknowledging the framework’s value, we highlight the necessity of significant enhancements to address the diverse array of factors in modeling climate negotiations. Building upon our previous work on the "Conditional Commitments Mechanism" (CCF mechanism) we discuss ways to bridge the gap between simulation and reality. We suggest the inclusion of a recommender or planner agent to enhance coordination, address the Real2Sim gap by incorporating social factors and non-party stakeholder sub-agents, and propose enhancements to the underlying Reinforcement learning solution algorithm. These proposed improvements aim to advance the evaluation and formulation of negotiation protocols for more effective international climate policy decision-making in Rice-N. However, further experimentation and testing are required to determine the implications and effectiveness of these suggestions. © 2023, CC BY.

关键词： Decision making

Improving International Climate Policy via Mutually Conditional Binding Commitments

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Heitzig, Jobst Oechssler, Jörg Pröschel, Christoph Ragavan, Niranjana Lo, Yat Long FutureLab on Game Theory and Networks of Interacting Agents Complexity Science Department Potsdam Institute for Climate Impact Research Germany Alfred Weber Institute for Economics University of Heidelberg Germany Technical University of Berlin Germany Dyson Robot Learning Lab United Kingdom

The Paris Agreement, considered a significant milestone in climate negotiations, has faced challenges in effectively addressing climate change due to the unconditional nature of most Nationally Determined Contributions (NDCs). This has resulted in a prevalence of free-riding behavior among major polluters and a lack of concrete conditionality in NDCs. To address this issue, we propose the implementation of a decentralized, bottom-up approach called the Conditional Commitment Mechanism. This mechanism, inspired by the National Popular Vote Interstate Compact, offers flexibility and incentives for early adopters, aiming to formalize conditional cooperation in international climate policy. In this paper, we provide an overview of the mechanism, its performance in the AI4ClimateCooperation challenge, and discuss potential real-world implementation aspects. Prior knowledge of the climate mitigation collective action problem, basic economic principles, and game theory concepts are assumed. © 2023, CC BY.

关键词： Game theory