检索结果-内蒙古大学图书馆

Effective Exploration via Tsallis Actor-Critic on 6D Robot Grasping

学校读者我要写书评

暂无评论

Effective Exploration via Tsallis Actor-Critic on 6D Robot G...

International Conference on Control, Automation and Systems ( ICCAS)

作者： Jaeyeon Jeong Jonghyun Park Songhwai Oh Department of Electrical and Computer Engineering Seoul National University Seoul Korea ASRI Seoul National University Seoul Korea Department of Naval Architecture and Ocean Engineering Seoul National University Seoul Korea

ISBN: (数字)9788993215380

ISBN: (纸本)9798331517939

Grasping arbitrary objects is challenging due to the need to understand unseen object shapes, generate plausible grasp poses, and plan trajectories simultaneously. In this paper, we propose Goal-Auxiliary Tsallis Actor-Critic (GA-TAC) method, which enhances grasping policy learning by integrating goal prediction auxiliary tasks with Tsallis Actor-Critic [1] methods. Building on the Goal-Auxiliary Deep Deterministic Policy Gradient (GA-DDPG) [2], GA-TAC addresses the limitations of suboptimal grasp policies by dynamically adjusting the exploration-exploitation trade-off using the Tsallis Actor-Critic method. This hybrid approach leverages the strengths of both learning paradigms and adaptive exploration, resulting in more robust and optimal grasping policies. We demonstrate that GA-TAC significantly increases the success rate of grasping tasks by ensuring effective exploration near critical grasping points in Dex-YCB dataset [3] with different q-indices. We also show that GA-TAC outperforms other baselines in the Handover-Sim [4] environment and our method is robust to unseen environments.

关键词： Automation Shape Buildings Grasping Reinforcement learning Handover Control systems Trajectory Collision avoidance Robots

Sequential preference ranking for efficient reinforcement learning from human feedback 23

学校读者我要写书评

暂无评论

Sequential preference ranking for efficient reinforcement le...

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Minyoung Hwang Gunmin Lee Hogun Kee Chanwoo Kim Kyungjae Lee Songhwai Oh Electrical and Computer Engineering and ASRI Seoul National University Seoul Korea Department of Artificial Intelligence Chung-Ang University Seoul Korea

Reinforcement learning from human feedback (RLHF) alleviates the problem of designing a task-specific reward function in reinforcement learning by learning it from human preference. However, existing RLHF models are considered inefficient as they produce only a single preference data from each human feedback. To tackle this problem, we propose a novel RLHF framework called SeqRank, that uses sequential preference ranking to enhance the feedback efficiency. Our method samples trajectories in a sequential manner by iteratively selecting a defender from the set of previously chosen trajectories Κ and a challenger from the set of unchosen trajectories u \ Κ, where u is the replay buffer. We propose two trajectory comparison methods with different defender sampling strategies: (1) sequential pairwise comparison that selects the most recent trajectory and (2) root pairwise comparison that selects the most preferred trajectory from Κ. We construct a data structure and rank trajectories by preference to augment additional queries. The proposed method results in at least 39.2% higher average feedback efficiency than the baseline and also achieves a balance between feedback efficiency and data dependency. We examine the convergence of the empirical risk and the generalization bound of the reward model with Rademacher complexity. While both trajectory comparison methods outperform conventional pairwise comparison, root pairwise comparison improves the average reward in locomotion tasks and the average success rate in manipulation tasks by 29.0% and 25.0%, respectively. Project page: https://***/projects/SeqRank

关键词：

Authentication of Multi-agent System with Verifiable Computation and Distributed Aggregation

学校读者我要写书评

暂无评论

Authentication of Multi-agent System with Verifiable Computa...

IEEE Conference on Decision and Control

作者： Seungbeom Lee Dongwoo Kim Heewon Chung Junsoo Kim Hyungbo Shim Department of Electrical and Computer Engineering ASRI Seoul National University Korea Department of AI · SW Convergence Dongguk University Korea DESILO Inc Korea Department of Electrical and Information Engineering Seoul National University of Science and Technology Korea

ISBN: (数字)9798350316339

ISBN: (纸本)9798350316346

We propose an authentication scheme for a multiagent system over integers, based on verifiable computation primitives. The naive approach, employing Freivalds’ algorithm in centralized way, faces several challenges. Specifically, unreliability of the network introduces the risk of information tampering by other agents. To this end, it requires locally updating and merging the proofs of the individual states in a distributed manner. Our proposed method addresses these issues with assuming presence of a leader agent who is responsible for validating the correctness of all the states of the agents. This can be achieved by a distributed protocol that aggregates proofs from the individual agents, relying on the well-known knowledge-of-exponent assumption. By using this distributed protocol, computational complexity and communication cost are reduced compared to centralized verification. Furthermore, we incorporate a clustering technique to distribute storage requirements.

关键词： Protocols Costs Data integrity Aggregates Merging Authentication Clustering algorithms Computational complexity Faces Multi-agent systems

Ring-LWE based Encrypted Controller with Unlimited Number of Recursive Multiplications and Effect of Error Growth

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jang, Yeongjun Lee, Joowon Min, Seonhong Kwak, Hyesun Kim, Junsoo Song, Yongsoo ASRI The Department of Electrical and Computer Engineering Seoul National University Seoul08826 Korea Republic of Department of Computer Science and Engineering Seoul National University Seoul08826 Korea Republic of Department of Electrical and Information Engineering Seoul National University of Science and Technology Seoul01811 Korea Republic of

In this paper, we propose an encrypted dynamic controller that executes an unlimited number of recursive homomorphic multiplications on a Ring Learning With Errors (Ring-LWE) based cryptosystem without bootstrapping. The proposed controller exhibits lower computational complexity compared to existing encrypted controllers implemented on LWE based schemes due to the polynomial structure of Ring-LWE. However, the structural difference introduces additional difficulties in analyzing the effect of error growth;Ring-LWE based schemes inject multiple error coefficients when encrypting a single message, which accumulate under recursive homomorphic multiplications. We show that their effect on the control performance can be arbitrarily bounded by the closed-loop stability, thus recovering the performance of the unencrypted controller. Furthermore, a novel method to "pack’’ a vector into a polynomial is presented, which enhances computational and memory efficiency when applied to the proposed encrypted controller. The effectiveness of the proposed design is demonstrated through numerical simulations. © 2024, CC BY.

关键词： Networked control systems

Encrypted Dynamic Control exploiting Limited Number of Multiplications and a Method using RLWE-based Cryptosystem

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lee, Joowon Lee, Donggil Kim, Junsoo Shim, Hyungbo ASRI Department of Electrical and Computer Engineering Seoul National University Korea Republic of Department of Electrical Engineering Incheon National University Korea Republic of Department of Electrical and Information Engineering Seoul National University of Science and Technology Korea Republic of

In this paper, we present a method to encrypt dynamic controllers that can be implemented through most homomorphic encryption schemes, including somewhat, leveled fully, and fully homomorphic encryption. To this end, we represent the output of the given controller as a linear combination of a fixed number of previous inputs and outputs. As a result, the encrypted controller involves only a limited number of homomorphic multiplications on every encrypted data, assuming that the output is re-encrypted and transmitted back from the actuator. A guidance for parameter choice is also provided, ensuring that the encrypted controller achieves predefined performance for an infinite time horizon. Furthermore, we propose a customization of the method for Ring Learning With Errors (RLWE)-based cryptosystems, where a vector of messages can be encrypted into a single ciphertext and operated simultaneously, thus reducing computation and communication loads. Unlike previous results, the proposed customization does not require extra algorithms such as rotation, other than basic addition and multiplication. Simulation results demonstrate the effectiveness of the proposed method. Copyright © 2023, The Authors. All rights reserved.

关键词： Encryption algorithms

TOWARDS DIVERSE EVALUATION OF CLASS INCREMENTAL LEARNING: A REPRESENTATION LEARNING PERSPECTIVE 3

学校读者我要写书评

暂无评论

TOWARDS DIVERSE EVALUATION OF CLASS INCREMENTAL LEARNING: A ...

3rd Conference on Lifelong Learning Agents, CoLLAs 2024

作者： Cha, Sungmin Kwak, Jihwan Shim, Dongsub Kim, Hyunwoo Lee, Moontae Lee, Honglak Moon, Taesup Computer Science Department Courant Institute of Mathematical Sciences New York University United States Department of Electrical and Computer Engineering Seoul National University Korea Republic of Advanced Machine Learning Lab LG AI Research Korea Republic of Zhejiang Lab China Department of Information and Decision Sciences University of Illinois Chicago United States ASRI INMC IPAI AIIS Seoul National University Korea Republic of

Class incremental learning (CIL) algorithms aim to continually learn new object classes from incrementally arriving data while not forgetting past learned classes. The common evaluation protocol for CIL algorithms is to measure the average test accuracy across all classes learned so far - however, we argue that solely focusing on maximizing the test accuracy may not necessarily lead to developing a CIL algorithm that also continually learns and updates the representations, which may be transferred to the downstream tasks. To that end, we experimentally analyze neural network models trained by CIL algorithms using various evaluation protocols in representation learning and propose new analysis methods. Our experiments show that most state-of-the-art algorithms prioritize high stability and do not significantly change the learned representation, and sometimes even learn a representation of lower quality than a naive baseline. However, we observe that these algorithms can still achieve high test accuracy because they enable a model to learn a classifier that closely resembles an estimated linear classifier trained for linear probing. Furthermore, the base model learned in the first task, which involves single-task learning, exhibits varying levels of representation quality across different algorithms, and this variance impacts the final performance of CIL algorithms. Therefore, we suggest that the representation-level evaluation should be considered as an additional recipe for more diverse evaluation for CIL algorithms. © 2024 CoLLAs. All Rights Reserved.

关键词： Contrastive Learning

Wasserstein Distributionally Robust Control of Partially Observable Linear Systems: Tractable Approximation and Performance Guarantee

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hakobyan, Astghik Yang, Insoon The Department of Electrical and Computer Engineering ASRI Seoul National University Seoul08826 Korea Republic of

Wasserstein distributionally robust control (WDRC) is an effective method for addressing inaccurate distribution information about disturbances in stochastic systems. It provides various salient features, such as an out-of-sample performance guarantee, while most of the existing methods use full-state observations. In this paper, we develop a computationally tractable WDRC method for discrete-time partially observable linear-quadratic (LQ) control problems. The key idea is to reformulate the WDRC problem as a novel minimax control problem with an approximate Wasserstein penalty. We derive a closed-form expression of the optimal control policy of the approximate problem using a nontrivial Riccati equation. We further show the guaranteed cost property of the resulting controller and identify a provable bound for the optimality gap. Finally, we evaluate the performance of our method through numerical experiments using both Gaussian and non-Gaussian disturbances. Copyright © 2022, The Authors. All rights reserved.

关键词： Robust control

User-Friendly Vehicle-to-Grid Optimal Scheduling Problem and Distributed Implementation for Plug-and-Play Operation

学校读者我要写书评

暂无评论

User-Friendly Vehicle-to-Grid Optimal Scheduling Problem and...

International Conference on Control, Automation and Systems ( ICCAS)

作者： Seungbeom Lee Jinwook Heo Soojeong Hyeon Jiyeon Nam Hyungbo Shim Jinsung Kim Department of Electrical and Computer Engineering ASRI Seoul National University Seoul Korea Vehicle SW Division Hyundai Motor Company Hwaseong Korea

In this paper, we propose an optimization method of charging scheduling algorithm based on preference of vehicle owners. To suffice various desires for vehicle charging, we design objective function of optimization problem as weighted sum of some objective function to reflect various desires. To solve the problem without aggregating weight coefficient, distributed optimization based on dynamic average consensus method is implemented. By initialization-free property of dynamic average consensus, plug-and-play operation of electric vehicle is applicable. Finally, we simulate the proposed algorithm to enlighten the meaning of weighted sum for objective function and plug-and-play scenario.

关键词：

Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yu, Sangwon Song, Jongyoon Hwang, Bongkyu Kang, Hoyoung Cho, Sooah Choi, Junhwa Joe, Seongho Lee, Taehee Gwon, Youngjune Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of Samsung SDS Korea Republic of AIIS ASRI INMC ISRC and IPAI Seoul National University Korea Republic of

A binary decision task, like yes-no questions or answer verification, reflects a significant real-world scenario such as where users look for confirmation about the correctness of their decisions on specific issues. In this work, we observe that language models exhibit a negative bias in the binary decisions of complex reasoning tasks. Based on our observations and the rationale about attention-based model dynamics, we propose a negative attention score (NAS) to systematically and quantitatively formulate negative bias. Based on NAS, we identify attention heads that attend to negative tokens provided in the instructions as answer candidate of binary decisions, regardless of the question in the prompt, and validate their association with the negative bias. Additionally, we propose the negative attention score alignment (NASA) method, which is a parameter-efficient fine-tuning technique to address the extracted negatively biased attention heads. Experimental results from various domains of reasoning tasks and large model search space demonstrate that NASA significantly reduces the gap between precision and recall caused by negative bias while preserving their generalization abilities. Copyright © 2024, The Authors. All rights reserved.

关键词： NASA