检索结果-内蒙古大学图书馆

Nonlinear Value Function Approximation Method with Easy Hyperparameter Tuning and Convergence Guarantee

IEEE Access 2025年 13卷 30117-30126页

作者： Watabe, Yuuya Shibuya, Takeshi University of Tsukuba Master's Program in Intelligent and Mechanical Interaction Systems Ibaraki305-8577 Japan University of Tsukuba Institute of Systems and Information Engineering Ibaraki305-8577 Japan

Reinforcement Learning is a branch of machine learning to learn control strategies that achieve a given objective through trial-And-error in the environment. Because this can be applicable even when the state transition function of the control object is unknown or it is difficult to create its model properly, this can reduce the designer's burden. Reinforcement learning repeats evaluating a policy and improving it based on that evaluation. An algorithm called value function approximation is used in this evaluation. Value function approximation is a general term for algorithms of approximating a function called value function, which maps an arbitrary state to how much reward can be obtained in the future when acting according to a policy from that state. Value function approximation should have high approximation accuracy and hyperparameter easy to tune. Although various value function approximation methods have been proposed so far, some of them does not have convergent guarantee and the others' hyperparameter tuning is difficult. The purpose of this paper is to propose a value function approximation method that have convergence guarantee even with nonlinear function approximator and whose hyperparameter tuning is easy and to verify its performance through numerical experiments. To this end, we focused on GTD2 method, which is one of the value function approximation methods that has convergence guarantee even with a nonlinear function approximator. GTD2 method has the drawback that its hyperparameter is difficult to tune appropriately so that the approximation does not diverge during learning. Therefore, we firstly clarified the cause of the divergence of GTD2 method. Secondly, based on this cause, we proposed Normalized and Regularized GTD2 (NRGTD2) method, that incorporates a method for suppressing divergence into GTD2 method. Finally, through numerical experiments, it was clarified that the proposed method suppresses the divergence and can achieve better approxim

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

A Dialogue Support System for Visualizing Latent and Dynamic Perspectives as Conceptual Structures for Comparative Shopping 24

A Dialogue Support System for Visualizing Latent and Dynamic...

引用

Proceedings of the 2024 8th International Conference on Natural Language Processing and Information Retrieval

作者： Kazuki Aikawa Hajime Nobuhara Master's and Doctoral Programs in Intelligent and Mechanical Interaction Systems University of Tsukuba Tsukuba Japan Institute of Systems and Information Engineering University of Tsukuba Tsukuba Japan

来源：评论

学校读者我要写书评

暂无评论

Evaluation of Cuffless Blood Pressure Estimation Accuracy at Each Heartbeat During Exercise 6th

Evaluation of Cuffless Blood Pressure Estimation Accuracy a...

引用

6th International Conference on Biomedical and Health Informatics, ICBHI 2024

作者： Ono, Kouki Maeda, Yuka Yoshimura, Takumi Huang, Ming Tamura, Toshiyo Master’s Program in Intelligent and Mechanical Interaction Systems University of Tsukuba Tsukuba305-8577 Japan Institute of Systems and Information Engineering University of Tsukuba Tsukuba305-8577 Japan Medical and Welfare Engineering Tokyo Metropolitan College of Industrial Technology Tokyo116-8523 Japan School of Data Science Nagoya City University Nagoya467-8501 Japan Future Robotics Organization Waseda University Tokyo169-8050 Japan

ISBN: (纸本)9783031863226

Continuous blood pressure monitoring is crucial because of its dynamic nature, influenced by factors such as physical activity and mental stress. Although cuffless blood pressure estimation is a promising technique for continuous monitoring, its accuracy during drastic fluctuations, especially during exercise, remains insufficiently validated. In this study, we evaluated a simple Pulse Arrival Time based blood pressure estimation method and several improved methods for sharp fluctuations induced by exercise. Sixteen subjects in the experiment, with data from 13 valid for analysis. The proposed method achieved an average correlation of 0.527 in systolic blood pressure, suggesting that cuffless blood pressure estimation could be applied to estimate continuous blood pressure variability during exercise. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Blood pressure

来源：评论

学校读者我要写书评

暂无评论

Variable-speed teaching–playback as real-world data augmentation for imitation learning

引用

Advanced Robotics 2025年

作者： Masuya, Nozomu Sato, Hiroshi Yamane, Koki Kusume, Takuya Sakaino, Sho Tsuji, Toshiaki Doctoral Program in Intelligent and Mechanical Interaction Systems Degree Programs in Systems and Information and Engineering Graduate School of Science and Technology University of Tsukuba Tsukuba Japan Master’s Program in Intelligent and Mechanical Interaction Systems Degree Programs in Systems and Information and Engineering Graduate School of Science and Technology University of Tsukuba Tsukuba Japan Department of Intelligent Interaction Technologies Institute of Systems and Information Engineering University of Tsukuba Tsukuba Japan Graduate School of Science and Engineering Saitama University Saitama Japan

Imitation learning’s inevitable reliance on human demonstrations in hard-to-simulate settings has resulted in a shortage of training data, even with a simple change in speed. Although the field of data augmentation has addressed the lack of data, conventional methods of data augmentation for robot manipulation are limited to simulation-based methods or downsampling for position control. This paper proposes a novel method of data augmentation that is applicable to force control and preserves the advantages of real-world datasets. We applied teaching–playback at variable speeds as real-world data augmentation to increase both the quantity and quality of environmental reactions at variable speeds. An experiment was conducted on bilateral control-based imitation learning using a method of imitation learning equipped with position–force control. We evaluated the effect of real-world data augmentation on two tasks, pick-and-place and wiping, at variable speeds, each from two human demonstrations at fixed speed. The results showed that generation of variable-speed movement from human demonstrations with less than 10% variety of durations is feasible, and improved success rate and accuracy along the duration/frequency command by gathering environmental reactions at variable speeds, rather than simply altering speeds of gathered data. © 2025 Informa UK Limited, trading as Taylor & Francis Group and The Robotics Society of Japan.

关键词： Demonstrations

来源：评论

学校读者我要写书评

暂无评论

Reinforcement Learning to Efficiently Recover Control Performance of Robots Using Imitation Learning After Failure

Reinforcement Learning to Efficiently Recover Control Perfor...

引用

2022 IEEE International Conference on systems, Man, and Cybernetics, SMC 2022

作者： Kobayashi, Shoki Shibuya, Takeshi University of Tsukuba Master's Program in Intelligent and Mechanical Interaction Systems Ibaraki Japan University of Tsukuba Faculty of Engineering Information and Systems Ibaraki Japan

ISBN: (数字)9781665452588

ISBN: (纸本)9781665452588

Extreme environments, such as space and underwater, are difficult for humans to enter because they involve risks, hence it is necessary to employ autonomous robots instead of humans. When robots fail in extreme environments, it is essential for the robot to automatically recover control following the rules of failure because humans cannot repair the robot directly. Reinforcement learning is expected to automatically acquire the control rule;however, the retrieval of the control rule requires significant trial-and-error. Imitation learning cannot acquire the control rule if no suitable expert data exist. Methods combining imitation learning and reinforcement learning reduce the number of trial-and-errors;however, they are still not effective against robot failure because these methods cannot utilize expert data effectively. This paper proposes a reinforcement learning method that efficiently recovers control performance from failures by utilizing both the control rules prepared by the designer and multiple discriminators to calculate measures similar to expert data. Experimental results show that the proposed method recovers the control performance with fewer episodes than the conventional method. The main contribution of the proposed method is its efficiency against robot failure through utilizing expart data prepared for failure by the designers for imitation learning. © 2022 IEEE.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Action Robust Reinforcement Learning with Highly Expressive Policy

Action Robust Reinforcement Learning with Highly Expressive ...

引用

2024 IEEE International Conference on systems, Man, and Cybernetics, SMC 2024

作者： Kim, SeongIn Shibuya, Takeshi University of Tsukuba Doctor's Program in Intelligent and Mechanical Interaction Systems Ibaraki Japan University of Tsukuba Faculty of Engineering Information and Systems Ibaraki Japan

ISBN: (纸本)9781665410205

In traditional reinforcement learning, there can be a degradation in the control performance of the policy when the environmental parameters differ between the training and application phase. The policy that minimizes this degradation is referred to as a robust policy. A framework called Noisy action Robust Markov Decision Process (NR-MDP) was proposed for training robust policies, and the Action Robust Deep Determin-istic Policy Gradient (AR-DDPG) algorithm was introduced as a method for solving NR-MDP. The optimal policy in NR-MDP includes policies following various probability distributions, whereas AR-DDPG is restricted to deterministic policies. We propose a new robust reinforcement learning method called Action Robust Q-Learning (AR-QL) that enables the training of optimal policies in NR-MDP by leveraging various sampling techniques to extend the representational capacity of policies, targeting an improvement in policy robustness. To validate this, we confirmed that AR-QL can acquire the optimal policy for a simple NR-MDP problem, for which AR-DDPG fails to obtain the optimal policy. Furthermore, we confirmed that the robust performance of policy trained by AR-QL in the OpenAI's InvertedPendulum environment surpasses that of policy trained by AR-DDPG. © 2024 IEEE.

关键词： Deep reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Underwater visible light communication using phase-shift keying and rolling-shutter effect 9

Underwater visible light communication using phase-shift key...

引用

9th IEEE Global Conference on Consumer Electronics, GCCE 2020

作者： Hamagami, Ritsuki Ebihara, Tadashi Wakatsuki, Naoto Mizutani, Koichi University of Tsukuba Master's Program in Intelligent and Mechanical Interaction Systems Tsukuba Japan University of Tsukuba Faculty of Engineering Information and Systems Tsukuba Japan

ISBN: (纸本)9781728198026

We propose an underwater visible light communication (VLC) system for use in drones. The transmitter modulates a message using phase-shift keying (PSK) and emits the signal as LED fast-blinking light. The receiver captures this pattern with a CMOS image sensor using a rolling-shutter effect, and demodulates PSK. In existing VLC systems, rolling-shutter effect is used to transmit baseband signals;this system transmits a passband signal tolerant of low-frequency optical interference noise. System performance was evaluated in test-tank experiments. Findings suggested that the VLC system with PSK outperformed that using on-off keying when noise frequency did not exceed carrier frequency. © 2020 IEEE.

关键词： Phase shift keying

来源：评论

学校读者我要写书评

暂无评论

Centralized Training with Decentralized Execution Reinforcement Learning for Cooperative Multi-agent systems with Communication Delay 61

Centralized Training with Decentralized Execution Reinforcem...

引用

61st Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2022

作者： Ikeda, Takuma Shibuya, Takeshi University of Tsukuba Master's Program in Intelligent and Mechanical Interaction Systems Ibaraki Japan University of Tsukuba Faculty of Engineering Information and Systems Ibaraki Japan

ISBN: (数字)9784907764784

ISBN: (纸本)9784907764784

In cooperative multi-agent systems, efficient coordination among agents is important when accomplishing tasks. VFFAC is a method that learns the communication system between agents and their interactions with the environment to obtain policies with high performance. However, this method results in decreased performance of policy in environments with a delay in communication. Furthermore, there is no formulation of the control problem of a cooperative multi-agent system with communication delays in unknown environments. In this study, we formulated a decision-making problem in a cooperative multi-agent system with an unknown environment model and a certain length delay in communication. We also propose a method to handle communication delays by using the history of information obtained through communication. We demonstrated that the proposed method successfully learns policy with high rewards through simulated experiments in an environment with a communication delay. © 2022 The Society of Instrument and Control Engineers - SICE.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Rolling-Shutter Sensor-Based Visible Light Communication with Cross-Screen Filter: Communication and Positioning System Using a Commercial Camera 10

Rolling-Shutter Sensor-Based Visible Light Communication wit...

引用

10th IEEE Global Conference on Consumer Electronics, GCCE 2021

作者： Hamagami, Ritsuki Ebihara, Tadashi Wakatsuki, Naoto Maeda, Yuka Mizutani, Koichi University of Tsukuba Master's Program In Intelligent And Mechanical Interaction Systems Tsukuba Japan Faculty of Engineering Information And Systems University of Tsukuba Tsukuba Japan

ISBN: (纸本)9781665436762

We propose a visible light communication system using a commercial camera that achieves communication and positioning simultaneously, which is essential for intelligent transport systems (ITS). The transmitter (Tx) emits the data as fast-blinking light of the LED. The receiver (Rx) captures the signal using a rolling-shutter sensor with a cross-screen filter. Different from the previous system without a cross-screen filter, which can only achieve short-range and single-user communication, the proposed system can achieve long-range communication by diffusing the incoming light in a specific direction using a cross-screen filter. Furthermore, the proposed system has the potential to achieve multi-user communication and positioning, which used to require a high-speed camera, using a commercial camera. We evaluated the performance of the proposed system in indoor experiments. The experimental results suggest that the proposed system can improve communication range compared to the previous system. © 2021 IEEE.

关键词： Visible light communication

来源：评论

学校读者我要写书评

暂无评论

Explainable Reinforcement Learning via Causal Model Considering Agent's Intention

Explainable Reinforcement Learning via Causal Model Consider...

引用

2024 IEEE International Conference on systems, Man, and Cybernetics, SMC 2024

ISBN: (纸本)9781665410205

Explaining agent's decision can offer valuable insights for designers and end-users. One proposed method for explaining an agent's decision-making involves representing a causal relation between state components and action as a causal model and providing explanations for the decisions made using causal model. However, traditional causal model often faces structural limitation, restricting the range of representable control problems. Additionally, providing accurate explanation becomes challenging in control problems with various types of rewards because agent's intention of an action is unknown. In this study, we introduce a causal model capable of representing a broader range of control problems and a method to provide accurate explanations in control problems with various types of reward structures. Through redefining the relationships between nodes in the causal model, we have enabled a broader representation of control problems. Also, by incorporating agent's intention into the explanation, we have achieved to provide a more precise explanation. To validate the effectiveness of our proposed method, we conducted experiments using OpenAI's LunarLander environment. Using a proposed causal model, we defined the causal model of LunarLander, which could not be represented by conventional causal models. Furthermore, by incorporating the intentions of an agent into the explanation, novel interpretations previously inaccessible have become feasible. © 2024 IEEE.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：