检索结果-内蒙古大学图书馆

Nonlinear Value Function Approximation Method with Easy Hyperparameter Tuning and Convergence Guarantee

IEEE Access 2025年 13卷 30117-30126页

作者： Watabe, Yuuya Shibuya, Takeshi University of Tsukuba Master's Program in Intelligent and Mechanical Interaction Systems Ibaraki305-8577 Japan University of Tsukuba Institute of Systems and Information Engineering Ibaraki305-8577 Japan

Reinforcement Learning is a branch of machine learning to learn control strategies that achieve a given objective through trial-And-error in the environment. Because this can be applicable even when the state transition function of the control object is unknown or it is difficult to create its model properly, this can reduce the designer's burden. Reinforcement learning repeats evaluating a policy and improving it based on that evaluation. An algorithm called value function approximation is used in this evaluation. Value function approximation is a general term for algorithms of approximating a function called value function, which maps an arbitrary state to how much reward can be obtained in the future when acting according to a policy from that state. Value function approximation should have high approximation accuracy and hyperparameter easy to tune. Although various value function approximation methods have been proposed so far, some of them does not have convergent guarantee and the others' hyperparameter tuning is difficult. The purpose of this paper is to propose a value function approximation method that have convergence guarantee even with nonlinear function approximator and whose hyperparameter tuning is easy and to verify its performance through numerical experiments. To this end, we focused on GTD2 method, which is one of the value function approximation methods that has convergence guarantee even with a nonlinear function approximator. GTD2 method has the drawback that its hyperparameter is difficult to tune appropriately so that the approximation does not diverge during learning. Therefore, we firstly clarified the cause of the divergence of GTD2 method. Secondly, based on this cause, we proposed Normalized and Regularized GTD2 (NRGTD2) method, that incorporates a method for suppressing divergence into GTD2 method. Finally, through numerical experiments, it was clarified that the proposed method suppresses the divergence and can achieve better approxim

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Texture development and surface reconstruction of BiVO_(4)photoanode via one-pot hydrothermal reaction for enhanced photoelectrochemical water splitting

引用

Journal of Advanced Ceramics 2025年第3期14卷 133-141页

作者： Sung Won Hwang Yoo Jae Jeong Runfa Tan Indhujasri Saravanan Hyun Soo Han Dong Hoe Kim In Sun Cho Department of Energy Systems Research Ajou UniversitySuwon 16499Republic of Korea Department of Material Science&Engineering Ajou UniversitySuwon 16499Republic of Korea Department of Mechanical Engineering Stanford UniversityStanford 94305USA Department of Material Science&Engineering Korea UniversitySeoul 02841Republic of Korea Materials Science and Engineering Program University of Colorado BoulderBoulder 80303USA

The simultaneous optimization of the bulk and surface characteristics of photoelectrodes is essential to maximize their photoelectrochemical(PEC)*** report a novel one-pot hydrothermal synthesis of textured and surface-reconstructed BiVO_(4)photoanodes(ts-BVO),achieving significant improvements in PEC water *** controlling precursor molarity and ethylene glycol(EG)addition,we developed a stepwise dual reaction(SDR)mechanism,which enables simultaneous bulk texture development and surface *** optimized CoBi/ts-BVO photoanode exhibited a photocurrent density of 4.3 mA∙cm^(−2)at 1.23 V *** hydrogen electrode(RHE)with a high Faradaic efficiency of 98%under one sun *** with nontextured BiVO_(4),the charge transport efficiency increased from 8%to 70%,whereas the surface charge transfer efficiency improved from 9%to 85%.These results underscore the critical role of both bulk and surface engineering in enhancing PEC *** findings offer a streamlined approach for improving the intrinsic properties of photoanodes in solar water splitting.

关键词： BiVO_(4) hydrothermal texture surface reconstruction ethylene glycol(EG) photoelectrochemical water splitting

来源：评论

学校读者我要写书评

暂无评论

Sustainability of Manufacturing Enterprises: A Psychological Perspective of Linking Manufacturing Layers

Sustainability of Manufacturing Enterprises: A Psychological...

引用

IISE Annual Conference and Expo 2024

作者： Garbie, Ibrahim Garbie, Muhammad Department of Mechanical Engineering Industrial Engineering Program College of Engineering Helwan University Cairo Egypt Department of Industrial Systems and Manufacturing Engineering Texas Tech University LubbockTX United States

ISBN: (纸本)9781713877851

As sustainability/sustainable development (S/SD) is still considered the buzzword in manufacturing environments, particularly after showing up and joining Industry 4.0 (I4.0), S/SD is prescribed to be embraced as one of the major perspectives of actualizing I4.0. Manufacturing, in common, is separated into three primary streams or layers. These layers are represented by machine design (MD), manufacturing processes (MP), and manufacturing systems (MS). All these layers constitute manufacturing enterprises. Although these layers coincide or overlap at one point, there's a psychological link between them. The primary layer is called the core and is dependable for designing machine components. The second layer is called the chosen layer, and it is represented by the manufacturing process with which generation operations will be adopted. The final layer could be a given one and is utilized to demonstrate which manufacturing system can be chosen. This paper aims to discover the S/SD for these layers and the psychological interface between them. In this paper, components of these three layers will be distinguished and analyzed, and the S/SD index of linking these layers through a psychological perspective will be proposed and outlined independently and universally. © IISE Annual Conference and Expo *** rights reserved.

关键词： Smart manufacturing

来源：评论

学校读者我要写书评

暂无评论

Designing Manufacturing Enterprises System for Industry 4.0

Designing Manufacturing Enterprises System for Industry 4.0

引用

IISE Annual Conference and Expo 2024

ISBN: (纸本)9781713877851

Currently, researchers are concentrating their attention on design for manufacturing, design for assembly, design for cost and design for quality, design for complexity, design for reconfiguration, design for sustainability, and design for X…etc., but they do not mention design manufacturing enterprises system for Industry 4.0. The design manufacturing enterprises system for Industry 4.0 is a systemic approach that simultaneously identifies elements and components for designing manufacturing enterprises for Industry 4.0, taking into consideration the new urgency of adopting Industry 4.0 and the challenges and risk management for implementing Industry 4.0. In this paper, the designing analysis and level will be presented by identifying the relative importance of urgent adoption and the challenges of implementing Industry 4.0. © IISE Annual Conference and Expo *** rights reserved.

关键词： Smart manufacturing

来源：评论

学校读者我要写书评

暂无评论

Sub-optimal Perceptron Controller Using WOA Algorithm for Farming Watering Mobile Robot

Sub-optimal Perceptron Controller Using WOA Algorithm for Fa...

引用

2024 IEEE Latin American Conference on Computational Intelligence, LA-CCI 2024

作者： Pastrana, M.A. Oliveira, L.H. Bautista, J. Pinto-Lopera, Jesus Mendoza-Penaloza, Jose Munoz, Daniel M. University of the Amazonia Faculty of Engineering Systems Undergraduate Program Florencia Caquetá Colombia University of Brasilia Mechatronics Graduate Program Department of Mechanical Engineering Brasilia Brazil University of Brasilia Faculty of Gama Electronics Engineering Undergraduate Program Brasilia Brazil

ISBN: (纸本)9798350374575

Artificial neural networks (ANNs) are intricate mathematical models, drawing inspiration from the biological nervous system, and offering intelligence alongside nonparametric capabilities. The efficacy of ANNs heavily relies on their learning process. Heuristic search algorithms have been proposed in literature as reliable options for optimizing neural networks utilizing the Perceptron architecture. This paper presents an suboptimal neural network Perceptron training method employing the Whale Optimization Algorithm for the application of farming watering mobile robots. To achieve this, a fitness function based on system responses such as overshoot, settling Time, and steady-state Error was developed. The sub-optimal Perceptron controller was then implemented using MATLAB, CoppeliaSim software and the EVA robot, resulting in a 66.17% overshoot, a 4.7% steady-state error, and a settling time of 3.05 seconds. Moreover, the sub-optimal Perceptron controller was mapped onto a System on Chip FPGA AMD-Xilinx Zynq 7020, resulting in a hardware resources consumption of 1,870 (3.52%) Lookup table, 2 (0.9%) Digital Signal Processing blocks, 1,019 (0.96%) Flip-Flops and a power consumtion of 0.005 W. Finally, Hardware-in-The-Loop validation was performed using CoppeliaSim, Python, Universal Direct Memory Access, and the EVA robot. The results showed an overshoot of 94.83%, a settling time of 2 seconds, and a steady-state error of 4.5%. © 2024 IEEE.

关键词： MATLAB

来源：评论

学校读者我要写书评

暂无评论

Comparative analysis of bioinspired algorithms for tuning a flatness-based controller: A ball and beam case study

Comparative analysis of bioinspired algorithms for tuning a ...

引用

2024 IEEE Latin American Conference on Computational Intelligence, LA-CCI 2024

作者： Pastrana, M.A. Filho, Jose Oniram De A. Limaverde Oliveira, Luiz Henrique Bautista, Jaider Fortaleza, Eugenio Munoz, Daniel M. Department of Mechanical Engineering Brasília Brazil University of Brasília Department of Electrical Engineering Brasília Brazil University of the Amazonia Faculty of Engineering Systems Undergraduate Program Florencia Colombia

ISBN: (纸本)9798350374575

This paper provides a comprehensive comparative analysis of the performance of four bioinspired algorithms (Differential Evolution, Grasshopper Optimization Algorithm, Moth-flame Optimization, and Particle Swarm Optimization) in designing a flatness-based controller for a ball and beam system simulated in MATLAB/Simulink environment. The assessment involves the tuning of feedback gains utilizing four distinct fitness functions: Integral Square Error, Integral Absolute Error, Integral Time Absolute Error, and an ad-hoc cost function. Hypothesis tests are employed to choose the optimal controllers from each combination for every bioinspired algorithm and a fitness function. Subsequently, a control performance analysis of these controllers for the ball and beam system is conducted, focusing on critical parameters such as overshoot, settling time, and steady-state error. This analysis demonstrates that all the controllers achieve a zero steady-state error. Noteworthy is the flatness-based controller tuned with the bioinspired Grasshopper Optimization Algorithm and utilizing the Integral Time Absolute Error fitness function, which stands out for achieving the fastest response with the lowest overshoot. This particular controller emerges as the most recommended choice for the specified application. © 2024 IEEE.

关键词： Particle swarm optimization (PSO)

来源：评论

学校读者我要写书评

暂无评论

Action Robust Reinforcement Learning with Highly Expressive Policy

Action Robust Reinforcement Learning with Highly Expressive ...

引用

2024 IEEE International Conference on systems, Man, and Cybernetics, SMC 2024

作者： Kim, SeongIn Shibuya, Takeshi University of Tsukuba Doctor's Program in Intelligent and Mechanical Interaction Systems Ibaraki Japan University of Tsukuba Faculty of Engineering Information and Systems Ibaraki Japan

ISBN: (纸本)9781665410205

In traditional reinforcement learning, there can be a degradation in the control performance of the policy when the environmental parameters differ between the training and application phase. The policy that minimizes this degradation is referred to as a robust policy. A framework called Noisy action Robust Markov Decision Process (NR-MDP) was proposed for training robust policies, and the Action Robust Deep Determin-istic Policy Gradient (AR-DDPG) algorithm was introduced as a method for solving NR-MDP. The optimal policy in NR-MDP includes policies following various probability distributions, whereas AR-DDPG is restricted to deterministic policies. We propose a new robust reinforcement learning method called Action Robust Q-Learning (AR-QL) that enables the training of optimal policies in NR-MDP by leveraging various sampling techniques to extend the representational capacity of policies, targeting an improvement in policy robustness. To validate this, we confirmed that AR-QL can acquire the optimal policy for a simple NR-MDP problem, for which AR-DDPG fails to obtain the optimal policy. Furthermore, we confirmed that the robust performance of policy trained by AR-QL in the OpenAI's InvertedPendulum environment surpasses that of policy trained by AR-DDPG. © 2024 IEEE.

关键词： Deep reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Explainable Reinforcement Learning via Causal Model Considering Agent's Intention

Explainable Reinforcement Learning via Causal Model Consider...

引用

2024 IEEE International Conference on systems, Man, and Cybernetics, SMC 2024

ISBN: (纸本)9781665410205

Explaining agent's decision can offer valuable insights for designers and end-users. One proposed method for explaining an agent's decision-making involves representing a causal relation between state components and action as a causal model and providing explanations for the decisions made using causal model. However, traditional causal model often faces structural limitation, restricting the range of representable control problems. Additionally, providing accurate explanation becomes challenging in control problems with various types of rewards because agent's intention of an action is unknown. In this study, we introduce a causal model capable of representing a broader range of control problems and a method to provide accurate explanations in control problems with various types of reward structures. Through redefining the relationships between nodes in the causal model, we have enabled a broader representation of control problems. Also, by incorporating agent's intention into the explanation, we have achieved to provide a more precise explanation. To validate the effectiveness of our proposed method, we conducted experiments using OpenAI's LunarLander environment. Using a proposed causal model, we defined the causal model of LunarLander, which could not be represented by conventional causal models. Furthermore, by incorporating the intentions of an agent into the explanation, novel interpretations previously inaccessible have become feasible. © 2024 IEEE.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

0906 | Experimental Investigation of Increased Bearing Capacity When Imparting Vibration to Loose Ground in Low Atmospheric Pressure 21

0906 | Experimental Investigation of Increased Bearing Capac...

引用

21st International and 12th Asia-Pacific Regional Conference of the International Society for Terrain-Vehicle systems, ISTVS 2024

作者： Watanabe, Tomohiro Iizuka, Kojiro Department of Engineering Mechanical Engineering Program Niigata University Niigata Prefecture Niigata City Japan Department of Science and Engineering Machinery and Control Systems Shibaura Institute of Technology Saitama Prefecture Saitama City Japan

ISBN: (纸本)9781942112570

Recently, legged robots have attracted considerable attention as highly mobile rovers for planetary exploration. However, the surfaces of celestial bodies such as Mars and the Moon are primarily loose, causing slippage due to surface deformation from the rover's leg movements. To mitigate this, a walking method aimed at preventing slippage was proposed. In this walking method, the bearing capacity is increased by imparting vibration from the rover's leg to the loose soil on the ground. This walking method improved the mobility performance of the legged rover. It is crucial to investigate the increase in bearing capacity by imparting vibration in a realistic space environment to validate the effectiveness of the proposed method. This study examines the change in bearing capacity due to vibration under low atmospheric pressure, a condition found on Mars and the Moon. This condition affects the characteristics of ground sand due to decreased air resistance. Our findings provide valuable insights into using vibration in planetary exploration to enhance the performance of legged rovers. The experimental results indicate that the bearing capacity under low atmospheric pressure is nearly identical to that under standard atmospheric pressure, suggesting that atmospheric pressure conditions have minimal impact on the bearing capacity when vibration is applied. © 2024 International Society for Terrain-Vehicle systems. All rights reserved.

关键词： Atmospheric pressure

来源：评论

学校读者我要写书评

暂无评论

Application of Linear Electromagnetic Actuators in Vehicular Suspensions 12

Application of Linear Electromagnetic Actuators in Vehicular...

引用

12th SAE BRASIL Colloquium on Suspensions and Road Implements and engineering Exhibition, BRASILCOLL 2023

作者： Eckert, Paulo Roberto Boff, Ben Hur Bandeira Follmann, Lucas Da Silva Oliveira, Eduardo Filho, Aly Ferreira Flores Flores, Jeferson Vieira Perondi, Eduardo Andre Lenz, Augusto Schmidt Poli, Mariana Damm Marcolin, Vinicius Campagnaro Struck, Rodrigo Graduate Program in Electrical Engineering Federal Universi Russia Graduate Program in Mechanical Engineering Federal Universi Russia Suspensys Automotive Systems RS Caxias do Sul Brazil

This study comprehensively describes the application of linear electromagnetic actuators in automotive suspension systems, focusing on the electromagnetic force necessary in suspension systems operating in passive, semi-active, and active modes and their capability for energy regeneration. The use of electromagnetic actuators as an alternative to traditional shock absorbers makes easy the energy harvesting that is typically lost during vertical vehicle displacement. An initial sizing technique for dimensioning electromagnetic actuators is applied to estimate the requirements for heavy vehicles. Experimental tests are conducted to validate the proposed method and assess the performance of the electromagnetic actuator in the suspension system for passive, semi-active, and active operations. A suitable extrapolation of the validated data is performed to determine the feasibility of the application of the proposed solution for heavy vehicles and identify potential applications and markets based on the road conditions of the ISO 8608 standard. This study offers a comprehensive approach to determining the actuator force for the suspension system and/or studying its potential for energy regeneration. As a result, it becomes possible to analyze the viability of the actuator replacement and provides important insights for further research and development. © 2023 SAE International. All rights reserved.

关键词： Shock absorbers

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：