检索结果-内蒙古大学图书馆

Ensemble successor representations for task generalization in offline-to-online reinforcement learning

Science China(Information Sciences) 2024年第7期67卷 240-255页

作者： Changhong WANG Xudong YU Chenjia BAI Qiaosheng ZHANG Zhen WANG Space Control and Inertial Technology Research Center Harbin Institute of Technology Shanghai Artificial Intelligence Laboratory Shenzhen Research Institute of Northwestern Polytechnical University School of Cybersecurity Northwestern Polytechnical University

In reinforcement learning(RL), training a policy from scratch with online experiences can be inefficient because of the difficulties in exploration. Recently, offline RL provides a promising solution by giving an initialized offline policy, which can be refined through online interactions. However, existing approaches primarily perform offline and online learning in the same task, without considering the task generalization problem in offline-to-online adaptation. In real-world applications, it is common that we only have an offline dataset from a specific task while aiming for fast online-adaptation for several tasks. To address this problem, our work builds upon the investigation of successor representations for task generalization in online RL and extends the framework to incorporate offline-to-online learning. We demonstrate that the conventional paradigm using successor features cannot effectively utilize offline data and improve the performance for the new task by online fine-tuning. To mitigate this, we introduce a novel methodology that leverages offline data to acquire an ensemble of successor representations and subsequently constructs ensemble Q functions. This approach enables robust representation learning from datasets with different coverage and facilitates fast adaption of Q functions towards new tasks during the online fine-tuning *** empirical evaluations provide compelling evidence showcasing the superior performance of our method in generalizing to diverse or even unseen tasks.

关键词： offline reinforcement learning online fine-tuning task generalization successor representations ensembles

来源：评论

学校读者我要写书评

暂无评论

Deep reinforcement learning using least-squares truncated temporal-difference

引用

CAAI Transactions on intelligence technology 2024年第2期9卷 425-439页

作者： Junkai Ren Yixing Lan Xin Xu Yichuan Zhang Qiang Fang Yujun Zeng College of Intelligence Science and Technology National University of Defense TechnologyChangshaChina State Key Laboratory of Astronautic Dynamics Xi'an Satellite Control CenterXi'anChina

Policy evaluation(PE)is a critical sub-problem in reinforcement learning,which estimates the value function for a given policy and can be used for policy ***,there still exist some limitations in current PE methods,such as low sample efficiency and local convergence,especially on complex *** this study,a novel PE algorithm called Least-Squares Truncated Temporal-Difference learning(LST2D)is *** LST2D,an adaptive truncation mechanism is designed,which effectively takes advantage of the fast convergence property of Least-Squares Temporal Difference learning and the asymptotic convergence property of Temporal Difference learning(TD).Then,two feature pre-training methods are utilised to improve the approximation ability of ***,an Actor-Critic algorithm based on LST2D and pre-trained feature representations(ACLPF)is proposed,where LST2D is integrated into the critic network to improve learning-prediction *** simulation studies were conducted on four robotic tasks,and the corresponding results illustrate the effectiveness of *** proposed ACLPF algorithm outperformed DQN,ACER and PPO in terms of sample efficiency and stability,which demonstrated that LST2D can be applied to online learning control problems by incorporating it into the actor-critic architecture.

关键词： Deep reinforcement learning policy evaluation temporal difference value function approximation

来源：评论

学校读者我要写书评

暂无评论

Multi-Vehicle Cooperative Localization Using a TOA-Based Simulated Annealing Extended Kalman Filter in Urban Canyons

引用

IEEE Internet of Things Journal 2025年第13期12卷 22832-22846页

作者： Li, Duhao Deng, Heng Yu, Tianhong Zhang, Liguo Beijing University of Technology School of information Science and Technology Beijing100124 China Ministry of Education Engineering Research Center of Intelligence Perception and Autonomous Control Beijing100124 China

This paper proposes a new multi-vehicle cooperative localization approach that combines Time of Arrival (TOA) with a heuristic Simulated Annealing Extended Kalman Filter (SA-EKF) to enhance positioning accuracy and robustness in urban canyons. The method incorporates a path loss model to account for the complex communication environment between vehicles, using TOA measurements for distributed EKF estimation. The integration of a simulated annealing strategy within the method is instrumental in circumventing local minima, thereby facilitating global optimisation. Furthermore, an adaptive weighted filtering correction mechanism is employed to enhance estimation accuracy and system stability. Experimental results conducted on the SUMO simulation platform and in real-world scenarios demonstrate that the proposed method offers certain advantages over existing approaches in complex, noisy environments. © 2014 IEEE.

关键词： Extended Kalman filters

来源：评论

学校读者我要写书评

暂无评论

Curve-Suppression-Based Event-Triggered Mechanisms for Quasi-Synchronization of Fuzzy Delayed Neural Networks on Time Scales

引用

IEEE Transactions on Systems, Man, and Cybernetics: Systems 2025年第5期55卷 3174-3187页

作者： Wan, Peng Zhou, Yufeng Zeng, Zhigang Lai, Jingang Wuhan University of Science and Technology School of Artificial Intelligence and Automation Engineering Research Center of Metallurgical Automation and Measurement Technology Wuhan430081 China Huazhong University of Science and Technology School of Artificial Intelligence and Automation Key Laboratory of Image Processing and Intelligent control of Education Ministry of China Wuhan430074 China

The vast majority of published event-triggered mechanisms (ETMs) are constructed based on measurement errors, which introduces a problem naturally that they are updated when the measurement errors exceed the thresholds although the current obtained sampling states can make systems converge well. With this problem in mind, we redesign ETMs for quasi-synchronization of T-S fuzzy neural networks (FNNs) with time delays on time scales. First, a novel ETM is designed for continuous-time FNNs with time-varying delays to achieve quasi-synchronization, with which synchronization errors is suppressed to globally exponentially converge to a ball. Second, we introduce the ETM for continuous-time FNNs to discrete-time FNNs, owing to the existence of discrete-time states, the Lypunov function of synchronization errors run over the exponentially decay curve, but it can be suppressed to evolve under another exponentially decay curve. Third, for FNNs on time scales with constant and time-varying delays, we estimate the forward jump operator of the Lyapunov functions and design ETMs to guarantee that the Lypunov functions evolve under the exponentially decay curves, so quasi-synchronization can be achieved. Last but not least, we prove that Zeno behavior will not happen and four numerical examples are introduced to verify the validity and the superiority of the proposed ETMs in reducing information transmission. © 2013 IEEE.

关键词： Continuous time systems

来源：评论

学校读者我要写书评

暂无评论

Small Object Detection in Remote Sensing Based on Feature Fusion and Channel Attention 8

Small Object Detection in Remote Sensing Based on Feature Fu...

引用

8th International Symposium on Computer Science and Intelligent control, ISCSIC 2024

作者： Ge, Yueran Ji, Wenqiang Yin, Shasha Zhang, Wei Hebei University of Technology School of Artificial Intelligence Control Engineering Technology Innovation Center of Hebei Province Tianjin China

ISBN: (纸本)9798350380286

Remote sensing small target detection can accurately detect small-scale targets in remote sensing images. It is valuable for military reconnaissance and civilian image recognition. However, there are some challenges in small target detection in remote sensing images. For instance, small targets have limited feature information, making them difficult to locate. To address these issues, we propose a remote sensing small target algorithm based on feature fusion and channel attention. This algorithm aims to improve small object detection by constructing a strong feature module and fusing the receptive fields of different levels of the shallow feature map. The two shallow feature maps, which are extracted by the backbone network, are processed by the feature enhancement module, resulting in an increase in the extracted small target feature information. The feature fusion module is created using the feature map extracted from the backbone network, and the feature information of small targets is significantly enhanced after passing through the feature fusion module. To capture small target features more accurately, a new channel attention mechanism is designed. The output consists of three feature maps, and small target detection is performed on these three feature maps. Experimental results demonstrate that on the SIMD dataset and the WiderFace dataset, the proposed algorithm achieves a high average accuracy level and exhibits good robustness. © 2024 IEEE.

关键词： Military photography

来源：评论

学校读者我要写书评

暂无评论

FCSA: An ARM+FPGA-based Software and Hardware Collaboration DNN Classifier 8

FCSA: An ARM+FPGA-based Software and Hardware Collaboration ...

引用

8th International Symposium on Computer Science and Intelligent control, ISCSIC 2024

作者： Ye, Jinlin Zhang, Wei Hebei University of Technology School of Artificial Intelligence Control Engineering Technology Innovation Center of Hebei Province Tianjin China

ISBN: (纸本)9798350380286

The parallel architecture of FPGAs is well suited for parallel computation of DNNs. However, softmax-layer is one of the core operators of DNNs, which contains exponentiation and division operations and is not suitable for implementation on FPGAs. In this paper, we design an ARM+FPGA-based software and hardware collaboration DNN classifier (FCSA). The DNN classifier consists of a fully-connected layer and a softmax-layer. FPGA performs the full-connected-layer operations, ARM performs the softmax-layer operations. The FPGA and ARM interact with data through AXI-Stream. Fashion-MNIST dataset is used as test data. The experiments are carried out on ZYNQ7020 and ZU4EV. The result show that ZYNQ7020 processes each image in 206.44us and 1.701 W power consumption and ZU4EV processes each image in 84.3us and 3.385W power consumption. © 2024 IEEE.

关键词： Parallel architectures

来源：评论

学校读者我要写书评

暂无评论

An Improved Observer-based Security Estimation Scheme for Networked control Systems with Deception Attacks 8

An Improved Observer-based Security Estimation Scheme for Ne...

引用

8th International Symposium on Computer Science and Intelligent control, ISCSIC 2024

作者： Gao, Hongjie Li, Hongchao Zhao, Jianchao Liu, Jiao Hebei University of Technology School of Artificial Intelligence Control Engineering Technology Innovation Center of Hebei Province Tianjin China

ISBN: (纸本)9798350380286

A networked control systems (NCSs) with deception attacks is studied in this paper. Considering the energy constraints, the deception attacks are assumed to be bounded. To weaken the negative impact of deception attacks, an estimation method based on an improved observer is propose. In the proposed estimation method, the lagging nature of the observer is utilized to delay the effect of deception attacks on the controller. Based on this scheme, conditions ensuring the stability of the system and guiding the design of the controller gains have been laid out, respectively. Ultimately, the accuracy of the theoretical findings is verified via numerical illustrations. © 2024 IEEE.

关键词： control system stability

来源：评论

学校读者我要写书评

暂无评论

Finite-Time Consensus of Discrete-Time Multi-Agent Systems 8

Finite-Time Consensus of Discrete-Time Multi-Agent Systems

引用

8th International Symposium on Computer Science and Intelligent control, ISCSIC 2024

作者： Liang, Zhihong Ding, Sanbo Hebei University of Technology School of Artificial Intelligence Control Engineering Technology Innovation Center of Hebei Province Tianjin China

ISBN: (纸本)9798350380286

This paper studies the finite-time consensus of discrete-time multi-agent systems (MASs). First, a regionconditional switching controller is developed to realize finitetime consensus problem of the discrete-time system. Under this scheme, the controller is divided into two modes of operation, and whether or not the controller makes switching depends on a predetermined region condition. The study of finite-time consensus can achieve consistency by reducing the consumption of communication and computational resources. Compared with the existing results, the scheme realizes the complete consensus of the discrete-time system. It improves control performance and reduces excessive consumption of control resources. Then, the concept of finite-time consensus is introduced in discrete-time MASs to describe the control performance by the settling time function. The finite-time consensus criterion is obtained by constructing the Lyapunov function, and the settling time is obtained by solving difference fractional and linear matrix inequality. Finally, the validity of the method is verified through numerical simulation. © 2024 IEEE.

关键词： Discrete time control systems

来源：评论

学校读者我要写书评

暂无评论

Adaptive Output Feedback control for 2D TORA System Based on Velocity Observer 8

Adaptive Output Feedback Control for 2D TORA System Based on...

引用

8th International Symposium on Computer Science and Intelligent control, ISCSIC 2024

作者： Yang, Xinran Chen, He Hebei University of Technology School of Artificial Intelligence Control Engineering Technology Innovation Center of Hebei Province Tianjin China

ISBN: (纸本)9798350380286

The typical underactuated system two-dimensional translational oscillator with rotational actuator (2D TORA) that consist of two unactuated translational carts and an actuated rotational eccentric ball which acts as its control input. As an extension of TORA system, the 2D TORA system is more general, meanwhile, the dynamic characteristics of this system makes it challenging to design a proper control scheme. It is very important to realize the accurate positioning of payloads in special application environment. In this paper, an adaptive output feedback controller is proposed to realize all state variables positioning. Specifically, in order to solve the matter of uncertain friction and unknown disturbance, an adaptive method for unknown friction coefficient is designed. Moreover, we come up with a velocity observer to calculate the angular velocity instead of differentiating the output signals. Based on Lyapunov stability theory, the state convergence and stability of the 2D TORA system are analysed. Finally, simulation test results exhibit that the presented control method is effective. © 2024 IEEE.

关键词： Friction

来源：评论

学校读者我要写书评

暂无评论

Integral Sliding-mode Disturbance Observer-based control for T-S Fuzzy Systems 8

Integral Sliding-mode Disturbance Observer-based Control for...

引用

8th International Symposium on Computer Science and Intelligent control, ISCSIC 2024

作者： Zhang, Lei Wang, Likui Hebei University of Technology School of Artificial Intelligence Control Engineering Technology Innovation Center of Hebei Province Tianjin China

ISBN: (纸本)9798350380286

This paper is concerned with stabilization for Takagi-Sugeno (T-S) fuzzy systems with the unmatched disturbance. In this paper, an integral fuzzy switching surface function (IFSSF) containing state-dependent input matrices is used. Based on this, the integral sliding mode control (ISMC) is devised to make sure that the reachability condition is achieved for a finite period of time and the DOB for estimating the disturbance is given. The design of the Lyapunov function utilizes the properties of the membership function with the assumption that its derivatives are bounded to introduce a zero matrix, which reduce the conservatism. Finally, the applicability of the proposed method is verified by a practical example and in simulation we verify that the assumption conditions of the premise is fulfilled. © 2024 IEEE.

关键词： Membership functions

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：