检索结果-内蒙古大学图书馆

Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Tiantian Wang, Xueqian Liang, Bin Yuan, Bo The Intelligent Computing Lab Shenzhen International Graduate School Tsinghua University Shenzhen518055 China The Center for Artificial Intelligence and Robotics Shenzhen International Graduate School Tsinghua University Shenzhen518055 China The Research Center for Navigation and Control Department of Automation Tsinghua University Beijing100084 China

The powerful learning ability of deep neural networks enables reinforcement learning agents to learn competent control policies directly from continuous environments. In theory, to achieve stable performance, neural networks assume i.i.d. inputs, which unfortunately does no hold in the general reinforcement learning paradigm where the training data is temporally correlated and non-stationary. This issue may lead to the phenomenon of "catastrophic interference" and the collapse in performance. In this paper, we present IQ, i.e., interference-aware deep Q-learning, to mitigate catastrophic interference in single-task deep reinforcement learning. Specifically, we resort to online clustering to achieve on-the-fly context division, together with a multi-head network and a knowledge distillation regularization term for preserving the policy of learned contexts. Built upon deep Q networks, IQ consistently boosts the stability and performance when compared to existing methods, verified with extensive experiments on classic control and Atari tasks. The code is publicly available at: https://***/Sweety-dm/Interference-aware-Deep-Q-learning. © 2021, CC BY-NC-SA.

关键词： Reinforcement learning

On Embeddings and Inverse Embeddings of Input Design for Regularized System Identification

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Mu, Biqiang Chen, Tianshi Kong, He Jiang, Bo Wang, Lei Wu, Junfeng Key Laboratory of Systems and Control Institute of Systems Science Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China School of Data Science Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen518172 China Shenzhen Key Laboratory of Biomimetic Robotics and Intelligent Systems Department of Mechanical and Energy Engineering Southern University of Science and Technology Shenzhen518055 China Guangdong Provincial Key Laboratory of Human-Augmentation and Rehabilitation Robotics in Universities Southern University of Science and Technology Shenzhen518055 China Key Laboratory for NSLSCS of Jiangsu Province School of Mathematical Sciences Nanjing Normal University Nanjing China College of Control Science and Engineering Zhejiang University Hangzhou China

Input design is an important problem for system identification and has been well studied for the classical system identification, i.e., the maximum likelihood/prediction error method. For the emerging regularized system identification, the study on input design has just started, and it is often formulated as a non-convex optimization problem that minimizes a scalar measure of the Bayesian mean squared error matrix subject to certain constraints, and the state-of-art method is the so-called quadratic mapping and inverse embedding (QMIE) method, where a time domain inverse embedding (TDIE) is proposed to find the inverse of the quadratic mapping. In this paper, we report some new results on the embeddings/inverse embeddings of the QMIE method. Firstly, we present a general result on the frequency domain inverse embedding (FDIE) that is to find the inverse of the quadratic mapping described by the discrete-time Fourier transform. Then we show the relation between the TDIE and the FDIE from a graph signal processing perspective. Finally, motivated by this perspective, we further propose a graph induced embedding and its inverse, which include the previously introduced embeddings as special cases. This deepens the understanding of input design from a new viewpoint beyond the real domain and the frequency domain viewpoints. Copyright © 2022, The Authors. All rights reserved.

关键词： Embeddings

Urban Digital Twins for intelligent Road Inspection

学校读者我要写书评

暂无评论

Urban Digital Twins for Intelligent Road Inspection

IEEE International Conference on Big Data

作者： Rui Fan Yikang Zhang Sicen Guo Jiahang Li Yi Feng Shuai Su Yanting Zhang Wenshuo Wang Yu Jiang Mohammud Junaid Bocus Xingyi Zhu Qijun Chen the Department of Control Science & Engineering Frontiers Science Center for Intelligent Autonomous Systems and State Key Laboratory of Intelligent Autonomous Systems Robotics & Artificial Intelligence Laboratory (RAIL) Tongji University Shanghai P. R. China Department of Computer Science & Technology Donghua University Shanghai P. R. China Department of Civil Engineering McGill University Montreal Canada CTO office at ClearMotion Billerica MA USA Department of Electrical & Electronic Engineering the University of Bristol BS8 1UB U.K Key Laboratory of Road and Traffic Engineering of Ministry of Education Tongji University Shanghai P. R. China

ISBN: (纸本)9781665480468

Urban digital twin (UDT) technologies offer new opportunities for intelligent road inspection (IRI). This paper first reviews the state-of-the-art algorithms used in the two key components of UDT-based IRI systems: (1) multi-temporal, multi-dimension, multi-score, and heterogeneous road data acquisition, and (2) road distress detection. This paper then summarizes the UDTIRI competition, organized in conjunction with IEEE Bigdata 2022. More details on our competition are available at ***/view/udtiri-workshop/bigdata-2022.

关键词： Roads Data acquisition Inspection Big Data Rough surfaces Digital twins

Advancing Embodied Intelligence in Robotic-Assisted Endovascular Procedures: A Systematic Review of AI Solutions

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yao, Tianliang Lu, Bo Kowarschik, Markus Yuan, Yixuan Zhao, Hubin Ourselin, Sebastien Althoefer, Kaspar Ge, Junbo Qi, Peng Department of Control Science and Engineering College of Electronic and Information Engineering Shanghai Institute of Intelligent Science and Technology Tongji University Shanghai200092 China Department of Electronic Engineering Faculty of Engineering The Chinese University of Hong Kong 999077 Hong Kong Robotics and Microsystems Center School of Mechanical and Electrical Engineering Soochow University Suzhou215131 China Bavaria Forchheim91301 Germany CREATe Division of Surgery & Interventional Science UCL LondonHA7 4LP United Kingdom School of Biomedical Engineering & Imaging Sciences King’s College London LondonSE1 7EH United Kingdom Centre for Advanced Robotics @ Queen Mary School of Engineering and Materials Science Queen Mary University of London LondonE1 4NS United Kingdom Department of Cardiology Zhongshan Hospital Fudan University Shanghai Institute of Cardiovascular Diseases Shanghai200032 China State Key Laboratory of Cardiovascular Diseases Medical Innovation Center Shanghai East Hospital School of Medicine Tongji University Shanghai200092 China

Endovascular procedures have revolutionized the treatment of vascular diseases thanks to minimally invasive solutions that significantly reduce patient recovery time and enhance clinical outcomes. However, the precision and dexterity required during these procedures poses considerable challenges for interventionists. Robotic systems have emerged offering transformative solutions, addressing issues such as operator fatigue, radiation exposure, and the inherent limitations of human precision. The integration of Embodied Intelligence (EI) into these systems signifies a paradigm shift, enabling robots to navigate complex vascular networks and adapt to dynamic physiological conditions. Data-driven approaches, advanced computer vision, medical image analysis, and machine learning techniques, are at the forefront of this evolution. These methods augment procedural intelligence by facilitating real-time vessel segmentation, device tracking, and anatomical landmark detection. Reinforcement learning and imitation learning further refine navigation strategies and replicate experts’ techniques. This review systematically examines the integration of EI principles into robotic technologies, in relation to endovascular procedures. We discuss recent advancements in intelligent perception and data-driven control, and their practical applications in robot-assisted endovascular procedures. By critically evaluating current limitations and emerging opportunities, this review establishes a framework for future developments, emphasizing the potential for greater autonomy and improved clinical outcomes. Emerging trends and specific areas of research, such as federated learning for medical data sharing, explainable AI for clinical decision support, and advanced human-robot collaboration paradigms, are also explored, offering insights into the future direction of this rapidly evolving field. © 2025, CC BY.

关键词： Medical robotics

Catastrophic interference in reinforcement learning: A solution based on context division and knowledge distillation

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2021年

作者： Zhang, Tiantian Wang, Xueqian Liang, Bin Yuan, Bo The Intelligent Computing Lab Division of Informatics Shenzhen International Graduate School Tsinghua University Shenzhen518055 China The Center for Artificial Intelligence and Robotics Shenzhen International Graduate School Tsinghua University Shenzhen518055 China The Research Center for Navigation and Control Department of Automation Tsinghua University Beijing100084 China

The powerful learning ability of deep neural networks enables reinforcement learning (RL) agents to learn competent control policies directly from high-dimensional and continuous environments. In theory, to achieve stable performance, neural networks assume i.i.d. inputs, which unfortunately does no hold in the general RL paradigm where the training data is temporally correlated and non-stationary. This issue may lead to the phenomenon of "catastrophic interference" (a.k.a. "catastrophic forgetting") and the collapse in performance as later training is likely to overwrite and interfer with previously learned good policies. In this paper, we introduce the concept of "context" into the single-task RL and develop a novel scheme, termed as Context Division and Knowledge Distillation (CDaKD) driven RL, to divide all states experienced during training into a series of contexts. Its motivation is to mitigate the challenge of aforementioned catastrophic interference in deep RL, thereby improving the stability and plasticity of RL models. At the heart of CDaKD is a value function, parameterized by a neural network feature extractor shared across all contexts, and a set of output heads, each specializing on an individual context. In CDaKD, we exploit online clustering to achieve context division, and interference is further alleviated by a knowledge distillation regularization term on the output layers for learned contexts. In addition, to effectively obtain the context division in high-dimensional state spaces (e.g., image inputs), we perform clustering in the lower-dimensional representation space of a randomly initialized convolutional encoder, which is fixed throughout training. Our results show that, with various replay memory capacities, CDaKD can consistently improve the performance of existing RL algorithms on classic OpenAI Gym tasks and the more complex high-dimensional Atari tasks, incurring only moderate computational overhead. © 2021, CC BY.

关键词： Distillation

Distributed linear equations over random networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yi, Peng Lei, Jinlong Chen, Jie Hong, Yiguang Shi, Guodong Department of Control Science and Engineering Tongji University Shanghai201804 China Institute of Advanced Study Tongji University Shanghai201804 China Australian Centre for Field Robotics Sydney Institute for Robotics and Intelligent Systems University of Sydney NSW2006 Australia

Distributed linear algebraic equation over networks, where nodes hold a part of problem data and cooperatively solve the equation via node-to-node communications, is a basic distributed computation task receiving an increasing research attention. Communications over a network have a stochastic nature, with both temporal and spatial dependence due to link failures, packet dropouts or node recreation, etc. In this paper, we study the convergence and convergence rate of distributed linear equation protocols over a ∗-mixing random network, where the temporal and spatial dependencies between the node-to-node communications are allowed. When the network linear equation admits exact solutions, we prove the mean-squared exponential convergence rate of the distributed projection consensus algorithm, while the lower and upper bound estimations of the convergence rate are also given for independent and identically distributed (i.i.d.) random graphs. Motivated by the randomized Kaczmarz algorithm, we also propose a distributed randomized projection consensus algorithm, where each node randomly selects one row of local linear equations for projection per iteration, and establish an exponential convergence rate. When the network linear equation admits no exact solution, we prove that a distributed gradient-descent-like algorithm with diminishing step-sizes can drive all nodes' states to a least-squares solution at a sublinear rate. These results collectively illustrate that distributed computations may overcome communication correlations if the prototype algorithms enjoy certain contractive properties or are designed with suitable parameters. Copyright © 2020, The Authors. All rights reserved.

关键词： Graph theory

An efficient and adaptable multimedia system for converting PAL to VGA in real-time video processing

学校读者我要写书评

暂无评论

An efficient and adaptable multimedia system for converting ...

作者： Jain, Deepak Kumar Jacob, Sunil Alzubi, Jafar Menon, Varun Key Laboratory of Intelligent Air-Ground Cooperative Control for Universities in Chongqing College of Automation Chongqing University of Posts and Telecommunications Chongqing400065 China Center for Robotics SCMS School of Engineering and Technology Kerala683576 India School of Engineering Al-Balqa Applied University Salt19117 Jordan Department of Computer Science and Engineering SCMS School of Engineering and Technology Kerala683576 India

Real-time video processing has found its range of applications from defense to consumer electronics for surveillance, video conferencing, etc. With the advent of Field Programmable Gate Arrays (FPGAs), flexible real-time video processing systems which can meet hard real-time constraints are easily realized with short development time. Most of the existing solutions have high utilization of system resources and are not quite flexible with many applications. Here we propose a hardware–software co-design for an FPGA-based real-time video processing system to convert video in standard Phase Alternating Line (PAL) 576i format to standard video of Video Graphics Array (VGA)/Super Video Graphics Array (SVGA) format with little utilization of resources. Switching between multiple video streams, character/text overlaying, and skin color detection are also incorporated with the system. The system is also adaptable for rugged applications. VHSIC Hardware Description Language (VHDL) codes for the architecture were synthesized using Altera Quartus II and targeted for Altera Stratix I FPGA. Results achieved confirm that the proposed system performs efficient conversion with very less resource utilization compared to the existing solutions. Since the proposed system is also flexible, many other applications can be incorporated in the future. © 2019, Springer-Verlag GmbH Germany, part of Springer Nature.

关键词： Field programmable gate arrays (FPGA)

Retinal Disorder Diagnosis Based on Hybrid Deep Learning Models

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Sedik, Ahmed El-Hag, Noha A. El-Hoseny, Heba M. El-Banby, Ghada M. Khalaf, Ashraf A.M. Abd El-Samie, Fathi E. El-Shafai, Walid Department of the Robotics and Intelligent Machines Faculty of Artificial Intelligence Kafrelsheikh University Egypt Menoufia University Egypt Department of Electronics and Electrical Communication Engineering Al-Obour High Institute for Engineering and Technology Egypt Department of Industrial Electronics and Control Engineering Faculty of Electronic Engineering Menoufia University Menouf32952 Egypt Deptartment of Electronics and Electrical Communication Engineering Faculty of Electronic Engineering Menoufia University Menouf32952 Egypt Department of Information Technology College of Computer and Information Sciences Princess Nourah Bint Abdulrahman University Riyadh84428 Saudi Arabia Security Engineering Lab Computer Science Department Prince Sultan University Riyadh11586 Saudi Arabia

Anomaly detection from medical images is badly needed for automated diagnosis. For example, medical images obtainedwith several modalities such as magnetic resonance (MR) and confocal microscopy need to be classified for anomaly *** MR images may contain tumors that need to be detected with anomaly detection techniques. In MR imaging, braintumors are represented as areas composed of abnormal cells that need to be detected. In addition, confocal microscopy can be used for retinal imaging. Hence, there is a need to apply classification techniques for diagnosing diseases of the eye such as Diabetic Retinopathy (DR), which leads to exudates in different regions of the retina. The detection of exudates is necessary for the diagnosis of DR. In this paper, two models are presented for the classification of MR and retinal images. Much research has been done in this area, but it did not achieve high accuracy of classification in an acceptable time. In addition, the previous models did not consider both types of image modalities considered in this paper. The suggested models are Convolutional Neural Network (CNN) and Convolutional Long Short-Term Memory (Conv-LSTM). They are used to classify © 2022, The Authors. All rights reserved.

关键词： Ophthalmology

The Method of Neural Network control over the Process of Manufacturing Foil Solar Panels 2

学校读者我要写书评

暂无评论

The Method of Neural Network Control over the Process of Man...

2nd IEEE International Conference on System Analysis and intelligent Computing, SAIC 2020

作者： Sachenko, Anatoliy Ivakhiv, Orest Vyshnia, Volodymyr Grzeszczyk, Konrad Osolinskyi, Oleksandr Novosad, Stanislav Kochan, Volodymyr Nakonechnyi, Markiyan Kochan, Orest Kopania, Lukasz Kazimierz Pulaski University of Technology and Humanities in Radom Department of Informatics Radom Poland Lviv Polytechnic National University Department of Intelligent Mechatronics and Robotics Lviv Ukraine Dnipropetrovsk State University of Internal Affairs Department of Economic and Information Security Dnipro Ukraine International Vision Machinery VISORT Sp. J Radom Poland Ternopil National Economic University Department for Information Computer Systems and Control Ternopil Ukraine Ternopil National Economic University Research Institute for Intelligent Computer Systems Ternopil Ukraine Lviv Polytechnic National University Department of Computerized Automation Systems Lviv Ukraine Lviv Polytechnic National University Department of Measuring Information Technologies Lviv Ukraine

ISBN: (纸本)9781728190822

Authors have developed a method of control over one of the main technological processes of manufacturing foil solar panels, i.e., burning tracks in the layers of semiconductor material and insulation. The proposed method is based on neural networks trained directly on the control object. This eliminates the laborious process of identifying the parameters of the control object mathematical model and the impact of its error on the training outcomes. To clarify the conditions for optimizing a certain law for generating the control action by neural networks various laws of control have been simulated. It's established that laws of control can be changed according to the current situation of the control object. Simulation outcomes proofed that the most efficient is the system with the PI- controller. © 2020 IEEE.

关键词： Process control