检索结果-内蒙古大学图书馆

IEEE International Conference on Robotics and Automation (ICRA)

作者： Evangelos Theodorou Jonas Buchli Stefan Schaal Computational Learning and Motor Control Laboratory in Computer Science Biomedical Engineering Neuroscience University of Southern California USA

Reinforcement learning (RL) is one of the most general approaches to learning control. Its applicability to complex motor systems, however, has been largely impossible so far due to the computational difficulties that reinforcement learning encounters in high dimensional continuous state-action spaces. In this paper, we derive a novel approach to RL for parameterized control policies based on the framework of stochastic optimal control with path integrals. While solidly grounded in optimal control theory and estimation theory, the update equations for learning are surprisingly simple and have no danger of numerical instabilities as neither matrix inversions nor gradient learning rates are required. Empirical evaluations demonstrate significant performance improvements over gradient-based policy learning and scalability to high-dimensional control problems. Finally, a learning experiment on a robot dog illustrates the functionality of our algorithm in a real-world scenario. We believe that our new algorithm, Policy Improvement with Path Integrals (PI 2 ), offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL in robotics.

关键词： Optimal control Function approximation Stochastic processes Integral equations Scalability Robots control systems learning systems Inference algorithms Stochastic resonance

来源：评论

学校读者我要写书评

暂无评论

Stochastic differential dynamic programming

Stochastic differential dynamic programming

引用

作者： Theodorou, Evangelos Tassa, Yuval Todorov, Emo Departments of Computer Science and Neuroscience Computational Learning and Motor Control Lab. University of Southern California United States Interdisciplinary Center for Neural Computation Hebrew University Jerusalem Israel Department of Computer Science and Engineering University of Washington Seattle WA United States

ISBN: (纸本)9781424474264

Although there has been a significant amount of work in the area of stochastic optimal control theory towards the development of new algorithms, the problem of how to control a stochastic nonlinear system remains an open research topic. Recent iterative linear quadratic optimal control methods iLQG [1], [2] handle control and state multiplicative noise while they are derived based on first order approximation of dynamics. On the other hand, methods such as Differential Dynamic Programming expand the dynamics up to the second order but so far they can handle nonlinear systems with additive noise. In this work we present a generalization of the classic Differential Dynamic Programming algorithm. We assume the existence of state and control multiplicative process noise, and proceed to derive the second-order expansion of the cost-to-go. We find the correction terms that arise from the stochastic assumption. Despite having quartic and cubic terms in the initial expression, we show that these vanish, leaving us with the same quadratic structure as standard DDP. © 2010 AACC.

关键词： Additive noise

来源：评论

学校读者我要写书评

暂无评论

Movement segmentation and recognition for imitation learning 15

Movement segmentation and recognition for imitation learning

引用

15th International Conference on Artificial Intelligence and Statistics, AISTATS 2012

作者： Meier, Franziska Theodorou, Evangelos Schaal, Stefan Computational Learning and Motor Control Lab University of Southern California Los Angeles United States Department of Computer Science and Engineering University of Washington Seattle United States Max-Planck-Institute for Intelligent Systems Tübingen Germany

In human movement learning, it is most common to teach constituent elements of complex movements in isolation, before chaining them into complex movements. Segmentation and recognition of observed movement could thus proceed out of this existing knowledge, which is directly compatible with movement generation. In this paper, we address exactly this scenario. We assume that a library of movement primitives has already been taught, and we wish to identify elements of the library in a complex motor act, where the individual elements have been smoothed together, and, occasionally, there might be a movement segment that is not in our library yet. We employ a flexible machine learning representation of movement primitives based on learnable nonlinear attractor system. For the purpose of movement segmentation and recognition, it is possible to reformulate this representation as a controlled linear dynamical system. An Expectation-Maximization algorithm can be developed to estimate the open parameters of a movement primitive from the library, using as input an observed trajectory piece. If no matching primitive from the library can be found, a new primitive is created. This process allows a straightforward sequential segmentation of observed movement into known and new primitives, which are suitable for robot imitation learning. We illustrate our approach with synthetic examples and data collected from human movement.

关键词： Maximum principle

来源：评论

学校读者我要写书评

暂无评论

Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization

Virtual vs. real: Trading off simulations and physical exper...

引用

2017 IEEE International Conference on Robotics and Automation, ICRA 2017

作者： Marco, Alonso Berkenkamp, Felix Hennig, Philipp Schoellig, Angela P. Krause, Andreas Schaal, Stefan Trimpe, Sebastian Max Planck Institute for Intelligent Systems Tübingen Germany Department of Computer Science ETH Zurich Switzerland Canada Computational Learning and Motor Control Lab University of Southern California United States Max Planck ETH Center for Learning Systems Tubingen Germany Zurich Switzerland

ISBN: (纸本)9781509046331

In practice, the parameters of control policies are often tuned manually. This is time-consuming and frustrating. Reinforcement learning is a promising alternative that aims to automate this process, yet often requires too many experiments to be practical. In this paper, we propose a solution to this problem by exploiting prior knowledge from simulations, which are readily available for most robotic platforms. Specifically, we extend Entropy Search, a Bayesian optimization algorithm that maximizes information gain from each experiment, to the case of multiple information sources. The result is a principled way to automatically combine cheap, but inaccurate information from simulations with expensive and accurate physical experiments in a cost-effective manner. We apply the resulting method to a cart-pole system, which confirms that the algorithm can find good control policies with fewer experiments than standard Bayesian optimization on the physical system only. © 2017 IEEE.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Editorial

引用

International Journal of Humanoid Robotics 2005年第4期2卷 389-390页

作者： CHENG, GORDON SCHAAL, STEFAN ATKESON, CHRISTOPHER G. JST-ICORP Computational Brain Project ATR Computational Neuroscience Laboratory 2-2-2 Keihanna Science City Soraku-gun Kyoto619-0288 Japan Computational Learning and Motor Control Lab University of Southern California Hedco Neurosciences Building HNB-103 3641 Watt Way Los AngelesCA90089-2520 United States Robotics Institute Carnegie Mellon University 5000 Forbes Avenue PittsburghPA15213 United States

来源：评论

学校读者我要写书评

暂无评论

Toward simple control for complex, autonomous robotic applications: Combining discrete and rhythmic motor primitives

Toward simple control for complex, autonomous robotic applic...

引用

作者： Degallier, Sarah Righetti, Ludovic Gay, Sebastien Ijspeert, Auke CNBI Laboratory School of Engineering EPFL Ecole Polytechnique Fédérale de Lausanne Lausanne 1015 Switzerland Biorobotics Laboratory School of Engineering EPFL Ecole Polytechnique Fédérale de Lausanne Lausanne 1015 Switzerland Computational Learning and Motor Control Lab Computer Science Neurosciences and Biomedical Engineering University of Southern California Los Angeles CA 90089 United States

Vertebrates are able to quickly adapt to new environments in a very robust, seemingly effortless way. To explain both this adaptivity and robustness, a very promising perspective in neurosciences is the modular approach to movement generation: Movements results from combinations of a finite set of stable motor primitives organized at the spinal level. In this article we apply this concept of modular generation of movements to the control of robots with a high number of degrees of freedom, an issue that is challenging notably because planning complex, multidimensional trajectories in time-varying environments is a laborious and costly process. We thus propose to decrease the complexity of the planning phase through the use of a combination of discrete and rhythmic motor primitives, leading to the decoupling of the planning phase (i.e. the choice of behavior) and the actual trajectory generation. Such implementation eases the control of, and the switch between, different behaviors by reducing the dimensionality of the high-level commands. Moreover, since the motor primitives are generated by dynamical systems, the trajectories can be smoothly modulated, either by high-level commands to change the current behavior or by sensory feedback information to adapt to environmental constraints. In order to show the generality of our approach, we apply the framework to interactive drumming and infant crawling in a humanoid robot. These experiments illustrate the simplicity of the control architecture in terms of planning, the integration of different types of feedback (vision and contact) and the capacity of autonomously switching between different behaviors (crawling and simple reaching). © 2011 Springer science+Business Media, LLC.

关键词： Anthropomorphic robots

来源：评论

学校读者我要写书评

暂无评论

learning movement primitives

Springer Tracts in Advanced Robotics

引用

Springer Tracts in Advanced Robotics 2005年 15卷 561-572页

作者： Schaal, Stefan Peters, Jan Nakanishi, Jun Ijspeert, Auke Computational Learning and Motor Control Laboratory Computer Science and Neuroscience University of Southern California Los Angeles CA 90089-2520 United States Dept of Humanoid Robotics and Computational Neuroscience ATR Computational Neuroscience Laboratory 2-2-2 Hikaridai Seika-cho Soraku-gun 619-0288 Kyoto Japan School of Computer and Communication Sciences EPFL Swiss Federal Institute of Technology Lausanne CH 1015 Lausanne Switzerland

This paper discusses a comprehensive framework for modular motor control based on a recently developed theory of dynamic movement primitives (DMP). DMPs are a formulation of movement primitives with autonomous nonlinear differential equations, whose time evolution creates smooth kinematic control policies. Model-based control theory is used to convert the outputs of these policies into motor commands. By means of coupling terms, on-line modifications can be incorporated into the time evolution of the differential equations, thus providing a rather flexible and reactive framework for motor planning and execution. The linear parameterization of DMPs lends itself naturally to supervised learning from demonstration. Moreover, the temporal, scale, and translation invariance of the differential equations with respect to these parameters provides a useful means for movement recognition. A novel reinforcement learning technique based on natural stochastic policy gradients allows a general approach of improving DMPs by trial and error learning with respect to almost arbitrary optimization criteria. We demonstrate the different ingredients of the DMP approach in various examples, involving skill learning from demonstration on the humanoid robot DB, and learning biped walking from demonstration in simulation, including self-improvement of the movement patterns towards energy efficiency through resonance tuning. © Springer-Verlag Berlin Heidelberg 2005.

关键词： Differential equations

来源：评论

学校读者我要写书评

暂无评论

Inverse kinematics for humanoid robots

Inverse kinematics for humanoid robots

引用

IEEE International Conference on Robotics and Automation (ICRA)

作者： G. Tevatia S. Schaal Kawato Dynamic Brain Project ERATO Japan Science and Technology Agency Kyoto Japan Computational Learning and Motor Control Laboratory University of Southern California Los Angeles CA USA

Real-time control of the end-effector of a humanoid robot in external coordinates requires computationally efficient solutions of the inverse kinematics problem. In this context, this paper investigates methods of resolved motion rate control (RMRC) that employ optimization criteria to resolve kinematic redundancies. In particular we focus on two established techniques, the pseudo inverse with explicit optimization and the extended Jacobian method. We prove that the extended Jacobian method includes pseudo-inverse methods as a special solution. In terms of computational complexity, however pseudo-inverse and extended Jacobian differ significantly in favor of pseudo-inverse methods. Employing numerical estimation techniques, we introduce a computationally efficient version of the extended Jacobian with performance comparable to the original version. Our results are illustrated in simulation studies with a multiple degree-of-freedom robot, and were tested on a 30 degree-of-freedom humanoid robot.

关键词： Humanoid robots Jacobian matrices Robot kinematics Optimization methods Manipulators Motion control Orbital robotics Actuators Null space motor drives

来源：评论

学校读者我要写书评

暂无评论

Dependence of orientation tuning on recurrent excitation and inhibition in a network model of V1

Dependence of orientation tuning on recurrent excitation and...

引用

22nd Annual Conference on Neural Information Processing Systems, NIPS 2008

作者： Wimmer, Klaus Stimberg, Marcel Martin, Robert Schwabe, Lars Mariño, Jorge Schummers, James Lyon, David C. Sur, Mriganka Obermayer, Klaus Bernstein Center for Computational Neuroscience Technische Universität Berlin Germany Dept of Computer Science and Electrical Engineering University of Rostock Germany Dept of Medicine Neuroscience and Motor Control Group Univ. A Coruña Spain Dept of Brain and Cognitive Sci Picower Ctr for Learning and Memory MIT Cambridge United States Dept of Anatomy and Neurobiology University of California Irvine United States

ISBN: (纸本)9781605609492

The computational role of the local recurrent network in primary visual cortex is still a matter of debate. To address this issue, we analyze intracellular recording data of cat V1, which combine measuring the tuning of a range of neuronal properties with a precise localization of the recording sites in the orientation preference map. For the analysis, we consider a network model of Hodgkin-Huxley type neurons arranged according to a biologically plausible two-dimensional topographic orientation preference map. We then systematically vary the strength of the recurrent excitation and inhibition relative to the strength of the afferent input. Each parametrization gives rise to a different model instance for which the tuning of model neurons at different locations of the orientation map is compared to the experimentally measured orientation tuning of membrane potential, spike output, excitatory, and inhibitory conductances. A quantitative analysis shows that the data provides strong evidence for a network model in which the afferent input is dominated by strong, balanced contributions of recurrent excitation and inhibition. This recurrent regime is close to a regime of "instability", where strong, self-sustained activity of the network occurs. The firing rate of neurons in the best-fitting network is particularly sensitive to small modulations of model parameters, which could be one of the functional benefits of a network operating in this particular regime.

关键词： Neurons

来源：评论

学校读者我要写书评

暂无评论

Stochastic Differential Dynamic Programming

Stochastic Differential Dynamic Programming

引用

American control Conference

作者： Evangelos Theodorou Yuval Tassa Emo Todorov Computational Learning and Motor Control Lab Departments of Computer Science and Neuroscience University of Southern California Interdisciplinary Center for Neural Computation Hebrew University Jerusalem Israel Department of Computer Science and Engineering and the Department of Applied Mathematics University of Washington Seattle WA

ISBN: (纸本)9781424474264

关键词： dynamic programming theory of optimal control multiplicative noise Quartic stochastic derivative Additive noise correct entry dynamics Work area Handles

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：