检索结果-内蒙古大学图书馆

Framing reinforcement learning from human reward: Reward positivity, temporal discounting, episodicity, and performance

引用

ARTIFICIAL INTELLIGENCE 2015年 225卷 24-50页

作者： Knox, W. Bradley Stone, Peter MIT Media Lab Cambridge MA 02139 USA Univ Texas Austin Dept Comp Sci Austin TX 78712 USA

Several studies have demonstrated that reward from a human trainer can be a powerful feedback signal for control-learning algorithms. However, the space of algorithms for learning from such human reward has hitherto not been explored systematically. Using model-based reinforcement learning from human reward, this article investigates the problem of learning from human reward through six experiments, focusing on the relationships between reward positivity, which is how generally positive a trainer's reward values are;temporal discounting, the extent to which future reward is discounted in value;episodicity, whether task learning occurs in discrete learning episodes instead of one continuing session;and task performance, the agent's performance on the task the trainer intends to teach. This investigation is motivated by the observation that an agent can pursue different learning objectives, leading to different resulting behaviors. We search for learning objectives that lead the agent to behave as the trainer intends. We identify and empirically support a "positive circuits" problem with low discounting (i.e., high discount factors) for episodic, goal-based tasks that arises from an observed bias among humans towards giving positive reward, resulting in an endorsement of myopic learning for such domains. We then show that converting simple episodic tasks to be non-episodic (i.e., continuing) reduces and in some cases resolves issues present in episodic tasks with generally positive reward and-relatedly-enables highly successful learning with non-myopic valuation in multiple user studies. The primary learning algorithm introduced in this article, which we call "VI-TAMER", is the first algorithm to successfully learn non-myopically from reward generated by a human trainer;we also empirically show that such non-myopic valuation facilitates higher-level understanding of the task. Anticipating the complexity of real-world problems, we perform further studies-one with a fai

关键词： Reinforcement learning Modeling user behavior end-user programming Human-agent interaction Interactive machine learning Human teachers

来源：评论

学校读者我要写书评

暂无评论

Design patterns for context-aware services

引用

MULTIMEDIA TOOLS AND APPLICATIONS 2015年第7期74卷 2337-2358页

作者： Park, Jeongkyu Lee, Keung Hae LIG Syst R&D Ctr Seoul South Korea Korea Aerosp Univ Dept Comp Engn Hwajondong 200-1Hanggongdae Gil 100 Goyang City 412791 Gyeonggi Do South Korea

Numerous context-aware applications were proposed in the literature. Relatively, little attention was paid to understanding how they are related. This paper introduces a framework for classifying context-aware services. We studied 48 context-aware service scenarios discussed in the literature, deriving a model of seven design patterns for context-aware services. This paper discusses the model and design patterns in detail. Also presented in this paper is a novel programming model for enabling the end-user to build context-aware services. The concept of programming by Selection and its prototype implementation are presented. The feasibility of this method is demonstrated. It is argued that programming by Selection offers an alternative end-user programming method for context-aware services.

关键词： Context-aware Pervasive computing Uniquitous computing end-user programming Smart home Smart space Design pattern

来源：评论

学校读者我要写书评

暂无评论

Human Tutorial Instruction in the Raw

引用

ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS 2015年第1期5卷 1-29页

作者： Gil, Yolanda Univ So Calif Inst Informat Sci 4676 Admiralty Way Marina Del Rey CA 90292 USA Univ So Calif Dept Comp Sci Marina Del Rey CA 90292 USA

Humans learn procedures from one another through a variety of methods, such as observing someone do the task, practicing by themselves, reading manuals or textbooks, or getting instruction from a teacher. Some of these methods generate examples that require the learner to generalize appropriately. When procedures are complex, however, it becomes unmanageable to induce the procedures from examples alone. An alternative and very common method for teaching procedures is tutorial instruction, where a teacher describes in general terms what actions to perform and possibly includes explanations of the rationale for the actions. This article provides an overview of the challenges in using human tutorial instruction for teaching procedures to computers. First, procedures can be very complex and can involve many different types of interrelated information, including (1) situating the instruction in the context of relevant objects and their properties, (2) describing the steps involved, (3) specifying the organization of the procedure in terms of relationships among steps and substeps, and (4) conveying control structures. Second, human tutorial instruction is naturally plagued with omissions, oversights, unintentional inconsistencies, errors, and simply poor design. The article presents a survey of work from the literature that highlights the nature of these challenges and illustrates them with numerous examples of instruction in many domains. Major research challenges in this area are highlighted, including the difficulty of the learning task when procedures are complex, the need to overcome omissions and errors in the instruction, the design of a natural user interface to specify procedures, the management of the interaction of a human with a learning system, and the combination of tutorial instruction with other teaching modalities.

关键词： Procedure learning intelligent user interfaces natural language interpretation instruction end-user programming interactive learning

来源：评论

学校读者我要写书评

暂无评论

Impact and utility of smell-driven performance tuning for end-user programmers

引用

JOURNAL OF VISUAL LANGUAGES AND COMPUTING 2015年 28卷 176-194页

作者： Chambers, Christopher Scaffidi, Christopher Oregon State Univ Sch Elect Engn & Comp Sci Corvallis OR 97331 USA

This paper proposes a technique, called Smell-driven performance tuning (SDPT), which semi-automatically assists end-user programmers with fixing performance problems in visual dataflow programming languages. A within-subjects laboratory experiment showed SDPT increased end-user programmers' success rate and decreased the time they required. Another study, based on using SDPT to analyze a corpus of example end-user programs, demonstrated that applying all available SDPT transformations achieved an execution time improvement of 42% and a memory usage improvement of 20%, comparable to improvements that expert programmers historically had manually achieved on the same programs. These results indicate that SDPT is an effective method for helping end-user programmers to fix performance problems. (C) 2015 Elsevier Ltd. All rights reserved.

关键词： end-user programming Performance Visual language

来源：评论

学校读者我要写书评

暂无评论

Utility and accuracy of smell-driven performance analysis for end-user programmers

引用

JOURNAL OF VISUAL LANGUAGES AND COMPUTING 2015年 26卷 1-14页

作者： Chambers, Christopher Scaffidi, Christopher Oregon State Univ Sch Elect Engn & Comp Sci Corvallis OR 97331 USA

This paper proposes a technique, called smell-driven performance analysis (SDPA), which automatically provides situated explanations within a visual dataflow language IDE to help end-user programmers to overcome performance problems without leaving the visual dataflow paradigm. An experiment showed SDPA increased end-user programmers' success rates at finding performance problems and decreased the time required for finding solutions. Another study, based on using SDPA to analyze a corpus of example end-user programs, revealed that it is usually accurate at identifying performance problems. Based on these results, we conclude that SDPA provides a reliable basis for helping end-user programmers to troubleshoot performance problems, as well as a potential foundation for future work aimed at training users and at aiding code reuse. (C) 2014 Elsevier Ltd. All rights reserved.

关键词： end-user programming Performance Visual language

来源：评论

学校读者我要写书评

暂无评论

Investigating the Role of Mutual Cognitive Environment for end-user programming

引用

6th IFIP Conference on Artificial Intelligence Applications and Innovations

作者： Barraquand, Remi Reignier, Patrick INRIA Grenoble Rhones Alpes Res Ctr F-38330 Montbonnot St Martin France

ISBN: (纸本)9783642162381

In this paper we present a situated end user programming approach where user co-constructs, in an iterative process, a mutual cognitive environment with the system. We argue that co-construction of a mutual cognitive environment, between both the human and the system, is a key toward social human-computer interaction. Preliminary results are illustrated with a step by step case study: a user teaches the system new perceptual and abstract concepts using hand gesture and an interactive learning table.

关键词： end-user programming Situation Models Cognitive Environment

来源：评论

学校读者我要写书评

暂无评论

Dexen: A scalable and extensible platform for experimenting with population-based design exploration algorithms

引用

AI EDAM-ARTIFICIAL INTELLIGENCE FOR ENGINEERING DESIGN ANALYSIS AND MANUFACTURING 2015年第4期29卷 443-455页

作者： Janssen, Patrick Natl Univ Singapore Dept Architecture Singapore 117566 Singapore

A platform for experimenting with population-based design exploration algorithms is presented, called Dexen. The platform has been developed in order to address the needs of two distinct groups of users loosely labeled as researchers and designers. Whereas the researchers group focuses on creating and testing customized toolkits, the designers group focuses on applying these toolkits in the design process. A platform is required that is scalable and extensible: scalable to allow computationally demanding population-based exploration algorithms to be executed on distributed hardware within reasonable time frames, and extensible to allow researchers to easily implement their own customized toolkits consisting of specialized algorithms and user interfaces. In order to address these requirements, a three-tier client-server system architecture has been used that separates data storage, domain logic, and presentation. This separation allows customized toolkits to be created for Dexen without requiring any changes to the data or logic tiers. In the logic tier, Dexen uses a programming model in which tasks only communicate through data objects stored in a key-value database. The paper ends with a case study experiment that uses a multicriteria evolutionary algorithm toolkit to explore alternative configurations for the massing and facade design of a large residential development. The parametric models for developing and evaluating design variants are described in detail. A population of design variants are evolved, a number of which are selected for further analysis. The case study demonstrates how evolutionary exploration methods can be applied to a complex design scenario without requiring any scripting.

关键词： Design Exploration Design Optimization Distributed Computing end-user programming Evolutionary Algorithms

来源：评论

学校读者我要写书评

暂无评论

A Spreadsheet Model for Handling Streaming Data 15

A Spreadsheet Model for Handling Streaming Data

引用

33rd Annual CHI Conference on Human Factors in Computing Systems (CHI)

作者： Chang, Kerry Shih-Ping Myers, Brad A. Carnegie Mellon Univ Human Comp Interact Inst Pittsburgh PA 15213 USA

ISBN: (纸本)9781450331456

We present a spreadsheet model for working with streaming data. Our prototype tool presents techniques to let the user stream data from web services and web input elements to a spreadsheet without preprogramming those sources into the tool. Spreadsheet cells record metadata about where and when the data came from, allowing the user to view and manipulate streaming data using temporal information. Starting and pausing a data stream in the spreadsheet can be controlled programmatically using values computed by spreadsheet cells, making the spreadsheet program highly dynamic and interactive. We demonstrate the range of our design with a series of examples highlighting its ability to create different kinds of applications that process real-time data from the web using simple spreadsheet formulas.

关键词： Spreadsheets streaming data web services end-user programming live programming

来源：评论

学校读者我要写书评

暂无评论

Natural Notation for the Domestic Internet of Things 5

Natural Notation for the Domestic Internet of Things

引用

5th International Symposium of end-user Development (IS-EUD)

作者： Perera, Charith Aghaee, Saeed Blackwell, Alan Australian Natl Univ Res Sch Comp Sci Canberra ACT Australia Open Univ Fac Maths Comp & Technol Milton Keynes MK7 6AA Bucks England Univ Cambridge Comp Lab Cambridge CB2 3QG England

ISBN: (纸本)9783319184258;9783319184241

This study explores the use of natural language to give instructions that might be interpreted by Internet of Things (IoT) devices in a domestic 'smart home' environment. We start from the proposition that reminders can be considered as a type of end-user programming, in which the executed actions might be performed either by an automated agent or by the author of the reminder. We conducted an experiment in which people wrote sticky notes specifying future actions in their home. In different conditions, these notes were addressed to themselves, to others, or to a computer agent. We analyse the linguistic features and strategies that are used to achieve these tasks, including the use of graphical resources as an informal visual language. The findings provide a basis for design guidance related to end-user development for the Internet of Things.

关键词： Internet of things Smart home end-user programming user study

来源：评论

学校读者我要写书评

暂无评论

Browser Record and Replay as a Building Block for end-user Web Automation Tools 24

Browser Record and Replay as a Building Block for End-User W...

引用

24th International Conference on World Wide Web (WWW)

作者： Chasins, Sarah Barman, Shaon Gulwani, Sumit Bodik, Rastislav Univ Calif Berkeley Berkeley CA 94720 USA Microsoft Res Redmond WA USA

ISBN: (纸本)9781450334730

To build a programming by demonstration (PBD) web scraping tool for end users, one needs two central components: a list finder, and a record and replay tool. A list finder extracts logical tables from a webpage. A record and replay (R+R) system records a user's interactions with a webpage, and replays them programmatically. The research community has invested substantial work in list finding variously called wrapper induction, structured data extraction, and template detection. In contrast, researchers largely considered the browser R+R problem solved until recently, when webpage complexity and interactivity began to rise. We argue that the increase in interactivity necessitates the use of new, more robust R+R approaches, which will facilitate the PBD web tools of the future. Because robust R+R is difficult to build and understand, we argue that tool developers need an R+R layer that they can treat as a black box. We have designed an easy -to -use API that allows programmers to use and even customize R+R, without having to understand R+R internals. We have instantiated our API in Ringer, our robust R+R tool. We use the API to implement WebCombine, a PBD scraping tool. A WebCombine user demonstrates how to collect the first row of a relational dataset, and the tool collects all remaining rows. WebCombine uses the Ringer API to handle navigation between pages, enabling users to scrape from modern, interaction -heavy pages. We demonstrate WebCombine by collecting a 3,787,146 row dataset from Google Scholar that allows us to explore the relationship between researchers' years of experience and their papers' citation counts.

关键词： Record and Replay end-user programming programming by Demonstration Automation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：