While there is abundant access to data management technology today, working with data is still challenging for the average user. One common means of manipulating data is with SQL on relational databases, but this requ...
详细信息
While there is abundant access to data management technology today, working with data is still challenging for the average user. One common means of manipulating data is with SQL on relational databases, but this requires knowledge of SQL as well as the database’s schema and contents. Consequently, previous work has proposed oblique query specification (OQS) methods such as natural language or programming-by-example to allow users to imprecisely specify their query intent. These methods, however, suffer from either low precision or low expressivity and, in addition, produce a list of candidate SQL queries that make it difficult for users to select their final target query.
My thesis is that OQS systems should maximize user domain expertise to triangulate the user’s desired query. First, I demonstrate how to leverage previously-issued SQL queries to improve the accuracy of natural language interfaces. Second, I propose a system allowing users to specify a query with both natural language and programming-by-example. Finally, I develop a system where users provide feedback on system-suggested tuples to select a SQL query from a set of candidate queries generated by an OQS system.
Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable d...
详细信息
Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high-quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI-based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it's often hard to interpret or manipulate these representations. In this state-of-the-art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research.
Context: programming by example (PBE) is increasingly assisting human users by recognizing and executing repetitive tasks, such as text editing and spreadsheet manipulation. Yet, existing work falls short on dealing w...
详细信息
Context: programming by example (PBE) is increasingly assisting human users by recognizing and executing repetitive tasks, such as text editing and spreadsheet manipulation. Yet, existing work falls short on dealing with rich-formatted documents like PowerPoint (PPT) files, when examples are few and collecting them is intrusive. Objective: This article presents TIDY, a PBE-based framework, to assist automated entity transformations for their layout and style consistency in rich-formatted documents like PowerPoint, in a way adaptive to entity contexts and flexible with user selections. Methods: TIDY achieves this by examining entities' operation histories, and proposes a two-stage framework to first identify user intentions behind histories and then make wise next-operation recommendations for users, in order to maintain the entity consistency for rich-formatted documents. Results: We implemented TIDY as a prototype tool and integrated it into PowerPoint as a plug-in module. We experimentally evaluated TIDY with real-world user operation data. The evaluation reports that TIDY achieved promising effectiveness with a hit rate of 77.3% on average, which was stably holding for a variety of editing tasks. Besides, TIDY took only marginal time overhead, costing several to several tens of milliseconds, to complete each recommendation. Conclusion: TIDY assists users to complete repetitive tasks in rich-formatted documents by non-intrusive user intention recognition and smart next-operation recommendations, which is effective and practically useful.
This paper presents a new approach for synthesizing transformations on tree-structured data, such as Unix directories and XML documents. We consider a general abstraction for such data, called hierarchical data trees ...
详细信息
This paper presents a new approach for synthesizing transformations on tree-structured data, such as Unix directories and XML documents. We consider a general abstraction for such data, called hierarchical data trees (HDTs) and present a novel example-driven synthesis algorithm for HDT transformations. Our central insight is to reduce the problem of synthesizing tree transformers to the synthesis of list transformations that are applied to the paths of the tree. The synthesis problem over lists is solved using a new algorithm that combines SMT solving and decision tree learning. We have implemented our technique in a system called HADES and show that HADES can automatically synthesize a variety of interesting transformations collected from online forums.
The Lapidary interface design tool is a demonstrational system that allows the graphics and run-time behaviors that go inside an application window to be specified pictorially. In particular, Lapidary allows the desig...
详细信息
The Lapidary interface design tool is a demonstrational system that allows the graphics and run-time behaviors that go inside an application window to be specified pictorially. In particular, Lapidary allows the designer to draw example pictures of application-specific graphical objects that the end user will manipulate (such as boxes, arrows, or elements of a list), the feedback that shows which objects are selected (such as small boxes on the sides and corners of an object), and the dynamic feedback objects (such as hairline boxes to show where an object is being dragged). The run-time behavior of all these objects can be specified ina straightforward way using constraints, demonstration, and dialog boxes that allow the designer to provide abstract descriptions of the interactive response to the input devices. Lapidary generalizes from these specific example pictures and behaviors to create prototype objects and behaviors from which instances can be made at run-time. A novel feature of Lapidary's implementation is its use of constraints that have been explicitly specified by the designer to help it generalize example objects and behaviors and to guide it in making inferences.
With the maturing of technology for model checking and constraint solving, there is an emerging opportunity to develop programming tools that can transform the way systems are specified. In this paper, we propose a ne...
详细信息
ISBN:
(纸本)9781450320146
With the maturing of technology for model checking and constraint solving, there is an emerging opportunity to develop programming tools that can transform the way systems are specified. In this paper, we propose a new way to program distributed protocols using concolic snippets. Concolic snippets are sample execution fragments that contain both concrete and symbolic values. The proposed approach allows the programmer to describe the desired system partially using the traditional model of communicating extended finite-state-machines (EFSM), along with high-level invariants and concrete execution fragments. Our synthesis engine completes an EFSM skeleton by inferring guards and updates from the given fragments which is then automatically analyzed using a model checker with respect to the desired invariants. The counterexamples produced by the model checker can then be used by the programmer to add new concrete execution fragments that describe the correct behavior in the specific scenario corresponding to the counterexample. We describe TRANSIT, a language and prototype implementation of the proposed specification methodology for distributed protocols. Experimental evaluations of TRANSIT to specify cache coherence protocols show that (1) the algorithm for expression inference from concolic snippets can synthesize expressions of size 15 involving typical operators over commonly occurring types, (2) for a classical directory-based protocol, TRANSIT automatically generates, in a few seconds, a complete implementation from a specification consisting of the EFSM structure and a few concrete examples for every transition, and (3) a published partial description of the SGI Origin cache coherence protocol maps directly to symbolic examples and leads to a complete implementation in a few iterations, with the programmer correcting counterexamples resulting from underspecified transitions by adding concrete examples in each iteration.
Multi-robot manipulation tasks can be complicated, due to the need for tight temporal coupling between the robots. However, this is an ideal scenario for human-agent-robot teams, since performing all of the manipulati...
详细信息
ISBN:
(纸本)9780981738130
Multi-robot manipulation tasks can be complicated, due to the need for tight temporal coupling between the robots. However, this is an ideal scenario for human-agent-robot teams, since performing all of the manipulation aspects of the task autonomously is not feasible without additional sensors. To ameliorate this problem, we present a paradigm for allowing subjects to configure a user interface for multi-robot manipulation tasks; using a macro acquisition system for learning combined manipulation/driving tasks. Learning takes place within this social setting; the human demonstrates the task to the single robot, but the robot uses an internal teamwork model to modify the macro to account for the actions of the second robot during execution. This allows the same macro to be useful in a variety of cooperative situations. In this paper, we show that our system is highly effective at empowering human-agent-robot teams within a household multi-robot manipulation setting and is rated favorably over a non-configurable user interface by a significant portion of the users.
暂无评论