检索结果-内蒙古大学图书馆

human-in-the-loop learning for Dynamic Congestion Games

IEEE TRANSACTIONS ON MOBILE COMPUTING 2024年第12期23卷 11159-11171页

作者： Li, Hongbo Duan, Lingjie Singapore Univ Technol & Design Pillar Engn Syst & Design Singapore 487372 Singapore

Today mobile users learn and share their traffic observations via crowdsourcing platforms (e.g., Google Maps and Waze). Yet such platforms simply cater to selfish users' myopic interests to recommend the shortest path, and do not encourage enough users to travel and learn other paths for future others. Prior studies focus on one-shot congestion games without considering users' information learning, while our work studies how users learn and alter traffic conditions on stochastic paths in a human-in-the-loop manner. In a typical parallel routing network with one deterministic path and multiple stochastic paths, our analysis shows that the myopic routing policy (used by Google Maps and Waze) leads to severe under-exploration of stochastic paths. This results in a price of anarchy (PoA) greater than 2, as compared to the socially optimal policy achieved through optimal exploration-exploitation tradeoff in minimizing the long-term social cost. Besides, the myopic policy fails to ensure the correct learning convergence about users' traffic hazard beliefs. To address this, we focus on informational (non-monetary) mechanisms as they are easier to implement than pricing. We first show that existing information-hiding mechanisms and deterministic path-recommendation mechanisms in Bayesian persuasion literature do not work with even PoA=infinity. Accordingly, we propose a new combined hiding and probabilistic recommendation (CHAR) mechanism to hide all information from a selected user group and provide state-dependent probabilistic recommendations to the other user group. Our CHAR mechanism successfully ensures PoA less than $\frac{5}{4}$54, which cannot be further reduced by any other informational (non-monetary) mechanism. Besides the parallel network, we further extend our analysis and CHAR mechanism to more general linear path graphs with multiple intermediate nodes, and we prove that the PoA results remain unchanged. Additionally, we carry out experiments with real-world

关键词： Stochastic processes Games Routing Hazards Costs Mobile computing human in the loop Dynamic congestion games human-in-the-loop learning mechanism design price of anarchy

来源：评论

学校读者我要写书评

暂无评论

To Optimize human-in-the-loop learning in Repeated Routing Games

引用

IEEE TRANSACTIONS ON MOBILE COMPUTING 2025年第4期24卷 2889-2899页

作者： Li, Hongbo Duan, Lingjie Singapore Univ Technol & Design Pillar Engn Syst & Design Singapore 487372 Singapore

Today navigation applications (e.g., Waze and Google Maps) enable human users to learn and share the latest traffic observations, yet such information sharing simply aids selfish users to predict and choose the shortest paths to jam each other. Prior routing game studies focus on myopic users in oversimplified one-shot scenarios to regulate selfish routing via information hiding or pricing mechanisms. For practical human-in-the-loop learning (HILL) in repeated routing games, we face non-myopic users of differential past observations and need new mechanisms (preferably non-monetary) to persuade users to adhere to the optimal path recommendations. We model the repeated routing game in a typical parallel transportation network, which generally contains one deterministic path and N stochastic paths. We first prove that no matter under the information sharing mechanism in use or the latest routing literature's hiding mechanism, the resultant price of anarchy (PoA) for measuring the efficiency loss from social optimum can approach infinity, telling arbitrarily poor exploration-exploitation tradeoff over time. Then we propose a novel user-differential probabilistic recommendation (UPR) mechanism to differentiate and randomize path recommendations for users with differential learning histories. We prove that our UPR mechanism ensures interim individual rationality for all users and significantly reduces PoA = infinity to close-to-optimal PoA = 1 + 1/4N+3, which cannot be further reduced by any other non-monetary mechanism. In addition to theoretical analysis, we conduct extensive experiments using real-world datasets to generalize our routing graphs and validate the close-to-optimal performance of UPR mechanism.

关键词： Routing Games Costs Stochastic processes Mobile computing Information sharing Transportation human in the loop Hazards Web services human-in-the-loop learning mechanism design price of anarchy repeated routing games

来源：评论

学校读者我要写书评

暂无评论

human-in-the-loop learning Methods Toward Safe DL-Based Autonomous Systems: A Review 1

引用

SAFECOMP Conference

作者： Rajendran, Prajit T. Espinoza, Huascar Delaborde, Agnes Mraidha, Chokri Univ Paris Saclay List CEA F-91120 Palaiseau France Lab Natl Metrol & Essais Trappes France

ISBN: (数字)9783030839062

ISBN: (纸本)9783030839062;9783030839055

The involvement of humans during the training phase can play a crucial role in mitigating some safety issues of Deep learning (DL)-based autonomous systems. This paper reviews the main concepts and methods for human-in-the-loop learning as a first step towards the development of a framework for human-machine teaming through safe learning and anomaly prediction. The methods come with their own set of challenges such as the variation in the training data provided by the human and test-time distributions, the cost involved to keep the human in the loop during the long training phase and the imperfection of the human to deal with unforeseen circumstances and define safer policies.

关键词： AI safety human-in-the-loop learning Deep learning

来源：评论

学校读者我要写书评

暂无评论

Robot learning on the job: human-in-the-loop autonomy and learning during deployment

引用

INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH 2024年

作者： Liu, Huihan Nasiriany, Soroush Zhang, Lance Bao, Zhiyao Zhu, Yuke Univ Texas Austin 2317 Speedway Austin TX 78712 USA

With the rapid growth of computing powers and recent advances in deep learning, we have witnessed impressive demonstrations of novel robot capabilities in research settings. Nonetheless, these learning systems exhibit brittle generalization and require excessive training data for practical tasks. To harness the capabilities of state-of-the-art robot learning models while embracing their imperfections, we present Sirius, a principled framework for humans and robots to collaborate through a division of work. In this framework, partially autonomous robots are tasked with handling a major portion of decision-making where they work reliably;meanwhile, human operators monitor the process and intervene in challenging situations. Such a human-robot team ensures safe deployments in complex tasks. Further, we introduce a new learning algorithm to improve the policy's performance on the data collected from the task executions. The core idea is re-weighing training samples with approximated human trust and optimizing the policies with weighted behavioral cloning. We evaluate Sirius in simulation and on real hardware, showing that Sirius consistently outperforms baselines over a collection of contact-rich manipulation tasks, achieving an 8% boost in simulation and 27% on real hardware than the state-of-the-art methods in policy success rate, with twice faster convergence and 85% memory size reduction. Videos and more details are available at https://***/sirius/.

关键词： human-in-the-loop learning robot learning imitation learning

来源：评论

学校读者我要写书评

暂无评论

human-in-the-loop learning for Personalized Diet Monitoring from Unstructured Mobile Data

引用

ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS 2019年第4期9卷 23-23页

作者： Hezarjaribi, Niloofar Mazrouee, Sepideh Hemati, Saied Chaytor, Naomi S. Perrigue, Martine Ghasemzadeh, Hassan Washington State Univ Elect Engn & Comp Sci Pullman WA 99163 USA Univ Calif San Diego Elect & Comp Engn La Jolla CA 92093 USA Univ Idaho Elect & Comp Engn Moscow ID 83843 USA Washington State Univ Elson S Floyd Coll Med Spokane WA 99210 USA

Lifestyle interventions with the focus on diet are crucial in self-management and prevention of many chronic conditions, such as obesity, cardiovascular disease, diabetes, and cancer. Such interventions require a diet monitoring approach to estimate overall dietary composition and energy intake. Although wearable sensors have been used to estimate eating context (e.g., food type and eating time), accurate monitoring of dietary intake has remained a challenging problem. In particular, because monitoring dietary intake is a self-administered task that involves the end-user to record or report their nutrition intake, current diet monitoring technologies are prone to measurement errors related to challenges of human memory, estimation, and bias. New approaches based on mobile devices have been proposed to facilitate the process of dietary intake recording. These technologies require individuals to use mobile devices such as smartphones to record nutrition intake by either entering text or taking images of the food. Such approaches, however, suffer from errors due to low adherence to technology adoption and time sensitivity to the dietary intake context. In this article, we introduce EZNutriPal,(1) an interactive diet monitoring system that operates on unstructured mobile data such as speech and free-text to facilitate dietary recording, real-time prompting, and personalized nutrition monitoring. EZNutriPal features a natural language processing unit that learns incrementally to add user-specific nutrition data and rules to the system. To prevent missing data that are required for dietary monitoring (e.g., calorie intake estimation), EZNutriPal devises an interactive operating mode that prompts the end-user to complete missing data in real-time. Additionally, we propose a combinatorial optimization approach to identify the most appropriate pairs of food names and food quantities in complex input sentences. We evaluate the performance of EZNutriPal using real data collect

关键词： Diet monitoring wireless health mobile computing human-in-the-loop learning unstructured data real-time prompting combinatorial optimization assignment problem

来源：评论

学校读者我要写书评

暂无评论

HiLoTEL: Virtual Reality Robot Interface-Based human-in-the-loop Task Execution and learning in the Physical World Through Its Digital Twin.

HiLoTEL: Virtual Reality Robot Interface-Based Human-in-the-...

引用

IEEE Conference on Virtual Reality and 3D User Interfaces (VR)

作者： Ergogo, Amanuel Dall'Alba, Diego Korzeniowski, Przemyslaw SANO Ctr Computat Personalized Med Krakow Poland

ISBN: (纸本)9798350374490;9798350374506

HiLoTEL is a flexible virtual-reality framework for executing and learning tasks in virtual and physical environments. It enables human experts to collaborate with learning agents and intervene when necessary through human-in-the-loop imitation learning. HiLoTEL reduces the need to carry out repetitive tasks, providing the user with an intuitive supervision interface. The system is tested on Pick-and-Place task, considering both teleoperated and passthrough interaction modalities. The results show that HiLoTEL improves success rates while maintaining human-level completion time and providing users with 71% hands free supervision time, thus enabling effective human-robot collaboration.

关键词： human-centered computing human computer interaction (HCI) human-in-the-loop learning Imitation learning human-robot collaboration

来源：评论

学校读者我要写书评

暂无评论

human-In-The-loop Methods for Data-Driven and Reinforcement learning Systems

Human-In-The-Loop Methods for Data-Driven and Reinforcement ...

引用

作者： Guimaraes Goecks, Vinicius Texas A&M University

学位级别：Ph.D., Doctor of Philosophy

Recent successes combine reinforcement learning algorithms and deep neural networks, despite reinforcement learning not being widely applied to robotics and real world scenarios. This can be attributed to the fact that current state-of-the-art, end-to-end reinforcement learning approaches still requires thousands or millions of data samples to converge to a satisfactory policy and are subject to catastrophic failures during training. Conversely, in real world scenarios and after just a few data samples, humans are able to either provide demonstrations of the task, intervene to prevent catastrophic actions, or simply evaluate if the policy is performing correctly. This dissertation investigates how to integrate these human interaction modalities to the reinforcement learning loop, increasing sample efficiency and enabling real-time reinforcement learning in robotics and real world scenarios. The theoretical foundation of this dissertation builds upon the actor- critic reinforcement learning architecture, the use of function approximation to represent action- and value-based functions, and the integration of different human interaction modalities; namely, task demonstration, intervention, and evaluation, to these functions and to reward signals. This novel theoretical foundation is called Cycle-of-learning, a reference to how different human interaction modalities are cycled and combined to reinforcement learning algorithms. This approach is validated on an Unmanned Air System (UAS) collision avoidance and landing scenario using a high-fidelity simulated environment and several continuous control tasks standardized to benchmark reinforcement learning algorithms. Results presented in this dissertation show that the reward signal that is learned based upon human interaction accelerates the rate of learning of reinforcement learning algorithms, when compared to traditional handcrafted or binary reward signals returned by the environment. Results also show that learning f

关键词： human-in-the-loop learning Reinforcement learning Deep learning Machine learning human-Robot Interaction human-Machine Interaction Autonomous systems Unmanned Air Systems Unmanned Air Vehicles Cycle-of-learning

来源：评论

学校读者我要写书评

暂无评论

learning by Asking Questions for Knowledge-Based Novel Object Recognition

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2024年第6期132卷 2290-2309页

作者： Uehara, Kohei Harada, Tatsuya Univ Tokyo Tokyo Japan RIKEN Tokyo Japan

In real-world object recognition, there are numerous object classes to be recognized. Traditional image recognition methods based on supervised learning can only recognize object classes present in the training data, and have limited applicability in the real world. In contrast, humans can recognize novel objects by questioning and acquiring knowledge about them. Inspired by this, we propose a framework for acquiring external knowledge by generating questions that enable the model to instantly recognize novel objects. Our framework comprises three components: the object classifier (OC), which performs knowledge-based object recognition, the question generator (QG), which generates knowledge-aware questions to acquire novel knowledge, and the policy decision (PD) Model, which determines the "policy" of questions to be asked. The PD model utilizes two strategies, namely "confirmation" and "exploration"-the former confirms candidate knowledge while the latter explores completely new knowledge. Our experiments demonstrate that the proposed pipeline effectively acquires knowledge about novel objects compared to several baselines, and realizes novel object recognition utilizing the obtained knowledge. We also performed a real-world evaluation in which humans responded to the generated questions, and the model used the acquired knowledge to retrain the OC, which is a fundamental step toward a real-world human-in-the-loop learning-by-asking framework. We plan to release the dataset immediately upon acceptance of our work.

关键词： Visual question generation Novel object recognition human-in-the-loop learning Knowledge acquisition

来源：评论

学校读者我要写书评

暂无评论

human as AI *** human-in the-loop reinforcement leaming for safe and efficient autonomous driving

引用

Communications in Transportation Research 2024年第1期4卷 124-147页

作者： Zilin Huang Zihao Sheng Chengyuan Ma Sikai Chen Department of Civil and Environmental Engineering University of Wisconsin-MadisonMadisonWI53706USA

Despite significant progress in autonomous vehicles(AVs),the development of driving policies that ensure both the safety of AVs and traffic flow efficiency has not yet been fully *** this paper,we propose an enhanced human-in-the-loop reinforcement learning method,termed the human as AI mentor-based deep reinforcement learning(HAIM-DRL)framework,which facilitates safe and efficient autonomous driving in mixed traffic *** inspiration from the human learning process,we first introduce an innovative learning paradigm that effectively injects human intelligence into AI,termed human as AI mentor(HAIM).In this paradigm,the human expert serves as a mentor to the AI *** allowing the agent to sufficiently explore uncertain environments,the human expert can take control in dangerous situations and demonstrate correct actions to avoid potential *** the other hand,the agent could be guided to minimize traffic flow disturbance,thereby optimizing traffic flow *** detail,HAIM-DRL leverages data collected from free exploration and partial human demonstrations as its two training ***,we circumvent the intricate process of manually designing reward functions;instead,we directly derive proxy state-action values from partial human demonstrations to guide the agents’policy ***,we employ a minimal intervention technique to reduce the human mentor’s cognitive *** results show that HAIM-DRL outperforms traditional methods in driving safety,sampling efficiency,mitigation of traffic flow disturbance,and generalizability to unseen traffic scenarios.

关键词： human as AI mentor paradigm Autonomous driving Deep reinforcement learning human-in-the-loop learning Driving policy Mixed traffic platoon

来源：评论

学校读者我要写书评

暂无评论

Selective eye-gaze augmentation to enhance imitation learning in Atari games

引用

NEURAL COMPUTING & APPLICATIONS 2023年第32期35卷 23401-23410页

作者： Thammineni, Chaitanya Manjunatha, Hemanth Esfahani, Ehsan T. Univ Buffalo Human Loop Syst Lab Buffalo NY 14260 USA

This paper presents the selective use of eye-gaze information in learning human actions in Atari games. Extensive evidence suggests that our eye movements convey a wealth of information about the direction of our attention and mental states and encode the information necessary to complete a task. Based on this evidence, we hypothesize that selective use of eye-gaze, as a clue for attention direction, will enhance the learning from demonstration. For this purpose, we propose a selective eye-gaze augmentation (SEA) network that learns when to use the eye-gaze information. The proposed network architecture consists of three sub-networks: gaze prediction, gating, and action prediction network. Using the prior 4 game frames, a gaze map is predicted by the gaze prediction network, which is used for augmenting the input frame. The gating network will determine whether the predicted gaze map should be used in learning and is fed to the final network to predict the action at the current frame. To validate this approach, we use publicly available Atari human Eye-Tracking And Demonstration (Atari-HEAD) dataset consists of 20 Atari games with 28 million human demonstrations and 328 million eye-gazes (over game frames) collected from four subjects. We demonstrate the efficacy of selective eye-gaze augmentation compared to the state-of-the-art Attention Guided Imitation learning (AGIL) and Behavior Cloning (BC). The results indicate that the selective augmentation approach (the SEA network) performs significantly better than the AGIL and BC. Moreover, to demonstrate the significance of selective use of gaze through the gating network, we compare our approach with the random selection of the gaze. Even in this case, the SEA network performs significantly better, validating the advantage of selectively using the gaze in demonstration learning.

关键词： Imitation learning human-in-the-loop learning learning by demonstration

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：