In this demo paper, we present an Extended Reality (XR) framework providing a streamlined workflow for creating and interacting with intelligent virtual agents (IVAs) with multimodal information processing capabilitie...
详细信息
ISBN:
(数字)9798331514846
ISBN:
(纸本)9798331525637
In this demo paper, we present an Extended Reality (XR) framework providing a streamlined workflow for creating and interacting with intelligent virtual agents (IVAs) with multimodal information processing capabilities using commercially available artificial intelligence (AI) tools and cloud services such as large language and vision models. The system supports (i) the integration of high-quality, customizable virtual 3D human models for visual representations of IVAs and (ii) multimodal communication with generative AI-driven IVAs in immersive XR, featuring realistic human behavior simulations. Our demo showcases the enormous potential and vast design space of embodied IVAs for various XR applications.
Understanding and evaluating user behavior in virtual reality environments is challenging for researchers. Stereoscopic perception is highly dependent on the point of view, so it is necessary to account for multiple s...
详细信息
We present a music interface implementing a bowed string. The bow is realised using a commercially available haptic device, consisting of a stylus attached to a robotic arm. While playing the virtual strings with the ...
详细信息
The shift to online education, accelerated by the COVID-19 pandemic, has introduced challenges in monitoring student engagement, an essential aspect of effective teaching. In response, real-time student monitoring int...
详细信息
ISBN:
(纸本)9798400703300
The shift to online education, accelerated by the COVID-19 pandemic, has introduced challenges in monitoring student engagement, an essential aspect of effective teaching. In response, real-time student monitoring interfaces have emerged as potential tools to aid instructors, yet their efficacy has not been thoroughly examined. Addressing this gap, we conducted a controlled experiment with 20 instructors examining the impact of engagement cues (presence versus absence) and student engagement levels (high versus low) on instructors’ monitoring effectiveness, teaching behavior adjustments, and cognitive load in online classes. Our findings underscored the fundamental benefits of student engagement monitoring interfaces for improving monitoring quality and effectiveness. Furthermore, our study highlighted the critical need for customizable interfaces that could balance the informational utility of engagement cues with the associated cognitive load and psychological stress on instructors. These insights may offer design implications for the design of future student engagement monitoring interfaces.
Social robots are widely used for educational activities, especially to attract children’s attention. As a side effect, pupils’ excitement can suppress their focus on the tutors and the content being approached. A p...
详细信息
ISBN:
(数字)9798350375022
ISBN:
(纸本)9798350375039
Social robots are widely used for educational activities, especially to attract children’s attention. As a side effect, pupils’ excitement can suppress their focus on the tutors and the content being approached. A potential solution to tackle this issue is to equip the robot with storytelling strategies, which have been growing remarkably in recent years thanks to advances in Large Language Models. However, few studies are still addressing the resulting application in real-world conditions. In this work, we are exploring the GPT-3.5 model for story generation based on content to be approached in maker-space classes. To achieve our goals, we implemented a web application for content insertion that connects to the robot through ROS. The proposal was validated in two phases: a first phase of interviews with 5 tutors of maker-space to present our solution and get their feedback, and two 90-minute sessions with pupils for real-world validation. Results suggested the proposal has high potential for supporting multiple languages and generating suitable stories for diverse contexts. Furthermore, adding social behaviors, as encouragement and sentiment analysis, can help in the students’ expectation handling.
Handheld-style head-mounted displays (HMDs) are becoming increasingly popular as a convenient option for onsite exhibitions. However, they lack established practices for basic interactions, particularly pointing metho...
详细信息
The efficient generation of datasets for deep neural networks is crucial for enhancing their performance, especially in complex tasks like grasp pose detection in robotic grasp planning. Traditional methods often reso...
详细信息
ISBN:
(数字)9798350376340
ISBN:
(纸本)9798350376357
The efficient generation of datasets for deep neural networks is crucial for enhancing their performance, especially in complex tasks like grasp pose detection in robotic grasp planning. Traditional methods often resort to random selection or rely on human-annotated datasets, making it challenging for networks to accurately learn graspable areas. For 3D grasp pose detection, an exhaustive search within an expansive space of points consumes substantial time and often overlooks an object's potential graspable regions. This paper introduces an algorithm capable of leveraging depth sensor information, particularly from devices like Kinect, to precisely determine the best grasp poses for object handovers. The proposed method, named Explorable Grasp Detection is based on the idea of exploitation and exploration, ensuring there is no redundant grasp detection within the same region. The results derived from DexNet's mesh objects dataset indicate remarkable computational efficiency, achieving grasp results in less than one second for various objects and maintaining a high success rate. Furthermore, error functions as geometry grasp quality metrics tailored for a two-finger gripper are incorporated to evaluate the grasp quality. One of the most notable contributions of this paper is the algorithm's ability to not only detect optimal grasping regions rapidly but also to serve as an instrumental tool for generating annotated datasets, thereby pushing forward the boundaries in robotic object manipulation techniques.
This study presents a three-dimensional localization of a ball drop in a multi-surface environment using cost-effective data acquisition devices, and proposes two learning-based methods to improve the baseline classic...
详细信息
ISBN:
(数字)9798331529734
ISBN:
(纸本)9798331529741
This study presents a three-dimensional localization of a ball drop in a multi-surface environment using cost-effective data acquisition devices, and proposes two learning-based methods to improve the baseline classical localization method based on GCC-PHAT-estimated TDOA values. The first proposed method employs a ball drop surface classification executed by Random Forest and XGBoost models to enhance localization performance, whereas the second method conducts region classification by using both basic and multi-branch neural network models, achieving improved and more efficient localization performance. A dataset of real-world sparse ball drop samples was collected in a controlled multi-surface environment for experimental execution, where results demonstrate that the two proposed methods improve the localization performance across various scenarios, with the region classification method significantly outperforming the baseline method, achieving a 17% reduction in the Mean Euclidean Distance error metric. Furthermore, the surface and region classification methods reduce the localization time compared to the baseline method by 50% and 97%, respectively. The obtained results reveal the potential of combining simple learning-based models with affordable microphone sensors for achieving an accurate localization of sparse audio samples in a multi-surface environment.
As tech giants such as Apple and Meta invest heavily in Virtual and Augmented Reality (VR/AR) technologies, often collectively termed Extended Reality (XR) devices, a significant societal concern emerges: The use of e...
详细信息
暂无评论