检索结果-内蒙古大学图书馆

Using Language-Aligned Gesture Embeddings for Understanding Gestures Accompanying Math Terms

学校读者我要写书评

暂无评论

Using Language-Aligned Gesture Embeddings for Understanding ...

IEEE computer Society Conference on computer Vision and Pattern Recognition Workshops (CVPRW)

作者： Tristan Maidment Purav J Patel Erin Walker Adriana Kovashka Intelligent Systems Program University of Pittsburgh Pittsburgh PA USA University of Maryland College Park MD USA Computer Science University of Pittsburgh Pittsburgh PA USA Learning Research and Development Center University of Pittsburgh Pittsburgh PA USA

ISBN: (数字)9798350365474

ISBN: (纸本)9798350365481

In this paper, we introduce an approach for recognizing and classifying gestures that accompany mathematical terms, in a new collection we name the "GAMT" dataset. Our method uses language as a means of providing context to classify gestures. Specifically, we use a CLIP-style framework to construct a shared embedding space for gestures and language, experimenting with various methods for encoding gestures within this space. We evaluate our method on our new dataset containing a wide array of gestures associated with mathematical terms. The shared embedding space leads to a substantial improvement in gesture classification. Furthermore, we identify an efficient model that excelled at classifying gestures from our unique dataset, thus contributing to the further development of gesture recognition in diverse interaction scenarios.

关键词： computer vision Conferences Computational modeling Gesture recognition Mathematical models Encoding

Exploring Augmented Reality's Influence on Cognitive Load and Emotional Dynamics within UAV Training Environments

学校读者我要写书评

暂无评论

IEEE Transactions on Learning Technologies 2025年

作者： Rahimi, Fatema Sadeghi-Niaraki, Abolghasem Song, Houbing Wang, Huihui Choi, Soo-Mi Sejong University Department of Computer Science and Engineering Convergence Engineering for Intelligent Drone XR Research Center Seoul Korea Republic of Department of Information Systems BaltimoreMD21250 United States St. Bonaventure University Cybersecurity Program St. BonaventureNY14778 United States

This study investigates the cognitive and emotional processes involved in augmented reality (AR)-based learning. The study looks at learning outcomes, emotional responses, meditation, and attention using a comprehensive approach that includes self-assessment, EEG data gathering, and post-experiment questionnaires. Twelve participants, selected based on their English proficiency and lack of prior knowledge of the course material, engaged in AR-based learning, while a baseline reading condition was included to contextualize cognitive and emotional engagement. The study findings indicate that the AR group's participants demonstrated notably elevated attention and meditation levels, indicating heightened engagement and focus that is advantageous for efficient assimilation and retention of knowledge. Furthermore, AR learners reported feeling less tired and exhausted, which may have mitigated the negative emotional states that are frequently connected to learning activities. However, no significant differences in negative emotions were observed between the reading and AR groups. These results emphasize the value of customized AR environments for education goals and the need for more study to maximize learning outcomes and affective experiences in AR learning contexts. © 2008-2011 IEEE.

关键词： Engineering education

Multimodal Temporal Modeling of Emotion using Physiological Signals

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2023年

作者： Hinduja, Saurabh Bilalpur, Maneesh Jivnani, Liza Vimalkumar Canavan, Shaun Dept. of Psychology University of Pittsburgh United States Dept. of Intelligent Systems Program University of Pittsburgh United States Dept. of Computer Science University of South Florida United States

Modeling of human emotion is a challenging problem that can require multiple signals types, as well as contextual information that has been obtained over time. Considering this, in this paper we present our approach, based on physiological signals, to the Emotion Physiology and Experience Collaboration (EPIC) challenge at Affective Computing & intelligent Interaction (ACII), 2023. In total there are four scenarios that we model: 1) across time;2) across subjects;3) across elicitor;and 4) across version. To tackle this challenge, we propose to use a physiological fusion-based approach to solve each scenario. Along with this, we give a detailed analysis of the evaluated physiological signals and personalized predictions for each subject are shown. Our proposed approach shows encouraging results with the lowest root mean square error achieved for scenario 4 (across version) for both valence and arousal on the challenge test set. © 2023, CC BY.

关键词： Physiology

A Multimodal Recaptioning Framework to Account for Perceptual Diversity in Multilingual Vision-Language Modeling

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Buettner, Kyle Emmerson, Jacob Kovashka, Adriana Intelligent Systems Program Department of Computer Science University of Pittsburgh United States

There are many ways to describe, name, and group objects when captioning an image. Differences are evident when speakers come from diverse cultures due to the unique experiences that shape perception. Machine translation of captions has pushed multilingual capabilities in vision-language models (VLMs), but data comes mainly from English speakers, indicating a perceptual bias and lack of model flexibility. In this work, we address this challenge and outline a data-efficient framework to instill multilingual VLMs with greater understanding of perceptual diversity. We specifically propose an LLM-based, multimodal recaptioning strategy that alters the object descriptions of English captions before translation. The greatest benefits are demonstrated in a targeted multimodal mechanism guided by native speaker data. By adding produced rewrites as augmentations in training, we improve on German and Japanese text-image retrieval cases studies (up to +3.5 mean recall overall, +4.7 on non-native error cases). We further propose a mechanism to analyze the specific object description differences across datasets, and we offer insights into cross-dataset and cross-language generalization. Copyright © 2025, The Authors. All rights reserved.

关键词： Computational grammars

Notes on the pursuit-evasion games between unmanned aerial vehicles operating in uncertain environments

学校读者我要写书评

暂无评论

Notes on the pursuit-evasion games between unmanned aerial v...

2021 International Conference Engineering and Telecommunication, En and T 2021

作者： Khachumov, Mikhail Khachumov, Vyacheslav Systems Institute of RAS Intelligent Control Laboratory Program Pereslavl-Zalessky Russia RUDN University Federal Research Center 'Computer Science and Control' of RAS Information Technologies Department Moscow Russia

ISBN: (纸本)9781728183909

We solve the planar case of the urgent pursuit-evasion problem for players using various strategies of motion. The following options are considered: players perform piecewise-linear motions;the pursuer moves in a straight line, and the evader moves in a circle;both players move along circular arcs. We propose schemes to solve the formulated problem based on the construction of Apollonius circles, which are geometric invariants. As participants of the game, we assume unmanned aerial vehicles (UAVs) operating in a non-deterministic environment. To solve the problem under uncertainty we use the theory of intelligent geometric control aimed at the joint application of complementary methods of precise geometric and flexible intelligent control as parts of one robotic system. Experimental studies of pursuit-evasion games are carried out in the MATLAB system using mathematical models of UAVs motion and wind loads. © 2021 IEEE.

关键词： intelligent control

PAC Bayesian performance guarantees for deep (Stochastic) networks in medical imaging

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Sicilia, Anthony Zhao, Xingchen Sosnovskikh, Anastasia Hwang, Seong Jae Intelligent Systems Program University of Pittsburgh Department of Computer Science University of Pittsburgh

Application of deep neural networks to medical imaging tasks has in some sense become commonplace. Still, a "thorn in the side" of the deep learning movement is the argument that deep networks are prone to overfitting and are thus unable to generalize well when datasets are small (as is common in medical imaging tasks). One way to bolster confidence is to provide mathematical guarantees, or bounds, on network performance after training which explicitly quantify the possibility of overfitting. In this work, we explore recent advances using the PAC-Bayesian framework to provide bounds on generalization error for large (stochastic) networks. While previous efforts focus on classification in larger natural image datasets (e.g., MNIST and CIFAR-10), we apply these techniques to both classification and segmentation in a smaller medical imagining dataset: the ISIC 2018 challenge set. We observe the resultant bounds are competitive compared to a simpler baseline, while also being more explainable and alleviating the need for holdout sets. © 2021, CC BY.

关键词： Medical imaging

Improvement of Spatial Ambiguity in Multi-Channel Speech Separation Using Channel Attention

学校读者我要写书评

暂无评论

Improvement of Spatial Ambiguity in Multi-Channel Speech Sep...

2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2021

作者： Hong, Qian-Bei Wu, Chung-Hsien Nguyen, Thanh Binh Wang, Hsin-Min National Cheng Kung University and Academia Sinica Graduate Program of Multimedia Systems and Intelligent Computing Tainan Taiwan National Cheng Kung University Department of Computer Science and Information Engineering Tainan Taiwan

ISBN: (纸本)9789881476890

Multi-channel speech separation has been successfully applied in a complex real-world environment such as the far-field condition. The common solution to deal with the far-field condition is using a multi-channel signal captured by a structured microphone array and leveraging the inner difference between channels to enhance the speech separation performance. The spatial feature has been widely used in recent speech separation research. This feature appears to be insufficient when the location information becomes ambiguous. This is known as the spatial ambiguity problem. In order to deal with the spatial ambiguity problem, this study proposes an attention mechanism for the Temporal-Spatial Neural Filter (TSNF), in which the channel attention on merged features and the feature map of 1D convolution block in the temporal convolution network is proposed. The proposed method is evaluated on the multi-channel reverberant dataset which is built based on the WSJ0-2mix dataset. The dataset is simulated in the real-environment room by using the Room Impulse Response generator. In the experimental results, the proposed methods produced the SI -SNR improvement of about 1.2dB in close speakers' case, while a small decrease of 0.1dB in other cases. © 2021 APSIPA.

关键词： Convolution

Explaining Legal Concepts with Augmented Large Language Models (GPT-4)

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Savelka, Jaromir Ashley, Kevin D. Gray, Morgan A. Westermann, Hannes Xu, Huihui Computer Science Department Carnegie Mellon University Pittsburgh United States Intelligent Systems Program University of Pittsburgh Pittsburgh United States Cyberjustice Laboratory Faculté de droit Université de Montréal Québec Canada

Interpreting the meaning of legal open-textured terms is a key task of legal professionals. An important source for this interpretation is how the term was applied in previous court cases. In this paper, we evaluate the performance of GPT-4 in generating factually accurate, clear and relevant explanations of terms in legislation. We compare the performance of a baseline setup, where GPT-4 is directly asked to explain a legal term, to an augmented approach, where a legal information retrieval module is used to provide relevant context to the model, in the form of sentences from case law. We found that the direct application of GPT-4 yields explanations that appear to be of very high quality on their surface. However, detailed analysis uncovered limitations in terms of the factual accuracy of the explanations. Further, we found that the augmentation leads to improved quality, and appears to eliminate the issue of hallucination, where models invent incorrect statements. These findings open the door to the building of systems that can autonomously retrieve relevant sentences from case law and condense them into a useful explanation for legal scholars, educators or practicing lawyers alike. © 2023, CC BY.

关键词： Laws and legislation

The Change that Matters in Discourse Parsing: Estimating the Impact of Domain Shift on Parser Error

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Atwell, Katherine Siciliaz, Anthony Hwang, Seong Jae Alikhani, Malihe Department Of Computer Science University Of Pittsburgh United States Intelligent Systems Program University Of Pittsburgh United States Department Of Artificial Intelligence Yonsei University Korea Republic of

Discourse analysis allows us to attain inferences of a text document that extend beyond the sentence-level. The current performance of discourse models is very low on texts outside of the training distribution's coverage, diminishing the practical utility of existing models. There is need for a measure that can inform us to what extent our model generalizes from the training to the test sample when these samples may be drawn from distinct distributions. While this can be estimated via distribution shift, we argue that this does not directly correlate with change in the observed error of a classifier (i.e. error-gap). Thus, we propose to use a statistic from the theoretical domain adaptation literature which can be directly tied to error-gap. We study the bias of this statistic as an estimator of error-gap both theoretically and through a large-scale empirical study of over 2400 experiments on 6 discourse datasets from domains including, but not limited to: news, biomedical texts, TED talks, Reddit posts, and fiction. Our results not only motivate our proposal and help us to understand its limitations, but also provide insight on the properties of discourse models and datasets which improve performance in domain adaptation. For instance, we find that non-news datasets are slightly easier to transfer to than news datasets when the training and test sets are very different. Our code and an associated Python package are available to allow practitioners to make more informed model and dataset choices. © 2022, CC BY.

关键词： Errors