检索结果-内蒙古大学图书馆

How Good is Google Bard's Visual Understanding? An Empirical Study on Open Challenges

Machine intelligence research 2023年第5期20卷 605-613页

作者： Haotong Qin Ge-Peng Ji Salman Khan Deng-Ping Fan Fahad Shahbaz Khan Luc Van Gool Computer Vision Lab(CVL) ETH ZurichZurich 8001Switzerland College of Engineering Computing&CyberneticsAustralian National UniversityCanberra 8105Australia Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi 999041UAE

Google's Bard has emerged as a formidable competitor to OpenAI's ChatGPT in the field of conversational ***,Bard has recently been updated to handle visual inputs alongside text prompts during *** Bard's impressive track record in handling textual inputs,we explore its capabilities in understanding and interpreting visual data(images)conditioned by text *** exploration holds the potential to unveil new insights and challenges for Bard and other forthcoming multi-modal Generative models,especially in addressing complex computer vision problems that demand accurate visual and language ***,in this study,we focus on 15 diverse task scenarios encompassing regular,camouflaged,medical,under-water and remote sensing data to comprehensively evaluate Bard's *** primary finding indicates that Bard still struggles in these vision scenarios,highlighting the significant gap in vision-based understanding that needs to be bridged in future *** expect that this empirical study will prove valuable in advancing future models,leading to enhanced capabilities in comprehending and interpreting finegrained visual *** project is released on https://***/htqin/GoogleBard-VisUnderstand.

关键词： Google Bard multi-modal understanding visual comprehension large language models conversational AI chatbot.

来源：评论

学校读者我要写书评

暂无评论

Development of a Hand Gesture Recognition Model Capable of Online Readjustment Using EMGs and Double Deep-Q Networks 1

引用

International Conference on Information Technology and Systems, ICITS 2023

作者： Díaz, Danny Benalcázar, Marco E. Barona, Lorena Valdivieso, Ángel Leonardo Artificial Intelligence and Computer Vision Research Lab Departamento de Informática y Ciencias de la Computación Escuela Politécnica Nacional Quito Ecuador

ISBN: (数字)9783031332586

ISBN: (纸本)9783031332579

Hand Gesture Recognition (HGR) has enabled the development of alternative forms of human-machine interaction in recent years. HGR models based on supervised learning have been developed with high accuracy. However, over time, the need to add new gestures appears. Consequently, it is necessary to establish a model to adapt to this need. Reinforcement learning permits developing agents capable of adapting their behavior to dynamic environments. In this work, we use Double Deep-Q Networks (DDQN), a reinforcement learning algorithm, to build an agent (an HGR model in this case), based on electromyography signals (EMGs), capable of recognizing new gestures over time. The proposed model is able to recognize 5 hand gestures, achieving an accuracy of 97.36% for classification and 94.83% for recognition. We achieved this result due to a modification in the experience replay, the post-processing of the algorithm, and its Markov Decision Process definition. This model can recalibrate its accuracy over time by adapting to a dynamic environment, while maintaining high performance. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Face to Cartoon Incremental Super-Resolution Using Knowledge Distillation 27th

Face to Cartoon Incremental Super-Resolution Using Knowledg...

引用

27th International Conference on Pattern Recognition, ICPR 2024

作者： Devkatte, Trinetra Dubey, Shiv Ram Singh, Satish Kumar Hadid, Abdenour Computer Vision and Biometrics Lab Indian Institute of Information Technology Allahabad Allahabad India Sorbonne Centre for Artificial Intelligence Sorbonne University Abu Dhabi Abu Dhabi United Arab Emirates

ISBN: (纸本)9783031781827

Facial super-resolution/hallucination is an important area of research that seeks to enhance low-resolution facial images for a variety of applications. While Generative Adversarial Networks (GANs) have shown promise in this area, their ability to adapt to new, unseen data remains a challenge. This paper addresses this problem by proposing an incremental super-resolution using GANs with knowledge distillation (ISR-KD) for face to cartoon. Previous research in this area has not investigated incremental learning, which is critical for real-world applications where new data is continually being generated. The proposed ISR-KD aims to develop a novel unified framework for facial super-resolution that can handle different settings, including different types of faces such as cartoon face and various levels of detail. To achieve this, a GAN-based super-resolution network was pre-trained on the CelebA dataset and then incrementally trained on the iCartoonFace dataset, using knowledge distillation to retain performance on the CelebA test set while improving the performance on iCartoonFace test set. Our experiments demonstrate the effectiveness of knowledge distillation in incrementally adding capability to the model for cartoon face super-resolution while retaining the learned knowledge for facial hallucination tasks in GANs. The code is released at https://***/TRINETRA-DEVKATTE/ISR-KD-Network. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

ClickDiff: Click to Induce Semantic Contact Map for Controllable Grasp Generation with Diffusion Models 24

ClickDiff: Click to Induce Semantic Contact Map for Controll...

引用

32nd ACM International Conference on Multimedia, MM 2024

作者： Li, Peiming Wang, Ziyi Liu, Mengyuan Liu, Hong Chen, Chen State Key Laboratory of General Artificial Intelligence Peking University Shenzhen Graduate School Shenzhen China Center for Research in Computer Vision University of Central Florida Orlando United States

ISBN: (纸本)9798400706868

Grasp generation aims to create complex hand-object interactions with a specified object. While traditional approaches for hand generation have primarily focused on visibility and diversity under scene constraints, they tend to overlook the fine-grained hand-object interactions such as contacts, resulting in inaccurate and undesired grasps. To address these challenges, we propose a controllable grasp generation task and introduce ClickDiff, a controllable conditional generation model that leverages a fine-grained Semantic Contact Map (SCM). Particularly when synthesizing interactive grasps, the method enables the precise control of grasp synthesis through either user-specified or algorithmically predicted Semantic Contact Map. Specifically, to optimally utilize contact supervision constraints and to accurately model the complex physical structure of hands, we propose a Dual Generation Framework. Within this framework, the Semantic Conditional Module generates reasonable contact maps based on fine-grained contact information, while the Contact Conditional Module utilizes contact maps alongside object point clouds to generate realistic grasps. We evaluate the evaluation criteria applicable to controllable grasp generation. Both unimanual and bimanual generation experiments on GRAB and ARCTIC datasets verify the validity of our proposed method, demonstrating the efficacy and robustness of ClickDiff, even with previously unseen objects. Our code is available at https://***/adventurer-w/ClickDiff. © 2024 ACM.

关键词： controllable grasp generation

来源：评论

学校读者我要写书评

暂无评论

Explaining Deep Learning Time Series Classification Models using a Decision Tree-Based Post-Hoc XAI Method 1

Explaining Deep Learning Time Series Classification Models u...

引用

Joint 1st World Conference on eXplainable artificial intelligence: Late-Breaking Work, Demos and Doctoral Consortium, xAI-2023: LB-D-DC

作者： Mekonnen, Ephrem T. Dondio, Pierpaolo Longo, Luca Artificial Intelligence and Cognitive Load Research Lab Ireland School of Computer Science Technological University Dublin Ireland

This preliminary study proposes a new post hoc method to explain deep learning-based time series classification models using a decision tree. Our approach generates a decision tree graph or rulesets as an explanation, improving interpretability compared to saliency map-based methods. The method involves two phases: training and evaluating the deep learning-based time series classification model and extracting prototypical events from the evaluation set to train the decision tree classifier. We conducted experiments on artificial and real datasets, evaluating the explanations based on accuracy, fidelity, number of nodes, and depth. Our preliminary findings suggest that our post-hoc method improves the interpretability and trust of complex time series classification models. © 2023 CEUR-WS. All rights reserved.

关键词： Decision trees

来源：评论

学校读者我要写书评

暂无评论

An examination of the effect of the inconsistency budget in weighted argumentation frameworks and their impact on the interpretation of deep neural networks 1

An examination of the effect of the inconsistency budget in ...

引用

Joint 1st World Conference on eXplainable artificial intelligence: Late-Breaking Work, Demos and Doctoral Consortium, xAI-2023: LB-D-DC

作者： Vilone, Giulia Longo, Luca Artificial Intelligence and Cognitive Load Research Lab Ireland School of Computer Science Technological University Dublin Ireland

Explaining the logic of a data-driven Machine Learning (ML) model can be seen as a defeasible reasoning process that is likely non-monotonic. This means a conclusion linked to a set of premises can be withdrawn when new information becomes available. Argumentation Theory (AT) formalises reasoning with a defeasible knowledge base. Abstract Argumentation Frameworks (AAF) organise conflicting arguments in a dialogical structure, allowing formal semantics to resolve conflicts. This study proposes an XAI method for automatically forming an AAF-based representation, using weighted attacks to model conflictual information. The concept of inconsistency budget is employed to eliminate the weakest attacks. Findings showed that the variation of the inconsistency budget could affect, albeit limited, the evaluation metrics computed over the resulting rulesets. © 2023 CEUR-WS. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Optimizing Deep Q-Learning Experience Replay with SHAP Explanations: Exploring Minimum Experience Replay Buffer Sizes in Reinforcement Learning 1

Optimizing Deep Q-Learning Experience Replay with SHAP Expla...

引用

Joint 1st World Conference on eXplainable artificial intelligence: Late-Breaking Work, Demos and Doctoral Consortium, xAI-2023: LB-D-DC

作者： Sullivan, Robert S. Longo, Luca Artificial Intelligence and Cognitive Load Research Lab Ireland School of Computer Science Technological University Dublin Ireland

Explainable Reinforcement Learning (xRL) faces challenges in debugging and interpreting Deep Reinforcement Learning (DRL) models. A lack of understanding for internal components like Experience Replay, which samples and stores data from the environment, risks burdening resources. This paper presents an xRL-based Deep Q-Learning (DQL) system using SHAP (SHapley Additive exPlanations) to explain input feature contributions. Data is sampled from Experience Replay, creating SHAP Heatmaps to understand how it influences the neural network Q-value approximator's actions. The xRL-based system aids in determining the smallest Experience Replay size for 23 simulations of varying complexities. It contributes an xRL optimization method, alongside traditional approaches, for tuning the Experience Replay size hyperparameter. This visual and creative approach achieves over 40% reduction in Experience Replay size for 18 of the 23 tested simulations, smaller than the commonly used sizes of 1 million transitions or 90% of total environment transitions. © 2023 CEUR-WS. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Latent space interpretation and visualisation for understanding the decisions of convolutional variational autoencoders trained with EEG topographic maps 1

Latent space interpretation and visualisation for understand...

引用

Joint 1st World Conference on eXplainable artificial intelligence: Late-Breaking Work, Demos and Doctoral Consortium, xAI-2023: LB-D-DC

作者： Ahmed, Taufique Longo, Luca Artificial Intelligence and Cognitive Load Research Lab Ireland School of Computer Science Technological University Dublin Ireland

Learning essential features and forming simple representations of electroencephalography (EEG) signals are difficult problems. Variational autoencoders (VAEs) can be used with EEG signals to learn the salient features of EEG data. But explainability should disclose knowledge of how the model makes its decision. The key contribution of this research is the combining of known components in a pipeline that allows us to give meaningful visualisations that help us understand which component of latent space is responsible for capturing which region of brain activation in EEG topographic maps. The results reveal that each component in the latent space contributes to capturing at least two generating factors in topographic maps. This pipeline can be used to produce EEG topographic maps of any scale. Furthermore, assist us in understanding each component of latent space responsible for activating a portion of the brain. © 2023 CEUR-WS. All rights reserved.

关键词： Electroencephalography

来源：评论

学校读者我要写书评

暂无评论

Multi-Modality Co-Learning for Efficient Skeleton-based Action Recognition 24

Multi-Modality Co-Learning for Efficient Skeleton-based Acti...

引用

32nd ACM International Conference on Multimedia, MM 2024

作者： Liu, Jinfu Chen, Chen Liu, Mengyuan State Key Laboratory of General Artificial Intelligence Peking University Shenzhen Graduate School Shenzhen China Center for Research in Computer Vision University of Central Florida OrlandoFL United States

ISBN: (纸本)9798400706868

Skeleton-based action recognition has garnered significant attention due to the utilization of concise and resilient skeletons. Nevertheless, the absence of detailed body information in skeletons restricts performance, while other multimodal methods require substantial inference resources and are inefficient when using multimodal data during both training and inference stages. To address this and fully harness the complementary multimodal features, we propose a novel multi-modality co-learning (MMCL) framework by leveraging the multimodal large language models (LLMs) as auxiliary networks for efficient skeleton-based action recognition, which engages in multi-modality co-learning during the training stage and keeps efficiency by employing only concise skeletons in inference. Our MMCL framework primarily consists of two modules. First, the Feature Alignment Module (FAM) extracts rich RGB features from video frames and aligns them with global skeleton features via contrastive learning. Second, the Feature Refinement Module (FRM) uses RGB images with temporal information and text instruction to generate instructive features based on the powerful generalization of multimodal LLMs. These instructive text features will further refine the classification scores and the refined scores will enhance the model's robustness and generalization in a manner similar to soft labels. Extensive experiments on NTU RGB+D, NTU RGB+D 120 and Northwestern-UCLA benchmarks consistently verify the effectiveness of our MMCL, which outperforms the existing skeleton-based action recognition methods. Meanwhile, experiments on UTD-MHAD and SYSU-Action datasets demonstrate the commendable generalization of our MMCL in zero-shot and domain-adaptive action recognition. Our code is publicly available at: https://***/liujf69/MMCL-Action. © 2024 ACM.

关键词： Musculoskeletal system

来源：评论

学校读者我要写书评

暂无评论

Interpreting Black-Box Time Series Classifiers using Parameterised Event Primitives 2

Interpreting Black-Box Time Series Classifiers using Paramet...

引用

Joint of the 2nd World Conference on eXplainable artificial intelligence Late-Breaking Work, Demos and Doctoral Consortium, xAI-2024:LB/D/DC

作者： Mekonnen, Ephrem T. Longo, Luca Dondio, Pierpaolo School of Computer Science Technological University Dublin Ireland Artificial Intelligence and Cognitive Load Research Lab. Technological University Dublin Ireland

Amidst the remarkable performance of deep learning models in time series classification, there is a pressing demand for methods that unveil their prediction rationale. Existing feature importance techniques often neglect the temporal nature of time series data, focusing solely on segment importance. Addressing this gap, this paper introduces a local model-agnostic method akin to LIME, which generates neighbouring samples by randomly perturbing segments of the original instance. Subsequently, weights are computed for each neighbouring instance based on its distance from the original, elucidating its influence. Parameterised event primitives (PEPs) are then extracted from these perturbed samples, encompassing increasing and decreasing events and local maxima and minima points. These primitives are clustered to form prototypical events that capture the temporal essence of the data. Leveraging these events, computed weights, and black box predictions, a simple linear regression model is trained to provide local explanations. Preliminary experiments on real-world datasets showcase the method’s efficacy in identifying salient subsequences and points and their importance scores, thereby enhancing comprehension of produced explanations. © 2022 Copyright for this paper by its authors.

关键词： Spatio-temporal data

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：