检索结果-内蒙古大学图书馆

Enhancing Visual Reasoning with LLM-Powered Knowledge Graphs for Visual Question Localized-Answering in Robotic Surgery

引用

IEEE Journal of Biomedical and Health informatics 2025年 PP卷 PP页

作者： Hao, Pengfei Wang, Hongqiu Yang, Guang Zhu, Lei Hong Kong University of Science and Technology ROAS Thrust Systems Hub Guangzhou China Imperial College London Bioengineering/Imperial-X London United Kingdom Guangdong Guangzhou511400 China Hong Kong University of Science and Technology Henan Key Laboratory of Imaging and Intelligent Processing Department of Electronic and Computer Engineering SAR Hong Kong Hong Kong

Expert surgeons often have heavy workloads and cannot promptly respond to queries from medical students and junior doctors about surgical procedures. Thus, research on Visual Question Localized-Answering in Surgery (Surgical-VQLA) is essential to assist medical students and junior doctors in understanding surgical scenarios. Surgical-VQLA aims to generate accurate answers and locate relevant areas in the surgical scene, requiring models to identify and understand surgical instruments, operative organs, and procedures. A key issue is the model's ability to accurately distinguish surgical instruments. Current Surgical-VQLA models rely primarily on sparse textual information, limiting their visual reasoning capabilities. To address this issue, we propose a framework called Enhancing Visual Reasoning with LLM-Powered Knowledge Graphs (EnVR-LPKG) for the Surgical-VQLA task. This framework enhances the model's understanding of the surgical scenario by utilizing knowledge graphs of surgical instruments constructed by the Large Language Model (LLM). Specifically, we design a Fine-grained Knowledge Extractor (FKE) to extract the most relevant information from knowledge graphs and perform contrastive learning with the extracted knowledge graphs and local image. Furthermore, we design a Multi-attention-based Surgical Instrument Enhancer (MSIE) module, which employs knowledge graphs to obtain an enhanced representation of the corresponding surgical instrument in the global scene. Through the MSIE module, the model can learn how to fuse visual features with knowledge graph text features, thereby strengthening the understanding of surgical instruments and further improving visual reasoning capabilities. Extensive experimental results on the EndoVis-17-VQLA and EndoVis-18-VQLA datasets demonstrate that our proposed method outperforms other state-of-the-art methods. We will release our code for future research. © 2013 IEEE.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

Results of the Ontology Alignment Evaluation Initiative 2024 19

Results of the Ontology Alignment Evaluation Initiative 2024

引用

19th International Workshop on Ontology Matching, OM 2024

作者： Pour, Mina Abd Nikooie Algergawy, Alsayed Blomqvist, Eva Buche, Patrice Chen, Jiaoyan Cotovio, Pedro Giesteira Coulet, Adrien Cufi, Julien Dong, Hang Faria, Daniel Ferraz, Lucas Hertling, Sven He, Yuan Horrocks, Ian Ibanescu, Liliana Jain, Sarika Jiménez-Ruiz, Ernesto Karam, Naouel Kraus, Felix Lambrix, Patrick Li, Huanyu Li, Ying Monnin, Pierre Paulheim, Heiko Pesquita, Catia Sharma, Abhisek Shvaiko, Pavel Silva, Marta Sousa, Guilherme Trojahn, Cassia Vataščinová, Jana Yaman, Beyza Zamazal, Ondřej Zhou, Lu Department of Computer and Information Science Linköping University Linköping Sweden Swedish e-Science Research Centre Linköping Sweden Friedrich Schiller University Jena Germany University of Passau Germany UMR IATE INRAE University of Montpellier France Department of Computer Science The University of Manchester United Kingdom City St George's University of London United Kingdom University of Oslo Norway LASIGE Faculdade de Ciências Universidade de Lisboa Portugal Inria Paris France Centre de Recherche des Cordeliers Inserm Université Paris Cité Sorbonne Université France Department of Computer Science University of Exeter United Kingdom INESC-ID/IST University of Lisbon Portugal Data and Web Science Group University of Mannheim Germany Department of Computer Science University of Oxford United Kingdom Université Paris-Saclay INRAE AgroParisTech UMR MIA Paris-Saclay France National Institute of Technology Kurukshetra Haryana India Institute for Applied Informatics University of Leipzig Germany Karlsruhe Institute of Technology Karlsruhe Germany Université Côte d'Azur Inria CNRS I3S Sophia Antipolis France Trentino Digitale SpA Trento Italy Institut de Recherche en Informatique de Toulouse France Univ. Grenoble Alpes Inria CNRS Grenoble INP LIG GrenobleF-38000 France Prague University of Economics and Business Czech Republic ADAPT Centre Trinity College Dublin Ireland Flatfee Corp United States

The Ontology Alignment Evaluation Initiative (OAEI) aims at comparing ontology matching systems on precisely defined test cases. These test cases can be based on ontologies of different levels of complexity and use different evaluation modalities. The OAEI 2024 campaign offered 13 tracks and was attended by 13 participants. This paper is an overall presentation of that campaign. © 2024 Copyright for this paper by its authors.

关键词： Ontology

来源：评论

学校读者我要写书评

暂无评论

End-to-end Semantic Segmentation Network for Low-Light Scenes

End-to-end Semantic Segmentation Network for Low-Light Scene...

引用

IEEE International Conference on Robotics and Automation (ICRA)

作者： Hongmin Mu Gang Zhang MengChu Zhou Zhengcai Cao College of Information Science and Technology Beijing University of Chemical Technology Beijing China Department of Macao Institute of Systems Engineering Macau University of Science and Technology Macao China Helen and John C. Hartmann Department of Electrical and Computer Engineering New Jersey Institute of Technology Newark NJ USA State Key Laboratory of Robotics and Systems Harbin Institute of Technology Harbin China

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

In the fields of robotic perception and computer vision, achieving accurate semantic segmentation of low-light or nighttime scenes is challenging. This is primarily due to the limited visibility of objects and the reduced texture and color contrasts among them. To address the issue of limited visibility, we propose a hierarchical gated convolution unit, which simultaneously expands the receptive field and restores edge texture. To address the issue of reduced texture among objects, we propose a dual closed-loop bipartite matching algorithm to establish a total loss function consisting of the unsupervised illumination enhancement loss and supervised intersection-over-union loss, thus enabling the joint minimization of both losses via the Hungarian algorithm. We thus achieve end-to-end training for a semantic segmentation network especially suitable for handling low-light scenes. Experimental results demonstrate that the proposed network surpasses existing methods on the Cityscapes dataset and notably outperforms state-of-the-art methods on both Dark Zurich and Nighttime Driving datasets.

关键词： Training computer vision Accuracy Convolution Semantic segmentation Lighting Logic gates

来源：评论

学校读者我要写书评

暂无评论

Optimal AoI-based Block Propagation and Incentive Mechanism for Blockchain Networks in Web 3.0

引用

IEEE Transactions on Cognitive Communications and Networking 2024年

作者： Wen, Jinbo Kang, Jiawen Xiong, Zehui Du, Hongyang Yang, Zhaohui Niyato, Dusit Shen, Meng Jiao, Yutao Zhang, Yang Nanjing University of Aeronautics and Astronautics College of Computer Science and Technology China Guangdong University of Technology School of Automation China Singapore University of Technology and Design Pillar of Information Systems Technology and Design Singapore Hong Kong University Department of Electrical and Electronic Engineering Hong Kong Zhejiang University College of Information Science and Electronic Engineering China Nanyang Technological University College of Computing and Data Science Singapore Beijing Institute of Technology School of Cyberspace Science and Technology China Army Engineering University of PLA College of Communications Engineering China

Web 3.0 is regarded as a revolutionary paradigm that enables users to securely manage data without a centralized authority. Blockchains, which enable data to be managed in a decentralized and transparent manner, are key technologies for achieving Web 3.0 goals. However, blockchain-based Web 3.0 is still in its infancy, such as ensuring block freshness and optimizing block propagation to improve blockchain performance. In this paper, we develop a freshness-aware block propagation optimization framework for Web 3.0. We first propose a novel metric called Age of Block Information (AoBI) based on the concept of age of information to quantify block freshness. AoBI measures the time elapsed from the freshest transaction generation to the completion of block consensus. To make block propagation optimization tractable, we classify miners into five different states and propose a block propagation model for public blockchains inspired by epidemic models. Moreover, considering that the miners are bounded rational, we propose an incentive mechanism based on the evolutionary game for block propagation to improve block propagation efficiency. Numerical results demonstrate that compared with other block propagation mechanisms in public blockchains, the proposed scheme has a higher block forwarding probability, which improves block propagation efficiency and decreases the minimum value of average AoBI. © 2015 IEEE.

关键词： age of information block propagation evolutionary game Web 3.0 wireless networks

来源：评论

学校读者我要写书评

暂无评论

What is the Visual Cognition Gap between Humans and Multimodal LLMs?

arXiv

引用

arXiv 2024年

作者： Cao, Xu Lai, Bolin Ye, Wenqian Ma, Yunsheng Heintz, George Chen, Jintai Cao, Jianguo Rehg, James M. Health Care Engineering Systems Center University of Illinois Urbana-Champaign United States College of Computing Georgia Institute of Technology United States Department of Computer Science University of Virginia United States College of Engineering Purdue University United States Department of Rehabilitation Medicine Shenzhen Children's Hospital China

Recently, Multimodal Large Language Models (MLLMs) have shown great promise in language-guided perceptual tasks such as recognition, segmentation, and object detection. However, their effectiveness in addressing visual cognition problems that require high-level reasoning is not well-established. One such challenge is abstract visual reasoning (AVR) - the cognitive ability to discern relationships among patterns in a set of images and extrapolate to predict subsequent patterns. This skill is crucial during the early neurodevelopmental stages of children. Inspired by the AVR tasks in Raven's Progressive Matrices (RPM) and Wechsler Intelligence Scale for Children (WISC), we propose a new dataset MaRs-VQA and a new benchmark VCog-Bench containing three datasets to evaluate the zero-shot AVR capability of MLLMs and compare their performance with existing human intelligent investigation. Our comparative experiments with different open-source and closed-source MLLMs on the VCog-Bench revealed a gap between MLLMs and human intelligence, highlighting the visual cognitive limitations of current MLLMs. We believe that the public release of VCog-Bench, consisting of MaRs-VQA, and the inference pipeline will drive progress toward the next generation of MLLMs with human-like visual cognition abilities. The code and datasets for our benchmark are at ***/IrohXu/*** Codes 68T01 © 2024, CC BY.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling

arXiv

引用

arXiv 2022年

作者： Cheng, Lei Yin, Feng Theodoridis, Sergios Chatzis, Sotirios Chang, Tsung-Hui College of Information Science and Electronic Engineering Zhejiang University Hangzhou China School of Science and Engineering The Chinese University of Hong Kong Shenzhen China Department of Informatics and Telecommunications National and Kapodistrian University of Athens Greece Department of Electronics Systems Aalborg University Denmark Department of Electrical & Computer Engineering and Informatics Cyprus University of Technology Cyprus Cyprus

Sparse modeling for signal processing and machine learning, in general, has been at the focus of scientific research for over two decades. Among others, supervised sparsity-aware learning comprises two major paths paved by: a) discriminative methods that establish direct input-output mapping based on a regularized cost function optimization, and b) generative methods that learn the underlying distributions. The latter, more widely known as Bayesian methods, enable uncertainty evaluation with respect to the performed predictions. Furthermore, they can better exploit related prior information and also, in principle, can naturally introduce robustness into the model, due to their unique capacity to marginalize out uncertainties related to the parameter estimates. Moreover, hyper-parameters (tuning parameters) associated with the adopted priors, which correspond to cost function regularizers, can be learnt via the training data and not via costly cross-validation techniques, which is, in general, the case with the discriminative methods. To implement sparsity-aware learning, the crucial point lies in the choice of the function regularizer for discriminative methods and the choice of the prior distribution for Bayesian learning. Over the last decade or so, due to the intense research on deep learning, emphasis has been put on discriminative techniques. However, a come back of Bayesian methods is taking place that sheds new light on the design of deep neural networks, which also establish firm links with Bayesian models, such as Gaussian processes, and, also, inspire new paths for unsupervised learning, such as Bayesian tensor decomposition. The goal of this article is two-fold. First, to review, in a unified way, some recent advances in incorporating sparsity-promoting priors into three highly popular data modeling/analysis tools, namely deep neural networks, Gaussian processes, and tensor decomposition. Second, to review their associated inference techniques from differ

关键词： Bayesian networks

来源：评论

学校读者我要写书评

暂无评论

Implementation of Knowledge based Collaborative Filtering and Machine Learning for E-Commerce Recommendation System 3

Implementation of Knowledge based Collaborative Filtering an...

引用

3rd International Conference on Computational and Experimental Methods in Mechanical Engineering, ICCEMME 2021

作者： Singh, Mahesh Kumar Rishi, Om Prakash Singh, Akhilesh Kumar Singh, Pushpendra Choudhary, Pushpa Department of Information Technology G L Bajaj Institute of Technology and Management UP Greater Noida India Department of Computer Science and Informatics University of Kota Rajasthan Kota India Department of Information Technology Indraprastha Engineering College UP Ghaziabad India

This is the era of I-way. The development of high-speed computing and huge storage devices change the working culture of human. It affects the traditional business processes and shifted towards online business. It creates huge problems like overload and irrelevant information which are the causes of confusion both customers as well as enterprise. Recommendation system solves these problems. Design and development of efficient system is one of the key areas of the recent researchers. Collaborative filtering (CF) and content-based filtering algorithms are widely used in the implementation of such system. Collaborative used user's features while content-based used item's features. Most of the CFS are rating or review based processed homogeneous information. In this paper we proposed knowledge-based collaborative filtering algorithm for large data set that uses various activities done by users during interaction of item through E-commerce web site like clicks, select and purchase. The performance of the system is compared with the base models using real time Amazon E-commerce dataset using precession, recall and NDCG evaluation parameters in various combinations of activities performed by users on items. © 2021 Institute of Physics Publishing. All rights reserved.

关键词： Collaborative filtering

来源：评论

学校读者我要写书评

暂无评论

Detecting Nonsynchronous Heart Cells from Video – An Unsupervised Machine Learning Approach 38th

Detecting Nonsynchronous Heart Cells from Video – An Unsupe...

引用

38th IMAC, A Conference and Exposition on Structural Dynamics, 2020

作者： Anderson, William Schneider, Lauren Yeong, Li-Ming Richard Coombs, Kent Nath, Pulak Harris, Jennifer Mascareñas, David Martinez, Bridget School of Biological and Health Systems Engineering Arizona State University TempeAZ United States Department of Computer Science Maseeh College of Engineering & Computer Science Portland State University PortlandOR United States Department of Mechanical Engineering Bourns College of Engineering University of California RiversideCA United States Los Alamos National Laboratory Los AlamosNM United States

ISBN: (纸本)9783030477165

Cardiac myocytes possess the property of automaticity;however, irregularities in the automaticity can lead to cardiac arrhythmias and heart blocks, which can cause life-threatening health concerns. Currently, full-field, high-resolution, anomaly detection methods for non-contact, electro-mechanical dynamics of cardiac myocytes do not exist. This research uses emerging structural dynamic techniques coupled with cutting edge signal processing methods to examine the dynamics of beating heart cells. Applications of this research could offer novel advances in medical diagnostics for heart disorders. A properly functioning heart will have cells that contract together in one global motion. However, in certain pathologic conditions, one cell –or a collection of cells – will beat out of phase with the rest of the heart. Using a combination of blind source separation techniques (non-negative matrix factorization, sparse and low-rank matrix decompositions) and mutual information on videos, these nonsynchronous cardiac myocytes can be identified. © 2021, The Society for Experimental Mechanics, Inc.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

A Reasoning Based Model for Anomaly Detection in the Smart City Domain

A Reasoning Based Model for Anomaly Detection in the Smart C...

引用

Intelligent systems Conference, IntelliSys 2020

作者： Hammer, Patrick Lofthouse, Tony Fenoglio, Enzo Latapie, Hugo Wang, Pei Department of Computer and Information Sciences College of Science and Technology Temple University PhiladelphiaPA19122 United States Reasoning Systems Ltd. London United Kingdom Cisco Systems Inc. San Jose United States

ISBN: (纸本)9783030551865

Using a visual scene object tracker and a Non-Axiomatic Reasoning System we demonstrate how to predict and detect various anomaly classes. The approach combines an object tracker with a base ontology and the OpenNARS reasoner to learn to classify scene regions based on accumulating evidence from typical entity class (tracked object) behaviours. The system can autonomously satisfy goals related to anomaly detection and respond to user Q&A in real time. The system learns directly from experience with no initial training required (one-shot). The solution is a fusion of neural techniques (object tracker) and a reasoning system (with ontology). © 2021, Springer Nature Switzerland AG.

关键词： Anomaly detection

来源：评论

学校读者我要写书评

暂无评论

Your Eyes Show What Your Eyes See (Y-EYES): Challenge-Response Anti-Spoofing Method for Mobile Security Using Corneal Specular Reflections 1

Your Eyes Show What Your Eyes See (Y-EYES): Challenge-Respon...

引用

1st Workshop on Security and Privacy for Mobile AI, MAISP 2021

作者： Mohzary, Muhammad Almalki, Khalid J. Choi, Baek-Young Song, Sejun School of Computing and Engineering University of Missouri Kansas CityMO United States Department of Computer Science Jazan University Jazan Saudi Arabia College of Computing and Informatics Saudi Electronic University Riyadh Saudi Arabia

ISBN: (纸本)9781450386012

As the need for contactless biometric authentication becomes more significant during COVID-19, and beyond, the popular biometric authentication method for mobile devices, iris detection, and facial recognition confronts various usability, security, and privacy concerns, including mask-wearing and various Presentation Attacks (PA). Specifically, liveness detection against spoofed artifacts is one of the most challenging tasks as many existing methods cannot conclusively assess the user's physical presence in unsupervised environments. Even though several methods have been proposed for tackling PA with motion challenges and 3D mapping, most of them require expensive depth sensors and fail to detect sophisticated 3D reconstruction attacks. We present a software-based face PA Detection (PAD) method named "Your Eyes Show What Your Eyes See (Y-EYES),"which creates challenges and detects meaningful corneal specular reflection responses from human eyes. To detect human liveness, Y-EYES creates multiple screen image patterns as a challenge, then captures the response of corneal specular reflections using the front camera and analyzes the images using lightweight Machine Learning (ML) techniques. Y-EYES system components include challenge pattern generation, reflection image augmentation (e.g., super-resolution), and ML-based analyses. We have implemented Y-EYES as Android, iOS, and web apps. Our extensive experimental results show that Y-EYES achieves liveness detection with high accuracy at around 200 ms against various types of sophisticated PA. Y-EYES liveness detection can be applied for multiple contactless biometric authentications accurately and efficiently without any costly extra sensors. © 2021 ACM.

关键词： Mobile security

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：