检索结果-内蒙古大学图书馆

Bidirectional Logits Tree: Pursuing Granularity Reconcilement in Fine-Grained Classification

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lu, Zhiguang Xu, Qianqian Bao, Shilong Yang, Zhiyong Huang, Qingming Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China School of Computer Science and Technology University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management University of Chinese Academy of Sciences China

This paper addresses the challenge of Granularity Competition in fine-grained classification tasks, which arises due to the semantic gap between multi-granularity labels. Existing approaches typically develop independent hierarchy-aware models based on shared features extracted from a common base encoder. However, because coarse-grained levels are inherently easier to learn than finer ones, the base encoder tends to prioritize coarse feature abstractions, which impedes the learning of fine-grained features. To overcome this challenge, we propose a novel framework called the Bidirectional Logits Tree (BiLT) for Granularity Reconcilement. The key idea is to develop classifiers sequentially from the finest to the coarsest granularities, rather than parallelly constructing a set of classifiers based on the same input features. In this setup, the outputs of finer-grained classifiers serve as inputs for coarser-grained ones, facilitating the flow of hierarchical semantic information across different granularities. On top of this, we further introduce an Adaptive Intra-Granularity Difference Learning (AIGDL) approach to uncover subtle semantic differences between classes within the same granularity. Extensive experiments demonstrate the effectiveness of our proposed method. Copyright © 2024, The Authors. All rights reserved.

关键词： Coarse-grained modeling

AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Han, Boyu Xu, Qianqian Yang, Zhiyong Bao, Shilong Wen, Peisong Jiang, Yangbangyan Huang, Qingming Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China School of Computer Science and Tech. University of Chinese Academy of Sciences China Peng Cheng Laboratory China Key Laboratory of Big Data Mining and Knowledge Management CAS China

The Area Under the ROC Curve (AUC) is a well-known metric for evaluating instance-level long-tail learning problems. In the past two decades, many AUC optimization methods have been proposed to improve model performance under long-tail distributions. In this paper, we explore AUC optimization methods in the context of pixel-level long-tail semantic segmentation, a much more complicated scenario. This task introduces two major challenges for AUC optimization techniques. On one hand, AUC optimization in a pixel-level task involves complex coupling across loss terms, with structured inner-image and pairwise inter-image dependencies, complicating theoretical analysis. On the other hand, we find that mini-batch estimation of AUC loss in this case requires a larger batch size, resulting in an unaffordable space complexity. To address these issues, we develop a pixel-level AUC loss function and conduct a dependency-graph-based theoretical analysis of the algorithm’s generalization ability. Additionally, we design a Tail-Classes Memory Bank (T-Memory Bank) to manage the significant memory demand. Finally, comprehensive experiments across various benchmarks confirm the effectiveness of our proposed AUCSeg method. The code is available at https://***/boyuh/AUCSeg. © 2024, CC BY.

关键词： Semantic Segmentation

Automatic Life Event Tree Generation for Older Adults 1

学校读者我要写书评

暂无评论

24th International Conference on Human-Computer Interaction, HCII 2022

作者： Gui, Fang Wu, Xi Hu, Min Yang, Jiaoyun School of Computer Science and Information Engineering Hefei University of Technology Hefei China Key Laboratory of Knowledge Engineering with Big Data of Ministry of Education Hefei University of Technology Hefei China National Smart Eldercare International S&T Cooperation Base Hefei University of Technology Hefei China Laboratory of Affective Computing and Advanced Intelligent Machine Hefei University of Technology Hefei China Intelligent Interconnected Systems Laboratory of Anhui Province Hefei University of Technology Hefei China

ISBN: (数字)9783031179020

ISBN: (纸本)9783031179013

Studies have shown that learning personal stories could help provide individualized eldercare services. However, personal stories are often disordered because of the scattered collection, including informal interviews or daily interactions, which brings difficulties in acquiring valuable information quickly. One solution to this problem is to extract events from personal stories and automatically organize them in chronological order. Events extracted by current methods from social media or news corpus are mainly organized in a linear structure. These works usually focus on the event time and ignore the consistency of event contents when organizing events. This paper aims to organize events into a tree structure based on an event network, with stem nodes representing key event topics and branch nodes representing detailed events. Social workers or caregivers can clarify the life experience of the older adults quickly through the event tree and have a preliminary understanding of them. The experiments show that the event tree generated by our method has a better performance in consistency than current event organization methods. A survey study shows that our method achieves the highest logical coherence for the event tree branches compared with other algorithms. © 2022, Springer Nature Switzerland AG.

关键词： Trees (mathematics)

K-ON: Stacking knowledge On the Head Layer of Large Language Model

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Guo, Lingbing Zhang, Yichi Bo, Zhongpu Chen, Zhuo Sun, Mengshu Zhang, Zhiqiang Zhang, Wen Chen, Huajun College of Computer Science and Technology Zhejiang University China ZJU-Ant Group Joint Lab of Knowledge Graph China Ant Group China School of Software Technology Zhejiang University China Zhejiang Key Laboratory of Big Data Intelligent Computing China

Recent advancements in large language models (LLMs) have significantly improved various natural language processing (NLP) tasks. Typically, LLMs are trained to predict the next token, aligning well with many NLP tasks. However, in knowledge graph (KG) scenarios, entities are the fundamental units and identifying an entity requires at least several tokens. This leads to a granularity mismatch between KGs and natural languages. To address this issue, we propose K-ON, which integrates KG knowledge into the LLM by employing multiple head layers for next k-step prediction. K-ON can not only generate entity-level results in one step, but also enables contrastive loss against entities, which is the most powerful tool in KG representation learning. Experimental results show that K-ON outperforms state-of-the-art methods that incorporate text and even the other modalities. Copyright © 2025, The Authors. All rights reserved.

关键词： knowledge graph

Towards Holistic Analysis of Wireless Capsule Endoscopic Videos: A Taxonomy-Driven Machine Learning Framework for Clinically Comprehensive WCE Frame Level Analysis

学校读者我要写书评

暂无评论

Towards Holistic Analysis of Wireless Capsule Endoscopic Vid...

Electrics and Computer (INTCEC), Interdisciplinary Conference on

作者： Haroon Wahab Rita Goel Maida Alamgir Irfan Mehmood Khan Muhammad Hassan Ugail Centre for Visual Computing & Intelligent Systems School of Computer Science AI and Electronics University of Bradford Bradford England UK Department of Applied Artificial Intelligence Visual Analytics Knowledge Laboratory (VIS2KNOW) Sungkyunkwan University Seoul Republic of Korea

ISBN: (数字)9798350349450

ISBN: (纸本)9798350349467

Manual perusal of capsule endoscopic videos is laborious due to their length, redundancy, and poor quality frames, risking misdetection. Limited representation of diverse anomalies and subtle lesions led to less generalized approaches targeting only individual or few anomalous findings. To deploy automated analysis in clinical settings effectively, a holistic, end-to-end approach is crucial, distinguishing between frame-level analysis and disease-level diagnosis. This study proposes a holistic framework for frame-level analysis based on a taxonomy of findings, leveraging clinical consensus. Clinical knowledge implicit in the taxonomy can enhance Computer Aided Diagnostics (CAD) systems. Convolutional Neural Networks (CNNs) trained at appropriate granularity levels in the taxonomy show improved performance metrics. Validation on the Kvasir-capsule dataset demonstrates a 2-4% performance boost. Taxonomy-based inference scores effectively resolve mimicked anomalies without additional learning.

关键词： Training Wireless communication Measurement Taxonomy Redundancy Merging Manuals

MULTIPLE HEADS ARE BETTER THAN ONE: MIXTURE OF MODALITY knowledge EXPERTS FOR ENTITY REPRESENTATION LEARNING

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Yichi Chen, Zhuo Guo, Lingbing Xu, Yajing Hu, Binbin Liu, Ziqi Zhang, Wen Chen, Huajun College of Computer Science and Technology Zhejiang University China ZJU-Ant Group Joint Lab of Knowledge Graph China Ant Group China School of Software Technology Zhejiang University China Zhejiang Key Laboratory of Big Data Intelligent Computing China

Learning high-quality multi-modal entity representations is an important goal of multi-modal knowledge graph (MMKG) representation learning, which can enhance reasoning tasks within the MMKGs, such as MMKG completion (MMKGC). The main challenge is to collaboratively model the structural information concealed in massive triples and the multi-modal features of the entities. Existing methods focus on crafting elegant entity-wise multi-modal fusion strategies, yet they overlook the utilization of multi-perspective features concealed within the modalities under diverse relational contexts. To address this issue, we introduce a novel framework with Mixture of Modality knowledge experts (MOMOK for short) to learn adaptive multi-modal entity representations for better MMKGC. We design relation-guided modality knowledge experts to acquire relation-aware modality embeddings and integrate the predictions from multi-modalities to achieve joint decisions. Additionally, we disentangle the experts by minimizing their mutual information. Experiments on four public MMKG benchmarks demonstrate the outstanding performance of MOMOK under complex scenarios. Our code and data are available at https://***/zjukg/MoMoK. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation

学校读者我要写书评

暂无评论

arXiv 2024年

Multi-modal knowledge graph completion (MMKGC) aims to discover unobserved knowledge from given knowledge graphs, collaboratively leveraging structural information from the triples and multi-modal information of the entities to overcome the inherent incompleteness. Existing MMKGC methods usually extract multi-modal features with pre-trained models, resulting in coarse handling of multi-modal entity information, overlooking the nuanced, fine-grained semantic details and their complex interactions. To tackle this shortfall, we introduce a novel framework MYGO to tokenize, fuse, and augment the fine-grained multi-modal representations of entities and enhance the MMKGC performance. Motivated by the tokenization technology, MYGO tokenizes multi-modal entity information as fine-grained discrete tokens and learns entity representations with a cross-modal entity encoder. To further augment the multi-modal representations, MYGO incorporates fine-grained contrastive learning to highlight the specificity of the entity representations. Experiments on standard MMKGC benchmarks reveal that our method surpasses 19 of the latest models, underlining its superior performance. Code and data can be found in https://***/zjukg/MyGO. Copyright © 2024, The Authors. All rights reserved.

关键词： knowledge graph

Rethink Video Retrieval Representation for Video Captioning

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Tian, Mingkai Li, Guorong Qi, Yuankai Wang, Shuhui Sheng, Quan Z. Huang, Qingming School of Computer Science and Technology Key Lab of Big Data Mining and Knowledge Management University of Chinese Academy of Sciences China School of Computing Macquarie University Australia Key Laboratory of Intelligent Information Processing Institute of Computer Technology Chinese Academy of Sciences China

Video captioning, a challenging task targeting the automatic generation of accurate and comprehensive descriptions based on video content, has witnessed substantial success recently driven by bridging video representations and textual semantics. Inspired by the nature of the video retrieval task, which learns visual features strongly related to text queries, we propose to take advantage of visual representation learning from the video retrieval framework to tackle video captioning tasks. However, a simple direct application of video retrieval models performs poorly due to the weak ability to capture sufficient video details and temporal information required for video captioning. To increase the attention on details, we propose a learnable token shift module, which flexibly captures subtle movements in local regions across the temporal sequence. Furthermore, we devise a Refineformer, which learns to integrate local video tokens strongly related to desired captions via a cross-attention mechanism. Extensive experiments on MSVD, MSR-VTT and VATEX demonstrate the favorable performance of our method. © 2024, The Authors. All rights reserved.

关键词： Semantics

Article reranking by memory-enhanced key sentence matching for detecting previously fact-checked claims 59

学校读者我要写书评

暂无评论

Article reranking by memory-enhanced key sentence matching f...

Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021

作者： Sheng, Qiang Cao, Juan Zhang, Xueyao Li, Xirong Zhong, Lei Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Key Lab of Data Engineering and Knowledge Engineering Renmin University of China China

ISBN: (纸本)9781954085527

False claims that have been previously fact-checked can still spread on social media. To mitigate their continual spread, detecting previously fact-checked claims is indispensable. Given a claim, existing works retrieve fact-checking articles (FC-articles) for detection and focus on reranking candidate articles in the typical two-stage retrieval framework. However, their performance may be limited as they ignore the following characteristics of FC-articles: (1) claims are often quoted to describe the checked events, providing lexical information besides semantics;and (2) sentence templates to introduce or debunk claims are common across articles, providing pattern information. In this paper, we propose a novel reranker, MTM (Memoryenhanced Transformers for Matching), to rank FC-articles using key sentences selected using event (lexical and semantic) and pattern information. For event information, we propose to finetune the Transformer with regression of ROUGE. For pattern information, we generate pattern vectors as a memory bank to match with the parts containing patterns. By fusing event and pattern information, we select key sentences to represent an article and then predict if the article fact-checks the given claim using the claim, key sentences, and patterns. Experiments on two real-world datasets show that MTM outperforms existing methods. Human evaluation proves that MTM can capture key sentences for explanations. The code and the dataset are at https://***/ICTMCG/MTM. © 2021 Association for Computational Linguistics

关键词： Semantics