检索结果-内蒙古大学图书馆

Unifying Large Language Models and knowledge Graphs: A Roadmap

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Pan, Shirui Luo, Linhao Wang, Yufei Chen, Chen Wang, Jiapu Wu, Xindong Griffith University QLD Australia The Department of Data Science and AI Monash University Melbourne Australia The Nanyang Technological University Singapore The Faculty of Information Technology Beijing University of Technology Beijing China The Key Laboratory of Knowledge Engineering with Big Data The Ministry of Education of China Hefei University of Technology Hefei China The Research Center for Knowledge Engineering Zhejiang Lab Hangzhou China

Large language models (LLMs), such as ChatGPT and GPT4, are making new waves in the field of natural language processing and artificial intelligence, due to their emergent ability and generalizability. However, LLMs are black-box models, which often fall short of capturing and accessing factual knowledge. In contrast, knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge. KGs can enhance LLMs by providing external knowledge for inference and interpretability. Meanwhile, KGs are difficult to construct and evolve by nature, which challenges the existing methods in KGs to generate new facts and represent unseen knowledge. Therefore, it is complementary to unify LLMs and KGs together and simultaneously leverage their advantages. In this article, we present a forward-looking roadmap for the unification of LLMs and KGs. Our roadmap consists of three general frameworks, namely, 1) KG-enhanced LLMs, which incorporate KGs during the pre-training and inference phases of LLMs, or for the purpose of enhancing understanding of the knowledge learned by LLMs;2) LLM-augmented KGs, that leverage LLMs for different KG tasks such as embedding, completion, construction, graph-to-text generation, and question answering;and 3) Synergized LLMs + KGs, in which LLMs and KGs play equal roles and work in a mutually beneficial way to enhance both LLMs and KGs for bidirectional reasoning driven by both data and knowledge. We review and summarize existing efforts within these three frameworks in our roadmap and pinpoint their future research directions. Copyright © 2023, The Authors. All rights reserved.

关键词： knowledge graph

Dual encoding for zero-example video retrieval

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Dong, Jianfeng Li, Xirong Xu, Chaoxi Ji, Shouling He, Yuan Yang, Gang Wang, Xun College of Computer and Information Engineering Zhejiang Gongshang University Key Lab of Data Engineering and Knowledge Engineering Renmin University of China AI & Media Computing Lab School of Information Renmin University of China College of Computer Science Zhejiang University Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies Alibaba Group

This paper attacks the challenging problem of zero-example video retrieval. In such a retrieval paradigm, an end user searches for unlabeled videos by ad-hoc queries described in natural language text with no visual example provided. Given videos as sequences of frames and queries as sequences of words, an effective sequence-to-sequence cross-modal matching is required. The majority of existing methods are concept based, extracting relevant concepts from queries and videos and accordingly establishing associations between the two modalities. In contrast, this paper takes a concept-free approach, proposing a dual deep encoding network that encodes videos and queries into powerful dense representations of their own. Dual encoding is conceptually simple, practically effective and end-to-end. As experiments on three benchmarks, i.e. MSR-VTT, TRECVID 2016 and 2017 Ad-hoc Video Search show, the proposed solution establishes a new state-of-the-art for zero-example video retrieval. Copyright © 2018, The Authors. All rights reserved.

关键词： Signal encoding

GPTR: Gestalt-Perception Transformer for Diagram Object Detection

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hu, Xin Zhang, Lingling Liu, Jun Fan, Jinfu You, Yang Wu, Yaqiang Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering School of Computer Science and Technology Xi’an Jiaotong University China National Engineering Lab for Big Data Analytics Xi’an Jiaotong University China Department of Control Science and Engineering Tongji University Shanghai China Department of Computer Science National University of Singapore Singapore Lenovo Research Beijing China

Diagram object detection is the key basis of practical applications such as textbook question answering. Because the diagram mainly consists of simple lines and color blocks, its visual features are sparser than those of natural images. In addition, diagrams usually express diverse knowledge, in which there are many low-frequency object categories in diagrams. These lead to the fact that traditional data-driven detection model is not suitable for diagrams. In this work, we propose a gestalt-perception transformer model for diagram object detection, which is based on an encoder-decoder architecture. Gestalt perception contains a series of laws to explain human perception, that the human visual system tends to perceive patches in an image that are similar, close or connected without abrupt directional changes as a perceptual whole object. Inspired by these thoughts, we build a gestalt-perception graph in transformer encoder, which is composed of diagram patches as nodes and the relationships between patches as edges. This graph aims to group these patches into objects via laws of similarity, proximity, and smoothness implied in these edges, so that the meaningful objects can be effectively detected. The experimental results demonstrate that the proposed GPTR achieves the best results in the diagram object detection task. Our model also obtains comparable results over the competitors in natural image object detection. Copyright © 2022, The Authors. All rights reserved.

关键词： Object detection

Distinguishing re-sharing behaviors from re-creating behaviors in information diffusion

学校读者我要写书评

暂无评论

Distinguishing re-sharing behaviors from re-creating behavio...

International Conference on data engineering Workshops

作者： Yiran Xie Hongzhi Yin Bin Cui Junjie Yao Quanqing Xu Key Lab of High Confidence Software Technologies (MOE) Peking University China The University of Queensland School of Information Technology and Electrical Engineering Australia East China Normal University Data Storage Institute Singapore

ISBN: (纸本)9781479984435

Social media plays a fundamental role in the diffusion of information. There are two different ways of information diffusion in social media platforms such as Twitter and Weibo. Users can either re-share messages posted by their friends or re-create messages based on the information acquired from other non-local information sources such as the mass media. By analyzing around 60 million messages from a large micro-blog site, we find that about 69% of the diffusion volume can be attributed to users' re-sharing behaviors, and the remaining 31% are caused by users' re-creating behaviors. The information diffusions caused by the two kinds of behaviors have different characteristics and variation trends, but most existing models of information diffusion do not distinguish them. The recent availability of massive online social streams allows us to study the process of information diffusion in much finer detail. In this paper, we introduce a novel model to capture and simulate the process of information diffusion in the micro-blog platforms, which distinguishes users' re-sharing behaviors from re-creating behaviors by introducing two different components. Thus, our model not only considers the effect of the underlying network structure, but also the influence of other non-local information sources. The empirical results show the superiority of our proposed model in the fitting and prediction tasks of information diffusion.

关键词： Blogs Uniform resource locators

Learning from Semi-Factuals: A Debiased and Semantic-Aware Framework for Generalized Relation Discovery

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Jiaxin Zhang, Lingling Liu, Jun Guo, Tianlin Wu, Wenjun The School of Computer Science and Technology Xi’an Jiaotong University Shaanxi Xi’an710049 China The Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Xi’an Jiaotong University Shaanxi Xi’an710049 China National Engineering Lab for Big Data Analytics Xi’an Jiaotong University Shaanxi Xi’an710049 China

We introduce a novel task, called Generalized Relation Discovery (GRD), for open-world relation extraction. GRD aims to identify unlabeled instances in existing pre-defined relations or discover novel relations by assigning instances to clusters as well as providing specific meanings for these clusters. The key challenges of GRD are how to mitigate the serious model biases caused by labeled pre-defined relations to learn effective relational representations and how to determine the specific semantics of novel relations during classifying or clustering unlabeled instances. We then propose a novel framework, SFGRD, for this task to solve the above issues by learning from semi-factuals in two stages. The first stage is semi-factual generation implemented by a tri-view debiased relation representation module, in which we take each original sentence as the main view and design two debiased views to generate semi-factual examples for this sentence. The second stage is semi-factual thinking executed by a dual-space tri-view collaborative relation learning module, where we design a cluster-semantic space and a class-index space to learn relational semantics and relation label indices, respectively. In addition, we devise alignment and selection strategies to integrate two spaces and establish a self-supervised learning loop for unlabeled data by doing semi-factual thinking across three views. Extensive experimental results show that SFGRD surpasses state-of-the-art models in terms of accuracy by 2.36% ∼5.78% and cosine similarity by 32.19%∼ 84.45% for relation label index and relation semantic quality, respectively. To the best of our knowledge, we are the first to exploit the efficacy of semi-factuals in relation extraction. Copyright © 2024, The Authors. All rights reserved.

关键词： Extraction

Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques 38

学校读者我要写书评

暂无评论

Suppress Content Shift: Better Diffusion Features via Off-th...

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Yang, Zhiyong Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Computer Science and Tech. University of Chinese Academy of Sciences China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are powerful generative models, and this capability can also be applied to discrimination. The inner activations of a pre-trained diffusion model can serve as features for discriminative tasks, namely, diffusion feature. We discover that diffusion feature has been hindered by a hidden yet universal phenomenon that we call content shift. To be specific, there are content differences between features and the input image, such as the exact shape of a certain object. We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature. Further empirical study also indicates that its negative impact is not negligible even when content shift is not visually perceivable. Hence, we propose to suppress content shift to enhance the overall quality of diffusion features. Specifically, content shift is related to the information drift during the process of recovering an image from the noisy input, pointing out the possibility of turning off-the-shelf generation techniques into tools for content shift suppression. We further propose a practical guideline named GATE to efficiently evaluate the potential benefit of a technique and provide an implementation of our methodology. Despite the simplicity, the proposed approach has achieved superior results on various tasks and datasets, validating its potential as a generic booster for diffusion features. Our code is available at this url. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features 38

学校读者我要写书评

暂无评论

Not All Diffusion Model Activations Have Been Evaluated as D...

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are initially designed for image generation. Recent research shows that the internal signals within their backbones, named activations, can also serve as dense features for various discriminative tasks such as semantic segmentation. Given numerous activations, selecting a small yet effective subset poses a fundamental problem. To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations. However, we find that many potential activations have not been evaluated, such as the queries and keys used to compute attention scores. Moreover, recent advancements in diffusion architectures bring many new activations, such as those within embedded ViT modules. Both combined, activation selection remains unresolved but overlooked. To tackle this issue, this paper takes a further step with a much broader range of activations evaluated. Considering the significant increase in activations, a full-scale quantitative comparison is no longer operational. Instead, we seek to understand the properties of these activations, such that the activations that are clearly inferior can be filtered out in advance via simple qualitative evaluation. After careful analysis, we discover three properties universal among diffusion models, enabling this study to go beyond specific models. On top of this, we present effective feature selection solutions for several popular diffusion models. Finally, the experiments across multiple discriminative tasks validate the superiority of our method over the SOTA competitors. Our code is available at this url. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

Breaking the moments condition barrier: no-regret algorithm for bandits with super heavy-tailed payoffs 21

学校读者我要写书评

暂无评论

Breaking the moments condition barrier: no-regret algorithm ...

Proceedings of the 35th International Conference on Neural Information Processing Systems

作者： Han Zhong Jiayi Huang Lin F. Yang Liwei Wang Center for Data Science Peking University Center for Data Science Peking University Pazhou Lab Department of Electrical and Computer Engineering University of California Los Angles Key Laboratory of Machine Perception MOE School of EECS Institute for Artificial Intelligence Peking University

ISBN: (纸本)9781713845393

Despite a large amount of effort in dealing with heavy-tailed error in machine learning, little is known when moments of the error can become non-existential: the random noise η satisfies Pr[|η| > |y|] ≤ 1/|y|α for some α > 0. We make the first attempt to actively handle such super heavy-tailed noise in bandit learning problems: We propose a novel robust statistical estimator, mean of medians, which estimates a random variable by computing the empirical mean of a sequence of empirical medians. We then present a generic reductionist algorithmic framework for solving bandit learning problems (including multi-armed and linear bandit problem): the mean of medians estimator can be applied to nearly any bandit learning algorithm as a black-box filtering for its reward signals and obtain similar regret bound as if the reward is sub-Gaussian. We show that the regret bound is near-optimal even with very heavy-tailed noise. We also empirically demonstrate the effectiveness of the proposed algorithm, which further corroborates our theoretical results.

关键词：

Adaptive gradient methods with dynamic bound of learning rate

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Luo, Liangchen Xiong, Yuanhao Liu, Yan Sun, Xu MOE Key Lab of Computational Linguistics School of EECS Peking University College of Information Science and Electronic Engineering Zhejiang University Department of Computer Science University of Southern California Center for Data Science Beijing Institute of Big Data Research Peking University

Adaptive optimization methods such as ADAGRAD, RMSPROP and ADAM have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates. Though prevailing, they are observed to generalize poorly compared with SGD or even fail to converge due to unstable and extreme learning rates. Recent work has put forward some algorithms such as AMSGRAD to tackle this issue but they failed to achieve considerable improvement over existing methods. In our paper, we demonstrate that extreme learning rates can lead to poor performance. We provide new variants of ADAM and AMSGRAD, called ADABOUND and AMSBOUND respectively, which employ dynamic bounds on learning rates to achieve a gradual and smooth transition from adaptive methods to SGD and give a theoretical proof of convergence. We further conduct experiments on various popular tasks and models, which is often insufficient in previous work. Experimental results show that new variants can eliminate the generalization gap between adaptive methods and SGD and maintain higher learning speed early in training at the same time. Moreover, they can bring significant improvement over their prototypes, especially on complex deep networks. The implementation of the algorithm can be found at https://***/Luolc/AdaBound. Copyright © 2019, The Authors. All rights reserved.

关键词： Gradient methods