检索结果-内蒙古大学图书馆

query by example: Semantic Traffic Scene Retrieval Using LLM-Based Scene Graph Representation

SENSORS 2025年第8期25卷 2546-2546页

作者： Tian, Yafu Carballo, Alexander Li, Ruifeng Thompson, Simon Takeda, Kazuya Nagoya Univ Grad Sch Informat Furo ChoChikusa Ku Nagoya 4648603 Japan Harbin Inst Technol State Key Lab Robot & Intelligent Syst Harbin 150000 Peoples R China Nagoya Univ Tier IV Inc Nakamura Ward Open Innovat Ctr 1-3 Mei Eki 1 Chome Nagoya 4506610 Japan Gifu Univ Fac Engn 1-1 Yanagido Gifu 5011193 Japan Nagoya Univ Inst Innovat Future Soc Furo ChoChikusa Ku Nagoya 4648601 Japan

In autonomous driving, retrieving a specific traffic scene in huge datasets is a significant challenge. Traditional scene retrieval methods struggle to cope with the semantic complexity and heterogeneity of traffic scenes and are unable to meet the variable needs of different users. This paper proposes "query-by-example", a traffic scene retrieval approach based on Visual-Large Language Model (VLM)-generated Road Scene Graph (RSG) representation. Our method uses VLMs to generate structured scene graphs from video data, capturing high-level semantic attributes and detailed object relationships in traffic scenes. We introduce an extensible set of scene attributes and a graph-based scene description to quantify scene similarity. We also propose a RSG-LLM benchmark dataset containing 1000 traffic scenes, their corresponding natural language descriptions, and RSGs to evaluate the performance of LLMs in generating RSGs. Experiments show that our method can effectively retrieve semantically similar traffic scenes from large databases, supporting various query formats, including natural language, images, video clips, rosbag, etc. Our method provides a comprehensive and flexible framework for traffic scene retrieval, promoting its application in autonomous driving systems.

关键词： traffic scene retrieval visual LLMs subgraph isomorphism matching scene graph query by example

来源：评论

学校读者我要写书评

暂无评论

Webpage retrieval based on query by example for think tank construction

引用

INFORMATION PROCESSING & MANAGEMENT 2022年第1期59卷 102767-102767页

作者： Geng, Qian Chuai, Ziang Jin, Jian Beijing Normal Univ Zhuhai Ctr Governance Studies Zhuhai Peoples R China Beijing Normal Univ Sch Govt Dept Informat Management Beijing Peoples R China

Think tanks have been proved helpful for decision-making in various communities. However, collecting information manually for think tank construction implies too much time and labor cost as well as inevitable subjectivity. A probable solution is to retrieve webpages of renowned experts and institutes similar to a given example, denoted as query by webpage (QBW). Considering users' searching behaviors, a novel QBW model based on webpages' visual and textual features is proposed. Specifically, a visual feature extraction module based on pre trained neural networks and a heuristic pooling scheme is proposed, which bridges the gap that existing extractors fail to extract snapshots' high-level features and are sensitive to the noise effect brought by images. Moreover, a textual feature extraction module is proposed to represent textual content in both term and topic grains, while most existing extractors merely focus on the term grain. In addition, a series of similarity metrics are proposed, including a textual similarity metric based on feature bootstrapping to improve model's robustness and an adaptive weighting scheme to balance the effect of different types of features. The proposed QBW model is evaluated on expert and institute introduction retrieval tasks in academic and medical scenarios, in which the average value of MAP has been improved by 10% compared to existing baselines. Practically, useful insights can be derived from this study for various applications involved with webpage retrieval besides think tank construction.

关键词： Feature bootstrapping Pre-trained neural networks query by example Textual features Visual features Webpage retrieval

来源：评论

学校读者我要写书评

暂无评论

Neural Network Based End-to-End query by example Spoken Term Detection

引用

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 2020年 28卷 1416-1427页

作者： Ram, Dhananjay Miculicich, Lesly Bourlard, Herve Idiap Res Inst CH-1920 Martigny Switzerland Idiap Res Inst CH-1920 Martigny Switzerland Ecole Polytech Fed Lausanne EPFL CH-1015 Lausanne Switzerland

This article focuses on the problem of query by example spoken term detection (QbE-STD) in zero-resource scenario. State-of-the-art approaches primarily rely on dynamic time warping (DTW) based template matching techniques using phone posterior or bottleneck features extracted from a deep neural network (DNN). We use both monolingual and multilingual bottleneck features, and show that multilingual features perform increasingly better with more training languages. Previously, it has been shown that the DTW based matching can be replaced with a CNN based matching while using posterior features. Here, we show that the CNN based matching outperforms DTW based matching using bottleneck features as well. In this case, the feature extraction and pattern matching stages of our QbE-STD system are optimized independently of each other. We propose to integrate these two stages in a fully neural network based end-to-end learning framework to enable joint optimization of those two stages simultaneously. The proposed approaches are evaluated on two challenging multilingual datasets: Spoken Web Search 2013 and query by example Search on Speech Task 2014, demonstrating in each case significant improvements.

关键词： Feature extraction Task analysis Neural networks Training Hidden Markov models Neurons Speech processing Spoken term detection query by example deep neural network bottleneck features end-to-end subsequence detection

来源：评论

学校读者我要写书评

暂无评论

Reduction of Speech Data Posteriorgrams by Compressing Maximum-likelihood State Sequences in query by example

Reduction of Speech Data Posteriorgrams by Compressing Maxim...

引用

Asia-Pacific-Signal-and-Information-Processing-Association Annual Summit and Conference (APSIPA ASC)

作者： Yokota, Takashi Kojima, Kazunori Lee, Shi-wook Itoh, Yoshiaki Iwate Prefectural Univ Takizawa Iwate Japan Natl Inst Adv Ind Sci & Technol Tokyo Japan

ISBN: (纸本)9789881476883

Spoken-term detection (STD) has recently attracted increased interest in speech-based retrieval research. STD is the task of finding sections in speech data matching a query consisting of one or more words. query by example (QbE) using spoken queries is another important research topic in STD. Although the use of posteriorgrams (sequences of output probabilities generated by deep neural networks from speech data) is a promising approach for QbE, that method results in long retrieval times and excessive memory usage. We previously proposed a method that replaces posteriorgrams for spoken queries with sequences of state numbers for triphone hidden Markov models, omitting calculations of local distance. While that method greatly reduced retrieval times, it still required large amounts of memory to store speech data posteriorgrams. We therefore newly propose a method for reducing memory usage and retrieval times by compressing speech data posteriorgrams into sets of posterior probability vectors for each utterance, each speech document, or all speech data, rather than storing all posterior probability vectors for each frame of speech data. Evaluation experiments conducted using open test collections for the "SpokenDoc" tasks of the NTCIR-10 and NTCIR-12 workshops demonstrate memory usage reduction by the proposed method.

关键词： Spoken-term detection query by example Average posterior probability vector

来源：评论

学校读者我要写书评

暂无评论

Multiple Deep Learning Models and Architectures with Different Numbers of States Used to Improve Retrieval Accuracy of query-by-example

Multiple Deep Learning Models and Architectures with Differe...

引用

Asia-Pacific-Signal-and-Information-Processing-Association Annual Summit and Conference (APSIPA ASC)

作者： Hatakeyama, Kazuki Nishino, Masahiro Kojima, Kazunori Lee, Shi-wook Itoh, Yoshiaki Iwate Prefectural Univ Takizawa Japan Toyota Syst Corp Tokyo Japan Natl Inst Adv Ind Sci & Technol Tokyo Japan

ISBN: (纸本)9789881476890

Studies examining Spoken Term Detection (STD) and Spoken query STD (SQ-STD) or query by example (QbE) using a spoken query have been conducted actively in recent years. When a spoken query is transcribed into a text using an automatic speech recognizer in SQ-STD, some misrecognition leads to retrieval accuracy deterioration. Posteriorgrams obtained using Deep Neural Network (DNN) and so on can be regarded as speaker-independent features. Although posteriorgram matching between a posteriorgram of a spoken query and posteriorgram of speech data showed high retrieval accuracy, it requires a long retrieval time and a large memory space. In earlier papers, we proposed a maximum likelihood state sequence method (MLSS) for retrieval time reduction. As described herein, we propose a method for reducing both the retrieval time and the memory space using MLSS method and multiple machine learning models with different numbers of states. The models show heterogeneous retrieval results. Their integration is probably mutually complementary and engenders retrieval accuracy improvement. Evaluation results demonstrate that the proposed method improves the retrieval accuracy, thereby reducing the retrieval time and the memory space.

关键词： query by example Spoken term detection maximum likelihood state sequence

来源：评论

学校读者我要写书评

暂无评论

Acoustic Word Embedding System for Code-Switching query-by-example Spoken Term Detection 12

Acoustic Word Embedding System for Code-Switching Query-by-e...

引用

12th International Symposium on Chinese Spoken Language Processing (ISCSLP)

作者： Ma, Murong Wu, Haiwei Wang, Xuyang Yang, Lin Wang, Junjie Li, Ming Duke Kunshan Univ Data Sci Res Ctr Kunshan Peoples R China Wuhan Univ Sch Comp Sci Wuhan Peoples R China AI Lab Lenovo Res Beijing Peoples R China

ISBN: (纸本)9781728169941

In this paper, we propose a deep convolutional neural network-based acoustic word embedding system for code-switching query by example spoken term detection. Different from previous configurations, we combine audio data in two languages for training instead of only using one single language. We transform the acoustic features of keyword templates and searching content segments obtained in a sliding manner to fixed-dimensional vectors and calculate the distances between them. An auxiliary variability-invariant loss is also applied to training data within the same word but different speakers. This strategy is used to prevent the extractor from encoding undesired speaker- or accent-related information into the acoustic word embeddings. Experimental results show that our proposed system produces promising searching results in the code-switching test scenario. With the employment of variability-invariant loss, the searching performance is further enhanced.

关键词： convolutional neural network acoustic word embedding code-switching query by example

来源：评论

学校读者我要写书评

暂无评论

Phonetic subspace features for improved query by example spoken term detection

引用

SPEECH COMMUNICATION 2018年 103卷 27-36页

作者： Ram, Dhananjay Asaei, Afsaneh Bourlard, Herve Idiap Res Inst Martigny Switzerland Ecole Polytech Fed Lausanne Lausanne Switzerland

This paper addresses the problem of detecting speech utterances from a large audio archive using a simple spoken query, hence referring to this problem as "query by example Spoken Term Detection" (QbE-STD). This still open pattern matching problem has been addressed in different contexts, often based on variants of the Dynamic Time Warping (DTW) algorithm. In the work reported here, we exploit Deep Neural Networks (DNN) and the so inferred phone posteriors to better model the phonetic subspaces and, consequently, improve the QbE-STD performance. Those phone posteriors have indeed been shown to properly model the union of the underlying low-dimensional phonetic subspaces. Exploiting this property. we investigate here two methods relying on sparse modeling and linguistic knowledge of sub-phonetic components. Sparse modeling characterizes the phonetic subspaces through a dictionary for sparse coding. Projection of the phone posteriors through reconstruction on the corresponding subspaces using their sparse representation enhance those phone posteriors. On the other hand, linguistic knowledge driven sub-phonetic structures are identified using phonological posteriors which consists of the probabilities of phone attributes estimated by DNNs, resulting in a new set of feature vectors. These phonological posteriors provide complementary information and a distance fusion method is proposed to integrate information from phone and phonological posterior features. Both posterior features are used for query detection using DTW and evaluated on AMI database. We demonstrate that the subspace enhanced phone posteriors obtained using sparse reconstruction outperforms the conventional DNN posteriors. The distance fusion technique gives further improvement in QbE-STD performance. (C) 2018 Elsevier B.V. All rights reserved.

关键词： Deep neural network Phone posterior Phonological posterior Sparse representation Dictionary learning query by example Spoken term detection

来源：评论

学校读者我要写书评

暂无评论

Sparse Subspace Modeling for query by example Spoken Term Detection

引用

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 2018年第6期26卷 1126-1139页

作者： Ram, Dhananjay Asaei, Afsaneh Bourlard, Herve Idiap Res Inst CH-1920 Martigny Switzerland Ecole Polytech Fed Lausanne CH-1015 Lausanne Switzerland

This paper focuses on the problem of query by example spoken term detection (QbE-STD) in zero-resource scenario. Current state-of-the-art approaches to tackle this problem rely on dynamic programming based template matching techniques using phone posterior features extracted at the output of a deep neural network. Previously, it has been shown that the space of phone posteriors is highly structured, as a union of low-dimensional sub-spaces. To exploit the temporal and sparse structure of the speech data, we investigate here three different QbE-STD systems based on sparse model recovery. More specifically, we use query examples to model the query subspace using dictionary for sparse coding. Reconstruction errors calculated using sparse representation of feature vectors are then used to characterize the underlying subspaces. The first approach uses these reconstruction errors in a dynamic programming framework to detect the spoken query, resulting in a much faster search compared to standard template matching. The other two methods aim at merging template matching and sparsity-based approaches to further improve the performance. The first one proposes to regularize the template matching local distances using sparse reconstruction errors. The second approach aims at using the sparse reconstruction errors to rescore (improve) the template matching likelihood. Experiments on two different databases (AMI and MediaEval) show that the proposed hybrid systems perform better than a highly competitive QbE-STD baseline system.

关键词： Speech processing spoken term detection query by example deep neural network posterior probabilities sparse recovery modeling sparse representation subspace detection subspace regularization

来源：评论

学校读者我要写书评

暂无评论

Fuzzy query by example 18

Fuzzy Query By Example

引用

33rd Annual ACM Symposium on Applied Computing (ACM SAC)

作者： Moreau, Aurelien Pivert, Olivier Smits, Gregory Univ Rennes CNRS IRISA UMR 6074 F-22305 Lannion France

ISBN: (纸本)9781450351911

This paper describes Fuzzy query by example, an approach helping users retrieve data without any prior knowledge of the database schema or any formal querying language. The user is solicited to evaluate, in a binary way, pre-selected items of the database. We provide a characterization-based strategy that identifies the properties shared by the examples (resp. counter-examples) positively (resp. negatively) evaluated by the user. These properties are expressed using linguistic terms from a fuzzy vocabulary to ensure that the user has a good understanding of the inferred query.

关键词： query by example Fuzzy logic Databases

来源：评论

学校读者我要写书评

暂无评论

Unsupervised query by example spoken term detection using features concatenated with Self-Organizing Map distances 11

Unsupervised query by example spoken term detection using fe...

引用

11th International Symposium on Chinese Spoken Language Processing (ISCSLP)

作者： Wu, Haiwei Li, Ming Cai, Zexin Zhong, Haibin Duke Kunshan Univ Kunshan Peoples R China Sun Yat Sen Univ Sch Elect & Informat Technol Guangzhou Guangdong Peoples R China Jiangsu Jinling Sci & Technol Grp Ltd Nanjing Jiangsu Peoples R China

ISBN: (纸本)9781538656273

In the task of the unsupervised query by example spoken term detection (QbE-STD), we concatenate the features extracted by a Self-Organizing Map (SOM) and features learned by an unsupervised GMM based model at the feature level to enhance the performance. More specifically, The SOM features are represented by the distances between the current feature vector and the weight vectors of SOM neurons learned in an unsupervised manner. After fetching these features, we apply sub-sequence Dynamic Time Warping (S-DTW) to detect the occurrences of keywords in the test data. We evaluate the performance of these features on the TIMIT English database. After concatenating the SOM features and the GMM based features together, we achieve an improvement of 7.77% and 7.74% on Mean Average Precision (MAP) and P@10 on average.

关键词： unsupervised learning query by example self-organizing map

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：