检索结果-内蒙古大学图书馆

CIR at the NTCIR-17 ULTRE-2 Task

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yu, Lulu Bi, Keping Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China

The Chinese academy of sciences Information Retrieval team (CIR) has participated in the NTCIR-17 ULTRE-2 task. This paper describes our approaches and reports our results on the ULTRE-2 task. We recognize the issue of false negatives in the Baidu search data in this competition is very severe, much more severe than position bias. Hence, we adopt the Dual Learning Algorithm (DLA) to address the position bias and use it as an auxiliary model to study how to alleviate the false negative issue. We approach the problem from two perspectives: 1) correcting the labels for non-clicked items by a relevance judgment model trained from DLA, and learn a new ranker that is initialized from DLA;2) including random documents as true negatives and documents that have partial matching as hard negatives. Both methods can enhance the model performance and our best method has achieved nDCG@10 of 0.5355, which is 2.66% better than the best score from the organizer. Copyright © 2023, The Authors. All rights reserved.

关键词： Learning algorithms

Feature-Enhanced network with Hybrid Debiasing Strategies for Unbiased Learning to Rank

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yu, Lulu Wang, Yiting Sun, Xiaojie Bi, Keping Guo, Jiafeng CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China

Unbiased learning to rank (ULTR) aims to mitigate various biases existing in user clicks, such as position bias, trust bias, presentation bias, and learn an effective ranker. In this paper, we introduce our winning approach for the "Unbiased Learning to Rank" task in WSDM Cup 2023. We find that the provided data is severely biased so neural models trained directly with the top 10 results with click information are unsatisfactory. So we extract multiple heuristic-based features for multi-fields of the results, adjust the click labels, add true negatives, and re-weight the samples during model training. Since the propensities learned by existing ULTR methods are not decreasing w.r.t. positions, we also calibrate the propensities according to the click ratios and ensemble the models trained in two different ways. Our method won the 3rd prize with a DCG@10 score of 9.80, which is 1.1% worse than the 2nd and 25.3% higher than the 4th. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

Ensemble Ranking Model with Multiple Pretraining Strategies for Web Search

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Sun, Xiaojie Yu, Lulu Wang, Yiting Bi, Keping Guo, Jiafeng CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China

An effective ranking model usually requires a large amount of training data to learn the relevance between documents and queries. User clicks are often used as training data since they can indicate relevance and are cheap to collect, but they contain substantial bias and noise. There has been some work on mitigating various types of bias in simulated user clicks to train effective learning-to-rank models based on multiple features. However, how to effectively use such methods on large-scale pre-trained models with real-world click data is unknown. To alleviate the data bias in the real world, we incorporate heuristic-based features, refine the ranking objective, add random negatives, and calibrate the propensity calculation in the pre-training stage. Then we fine-tune several pre-trained models and train an ensemble model to aggregate all the predictions from various pre-trained models with human-annotation data in the fine-tuning stage. Our approaches won 3rd place in the "Pretraining for Web Search" task in WSDM Cup 2023 and are 22.6% better than the 4th-ranked team. Copyright © 2023, The Authors. All rights reserved.

关键词： Websites

Finformer: A Static-dynamic Spatiotemporal Framework for Stock Trend Prediction

学校读者我要写书评

暂无评论

Finformer: A Static-dynamic Spatiotemporal Framework for Sto...

2023 IEEE International Conference on Big Data, BigData 2023

作者： Zu, Yi Mi, Jiacong Song, Lingning Lu, Shan He, Jieyue Southeast University School of Computer Science and Engineering Key Lab of Computer Network and Information Integration MOE Nanjing China Southeast University School of Software Engineering Nanjing China Nanjing Fenghuo Tiandi Communication Technology Co. Ltd Nanjing China

ISBN: (纸本)9798350324457

The core of quantitative investment lies in predicting future trends in stock prices. The future trend of a stock is closely related to the industry it belongs to and its relationship with other stocks. Although some research has focused on stock trend prediction in recent years, most studies have only considered the stock's own time series feature, neglecting the spatial features between stocks. Some research has incorporated spatial information, but typically only considered predefined static relationships. At the same time, capturing dynamic spatial information in the market has been a long-standing challenge. Thus, we propose a spatio-temporal model, Finformer, in order to go beyond traditional time series models. We designed a sparse static-dynamic transformer to capture dynamic market spatial information as it changes over time and combined predefined relationships to extract highly correlated spatial features in the stock market. To effectively integrate spatial and temporal features, we introduced an adaptive spatio-temporal fusion module that dynamically fuses spatio-temporal features based on market conditions at different periods. Experiments on two real-world stock market datasets show that our proposed model outperforms the state-of-the-art baselines in the signal-based and portfolio-based metrics, which are widely concerned in the financial field. Ablation study and hyper-parameter study further reveal the effectiveness of each module in the model and the impact of hyper-parameters. The code will be made publicly available. 1 © 2023 IEEE.

关键词： computational finance deep learning stock trend prediction transformer

Are Large Language Models Good at Utility Judgments?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Hengran Zhang, Ruqing Guo, Jiafeng de Rijke, Maarten Fan, Yixing Cheng, Xueqi CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China University of Amsterdam Amsterdam Netherlands

Retrieval-augmented generation (RAG) is considered to be a promising approach to alleviate the hallucination issue of large language models (LLMs), and it has received widespread attention from researchers recently. Due to the limitation in the semantic understanding of retrieval models, the success of RAG heavily lies on the ability of LLMs to identify passages with utility. Recent efforts have explored the ability of LLMs to assess the relevance of passages in retrieval, but there has been limited work on evaluating the utility of passages in supporting question answering. In this work, we conduct a comprehensive study about the capabilities of large language models (LLMs) in utility evaluation for open-domain question answering (QA). Specifically, we introduce a benchmarking procedure and collection of candidate passages with different characteristics, facilitating a series of experiments with five representative LLMs. Our experiments reveal that: (i) well-instructed LLMs can distinguish between relevance and utility, and that LLMs are highly receptive to newly generated counterfactual passages. Moreover, (ii) we scrutinize key factors that affect utility judgments in the instruction design. And finally, (iii) to verify the efficacy of utility judgments in practical retrieval augmentation applications, we delve into LLMs’ QA capabilities using the evidence judged with utility and direct dense retrieval results. (iv) We propose a k-sampling, listwise approach to reduce the dependency of LLMs on the sequence of input passages, thereby facilitating subsequent answer generation. We believe that the way we formalize and study the problem along with our findings contributes to a critical assessment of retrieval-augmented LLMs. Our code and benchmark can be found at https://***/ict-bigdatalab/utility_judgments. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

On the Robustness of Generative Retrieval Models: An Out-of-Distribution Perspective

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Yu-An Zhang, Ruqing Guo, Jiafeng Chen, Wei Cheng, Xueqi CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China

Recently, we have witnessed generative retrieval increasingly gaining attention in the information retrieval (IR) field, which retrieves documents by directly generating their identifiers. So far, much effort has been devoted to developing effective generative retrieval models. There has been less attention paid to the robustness perspective. When a new retrieval paradigm enters into the real-world application, it is also critical to measure the out-of-distribution (OOD) generalization, i.e., how would generative retrieval models generalize to new distributions. To answer this question, firstly, we define OOD robustness from three perspectives in retrieval problems: 1) The query variations;2) The unforeseen query types;and 3) The unforeseen tasks. Based on this taxonomy, we conduct empirical studies to analyze the OOD robustness of several representative generative retrieval models against dense retrieval models. The empirical results indicate that the OOD robustness of generative retrieval models requires enhancement. We hope studying the OOD robustness of generative retrieval models would be advantageous to the IR community. Copyright © 2023, The Authors. All rights reserved.

关键词： Query processing

CLIPURE: PURIFICATION IN LATENT SPACE VIA CLIP FOR ADVERSARIALLY ROBUST ZERO-SHOT CLASSIFICATION

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Mingkun Bi, Keping Chen, Wei Guo, Jiafeng Cheng, Xueqi State Key Lab of AI Safety CAS Key Lab of AI Safety Institute of Computing Technology Chinese Academy of Sciences Beijing China CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

In this paper, we aim to build an adversarially robust zero-shot image classifier. We ground our work on CLIP, a vision-language pre-trained encoder model that can perform zero-shot classification by matching an image with text prompts "a photo of a .". Purification is the path we choose since it does not require adversarial training on specific attack types and thus can cope with any foreseen attacks. We then formulate purification risk as the KL divergence between the joint distributions of the purification process of denoising the adversarial samples and the attack process of adding perturbations to benign samples, through bidirectional Stochastic Differential Equations (SDEs). The final derived results inspire us to explore purification in the multi-modal latent space of CLIP. We propose two variants for our CLIPure approach: CLIPure-Diff which models the likelihood of images’ latent vectors with the DiffusionPrior module in DaLLE-2 (modeling the generation process of CLIP’s latent vectors), and CLIPure-Cos which models the likelihood with the cosine similarity between the embeddings of an image and "a photo of a.". As far as we know, CLIPure is the first purification method in multi-modal latent space and CLIPure-Cos is the first purification method that is not based on generative models, which substantially improves defense efficiency. We conducted extensive experiments on CIFAR-10, ImageNet, and 13 datasets that previous CLIP-based defense methods used for evaluating zero-shot classification robustness. Results show that CLIPure boosts the SOTA robustness by a large margin, e.g., from 71.7% to 91.1% on CIFAR10, from 59.6% to 72.6% on ImageNet, and 108% relative improvements of average robustness on the 13 datasets over previous SOTA. The code is available at https://***/TMLResearchGroup-CAS/CLIPure. Copyright © 2025, The Authors. All rights reserved.

关键词： Stochastic systems

Generative Retrieval for Book Search 25

学校读者我要写书评

暂无评论

Generative Retrieval for Book Search

Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1

作者： Yubao Tang Ruqing Zhang Jiafeng Guo Maarten de Rijke Shihao Liu Shuaiqiang Wang Dawei Yin Xueqi Cheng CAS Key Lab of Network Data Science and Technology ICT CAS Beijing China University of the Chinese Academy of Sciences Beijing China & University of Amsterdam Amsterdam Netherlands CAS Key Lab of Network Data Science and Technology ICT CAS Beijing China & University of Chinese Academy of Sciences Beijing China University of Amsterdam Amsterdam Netherlands Baidu Inc. Beijing China

ISBN: (纸本)9798400712456

In book search, relevant book information should be returned in response to a query. Books contain complex, multi-faceted information such as metadata, outlines, and main text, where the outline provides hierarchical information between chapters and sections. Generative retrieval (GR) is a new retrieval paradigm that consolidates corpus information into a single model to generate identifiers of documents that are relevant to a given query. How can GR be applied to book search? Directly applying GR to book search is a challenge due to the unique characteristics of book search: The model needs to retain the complex, multi-faceted information of the book, which increases the demand for labeled data. Splitting book information and treating it as a collection of separate segments for learning might result in a loss of hierarchical *** propose an effective Generative retrieval framework for Book Search (GBS) that features two main components: data augmentation and outline-oriented book encoding. For data augmentation, GBS constructs multiple query-book pairs for training; it constructs multiple book identifiers based on the outline, various forms of book contents, and simulates real book retrieval scenarios with varied pseudo-queries. This includes coverage-promoting book identifier augmentation, allowing the model to learn to index effectively, and diversity-enhanced query augmentation, allowing the model to learn to retrieve effectively. Outline-oriented book encoding improves length extrapolation through bi-level positional encoding and retentive attention mechanisms to maintain context over long sequences. Experiments on a proprietary Baidu dataset demonstrate that GBS outperforms strong baselines, achieving a 9.8% improvement in terms of MRR@20, over the state-of-the-art RIPOR method. Experiments on public datasets confirm the robustness and generalizability of GBS, highlighting its potential to enhance book retrieval.

关键词： book retrieval

Design and Implementation of Security Vulnerability Sharing Platform Based on Web Crawler 11th

学校读者我要写书评

暂无评论

Design and Implementation of Security Vulnerability Sharing ...

11th International Conference on Computer Engineering and networks, CENet2021

作者： Wang, Zhiqiang Wang, Ziyi Wang, Zhuoyue Zhang, Zhirui Yang, Tao Beijing Electronic Science and Technology Institute Beijing100070 China State Information Center Beijing100045 China Key Lab of Information Network Security Ministry of Public Security Shanghai China

ISBN: (纸本)9789811665530

network attacks affect the security of basic network facilities and critical information systems and cause serious damage to the national security. It is vital and important to establish a platform of software and hardware security vulnerability for collecting, verifying, sharing vulnerabilities. Aiming at the problems of low automation, high labor cost in the current domestic and foreign vulnerability sharing platform, this paper designs and implements a security vulnerability sharing platform based on a web crawler. The crawler technology is used to crawling the vulnerabilities’ data automatically. Moreover, more comprehensive vulnerability information query and sharing are realized, which is convenient for early warning and repair of security vulnerabilities. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

关键词： network security