检索结果-内蒙古大学图书馆

15th ACM International Conference on Web Search and Data Mining (WSDM)

作者： Fu, Chengzhen Hu, Enrui Feng, Letian Dou, Zhicheng Jia, Yantao Chen, Lei Yu, Fan Cao, Zhao Huawei Technol Co Ltd Distributed & Parallel Software Lab Shenzhen Peoples R China Renmin Univ China Gaoling Sch Artificial Intelligence Beijing Peoples R China Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Hong Kong Peoples R China

ISBN: (纸本)9781450391320

The configuration of 512 window size prevents transformers from being directly applicable to document ranking that requires larger context. Hence, recent works propose to estimate document relevance with fine-grained passage-level relevance signals. A limitation of such models, however, is that scoring each passage independently falls short in modeling inter-passage interactions and leads to unsatisfactory results. In this paper, we propose a Multiview inter-passage Interaction based Ranking model (MIR), to combine intra-passage interactions and inter-passage interactions in a complementary manner. The former captures local semantic relations inside each passage, whereas the latter draws global dependencies between different passages. Moreover, we represent inter-passage relationships via multi-view attention patterns, allowing information propagation at token, sentence, and passage-level. The representations at different levels of granularity, being aware of global context, are then aggregated into a document-level representation for ranking. Experimental results on two benchmarks show that modeling inter-passage interactions brings substantial improvements over existing passage-level methods.

关键词： Document ranking Inter-passage attention Intra-passage attention

来源：评论

学校读者我要写书评

暂无评论

Contrastive Learning of User Behavior Sequence for Context-Aware Document Ranking 21

Contrastive Learning of User Behavior Sequence for Context-A...

引用

30th ACM International Conference on Information and Knowledge Management (CIKM)

作者： Zhu, Yutao Nie, Jian-Yun Dou, Zhicheng Ma, Zhengyi Zhang, Xinyu Du, Pan Zuo, Xiaochen Jiang, Hao Univ Montreal Montreal PQ Canada Renmin Univ China Gaoling Sch Artificial Intelligence Beijing Peoples R China Huawei Distributed & Parallel Software Lab Hangzhou Zhejiang Peoples R China

ISBN: (纸本)9781450384469

Context information in search sessions has proven to be useful for capturing user search intent. Existing studies explored user behavior sequences in sessions in different ways to enhance query suggestion or document ranking. However, a user behavior sequence has often been viewed as a definite and exact signal reflecting a user's behavior. In reality, it is highly variable: user's queries for the same intent can vary, and different documents can be clicked. To learn a more robust representation of the user behavior sequence, we propose a method based on contrastive learning, which takes into account the possible variations in user's behavior sequences. Specifically, we propose three data augmentation strategies to generate similar variants of user behavior sequences and contrast them with other sequences. In so doing, the model is forced to be more robust regarding the possible variations. The optimized sequence representation is incorporated into document ranking. Experiments on two real query log datasets show that our proposed model outperforms the state-of-the-art methods significantly, which demonstrates the effectiveness of our method for context-aware document ranking.

关键词： Context-aware Document Ranking Contrastive Learning User Behavior Sequence Data Augmentation

来源：评论

学校读者我要写书评

暂无评论

You Ought to Look Around: Precise, Large Span Action Detection 25

You Ought to Look Around: Precise, Large Span Action Detecti...

引用

25th International Conference on Pattern Recognition (ICPR)

作者： Pan, Ge Zhang, Han Yu, Fan Song, Yonghong Zhang, Yuanlin Yuan, Han Xi An Jiao Tong Univ Sch Software Engn Xian Peoples R China Huawei Technol Distributed & Parallel Software Lab Shenzhen Guangdong Peoples R China Xi An Jiao Tong Univ Inst Artificial Intelligence & Robot Xian Peoples R China

ISBN: (纸本)9781728188089

For the action localization task, pre-defined action anchors are the cornerstone of mainstream techniques. State-of-the-art models mostly rely on a dense segmenting scheme, where anchors are sampled uniformly over the temporal domain with a predefined set of scales. However, it is not sufficient because action duration varies greatly. Therefore, it is necessary for the anchors or proposals to have a variable receptive field. In this paper, we propose a method called YOLA (You Ought to Look Around) which includes three parts: 1) a robust backbone SPN-I3D for extracting spatio-temporal features. In this part, we employ a stronger backbone I3D with SPN (Segment Pyramid Network) instead of C3D to obtain multi-scale features;2) a simple but useful feature fusion module named LFE (Local Feature Extraction). Compared with the fully connected layer and global average pooling, our LFE model is more advantageous for network to fit and fuse features. 3) a new feature segment aligning method called TPGC (Two Pathway Graph Convolution), which allows one proposal to leverage semantic features of adjacent proposals to update its content and make sure the proposals have a variable receptive field. YOLA add only a small overhead to the baseline network, and is easy to train in an end-to-end manner, running at a speed of 1097 fps. YOLA achieves a mAP of 58.3%, outperforming all existing models including both RGB-based and two stream on THUMOS'14, and achieves competitive results on ActivityNet 1.3.

关键词： Anchors

来源：评论

学校读者我要写书评

暂无评论

An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models

arXiv

引用

arXiv 2024年

作者： Liu, Qi Guo, Gang Mao, Jiaxin Dou, Zhicheng Wen, Ji-Rong Jiang, Hao Zhang, Xinyu Cao, Zhao Being Key Laboratory of Data Management and Analysis Methods Gaoling School of Artificial Intelligence Renmin University of China No.59 Zhongguancun Street Beijing100872 China Distributed and Parallel Software Lab Huawei Zhejiang No. 360 Jiangshu Road Hangzhou310056 China

With the development of pre-trained language models, the dense retrieval models have become promising alternatives to the traditional retrieval models that rely on exact match and sparse bag-of-words representations. Different from most dense retrieval models using a bi-encoder to encode each query or document into a dense vector, the recently proposed late-interaction multi-vector models (i.e., ColBERT and COIL) achieve state-of-the-art retrieval effectiveness by using all token embeddings to represent documents and queries and modeling their relevance with a sum-of-max operation. However, these fine-grained representations may cause unacceptable storage overhead for practical search systems. In this study, we systematically analyze the matching mechanism of these late-interaction models and show that the sum-of-max operation heavily relies on the co-occurrence signals and some important words in the document. Based on these findings, we then propose several simple document pruning methods to reduce the storage overhead and compare the effectiveness of different pruning methods on different late-interaction models. We also leverage query pruning methods to further reduce the retrieval latency. We conduct extensive experiments on both in-domain and out-domain datasets and show that some of the used pruning methods can significantly improve the efficiency of these late-interaction models without substantially hurting their retrieval effectiveness. © 2024, CC BY.

关键词： Efficiency

来源：评论

学校读者我要写书评

暂无评论

Variational Distillation for Multi-View Learning

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024年第7期46卷 4551-4566页

作者： Tian, Xudong Zhang, Zhizhong Wang, Cong Zhang, Wensheng Qu, Yanyun Ma, Lizhuang Wu, Zongze Xie, Yuan Tao, Dacheng East China Normal Univ Sch Comp Sci & Technol Shanghai 200062 Peoples R China Huawei Technol Distributed & Parallel Software Lab Labs 2012 Hangzhou 518129 Peoples R China Chinese Acad Sci Inst Automat Beijing 100190 Peoples R China Xiamen Univ Sch Informat Sci & Technol Xiamen 361005 Fujian Peoples R China East China Normal Univ Sch Comp Sci & Software Engn Shanghai 200050 Peoples R China Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn Shanghai 200240 Peoples R China Shenzhen Univ Coll Mech & Control Engn Shenzhen 518060 Peoples R China Normal Univ Chongqing Inst East China Shanghai 200062 Peoples R China JD Exploer Acad Beijing Peoples R China Univ Sydney Camperdown NSW 2050 Australia

Information Bottleneck (IB) provides an information-theoretic principle for multi-view learning by revealing the various components contained in each viewpoint. This highlights the necessity to capture their distinct roles to achieve view-invariance and predictive representations but remains under-explored due to the technical intractability of modeling and organizing innumerable mutual information (MI) terms. Recent studies show that sufficiency and consistency play such key roles in multi-view representation learning, and could be preserved via a variational distillation framework. But when it generalizes to arbitrary viewpoints, such strategy fails as the mutual information terms of consistency become complicated. This paper presents Multi-View Variational Distillation ((MVD)-D-2), tackling the above limitations for generalized multi-view learning. Uniquely, (MVD)-D-2 can recognize useful consistent information and prioritize diverse components by their generalization ability. This guides an analytical and scalable solution to achieving both sufficiency and consistency. Additionally, by rigorously reformulating the IB objective, (MVD)-D-2 tackles the difficulties in MI optimization and fully realizes the theoretical advantages of the information bottleneck principle. We extensively evaluate our model on diverse tasks to verify its effectiveness, where the considerable gains provide key insights into achieving generalized multi-view representations under a rigorous information-theoretic principle.

关键词： Mutual information Task analysis Representation learning Predictive models Optimization Visualization Pattern analysis Multi-view learning information bottleneck mutual information variational inference knowledge distillation

来源：评论

学校读者我要写书评

暂无评论

EPP-MVSNet: Epipolar-assembling based Depth Prediction for Multi-view Stereo

EPP-MVSNet: Epipolar-assembling based Depth Prediction for M...

引用

International Conference on Computer Vision (ICCV)

作者： Xinjun Ma Yue Gong Qirui Wang Jingwei Huang Lei Chen Fan Yu Distributed and Parallel Software Lab Huawei Technologies Hong Kong University of Science and Technology

ISBN: (纸本)9781665428132

In this paper, we proposed EPP-MVSNet, a novel deep learning network for 3D reconstruction from multi-view stereo (MVS). EPP-MVSNet can accurately aggregate features at high resolution to a limited cost volume with an optimal depth range, thus, leads to effective and efficient 3D construction. Distinct from existing works which measure feature cost at discrete positions which affects the 3D reconstruction accuracy, EPP-MVSNet introduces an epipolar-assembling-based kernel that operates on adaptive intervals along epipolar lines for making full use of the image resolution. Further, we introduce an entropy-based refining strategy where the cost volume describes the space geometry with the little redundancy. Moreover, we design a light-weighted network with Pseudo-3D convolutions integrated to achieve high accuracy and efficiency. We have conducted extensive experiments on challenging datasets Tanks & Temples(TNT), ETH3D and DTU. As a result, we achieve promising results on all datasets and the highest F-Score on the online TNT intermediate benchmark. Code is available at https://***/mindspore/mindspore/tree/master/model_zoo/research/cv/eppmvsnet.

关键词： Learning systems Costs Three-dimensional displays Image resolution Refining Redundancy Memory management

来源：评论

学校读者我要写书评

暂无评论

YES SIR!Optimizing semantic space of Negatives with Self-Involvement Ranker

arXiv

引用

arXiv 2021年

作者： Pu, Ruizhi Zhang, Xinyu Lai, Ruofei Guo, Zikai Zhang, Yinxia Jiang, Hao Wu, Yongkang Jia, Yantao Dou, Zhicheng Cao, Zhao Distributed and Parallel Software Lab Huawei China Gaoling School of Artificial Intelligence Renmin University of China

Pre-trained model such as BERT has been proved to be an effective tool for dealing with Information Retrieval (IR) problems. Due to its inspiring performance, it has been widely used to tackle with real-world IR problems such as document ranking. Recently, researchers have found that selecting "hard" rather than"random" negative samples would be beneficial for fine-tuning pre-trained models on ranking tasks. However, it remains elusive how to leverage hard negative samples in a principled way. To address aforementioned issues, we propose a fine-tuning strategy for document ranking, namely Self-Involvement Ranker (SIR), to dynamically select hard negative samples to construct high-quality semantic space for training a high-quality ranking model. Specifically, SIR consists of sequential compressors implemented with pre-trained models. Front compressor selects hard negative samples for rear compressor. Moreover, SIR leverages supervisory signal to adaptively adjust semantic space of negative samples. Finally, supervisory signal in rear compressor is computed based on condition probability and thus can control sample dynamic and further enhance the model performance. SIR is a lightweight and general framework for pre-trained models, which simplifies the ranking process in industry practice. We test our proposed solution on MS MARCO with document ranking setting, and the results show that SIR can significantly improve the ranking performance of various pre-trained models. Moreover, our method became the new SOTA model anonymously on MS MARCO Document ranking leaderboard in May 2021. Copyright © 2021, The Authors. All rights reserved.

关键词： Search engines

来源：评论

学校读者我要写书评

暂无评论

Answer Complex Questions: Path Ranker Is All You Need 21

Answer Complex Questions: Path Ranker Is All You Need

引用

44th International ACM SIGIR Conference on Research and Development in Information Retrieval

作者： Zhang, Xinyu Zhan, Ke Hu, Enrui Fu, Chengzhen Luo, Lan Jiang, Hao Jia, Yantao Yu, Fan Dou, Zhicheng Cao, Zhao Chen, Lei Huawei Distributed & Parallel Software Lab Shenzhen Peoples R China Renmin Univ China Gaoling Sch Artificial Intelligence Beijing Peoples R China Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Hong Kong Peoples R China

ISBN: (纸本)9781450380379

Currently, the most popular method for open-domain Question Answering (QA) adopts "Retriever and Reader" pipeline, where the retriever extracts a list of candidate documents from a large set of documents followed by a ranker to rank the most relevant documents and the reader extracts answer from the candidates. Existing studies take the greedy strategy in the sense that they only use samples for ranking at the current hop, and ignore the global information across the whole documents. In this paper, we propose a purely rank-based framework Thinking Path Re-Ranker (TPRR), which is comprised of Thinking Path Ranker (TPR) for generating document sequences called "a path" and External Path Reranker (EPR) for selecting the best path from candidate paths generated by TPR. Specifically, TPR leverages the scores of a dense model and conditional probabilities to score the full paths. Moreover, to further enhance the performance of the dense ranker in the iterative training, we propose a "thinking" negatives selection method that the top-K candidates treated as negatives in the current hop are adjusted dynamically through supervised signals. After achieving multiple supporting paths through TPR, the EPR component which integrates several fine-grained training tasks for QA is used to select the best path for answer extraction. We have tested our proposed solution on the multi-hop dataset "HotpotQA" with a full wiki setting, and the results show that TPRR significantly outperforms the existing state-of-the-art models. Moreover, our method haswon the first place(1) in the HotpotQA official leaderboard since Feb 1, 2021 under the Fullwiki setting. Code is available at https:// ***/ mindspore/ mindspore/ tree/ master/ model_zoo/ research/ nlp/ tprr.

关键词： Passage ranking Path ranker Multi-hop QA Multi-turn ranking

来源：评论

学校读者我要写书评

暂无评论

Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Need 21

Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Nee...

引用

30th ACM International Conference on Information and Knowledge Management (CIKM)

作者： Ma, Zhengyi Dou, Zhicheng Xu, Wei Zhang, Xinyu Jiang, Hao Cao, Zhao Wen, Ji-Rong Renmin Univ China Gaoling Sch Artificial Intelligence Beijing Peoples R China Huawei Distributed & Parallel Software Lab Beijing Peoples R China Beijing Key Lab Big Data Management & Anal Method Beijing Peoples R China MOE Key Lab Data Engn & Knowledge Engn Beijing Peoples R China

ISBN: (纸本)9781450384469

Designing pre-training objectives that more closely resemble the downstream tasks for pre-trained language models can lead to better performance at the fine-tuning stage, especially in the ad-hoc retrieval area. Existing pre-training approaches tailored for IR tried to incorporate weak supervised signals, such as query-likelihood based sampling, to construct pseudo query-document pairs from the raw textual corpus. However, these signals rely heavily on the sampling method. For example, the query likelihood model may lead to much noise in the constructed pre-training data. In this paper, we propose to leverage the large-scale hyperlinks and anchor texts to pre-train the language model for ad-hoc retrieval. Since the anchor texts are created by webmasters and can usually summarize the target document, it can help to build more accurate and reliable pre-training samples than a specific algorithm. Considering different views of the downstream ad-hoc retrieval, we devise four pre-training tasks based on the hyperlinks. We then pre-train the Transformer model to predict the pair-wise preference, jointly with the Masked Language Model objective. Experimental results on two large-scale ad-hoc retrieval datasets show the significant improvement of our model compared with the existing methods.

关键词： Ad-hoc Retrieval Pre-training Hyperlinks Anchor Texts

来源：评论

学校读者我要写书评

暂无评论

distributed and parallel Sparse Computing for Very Large Graph Neural Networks

Distributed and Parallel Sparse Computing for Very Large Gra...

引用

IEEE International Conference on Big Data

作者： Quentin R. Petit Chong Li Nahid Emad Huawei Paris Research Center & Université Paris-Saclay Boulogne-Billancourt France Distributed and Parallel Software Lab Huawei Paris Research Center Boulogne-Billancourt France Maison de la Simulation & LI-PaRAD Université Versailles Saint-Quentin Saclay France

ISBN: (纸本)9781665480468

Deep learning (DL) requires high-performance processing on big data. Graph Neural Networks, a challenging topic in DL using linear algebra methods, need algorithmic solutions to efficiently assign and process graph data on modern distributed and parallel machines, which are considered with mixed arithmetic and various types of tensor/matrix accelerators. Determining compression techniques for the graph’s sparse data structures is one of the key *** first objective is to design and implement a reusable parallel numerical library to resolve large neural network graphs. Our design strategy is drawn on a component-based approach and targets maximum code reuse in various parallel contexts while allowing for performance optimization. The solution could be later integrated into a DL framework like MindSpore.

关键词： Deep learning distributed databases Linear algebra Big Data parallel machines Graph neural networks Libraries

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：