检索结果-内蒙古大学图书馆

Learning to truncate ranked lists for information retrieval

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wu, Chen Zhang, Ruqing Guo, Jiafeng Fan, Yixing Lan, Yanyan Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Ranked list truncation is of critical importance in a variety of professional information retrieval applications such as patent search or legal search. The goal is to dynamically determine the number of returned documents according to some user-defined objectives, in order to reach a balance between the overall utility of the results and user efforts. Existing methods formulate this task as a sequential decision problem and take some pre-defined loss as a proxy objective, which suffers from the limitation of local decision and non-direct optimization. In this work, we propose a global decision based truncation model named AttnCut, which directly optimizes user-defined objectives for the ranked list truncation. Specifically, we take the successful transformer architecture to capture the global dependency within the ranked list for truncation decision, and employ the reward augmented maximum likelihood (RAML) for direct optimization. We consider two types of user-defined objectives which are of practical usage. One is the widely adopted metric such as F1 which acts as a balanced objective, and the other is the best F1 under some minimal recall constraint which represents a typical objective in professional search. Empirical results over the Robust04 and MQ2007 datasets demonstrate the effectiveness of our approach as compared with the state-of-the-art baselines. Copyright © 2021, The Authors. All rights reserved.

关键词： Information retrieval

A linguistic study on relevance modeling in information retrieval

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Fan, Yixing Guo, Jiafeng Ma, Xinyu Zhang, Ruqing Lan, Yanyan Cheng, Xueqi University of Chinese Academy of Sciences Beijing China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

Relevance plays a central role in information retrieval (IR), which has received extensive studies starting from the 20th century. The definition and the modeling of relevance has always been critical challenges in both information science and computer science research areas. Along with the debate and exploration on relevance, IR has already become a core task in many real-world applications, such as Web search engines, question answering systems, conversational bots, and so on. While relevance acts as a unified concept in all these retrieval tasks, the inherent definitions are quite different due to the heterogeneity of these tasks. This raises a question to us: Do these different forms of relevance really lead to different modeling focuses? To answer this question, in this work, we conduct an empirical study on relevance modeling in three representative IR tasks, i.e., document retrieval, answer retrieval, and response retrieval. Specifically, we attempt to study the following two questions: 1) Does relevance modeling in these tasks really show differences in terms of natural language understanding (NLU)? We employ 16 linguistic tasks to probe a unified retrieval model over these three retrieval tasks to answer this question. 2) If there do exist differences, how can we leverage the findings to enhance the relevance modeling? We proposed three intervention methods to investigate how to leverage different modeling focuses of relevance to improve these IR tasks. We believe the way we study the problem as well as our findings would be beneficial to the IR community. © 2021, CC BY.

关键词： Information retrieval

BiKT: Unleashing the potential of GNNs via Bi-directional Knowledge Transfer

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zheng, Shuai Liu, Zhizhe Zhu, Zhenfeng Zhang, Xingxing Li, Jianxin Zhao, Yao The Institute of Information Science Beijing Jiaotong University Beijing100044 China The Beijing Key Laboratory of Advanced Information Science and Network Technology Beijing100044 China Qiyuan Lab Beijing China The Beijing Advanced Innovation Center for Big Data and Brain Computing School of Computer Science and Engineering Beihang University Beijing100083 China

Based on the message-passing paradigm, there has been an amount of research proposing diverse and impressive feature propagation mechanisms to improve the performance of GNNs. However, less focus has been put on feature transformation, another major operation of the message-passing framework. In this paper, we first empirically investigate the performance of the feature transformation operation in several typical GNNs. Unexpectedly, we notice that GNNs do not completely free up the power of the inherent feature transformation operation. By this observation, we propose the Bi-directional Knowledge Transfer (BiKT), a plug-and-play approach to unleash the potential of the feature transformation operations without modifying the original architecture. Taking the feature transformation operation as a derived representation learning model that shares parameters with the original GNN, the direct prediction by this model provides a topological-agnostic knowledge feedback that can further instruct the learning of GNN and the feature transformations therein. On this basis, BiKT not only allows us to acquire knowledge from both the GNN and its derived model but promotes each other by injecting the knowledge into the other. In addition, a theoretical analysis is further provided to demonstrate that BiKT improves the generalization bound of the GNNs from the perspective of domain adaption. An extensive group of experiments on up to 7 datasets with 5 typical GNNs demonstrates that BiKT brings up to 0.5% - 4% performance gain over the original GNN, which means a boosted GNN is obtained. Meanwhile, the derived model also shows a powerful performance to compete with or even surpass the original GNN, enabling us to flexibly apply it independently to some other specific downstream tasks. Copyright © 2023, The Authors. All rights reserved.

关键词： Message passing

FedMatch: Federated learning over heterogeneous question answering data

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Chen, Jiangui Zhang, Ruqing Guo, Jiafeng Fan, Yixing Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Question Answering (QA), a popular and promising technique for intelligent information access, faces a dilemma about data as most other AI techniques. On one hand, modern QA methods rely on deep learning models which are typically data-hungry. Therefore, it is expected to collect and fuse all the available QA datasets together in a common site for developing a powerful QA model. On the other hand, real-world QA datasets are typically distributed in the form of isolated islands belonging to different parties. Due to the increasing awareness of privacy security, it is almost impossible to integrate the data scattered around, or the cost is prohibited. A possible solution to this dilemma is a new approach known as federated learning, which is a privacy-preserving machine learning technique over distributed datasets. In this work, we propose to adopt federated learning for QA with the special concern on the statistical heterogeneity of the QA data. Here the heterogeneity refers to the fact that annotated QA data are typically with non-identical and independent distribution (non-IID) and unbalanced sizes in practice. Traditional federated learning methods may sacrifice the accuracy of individual models under the heterogeneous situation. To tackle this problem, we propose a novel Federated Matching framework for QA, named FedMatch, with a backbone-patch architecture. The shared backbone is to distill the common knowledge of all the participants while the private patch is a compact and efficient module to retain the domain information for each participant. To facilitate the evaluation, we build a benchmark collection based on several QA datasets from different domains to simulate the heterogeneous situation in practice. Empirical studies demonstrate that our model can achieve significant improvements against the baselines over all the datasets. © 2021, CC BY-NC-ND.

关键词： Deep learning

Focus on the Stability of Large Systems: Toward Automatic Prediction and Analysis of Vulnerability Threat Intelligence

学校读者我要写书评

暂无评论

Focus on the Stability of Large Systems: Toward Automatic Pr...

IEEE International Conference on data science in Cyberspace (DSC)

作者： Shiwen Song Qiong Wu Xin Zheng Peng Wang Yuchen Dou Zhongwen Li Lidong Zhai Institute of Information Engineering Chinese Academy of Sciences BeiJing China Bag Data Academy Zhongke Beijing China CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

ISBN: (纸本)9781665418164

With the increase in the number of users and business volume, the business systems of Internet companies are becoming more and more complex, resulting in a surge in the number of alarms. A large number of dirty alarms add a huge workload to security operations, which indirectly pose a large number of threats to business systems. At present, most systems use the method of accessing third-party Threat Intelligence to assist operators to realize automatic handling of alarms. However, this method has lagging and accuracy problems, making this work always difficult to meet the requirements of fast and accurate. This article proposes a new method for gathering vulnerability Threat Intelligence, which can obtain vulnerability information in advance of security announcements issued by security vendors. By analyzing the vulnerability disclosure process, this method obtains vulnerability information from the original source submitted by open source mail group, of developers. We used NLP technology and XGBoost model to automatically analyze the vulnerability information, and finally generate FINTEL. The experimental result shows that this method has an accuracy of 93%, and can obtain vulnerability information 10h to 7 days before security vendors release. The scope of application covers all open source code repositories and some closed source repositories.

关键词： Machine learning algorithms Scalability Prediction methods Stability analysis Security Surges Research and development

B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ma, Xinyu Guo, Jiafeng Zhang, Ruqing Fan, Yixing Li, Yingyan Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Pre-training and fine-tuning have achieved remarkable success in many downstream natural language processing (NLP) tasks. Recently, pre-training methods tailored for information retrieval (IR) have also been explored, and the latest success is the PROP method which has reached new SOTA on a variety of ad-hoc retrieval benchmarks. The basic idea of PROP is to construct the representative words prediction (ROP) task for pre-training inspired by the query likelihood model. Despite its exciting performance, the effectiveness of PROP might be bounded by the classical unigram language model adopted in the ROP task construction process. To tackle this problem, we propose a bootstrapped pre-training method (namely B-PROP) based on BERT for ad-hoc retrieval. The key idea is to use the powerful contextual language model BERT to replace the classical unigram language model for the ROP task construction, and re-train BERT itself towards the tailored objective for IR. Specifically, we introduce a novel contrastive method, inspired by the divergence-from-randomness idea, to leverage BERT's self-attention mechanism to sample representative words from the document. By further fine-tuning on downstream ad-hoc retrieval tasks, our method achieves significant improvements over PROP and other baselines, and further pushes forward the SOTA on a variety of ad-hoc retrieval tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： Natural language processing systems

DPER: Diffusion Prior Driven Neural Representation for Limited Angle and Sparse View CT Reconstruction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Du, Chenhe Lin, Xiyue Wu, Qing Tian, Xuanyu Su, Ying Luo, Zhe Zheng, Rui Chen, Yang Wei, Hongjiang Zhou, S. Kevin Yu, Jingyi Zhang, Yuyao The School of Information Science and Technology ShanghaiTech University Shanghai China The Department of Critical Care Medicine Zhongshan Hospital Fudan University Shanghai China The Laboratory of Image Science and Technology The School of Computer Science and Engineering The Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications Southeast University Ministry of Education Nanjing210096 China The School of Biomedical Engineering Institute of Medical Robotics Shanghai Jiao Tong University Shanghai China School of Biomedical Engineering Suzhou Institute for Advanced Research University of Science and Technology of China Suzhou215123 China Institute of Computing Technology CAS Beijing100190 China

Limited-angle and sparse-view computed tomography (LACT and SVCT) are crucial for expanding the scope of X-ray CT applications. However, they face challenges due to incomplete data acquisition, resulting in diverse artifacts in the reconstructed CT images. Emerging implicit neural representation (INR) techniques, such as NeRF, NeAT, and NeRP, have shown promise in under-determined CT imaging reconstruction tasks. However, the unsupervised nature of INR architecture imposes limited constraints on the solution space, particularly for the highly ill-posed reconstruction task posed by LACT and ultra-SVCT. In this study, we introduce the Diffusion Prior Driven Neural Representation (DPER), an advanced unsupervised framework designed to address the exceptionally ill-posed CT reconstruction inverse problems. DPER adopts the Half Quadratic Splitting (HQS) algorithm to decompose the inverse problem into data fidelity and distribution prior sub-problems. The two sub-problems are respectively addressed by INR reconstruction scheme and pre-trained score-based diffusion model. This combination first injects the implicit image local consistency prior from INR. Additionally, it effectively augments the feasibility of the solution space for the inverse problem through the generative diffusion model, resulting in increased stability and precision in the solutions. We conduct comprehensive experiments to evaluate the performance of DPER on LACT and ultra-SVCT reconstruction with two public datasets (AAPM and LIDC), an in-house clinical COVID-19 dataset and a public raw projection dataset created by Mayo Clinic. The results show that our method outperforms the state-of-the-art reconstruction methods on in-domain datasets, while achieving significant performance improvements on out-of-domain (OOD) datasets. Copyright © 2024, The Authors. All rights reserved.

关键词： Inverse problems

Text Information Mining in Cyberspace: An Information Extraction Method Based on T5 and keyBERT

学校读者我要写书评

暂无评论

Text Information Mining in Cyberspace: An Information Extrac...

IEEE International Conference on data science in Cyberspace (DSC)

作者： Kuan Liu Yumei Li Yuanhua Qi Ning Qi Mengran Zhai Information Research Institute of Shandong Academy of Sciences Qilu University of Technology (Shandong Academy of Sciences) Jinan China Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center (National Supercomputer Center in Jinan) Qilu University of Technology (Shandong Academy of Sciences) Shandong Institute of Economy and Informatization Development Jinan China Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center (National Supercomputer Center in Jinan) Qilu University of Technology (Shandong Academy of Sciences) Shandong Provincial Key Laboratory of Computer Networks Shandong Fundamental Research Center for Computer Science Jinan China SHANDONG SCICOM Information and Economy Research Institute Co. Ltd Jinan China Shandong Institute of Economy and Informatization Development Jinan China

ISBN: (数字)9798350391367

ISBN: (纸本)9798350391374

With the development of the digital economy and the advent of the big data era, the rapid growth of textual information in cyberspace has placed higher demands on information processing technology. This paper proposes an information extraction technique based on the Text-to-Text Transfer Transformer, known as T5, and keyBERT models, aiming to efficiently distill the key content from textual information in cyberspace. This method combines automatic summarization and keyword extraction to form information briefs. The experiments selected representative policy documents related to data elements as input data and employed information entropy and ROUGE scores as metrics to evaluate the automatic summarization model. The results were compared with the outputs of similar models. Experimental results indicate that the T5 model outperforms other models in terms of summarization effectiveness, and the proposed information extraction method shows significant advantages in readability and processing efficiency, demonstrating practical application value.

关键词： Measurement Biological system modeling Cyberspace Text summarization Information processing Information retrieval Transformers data models data mining Information entropy

从信号到知识——基于人工智能的医学影像裸数据诊断价值初探

学校读者我要写书评

暂无评论

Engineering 2024年第3期34卷 60-69页

作者： Bingxi He Yu Guo Yongbei Zhu Lixia Tong Boyu Kong Kun Wang Caixia Sun Hailin Li Feng Huang Liwei Wu Meng Wang Fanyang Meng Le Dou Kai Sun Tong Tong Zhenyu Liu Ziqi Wei Wei Mu Shuo Wang Zhenchao Tang Shuaitong Zhang Jingwei Wei Lizhi Shao Mengjie Fang Juntao Li Shouping Zhu Lili Zhou Shuo Wang Di Dong Huimao Zhang Jie Tian Beijing Advanced Innovation Center for Big Data-Based Precision Medicine School of Engineering MedicineBeihang UniversityBeijing 100191China Key Laboratory of Big Data-Based Precision Medicine Beihang UniversityMinistry of Industry and Information TechnologyBeijing 100191China CAS Key Laboratory of Molecular Imaging Institute of AutomationChinese Academy of SciencesBeijing 100190China Department of Radiology The First Hospital of Jilin UniversityChangchun 130021China Neusoft Medical Systems Co.Ltd. Shenyang 110167China Neusoft Research of Intelligent Healthcare Technology Co.Ltd. Shenyang 110167China School of Life Science and Technology Xidian UniversityXi’an 710126China Engineering Research Center of Molecular and Neuro Imaging Ministry of EducationXi’an 710126China School of Artificial Intelligence University of Chinese Academy of SciencesBeijing 100049China

Encouraging and astonishing developments have recently been achieved in image-based diagnostic *** medical care and imaging technology are becoming increasingly ***,the current diagnosis pattern of signal to image to knowledge inevitably leads to information distortion and noise introduction in the procedure of image reconstruction(from signal to image).Artificial intelligence(AI)technologies that can mine knowledge from vast amounts of data offer opportunities to disrupt established *** this prospective study,for the first time,we develop an AI-based signal-toknowledge diagnostic scheme for lung nodule classification directly from the computed tomography(CT)raw data(the signal).We find that the raw data achieves almost comparable performance with CT,indicating that it is possible to diagnose diseases without reconstructing ***,the incorporation of raw data through three common convolutional network structures greatly improves the performance of the CT models in all cohorts(with a gain ranging from 0.01 to 0.12),demonstrating that raw data contains diagnostic information that CT does not *** results break new ground and demonstrate the potential for direct signal-to-knowledge domain analysis.

关键词： Computed tomography Diagnosis Deep learning Lung cancer Raw data