The dense retrieval model offers remarkable capabilities, yet it exhibits inconsistencies in the embedding space of queries and documents due to its dual-encoder structure. Addressing this limitation, we introduce Pse...
ISBN:
(纸本)9789819794300;9789819794317
The dense retrieval model offers remarkable capabilities, yet it exhibits inconsistencies in the embedding space of queries and documents due to its dual-encoder structure. Addressing this limitation, we introduce Pseudo-query Embedding (PqE), a document expansion approach that eliminates the need for supervised data. By zero-shot prompting large language models (LLMs), we generate a specific number of pseudo-queries for each document, which are used to mitigate inconsistencies in the embeddings between queries and documents. This innovative strategy employs a multi-stage retrieval process to expand documents, enhancing the performance of the dense retrieval model without unduly impacting retrieval time. Experimental results demonstrate the efficacy of PqE. On the TREC DL dataset, PqE enhances the nDCG@10 metric of the unsupervised dense retrieval model Contriever by 7% points, and the Recall@1k metric by 4% points, surpassing the performance of the BM25 algorithm. Even for contrieverFT, fine-tuned on the massive dataset MS-MARCO, and BGE, trained on hundreds of millions of query-document pairs, PqE boosts their Recall@1k metrics on the TREC DL dataset by 1 to 2% points. Notably, on out-of-domain datasets from BEIR, PqE elevates the performance of most models across all metrics.
Automatically generating scientific literature surveys is a valuable task that can significantly enhance research efficiency. However, the diverse and complex nature of information within a literature survey poses sub...
ISBN:
(纸本)9789819794423;9789819794430
Automatically generating scientific literature surveys is a valuable task that can significantly enhance research efficiency. However, the diverse and complex nature of information within a literature survey poses substantial challenges for generative models. In this paper, we design a series of prompts to systematically leverage large language models (LLMs), enabling the creation of comprehensive literature surveys through a step-by-step approach. Specifically, we design prompts to guide LLMs to sequentially generate the title, abstract, hierarchical headings, and the main content of the literature survey. We argue that this design enables the generation of the headings from a high-level perspective. During the content generation process, this design effectively harnesses relevant information while minimizing costs by restricting the length of both input and output content in LLM queries. Our implementation with Qwen-long achieved third place in the NLPCC 2024 Scientific Literature Survey Generation evaluation task, with an overall score only 0.03% lower than the second-place team. Additionally, our soft heading recall is 95.84%, the second best among the submissions. Thanks to the efficient prompt design and the low cost of the Qwen-long API, our method reduces the expense for generating each literature survey to 0.1 RMB, enhancing the practical value of our method.
Document-level Event Argument Extraction (DEAE) is a highly challenging subtask of information extraction, especially in news scenarios where the event structure is complex. Most of the current methods are entity-base...
ISBN:
(纸本)9789819794331;9789819794348
Document-level Event Argument Extraction (DEAE) is a highly challenging subtask of information extraction, especially in news scenarios where the event structure is complex. Most of the current methods are entity-based classification or generative frameworks, facing significant challenges when dealing with argument types that are not entities and handling complex event types. In this paper, we propose an iterative extraction framework for DEAE, which simulates human reading habits to iterate documents sentence by sentence. We utilize a long-term memory to effectively capture and utilize document context during iteration, compensating for the model's limited global information. To assist the model in understanding the complex events during iteration, the extracted arguments, considered short-term information, are used to enrich a dynamic prompt for extraction. Experiments on the news dataset DocEE demonstrate that our model outperforms previous methods. The ablation study also proves the effectiveness of each module.
Speech technologies such as text-to-speech (TTS) and speech-to-text (STT) are becoming increasingly applicable. Significant improvements in their quality are driven by advancements in deep machine learning. The abilit...
ISBN:
(纸本)9783031779602;9783031779619
Speech technologies such as text-to-speech (TTS) and speech-to-text (STT) are becoming increasingly applicable. Significant improvements in their quality are driven by advancements in deep machine learning. The ability of devices to deeply understand human speech and generate appropriate responses is a hallmark of AI capabilities. Developing speech technology requires extensive speech and language resources, which is why many languages with smaller speaker bases lag behind widely spoken languages in the development of speech technologys. Prior to the deep learning (DL) paradigm, hidden Markov models (HMM) and probabilistic approaches dominated speech technology development. This paper reviews the challenges and solutions in TTS and STT development for Serbian, highlighting the transition from HMM to DL. It also explores the future prospects of speech technology development for under-resourced languages and its role in preserving these languages.
This paper presents the results of the shared task on Chinese metaphor generation, hosted at the 13th CCF Conference on Natural Language Processing and Chinese Computing (NLPCC 2024). The goal of this shared task is t...
ISBN:
(纸本)9789819794423;9789819794430
This paper presents the results of the shared task on Chinese metaphor generation, hosted at the 13th CCF Conference on Natural Language Processing and Chinese Computing (NLPCC 2024). The goal of this shared task is to generate Chinese metaphors using machine learning techniques and effectively identifying basic components of metaphorical sentences. It is divided into two subtasks: 1) Metaphor Generation, which involves creating a metaphor from a provided tuple consisting of TENOR, GROUND, and VEHICLE. The goal here is to synthesize a metaphor that connects the subject (i.e. TENOR) with the object (i.e. VEHICLE), guided by the concept of the GROUND. 2) Metaphor Components Identification, which extracts the most fitting TENORs, GROUNDs, and VEHICLEs from a metaphorical sentence. This component requires the identification of the most fitting metaphor elements that correspond to the specified grounds. In addition to overall results, we report on the setup and insights from the metaphor generation shared task, which attracted a total of 4 participating teams across both subtasks.
With the rapid growth of multi-modal data, deep cross-modal hashing algorithms provide a perfect solution for cross-modal retrieval tasks for their advantages of efficient retrieval speed and low storage consumption. ...
ISBN:
(纸本)9789819794362;9789819794379
With the rapid growth of multi-modal data, deep cross-modal hashing algorithms provide a perfect solution for cross-modal retrieval tasks for their advantages of efficient retrieval speed and low storage consumption. Currently, the existing supervised cross-modal hashing methods, in order to efficiently extract structured information from raw data, generally gather on feature extraction of global information, however, all those methods ignore the weight differentiation between foreground and background information in a image. To address the issue, we propose a novel Deep Foreground-Background Weighted Cross-Modal Hashing(DFBWH) for supervised cross-modal retrieval. Specifically, the proposed method firstly performs target detection on the original image and select out candidate regions as target foreground entities. Then, the proposed method utilize the semantic interactions in the textual descriptions and tagging information as evaluation criteria, and use CLIP to detect the matching degree of the candidate regions. Eventually, under the supervision of the category labeling information, the hash loss function is utilized to obtain a high-quality hash code. Extensive experiments were carried out on two benchmark datasets, which demonstrate that DFBWH achieves better performance than the state-of-the-art baselines.
Due to the high complexity and diversity of writing, automated essay evaluation systems face significant challenges. Large language models (LLMs), representing the latest peak in NLP technology for semantic understand...
ISBN:
(纸本)9789819794423;9789819794430
Due to the high complexity and diversity of writing, automated essay evaluation systems face significant challenges. Large language models (LLMs), representing the latest peak in NLP technology for semantic understanding, hold immense potential for advancing essay evaluation systems. In the NLPCC 2024 Shared Task 4 Chinese Essay Discourse Logic Evaluation and Integration, we investigated improving LLMs' capabilities in evaluating essay logic, coherence, and quality. Considering the characteristics of different tasks, we adopted MRC-style instructions to optimize output formats and implemented undersampling to address data imbalance. To enhance efficiency and model performance, we explored LLM fine-tuning methods that decouple tasks and applied similarity comparison to refine model outputs. Additionally, we utilized noisy embedding fine-tuning to mitigate overfitting. Our approach achieved the top ranking in the NLPCC 2024 Shared Task 4.
The professional nature and confidentiality of the power domain hinder the public to accurately assess the authenticity of online electricity-related statements, fostering an environment conducive to the spread of ele...
ISBN:
(纸本)9789819794393;9789819794409
The professional nature and confidentiality of the power domain hinder the public to accurately assess the authenticity of online electricity-related statements, fostering an environment conducive to the spread of electricity-related hate speech on social media. To address this challenge, we introduce a new hate speech detection task for the electric power domain. A dataset for electric power domain hate speech detection is constructed, consisting of 6000 electricity-related Weibo posts. We propose a prompt learning approach for hate speech detection in the electric power domain, which integrates power domain knowledge, such as work scenarios and terms. Subsequently, a prompt template is formulated to facilitate hate speech detection. Experimental results on the dataset indicate that the proposed prompt learning method surpasses the baseline model.
This technical report presents our approach for the NLPCC 2024 Shared Task 3. It outlines the task content, and our proposed methodology and introduces some related works of this task. The report provides a brief desc...
ISBN:
(纸本)9789819794423;9789819794430
This technical report presents our approach for the NLPCC 2024 Shared Task 3. It outlines the task content, and our proposed methodology and introduces some related works of this task. The report provides a brief description of the task, our methodology, and the experimental setup.
The rapid development of large-scale language models has garnered widespread interest from both academia and industry. Efficiently applying those models across various domains is now posing a challenge to researchers....
ISBN:
(纸本)9789819794362;9789819794379
The rapid development of large-scale language models has garnered widespread interest from both academia and industry. Efficiently applying those models across various domains is now posing a challenge to researchers. High training costs and the relative scarcity of domain-specific data have rendered continual learning on general pretrained language models as one preferable approach. In this paper, we provide a comprehensive analysis and modification of these continual learning strategies for large language models, as they were initially designed for encoderonly architectures. Then a probing algorithm for the token representation shift was proposed to better alleviate forgetting. Additionally, corresponding evaluation metrics were modified for quantitative analysis of our methods. Through the experiment across three different domains, we verified the effectiveness of continual learning and probing algorithms on recent models. Results showed that knowledge distillation outperforms other methods in cross-domain continual learning. Moreover, the introduction of probing can further enhance the accuracy with a relatively small calculation budget.
暂无评论