检索结果-内蒙古大学图书馆

Prompt Engineering 101 Prompt Engineering Guidelines from a Linguistic Perspective 23rd

学校读者我要写书评

暂无评论

Prompt Engineering 101 Prompt Engineering Guidelines from a...

23rd China National Conference on Computational Linguistics, CCL 2024

作者： Han, Wenjuan Wei, Xiang Cui, Xingyu Cheng, Ning Jiang, Guangyuan Qian, Weinan Zhang, Chi Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing China Peking University Beijing China

ISBN: (纸本)9789819783663

Deploying tuning-free prompting is challenging in engineering practice: it not only requires users to engage in cumbersome trials and errors but is also extremely time-consuming, as even a slight change in wording and phrasing could have a huge impact on the final performance. To further investigate the impact of different prompts, in this work, we perform a systematic inspection of four factors in linguistics involved in prompt engineering: syntax, semantics, lexicon, and pragmatics. The empirical results quantify the sensitivity of the output to small textual perturbations in four linguistic factors of prompts. Based on the analysis of these four factors, we present a series of design guidelines to help human users write effective prompts. Human evaluation on amateurs shows that using the proposed guidelines helps humans produce prompts with significant gains in zero-shot performance in Pre-trained Language Models (PLMs) and hence validates the utility of the guidelines. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Semantics

Enhancing Rumor Detection Methods with Propagation Structure Infused Language Model 31

学校读者我要写书评

暂无评论

Enhancing Rumor Detection Methods with Propagation Structure...

31st International Conference on Computational Linguistics, COLING 2025

作者： Cui, Chaoqun Li, Siyuan Ma, Kunkun Jia, Caiyan School of Computer Science and Technology & Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing100044 China

ISBN: (纸本)9798891761964

Pretrained Language Models (PLMs) have excelled in various Natural Language Processing tasks, benefiting from large-scale pretraining and self-attention mechanism's ability to capture long-range dependencies. However, their performance on social media application tasks like rumor detection remains suboptimal. We attribute this to mismatches between pretraining corpora and social texts, inadequate handling of unique social symbols, and pretraining tasks ill-suited for modeling user engagements implicit in propagation structures. To address these issues, we propose a continue pretraining strategy called Post Engagement Prediction (PEP) to infuse information from propagation structures into PLMs. PEP makes models to predict root, branch, and parent relations between posts, capturing interactions of stance and sentiment crucial for rumor detection. We also curate and release large-scale Twitter corpus: TwitterCorpus (269GB text), and two unlabeled claim conversation datasets with propagation structures (UTwitter and UWeibo). Utilizing these resources and PEP strategy, we train a Twitter-tailored PLM called SoLM. Extensive experiments demonstrate PEP significantly boosts rumor detection performance across universal and social media PLMs, even in few-shot scenarios. On benchmark datasets, PEP enhances baseline models by 1.0-3.7% accuracy, even enabling it to outperform current state-of-the-art methods on multiple datasets. SoLM alone, without high-level modules, also achieves competitive results, highlighting the strategy's effectiveness in learning discriminative post interaction features. © 2025 Association for Computational Linguistics.

关键词： Tweets

Towards Real-World Rumor Detection: Anomaly Detection Framework with Graph Supervised Contrastive Learning 31

学校读者我要写书评

暂无评论

Towards Real-World Rumor Detection: Anomaly Detection Framew...

31st International Conference on Computational Linguistics, COLING 2025

作者： Cui, Chaoqun Jia, Caiyan School of Computer Science and Technology & Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing100044 China

ISBN: (纸本)9798891761964

Current rumor detection methods based on propagation structure learning predominately treat rumor detection as a class-balanced classification task on limited labeled data. However, real-world social media data exhibits an imbalanced distribution with a minority of rumors among massive regular posts. To address the data scarcity and imbalance issues, we construct two large-scale conversation datasets from Weibo and Twitter and analyze the domain distributions. We find obvious differences between rumor and non-rumor distributions, with non-rumors mostly in entertainment domains while rumors concentrate in news, indicating the conformity of rumor detection to an anomaly detection paradigm. Correspondingly, we propose the Anomaly Detection framework with Graph Supervised Contrastive Learning (AD-GSCL). It heuristically treats unlabeled data as non-rumors and adapts graph contrastive learning for rumor detection. Extensive experiments demonstrate AD-GSCL's superiority under class-balanced, imbalanced, and few-shot conditions. Our findings provide valuable insights for real-world rumor detection featuring imbalanced data distributions. © 2025 Association for Computational Linguistics.

关键词： Contrastive Learning

Multilingual Knowledge Editing with Language-Agnostic Factual Neurons 31

学校读者我要写书评

暂无评论

Multilingual Knowledge Editing with Language-Agnostic Factua...

31st International Conference on Computational Linguistics, COLING 2025

作者： Zhang, Xue Liang, Yunlong Meng, Fandong Zhang, Songming Chen, Yufeng Xu, Jinan Zhou, Jie Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing China Pattern Recognition Center WeChat AI Tencent Inc China

ISBN: (纸本)9798891761964

Multilingual knowledge editing (MKE) aims to simultaneously update factual knowledge across multiple languages within large language models (LLMs). Previous research indicates that the same knowledge across different languages within LLMs exhibits a degree of shareability. However, most existing MKE methods overlook the connections of the same knowledge between different languages, resulting in knowledge conflicts and limited edit performance. To address this issue, we first investigate how LLMs process multilingual factual knowledge and discover that the same factual knowledge in different languages generally activates a shared set of neurons, which we call language-agnostic factual neurons (LAFNs). These neurons represent the same factual knowledge shared across languages and imply the semantic connections among multilingual knowledge. Inspired by this finding, we propose a new MKE method by Locating and Updating Language-Agnostic Factual Neurons (LU-LAFNs) to edit multilingual knowledge simultaneously, which avoids knowledge conflicts and thus improves edit performance. Experimental results on Bi-ZsRE and MzsRE benchmarks demonstrate that our method achieves the best edit performance, indicating the effectiveness and importance of modeling the semantic connections among multilingual knowledge. © 2025 Association for Computational Linguistics.

关键词： Neurons

WTU-EVAL: A Whether-or-Not Tool Usage Evaluation Benchmark for Large Language Models

学校读者我要写书评

暂无评论

WTU-EVAL: A Whether-or-Not Tool Usage Evaluation Benchmark f...

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Liu, Jian Ning, Kangyun Su, Yisong Han, Wenjuan Xu, Jinan Zhang, Yuanzhe University of Science and Technology Beijing China Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing China College of Computer and Data Science Fuzhou University Fuzhou China National Science Library Chinese Academy of Sciences Beijing China

ISBN: (纸本)9798350368741

Although Large Language Models (LLMs) excel in NLP tasks, they still need external tools to extend their ability. Current research on tool learning with LLMs often assumes mandatory tool use, which does not always align with real-world situations, where the necessity for tools is uncertain, and incorrect or unnecessary use of tools can damage the general abilities of LLMs. Therefore, we propose to explore whether LLMs can discern their ability boundaries and use tools flexibly. We then introduce the Whether-or-not tool usage Evaluation benchmark (WTU-Eval) to assess LLMs with eleven datasets, where six of them are tool-usage datasets, and five are general datasets. LLMs are prompted to use tools according to their needs. The results of eight LLMs on WTU-Eval reveal that LLMs frequently struggle to determine tool use in general datasets, and LLMs' performance in tool-usage datasets improves when their ability is similar to ChatGPT. © 2025 IEEE.

关键词： Ability Boundary Evaluation Large Language Models Tool Usage

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Songming Zhang, Xue Zhang, Tong Hu, Bojie Chen, Yufeng Xu, Jinan Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing China Tencent Inc China

In modern large language models (LLMs), LLM alignment is of crucial importance and is typically achieved through methods such as reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO). However, in most existing methods for LLM alignment, all tokens in the response are optimized using a sparse, response-level reward or preference annotation. The ignorance of token-level rewards may erroneously punish high-quality tokens or encourage low-quality tokens, resulting in suboptimal performance and slow convergence speed. To address this issue, we propose AlignDistil, an RLHF-equivalent distillation method for token-level reward optimization. Specifically, we introduce the reward learned by DPO into the RLHF objective and theoretically prove the equivalence between this objective and a token-level distillation process, where the teacher distribution linearly combines the logits from the DPO model and a reference model. On this basis, we further bridge the accuracy gap between the reward from the DPO model and the pure reward model, by building a contrastive DPO reward with a normal and a reverse DPO model. Moreover, to avoid under- and over-optimization on different tokens, we design a token adaptive logit extrapolation mechanism to construct an appropriate teacher distribution for each token. Experimental results demonstrate the superiority of our AlignDistil over existing methods and showcase fast convergence due to its token-level distributional reward optimization. © 2025, CC BY-NC-ND.

关键词： Reinforcement learning

Debiased Prompt Tuning in Vision-Language Model without Annotations

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Jiang, Chaoquan Yang, Yunfan Hu, Rui Sang, Jitao School of Computer and Information Technology Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University China

Prompt tuning of Vision-Language Models (VLMs) such as CLIP, has demonstrated the ability to rapidly adapt to various downstream tasks. However, recent studies indicate that tuned VLMs may suffer from the problem of spurious correlations, where the model relies on spurious features (e.g. background and gender) in the data. This may lead to the model having worse robustness in out-of-distribution data. Standard methods for eliminating spurious correlation typically require us to know the spurious attribute labels of each sample, which is hard in the real world. In this work, we explore improving the group robustness of prompt tuning in VLMs without relying on manual annotation of spurious features. We notice the zero - shot image recognition ability of VLMs and use this ability to identify spurious features, thus avoiding the cost of manual annotation. By leveraging pseudo-spurious attribute annotations, we further propose a method to automatically adjust the training weights of different groups. Extensive experiments show that our approach efficiently improves the worst-group accuracy on CelebA, Waterbirds, and MetaShift datasets, achieving the best robustness gap between the worst-group accuracy and the overall accuracy. Copyright © 2025, The Authors. All rights reserved.

关键词： Visual languages

Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Yijie Liu, Yijin Meng, Fandong Chen, Yufeng Xu, Jinan Zhou, Jie Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing China Pattern Recognition Center WeChat AI Tencent Inc China

Knowledge Distillation (KD) has emerged as a prominent technique for model compression. However, conventional KD approaches primarily focus on homogeneous architectures with identical tokenizers, constraining their applicability in cross-architecture scenarios. As for the cross-tokenizer KD, the differences in the tokenizers give rise to two fundamental challenges: (1) sequence misalignment caused by divergent tokenization strategies, and (2) mismatched vocabulary size and composition. While existing probability-matching methods attempt to address these issues, their efficacy remains limited due to suboptimal alignment in both the sequence and vocabulary aspects. To overcome these limitations, we propose Contextual Dynamic Mapping (CDM), a novel cross-tokenizer distillation framework that employs contextual information to enhance sequence alignment precision and dynamically improves vocabulary mapping. We evaluated the effectiveness of our approach across five advanced and widely-used model families (i.e, LLama3, Phi3, Gemma2, OPT and Qwen2), which were configured into three distinct teacher-student pairs. Our method shows significant advantages over existing cross-tokenizer distillation baselines across diverse benchmarks, including instruction-following, code generation and math. Notably, our analysis reveals that combining conventional same-tokenizer distillation and cross-tokenizer distillation through CDM yields further performance improvements. The code is available at https://***/pppa2019/ContexualDynamicMapping Copyright © 2025, The Authors. All rights reserved.

关键词： Teaching

Enhancing LLM Generation with Knowledge Hypergraph for Evidence-Based Medicine

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Dou, Chengfeng Zhang, Ying Jin, Zhi Jiao, Wenpin Zhao, Haiyan Zhao, Yongqiang Tao, Zhengwei School of Computer Science Peking University China MOE China Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing China

Evidence-based medicine (EBM) plays a crucial role in the application of large language models (LLMs) in healthcare, as it provides reliable support for medical decision-making processes. Although it benefits from current retrieval-augmented generation (RAG) technologies, it still faces two significant challenges: the collection of dispersed evidence and the efficient organization of this evidence to support the complex queries necessary for EBM. To tackle these issues, we propose using LLMs to gather scattered evidence from multiple sources and present a knowledge hypergraph-based evidence management model to integrate these evidence while capturing intricate relationships. Furthermore, to better support complex queries, we have developed an Importance-Driven Evidence Prioritization (IDEP) algorithm that utilizes the LLM to generate multiple evidence features, each with an associated importance score, which are then used to rank the evidence and produce the final retrieval results. Experimental results from six datasets demonstrate that our approach outperforms existing RAG techniques in application domains of interest to EBM, such as medical quizzing, hallucination detection, and decision support. Testsets and the constructed knowledge graph can be accessed at https://***/rag4ebm. © 2025, CC BY-NC-ND.

关键词： Knowledge graph