检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Mei, Lingrui Liu, Shenghua Wang, Yiwei Bi, Baolong Yuan, Ruibin Cheng, Xueqi CAS Key Laboratory of AI Safety Institute of Computing Technology CAS China University of Chinese Academy of Sciences China University of California Merced United States HKUST Hong Kong

As Large Language Models (LLMs) grow increasingly powerful, ensuring their safety and alignment with human values remains a critical challenge. Ideally, LLMs should provide informative responses while avoiding the disclosure of harmful or sensitive information. However, current alignment approaches, which rely heavily on refusal strategies—such as training models to completely reject harmful prompts or applying coarse filters—are limited by their binary nature. These methods either fully deny access to information or grant it without sufficient nuance, leading to overly cautious responses or failures to detect subtle harmful content. For example, LLMs may refuse to provide basic, public information about medication due to misuse concerns. Moreover, these refusal-based methods struggle to handle mixed-content scenarios and lack the ability to adapt to context-dependent sensitivities, which can result in over-censorship of benign content. To overcome these challenges, we introduce HIDDENGUARD, a novel framework for fine-grained, safe generation in LLMs. HIDDENGUARD incorporates PRISM (Representation Router for In-Stream Moderation), which operates alongside the LLM to enable real-time, token-level detection and redaction of harmful content by leveraging intermediate hidden states. This fine-grained approach allows for more nuanced, context-aware moderation, enabling the model to generate informative responses while selectively redacting or replacing sensitive information, rather than outright refusal. We also contribute a comprehensive dataset with token-level fine-grained annotations of potentially harmful information across diverse contexts. Our experiments demonstrate that HIDDENGUARD achieves over 90% in F1 score for detecting and redacting harmful content while preserving the overall utility and informativeness of the model’s responses. Our code is available at https://***/Meirtz/HiddenGuard. © 2024, CC BY.

关键词： Information hiding

来源：评论

学校读者我要写书评

暂无评论

Qsnail: A Questionnaire Dataset for Sequential Question Generation

arXiv

引用

arXiv 2024年

作者： Lei, Yan Pang, Liang Wang, Yuanzhuo Shen, Huawei Cheng, Xueqi CAS Key Laboratory of AI Safety & Security Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

The questionnaire is a professional research methodology used for both qualitative and quantitative analysis of human opinions, preferences, attitudes, and behaviors. However, designing and evaluating questionnaires demands significant effort due to their intricate and complex structure. Questionnaires entail a series of questions that must conform to intricate constraints involving the questions, options, and overall structure. Specifically, the questions should be relevant and specific to the given research topic and intent. The options should be tailored to the questions, ensuring they are mutually exclusive, completed, and ordered sensibly. Moreover, the sequence of questions should follow a logical order, grouping similar topics together. As a result, automatically generating questionnaires presents a significant challenge and this area has received limited attention primarily due to the scarcity of high-quality datasets. To address these issues, we present Qsnail, the first dataset specifically constructed for the questionnaire generation task, which comprises 13,168 human-written questionnaires gathered from online platforms. We further conduct experiments on Qsnail, and the results reveal that retrieval models and traditional generative models do not fully align with the given research topic and intents. Large language models, while more closely related to the research topic and intents, exhibit significant limitations in terms of diversity and specificity. Despite enhancements through the chain-of-thought prompt and finetuning, questionnaires generated by language models still fall short of human-written questionnaires. Therefore, questionnaire generation is challenging and needs to be further explored. The dataset is available at: https://***/LeiyanGithub/qsnail. Copyright © 2024, The Authors. All rights reserved.

关键词： Large datasets

来源：评论

学校读者我要写书评

暂无评论

Plot Retrieval as an Assessment of Abstract Semantic Association

arXiv

引用

arXiv 2023年

作者： Xu, Shicheng Pang, Liang Li, Jiangnan Yu, Mo Meng, Fandong Shen, Huawei Cheng, Xueqi Zhou, Jie CAS Key Laboratory of AI Security Institute of Computing Technology CAS China Pattern Recognition Center WeChat AI China

Retrieving relevant plots from the book for a query is a critical task, which can improve the reading experience and efficiency of readers. Readers usually only give an abstract and vague description as the query based on their own understanding, summaries, or speculations of the plot, which requires the retrieval model to have a strong ability to estimate the abstract semantic associations between the query and candidate plots. However, existing information retrieval (IR) datasets cannot reflect this ability well. In this paper, we propose PLOTRETRIEVAL, a labeled dataset to train and evaluate the performance of IR models on the novel task Plot Retrieval. Text pairs in PLOTRETRIEVAL have less word overlap and more abstract semantic association, which can reflect the ability of the IR models to estimate the abstract semantic association, rather than just traditional lexical or semantic matching. Extensive experiments across various lexical retrieval, sparse retrieval, dense retrieval, and cross-encoder methods compared with human studies on PLOTRETRIEVAL show current IR models still struggle in capturing abstract semantic association between texts. PLOTRETRIEVAL can be the benchmark for further research on the semantic association modeling ability of IR models. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Classifier Guidance Enhances Diffusion-based Adversarial Purification by Preserving Predictive Information

arXiv

引用

arXiv 2024年

作者： Zhang, Mingkun Li, Jianing Chen, Wei Guo, Jiafeng Cheng, Xueqi CAS Key Laboratory of AI Safety Institute of Computing Technology Chinese Academy of Sciences Beijing China Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Adversarial purification is one of the promising approaches to defend neural networks against adversarial attacks. Recently, methods utilizing diffusion probabilistic models have achieved great success for adversarial purification in image classification tasks. However, such methods fall into the dilemma of balancing the needs for noise removal and information preservation. This paper points out that existing adversarial purification methods based on diffusion models gradually lose sample information during the core denoising process, causing occasional label shift in subsequent classification tasks. As a remedy, we suggest to suppress such information loss by introducing guidance from the classifier confidence. Specifically, we propose Classifier-cOnfidence gUided Purification (COUP) algorithm, which purifies adversarial examples while keeping away from the classifier decision boundary. Experimental results show that COUP can achieve better adversarial robustness under strong attack methods. Copyright © 2024, The Authors. All rights reserved.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

Unlocking the Power of Large Language Models for Entity Alignment

arXiv

引用

arXiv 2024年

作者： Jiang, Xuhui Shen, Yinghan Shi, Zhichao Xu, Chengjin Li, Wei Li, Zixuan Guo, Jian Shen, Huawei Wang, Yuanzhuo CAS Key Laboratory of AI Safety Institute of Computing Technology CAS China School of Computer Science and Technology University of Chinese Academy of Science China IDEA Research International Digital Economy Academy China

Entity Alignment (EA) is vital for integrating diverse knowledge graph (KG) data, playing a crucial role in data-driven ai applications. Traditional EA methods primarily rely on comparing entity embeddings, but their effectiveness is constrained by the limited input KG data and the capabilities of the representation learning techniques. Against this backdrop, we introduce ChatEA, an innovative framework that incorporates large language models (LLMs) to improve EA. To address the constraints of limited input KG data, ChatEA introduces a KG-code translation module that translates KG structures into a format understandable by LLMs, thereby allowing LLMs to utilize their extensive background knowledge to improve EA accuracy. To overcome the over-reliance on entity embedding comparisons, ChatEA implements a two-stage EA strategy that capitalizes on LLMs’ capability for multi-step reasoning in a dialogue format, thereby enhancing accuracy while preserving efficiency. Our experimental results verify ChatEA’s superior performance, highlighting LLMs’ potential in facilitating EA tasks. The source code is available at https://***/IDEA-Finai/ChatEA. Copyright © 2024, The Authors. All rights reserved.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

CROSS-MODAL safety MECHANISM TRANSFER IN LARGE VISION-LANGUAGE MODELS

arXiv

引用

arXiv 2024年

作者： Xu, Shicheng Pang, Liang Zhu, Yunchang Shen, Huawei Cheng, Xueqi CAS Key Laboratory of AI Safety Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Huawei Inc. China

Content warning: This paper contains harmful images and texts! Vision-language alignment in Large Vision-Language Models (LVLMs) successfully enables LLMs to understand visual input. However, we find that existing vision-language alignment methods fail to transfer the existing safety mechanism for text in LLMs to vision, which leads to vulnerabilities in toxic image. To explore the cause of this problem, we give the insightful explanation of where and how the safety mechanism of LVLMs operates and conduct comparative analysis between text and vision. We find that the hidden states at the specific transformer layers play a crucial role in the successful activation of safety mechanism, while the vision-language alignment at hidden states level in current methods is insufficient. This results in a semantic shift for input images compared to text in hidden states, therefore misleads the safety mechanism. To address this, we propose a novel Text-Guided vision-language Alignment method (TGA) for LVLMs. TGA retrieves the texts related to input vision and uses them to guide the projection of vision into the hidden states space in LLMs. Experiments show that TGA not only successfully transfers the safety mechanism for text in basic LLMs to vision in vision-language alignment for LVLMs without any safety fine-tuning on the visual modality but also maintains the general performance on various vision tasks. Code is available1 Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration

arXiv

引用

arXiv 2024年

作者： Dai, Sunhao Liu, Weihao Zhou, Yuqi Pang, Liang Ruan, Rongju Wang, Gang Dong, Zhenhua Xu, Jun Wen, Ji-Rong Gaoling School of Artificial Intelligence Renmin University of China China CAS Key Laboratory of AI Safety Institute of Computing Technology CAS China Huawei Noah’s Ark Lab Hong Kong

The proliferation of Large Language Models (LLMs) has led to an influx of ai-generated content (aiGC) on the internet, transforming the corpus of Information Retrieval (IR) systems from solely human-written to a coexistence with LLM-generated content. The impact of this surge in aiGC on IR systems remains an open question, with the primary challenge being the lack of a dedicated benchmark for researchers. In this paper, we introduce Cocktail, a comprehensive benchmark tailored for evaluating IR models in this mixed-sourced data landscape of the LLM era. Cocktail consists of 16 diverse datasets with mixed human-written and LLM-generated corpora across various text retrieval tasks and domains. Additionally, to avoid the potential bias from previously included dataset information in LLMs, we also introduce an up-to-date dataset, named NQ-UTD, with queries derived from recent events. Through conducting over 1,000 experiments to assess state-of-the-art retrieval models against the benchmarked datasets in Cocktail, we uncover a clear trade-off between ranking performance and source bias in neural retrieval models, highlighting the necessity for a balanced approach in designing future IR systems. We hope Cocktail can serve as a foundational resource for IR research in the LLM era, with all data and code publicly available at https://***/KID-22/Cocktail. © 2024, CC BY-NC-ND.

关键词： Content based retrieval

来源：评论

学校读者我要写书评

暂无评论

Invisible Relevance Bias: Text-Image Retrieval Models Prefer ai-Generated Images

arXiv

引用

arXiv 2023年

作者： Xu, Shicheng Deng, Jingcheng Hou, Danyang Xu, Jun Pang, Liang Shen, Huawei Cheng, Xueqi CAS Key Laboratory of AI Safety Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences Beijing China Gaoling School of Artificial Intelligence Renmin University of China Beijing China CAS Key Laboratory of AI Safety Institute of Computing Technology Chinese Academy of Sciences Beijing China

With the application of generation models, internet is increasingly inundated with ai-generated content (aiGC), causing both real and ai-generated content indexed in corpus for search. This paper explores the impact of ai-generated images on text-image search in this scenario. Firstly, we construct a benchmark consisting of both real and ai-generated images for this study. In this benchmark, ai-generated images possess visual semantics sufficiently similar to real images. Experiments on this benchmark reveal that text-image retrieval models tend to rank the ai-generated images higher than the real images, even though the ai-generated images do not exhibit more visually relevant semantics to the queries than real images. We call this bias as invisible relevance bias. This bias is detected across retrieval models with different training data and architectures. Further exploration reveals that mixing ai-generated images into the training data of retrieval models exacerbates the invisible relevance bias. These problems cause a vicious cycle in which ai-generated images have a higher chance of exposing from massive data, which makes them more likely to be mixed into the training of retrieval models and such training makes the invisible relevance bias more and more serious. To mitigate this bias and elucidate the potential causes of the bias, firstly, we propose an effective method to alleviate this bias. Subsequently, we apply our proposed debiasing method to retroactively identify the causes of this bias, revealing that the ai-generated images induce the image encoder to embed additional information into their representation. This information makes the retriever estimate a higher relevance score. We conduct experiments to support this assertion. Findings in this paper reveal the potential impact of ai-generated images on retrieval and have implications for further research. Code is released at https://***/xsc1234/Invisible-Relevance-Bias. Copyright © 2023, The Au

关键词： Image retrieval

来源：评论

学校读者我要写书评

暂无评论

A Survey on Large Language Model Hallucination via a Creativity Perspective

arXiv

引用

arXiv 2024年

作者： Jiang, Xuhui Tian, Yuxing Hua, Fengrui Xu, Chengjin Wang, Yuanzhuo Guo, Jian CAS Key Laboratory of AI Safety & Security Institute of Computing Technology CAS China School of Computer Science and Technology University of Chinese Academy of Science China International Digital Economy Academy IDEA Research China

Hallucinations in large language models (LLMs) are always seen as limitations. However, could they also be a source of creativity? This survey explores this possibility, suggesting that hallucinations may contribute to LLM application by fostering creativity. This survey begins with a review of the taxonomy of hallucinations and their negative impact on LLM reliability in critical applications. Then, through historical examples and recent relevant theories, the survey explores the potential creative benefits of hallucinations in LLMs. To elucidate the value and evaluation criteria of this connection, we delve into the definitions and assessment methods of creativity. Following the framework of divergent and convergent thinking phases, the survey systematically reviews the literature on transforming and harnessing hallucinations for creativity in LLMs. Finally, the survey discusses future research directions, emphasizing the need to further explore and refine the application of hallucinations in creative processes within LLMs. Copyright © 2024, The Authors. All rights reserved.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

Agent-SiMT: Agent-Assisted Simultaneous Translation With Large Language Models

IEEE Transactions on Audio, Speech and Language Processing

引用

IEEE Transactions on Audio, Speech and Language Processing 2025年 33卷 2074-2083页

作者： Shoutao Guo Shaolei Zhang Zhengrui Ma Min Zhang Yang Feng Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences (ICT/CAS) Beijing China University of Chinese Academy of Sciences Beijing China School of Computer Science and Technology Soochow University Suzhou China Key Laboratory of AI Safety Chinese Academy of Sciences Beijing China

Simultaneous Machine Translation (SiMT) generates target translations in real-time while reading the source sentence. It relies on a policy to determine the optimal timing for producing translations, aiming to achieve both low response latency and high translation quality. Existing SiMT methods typically utilize the traditional Transformer architecture, employing complicated dynamic programming algorithms to jointly optimize policy decisions and translation generation. While these methods excel at determining policies, their translation capabilities are suboptimal. In contrast, Large Language Models (LLMs), trained on extensive corpora, exhibit exceptional generation capabilities but struggle to learn translation policies through traditional Transformer-based training methods. To address these limitations, we introduce Agent-SiMT, a novel framework combining the strengths of LLMs and traditional SiMT methods. Agent-SiMT contains a policy-decision agent and a translation agent. The policy-decision agent is managed by a SiMT model, which determines the policy using partial source sentences and translations. The translation agent, powered by an LLM, generates the target translation according to the policy decisions. These two agents work collaboratively to perform SiMT. Experiments demonstrate that our method attains state-of-the-art performance.

关键词： Translation Hidden Markov models Transformers Training Large language models Collaboration Vocabulary Speech processing Memory modules Machine translation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：