检索结果-内蒙古大学图书馆

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Gao, Hongfu Zhang, Feipeng Jiang, Wenyu Shu, Jun Zheng, Feng Wei, Hongxin Department of Statistics and Data Science Southern University of Science and Technology China School of Economics and Finance Xi'an Jiaotong University China National Key Laboratory for Novel Software Technology Nanjing University China School of Mathematics and Statistics Xi'an Jiaotong University China Department of Computer Science and Engineering Southern University of Science and Technology China

Large language models (LLMs) have shown impressive performance on downstream tasks by in-context learning (ICL), which heavily relies on the quality of demonstrations selected from a large set of annotated examples. Recent works claim that in-context learning is robust to noisy demonstrations in text classification. In this work, we show that, on text generation tasks, noisy annotations significantly hurt the performance of in-context learning. To circumvent the issue, we propose a simple and effective approach called Local Perplexity Ranking (LPR), which replaces the "noisy" candidates with their nearest neighbors that are more likely to be clean. Our method is motivated by analyzing the perplexity deviation caused by noisy labels and decomposing perplexity into inherent perplexity and matching perplexity. Our key idea behind LPR is thus to decouple the matching perplexity by performing the ranking among the neighbors in semantic space. Our approach can prevent the selected demonstrations from including mismatched input-label pairs while preserving the effectiveness of the original selection methods. Extensive experiments demonstrate the effectiveness of LPR, improving the EM score by up to 18.75 on common benchmarks with noisy annotations. Our code is available at https://***/ml-stat-Sustech/Local-Perplexity-Ranking. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning

RoTBench: A Multi-Level Benchmark for Evaluating the Robustn...

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Ye, Junjie Wu, Yilong Gao, Songyang Huang, Caishuang Li, Sixian Li, Guanyu Fan, Xiaoran Zhang, Qi Gui, Tao Huang, Xuanjing School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China

ISBN: (纸本)9798891761643

Tool learning has generated widespread interest as a vital means of interaction between Large Language Models (LLMs) and the physical world. Current research predominantly emphasizes LLMs' capacity to utilize tools in well-structured environments while overlooking their stability when confronted with the inevitable noise of the real world. To bridge this gap, we introduce RoTBench, a multi-level benchmark for evaluating the robustness of LLMs in tool learning. Specifically, we establish five external environments, each featuring varying levels of noise (i.e., Clean, Slight, Medium, Heavy, and Union), providing an in-depth analysis of the model's resilience across three critical phases: tool selection, parameter identification, and content filling. Experiments involving six widely-used models underscore the urgent necessity for enhancing the robustness of LLMs in tool learning. For instance, the performance of GPT-4 even drops significantly from 80.00 to 58.10 when there is no substantial change in manual accuracy. More surprisingly, the noise correction capability inherent in the GPT family paradoxically impedes its adaptability in the face of mild noise. In light of these findings, we propose RoTTuning, a strategy that enriches the diversity of training environments to bolster the robustness of LLMs in tool learning. The code and data are available at https://***/Junjie-Ye/RoTBench. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

PDF-to-Tree: Parsing PDF Text Blocks into a Tree

PDF-to-Tree: Parsing PDF Text Blocks into a Tree

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Zhang, Yue Zhang, Zhihao Lai, Wenbin Zhang, Chong Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China

ISBN: (纸本)9798891761681

In many PDF documents, the reading order of text blocks is missing, which can hinder machine understanding of the document's content. Existing works try to extract one universal reading order for a PDF file. However, applications, like Retrieval Augmented Generation (RAG), require breaking long articles into sections, subsections and table cells for better indexing. For this reason, this paper introduces a new task and dataset, PDF-to-Tree, which organizes the text blocks of a PDF into a tree structure. Since a PDF may contain thousands of text blocks, this paper proposes a transition-based parser that uses a greedy strategy to build the tree structure. Compared to the parser for plain text, we also use multi-modal features to encode the parser state. Experiments show that our approach achieves an accuracy of 93.93%, surpassing the performance of baseline methods by an improvement of 6.72%. The dataset is public available at https://***/yuezh000/PDFto-Tree. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

AuRoRA: A One-for-all Platform for Augmented Reasoning and Refining with Task-Adaptive Chain-of-Thought Prompting 30

AuRoRA: A One-for-all Platform for Augmented Reasoning and R...

引用

Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024

作者： Zou, Anni Zhang, Zhuosheng Zhao, Hai Li, Xiaoshan Bian, Minjie Department of Computer Science and Engineering Shanghai Jiao Tong University China School of Electronic Information and Electrical Engineering Shanghai Jiao Tong University China Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University China Shanghai Technology Innovation Department Shanghai Data Group Co. Ltd. China Shanghai Data Group Co. Ltd. China

ISBN: (纸本)9782493814104

Large language models (LLMs) empowered by chain-of-thought (CoT) prompting have yielded remarkable prowess in reasoning tasks. Nevertheless, current methods predominantly lean on handcrafted or task-specific demonstrations, lack reliable knowledge basis and thus struggle for trustworthy responses in an automated pattern. While recent works endeavor to improve upon one certain aspect, they ignore the importance and necessity of establishing an integrated and interpretable reasoning system. To address these drawbacks and provide a universal solution, we propose AuRoRA: a one-for-all platform for augmented reasoning and refining based on CoT prompting that excels in adaptability, reliability, integrity, and interpretability. The system exhibits superior performances across six reasoning tasks and offers real-time visual analysis, which has pivotal academic and application value in the era of LLMs. The AuRoRA platform is available at https://***/spaces/Anni123/AuRoRA. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

关键词： Refining

来源：评论

学校读者我要写书评

暂无评论

Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning

Improving Discriminative Capability of Reward Models in RLHF...

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Chen, Lu Zheng, Rui Wang, Binghai Jin, Senjie Huang, Caishuang Ye, Junjie Zhang, Zhihao Zhou, Yuhao Xi, Zhiheng Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Key Laboratory of Intelligent Information Processing Fudan University Shanghai China

ISBN: (纸本)9798891761643

Reinforcement Learning from Human Feedback (RLHF) is a crucial approach to aligning language models with human values and intentions. A fundamental challenge in this method lies in ensuring that the reward model accurately understands and evaluates human preferences. Current methods rely on ranking losses to teach the reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions. To address this issue, we introduce contrastive learning into the reward modeling process. In addition to supervised ranking loss, we introduce an unsupervised contrastive loss to enable the reward model to fully capture the distinctions in contrastive data. Experimental results demonstrate that the proposed contrastive learning-based reward modeling method effectively enhances the generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences. © 2024 Association for Computational Linguistics.

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

***: A Benchmark for Evaluating Questioning Capability in Education for Large Language Models

arXiv

引用

arXiv 2024年

作者： Chen, Yuyan Wu, Chenwei Yan, Songzhou Liu, Panjun Zhou, Haoyu Xiao, Yanghua Shanghai Key Laboratory of Data Science School of Computer Science Fudan University China Electrical Engineering and Computer Science Department University of Michigan United States School of Computer Science Beijing Institute of Technology China

Teachers are important to imparting knowledge and guiding learners, and the role of large language models (LLMs) as potential educators is emerging as an important area of study. Recognizing LLMs’ capability to generate educational content can lead to advances in automated and personalized learning. While LLMs have been tested for their comprehension and problem-solving skills, their capability in teaching remains largely unexplored. In teaching, questioning is a key skill that guides students to analyze, evaluate, and synthesize core concepts and principles. Therefore, our research introduces a benchmark to evaluate the questioning capability in education as a teacher of LLMs through evaluating their generated educational questions, utilizing Anderson and Krathwohl’s taxonomy across general, monodisciplinary, and interdisciplinary domains. We shift the focus from LLMs as learners to LLMs as educators, assessing their teaching capability through guiding them to generate questions. We apply four metrics, including relevance, coverage, representativeness, and consistency, to evaluate the educational quality of LLMs’ outputs. Our results indicate that GPT-4 demonstrates significant potential in teaching general, humanities, and science courses;Claude2 appears more apt as an interdisciplinary teacher. Furthermore, the automatic scores align with human perspectives. © 2024, CC BY.

关键词： Problem solving

来源：评论

学校读者我要写书评

暂无评论

LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration

LONGAGENT: Achieving Question Answering for 128k-Token-Long ...

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Zhao, Jun Zu, Can Xu, Hao Lu, Yi He, Wei Ding, Yiwen Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China Institute of Modern Languages and Linguistics Fudan University China

ISBN: (纸本)9798891761643

Large language models (LLMs) have achieved tremendous success in understanding language and processing text. However, question-answering (QA) on lengthy documents faces challenges of resource constraints and a high propensity for errors, even for the most advanced models such as GPT-4 and Claude2. In this paper, we introduce LONGAGENT, a multi-agent collaboration method that enables efficient and effective QA over 128k-token-long documents. LONGAGENT adopts a divide- and-conquer strategy, breaking down lengthy documents into shorter, more manageable text chunks. A leader agent comprehends the user's query and organizes the member agents to read their assigned chunks, reasoning a final answer through multiple rounds of discussion. Due to members' hallucinations, it's difficult to guarantee that every response provided by each member is accurate. To address this, we develop an inter-member communication mechanism that facilitates information sharing, allowing for the detection and mitigation of hallucinatory responses. Experimental results show that a LLaMA-2 7B driven by LONGAGENT can effectively support QA over 128k-token documents, achieving 16.42% and 1.63% accuracy gains over GPT-4 on single-hop and multi-hop QA settings, respectively. © 2024 Association for Computational Linguistics.

关键词： Question answering

来源：评论

学校读者我要写书评

暂无评论

Multi-level Feature Fusion for Automated Essay Scoring

Journal of Network Intelligence

引用

Journal of Network Intelligence 2023年第1期8卷 76-88页

作者： Wang, Jinshui Chen, Junyan Ou, Xuewen Han, Qingfeng Tang, Zhengyi School of Computer Science and Mathematics Fujian University of Technology Fuzhou350118 China Fujian Provincial Key Laboratory of Big Data Mining and Applications Fujian University of Technology Fuzhou350118 China China National Petroleum Corporation China

Automatic Essay Scoring (AES) is one of the significant and challenging research topics in the Natural Language Processing(NLP) area. However, existing AES models majorly consider features derived from vocabulary while failing to integrate sentence and chapter information from multi-level perspectives. In this study, we proposed a multi-level feature fusion model, which was used to capture the multi-level features from different perspectives to improve the accuracy of scoring. Our model consisted of three components to respectively capture the vocabulary-level, sentence-level, and chapter-level features, which were then fed into a CNN and BiLSTM network for the final essay scor-ing. The results show that the proposed model outperforms a set of state-of-the-art AES models on the dataset of the Kaggle Automated Student Assessment Prize (ASAP) com-petition. The average Quadratic Weighted Kappa (QWK) value reaches 0.816, which verifies the efficacy of the model in the task of automated essay scoring. © 2023, Taiwan Ubiquitous Information CO LTD.

关键词： Automation

来源：评论

学校读者我要写书评

暂无评论

Physically Secure and Privacy-Preserving Authentication Scheme for VANET

Physically Secure and Privacy-Preserving Authentication Sche...

引用

2024 International Conference on Artificial Intelligence of Things and Systems, AIoTSys 2024

作者： Chen, Dongmei Liang, Yangfan Li, Lihui Liu, Yining School of Computer Science and Information Security Guilin University of Electronic Technology Guilin China Jiaxing University Provincial Key Laboratory of Multimodal Perceiving and Intelligent Systems Jiaxing China School of Data Science and Artificial Intelligence Wenzhou University of Technology Wenzhou China

ISBN: (纸本)9798331507886

The communication between vehicles and other entities occurs over open wireless networks, making them susceptible to various security attacks In Vehicular Ad Hoc Network (VANET). Consequently, many researchers have proposed numerous data transmission and key exchange protocols to ensure secure communication within VANET. The keys and sensitive information involved in these protocols need to be stored long-Term in the tamper-resistant devices of vehicles. However, due to the limited resources of these devices, many encryption schemes are difficult to embed. Even if effective encryption schemes can be embedded in VANET, the long-Term storage of data in vehicles still carries a significant risk of being attacked. To tackle this problem, we propose an authentication protocol designed for VANET that emphasizes both physical security and privacy protection. The protocol primarily leverages the properties of Physical Unclonable Functions to achieve enhanced security. This approach eliminates the need for vehicles to store keys and sensitive messages long-Term, reducing the storage burden on vehicles while enhancing the system's security and efficiency. Detailed security analyses demonstrate that our proposed scheme achieves the goals of physical security and privacy protection. A comparison with similar approaches demonstrates that our solution outperforms them in both time efficiency and communication overhead. © 2024 IEEE.

关键词： Authentication Protocol

来源：评论

学校读者我要写书评

暂无评论

CodeGen4Libs: A Two-Stage Approach for Library-Oriented Code Generation

CodeGen4Libs: A Two-Stage Approach for Library-Oriented Code...

引用

IEEE International Conference on Automated Software Engineering (ASE)

作者： Mingwei Liu Tianyong Yang Yiling Lou Xueying Du Ying Wang Xin Peng Fudan University Shanghai China Shanghai Key Laboratory of Data Science School of Computer Science Fudan University China

Automated code generation has been extensively studied in recent literature. In this work, we first survey 66 participants to motivate a more pragmatic code generation scenario, i.e., library-oriented code generation, where the generated code should implement the functionally of the natural language query with the given library. We then revisit existing learning-based code generation techniques and find they have limited effectiveness in such a library-oriented code generation scenario. To address this limitation, we propose a novel library-oriented code generation technique, CodeGen4Libs, which incorporates two stages: import generation and code generation. The import generation stage generates import statements for the natural language query with the given third-party libraries, while the code generation stage generates concrete code based on the generated imports and the query. To evaluate the effectiveness of our approach, we conduct extensive experiments on a dataset of 403,780 data items. Our results demonstrate that CodeGen4Libs outperforms baseline models in both import generation and code generation stages, achieving improvements of up to 97.4% on EM (Exact Match), 54.5% on BLEU, and 53.5% on Hit@All. Overall, our proposed CodeGen4Libs approach shows promising results in generating high-quality code with specific third-party libraries, which can improve the efficiency and effectiveness of software development.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：