检索结果-内蒙古大学图书馆

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Zhang, Yue Zhang, Zhihao Lai, Wenbin Zhang, Chong Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China

ISBN: (纸本)9798891761681

In many PDF documents, the reading order of text blocks is missing, which can hinder machine understanding of the document's content. Existing works try to extract one universal reading order for a PDF file. However, applications, like Retrieval Augmented Generation (RAG), require breaking long articles into sections, subsections and table cells for better indexing. For this reason, this paper introduces a new task and dataset, PDF-to-Tree, which organizes the text blocks of a PDF into a tree structure. Since a PDF may contain thousands of text blocks, this paper proposes a transition-based parser that uses a greedy strategy to build the tree structure. Compared to the parser for plain text, we also use multi-modal features to encode the parser state. Experiments show that our approach achieves an accuracy of 93.93%, surpassing the performance of baseline methods by an improvement of 6.72%. The dataset is public available at https://***/yuezh000/PDFto-Tree. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning

Improving Discriminative Capability of Reward Models in RLHF...

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Chen, Lu Zheng, Rui Wang, Binghai Jin, Senjie Huang, Caishuang Ye, Junjie Zhang, Zhihao Zhou, Yuhao Xi, Zhiheng Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Key Laboratory of Intelligent Information Processing Fudan University Shanghai China

ISBN: (纸本)9798891761643

Reinforcement Learning from Human Feedback (RLHF) is a crucial approach to aligning language models with human values and intentions. A fundamental challenge in this method lies in ensuring that the reward model accurately understands and evaluates human preferences. Current methods rely on ranking losses to teach the reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions. To address this issue, we introduce contrastive learning into the reward modeling process. In addition to supervised ranking loss, we introduce an unsupervised contrastive loss to enable the reward model to fully capture the distinctions in contrastive data. Experimental results demonstrate that the proposed contrastive learning-based reward modeling method effectively enhances the generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences. © 2024 Association for Computational Linguistics.

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration

LONGAGENT: Achieving Question Answering for 128k-Token-Long ...

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Zhao, Jun Zu, Can Xu, Hao Lu, Yi He, Wei Ding, Yiwen Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China Institute of Modern Languages and Linguistics Fudan University China

ISBN: (纸本)9798891761643

Large language models (LLMs) have achieved tremendous success in understanding language and processing text. However, question-answering (QA) on lengthy documents faces challenges of resource constraints and a high propensity for errors, even for the most advanced models such as GPT-4 and Claude2. In this paper, we introduce LONGAGENT, a multi-agent collaboration method that enables efficient and effective QA over 128k-token-long documents. LONGAGENT adopts a divide- and-conquer strategy, breaking down lengthy documents into shorter, more manageable text chunks. A leader agent comprehends the user's query and organizes the member agents to read their assigned chunks, reasoning a final answer through multiple rounds of discussion. Due to members' hallucinations, it's difficult to guarantee that every response provided by each member is accurate. To address this, we develop an inter-member communication mechanism that facilitates information sharing, allowing for the detection and mitigation of hallucinatory responses. Experimental results show that a LLaMA-2 7B driven by LONGAGENT can effectively support QA over 128k-token documents, achieving 16.42% and 1.63% accuracy gains over GPT-4 on single-hop and multi-hop QA settings, respectively. © 2024 Association for Computational Linguistics.

关键词： Question answering

来源：评论

学校读者我要写书评

暂无评论

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios 31

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabili...

引用

31st International Conference on Computational Linguistics, COLING 2025

作者： Ye, Junjie Li, Guanyu Gao, Songyang Huang, Caishuang Wu, Yilong Li, Sixian Fan, Xiaoran Dou, Shihan Ji, Tao Zhang, Qi Gui, Tao Huang, Xuanjing School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Research Institute of Intelligent Complex Systems Fudan University China Shanghai Key Laboratory of Intelligent Information Processing China Pengcheng Laboratory China

ISBN: (纸本)9798891761964

Existing evaluations of tool learning primarily focus on validating the alignment of selected tools (e.g., various APIs) for large language models (LLMs) with expected outcomes. However, these approaches rely on a limited set of scenarios where answers can be pre-determined. Furthermore, a sole emphasis on outcomes disregards the complex capabilities required for LLMs to effectively use tools. To tackle this issue, we propose ToolEyes, a fine-grained system tailored for the evaluation of the LLMs' tool learning capabilities in authentic scenarios. The system meticulously examines seven real-world scenarios, analyzing five dimensions crucial to LLMs in tool learning: format alignment, intent comprehension, behavior planning, tool selection, and answer organization. Additionally, ToolEyes incorporates a tool library boasting approximately 600 tools, serving as an intermediary between LLMs and the physical world. Evaluations involving ten LLMs across three categories reveal a preference for specific scenarios and limited cognitive abilities in tool learning. Intriguingly, expanding the model size even exacerbates the hindrance to tool learning. The code and data are available at https://***/Junjie-Ye/ToolEyes. © 2025 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

LONGHEADS: Multi-Head Attention is Secretly a Long Context Processor

LONGHEADS: Multi-Head Attention is Secretly a Long Context P...

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Lu, Yi Zhou, Xin He, Wei Zhao, Jun Ji, Tao Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University Shanghai China Institute of Modern Languages and Linguistics Fudan University Shanghai China Key Laboratory of Intelligent Information Processing Fudan University Shanghai China

ISBN: (纸本)9798891761681

Large language models (LLMs) have achieved impressive performance in numerous domains but often struggle to process lengthy inputs effectively and efficiently due to limited length generalization and attention's quadratic computational demands. Many sought to mitigate this by restricting the attention window within the pre-trained length. However, these methods introduce new issues such as ignoring the middle context and requiring additional training. To address these problems, we propose LONGHEADS, a training-free framework that enhances LLM's long context ability by unlocking multi-head attention's untapped potential. Instead of allowing each head to attend to the full sentence, which struggles with generalizing to longer sequences, we allow each head to process in-distribution length by selecting and attending to important context chunks. To this end, we propose a chunk selection strategy that relies on the inherent correlation between the query and the key representations, efficiently distributing context chunks to different heads. In this way, each head ensures it can effectively process attended tokens within the trained length, while different heads in different layers can collectively process longer contexts. LONGHEADS works efficiently and fits seamlessly with many LLMs that use relative positional encoding. LONGHEADS achieves 100% accuracy at the 128k length on passkey retrieval task, verifying LONGHEADS's efficacy in extending the usable context window for existing models. We release our code at https://***/LuLuLuyi/LongHeads. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

FlexPDA:A Flexible Programming Framework for Deep Learning Accelerators

引用

Journal of computer science & Technology 2022年第5期37卷 1200-1220页

作者： Lei Liu Xiu Ma Hua-xiao Liu Guang-li Li Lei Liu College of Computer Science and Technology Jilin UniversityChangchunChina Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin UniversityChangchunChina State Key Laboratory of Computer Architecture Institute of Computing TechnologyChinese Academy of SciencesBeijingChina University of Chinese Academy of Sciences BeijingChina

There are a wide variety of intelligence accelerators with promising performance and energy efficiency,deployed in a broad range of applications such as computer vision and speech ***,programming productivity hinders the deployment of deep learning *** low-level library invoked in the high-level deep learning framework which supports the end-to-end execution with a given model,is designed to reduce the programming burden on the intelligence ***,it is inflexible for developers to build a network model for every deep learning application,which probably brings unnecessary repetitive *** this paper,a flexible and efficient programming framework for deep learning accelerators,FlexPDA,is proposed,which provides more optimization opportunities than the low-level library and realizes quick transplantation of applications to intelligence accelerators for fast *** evaluate FlexPDA by using 10 representative operators selected from deep learning algorithms and an end-to-end *** experimental results validate the effectiveness of FlexPDA,which achieves an end-to-end performance improvement of 1.620x over the low-level library.

关键词： deep learning accelerator programming framework domain-specific language

来源：评论

学校读者我要写书评

暂无评论

NfvInsight:A Framework for Automatically Deploying and Benchmarking VNF Chains

引用

Journal of computer science & Technology 2022年第3期37卷 680-698页

作者： Tian-Ni Xu Hai-Feng Sun Di Zhang Xiao-Ming Zhou Xiu-Feng Sui Sa Wang Qun Huang Yun-Gang Bao State Key Laboratory of Computer Architecture Institute of Computing TechnologyChinese Academy of SciencesBeijing 100190China University of Chinese Academy of Sciences Beijing 100049China School of Information and Electronics Beijing Institute of TechnologyBeijing 100081China Peng Cheng Laboratory Shenzhen 518055China Department of Computer Science and Technology Peking UniversityBeijing 100871China

With the advent of virtualization techniques and software-defined networking(SDN),network function virtualization(NFV)shifts network functions(NFs)from hardware implementations to software appliances,between which exists a performance *** to narrow the gap is an essential issue of current NFV ***,the cumbersomeness of deployment,the water pipe effect of virtual network function(VNF)chains,and the complexity of the system software stack together make it tough to figure out the cause of low performance in the NFV *** pinpoint the NFV system performance,we propose NfvInsight,a framework for automatic deployment and benchmarking VNF *** framework tackles the challenges in NFV performance *** framework components include chain graph generation,automatic deployment,and fine granularity *** design and implementation of each component have their *** the best of our knowledge,we make the first attempt to collect rules forming a knowledge base for generating reasonable chain *** deploys the generated chain graphs automatically,which frees the network operators from executing at least 391 lines of bash commands for a single *** diagnose the performance bottleneck,NfvInsight collects metrics from multiple layers of the software ***,we collect the network stack latency distribution ingeniously,introducing only less than 2.2%*** showcase the convenience and usability of NfvInsight in finding bottlenecks for both VNF chains and the underlying *** our framework,we find several design flaws of the network stack,which are unsuitable for packet forwarding inside one single server under the NFV *** optimization for these flaws gains at most 3x performance improvement.

关键词： network function virtualization(NFV) service chain performance bottleneck network stack latency

来源：评论

学校读者我要写书评

暂无评论

Fast and efficient parallel breadth-first search with power-law graph transformation

引用

Frontiers of computer science 2022年第5期16卷 225-227页

作者： Zite JIANG Tao LIU Shuai ZHANG Mengting YUAN Haihang YOU School of Computer Science and Technology University of Chinese Academy of SciencesBeijing 100049China State Key Laboratory of Computer Architecture Institute of Computing TechnologyChinese Academy of SciencesBeijing 100190China School of Comptuer Science Wuhan UniversityWuhan 430072China

1 Introduction Most real-world graphs are large-scale but unstructured and *** of the most notable characteristics of real-world graphs is the skewed power law degree distribution[1]:most vertices have a few neighbors while a few own a large number of *** characteristics present challenges for efficient parallel graph processing,such as load imbalance,poor locality,and redundant *** from modifying the graph programming abstraction or changing the execution models on different architectures,reducing the irregularity of graph data also improves the performance of graph processing[2].For example,it is wellknown that BFS has a bad temporal locality,but it is possible to transform irregular graphs to more regular ones to improve spatial locality and gain more performance.

关键词： neighbor programming redundant

来源：评论

学校读者我要写书评

暂无评论

Tetris:A Heuristic Static Memory Management Framework for Uniform Memory Multicore Neural Network Accelerators

引用

Journal of computer science & Technology 2022年第6期37卷 1255-1270页

作者： Xiao-Bing Chen Hao Qi Shao-Hui Peng Yi-Min Zhuang Tian Zhi Yun-Ji Chen Distinguished Member,CCF State Key Laboratory of Computer Architecture Institute of Computing TechnologyChinese Academy of SciencesBeijing 100190China University of Chinese Academy of Sciences Beijing 100049China School of Computer Science and Technology University of Science and Technology of ChinaHeFei230026China Chinese Academy of Sciences Center for Excellence in Brain Science and Intelligence Technology Shanghai200031China 不详

Uniform memory multicore neural network accelerators(UNNAs)furnish huge computing power to emerging neural network ***,with neural network architectures going deeper and wider,the limited memory capacity has become a constraint to deploy models on UNNA *** how to efficiently manage memory space and how to reduce workload footprints are urgently *** this paper,we propose Tetris:a heuristic static memory management framework for UNNA *** reconstructs execution flows and synchronization relationships among cores to analyze each tensor’s liveness *** the memory management problem is converted to a sequence permutation *** uses a genetic algorithm to explore the permutation space to optimize the memory management strategy and reduce memory *** evaluate several typical neural networks and the experimental results demonstrate that Tetris outperforms the state-of-the-art memory allocation methods,and achieves an average memory reduction ratio of 91.9%and 87.9%for a quad-core and a 16-core Cambricon-X platform,respectively.

关键词： multicore neural network accelerators liveness analysis static memory management memory reuse genetic algorithm

来源：评论

学校读者我要写书评

暂无评论

Sampling Methods for Efficient Training of Graph Convolutional Networks:A Survey

引用

IEEE/CAA Journal of Automatica Sinica 2022年第2期9卷 205-234页

作者： Xin Liu Mingyu Yan Lei Deng Guoqi Li Xiaochun Ye Dongrui Fan State Key Laboratory of Computer Architecture Institute of Computing TechnologyChinese Academy of SciencesBeijing 100086 School of Computer Science and Technology University of Chinese Academy of SciencesBeijing 100049 IEEE Department of Precision Instrument Center for Brain Inspired Computing ResearchTsinghua UniversityBeijing 100084China

Graph convolutional networks(GCNs)have received significant attention from various research fields due to the excellent performance in learning graph *** GCN performs well compared with other methods,it still faces *** a GCN model for large-scale graphs in a conventional way requires high computation and storage ***,motivated by an urgent need in terms of efficiency and scalability in training GCN,sampling methods have been proposed and achieved a significant *** this paper,we categorize sampling methods based on the sampling mechanisms and provide a comprehensive survey of sampling methods for efficient training of *** highlight the characteristics and differences of sampling methods,we present a detailed comparison within each category and further give an overall comparative analysis for the sampling methods in all ***,we discuss some challenges and future research directions of the sampling methods.

关键词： Efficient training graph convolutional networks(GCNs) graph neural networks(GNNs) sampling method

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：