检索结果-内蒙古大学图书馆

Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024

作者： Ma, Longxuan Ke, Changxin Zhou, Shuhan Sun, Churui Zhang, Weinan Liu, Ting Research Center for Social Computing and Information Retrieval School of Computer Science Harbin Institute of Technology China

ISBN: (纸本)9782493814104

Simile tasks are challenging in natural language processing (NLP) because models require adequate world knowledge to produce predictions. In recent years, pre-trained language models (PLMs) have succeeded in NLP since they learn generic knowledge from a large corpus. The knowledge embedded in PLMs can be used for different kinds of Simile tasks. However, previous work usually explored one type of simile knowledge for a specific simile task, how to fully utilize different types of knowledge embedded in the PLMs requires further exploration. This paper proposes a self-verified method for exploring simile knowledge from PLMs, which allows the PLMs to leverage one type of simile knowledge to self-validate another. To this end, we first enhance PLMs with a novel multi-level simile recognition (MLSR) task that trains PLMs to evaluate the quality of similes. Then the PLMs leverage this evaluation score to assist the simile interpretation and generation tasks. In this way, we connect different types of simile knowledge in PLMs and make better use of them. Experiments on different pre-trained models and multiple publicly available datasets show that our method works for different kinds of PLMs and can explore more accurate simile knowledge for PLMs. Our code/data will be released on GitHub. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

关键词： Quality control

来源：评论

学校读者我要写书评

暂无评论

How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers 38

How does Architecture Influence the Base Capabilities of Pre...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Lu, Xin Zhao, Yanyan Qin, Bing Huo, Liangyu Yang, Qing Xu, Dongliang Research Center for Social Computing and Information Retrieval Harbin Institute of Technology China Science Technology Co. Ltd. China

Pre-trained language models have been proven to possess strong base capabilities, which not only excel in in-distribution language modeling but also show powerful abilities in out-of-distribution language modeling, transfer learning and few-shot learning. Unlike existing work focusing on the influence of scale on base capabilities, our work examines the influence of architecture on those. Specifically, our concern is: How does architecture influence the base capabilities of pre-trained language models? In this work, we attempt to explain and reverse the decline in base capabilities caused by the architecture of FFN-Wider Transformers, seeking to provide some insights. Through analysis, we found the contribution ratio of Multi-Head Attention (a combination function) to pre-trained language modeling is a key factor affecting base capabilities. FFN-Wider Transformers reduce the contribution ratio of this combination function, leading to a decline in base capabilities. We confirmed this by experiments and proposed Combination Enhanced Architecture (CEA) to address the decline in base capabilities of such models. Significantly, we extended our explanation and CEA to Mixture of Experts (MoE) Transformers. We successfully achieved significant improvements in base capabilities on a 14B parameter MoE model, demonstrating the practical application value of our work. This also indicates that our analysis has a certain guiding significance for architecture analysis, architecture improvement and architecture design. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

OneBit: Towards Extremely Low-bit Large Language Models 38

OneBit: Towards Extremely Low-bit Large Language Models

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Xu, Yuzhuang Han, Xu Yang, Zonghan Wang, Shuo Zhu, Qingfu Liu, Zhiyuan Liu, Weidong Che, Wanxiang Research Center for Social Computing and Information Retrieval Harbin Institute of Technology Harbin China Department of Computer Science & Technology Tsinghua University Beijing China

Model quantification uses low bit-width values to represent the weight matrices of existing models to be quantized, which is a promising approach to reduce both storage and computational overheads of deploying highly anticipated LLMs. However, current quantization methods suffer severe performance degradation when the bit-width is extremely reduced, and thus focus on utilizing 4-bit or 8-bit values to quantize models. This paper boldly quantizes the weight matrices of LLMs to 1-bit, paving the way for the extremely low bit-width deployment of LLMs. For this target, we introduce a 1-bit model compressing framework named OneBit, including a novel 1-bit parameter representation method to better quantize LLMs as well as an effective parameter initialization method based on matrix decomposition to improve the convergence speed of the quantization framework. Sufficient experimental results indicate that OneBit achieves good performance (at least 81% of the non-quantized performance on LLaMA models) with robust training processes when only using 1-bit weight matrices. Code and checkpoints are available at https://***/xuyuzhuang11/OneBit. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

STB-GraCapsNet: A Novel Capsule Network Structure with Swin Transformer Block 25th

STB-GraCapsNet: A Novel Capsule Network Structure with Swin...

引用

25th International Conference on Parallel and Distributed computing, Applications and Technologies, PDCAT 2024

作者： Zhang, Chunying Dong, Ziao Wang, Liya Liu, Lu Ren, Jing Ma, Jiang Liu, Bin College of Science North China University of Science and Technology 21 Bohai Road Caofeidian Xincheng Hebei Tangshan063210 China Big data and Social Computing Research Center Hebei University of Science and Technology Hebei Shijiazhuang0500198 China

ISBN: (纸本)9789819642069

Capsule network is a new type of neural network encoding features into capsules and constructing the part-whole relationships, which demonstrated good performance in image classification. However, it has some issues such as weak feature extraction ability and large number of training parameters, leading to performance decline when processing images with complex background. To address these issues, this article proposes a novel capsule network structure called STB-GraCapsNet, with key improvements as follows: 1) It uses Swin Transformer Block with the shift window attention mechanism for data preprocessing to obtain richer features;2) the features are then processed through Linear Embedding and passed into the Primary Capsule Layer to generate high-dimension feature capsules, which reduces primary capsules number and improves feature extraction ability of the network. Ablation and comparative experiments based on multiple datasets show that STB-GraCapsNet performs better on complex datasets: 77.8% and 36.2% reduction in parameters number of capsule structure, and 12.07% and 5.22% improvement in model accuracy, as compared to CapsNet and GraCapsNet, respectively. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Signal encoding

来源：评论

学校读者我要写书评

暂无评论

POPA: Expressing High and Portable Performance across Spatial and Vector Architectures for Tensor Computations 24

POPA: Expressing High and Portable Performance across Spatia...

引用

32nd ACM International Symposium on Field-Programmable Gate Arrays, FPGA 2024

作者： Hao, Xiaochen Rong, Hongbo Zhang, Mingzhe Sun, Ce Jiang, Hong Liang, Yun Peking University China Parallel Computing Lab Intel United States Tsinghua University China University of Science and Technology of China China Intel United States Peking University & Beijing Advanced Innovation Center for Integrated Circuits China

ISBN: (纸本)9798400704185

This paper aims at high and portable performance for tensor computations across spatial (e.g., FPGAs) and vector architectures (e.g., GPUs). The state-of-the-art usually address performance portability across vector architectures (CPUs and GPUs). However, they either miss FPGAs or do not achieve high performance. Without a common architectural abstraction, they program and optimize spatial and vector devices separately, causing low portability. We propose a unified programming framework, POPA, which achieves portability via architectural abstraction and performance via specialization. A parallel dataflow machine is proposed as a unified, abstract hardware target that hides differences of concrete architectures. The machine consists of software-defined systolic arrays and a tensor-specific cache hierarchy, which captures pipeline parallelism and customizable memories on FPGAs, as well as multithreading parallelism on GPUs. The machine is specified in a unified programming model as two dataflow graphs for scheduling compute and data movement, respectively. A compiler then specializes the abstract machine to exploit the properties of FPGAs and GPUs, bridging the gap between the abstract machine and a concrete architecture. We evaluate POPA on several intel FPGAs and GPUs with high-profile tensor kernels, and this is the first system that achieves >=80% performance of expert-written code or machine peak across architectures, to the best of our knowledge. © 2024 ACM.

关键词： Graphics processing unit

来源：评论

学校读者我要写书评

暂无评论

Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance 38

Meaningful Learning: Enhancing Abstract Reasoning in Large L...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Xiong, Kai Ding, Xiao Liu, Ting Qin, Bing Xu, Dongliang Yang, Qing Liu, Hongtao Cao, Yixin Research Center for Social Computing and Information Retrieval Harbin Institute of Technology Harbin China Fudan University Shanghai China Science Technology Co. Ltd. Beijing China

Large language models (LLMs) have developed impressive performance and strong explainability across various reasoning scenarios, marking a significant stride towards mimicking human-like intelligence. Despite this, when tasked with several simple questions supported by a generic fact, LLMs often struggle to abstract and apply the generic fact to provide consistent and precise answers, revealing a deficiency in abstract reasoning abilities. This has sparked a vigorous debate about whether LLMs are genuinely reasoning or merely memorizing. In light of this, we design a preliminary study to quantify and delve into the abstract reasoning abilities of existing LLMs. Our findings reveal a substantial discrepancy between their general reasoning and abstract reasoning performances. To relieve this problem, we tailor an abstract reasoning dataset (AbsR) together with a meaningful learning paradigm to teach LLMs how to leverage generic facts for reasoning purposes. The results show that our approach not only boosts the general reasoning performance of LLMs but also makes considerable strides towards their capacity for abstract reasoning, moving beyond simple memorization or imitation to a more nuanced understanding and application of generic facts. The code is available at https://***/Waste-Wood/MeanLearn. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

OpenSLU: A unified, modularized, and extensible toolkit for spoken language understanding 61

OpenSLU: A unified, modularized, and extensible toolkit for ...

引用

61st Annual Meeting of the Association for Computational Linguistics, ACL-DEMO 2023

作者： Qin, Libo Chen, Qiguang Xu, Xiao Feng, Yunlong Che, Wanxiang School of Computer Science and Engineering Central South University China Research Center for Social Computing and Information Retrieval Harbin Institute of Technology China

ISBN: (纸本)9781959429708

Spoken Language Understanding (SLU) is one of the core components of a task-oriented dialogue system, which aims to extract the semantic meaning of user queries (e.g., intents and slots). In this work, we introduce OpenSLU, an open-source toolkit to provide a unified, modularized, and extensible toolkit for spoken language understanding. Specifically, OpenSLU unifies 10 SLU models for both single-intent and multi-intent scenarios, which support both non-pretrained and pretrained models simultaneously. Additionally, OpenSLU is highly modularized and extensible by decomposing the model architecture, inference, and learning process into reusable modules, which allows researchers to quickly set up SLU experiments with highly flexible configurations. OpenSLU is implemented based on PyTorch, and released at https://***/LightChen233/OpenSLU. © ACL-DEMO 2023. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Technological acceptance of non-native speakers on language learning mobile application using augmented reality

引用

Multimedia Tools and Applications 2025年 1-32页

作者： Suwadi, Nur Asylah Lam, Meng Chun Majid, Nazatul Aini Abd Jalaluddin, Nor Hashimah Aisyah, Aznur Kasdan, Junaini Hussain, Afifuddin Husairi Ahmad, Azlan Ma’arof, Daing Zairi Mixed Reality and Pervasive Computing Lab Center for Artificial Intelligence Technology Faculty of Information Science & Technology Universiti Kebangsaan Malaysia Bangi43600 Malaysia Center for Research in Language and Linguistics Faculty of Social Sciences and Humanities Universiti Kebangsaan Malaysia Bangi43600 Malaysia Universiti Kebangsaan Malaysia Bangi43600 Malaysia Pusat Citra Universiti Universiti Kebangsaan Malaysia Bangi43600 Malaysia

Although mobile applications have been widely used, there is limited research on the acceptance of mobile-based applications for language learning among foreign learners. Hence, we propose an extended version of the technology Acceptance Model (TAM) that analyzes the key aspects affecting the acceptance of the mobile application by foreign learners. The extended TAM includes external constructs such as augmented reality (AR), content quality, design quality, and interactivity to consider both system and individual levels. Eighty international students from various countries studying Malay were given a TAM questionnaire online. Results revealed positive effects, and notably, the external constructs of AR, content quality, and interactivity are prerequisite factors in extending the TAM. The findings revealed that interactivity is the most significant external construct of Perceived Ease of Use (PEOU), followed by Perceived Usefulness (PU), which exhibits a significant contribution to the user’s attitude. Therefore, these results could guide how educator and higher education centers should approach the incorporation of these technologies in classrooms. © The Author(s), under exclusive licence to Springer science+Business Media, LLC, part of Springer Nature 2025.

关键词： Augmented reality

来源：评论

学校读者我要写书评

暂无评论

What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration 38

What Factors Affect Multi-Modal In-Context Learning? An In-D...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Qin, Libo Chen, Qiguang Fei, Hao Chen, Zhi Li, Min Che, Wanxiang School of Computer Science and Engineering Central South University China Research Center for Social Computing and Information Retrieval China Harbin Institute of Technology China Tsinghua University China ByteDance China

Recently, rapid advancements in Multi-Modal In-Context Learning (MM-ICL) have achieved notable success, which is capable of achieving superior performance across various tasks without requiring additional parameter tuning. However, the underlying rules for the effectiveness of MM-ICL remain under-explored. To fill this gap, this work aims to investigate the research question: "What factors affect the performance of MM-ICL?" To this end, we investigate extensive experiments on the three core steps of MM-ICL including demonstration retrieval, demonstration ordering, and prompt construction using 6 vision large language models and 20 strategies. Our findings highlight (1) the necessity of a multi-modal retriever for demonstration retrieval, (2) the importance of intra-demonstration ordering over inter-demonstration ordering, and (3) the enhancement of task comprehension through introductory instructions in prompts. We hope this study can serve as a foundational guide for optimizing MM-ICL strategies in future research. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought 38

Unlocking the Capabilities of Thought: A Reasoning Boundary ...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Chen, Qiguang Qin, Libo Wang, Jiaqi Zhou, Jinxuan Che, Wanxiang Research Center for Social Computing and Information Retrieval China Harbin Institute of Technology China School of Computer Science and Engineering Central South University China The Chinese University of Hong Kong Hong Kong

Chain-of-Thought (CoT) reasoning has emerged as a promising approach for enhancing the performance of large language models (LLMs) on complex reasoning tasks. Recently, a series of studies attempt to explain the mechanisms underlying CoT, aiming to deepen the understanding of its efficacy. Nevertheless, the existing research faces two major challenges: (1) a lack of quantitative metrics to assess CoT capabilities and (2) a dearth of guidance on optimizing CoT performance. Motivated by this, in this work, we introduce a novel reasoning boundary framework (RBF) to address these challenges. To solve the lack of quantification, we first define a reasoning boundary (RB) to quantify the upper-bound of CoT and establish a combination law for RB, enabling a practical quantitative approach applicable to various real-world CoT tasks. To address the lack of optimization, we propose three categories of RBs. We further optimize these categories with combination laws focused on RB promotion and reasoning path optimization for CoT improvement. Through extensive experiments on 27 models and 5 tasks, the study validates the existence and rationality of the proposed framework. Furthermore, it explains the effectiveness of 10 CoT strategies and guides optimization from two perspectives. We hope this work can provide a comprehensive understanding of the boundaries and optimization strategies for reasoning in LLMs. Our code and data are available at https://***/LightChen233/reasoning-boundary. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：