检索结果-内蒙古大学图书馆

REASONING BIAS OF NEXT TOKEN PREDICTION TRAINING

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Lin, Pengxiao Zhang, Zhongwang Xu, Zhi-Qin John Institute of Natural Sciences MOE-LSC Shanghai Jiao Tong University China School of Mathematical Sciences Shanghai Jiao Tong University China School of Artificial Intelligence Shanghai Jiao Tong University China Key Laboratory of Marine Intelligent Equipment and System Ministry of Education China Center for LLM Institute for Advanced Algorithms Research Shanghai China

Since the inception of Large Language Models (LLMs), the quest to efficiently train them for superior reasoning capabilities has been a pivotal challenge. The dominant training paradigm for LLMs is based on next token prediction (NTP). Alternative methodologies, called Critical Token Prediction (CTP), focused exclusively on specific critical tokens (such as the answer in Q&A dataset), aiming to reduce the overfitting of extraneous information and noise. Contrary to initial assumptions, our research reveals that despite NTP’s exposure to noise during training, it surpasses CTP in reasoning ability. We attribute this counterintuitive outcome to the regularizing influence of noise on the training dynamics. Our empirical analysis shows that NTP-trained models exhibit enhanced generalization and robustness across various benchmark reasoning datasets, demonstrating greater resilience to perturbations and achieving flatter loss minima. These findings illuminate that NTP is instrumental in fostering reasoning abilities during pretraining, whereas CTP is more effective for finetuning, thereby enriching our comprehension of optimal training strategies in LLM development. © 2025, CC BY.

关键词： Prediction models

AN ANALYSIS FOR REASONING BIAS OF LANGUAGE MODELS WITH SMALL INITIALIZATION

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yao, Junjie Zhang, Zhongwang Xu, Zhi-Qin John Institute of Natural Sciences MOE-LSC Shanghai Jiao Tong University China School of Mathematical Sciences Shanghai Jiao Tong University China School of Artificial Intelligence Shanghai Jiao Tong University China Key Laboratory of Marine Intelligent Equipment and System Ministry of Education China Center for LLM Institute for Advanced Algorithms Research Shanghai China

Transformer-based Large Language Models (LLMs) have revolutionized Natural Language Processing by demonstrating exceptional performance across diverse tasks. This study investigates the impact of the parameter initialization scale on the training behavior and task preferences of LLMs. We discover that smaller initialization scales encourage models to favor reasoning tasks, whereas larger initialization scales lead to a preference for memorization tasks. We validate this reasoning bias via real datasets and meticulously designed anchor functions. Further analysis of initial training dynamics suggests that specific model components, particularly the embedding space and self-attention mechanisms, play pivotal roles in shaping these learning biases. We provide a theoretical framework from the perspective of model training dynamics to explain these phenomena. Additionally, experiments on real-world language tasks corroborate our theoretical insights. This work enhances our understanding of how initialization strategies influence LLM performance on reasoning tasks and offers valuable guidelines for training models. © 2025, CC BY.

关键词： Natural language processing systems

Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Zhongwang Lin, Pengxiao Wang, Zhiwei Zhang, Yaoyu Xu, Zhi-Qin John The Institute of Natural Sciences School of Mathematical Sciences MOE-LSC Shanghai Jiao Tong University Shanghai200240 China The School of Artificial Intelligence Shanghai Jiao Tong University Shanghai200240 China The School of Artificial Intelligence Shanghai Jiao Tong University Center for LLM Institute for Advanced Algorithms Research Shanghai Seres Information Technology Co. Ltd. Shanghai200040 China

Transformers have demonstrated impressive capabilities across various tasks, yet their performance on compositional problems remains a subject of debate. In this study, we investigate the internal mechanisms underlying Transformers' behavior in compositional tasks. We find that complexity control strategies-particularly the choice of parameter initialization scale and weight decay-significantly influence whether the model learns primitive-level rules that generalize out-of-distribution (reasoning-based solutions) or relies solely on memorized mappings (memory-based solutions). By applying masking strategies to the model's information circuits and employing multiple complexity metrics, we reveal distinct internal working mechanisms associated with different solution types. Further analysis reveals that reasoning-based solutions exhibit a lower complexity bias, which aligns with the well-studied neuron condensation phenomenon. This lower complexity bias is hypothesized to be the key factor enabling these solutions to learn reasoning rules. We validate these conclusions across multiple real-world datasets, including image generation and natural language processing tasks, confirming the broad applicability of our findings. Copyright © 2025, The Authors. All rights reserved.

关键词： Distribution transformers

Efficient Qubit Calibration by Binary-Search Hamiltonian Tracking

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Berritta, Fabrizio Benestad, Jacob Pahl, Lukas Mathews, Melvin Krzywda, Jan A. Assouly, Réouven Sung, Youngkyu Kim, David K. Niedzielski, Bethany M. Serniak, Kyle Schwartz, Mollie E. Yoder, Jonilyn L. Chatterjee, Anasua Grover, Jeffrey A. Danon, Jeroen Oliver, William D. Kuemmeth, Ferdinand Research Laboratory of Electronics Massachusetts Institute of Technology CambridgeMA02139 United States Center for Quantum Devices Niels Bohr Institute University of Copenhagen Copenhagen2100 Denmark Center for Quantum Spintronics Department of Physics Norwegian University of Science and Technology TrondheimNO-7491 Norway Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology CambridgeMA02139 United States Department of Information Technology and Electrical Engineering ETH Zürich Zürich8093 Switzerland 〈aQaL〉 Applied Quantum Algorithms — Lorentz Insitute for Theoretical Physics & Leiden Institute of Advanced Computer Science Universiteit Leiden Netherlands Lincoln Laboratory Massachusetts Institute of Technology LexingtonMA02421 United States QuTech and Kavli Institute of Nanoscience Delft University of Technology Delft Netherlands Department of Physics Massachusetts Institute of Technology CambridgeMA02139 United States Institute of Experimental and Applied Physics University of Regensburg Regensburg93040 Germany QDevil Quantum Machines Ballerup2750 Denmark

We present a real-time method for calibrating the frequency of a resonantly driven qubit. The real-time processing capabilities of a classical controller dynamically generate adaptive probing sequences for qubit-frequency estimation. Each probing evolution time and drive frequency are calculated to divide the prior probability distribution into two branches, following a locally optimal strategy that mimics a conventional binary search. We show the algorithm’s efficacy by stabilizing a flux-tunable transmon qubit, leading to improved coherence and gate fidelity. By feeding forward the updated qubit frequency, the FPGA-powered control electronics partially mitigates non-Markovian noise in the system, which is detrimental to quantum error correction. Our protocol highlights the importance of feedback in improving the calibration and stability of qubits subject to drift and can be readily applied to other qubit platforms. © 2025, CC BY.

关键词： Hamiltonians

RARE: Retrieval-Augmented Reasoning Modeling

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Zhengren Yu, Jiayang Ma, Dongsheng Chen, Zhe Wang, Yu Li, Zhiyu Xiong, Feiyu Wang, Yanfeng Weinan, E. Tang, Linpeng Zhang, Wentao Peking University China Shanghai Jiao Tong University China Northeastern University China Nankai University China Institute for Advanced Algorithms Research Shanghai China OriginHub Technology United States MemTensor China Shanghai Artificial Intelligence Laboratory China

Domain-specific intelligence demands specialized knowledge and sophisticated reasoning for problem-solving, posing significant challenges for large language models (LLMs) that struggle with knowledge hallucination and inadequate reasoning capabilities under constrained parameter budgets. Inspired by Bloom’s Taxonomy in educational theory, we propose Retrieval-Augmented Reasoning Modeling (RARE), a novel paradigm that decouples knowledge storage from reasoning optimization. RARE externalizes domain knowledge to retrievable sources and internalizes domain-specific reasoning patterns during training. Specifically, by injecting retrieved knowledge into training prompts, RARE transforms learning objectives from rote memorization to contextualized reasoning application. It enables models to bypass parameter-intensive memorization and prioritize the development of higher-order cognitive processes. Our experiments demonstrate that lightweight RARE-trained models (e.g., Llama-3.1-8B) could achieve state-of-the-art performance, surpassing retrieval-augmented GPT-4 and Deepseek-R1 distilled counterparts. RARE establishes a paradigm shift where maintainable external knowledge bases synergize with compact, reasoning-optimized models, collectively driving more scalable domain-specific intelligence. Copyright © 2025, The Authors. All rights reserved.

关键词： Domain Knowledge

OFF-POLICY PRIMAL-DUAL SAFE REINFORCEMENT LEARNING 12

学校读者我要写书评

暂无评论

OFF-POLICY PRIMAL-DUAL SAFE REINFORCEMENT LEARNING

12th International Conference on Learning Representations, ICLR 2024

作者： Wu, Zifan Tang, Bo Lin, Qian Yu, Chao Mao, Shangqin Xie, Qianlong Wang, Xingxing Wang, Dong Sun Yat-Sen University Guangzhou China Institute for Advanced Algorithms Research Shanghai China Meituan Beijing China

Primal-dual safe RL methods commonly perform iterations between the primal update of the policy and the dual update of the Lagrange Multiplier. Such a training paradigm is highly susceptible to the error in cumulative cost estimation since this estimation serves as the key bond connecting the primal and dual update processes. We show that this problem causes significant underestimation of cost when using off-policy methods, leading to the failure to satisfy the safety constraint. To address this issue, we propose conservative policy optimization, which learns a policy in a constraint-satisfying area by considering the uncertainty in cost estimation. This improves constraint satisfaction but also potentially hinders reward maximization. We then introduce local policy convexification to help eliminate such suboptimality by gradually reducing the estimation uncertainty. We provide theoretical interpretations of the joint coupling effect of these two ingredients and further verify them by extensive experiments. Results on benchmark tasks show that our method not only achieves an asymptotic performance comparable to state-of-the-art on-policy methods while using much fewer samples, but also significantly reduces constraint violation during training. Our code is available at https://***/ZifanWu/CAL. © 2024 12th International Conference on Learning Representations, ICLR 2024. All rights reserved.

关键词： Benchmarking

FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models

学校读者我要写书评

暂无评论

FastMem: Fast Memorization of Prompt Improves Context Awaren...

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Zhu, Junyi Liu, Shuochen Yu, Yu Tang, Bo Yan, Yibo Li, Zhiyu Xiong, Feiyu Xu, Tong Blaschko, Matthew B. ESAT-PSI KU Leuven Belgium University of Science and Technology of China China Institute for Advanced Algorithms Research Shanghai China National University of Singapore Singapore

ISBN: (纸本)9798891761681

Large language models (LLMs) excel in generating coherent text, but they often struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information. We introduce FastMem, a novel method designed to enhance instruction fine-tuned LLMs' context awareness through fast memorization of the prompt. FastMem maximizes the likelihood of the prompt before inference by updating only the last Feed-Forward Network (FFN) module. This targeted approach ensures efficient optimization without overfitting, significantly improving the model's ability to comprehend and accurately follow the context. Our experiments demonstrate substantial gains in reading comprehension, text summarization and adherence to output structures. For instance, FastMem improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6%, and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%. Extensive experimental results highlight FastMem's potential to offer a robust solution to enhance the reliability and accuracy of LLMs in various applications. Our code is available at: https://***/IAAR-Shanghai/FastMem. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation 62

学校读者我要写书评

暂无评论

UHGEval: Benchmarking the Hallucination of Chinese Large Lan...

62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024

作者： Liang, Xun Song, Shichao Niu, Simin Li, Zhiyu Xiong, Feiyu Tang, Bo Wang, Yezhaohui He, Dawei Cheng, Peng Wang, Zhonghao Deng, Haiying School of Information Renmin University of China Beijing China Institute for Advanced Algorithms Research Shanghai China State Key Laboratory of Media Convergence Production Technology and Systems Beijing China

ISBN: (纸本)9798891760943

Large language models (LLMs) produce hallucinated text, compromising their practical utility in professional contexts. To assess the reliability of LLMs, numerous initiatives have developed benchmark evaluations for hallucination phenomena. However, they often employ constrained generation techniques to produce the evaluation dataset due to cost and time limitations. For instance, this may involve employing directed hallucination induction or deliberately modifying authentic text to generate hallucinations. These are not congruent with the unrestricted text generation demanded by real-world applications. Furthermore, a well-established Chinese-language dataset dedicated to the evaluation of hallucinations is presently lacking. Consequently, we have developed an Unconstrained Hallucination Generation Evaluation (UHGEval) benchmark, containing hallucinations generated by LLMs with minimal restrictions. Concurrently, we have established a comprehensive benchmark evaluation framework to aid subsequent researchers in undertaking scalable and reproducible experiments. We have also evaluated prominent Chinese LLMs and the GPT series models to derive insights regarding hallucination. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

Learning Free Terminal Time Optimal Closed-loop Control of Manipulators with Neural Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hu, Wei Zhao, Yue Weinan, E. Han, Jiequn Long, Jihao AI for Science Institute Beijing and Institute for Advanced Algorithms Research Shanghai China Peking University Beijing China Flatiron Institute New York United States Institute for Advanced Algorithms Research Shanghai China

This paper presents a novel approach to learning free terminal time closed-loop control for robotic manipulation tasks, enabling dynamic adjustment of task duration and control inputs to enhance performance. We extend the supervised learning approach, namely solving selected optimal open-loop problems and utilizing them as training data for a policy network, to the free terminal time scenario. Three main challenges are addressed in this extension. First, we introduce a marching scheme that enhances the solution quality and increases the success rate of the open-loop solver by gradually refining time discretization. Second, we extend the QRnet in Nakamura-Zimmerer et al. (2021b) to the free terminal time setting to address discontinuity and improve stability at the terminal state. Third, we present a more automated version of the initial value problem (IVP) enhanced sampling method from previous work (Zhang et al., 2022) to adaptively update the training dataset, significantly improving its quality. By integrating these techniques, we develop a closed-loop policy that operates effectively over a broad domain with varying optimal time durations, achieving near globally optimal total costs. Copyright © 2023, The Authors. All rights reserved.

关键词： Manipulators