检索结果-内蒙古大学图书馆

Optimal and instance-dependent guarantees for Markovian linear stochastic approximation

Mathematical Statistics and Learning 2024年第1-2期7卷 41-153页

作者： Mou, Wenlong Pananjady, Ashwin Wainwright, Martin J. Bartlett, Peter L. Department of Statistical Sciences and Vector Institute for Artificial Intelligence University of Toronto 700 University Avenue Toronto M5G 1X6 Canada School of ISyE and School of ECE Georgia Institute of Technology 765 Ferst Dr. NW Atlanta 30332 GA United States Department of EECS Department of Mathematics Laboratory for Information and Decision Systems and Statistics and Data Science Center Massachusetts Institute of Technology 32 Vassar Street Cambridge 02139 MA United States Department of EECS and Department of Statistics University of California 367 Evans Hall Berkeley 94720 CA United States Google DeepMind 1600 Amphitheatre Parkway Mountain View 94043 CA United States

We study stochastic approximation procedures for approximately solving a d-dimensional linear fixed-point equation based on observing a trajectory of length n from an ergodic d Markov chain. We first exhibit a non-asymptotic bound of the order t_mix on the squared n error of the last iterate of a standard scheme, where t_mix is a mixing time. We then prove a non-asymptotic instance-dependent bound on a suitably averaged sequence of iterates, with a leading term that matches the local asymptotic minimax limit, including sharp dependence on the parameters (d;t_mix) in the higher-order terms. We complement these upper bounds with a non-asymptotic minimax lower bound that establishes the instance-optimality of the averaged SA estimator. We derive corollaries of these results for policy evaluation with Markov noise— covering the TD(λ) family of algorithms for all λ ∊ [0;1)—and linear autoregressive models. Our instance-dependent characterizations open the door to the design of fine-grained model selection procedures for hyperparameter tuning (e.g., choosing the value of λ when running the TD(λ) algorithm). © 2024 European Mathematical Society.

关键词： instance-dependent optimality Markov chains reinforcement learning stochastic approximation temporal difference methods

来源：评论

学校读者我要写书评

暂无评论

A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond

arXiv

引用

arXiv 2024年

作者： Sun, Qiushi Chen, Zhirui Xu, Fangzhi Cheng, Kanzhi Ma, Chang Yin, Zhangyue Wang, Jianing Han, Chengcheng Zhu, Renyu Yuan, Shuai Guo, Qipeng Qiu, Xipeng Yin, Pengcheng Li, Xiaoli Yuan, Fei Kong, Lingpeng Li, Xiang Wu, Zhiyong Shanghai AI Laboratory Shanghai China School of Data Science and Engineering East China Normal University Shanghai China School of Computer Science Fudan University Shanghai China NetEase Fuxi AI Lab Zhejiang China Google DeepMind Mountain ViewCA United States Singapore School of Computer Science and Engineering Nanyang Technological University Singapore Department of Computer Science The University of Hong Kong Hong Kong

Neural Code Intelligence – leveraging deep learning to understand, generate, and optimize code – holds immense potential for transformative impacts on the whole society. Bridging the gap between Natural Language and Programming Language, this domain has drawn significant attention from researchers in both research communities over the past few years. This survey presents a systematic and chronological review of the advancements in code intelligence, encompassing over 50 representative models and their variants, more than 20 categories of tasks, and over 700 related works. We follow the historical progression to trace the paradigm shifts across different research phases (e.g., from modeling code with recurrent neural networks to the era of Large Language Models). Concurrently, we highlight the major technical transitions in models, tasks, and evaluations spanning through different stages. For applications, we also observe a co-evolving shift. It spans from initial endeavors to tackling specific scenarios, through exploring a diverse array of tasks during its rapid expansion, to currently focusing on tackling increasingly complex and varied real-world challenges. Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence, uncovering new cross-domain opportunities and illustrating the substantial influence of code intelligence across various domains. Finally, we delve into both the opportunities and challenges associated with this field, alongside elucidating our insights on the most promising research directions. An ongoing, dynamically updated project and resources associated with this survey have been released at https://***/QiushiSun/Awesome-Code-Intelligence. © 2024, CC BY.

关键词： Recurrent neural networks

来源：评论

学校读者我要写书评

暂无评论

The Odyssey Journey: Top-Tier Medical Resource Seeking for Specialized Disorder in China

arXiv

引用

arXiv 2024年

作者： Chan, Ka I. Hu, Siying Wang, Yuntao Xu, Xuhai Lu, Zhicong Shi, Yuanchun Institute Tsinghua University Beijing China Department of Computer Science City University of Hong Kong Hong Kong Key Laboratory of Pervasive Computing Ministry of Education Department of Computer Science and Technology Tsinghua University Beijing China School of Computer Science and Technology Qinghai University Xining China Google New York CityNY United States Department of Computer Science George Mason University FairfaxVA United States Department of Computer Science and Technology Beijing National Research Center for Information Science and Technology Tsinghua University Beijing China Qinghai University Xining China

It is pivotal for patients to receive accurate health information, diagnoses, and timely treatments. However, in China, the significant imbalanced doctor-to-patient ratio intensifies the information and power asymmetries in doctor-patient relationships. Health information-seeking, which enables patients to collect information from sources beyond doctors, is a potential approach to mitigate these asymmetries. While HCI research predominantly focuses on common chronic conditions, our study focuses on specialized disorders, which are often familiar to specialists but not to general practitioners and the public. With Hemifacial Spasm (HFS) as an example, we aim to understand patients’ health information and top-tier1 medical resource seeking journeys in China. Through interviews with three neurosurgeons and 12 HFS patients from rural and urban areas, and applying Actor-Network Theory, we provide empirical insights into the roles, interactions, and workflows of various actors in the health information-seeking network. We also identified five strategies patients adopted to mitigate asymmetries and access top-tier medical resources, illustrating these strategies as subnetworks within the broader health information-seeking network and outlining their advantages and challenges. Copyright © 2024, The Authors. All rights reserved.

关键词： Electronic health record

来源：评论

学校读者我要写书评

暂无评论

Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use

arXiv

引用

arXiv 2025年

作者： Goldie, Anna Mirhoseini, Azalia Zhou, Hao Cai, Irene Manning, Christopher D. Department of Computer Science Stanford University United States Google DeepMind United Kingdom

Reinforcement learning has been shown to improve the performance of large language models. However, traditional approaches like RLHF or RLAIF treat the problem as single-step. As focus shifts toward more complex reasoning and agentic tasks, language models must take multiple steps of text generation, reasoning and environment interaction before generating a solution. We propose a synthetic data generation and RL methodology targeting multi-step optimization scenarios. This approach, called Step-Wise Reinforcement Learning (SWiRL), iteratively generates multi-step reasoning and tool use data, and then learns from that data. It employs a simple step-wise decomposition that breaks each multi-step trajectory into multiple sub-trajectories corresponding to each action by the original model. It then applies synthetic data filtering and RL optimization on these sub-trajectories. We evaluated SWiRL on a number of multi-step tool use, question answering, and mathematical reasoning tasks. Our experiments show that SWiRL outperforms baseline approaches by 21.5%, 12.3%, 14.8%, 11.1%, and 15.3% in relative accuracy on GSM8K, HotPotQA, CofCA, MuSiQue, and BeerQA, respectively. Excitingly, the approach exhibits generalization across tasks: for example, training only on HotPotQA (text question-answering) improves zero-shot performance on GSM8K (a math dataset) by a relative 16.9%. © 2025, CC BY.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Robust Knowledge Distillation from RNN-T Models with Noisy Training Labels Using Full-Sum Loss

Robust Knowledge Distillation from RNN-T Models with Noisy T...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Mohammad Zeineldeen Kartik Audhkhasi Murali Karthick Baskar Bhuvana Ramabhadran Computer Science Department Human Language Technology and Pattern Recognition RWTH Aachen University Aachen Germany Google LLC New York

This work studies knowledge distillation (KD) and addresses its constraints for recurrent neural network transducer (RNN-T) models. In hard distillation, a teacher model transcribes large amounts of unlabelled speech to train a student model. Soft distillation is another popular KD method that distills the output logits of the teacher model. Due to the nature of RNN-T alignments, applying soft distillation between RNNT architectures having different posterior distributions is challenging. In addition, bad teachers having high word-error-rate (WER) reduce the efficacy of KD. We investigate how to effectively distill knowledge from variable quality ASR teachers, which has not been studied before to the best of our knowledge. We show that a sequence-level KD, full-sum distillation, outperforms other distillation methods for RNN-T models, especially for bad teachers. We also propose a variant of full-sum distillation that distills the sequence discriminative knowledge of the teacher leading to further improvement in WER. We conduct experiments on public datasets namely SpeechStew and LibriSpeech, and on in-house production data.

关键词： Knowledge engineering Training Recurrent neural networks Transducers Production Signal processing Acoustics

来源：评论

学校读者我要写书评

暂无评论

Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems

arXiv

引用

arXiv 2022年

作者： Sitawarin, Chawin Tramèr, Florian Carlini, Nicholas Department of Computer Science University of California Berkeley United States ETH Zürich Zürich Switzerland Google DeepMind Mountain View United States

Decision-based attacks construct adversarial examples against a machine learning (ML) model by making only hard-label queries. These attacks have mainly been applied directly to standalone neural networks. However, in practice, ML models are just one component of a larger learning system. We find that by adding a single preprocessor in front of a classifier, state-of-the-art query-based attacks are up to seven× less effective at attacking a prediction pipeline than at attacking the model alone. We explain this discrepancy by the fact that most preprocessors introduce some notion of invariance to the input space. Hence, attacks that are unaware of this invariance inevitably waste a large number of queries to re-discover or overcome it. We therefore develop techniques to (i) reverse-engineer the preprocessor and then (ii) use this extracted information to attack the end-to-end system. Our preprocessors extraction method requires only a few hundreds queries, and our preprocessor-aware attacks recover the same efficacy as when attacking the model alone. The code can be found at https://***/google-research/ preprocessor-aware-black-box-attack. © 2022, CC BY-NC-SA.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiency

arXiv

引用

arXiv 2023年

作者： Zhao, Heyang He, Jiafan Zhou, Dongruo Zhang, Tong Gu, Quanquan Department of Computer Science University of California Los AngelesCA90095 United States Google Research The Hong Kong University of Science and Technology Hong Kong

Recently, several studies (Zhou et al., 2021a;Zhang et al., 2021b;Kim et al., 2021;Zhou and Gu, 2022) have provided variance-dependent regret bounds for linear contextual bandits, which interpolates the regret for the worst-case regime and the deterministic reward regime. However, these algorithms are either computationally intractable or unable to handle unknown variance of the noise. In this paper, we present a novel solution to this open problem by proposing the first computationally efficient algorithm for linear bandits with heteroscedastic noise. Our algorithm is adaptive to the unknown variance of noise and achieves an (Equation presented) regret, where σk2 is the variance of the noise at the round k, d is the dimension of the contexts and K is the total number of rounds. Our results are based on an adaptive variance-aware confidence set enabled by a new Freedman-type concentration inequality for self-normalized martingales and a multi-layer structure to stratify the context vectors into different layers with different uniform upper bounds on the uncertainty. Furthermore, our approach can be extended to linear mixture Markov decision processes (MDPs) in reinforcement learning. We propose a variance-adaptive algorithm for linear mixture MDPs, which achieves a problem-dependent horizon-free regret bound that can gracefully reduce to a nearly constant regret for deterministic MDPs. Unlike existing nearly minimax optimal algorithms for linear mixture MDPs, our algorithm does not require explicit variance estimation of the transitional probabilities or the use of high-order moment estimators to attain horizon-free regret. We believe the techniques developed in this paper can have independent value for general online decision making problems. Copyright © 2023, The Authors. All rights reserved.

关键词： Markov processes

来源：评论

学校读者我要写书评

暂无评论

Span Attention for Entity-Consistent Task-Oriented Dialogue Response Generation

Span Attention for Entity-Consistent Task-Oriented Dialogue ...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Jiale Chen Xuelian Dong Wenxiu Xie Tao Gong Fu Lee Wang Tianyong Hao School of Computer Science South China Normal University Guangzhou China Department of Computer Science City University of Hong Kong Hong Kong China Google Inc. New York USA School of Science and Technology Hong Kong Metropolitan University Hong Kong China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Task-oriented dialogue systems have recently gained increasing attention due to their capability of using natural language to fulfill specific user demands, such as restaurant reservation and hotel booking. Recent works directly model task-oriented dialogue response as a text generation task. However, these methods, utilizing generated response tokens as an attention query to obtain the vanilla attention distribution over an entire knowledge base, frequently lead to an entity inconsistency in final response generation. To tackle this problem, we propose a novel attention mechanism called span attention and a novel model named Span Attention GEnerator (SAGE). The span attention computes an attention score between a query vector and each knowledge record vector instead of computing a vanilla attention score among word vectors, which consisted of dialogue context and knowledge base. For effective training, we propose an attention constraint strategy that utilizes the entities appearing in response as pseudo-labels to supervise the training of the span attention. Experiments based on three publicly accessible datasets demonstrate that our model, utilizing the proposed mechanism, outperforms the state-of-the-art models with improvements of 8.94%, 2.29%, and 10.22% respectively in Entity F1.

关键词： Training Attention mechanisms Computational modeling Knowledge based systems Natural languages Signal processing Vectors Generators Acoustics Speech processing

来源：评论

学校读者我要写书评

暂无评论

Bidirectional Human-AI Alignment: Emerging Challenges and Opportunities

Bidirectional Human-AI Alignment: Emerging Challenges and Op...

引用

2025 CHI Conference on Human Factors in Computing Systems, CHI EA 2025

作者： Shen, Hua Knearem, Tiffany Ghosh, Reshmi Liu, Michael Xieyang Monroy-Hernández, Andrés Wu, Tongshuang Yang, Diyi Huang, Yun Mitra, Tanushree Li, Yang Hearst, Marti The Information School University of Washington Seattle WA United States Google San Francisco CA United States Microsoft Corp Microsoft AI Development Acceleration Program Cambridge MA United States Google DeepMind Pittsburgh PA United States Princeton University Princeton NJ United States Human-Computer Interaction Institute Carnegie Mellon University Pittsburgh PA United States Computer Science Department Stanford University Stanford CA United States School of Information Sciences University of Illinois at Urbana-Champaign Champaign IL United States University of Washington Seattle WA United States Google Research Mountain View CA United States UC Berkeley Berkeley CA United States

ISBN: (纸本)9798400713958

Recent advancements in general-purpose AI have highlighted the urgent need to align AI systems with the goals, ethical principles, and values of individuals and society. Existing alignment research has been primarily approached as an AI-centered, static, and unidirectional process. However, this unidirectional perspective falls short of taking into account the dynamic and evolving interaction between humans and AI, necessitating a shift toward a bidirectional, interconnected mode of human-AI alignment. This SIG aims to outline the emerging areas of bidirectinoal human-AI alignment research, propose a blueprint of future goals and challenges for fundamental alignment research, and establish a shared platform to bring together experts from HCI, AI, social sciences, and more to advance interdisciplinary research and collaboration on human-AI alignment. © 2025 Copyright held by the owner/author(s).

关键词： bidirectional human-AI alignment human-AI interactive alignment social impact of AI alignment value-centered design

来源：评论

学校读者我要写书评

暂无评论

Decoupling Semantic Similarity from Spatial Alignment for Neural Networks

arXiv

引用

arXiv 2024年

作者： Wald, Tassilo Ulrich, Constantin Köhler, Gregor Zimmerer, David Denner, Stefan Baumgartner, Michael Isensee, Fabian Jaini, Priyank Maier-Hein, Klaus H. Heidelberg Germany Helmholtz Imaging DKFZ Heidelberg Germany Faculty of Mathematics and Computer Science University of Heidelberg Germany Medical Faculty Heidelberg University of Heidelberg Germany Google Deepmind United Kingdom Pattern Analysis and Learning Group Department of Radiation Oncology Germany Heidelberg Germany

What representation do deep neural networks learn? How similar are images to each other for neural networks? Despite the overwhelming success of deep learning methods key questions about their internal workings still remain largely unanswered, due to their internal high dimensionality and complexity. To address this, one approach is to measure the similarity of activation responses to various inputs. Representational Similarity Matrices (RSMs) distill this similarity into scalar values for each input pair. These matrices encapsulate the entire similarity structure of a system, indicating which input leads to similar responses. While the similarity between images is ambiguous, we argue that the spatial location of semantic objects does neither influence human perception nor deep learning classifiers. Thus this should be reflected in the definition of similarity between image responses for computer vision systems. Revisiting the established similarity calculations for RSMs we expose their sensitivity to spatial alignment. In this paper, we propose to solve this through semantic RSMs, which are invariant to spatial permutation. We measure semantic similarity between input responses by formulating it as a set-matching problem. Further, we quantify the superiority of semantic RSMs over spatio-semantic RSMs through image retrieval and by comparing the similarity between representations to the similarity between predicted class probabilities. © 2024, CC BY-SA.

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：