检索结果-内蒙古大学图书馆

Monitoring AI-modified content at scale: a case study on the impact of ChatGPT on AI conference peer reviews 24

学校读者我要写书评

暂无评论

Monitoring AI-modified content at scale: a case study on the...

Proceedings of the 41st International Conference on Machine Learning

作者： Weixin Liang Zachary Izzo Yaohui Zhang Haley Lepp Hancheng Cao Xuandong Zhao Lingjiao Chen Haotian Ye Sheng Liu Zhi Huang Daniel A. McFarland James Y. Zou Department of Computer Science Stanford University Machine Learning Department NEC Labs America Department of Electrical Engineering Stanford University Graduate School of Education Stanford University Department of Computer Science and Department of Management Science and Engineering Stanford University Department of Computer Science UC Santa Barbara Department of Biomedical Data Science Stanford University Graduate School of Education and Department of Sociology and Graduate School of Business Stanford University Department of Computer Science and Department of Electrical Engineering and Department of Biomedical Data Science Stanford University

We present an approach for estimating the fraction of text in a large corpus which is likely to be substantially modified or produced by a large language model (LLM). Our maximum likelihood model leverages expert-written and AI-generated reference texts to accurately and efficiently examine real-world LLM-use at the corpus level. We apply this approach to a case study of scientific peer review in AI conferences that took place after the release of ChatGPT: ICLR 2024, NeurIPS 2023, CoRL 2023 and EMNLP 2023. Our results suggest that between 6.5% and 16.9% of text submitted as peer reviews to these conferences could have been substantially modified by LLMs, i.e. beyond spell-checking or minor writing updates. The circumstances in which generated text occurs offer insight into user behavior: the estimated fraction of LLM-generated text is higher in reviews which report lower confidence, were submitted close to the deadline, and from reviewers who are less likely to respond to author rebuttals. We also observe corpus-level trends in generated text which may be too subtle to detect at the individual level, and discuss the implications of such trends on peer review. We call for future interdisciplinary work to examine how LLM use is changing our information and knowledge practices.

关键词：

Edge Computing with Early Exiting for Adaptive Inference in Mobile Autonomous Systems

学校读者我要写书评

暂无评论

Edge Computing with Early Exiting for Adaptive Inference in ...

IEEE International Conference on Communications (ICC)

作者： Simone Angelucci Roberto Valentini Marco Levorato Fortunato Santucci Carla Fabiana Chiasserini Dept. of Information Engineering Computer Science and Mathematics Centre Ex-EMERGE University of L'Aquila IT The Donald Bren School of Information and Computer Science UC Irvine CA US Dept. of Electronics and Telecommunications Politecnico di Torino Torino IT

ISBN: (数字)9781728190549

ISBN: (纸本)9781728190556

Early Exiting (EE) is an emerging computing paradigm where Deep Neural Networks (DNNs) are equipped with earlier classifiers, enabling trading-off accuracy with inference latency. EE can be effectively combined with edge computing, a paradigm that allows mobile nodes to offload complex tasks, such as the execution of DNNs, to servers at the edge of the network, thus reducing computing times and energy consumption at the mobile devices. The integration of such technologies is particularly attractive for the support of applications for connected and automated driving. In this paper, we consider a system that jointly leverages the benefits of EE and edge computing, and we model their complex interactions by means of a Markov Decision Process (MDP). We then formulate an optimization problem to select the inference strategy that maximizes the average task accuracy. Importantly, such an optimization problem has low complexity, as the optimal policy can be derived by mapping the MDP into a linear program. Our numerical results focus on a use case centered on automated vehicles connected with an edge server under varying channel and network conditions, and show that our solution achieves up to 11% higher accuracy compared to the optimal policy with no EE.

关键词： Accuracy Computational modeling Markov decision processes Artificial neural networks Mobile handsets Servers Task analysis

Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Majumder, Bodhisattwa Prasad Jhamtani, Harsh Berg-Kirkpatrick, Taylor McAuley, Julian Department Of Computer Science And Engineering UC San Diego United States School Of Computer Science Carnegie Mellon University United States

A limitation of current neural dialog models is that they tend to suffer from a lack of specificity and informativeness in generated responses, primarily due to dependence on training data that covers a limited variety of scenarios and conveys limited knowledge. One way to alleviate this issue is to extract relevant knowledge from external sources at decoding time and incorporate it into the dialog response. In this paper, we propose a posthoc knowledge-injection technique where we first retrieve a diverse set of relevant knowledge snippets conditioned on both the dialog history and an initial response from an existing dialog model. We construct multiple candidate responses, individually injecting each retrieved snippet into the initial response using a gradient-based decoding method, and then select the final response with an unsupervised ranking step. Our experiments in goal-oriented and knowledge-grounded dialog settings demonstrate that human annotators judge the outputs from the proposed method to be more engaging and informative compared to responses from prior dialog systems. We further show that knowledge-augmentation promotes success in achieving conversational goals in both experimental settings. © 2022, CC BY.

关键词： Decoding

A Generalized Acquisition Function for Preference-based Reward Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ellis, Evan Ghosal, Gaurav R. Russell, Stuart J. Dragan, Anca Bıyık, Erdem Department of Electrical Engineering and Computer Sciences UC Berkeley United States Machine Learning Department Carnegie Mellon University United States Thomas Lord Department of Computer Science University of Southern California United States

Preference-based reward learning is a popular technique for teaching robots and autonomous systems how a human user wants them to perform a task. Previous works have shown that actively synthesizing preference queries to maximize information gain about the reward function parameters improves data efficiency. The information gain criterion focuses on precisely identifying all parameters of the reward function. This can potentially be wasteful as many parameters may result in the same reward, and many rewards may result in the same behavior in the downstream tasks. Instead, we show that it is possible to optimize for learning the reward function up to a behavioral equivalence class, such as inducing the same ranking over behaviors, distribution over choices, or other related definitions of what makes two rewards similar. We introduce a tractable framework that can capture such definitions of similarity. Our experiments in a synthetic environment, an assistive robotics environment with domain transfer, and a natural language processing problem with real datasets demonstrate the superior performance of our querying method over the state-of-the-art information gain method. © 2024, CC BY.

关键词： Equivalence classes

A New Federated Learning Framework Against Gradient Inversion Attacks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Guo, Pengxin Zeng, Shuang Chen, Wenhao Zhang, Xiaodan Ren, Weihong Zhou, Yuyin Qu, Liangqiong School of Computing and Data Science The University of Hong Kong Hong Kong Department of Mathematics The University of Hong Kong Hong Kong College of Computer Science Beijing University of Technology China School of Mechanical Engineering and Automation Harbin Institute of Technology Shenzhen China Department of Computer Science and Engineering UC Santa Cruz United States

Federated Learning (FL) aims to protect data privacy by enabling clients to collectively train machine learning models without sharing their raw data. However, recent studies demonstrate that information exchanged during FL is subject to Gradient Inversion Attacks (GIA) and, consequently, a variety of privacy-preserving methods have been integrated into FL to thwart such attacks, such as Secure Multi-party Computing (SMC), Homomorphic Encryption (HE), and Differential Privacy (DP). Despite their ability to protect data privacy, these approaches inherently involve substantial privacy-utility trade-offs. By revisiting the key to privacy exposure in FL under GIA, which lies in the frequent sharing of model gradients that contain private data, we take a new perspective by designing a novel privacy preserve FL framework that effectively "breaks the direct connection" between the shared parameters and the local private data to defend against GIA. Specifically, we propose a Hypernetwork Federated Learning (HyperFL) framework that utilizes hypernetworks to generate the parameters of the local model and only the hypernetwork parameters are uploaded to the server for aggregation. Theoretical analyses demonstrate the convergence rate of the proposed HyperFL, while extensive experimental results show the privacy-preserving capability and comparable performance of HyperFL. Code is available at https://***/Pengxin-Guo/HyperFL. © 2024, CC BY.

关键词： Federated learning

State-driven Implicit Modeling for Sparsity and Robustness in Neural Networks

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Tsai, Alicia Y. Decugis, Juliette El Ghaoui, Laurent Atamtürk, Alper Department of Electrical Engineering and Computer Science UC Berkeley United States Department of Mathematics UC Berkeley United States Department of Industrial Engineering and Operations Research UC Berkeley United States

Implicit models are a general class of learning models that forgo the hierarchical layer structure typical in neural networks and instead define the internal states based on an "equilibrium" equation, offering competitive performance and reduced memory consumption. However, training such models usually relies on expensive implicit differentiation for backward propagation. In this work, we present a new approach to training implicit models, called State-driven Implicit Modeling (SIM), where we constrain the internal states and outputs to match that of a baseline model, circumventing costly backward computations. The training problem becomes convex by construction and can be solved in a parallel fashion, thanks to its decomposable structure. We demonstrate how the SIM approach can be applied to significantly improve sparsity (parameter reduction) and robustness of baseline models trained on FashionMNIST and CIFAR-100 datasets. © 2022, CC BY.

关键词： Multilayer neural networks

LiteCON: An All-Photonic Neuromorphic Accelerator for Energy-efficient Deep Learning (Preprint)

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Dang, Dharanidhar Lin, Bill Sahoo, Debashis Department of Pediatrics Department of Computer Science & Engineering UC San Diego CA92093 United States Department of Electrical & Computer Engineering UC San Diego CA92093 United States

Deep learning is highly pervasive in today's data-intensive era. In particular, convolutional neural networks (CNNs) are being widely adopted in a variety of fields for superior accuracy. However, computing deep CNNs on traditional CPUs and GPUs brings several performance and energy pitfalls. Several novel approaches based on ASIC, FPGA, and resistive-memory devices have been recently demonstrated with promising results. Most of them target only the inference (testing) phase of deep learning. There have been very limited attempts to design a full-fledged deep learning accelerator capable of both training and inference. It is due to the highly compute and memory intensive nature of the training phase. In this paper, we propose LiteCON, a novel analog photonics CNN accelerator. LiteCON uses silicon microdisk-based convolution, memristor-based memory, and dense-wavelength-division-multiplexing for energy-efficient and ultrafast deep learning. We evaluate LiteCON using a commercial CAD framework (IPKISS) on deep learning benchmark models including LeNet and VGG-Net. Compared to the state-of-the-art, LiteCON improves the CNN throughput, energy-efficiency, and computational efficiency by up to 32×, 37×, and 5× respectively with trivial accuracy degradation. © 2022, CC BY.

关键词： Memristors

All Things Considered: Detecting Partisan Events from News Media with Cross-Article Comparison

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Yujian Zhang, Xinliang Frederick Zou, Kaijian Huang, Ruihong Beauchamp, Nick Wang, Lu Computer Science UC Santa Barbara Santa BarbaraCA United States Computer Science and Engineering University of Michigan Ann ArborMI United States Computer Science and Engineering Texas A&M University College StationTX United States Department of Political Science Northeastern University BostonMA United States

Public opinion is shaped by the information news media provide, and that information in turn may be shaped by the ideological preferences of media outlets. But while much attention has been devoted to media bias via overt ideological language or topic selection, a more unobtrusive way in which the media shape opinion is via the strategic inclusion or omission of partisan events that may support one side or the other. We develop a latent variable-based framework to predict the ideology of news articles by comparing multiple articles on the same story and identifying partisan events whose inclusion or omission reveals ideology. Our experiments first validate the existence of partisan event selection, and then show that article alignment and cross-document comparison detect partisan events and article ideology better than competitive baselines. Our results reveal the high-level form of media bias, which is present even among mainstream media with strong norms of objectivity and nonpartisan-ship. Our codebase and dataset are available at https://***/launchnlp/ATC. Copyright © 2023, The Authors. All rights reserved.

关键词： Social aspects

A theoretical framework for inference learning 22

学校读者我要写书评

暂无评论

A theoretical framework for inference learning

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Nick Alonso Beren Millidge Jeff Krichmar Emre Neftci Department of Cognitive Science UC Irvine MRC Brain Network Dynamics Unit University of Oxford Department of Cognitive Science UC Irvine and Department of Computer Science UC Irvine Department of Cognitive Science UC Irvine and Department of Computer Science UC Irvine and Electrical Engineering and Information Technology RWTH Aachen Germany and Peter Grünberg Institute Forschungszentrum Jülich Germany

ISBN: (纸本)9781713871088

Backpropagation (BP) is the most successful and widely used algorithm in deep learning. However, the computations required by BP are challenging to reconcile with known neurobiology. This difficulty has stimulated interest in more biologically plausible alternatives to BP. One such algorithm is the inference learning algorithm (IL). IL trains predictive coding models of neural circuits and has achieved equal performance to BP on supervised and auto-associative tasks. In contrast to BP, however, the mathematical foundations of IL are not well-understood. Here, we develop a novel theoretical framework for IL. Our main result is that IL closely approximates an optimization method known as implicit stochastic gradient descent (implicit SGD), which is distinct from the explicit SGD implemented by BP. Our results further show how the standard implementation of IL can be altered to better approximate implicit SGD. Our novel implementation considerably improves the stability of IL across learning rates, which is consistent with our theory, as a key property of implicit SGD is its stability. We provide extensive simulation results that further support our theoretical interpretations and find IL achieves quicker convergence when trained with mini-batch size one while performing competitively with BP for larger mini-batches when combined with Adam.

关键词：