检索结果-内蒙古大学图书馆

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Ye Zhu Yu Wu Zhiwei Deng Olga Russakovsky Yan Yan Department of Computer Science Illinois Institute of Technology and Department of Computer Science Princeton University School of Computer Science Wuhan University Google Research Department of Computer Science Princeton University Department of Computer Science Illinois Institute of Technology

Applying pre-trained generative denoising diffusion models (DDMs) for downstream tasks such as image semantic editing usually requires either fine-tuning DDMs or learning auxiliary editing networks in the existing literature. In this work, we present our BoundaryDijjusion method for efficient, effective and light-weight semantic control with frozen pre-trained DDMs, without learning any extra networks. As one of the first learning-free diffusion editing works, we start by seeking a comprehensive understanding of the intermediate high-dimensional latent spaces by theoretically and empirically analyzing their probabilistic and geometric behaviors in the Markov chain. We then propose to further explore the critical step for editing in the denoising trajectory that characterizes the convergence of a pre-trained DDM and introduce an automatic search method. Last but not least, in contrast to the conventional understanding that DDMs have relatively poor semantic behaviors, we prove that the critical latent space we found already exhibits semantic subspace boundaries at the generic level in unconditional DDMs, which allows us to do controllable manipulation by guiding the denoising trajectory towards the targeted boundary via a single-step operation. We conduct extensive experiments on multiple DPMs architectures (DDPM, iDDPM) and datasets (CelebA, CelebA-HQ, LSUN-church, LSUN-bedroom, AFHQ-dog) with different resolutions (64, 256), achieving superior or state-of-the-art performance in various task scenarios (image semantic editing, text-based editing, unconditional semantic control) to demonstrate the effectiveness. Project page at https://***/BoundaryDijfusion/.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Information complexity of stochastic convex optimization: applications to generalization, memorization, and tracing 24

Information complexity of stochastic convex optimization: ap...

引用

Proceedings of the 41st International Conference on Machine Learning

作者： Idan Attias Gintare Karolina Dziugaite Mahdi Haghifam Roi Livni Daniel M. Roy Department of Computer Science Ben-Gurion University and Vector Institute Google DeepMind Khoury College of Computer Sciences Northeastern University Department of Electrical Engineering Tel Aviv University Department of Statistical Sciences University of Toronto and Vector Institute

In this work, we investigate the interplay between memorization and learning in the context of stochastic convex optimization (SCO). We define memorization via the information a learning algorithm reveals about its training data points. We then quantify this information using the framework of conditional mutual information (CMI) proposed by Steinke and Zakynthinou [SZ20]. Our main result is a precise characterization of the tradeoff between the accuracy of a learning algorithm and its CMI, answering an open question posed by Livni [Liv23]. We show that, in the L2 Lipschitz-bounded setting and under strong convexity, every learner with an excess error ε has CMI bounded below by Ω(1/ε2) and Ω(1/ε), respectively. We further demonstrate the essential role of memorization in learning problems in SCO by designing an adversary capable of accurately identifying a significant fraction of the training samples in specific SCO problems. Finally, we enumerate several implications of our results, such as a limitation of generalization bounds based on CMI and the incompressibility of samples in SCO problems.

关键词：

来源：评论

学校读者我要写书评

暂无评论

REDUCR: Robust Data Downsampling using Class Priority Reweighting

arXiv

引用

arXiv 2023年

作者： Bankes, William Hughes, George Bogunovic, Ilija Wang, Zi Department of Computer Science University College London United Kingdom Department of Electrical Engineering University College London United Kingdom Google DeepMind United Kingdom

Modern machine learning models are becoming increasingly expensive to train for real-world image and text classification tasks, where massive web-scale data is collected in a streaming fashion. To reduce the training cost, online batch selection techniques have been developed to choose the most informative datapoints. However, many existing techniques are not robust to class imbalance and distributional shifts, and can suffer from poor worst-class generalization performance. This work introduces REDUCR, a robust and efficient data downsampling method that uses class priority reweighting. REDUCR reduces the training data while preserving worst-class generalization performance. REDUCR assigns priority weights to datapoints in a class-aware manner using an online learning algorithm. We demonstrate the data efficiency and robust performance of REDUCR on vision and text classification tasks. On web-scraped datasets with imbalanced class distributions, REDUCR significantly improves worst-class test accuracy (and average accuracy), surpassing state-of-the-art methods by around 15%. © 2023, CC BY.

关键词： Adversarial machine learning

来源：评论

学校读者我要写书评

暂无评论

Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Task

arXiv

引用

arXiv 2022年

作者： Kossen, Jannik Cangea, Cătălina Vértes, Eszter Jaegle, Andrew Patraucean, Viorica Ktena, Ira Tomasev, Nenad Belgrave, Danielle OATML Department of Computer Science University of Oxford United Kingdom Google DeepMind United Kingdom

We introduce a challenging decision-making task that we call active acquisition for multimodal temporal data (A2MT). In many real-world scenarios, input features are not readily available at test time and must instead be acquired at significant cost. With A2MT, we aim to learn agents that actively select which modalities of an input to acquire, trading off acquisition cost and predictive performance. A2MT extends a previous task called active feature acquisition to temporal decision making about high-dimensional inputs. We propose a method based on the Perceiver IO architecture to address A2MT in practice. Our agents are able to solve a novel synthetic scenario requiring practically relevant cross-modal reasoning skills. On two large-scale, real-world datasets, Kinetics-700 and AudioSet, our agents successfully learn cost-reactive acquisition behavior. However, an ablation reveals they are unable to learn adaptive acquisition strategies, emphasizing the difficulty of the task even for state-of-the-art models. Applications of A2MT may be impactful in domains like medicine, robotics, or finance, where modalities differ in acquisition cost and informativeness. Copyright © 2022, The Authors. All rights reserved.

关键词： Decision making

来源：评论

学校读者我要写书评

暂无评论

Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs

arXiv

引用

arXiv 2024年

作者： Guo, Siyuan Didolkar, Aniket Ke, Nan Rosemary Goyal, Anirudh Huszár, Ferenc Schölkopf, Bernhard Max Planck Institute for Intelligent Systems Tübingen Germany Department of Computer Science University of Cambridge United Kingdom Mila University of Montreal Canada Google DeepMind United Kingdom

We are beginning to see progress in language model assisted scientific discovery. Motivated by the use of LLMs as a general scientific assistant, this paper assesses the domain knowledge of LLMs through its understanding of different mathematical skills required to solve problems. In particular, we look at not just what the pre-trained model already knows, but how it learned to learn from information during in-context learning or instruction-tuning through exploiting the complex knowledge structure within mathematics. Motivated by the Neural Tangent Kernel (NTK), we propose NTKEval to assess changes in LLM’s probability distribution via training on different kinds of math data. Our systematic analysis finds evidence of domain understanding during in-context learning. By contrast, certain instruction-tuning leads to similar performance changes irrespective of training on different data, suggesting a lack of domain understanding across different skills. © 2024, CC BY.

关键词： Pattern matching

来源：评论

学校读者我要写书评

暂无评论

Chain of Code: Reasoning with a Language Model-Augmented Code Emulator

arXiv

引用

arXiv 2023年

作者： Li, Chengshu Liang, Jacky Zeng, Andy Chen, Xinyun Hausman, Karol Sadigh, Dorsa Levine, Sergey Fei-Fei, Li Xia, Fei Ichter, Brian Department of Computer Science Stanford University CA United States Google DeepMind CA United States Department of Electrical Engineering and Computer Sciences University of California BerkeleyCA United States

Code provides a general syntactic structure to build complex programs and perform precise computations when paired with a code interpreter - we hypothesize that language models (LMs) can leverage code-writing to improve Chain of Thought reasoning not only for logic and arithmetic tasks (Chen et al., 2022;Nye et al., 2021;Austin et al., 2021), but also for semantic ones (and in particular, those that are a mix of both). For example, consider prompting an LM to write code that counts the number of times it detects sarcasm in an essay: the LM may struggle to write an implementation for "detect_sarcasm(string)" that can be executed by the interpreter (handling the edge cases would be insurmountable). However, LMs may still produce a valid solution if they not only write code, but also selectively "emulate" the interpreter by generating the expected output of "detect_sarcasm(string)". In this work, we propose Chain of Code (CoC), a simple yet surprisingly effective extension that improves LM code-driven reasoning. The key idea is to encourage LMs to format semantic sub-tasks in a program as flexible pseudocode that the interpreter can explicitly catch undefined behaviors and hand off to simulate with an LM (as an "LMulator"). Experiments demonstrate that Chain of Code outperforms Chain of Thought and other baselines across a variety of benchmarks;on BIG-Bench Hard, Chain of Code achieves 84%, a gain of 12% over Chain of Thought. In a nutshell, CoC broadens the scope of reasoning questions that LMs can answer by "thinking in code". Copyright © 2023, The Authors. All rights reserved.

关键词： Program interpreters

来源：评论

学校读者我要写书评

暂无评论

Addendum: Accurate structure prediction of biomolecular interactions with AlphaFold 3

引用

Nature 2024年第8042期636卷 E4页

作者： Abramson, Josh Adler, Jonas Dunger, Jack Evans, Richard Green, Tim Pritzel, Alexander Ronneberger, Olaf Willmore, Lindsay Ballard, Andrew J. Bambrick, Joshua Bodenstein, Sebastian W. Evans, David A. Hung, Chia-Chun Reiman, David Tunyasuvunakool, Kathryn Wu, Zachary Žemgulytė, Akvilė Arvaniti, Eirini Beattie, Charles Bertolli, Ottavia Bridgland, Alex Cherepanov, Alexey Congreve, Miles Cowen-Rivers, Alexander I. Cowie, Andrew Figurnov, Michael Fuchs, Fabian B. Gladman, Hannah Jain, Rishub Khan, Yousuf A. Low, Caroline M R Perlin, Kuba Potapenko, Anna Savy, Pascal Singh, Sukhdeep Stecula, Adrian Thillaisundaram, Ashok Tong, Catherine Yakneen, Sergei Zhong, Ellen D. Zielinski, Michal Žídek, Augustin Bapst, Victor Kohli, Pushmeet Jaderberg, Max Hassabis, Demis Jumper, John M. Google DeepMind London United Kingdom Isomorphic Labs London United Kingdom Google DeepMind London United Kingdom Isomorphic Labs London United Kingdom Department of Molecular and Cellular Physiology Stanford University Stanford CA United States Department of Computer Science Princeton University Princeton NJ United States Google DeepMind London United Kingdom Isomorphic Labs London United Kingdom

来源：评论

学校读者我要写书评

暂无评论

Computational search for materials having a giant anomalous Hall effect in the pyrochlore and spinel crystal structures

arXiv

引用

arXiv 2025年

作者： Sullivan, Sean Lee, Seungjun Szymanski, Nathan J. Merchant, Amil Cubuk, Ekin Dogus Low, Tony Bartel, Christopher J. Department of Chemical Engineering and Materials Science University of Minnesota MinneapolisMN55455 United States Department of Electrical Engineering and Computer Engineering University of Minnesota MinneapolisMN55455 United States Google DeepMind United Kingdom

Ferromagnetic pyrochlore and spinel materials with topological flat bands are of interest for their potential to exhibit a giant anomalous Hall effect (AHE). In this work, we present computational predictions of stability and electronic structure for 448 compositions within the pyrochlore (A2B2O7) and spinel (AB2O4) frameworks. Of these, 92 are predicted to be thermodynamically stable or close (2Ir2O7 exhibits an exceptionally high AHA of 0.728 when its Fermi level is optimized. These findings provide a roadmap for the targeted synthesis of new pyrochlore and spinel compounds with enhanced AHE properties. They also broaden the compositional design space for these structures and support the discovery of high-performance materials for next-generation spintronic applications. Copyright © 2025, The Authors. All rights reserved.

关键词： Crystal atomic structure

来源：评论

学校读者我要写书评

暂无评论

Simplicity bias in 1-hidden layer neural networks 23

Simplicity bias in 1-hidden layer neural networks

引用

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Depen Morwani Jatin Batra Prateek Jain Praneeth Netrapalli Department of Computer Science Harvard University School of Technology and Computer Science Tata Institute of Fundamental Research (TIFR) Google Research

Recent works (Shah et al., 2020; Chen et al., 2021) have demonstrated that neural networks exhibit extreme simplicity bias (SB). That is, they learn only the simplest features to solve a task at hand, even in the presence of other, more robust but more complex features. Due to the lack of a general and rigorous definition of features, these works showcase SB on semi-synthetic datasets such as Color-MNIST, MNIST-CIFAR where defining features is relatively *** this work, we rigorously define as well as thoroughly establish SB for one hidden layer neural networks. More concretely, (i) we define SB as the network essentially being a function of a low dimensional projection of the inputs (ii) theoretically, in the infinite width regime, we show that when the data is linearly separable, the network primarily depends on only the linearly separable (1-dimensional) subspace even in the presence of an arbitrarily large number of other, more complex features which could have led to a significantly more robust classifier, (iii) empirically, we show that models trained on real datasets such as Imagenet and Waterbirds-Landbirds indeed depend on a low dimensional projection of the inputs, thereby demonstrating SB on these datasets, iv) finally, we present a natural ensemble approach that encourages diversity in models by training successive models on features not used by earlier models, and demonstrate that it yields models that are significantly more robust to Gaussian noise.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Rule-Bottleneck Reinforcement Learning: Joint Explanation and Decision Optimization for Resource Allocation with Language Agents

arXiv

引用

arXiv 2025年

作者： Tec, Mauricio Xiong, Guojun Wang, Haichuan Dominici, Francesca Tambe, Milind Department of Computer Science Harvard John A. Paulson School of Engineering and Applied Sciences United States Department of Biostatistics Harvard T.H. Chan School of Public Health United States Google DeepMind United Kingdom

Deep Reinforcement Learning (RL) is remarkably effective in addressing sequential resource allocation problems in domains such as healthcare, public policy, and resource management. However, deep RL policies often lack transparency and adaptability, challenging their deployment alongside human decision-makers. In contrast, Language Agents, powered by large language models (LLMs), provide human-understandable reasoning but may struggle with effective decision making. To bridge this gap, we propose Rule-Bottleneck Reinforcement Learning (RBRL), a novel framework that jointly optimizes decision and explanations. At each step, RBRL generates candidate rules with an LLM, selects among them using an attention-based RL policy, and determines the environment action with an explanation via chain-of-thought reasoning. The RL rule selection is optimized using the environment rewards and an explainability metric judged by the LLM. Evaluations in real-world scenarios highlight RBRL’s competitive performance with deep RL and efficiency gains over LLM fine-tuning. A survey further confirms the enhanced quality of its explanations. © 2025, CC BY.

关键词： Deep reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：