检索结果-内蒙古大学图书馆

IEEE Transactions on Artificial Intelligence 2021年第2期2卷 200-212页

作者： Qiu, Chen Zhou, Guangyou Cai, Zhihua Søgaard, Anders School of Computer Science and Technology Wuhan University of Science and Technology Wuhan430081 China School of Computer Science Central China Normal University Wuhan430079 China School of Computer Science China University of Geosciences Wuhan430074 China Department of Computer Science University of Copenhagen Copenhagen1165 Denmark Google Research Copenhagen1353 Denmark

Knowledge-based question answering (KBQA) is an essential but challenging task for artificial intelligence and natural language processing. A key challenge pertains to the design of effective algorithms for relation detection. Conventional methods model questions and candidate relations separately through the knowledge bases (KBs) without considering the rich word-level interactions between them. This approach may result in local optimal results. This article presents a global–local attentive relation detection model (GLAR) that utilizes the local module to learn the features of word-level interactions and employs the global module to acquire nonlinear relationships between questions and their candidate relations located in KBs. This article also reports on the application of an end-to-end retrieval-based KBQA system incorporating the proposed relation detection model. Experimental results obtained on two datasets demonstrated GLAR’s remarkable performance in the relation detection task. Furthermore, the functioning of end-to-end KBQA systems was significantly improved through the relation detection model, whose results on both datasets outperformed even state-of-the-art methods. Impact Statement—Knowledge-based question answering (KBQA) aims at answering user questions posed over the knowledge bases (KBs). KBQA helps users access knowledge in the KBs more easily, and it works on two subtasks: entity mention detection and relation detection. While existing relation detection algorithms perform well on the global representation of questions and relations sequences, they ignore some local semantic information on interaction cases between them. The technology proposed in this article takes both global and local interactions into account. With superior improvement on two relation detection tasks and two KBQA end tasks, the technology provides more precise answers. It could be used in more applications, including intelligent customer service, intelligent finance, and oth

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization

arXiv

引用

arXiv 2024年

作者： Attias, Idan Dziugaite, Gintare Karolina Haghifam, Mahdi Livni, Roi Roy, Daniel M. Department of Computer Science Ben-Gurion University Vector Institute Google DeepMind United Kingdom Khoury College of Computer Sciences Northeastern University United States Department of Electrical Engineering Tel Aviv University Department of Statistical Sciences University of Toronto Vector Institute Canada

In this work, we investigate the interplay between memorization and learning in the context of stochastic convex optimization (SCO). We define memorization via the information a learning algorithm reveals about its training data points. We then quantify this information using the framework of conditional mutual information (CMI) proposed by Steinke and Zakynthinou [SZ20]. Our main result is a precise characterization of the tradeoff between the accuracy of a learning algorithm and its CMI, answering an open question posed by Livni [Liv23]. We show that, in the L2 Lipschitz–bounded setting and under strong convexity, every learner with an excess error Ε has CMI bounded below by Ω(1/Ε2) and Ω(1/Ε), respectively. We further demonstrate the essential role of memorization in learning problems in SCO by designing an adversary capable of accurately identifying a significant fraction of the training samples in specific SCO problems. Finally, we enumerate several implications of our results, such as a limitation of generalization bounds based on CMI and the incompressibility of samples in SCO problems. Copyright © 2024, The Authors. All rights reserved.

关键词： Stochastic systems

来源：评论

学校读者我要写书评

暂无评论

REVEALING THE 3D COSMIC WEB THROUGH GRAVITATIONALLY CONSTRAINED NEURAL FIELDS

arXiv

引用

arXiv 2025年

作者： Zhao, Brandon Levis, Aviad Connor, Liam Srinivasan, Pratul P. Bouman, Katherine L. Department of Computing and Mathematical Sciences California Institute of Technology United States Department of Computer Science University of Toronto Canada David A. Dunlap Department of Astronomy & Astrophysics University of Toronto Canada Center for Astrophysics Harvard & Smithsonian United States Google DeepMind United Kingdom Departments of Astronomy and Electrical Engineering California Institute of Technology United States

Weak gravitational lensing is the slight distortion of galaxy shapes caused primarily by the gravitational effects of dark matter in the universe. In our work, we seek to invert the weak lensing signal from 2D telescope images to reconstruct a 3D map of the universe’s dark matter field. While inversion typically yields a 2D projection of the dark matter field, accurate 3D maps of the dark matter distribution are essential for localizing structures of interest and testing theories of our universe. However, 3D inversion poses significant challenges. First, unlike standard 3D reconstruction that relies on multiple viewpoints, in this case, images are only observed from a single viewpoint. This challenge can be partially addressed by observing how galaxy emitters throughout the volume are lensed. However, this leads to the second challenge: the shapes and exact locations of unlensed galaxies are unknown, and can only be estimated with a very large degree of uncertainty. This introduces an overwhelming amount of noise which nearly drowns out the lensing signal completely. Previous approaches tackle this by imposing strong assumptions about the structures in the volume. We instead propose a methodology using a gravitationally-constrained neural field to flexibly model the continuous matter distribution. We take an analysis-by-synthesis approach, optimizing the weights of the neural network through a fully differentiable physical forward model to reproduce the lensing signal present in image measurements. We showcase our method on simulations, including realistic simulated measurements of dark matter distributions that mimic data from upcoming telescope surveys. Our results show that our method can not only outperform previous methods, but importantly is also able to recover potentially surprising dark matter structures. © 2025, CC BY.

关键词： Dark Matter

来源：评论

学校读者我要写书评

暂无评论

MORE EXPERTS THAN GALAXIES: CONDITIONALLY-OVERLAPPING EXPERTS WITH BIOLOGICALLY-INSPIRED FIXED ROUTING

arXiv

引用

arXiv 2024年

作者： Shaier, Sagi Pereira, Francisco von der Wense, Katharina Hunter, Lawrence E. Jones, Matt Department of Computer Science University of Colorado Boulder United States Machine Learning Core National Institute of Mental Health United States Department of Computer Science University of Colorado Boulder Institute of Computer Science Johannes Gutenberg University Mainz Germany Department of Pediatrics University of Chicago United States Department of Psychology and Neuroscience University of Colorado Boulder Google DeepMind United States

The evolution of biological neural systems has led to both modularity and sparse coding, which enables energy efficiency and robustness across the diversity of tasks in the lifespan. In contrast, standard neural networks rely on dense, non-specialized architectures, where all model parameters are simultaneously updated to learn multiple tasks, leading to interference. Current sparse neural network approaches aim to alleviate this issue but are hindered by limitations such as 1) trainable gating functions that cause representation collapse, 2) disjoint experts that result in redundant computation and slow learning, and 3) reliance on explicit input or task IDs that limit flexibility and scalability. In this paper we propose Conditionally Overlapping Mixture of ExperTs (COMET), a general deep learning method that addresses these challenges by inducing a modular, sparse architecture with an exponential number of overlapping experts. COMET replaces the trainable gating function used in Sparse Mixture of Experts with a fixed, biologically inspired random projection applied to individual input representations. This design causes the degree of expert overlap to depend on input similarity, so that similar inputs tend to share more parameters. This results in faster learning per update step and improved out-of-sample generalization. We demonstrate the effectiveness of COMET on a range of tasks, including image classification, language modeling, and regression, using several popular deep learning architectures. Code can be found here: https://***/Shaier/***. © 2024, CC BY.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Generative Agent Simulations of 1,000 People

arXiv

引用

arXiv 2024年

作者： Park, Joon Sung Zou, Carolyn Q. Shaw, Aaron Hill, Benjamin Mako Cai, Carrie Morris, Meredith Ringel Willer, Robb Liang, Percy Bernstein, Michael S. Computer Science Department Stanford University StanfordCA94305 United States Department of Communication Studies Northwestern University EvanstonIL60208 United States Department of Communication University of Washington SeattleWA98195 United States Google DeepMind Mountain ViewCA94043 United States Google DeepMind SeattleWA98195 United States Department of Sociology Stanford University StanfordCA94305 United States

The promise of human behavioral simulation—general-purpose computational agents that replicate human behavior across domains—could enable broad applications in policymaking and social science. We present a novel agent architecture that simulates the attitudes and behaviors of 1,052 real individuals—applying large language models to qualitative interviews about their lives, then measuring how well these agents replicate the attitudes and behaviors of the individuals that they represent. The generative agents replicate participants' responses on the General Social Survey 85% as accurately as participants replicate their own answers two weeks later, and perform comparably in predicting personality traits and outcomes in experimental replications. Our architecture reduces accuracy biases across racial and ideological groups compared to agents given demographic descriptions. This work provides a foundation for new tools that can help investigate individual and collective behavior. Copyright © 2024, The Authors. All rights reserved.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

Boundary Guided Learning-Free Semantic Control with Diffusion Models

arXiv

引用

arXiv 2023年

作者： Zhu, Ye Wu, Yu Deng, Zhiwei Russakovsky, Olga Yan, Yan Department of Computer Science Illinois Institute of Technology United States Department of Computer Science Princeton University United States School of Computer Science Wuhan University China Google Research United States

Applying pre-trained generative denoising diffusion models (DDMs) for downstream tasks such as image semantic editing usually requires either fine-tuning DDMs or learning auxiliary editing networks in the existing literature. In this work, we present our BoundaryDiffusion method for efficient, effective and lightweight semantic control with frozen pre-trained DDMs, without learning any extra networks. As one of the first learning-free diffusion editing works, we start by seeking a comprehensive understanding of the intermediate high-dimensional latent spaces by theoretically and empirically analyzing their probabilistic and geometric behaviors in the Markov chain. We then propose to further explore the critical step for editing in the denoising trajectory that characterizes the convergence of a pre-trained DDM and introduce an automatic search method. Last but not least, in contrast to the conventional understanding that DDMs have relatively poor semantic behaviors, we prove that the critical latent space we found already exhibits semantic subspace boundaries at the generic level in unconditional DDMs, which allows us to do controllable manipulation by guiding the denoising trajectory towards the targeted boundary via a single-step operation. We conduct extensive experiments on multiple DPMs architectures (DDPM, iDDPM) and datasets (CelebA, CelebA-HQ, LSUN-church, LSUN-bedroom, AFHQ-dog) with different resolutions (64, 256), achieving superior or state-of-the-art performance in various task scenarios (image semantic editing, text-based editing, unconditional semantic control) to demonstrate the effectiveness. Project page at https://***/BoundaryDiffusion/. © 2023, CC BY-NC-ND.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysis

arXiv

引用

arXiv 2023年

作者： Meulemans, Alexander Schug, Simon Kobayashi, Seijin Daw, Nathaniel D. Wayne, Gregory Department of Computer Science ETH Zürich Switzerland Google DeepMind United Kingdom Princeton Neuroscience Institute Princeton University United States Department of Psychology Princeton University United States

To make reinforcement learning more sample efficient, we need better credit assignment methods that measure an action’s influence on future rewards. Building upon Hindsight Credit Assignment (HCA) [1], we introduce Counterfactual Contribution Analysis (COCOA), a new family of model-based credit assignment algorithms. Our algorithms achieve precise credit assignment by measuring the contribution of actions upon obtaining subsequent rewards, by quantifying a counterfactual query: ‘Would the agent still have reached this reward if it had taken another action?’. We show that measuring contributions w.r.t. rewarding states, as is done in HCA, results in spurious estimates of contributions, causing HCA to degrade towards the high-variance REINFORCE estimator in many relevant environments. Instead, we measure contributions w.r.t. rewards or learned representations of the rewarding objects, resulting in gradient estimates with lower variance. We run experiments on a suite of problems specifically designed to evaluate long-term credit assignment capabilities. By using dynamic programming, we measure ground-truth policy gradients and show that the improved performance of our new model-based credit assignment methods is due to lower bias and variance compared to HCA and common baselines. Our results demonstrate how modeling action contributions towards rewarding outcomes can be leveraged for credit assignment, opening a new path towards sample-efficient reinforcement learning. © 2023, CC BY-NC-SA.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Tree-D Fusion: Simulation-Ready Tree Dataset from Single Images with Diffusion Priors

arXiv

引用

arXiv 2024年

作者： Lee, Jae Joong Li, Bosheng Beery, Sara Huang, Jonathan Fei, Songlin Yeh, Raymond A. Benes, Bedrich Purdue University Department of Computer Science United States Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science United States Google United States Purdue University Department of Forestry Natural Resources United States

We introduce Tree-D Fusion, featuring the first collection of 600,000 environmentally aware, 3D simulation-ready tree models generated through Diffusion priors. Each reconstructed 3D tree model corresponds to an image from google's Auto Arborist Dataset, comprising street view images and associated genus labels of trees across North America. Our method distills the scores of two tree-adapted diffusion models by utilizing text prompts to specify a tree genus, thus facilitating shape reconstruction. This process involves reconstructing a 3D tree envelope filled with point markers, which are subsequently utilized to estimate the tree's branching structure using the space colonization algorithm conditioned on a specified genus. © 2024, CC BY.

关键词： Diffusion

来源：评论

学校读者我要写书评

暂无评论

LoopTree: Exploring the Fused-Layer Dataflow Accelerator Design Space

IEEE Transactions on Circuits and Systems for Artificial Int...

引用

IEEE Transactions on Circuits and Systems for Artificial Intelligence 2024年第1期1卷 97-111页

作者： Michael Gilbert Yannan Nellie Wu Joel S. Emer Vivienne Sze Department of Electrical Engineering and Computer Science School of Engineering Massachusetts Institute of Technology Cambridge MA USA Google Mountain View CA USA NVIDIA Santa Clara CA USA

Latency and energy consumption are key metrics in the performance of deep neural network (DNN) accelerators. A significant factor contributing to latency and energy is data transfers. One method to reduce transfers or data is reusing data when multiple operations use the same data. Fused-layer accelerators reuse data across operations in different layers by retaining intermediate data in on-chip buffers, which has been shown to reduce energy consumption and latency. Moreover, the intermediate data is often tiled ( i.e., broken into chunks) to reduce the on-chip buffer capacity required to reuse the data. Because on-chip buffer capacity is frequently more limited than computation units, fused-layer dataflow accelerators may also recompute certain parts of the intermediate data instead of retaining them in a buffer. Achieving efficient trade-offs between on-chip buffer capacity, off-chip transfers, and recomputation requires systematic exploration of the fused-layer dataflow design space. However, prior work only explored a subset of the design space, and more efficient designs are left unexplored. In this work, we propose (1) a more extensive design space that has more choices in terms of tiling, data retention, recomputation and, importantly, allows us to explore them in combination, (2) a taxonomy to systematically specify designs, and (3) a model, LoopTree, to evaluate the latency, energy consumption, buffer capacity requirements, and off-chip transfers of designs in this design space. We validate our model against a representative set of prior architectures, achieving a worst-case 4% error. Finally, we present case studies that show how exploring this larger space results in more efficient designs ( e.g., up to a 10 $\times$ buffer capacity reduction to achieve the same off-chip transfers).

关键词： System-on-chip Filters Tensors Space exploration Taxonomy Shape Energy consumption

来源：评论

学校读者我要写书评

暂无评论

MambaLRP: Explaining Selective State Space Sequence Models 38

MambaLRP: Explaining Selective State Space Sequence Models

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Jafari, Farnoush Rezaei Montavon, Grégoire Müller, Klaus-Robert Eberle, Oliver Machine Learning Group Technische Universität Berlin Berlin10587 Germany BIFOLD - Berlin Institute for the Foundations of Learning and Data Berlin10587 Germany Department of Mathematics and Computer Science Freie Universität Berlin Arnimallee 14 Berlin14195 Germany Department of Artificial Intelligence Korea University Seoul136-713 Korea Republic of Max Planck Institute for Informatics Stuhlsatzenhausweg 4 Saarbrücken66123 Germany Google DeepMind Berlin Germany

Recent sequence modeling approaches using selective state space sequence models, referred to as Mamba models, have seen a surge of interest. These models allow efficient processing of long sequences in linear time and are rapidly being adopted in a wide range of applications such as language modeling, demonstrating promising performance. To foster their reliable use in real-world scenarios, it is crucial to augment their transparency. Our work bridges this critical gap by bringing explainability, particularly Layer-wise Relevance Propagation (LRP), to the Mamba architecture. Guided by the axiom of relevance conservation, we identify specific components in the Mamba architecture, which cause unfaithful explanations. To remedy this issue, we propose MambaLRP, a novel algorithm within the LRP framework, which ensures a more stable and reliable relevance propagation through these components. Our proposed method is theoretically sound and excels in achieving state-of-the-art explanation performance across a diverse range of models and datasets. Moreover, MambaLRP facilitates a deeper inspection of Mamba architectures, uncovering various biases and evaluating their significance. It also enables the analysis of previous speculations regarding the long-range capabilities of Mamba models. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：