检索结果-内蒙古大学图书馆

Synthesizing Precise Static Analyzers for Automatic Differentiation

PROCEEDINGS OF THE ACM ON programming LANGUAGES-PACMPL 2023年第OOPSLA期7卷 1964–1992页

作者： Laurel, Jacob Qian, Siyuan Brant Singh, Gagandeep Misailovic, Sasa Univ Illinois Champaign IL 61820 USA Zhejiang Univ Hangzhou Peoples R China VMware Res Palo Alto CA USA

We present Pasado, a technique for synthesizing precise static analyzers for Automatic Differentiation. Our technique allows one to automatically construct a static analyzer specialized for the Chain Rule, Product Rule, and Quotient Rule computations for Automatic Differentiation in a way that abstracts all of the nonlinear operations of each respective rule simultaneously. By directly synthesizing an abstract transformer for the composite expressions of these 3 most common rules of AD, we are able to obtain significant precision improvement compared to prior works which compose standard abstract transformers together suboptimally. We prove our synthesized static analyzers sound and additionally demonstrate the generality of our approach by instantiating these AD static analyzers with different nonlinear functions, different abstract domains (both intervals and zonotopes) and both forward-mode and reverse-mode AD. We evaluate Pasado on multiple case studies, namely computing certified bounds on a neural network's local Lipschitz constant, soundly bounding the sensitivities of financial models, certifying monotonicity, and lastly, bounding sensitivities of the solutions of differential equations from climate science and chemistry for verified ranges of initial conditions and parameters. The local Lipschitz constants computed by Pasado on our largest CNN are up to 2750x more precise compared to the existing state-of-the-art zonotope analysis. Additionally, the bounds obtained on the sensitivities of the climate, chemical, and financial differential equation solutions are between 1.31 - 2.81x more precise (on average) compared to a state-of-the-art zonotope analysis.

关键词： differentiable programming Abstract Interpretation

来源：评论

学校读者我要写书评

暂无评论

Open-Source differentiable Lithography Imaging Framework 3

Open-Source Differentiable Lithography Imaging Framework

引用

Conference on DTCO and Computational Patterning III

作者： Chen, Guojin Geng, Hao Yu, Bei Pan, David Z. Chinese Univ Hong Kong Hong Kong Peoples R China Univ Texas Austin Austin TX 78712 USA ShanghaiTech Univ Shanghai Peoples R China

ISBN: (纸本)9781510672154;9781510672147

The rapid evolution of the electronics industry, driven by Moore's law and the proliferation of integrated circuits, has led to significant advancements in modern society, including the Internet, wireless communication, and artificial intelligence (AI). Central to this progress is optical lithography, a critical technology in semiconductor manufacturing that accounts for approximately 30% to 40% of production costs. As semiconductor nodes shrink and transistor numbers increase, optical lithography becomes increasingly vital in current integrated circuit (IC) fabrication technology. This paper introduces an open-source differentiable lithography imaging framework that leverages the principles of differentiable programming and the computational power of GPUs to enhance the precision of lithography modeling and simplify the optimization of resolution enhancement techniques (RETs). The framework models the core components of lithography as differentiable segments, allowing for the implementation of standard scalar imaging models, including the Abbe and Hopkins models, as well as their approximation models. The paper introduces a computational lithography framework that optimizes semiconductor manufacturing processes using advanced computational techniques and differentiable programming. It compares imaging models and provides tools for enhancing resolution, demonstrating improved semiconductor patterning performance. The open-sourced framework represents a significant advancement in lithography technology, facilitating collaboration in the field. The source code is available at https://***/TorchOPC/TorchLitho.

关键词： Lithography Computational Lithography differentiable programming Machine Learning

来源：评论

学校读者我要写书评

暂无评论

Learning Optimal Guidance Schemes with Safety Guarantees for Underactuated Marine Vehicles 15

Learning Optimal Guidance Schemes with Safety Guarantees for...

引用

15th IFAC Conference on Control Applications in Marine Systems, Robotics and Vehicles (CAMS)

作者： Foseid, Eirik L. Basso, Erlend A. Schmidt-Didlaukies, Henrik M. Marley, Mathias Pettersen, Kristin Y. Gravdahl, Jan Tommy Norwegian Univ Sci & Technol Trondheim Norway

In this paper, we propose a guidance scheme for underactuated marine vehicles with collision-avoidance guarantees. The guidance scheme is based on a line-of-sight approach and ensures path-following when safety allows it, but deviates from the desired path if safety mandates it. We provide formal safety guarantees by proving forward invariance of the collision-free safe set using a barrier function. However, while barrier-function-based control schemes provide formal safety guarantees, the real-life performance is often lacking due to the challenge of manually tuning such controllers. We demonstrate that the performance of the control law can be improved by replacing a part of the control law by a neural network, while retaining the same formal safety guarantees.

关键词： Line-of-sight guidance barrier functions safety-critical control neural networks differentiable programming

来源：评论

学校读者我要写书评

暂无评论

differentiable Hybrid Traffic Simulation

引用

ACM TRANSACTIONS ON GRAPHICS 2022年第6期41卷 p1-10页

作者： Son, Sanghyun Qiao, Yi-Ling Sewall, Jason Lin, Ming C. Univ Maryland College Pk MD 20742 USA NVIDIA Santa Clara CA USA

We introduce a novel differentiable hybrid traffic simulator, which simulates traffic using a hybrid model of both macroscopic and microscopic models and can be directly integrated into a neural network for traffic control and flow optimization. This is the first differentiable traffic simulator for macroscopic and hybrid models that can compute gradients for traffic states across time steps and inhomogeneous lanes. To compute the gradient flow between two types of traffic models in a hybrid framework, we present a novel intermediate conversion component that bridges the lanes in a differentiable manner as well. We also show that we can use analytical gradients to accelerate the overall process and enhance scalability. Thanks to these gradients, our simulator can provide more efficient and scalable solutions for complex learning and control problems posed in traffic engineering than other existing algorithms. Refer to https://***/***/diff-hybrid-traffic-sim for our project.

关键词： Traffic Simulation differentiable programming Machine Learning

来源：评论

学校读者我要写书评

暂无评论

Deep transform and metric learning network: Wedding deep dictionary learning and neural network

引用

NEUROCOMPUTING 2022年 509卷 244-256页

作者： Tang, Wen Chouzenoux, Emilie Pesquet, Jean-Christophe Krim, Hamid North Carolina State Univ Dept Elect & Comp Engn Raleigh NC 27606 USA Univ Paris Saclay Ctr Supelec F-91190 Gif Sur Yvette Paris France

On account of its many successes in inference tasks and imaging applications, Dictionary Learning (DL) and its related sparse optimization problems have garnered a lot of research interest. In DL area, most solutions are focused on single-layer dictionaries, whose reliance on handcrafted features achieves a somewhat limited performance. With the rapid development of deep learning, improved DL methods called Deep DL (DDL), have been recently proposed an end-to-end flexible inference solution with a much higher performance. The proposed DDL techniques have, however, also fallen short on a number of issues, namely, computational cost and the difficulties in gradient updating and initialization. While a few dif-ferential programming solutions have been proposed to speed-up the single-layer DL, none of them could ensure an efficient, scalable, and robust solution for DDL methods. To that end, we propose herein, a novel differentiable programming approach, which yields an efficient, competitive and reliable DDL solution. The novel DDL method jointly learns deep transforms and deep metrics, where each DL layer is theoret-ically reformulated as a combination of one linear layer and a Recurrent Neural Network (RNN). The RNN is also shown to flexibly account for the layer-associated approximation together with a learnable metric. Additionally, our proposed work unveils new insights into Neural Network (NN) and DDL, bridging the combinations of linear and RNN layers with DDL methods. Extensive experiments on image classification problems are carried out to demonstrate that the proposed method can not only outperform existing DDL several counts including, efficiency, scaling and discrimination, but also achieve better accuracy and increased robustness against adversarial perturbations than CNNs.(c) 2022 Elsevier B.V. All rights reserved.

关键词： Deep dictionary learning Deep neural network Metric learning Transform learning Proximal operator differentiable programming

来源：评论

学校读者我要写书评

暂无评论

Reverse-mode differentiation in arbitrary tensor network format: with application to supervised learning

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2022年第1期23卷 1-29页

作者： Gorodetsky, Alex A. Safta, Cosmin Jakeman, John D. Univ Michigan Dept Aerosp Engn Ann Arbor MI 48109 USA Sandia Natl Labs Quantitat Modeling & Anal Livermore CA 94550 USA Sandia Natl Labs Optimizat & Uncertainty Quantificat Albuquerque NM 87123 USA

This paper describes an efficient reverse-mode differentiation algorithm for contraction operations for arbitrary and unconventional tensor network topologies. The approach leverages the tensor contraction tree of Evenbly and Pfeifer (2014), which provides an instruction set for the contraction sequence of a network. We show that this tree can be efficiently leveraged for differentiation of a full tensor network contraction using a recursive scheme that exploits (1) the bilinear property of contraction and (2) the property that trees have a single path from root to leaves. While differentiation of tensor-tensor contraction is already possible in most automatic differentiation packages, we show that exploiting these two additional properties in the specific context of contraction sequences can improve efficiency. Following a description of the algorithm and computational complexity analysis, we investigate its utility for gradient-based supervised learning for low-rank function recovery and for fitting real-world unstructured datasets. We demonstrate improved performance over alternating least-squares optimization approaches and the capability to handle heterogeneous and arbitrary tensor network formats. When compared to alternating minimization algorithms, we find that the gradient-based approach requires a smaller oversampling ratio efficiency extends to fitting unstructured data of varying dimensionality and when emof benchmark problems.

关键词： Tensor networks supervised learning gradient descent alternating least squares differentiable programming

来源：评论

学校读者我要写书评

暂无评论

Aδ : Autodiff for Discontinuous Programs - Applied to Shaders

引用

ACM TRANSACTIONS ON GRAPHICS 2022年第4期41卷 p1-24页

作者： Yang, Yuting Barnes, Connelly Adams, Andrew Finkelstein, Adam Princeton Univ Princeton NJ 08544 USA Adobe Res San Francisco CA USA

Over the last decade, automatic differentiation (AD) has profoundly impacted graphics and vision applications - both broadly via deep learning and specifically for inverse rendering. Traditional AD methods ignore gradients at discontinuities, instead treating functions as continuous. Rendering algorithms intrinsically rely on discontinuities, crucial at object silhouettes and in general for any branching operation. Researchers have proposed fully-automatic differentiation approaches for handling discontinuities by restricting to affine functions, or semi-automatic processes restricted either to invertible functions or to specialized applications like vector graphics. This paper describes a compiler-based approach to extend reverse mode AD so as to accept arbitrary programs involving discontinuities. Our novel gradient rules generalize differentiation to work correctly, assuming there is a single discontinuity in a local neighborhood, by approximating the prefiltered gradient over a box kernel oriented along a 1D sampling axis. We describe when such approximation rules are first-order correct, and show that this correctness criterion applies to a relatively broad class of functions. Moreover, we show that the method is effective in practice for arbitrary programs, including features for which we cannot prove correctness. We evaluate this approach on procedural shader programs, where the task is to optimize unknown parameters in order to match a target image, and our method outperforms baselines in terms of both convergence and efficiency. Our compiler outputs gradient programs in TensorFlow, PyTorch (for quick prototypes) and Halide with an optional auto-scheduler (for efficiency). The compiler also outputs GLSL that renders the target image, allowing users to interactively modify and animate the shader, which would otherwise be cumbersome in other representations such as triangle meshes or vector art.

关键词： Automatic Differentiation differentiable programming differentiable Rendering Domain-Specific Language

来源：评论

学校读者我要写书评

暂无评论

Searching for Fast Demosaicking Algorithms

引用

ACM TRANSACTIONS ON GRAPHICS 2022年第5期41卷 p1-18页

作者： Ma, Karima Gharbi, Michael Adams, Andrew Kamil, Shoaib Li, Tzu-Mao Barnes, Connelly Ragan-Kelley, Jonathan MIT 32 Vassar St Cambridge MA 02139 USA Adobe Syst Inc 601 Townsend St San Francisco CA 94103 USA Adobe Syst Inc 345 Pk Ave San Jose CA 95110 USA Adobe Syst Inc 104 Fifth Ave4th Fl New York NY 10011 USA Adobe Syst Inc 801 N 34th St Seattle WA 98103 USA Univ Calif San Diego 9500 Gilman DrMail Code 0404 La Jolla CA 92093 USA

We present a method to automatically synthesize efficient, high-quality demosaicking algorithms, across a range of computational budgets, given a loss function and training data. It performs a multi-objective, discrete-continuous optimization which simultaneously solves for the program structure and parameters that best tradeoff computational cost and image quality. We design the method to exploit domain-specific structure for search efficiency. We apply it to several tasks, including demosaicking both Bayer and Fuji X-Trans color filter patterns, as well as joint demosaicking and super-resolution. In a few days on 8 GPUs, it produces a family of algorithms that significantly improves image quality relative to the prior state-of-the-art across a range of computational budgets from 10 s to 1000 s of operations per pixel (1 dB-3 dB higher quality at the same cost, or 8.5-200x higher throughput at same or better quality). The resulting programs combine features of both classical and deep learning-based demosaicking algorithms into more efficient hybrid combinations, which are bandwidth-efficient and vectorizable by construction. Finally, our method automatically schedules and compiles all generated programs into optimized SIMD code for modern processors.

关键词： Demosaicking super-resolution domain specific programming differentiable programming neural architecture search data driven methods

来源：评论

学校读者我要写书评

暂无评论

XAI Beyond Classification: Interpretable Neural Clustering

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2022年第1期23卷 1-28页

作者： Peng, Xi Li, Yunfan Tsang, Ivor W. Zhu, Hongyuan Lv, Jiancheng Zhou, Joey Tianyi Sichuan Univ Coll Comp Sci Chengdu Peoples R China ASTAR Ctr Frontier Artificial Intelligence Res Singapore Singapore Univ Technol Australian Artificial Intelligence Inst Sydney NSW Australia ASTAR Inst Infocomm Res Singapore Singapore ASTAR Inst High Performance Comp Singapore Singapore

In this paper, we study two challenging problems in explainable AI (XAI) and data clustering. The first is how to directly design a neural network with inherent interpretability, rather than giving post-hoc explanations of a black-box model. The second is implementing discrete k-means with a differentiable neural network that embraces the advantages of parallel computing, online clustering, and clustering-favorable representation learning. To address these two challenges, we design a novel neural network, which is a differentiable reformulation of the vanilla k-means, called inTerpretable nEuraL cLustering (TELL). Our contributions are threefold. First, to the best of our knowledge, most existing XAI works focus on supervised learning paradigms. This work is one of the few XAI studies on unsupervised learning, in particular, data clustering. Second, TELL is an interpretable, or the so-called intrinsically explainable and transparent model. In contrast, most existing XAI studies resort to various means for understanding a black-box model with post-hoc explanations. Third, from the view of data clustering, TELL possesses many properties highly desired by k-means, including but not limited to online clustering, plug-and-play module, parallel computing, and provable convergence. Extensive experiments show that our method achieves superior performance comparing with 14 clustering approaches on three challenging data sets.

关键词： transparent neural networks stochastic k-means clustering differentiable programming

来源：评论

学校读者我要写书评

暂无评论

Using Rewrite Strategies for Efficient Functional Automatic Differentiation 25

Using Rewrite Strategies for Efficient Functional Automatic ...

引用

25th ACM International Workshop on Formal Techniques for Java-like Programs (FTfJP)

作者： Boehler, Timon Richter, David Mezini, Mira Tech Univ Darmstadt Darmstadt Germany Hessian AI Darmstadt Germany

ISBN: (纸本)9798400702464

Automatic Differentiation (AD) has become a dominant technique in ML. AD frameworks have first been implemented for imperative languages using tapes. Meanwhile, functional implementations of AD have been developed, often based on dual numbers, which are close to the formal specification of differentiation and hence easier to prove correct. But these papers have focussed on correctness not efficiency. Recently, it was shown how an approach using dual numbers could be made efficient through the right optimizations. Optimizations are highly dependent on order, as one optimization can enable another. It can therefore be useful to have fine-grained control over the scheduling of optimizations. One method expresses compiler optimizations as rewrite rules, whose application can be combined and controlled using strategy languages. Previous work describes the use of term rewriting and strategies to generate high-performance code in a compiler for a functional language. In this work, we implement dual numbers AD in a functional array programming language using rewrite rules and strategy combinators for optimization. We aim to combine the elegance of differentiation using dual numbers with a succinct expression of the optimization schedule using a strategy language. We give preliminary evidence suggesting the viability of the approach on a micro-benchmark.

关键词： differentiable programming domain-specific language optimization term rewriting

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：