检索结果-内蒙古大学图书馆

Adversarial examples in multi-layer random ReLU networks 21

学校读者我要写书评

暂无评论

Adversarial examples in multi-layer random ReLU networks

Proceedings of the 35th International Conference on Neural Information Processing Systems

作者： Peter L. Bartlett Sébastien Bubeck Yeshwanth Cherapanamjeri Department of Electrical Engineering and Computer Science Department of Statistics UC Berkeley Microsoft Research Redmond Department of Electrical Engineering and Computer Science UC Berkeley

ISBN: (纸本)9781713845393

We consider the phenomenon of adversarial examples in ReLU networks with independent Gaussian parameters. For networks of constant depth and with a large range of widths (for instance, it suffices if the width of each layer is polynomial in that of any other layer), small perturbations of input vectors lead to large changes of outputs. This generalizes results of Daniely and Schacham (2020) for networks of rapidly decreasing width and of Bubeck et al (2021) for two-layer networks. Our proof shows that adversarial examples arise in these networks because the functions they compute are locally very similar to random linear functions. Bottleneck layers play a key role: the minimal width up to some point in the network determines scales and sensitivities of mappings computed up to that point. The main result is for networks with constant depth, but we also show that some constraint on depth is necessary for a result of this kind, because there are suitably deep networks that, with constant probability, compute a function that is close to constant.

关键词：

Silicon Microring Modulator for High SFDR Analog Links in Monolithic 45nm CMOS

学校读者我要写书评

暂无评论

Silicon Microring Modulator for High SFDR Analog Links in Mo...

Conference on Lasers and Electro-Optics (CLEO)

作者： S. Buchbinder R. Wang D. Kramnik D. Van Orden A. Khilo J. Fini C. Sun M. Wade V. Stojanović Department of Electrical Engineering and Computer Science UC Berkeley Berkeley CA USA Ayar Labs Emeryville CA USA

We characterize the linearity of a lateral junction microring modulator in a monolithic 45 nm CMOS platform versus modulator bias, optical wavelength, and input power, and achieve peak third-order SFDR of 96.4dB ·... 详细信息

ISBN: (数字)9781957171050

ISBN: (纸本)9781665466660

关键词： Semiconductor lasers Power lasers Linearity Lasers and electrooptics Electro-optic modulators Silicon Optical fiber communication

Adversarial examples in multi-layer random ReLU networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Bartlett, Peter L. Bubeck, Sébastien Cherapanamjeri, Yeshwanth Department of Electrical Engineering and Computer Science Department of Statistics UC Berkeley Microsoft Research Redmond Department of Electrical Engineering and Computer Science UC Berkeley

We consider the phenomenon of adversarial examples in ReLU networks with independent gaussian parameters. For networks of constant depth and with a large range of widths (for instance, it suffices if the width of each layer is polynomial in that of any other layer), small perturbations of input vectors lead to large changes of outputs. This generalizes results of Daniely and Schacham (2020) for networks of rapidly decreasing width and of Bubeck et al (2021) for two-layer networks. The proof shows that adversarial examples arise in these networks because the functions that they compute are very close to linear. Bottleneck layers in the network play a key role: the minimal width up to some point in the network determines scales and sensitivities of mappings computed up to that point. The main result is for networks with constant depth, but we also show that some constraint on depth is necessary for a result of this kind, because there are suitably deep networks that, with constant probability, compute a function that is close to constant. Copyright © 2021, The Authors. All rights reserved.

关键词： Network layers

Gradient-based Neuromorphic Learning on Dynamical RRAM Arrays

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhou, Peng Eshraghian, Jason K. Choi, Dong-Uk Lu, Wei D. Kang, Sung-Mo The Department of Electrical and Computer Engineering UC Santa CruzCA United States The Department of Electrical Engineering and Computer Science University of Michigan Ann ArborMI United States

We present MEMprop, the adoption of gradient-based learning to train fully memristive spiking neural networks (MSNNs). Our approach harnesses intrinsic device dynamics to trigger naturally arising voltage spikes. These spikes emitted by memristive dynamics are analog in nature, and thus fully differentiable, which eliminates the need for surrogate gradient methods that are prevalent in the spiking neural network (SNN) literature. Memristive neural networks typically either integrate memristors as synapses that map offline-trained networks, or otherwise rely on associative learning mechanisms to train networks of memristive neurons. We instead apply the backpropagation through time (BPTT) training algorithm directly on analog SPICE models of memristive neurons and synapses. Our implementation is fully memristive, in that synaptic weights and spiking neurons are both integrated on resistive RAM (RRAM) arrays without the need for additional circuits to implement spiking dynamics, e.g., analog-to-digital converters (ADCs) or thresholded comparators. As a result, higher-order electrophysical effects are fully exploited to use the state-driven dynamics of memristive neurons at run time. By moving towards non-approximate gradient-based learning, we obtain highly competitive accuracy amongst previously reported lightweight dense fully MSNNs on several benchmarks. © 2022, CC BY.

关键词： Memristors

AXI-REALM: Safe, Modular and Lightweight Traffic Monitoring and Regulation for Heterogeneous Mixed-Criticality Systems

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Benz, Thomas Ottaviano, Alessandro Liang, Chaoqun Balas, Robert Garofalo, Angelo Restuccia, Francesco Biondi, Alessandro Rossi, Davide Benini, Luca ETH Zurich Switzerland Department of Computer Science and Engineering UC San Diego San DiegoCA United States Department of Excellence in Robotics & AI Scuola Superiore Sant’Anna Pisa Italy University of Bologna Bologna Italy

The automotive industry is transitioning from federated, homogeneous, interconnected devices to integrated, heterogeneous, mixed-criticality systems (MCS). This leads to challenges in achieving timing predictability techniques due to access contention on shared resources, which can be mitigated using hardware-based spatial and temporal isolation techniques. Focusing on the interconnect as the point of access for shared resources, we propose AXI-REALM, a lightweight, modular, technology-independent, and open-source real-time extension to AXI4 interconnects. AXI-REALM uses a budget-based mechanism enforced on periodic time windows and transfer fragmentation to provide fair arbitration, coupled with execution predictability on real-time workloads. AXI-REALM features a comprehensive bandwidth and latency monitor at both the ingress and egress of the interconnect system. Latency information is also used to detect and reset malfunctioning subordinates, preventing missed deadlines. We provide a detailed cost assessment in a 12 nm node and an end-to-end case study implementing AXI-REALM into an open-source MCS, incurring an area overhead of less than 2 %. When running a mixed-criticality workload, with a time-critical application sharing the interconnect with non-critical applications, we demonstrate that the critical application can achieve up to 68.2 % of the isolated performance by enforcing fairness on the interconnect traffic through burst fragmentation, thus reducing the subordinate access latency by up to 24 times. Near-ideal performance, (above 95 % of the isolated performance) can be achieved by distributing the available bandwidth in favor of the critical application. Copyright © 2025, The Authors. All rights reserved.

关键词： Automotive industry

Electronic-photonic quantum systems on-chip

学校读者我要写书评

暂无评论

Electronic-photonic quantum systems on-chip

Quantum 2.0, QUANTUM 2022

作者： Wang, Imbert Ramesh, Anirudh Kramnik, Danielius Fargas Cabanillas, Josep M. Stojanović, Vladimir Kumar, Prem Popović, Miloš A. Department of Electrical and Computer Engineering Photonics Center Boston University BostonMA02215 United States Graduate Program in Applied Physics Northwestern University EvanstonIL60208 United States Department of Electrical Engineering and Computer Science UC Berkeley BerkeleyCA94720 United States Center for Photonic Communication and Computing Dept. of Electrical and Computer Eng. Northwestern University EvanstonIL60208 United States

We present progress towards realizing electronic-photonic quantum systems on-chip;particularly, entangled photon-pair sources, placing them in the context of previous work, and outlining our vision for mass-producible... 详细信息

ISBN: (纸本)9781957171111

关键词： Quantum entanglement

PPI++: Efficient Prediction-Powered Inference

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Angelopoulos, Anastasios N. Duchi, John C. Zrnic, Tijana Department of Electrical Engineering and Computer Sciences UC Berkeley United States Departments of Statistics and Electrical Engineering Stanford University United States Department of Statistics and Stanford Data Science Stanford University United States

We present PPI++: a computationally lightweight methodology for estimation and inference based on a small labeled dataset and a typically much larger dataset of machine-learning predictions. The methods automatically adapt to the quality of available predictions, yielding easy-to-compute confidence sets—for parameters of any dimensionality—that always improve on classical intervals using only the labeled data. PPI++ builds on prediction-powered inference (PPI), which targets the same problem setting, improving its computational and statistical efficiency. Real and synthetic experiments demonstrate the benefits of the proposed adaptations. Copyright © 2023, The Authors. All rights reserved.

关键词： Forecasting

Greedy Pruning with Group Lasso Provably Generalizes for Matrix Sensing

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Rajaraman, Nived Mokhtari, Aryan Devvrit Ramchandran, Kannan University of California Berkeley United States UT Austin United States Dept. of Electrical Engineering and Computer Sciences UC Berkeley United States Dept. of Computer Science UT Austin United States Dept. of Electrical and Computer Engineering UT Austin United States

Pruning schemes have been widely used in practice to reduce the complexity of trained models with a massive number of parameters. In fact, several practical studies have shown that if a pruned model is fine-tuned with some gradient-based updates it generalizes well to new samples. Although the above pipeline, which we refer to as pruning + fine-tuning, has been extremely successful in lowering the complexity of trained models, there is very little known about the theory behind this success. In this paper, we address this issue by investigating the pruning + fine-tuning framework on the overparameterized matrix sensing problem with the ground truth U* ∈ Rd×r and the overparameterized model U ∈ Rd×k with k r. We study the approximate local minima of the mean square error, augmented with a smooth version of a group Lasso regularizer, Pki=1 Uei2. In particular, we provably show that pruning all the columns below a certain explicit 2-norm threshold results in a solution Uprune which has the minimum number of columns r, yet close to the ground truth in training loss. Moreover, in the subsequent fine-tuning phase, gradient descent initialized at Uprune converges at a linear rate to its limit. While our analysis provides insights into the role of regularization in pruning, we also show that running gradient descent in the absence of regularization results in models which are not suitable for greedy pruning, i.e., many columns could have their 2 norm comparable to that of the maximum. To the best of our knowledge, our results provide the first rigorous insights on why greedy pruning + fine-tuning leads to smaller models which also generalize well. Copyright © 2023, The Authors. All rights reserved.

关键词： Mean square error

Timed I/O Automata: It Is Never Too Late to Complete Your Timed Specification Theory

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Goorden, Martijn A. Larsen, Kim G. Legay, Axel Lorber, Florian Nyman, Ulrik Wąsowski, Andrzej Department of Computer Science Aalborg University Selma Lagerlöfs Vej 300 Öst Aalborg9220 Denmark Department of Computer Science Engineering UC Louvain Place Sainte Barbe 2 Louvain-la-Neuve1348 Belgium Department of Computer Science IT University Rued Langgaards Vej 7 Copenhagen S2300 Denmark

A specification theory combines notions of specifications and implementations with a satisfaction relation, a refinement relation and a set of operators supporting stepwise design. We develop a complete specification framework for real-time systems using Timed I/O Automata as the specification formalism, with the semantics expressed in terms of Timed I/O Transition Systems. We provide constructs for refinement, consistency checking, logical and structural composition, and quotient of specifications—all indispensable ingredients of a compositional design methodology. The theory is backed by rigorous proofs and is being implemented in the open-source tool ECDAR. © 2023, CC BY.

关键词： Specifications