Deep Equilibrium (DEQ) models hold great promise for expanding the capabilities of neural networks and showcasing competitive performances with less memory cost. However, the backward propagation steps in their traini...
详细信息
ISBN:
(数字)9798350368741
ISBN:
(纸本)9798350368758
Deep Equilibrium (DEQ) models hold great promise for expanding the capabilities of neural networks and showcasing competitive performances with less memory cost. However, the backward propagation steps in their training process typically involve expensive Jacobian-inverse calculations, resulting in significantly higher computational costs than training conventional neural networks. Despite previous attempts to alleviate the burden, the trade-off between performance and efficiency remains unsatisfactory. In this study, we recast the training problem as a specific bilevel optimization problem. Then we propose BiDEQ, an efficient training algorithm based on penalty method to address the problem without calculating or approximating the hypergradient and matrix inverse in the backward propagation. Numerical experiments on various datasets demonstrate the superiority of our method over state-of-the-art methods for training DEQ models.
Oblivious dimension reduction, à la the Johnson-Lindenstrauss (JL) Lemma, is a fundamental approach for processing high-dimensional data. We study this approach for Uniform Facility Location (UFL) on a Euclidean ...
详细信息
ISBN:
(纸本)9798400715105
Oblivious dimension reduction, à la the Johnson-Lindenstrauss (JL) Lemma, is a fundamental approach for processing high-dimensional data. We study this approach for Uniform Facility Location (UFL) on a Euclidean input X ⊂ℝd, where facilities can lie in the ambient space (not restricted to X). Our main result is that target dimension m=Õ(є−2 ddim) suffices to (1+є)-approximate the optimal value of UFL on inputs whose doubling dimension is bounded by ddim. It significantly improves over previous results, that could only achieve O(1)-approximation [Narayanan, Silwal, Indyk, and Zamir, ICML 2021] or dimension m=O(є−2logn) for n=|X|, which follows from [Makarychev, Makarychev, and Razenshteyn, STOC 2019]. Our oblivious dimension reduction has immediate implications to streaming and offline algorithms, by employing known algorithms for low dimension. In dynamic geometric streams, it implies a (1+є)-approximation algorithm that uses O(є−1logn)Õ(ddim/є2) bits of space, which is the first streaming algorithm for UFL to utilize the doubling dimension. In the offline setting, it implies a (1+є)-approximation algorithm, which we further refine to run in time ((1/є)Õ(ddim) d + 2(1/є)Õ(ddim)) · Õ(n). Prior work has a similar running time but requires some restriction on the facilities [Cohen-Addad, Feldmann and Saulpic, JACM 2021]. Our main technical contribution is a fast procedure to decompose an input X into several k-median instances for small k. This decomposition is inspired by, but has several significant differences from [Czumaj, Lammersen, Monemizadeh and Sohler, SODA 2013], and is key to both our dimension reduction and our PTAS.
Distributed compressed sensing aims at the joint reconstruction of sparse signals with a common support. In some applications, complex-valued signals and sensing matrices are present. In this paper, we investigate rec...
详细信息
ISBN:
(数字)9798331522896
ISBN:
(纸本)9798331522902
Distributed compressed sensing aims at the joint reconstruction of sparse signals with a common support. In some applications, complex-valued signals and sensing matrices are present. In this paper, we investigate recovery algorithms for complex-valued distributed scenarios. To that end, we review a compact exposition of complex- and vector-valued MMSE estimators. These can be used in approximate-message-passing-type algorithms. We explain joint reconstruction via iterative algorithms and evaluate suitable recovery algorithms. The performance of these algorithms is evaluated by numerical simulations for different scenarios.
We provide improved upper and lower bounds for the Min-Sum-Radii (MSR) and Min-Sum-Diameters (MSD) clustering problems with a bounded number of clusters k. In particular, we propose an exact MSD algorithm with running...
详细信息
This paper studies fair division of divisible and indivisible items among agents whose cardinal preferences are not necessarily monotone. We establish the existence of fair divisions and develop approximation algorith...
详细信息
This paper studies fair division of divisible and indivisible items among agents whose cardinal preferences are not necessarily monotone. We establish the existence of fair divisions and develop approximation algorithms to compute them. We address two complementary valuation classes, subadditive and nonnegative, which go beyond monotone functions. Considering both the division of cake (divisible resources) and allocation of indivisible items, we obtain fairness guarantees in terms of (approximate) envy-freeness (EF) and equability (EQ). In the context of envy-freeness, we prove that an EF division of a cake always exists under cake valuations that are subadditive and globally nonnegative (i.e., the value of the entire cake for every agent is nonnegative, but parts of the cake can be burnt). This result notably complements the nonexistence of EF allocations for burnt cakes known for more general valuations. For envy-freeness in the indivisible-items setting, we establish the existence of EF3 allocations for subadditive and globally nonnegative valuations;again, such valuations can be non-monotone and can impart negative value to specific item subsets. In addition, we obtain universal existence of EF3 allocations under nonnegative valuations. We study equitability under nonnegative valuations. Here, we prove that EQ3 allocations always exist when the agents’ valuations are nonnegative (and possibly non-monotone). Also, in the indivisible-items setting, we develop an approximation algorithm that, for given nonnegative valuations, finds allocations that are equitable within additive margins. Our results have combinatorial implications. For instance, the developed results imply: (i) The universal existence of proximately dense subgraphs: Given any graph G = (V, E) and integer k (at most |V |), there always exists a partition V1, V2, . . ., Vk of the vertex set such that the edge densities within the parts, Vi, are additively within four of each other, and (ii) The univer
We consider a number of min-max coverage problems. In each problem, the input is an unweighted graph G and an integer k, and possibly some additional information, such as a root vertex r. In the Min-Max Path Cover pro...
详细信息
We consider a number of min-max coverage problems. In each problem, the input is an unweighted graph G and an integer k, and possibly some additional information, such as a root vertex r. In the Min-Max Path Cover problem, the task is to cover all vertices of the graph by k walks, minimizing the length of the longest walk. The variant of Min-Max Path Cover in which all walks start and end at the same prescribed root vertex r is called the k-Traveling Salesmen Problem. In the Min-Max Tree Cover problem, the task is to cover all vertices of the graph by k trees, minimizing the size (number of edges) of the largest tree. In the rooted version, Min-Max k-Rooted Tree Cover, the input also contains k roots r 1 , . . ., r k , and the i th tree must contain the root r i . These four problems are all known to be APX-hard and to admit a constant-factor approximation. In this paper, we initiate the systematic study of these problems on trees and, more generally, on graphs of constant treewidth. As opposed to most graph problems, all four of the above coverage problems remain NP-hard even when G is a tree. We obtain an n O(k) -time exact algorithm for all four problems on graphs of bounded treewidth. Our main contribution is a quasi-polynomial-time approximation scheme (QPTAS) for the k-Traveling Salesmen Problem, Min-Max Path Cover, and Min-Max Tree Cover on graphs of bounded treewidth.
In the Max-Cut problem in the streaming model, an algorithm is given the edges of an unknown graph G = (V, E) in some fixed order, and its goal is to approximate the size of the largest cut in G. Improving upon an ear...
详细信息
An important and often overlooked aspect of particle filtering methods is the estimation of unknown static parameters. A simple approach for addressing this problem is to augment the unknown static parameters as auxil...
详细信息
ISBN:
(数字)9798350368741
ISBN:
(纸本)9798350368758
An important and often overlooked aspect of particle filtering methods is the estimation of unknown static parameters. A simple approach for addressing this problem is to augment the unknown static parameters as auxiliary states that are jointly estimated with the time-varying parameters of interest. This can be impractical, especially when the system of interest is high-dimensional. Multiple particle filtering (MPF) methods were introduced to try to overcome the curse of dimensionality by using a "divide and conquer" approach, where the vector of unknowns is partitioned into a set of subvectors, each estimated by a separate particle filter. Each particle filter weighs its own particles by using predictions and estimates communicated from the other filters. Currently, there is no principled way to implement MPF methods where the particle filters share unknown parameters or states. In this work, we propose a fusion strategy to allow for the sharing of unknown static parameters in the MPF setting. Specifically, we study the systems which are separable in states and observations. It is proved that optimal Bayesian fusion can be obtained for state-space models with non-interacting states and observations. Simulations are performed to show that MPF with fusion strategy can provide more accurate estimates within fewer time steps comparing to existing algorithms.
We study the complexity of approximating the permanent of a positive semidefinite matrix A∈ ℂn× *** first result is a new approximation algorithm for per(A) with approximation ratio e−(0.9999 + γ)n, exponential...
详细信息
ISBN:
(纸本)9798400715105
We study the complexity of approximating the permanent of a positive semidefinite matrix A∈ ℂn× *** first result is a new approximation algorithm for per(A) with approximation ratio e−(0.9999 + γ)n, exponentially improving upon the current best bound of e−(1+γ−o(1))n (Anari-Gurvits-Oveis Gharan-Saberi 2017, Yuan-Parrilo 2022). Here, γ ≈ 0.577 is Euler’s *** second result is a hardness result. We prove that it is NP-hard to approximate per(A) within a factor e−(γ−)n for any >0. This is the first exponential hardness of approximation for this problem. Along the way, we prove optimal hardness of approximation results for the ||·||2→ q “norm” problem of a matrix for all −1 < q < 2.
Radio spectrum monitoring across 360-degree field-of-view (FoV) is a fundamental requirement for RF situational awareness. RF awareness demands sensed intelligence on the direction of arrival, frequency occupancy, mod...
详细信息
ISBN:
(数字)9798331507367
ISBN:
(纸本)9798331507374
Radio spectrum monitoring across 360-degree field-of-view (FoV) is a fundamental requirement for RF situational awareness. RF awareness demands sensed intelligence on the direction of arrival, frequency occupancy, modulation type signal power, and bandwidth for each waveform present. This article includes an end-to-end design and realization of an array processor, consisting of sixteen uniformly spaced antennas in a circular array at 2.4 GHz, providing 360-degrees FoV using digital multi-beam beamforming. The algorithms encapsulate a low-complexity multi-beam circular array processor with each RF beam being processed by an approximate discrete Fourier transform (ADFT) algorithm. All directional sensing covered by sixteen simultaneous RF beams are 100 MHz in baseband. A fully parallel systolic-array processor for multi-beams is realized using Xilinx Virtex-6 FPGA and CASPER toolchain.
暂无评论