We give new and efficient black-box reconstruction algorithms for some classes of depth-$3$ arithmetic circuits. As a consequence, we obtain the first efficient algorithm for computing the tensor rank and for finding the optimal tensor decomposition as a sum of rank-one tensors when then input is a constant-rank tensor. More specifically, we provide efficient learning algorithms that run in randomized polynomial time over general fields and in deterministic polynomial time over the reals and the complex numbers for the following classes:
(1) Set-multilinear depth-$3$ circuits of constant top fan-in $ΣΠΣ\{\sqcup_j X_j\}(k)$ circuits). As a consequence of our algorithm, we obtain the first polynomial time algorithm for tensor rank computation and optimal tensor decomposition of constant-rank tensors. This result holds for $d$ dimensional tensors for any $d$, but is interesting even for $d=3$.
(2) Sums of powers of constantly many linear forms ($Σ\wedgeΣ$ circuits). As a consequence we obtain the first polynomial-time algorithm for tensor rank computation and optimal tensor decomposition of constant-rank symmetric tensors.
(3) Multilinear depth-3 circuits of constant top fan-in (multilinear $ΣΠΣ(k)$ circuits). Our algorithm works over all fields of characteristic 0 or large enough characteristic. Prior to our work the only efficient algorithms known were over polynomially-sized finite fields (see. Karnin-Shpilka 09').
Prior to our work, the only polynomial-time or even subexponential-time algorithms known (deterministic or randomized) for subclasses of $ΣΠΣ(k)$ circuits that also work over large/infinite fields were for the setting when the top fan-in $k$ is at most $2$ (see Sinha 16' and Sinha 20').
In this paper, we study the computational complexity of the commutative determinant polynomial computed by a class of set-multilinear circuits which we call regular set-multilinear circuits. Regular set-multilinear circuits are commutative circuits with a restriction on the order in which they can compute polynomials. A regular circuit can be seen as the commutative analogue of the ordered circuit defined by Hrubes,Wigderson and Yehudayoff [HWY10]. We show that if the commutative determinant polynomial has small representation in the sum of constantly many regular set-multilinear circuits, the…
In this paper, we study the structure of set-multilinear arithmetic circuits and set-multilinear branching programs with the aim of showing lower bound results. We define some natural restrictions of these models for which we are able to show lower bound results. Some of our results extend existing lower bounds, while others are new and raise open questions. More specifically, our main results are the following: (1) We observe that set-multilinear arithmetic circuits can be transformed into shallow set-multilinear circuits efficiently, similar to depth reduction results of [VSBR83,RY08] for mo…
We prove a lower bound of $Ω(n^2/\log^2 n)$ on the size of any syntactically multilinear arithmetic circuit computing some explicit multilinear polynomial $f(x_1, \ldots, x_n)$. Our approach expands and improves upon a result of Raz, Shpilka and Yehudayoff ([RSY08]), who proved a lower bound of $Ω(n^{4/3}/\log^2 n)$ for the same polynomial. Our improvement follows from an asymptotically optimal lower bound for a generalized version of Galvin's problem in extremal set theory.
We study the size blow-up that is necessary to convert an algebraic circuit of product-depth $Δ+1$ to one of product-depth $Δ$ in the multilinear setting. We show that for every positive $Δ= Δ(n) = o(\log n/\log \log n),$ there is an explicit multilinear polynomial $P^{(Δ)}$ on $n$ variables that can be computed by a multilinear formula of product-depth $Δ+1$ and size $O(n)$, but not by any multilinear circuit of product-depth $Δ$ and size less than $\exp(n^{Ω(1/Δ)})$. This result is tight up to the constant implicit in the double exponent for all $Δ= o(\log n/\log \log n).$ This strengthens a…
We show that any $n$-variate polynomial computable by a syntactically multilinear circuit of size $\operatorname{poly}(n)$ can be computed by a depth-$4$ syntactically multilinear ($ΣΠΣΠ$) circuit of size at most $\exp\left({O\left(\sqrt{n\log n}\right)}\right)$. For degree $d = ω(n/\log n)$, this improves upon the upper bound of $\exp\left({O(\sqrt{d}\log n)}\right)$ obtained by Tavenas~\cite{T15} for general circuits, and is known to be asymptotically optimal in the exponent when $d < n^ε$ for a small enough constant $ε$. Our upper bound matches the lower bound of $\exp\left({Ω\left(\sqrt{n\…
In this paper, we study the algebraic formula complexity of multiplying $d$ many $2\times 2$ matrices, denoted $\mathrm{IMM}_{d}$, and show that the well-known divide-and-conquer algorithm cannot be significantly improved at any depth, as long as the formulas are multilinear. Formally, for each depth $Δ\leq \log d$, we show that any product-depth $Δ$ multilinear formula for $\mathrm{IMM}_d$ must have size $\exp(Ω(Δd^{1/Δ})).$ It also follows from this that any multilinear circuit of product-depth $Δ$ for the same polynomial of the above form must have a size of $\exp(Ω(d^{1/Δ})).$ In particula…
In this paper we give subexponential size hitting sets for bounded depth multilinear arithmetic formulas. Using the known relation between black-box PIT and lower bounds we obtain lower bounds for these models. For depth-3 multilinear formulas, of size $\exp(n^δ)$, we give a hitting set of size $\exp(\tilde{O}(n^{2/3 + 2δ/3}))$. This implies a lower bound of $\exp(\tildeΩ(n^{1/2}))$ for depth-3 multilinear formulas, for some explicit polynomial. For depth-4 multilinear formulas, of size $\exp(n^δ)$, we give a hitting set of size $\exp(\tilde{O}(n^{2/3 + 4δ/3}))$. This implies a lower bound of …
We prove strengthened lower bounds for constant-depth set-multilinear formulas. More precisely, we show that over any field, there is an explicit polynomial $f$ in VNP defined over $n^2$ variables, and of degree $n$, such that any product-depth $Δ$ set-multilinear formula computing $f$ has size at least $n^{Ω\left( n^{1/Δ}/Δ\right)}$. The hard polynomial $f$ comes from the class of Nisan-Wigderson (NW) design-based polynomials. Our lower bounds improve upon the recent work of Limaye, Srinivasan and Tavenas (STOC 2022), where a lower bound of the form $(\log n)^{Ω(Δn^{1/Δ})}$ was shown for the …
Algebraic Branching Programs(ABPs) are standard models for computing polynomials. Syntactic multilinear ABPs (smABPs) are restrictions of ABPs where every variable is allowed to occur at most once in every path from the start to the terminal node. Proving lower bounds against syntactic multilinear ABPs remains a challenging open question in Algebraic Complexity Theory. The current best known bound is only quadratic [Alon-Kumar-Volk, ECCC 2017]. In this article we develop a new approach upper bounding the rank of the partial derivative matrix of syntactic multlinear ABPs: Convert the ABP to a s…
We prove super-polynomial lower bounds on the size of propositional proof systems operating with constant-depth algebraic circuits over fields of zero characteristic. Specifically, we show that the subset-sum variant $\sum_{i,j,k,\ell\in[n]} z_{ijk\ell}x_ix_j x_k x_\ell - β=0$, for Boolean variables, does not have polynomial-size IPS refutations where the refutations are multilinear and written as constant-depth circuits. Andrews and Forbes (STOC'22) established recently a constant-depth IPS lower bound, but their hard instance does not have itself small constant-depth circuits, while our inst…
We prove super-polynomial lower bounds for low-depth arithmetic circuits using the shifted partials measure [Gupta-Kamath-Kayal-Saptharishi, CCC 2013], [Kayal, ECCC 2012] and the affine projections of partials measure [Garg-Kayal-Saha, FOCS 2020], [Kayal-Nair-Saha, STACS 2016]. The recent breakthrough work of Limaye, Srinivasan and Tavenas [FOCS 2021] proved these lower bounds by proving lower bounds for low-depth set-multilinear circuits. An interesting aspect of our proof is that it does not require conversion of a circuit to a set-multilinear circuit, nor does it involve a random restrictio…
Weakening Assumptions for Deterministic Subexponential Time Non-Singular Matrix Completion
在（Kabanets，Impagliazzo，2004）中，给出了如何在确定性次指数时间内确定电路多项式恒等式测试问题（CPIT），假设算术电路的一些显式多线性多项式族的硬度。本文考虑了CPIT的一种特殊情况，即低次非奇异矩阵完备化（NSMC）。对于这个子类的问题，它示出了如何获得相同的确定性的时间范围，使用较弱的假设，在确定性的复杂性。硬度随机性权衡也将显示在相反的方向，在努力取得进展的Valiant的VP与VNP的问题。为了分离VP和VNP，已知证明m × m持久式的行列式复杂度是$m^{ω(\log m)}$就足够了。本文证明了，对于一个适当的显式性概念，具有行列式复杂度m^{ω（\log m）} $ is equivalent to the existence of an efficiently computable generator $ G_n $ for multilinear NSMC with seed length $ O（n^{1/ \sqrt{\log n} }）$. The latter is a combinatorial object that provides an efficient deterministic black-box algorithm for NSMC. ``Multilinear NSMC'' indicates that $ G…
An arithmetic read-once formula (ROF) is a formula (circuit of fan-out 1) over $+,\times$ where each variable labels at most one leaf. Every multilinear polynomial can be expressed as the sum of ROFs. In this work, we prove, for certain multilinear polynomials, a tight lower bound on the number of summands in such an expression.
An arithmetic read-once formula (ROF) is a formula (circuit of fan-out 1) over $+, \times$ where each variable labels at most one leaf. Every multilinear polynomial can be expressed as the sum of ROFs. In this work, we prove, for certain multilinear polynomials, a tight lower bound on the number of summands in such an expression.
A {+,x}-circuit counts a given multivariate polynomial f, if its values on 0-1 inputs are the same as those of f; on other inputs the circuit may output arbitrary values. Such a circuit counts the number of monomials of f evaluated to 1 by a given 0-1 input vector (with multiplicities given by their coefficients). A circuit decides $f$ if it has the same 0-1 roots as f. We first show that some multilinear polynomials can be exponentially easier to count than to compute them, and can be exponentially easier to decide than to count them. Then we give general lower bounds on the size of counting …
The complexity class PPA consists of NP-search problems which are reducible to the parity principle in undirected graphs. It contains a wide variety of interesting problems from graph theory, combinatorics, algebra and number theory, but only a few of these are known to be complete in the class. Before this work, the known complete problems were all discretizations or combinatorial analogues of topological fixed point theorems. Here we prove the PPA-completeness of two problems of radically different style. They are PPA-Circuit CNSS and PPA-Circuit Chevalley, related respectively to the Combin…
A read-once oblivious arithmetic branching program (ROABP) is an arithmetic branching program (ABP) where each variable occurs in at most one layer. We give the first polynomial time whitebox identity test for a polynomial computed by a sum of constantly many ROABPs. We also give a corresponding blackbox algorithm with quasi-polynomial time complexity $n^{O(\log n)}$. In both the cases, our time complexity is double exponential in the number of ROABPs. ROABPs are a generalization of set-multilinear depth-$3$ circuits. The prior results for the sum of constantly many set-multilinear depth-$3$ c…
This text is a development of a preprint of Ankit Gupta. We present an approach for devising a deterministic polynomial time blackbox identity testing (PIT) algorithm for depth-$4$ circuits with bounded top fanin. This approach is similar to Kayal-Shubhangi approach for depth-$3$ circuits. Kayal and Shubhangi based their algorithm on Sylvester-Gallai-type theorem about linear polynomials. We show how it is possible to generalize this approach to depth-$4$ circuits. However we failed to implement this plan completely. We succeeded to construct a polynomial time deterministic algorithm for depth…
The best known size lower bounds against unrestricted circuits have remained around $3n$ for several decades. Moreover, the only known technique for proving lower bounds in this model, gate elimination, is inherently limited to proving lower bounds of less than $5n$. In this work, we propose a non-gate-elimination approach for obtaining circuit lower bounds, via certain depth-three lower bounds. We prove that every (unbounded-depth) circuit of size $s$ can be expressed as an OR of $2^{s/3.9}$ $16$-CNFs. For DeMorgan formulas, the best known size lower bounds have been stuck at around $n^{3-o(1…
We study the class of non-commutative Unambiguous circuits or Unique-Parse-Tree (UPT) circuits, and a related model of Few-Parse-Trees (FewPT) circuits (which were recently introduced by Lagarde, Malod and Perifel [LMP16] and Lagarde, Limaye and Srinivasan [LLS17]) and give the following constructions: (1) An explicit hitting set of quasipolynomial size for UPT circuits, (2) An explicit hitting set of quasipolynomial size for FewPT circuits (circuits with constantly many parse tree shapes), (3) An explicit hitting set of polynomial size for UPT circuits (of known parse tree shape), when a para…
Allen Institute for AI（艾伦人工智能研究所）；New York University（纽约大学）；University of Washington（华盛顿大学）
Transformers have become a standard neural network architecture for many NLP problems, motivating theoretical analysis of their power in terms of formal languages. Recent work has shown that transformers with hard attention are quite limited in power (Hahn, 2020), as they can be simulated by constant-depth AND/OR circuits (Hao et al. 2021). However, hard attention is a strong assumption, which may complicate the relevance of these results in practice. In this work, we analyze the circuit complexity of transformers with saturated attention: a generalization of hard attention that more closely c…
We develop efficient randomized algorithms to solve the black-box reconstruction problem for polynomials over finite fields, computable by depth three arithmetic circuits with alternating addition/multiplication gates, such that output gate is an addition gate with in-degree two. These circuits compute polynomials of form $G\times(T_1 + T_2)$, where $G,T_1,T_2$ are product of affine forms, and polynomials $T_1,T_2$ have no common factors. Rank of such a circuit is defined as dimension of vector space spanned by all affine factors of $T_1$ and $T_2$. For any polynomial $f$ computable by such a …
What is the power of constant-depth circuits with $MOD_m$ gates, that can count modulo $m$? Can they efficiently compute MAJORITY and other symmetric functions? When $m$ is a constant prime power, the answer is well understood: Razborov and Smolensky proved in the 1980s that MAJORITY and $MOD_m$ require super-polynomial-size $MOD_q$ circuits, where $q$ is any prime power not dividing $m$. However, relatively little is known about the power of $MOD_m$ circuits for non-prime-power $m$. For example, it is still open whether every problem in $EXP$ can be computed by depth-$3$ circuits of polynomia…
Proving super-polynomial size lower bounds for syntactic multilinear Algebraic Branching Programs(smABPs) computing an explicit polynomial is a challenging problem in Algebraic Complexity Theory. The order in which variables in $\{x_1,\ldots,x_n\}$ appear along source to sink paths in any smABP can be viewed as a permutation in $S_n$. In this article, we consider the following special classes of smABPs where the order of occurrence of variables along a source to sink path is restricted: Strict circular-interval ABPs: For every subprogram the index set of variables occurring in it is contained …
In this paper we give an Immerman's Theorem for real-valued computation. We define circuits operating over real numbers and show that families of such circuits of polynomial size and constant depth decide exactly those sets of vectors of reals that can be defined in first-order logic on R-structures in the sense of Cucker and Meer. Our characterization holds both non-uniformily as well as for many natural uniformity conditions.
We study the notion of "cancellation-free" circuits. This is a restriction of linear Boolean circuits (XOR circuits), but can be considered as being equivalent to previously studied models of computation. The notion was coined by Boyar and Peralta in a study of heuristics for a particular circuit minimization problem. They asked how large a gap there can be between the smallest cancellation-free circuit and the smallest linear circuit. We show that the difference can be a factor $Ω(n/\log^{2}n)$. This improves on a recent result by Sergeev and Gashkov who have studied a similar problem. Furthe…
We say that a circuit $C$ over a field $F$ functionally computes an $n$-variate polynomial $P$ if for every $x \in \{0,1\}^n$ we have that $C(x) = P(x)$. This is in contrast to syntactically computing $P$, when $C \equiv P$ as formal polynomials. In this paper, we study the question of proving lower bounds for homogeneous depth-$3$ and depth-$4$ arithmetic circuits for functional computation. We prove the following results : 1. Exponential lower bounds homogeneous depth-$3$ arithmetic circuits for a polynomial in $VNP$. 2. Exponential lower bounds for homogeneous depth-$4$ arithmetic circuits …
Let $C$ be a depth-3 arithmetic circuit of size at most $s$, computing a polynomial $ f \in \mathbb{F}[x_1,\ldots, x_n] $ (where $\mathbb{F}$ = $\mathbb{Q}$ or $\mathbb{C}$) and the fan-in of the product gates of $C$ is bounded by $d$. We give a deterministic polynomial identity testing algorithm to check whether $f\equiv 0$ or not in time $ 2^d \text{ poly}(n,s) $.
We show that over the field of complex numbers, \emph{every} homogeneous polynomial of degree $d$ can be approximated (in the border complexity sense) by a depth-$3$ arithmetic circuit of top fan-in at most $d+1$. This is quite surprising since there exist homogeneous polynomials $P$ on $n$ variables of degree $2$, such that any depth-$3$ arithmetic circuit computing $P$ must have top fan-in at least $Ω(n)$. As an application, we get a new tradeoff between the top fan-in and formal degree in an approximate analog of the celebrated depth reduction result of Gupta, Kamath, Kayal and Saptharishi …
We prove an average-case depth hierarchy theorem for Boolean circuits over the standard basis of $\mathsf{AND}$, $\mathsf{OR}$, and $\mathsf{NOT}$ gates. Our hierarchy theorem says that for every $d \geq 2$, there is an explicit $n$-variable Boolean function $f$, computed by a linear-size depth-$d$ formula, which is such that any depth-$(d-1)$ circuit that agrees with $f$ on $(1/2 + o_n(1))$ fraction of all inputs must have size $\exp({n^{Ω(1/d)}}).$ This answers an open question posed by Håstad in his Ph.D. thesis. Our average-case depth hierarchy theorem implies that the polynomial hierarchy…
In order to formally understand the power of neural computing, we first need to crack the frontier of threshold circuits with two and three layers, a regime that has been surprisingly intractable to analyze. We prove the first super-linear gate lower bounds and the first super-quadratic wire lower bounds for depth-two linear threshold circuits with arbitrary weights, and depth-three majority circuits computing an explicit function. $\bullet$ We prove that for all $ε\gg \sqrt{\log(n)/n}$, the linear-time computable Andreev's function cannot be computed on a $(1/2+ε)$-fraction of $n$-bit inputs …
In this paper, we show exponential lower bounds for the class of homogeneous depth-$5$ circuits over all small finite fields. More formally, we show that there is an explicit family $\{P_d : d \in \mathbb{N}\}$ of polynomials in $\mathsf{VNP}$, where $P_d$ is of degree $d$ in $n = d^{O(1)}$ variables, such that over all finite fields $\mathbb{F}_q$, any homogeneous depth-$5$ circuit which computes $P_d$ must have size at least $\exp(Ω_q(\sqrt{d}))$. To the best of our knowledge, this is the first super-polynomial lower bound for this class for any field $\mathbb{F}_q \neq \mathbb{F}_2$. Our pr…
