The motivation for this paper is to study the complexity of constant-width arithmetic circuits. Our main results are the following.
1. For every k > 1, we provide an explicit polynomial that can be computed by a linear-sized monotone circuit of width 2k but has no subexponential-sized monotone circuit of width k. It follows, from the definition of the polynomial, that the constant-width and the constant-depth hierarchies of monotone arithmetic circuits are infinite, both in the commutative and the noncommutative settings.
2. We prove hardness-randomness tradeoffs for identity testing constant-width commutative circuits analogous to [KI03,DSY08].
In their paper on the "chasm at depth four", Agrawal and Vinay have shown that polynomials in m variables of degree O(m) which admit arithmetic circuits of size 2^o(m) also admit arithmetic circuits of depth four and size 2^o(m). This theorem shows that for problems such as arithmetic circuit lower bounds or black-box derandomization of identity testing, the case of depth four circuits is in a certain sense the general case. In this paper we show that smaller depth four circuits can be obtained if we start from polynomial size arithmetic circuits. For instance, we show that if the permanent of…
We consider the complexity of two questions on polynomials given by arithmetic circuits: testing whether a monomial is present and counting the number of monomials. We show that these problems are complete for subclasses of the counting hierarchy which had few or no known natural complete problems. We also study these questions for circuits computing multilinear polynomials.
We show that the two main reduction notions in arithmetic circuit complexity, p-projections and c-reductions, differ in power. We do so by showing unconditionally that there are polynomials that are VNP-complete under c-reductions but not under p-projections. We also show that the question of which polynomials are VNP-complete under which type of reductions depends on the underlying field.
We say that a circuit $C$ over a field $F$ functionally computes an $n$-variate polynomial $P$ if for every $x \in \{0,1\}^n$ we have that $C(x) = P(x)$. This is in contrast to syntactically computing $P$, when $C \equiv P$ as formal polynomials. In this paper, we study the question of proving lower bounds for homogeneous depth-$3$ and depth-$4$ arithmetic circuits for functional computation. We prove the following results : 1. Exponential lower bounds homogeneous depth-$3$ arithmetic circuits for a polynomial in $VNP$. 2. Exponential lower bounds for homogeneous depth-$4$ arithmetic circuits …
In this paper we give a characterization of both Boolean and arithmetic circuit classes of logarithmic depth in the vein of descriptive complexity theory, i.e., the Boolean classes $\textrm{NC}^1$, $\textrm{SAC}^1$ and $\textrm{AC}^1$ as well as their arithmetic counterparts $\#\textrm{NC}^1$, $\#\textrm{SAC}^1$ and $\#\textrm{AC}^1$. We build on Immerman's characterization of constant-depth polynomial-size circuits by formulas of first-order logic, i.e., $\textrm{AC}^0 = \textrm{FO}$, and augment the logical language with an operator for defining relations in an inductive way. Considering sli…
We prove a lower bound of $Ω(n^2/\log^2 n)$ on the size of any syntactically multilinear arithmetic circuit computing some explicit multilinear polynomial $f(x_1, \ldots, x_n)$. Our approach expands and improves upon a result of Raz, Shpilka and Yehudayoff ([RSY08]), who proved a lower bound of $Ω(n^{4/3}/\log^2 n)$ for the same polynomial. Our improvement follows from an asymptotically optimal lower bound for a generalized version of Galvin's problem in extremal set theory.
A polynomial identity testing algorithm must determine whether a given input polynomial is identically equal to 0. We give a deterministic black-box identity testing algorithm for univariate polynomials of the form $\sum_{j=0}^t c_j X^{α_j} (a + b X)^{β_j}$. From our algorithm we derive an exponential lower bound for representations of polynomials such as $\prod_{i=1}^{2^n} (X^i-1)$ under this form. It has been conjectured that these polynomials are hard to compute by general arithmetic circuits. Our result shows that the "hardness from derandomization" approach to lower bounds is feasible for…
We prove exponential lower bounds on the size of homogeneous depth 4 arithmetic circuits computing an explicit polynomial in $VP$. Our results hold for the {\it Iterated Matrix Multiplication} polynomial - in particular we show that any homogeneous depth 4 circuit computing the $(1,1)$ entry in the product of $n$ generic matrices of dimension $n^{O(1)}$ must have size $n^{Ω(\sqrt{n})}$. Our results strengthen previous works in two significant ways. Our lower bounds hold for a polynomial in $VP$. Prior to our work, Kayal et al [KLSS14] proved an exponential lower bound for homogeneous depth 4 c…
We show that over the field of complex numbers, \emph{every} homogeneous polynomial of degree $d$ can be approximated (in the border complexity sense) by a depth-$3$ arithmetic circuit of top fan-in at most $d+1$. This is quite surprising since there exist homogeneous polynomials $P$ on $n$ variables of degree $2$, such that any depth-$3$ arithmetic circuit computing $P$ must have top fan-in at least $Ω(n)$. As an application, we get a new tradeoff between the top fan-in and formal degree in an approximate analog of the celebrated depth reduction result of Gupta, Kamath, Kayal and Saptharishi …
In this paper, we prove superpolynomial lower bounds for the class of homogeneous depth 4 arithmetic circuits. We give an explicit polynomial in VNP of degree $n$ in $n^2$ variables such that any homogeneous depth 4 arithmetic circuit computing it must have size $n^{Ω(\log \log n)}$. Our results extend the works of Nisan-Wigderson [NW95] (which showed superpolynomial lower bounds for homogeneous depth 3 circuits), Gupta-Kamath-Kayal-Saptharishi and Kayal-Saha-Saptharishi [GKKS13, KSS13] (which showed superpolynomial lower bounds for homogeneous depth 4 circuits with bounded bottom fan-in), Kum…
Motivated by the Hadamard product of matrices we define the Hadamard product of multivariate polynomials and study its arithmetic circuit and branching program complexity. We also give applications and connections to polynomial identity testing. Our main results are the following. 1. We show that noncommutative polynomial identity testing for algebraic branching programs over rationals is complete for the logspace counting class $\ceql$, and over fields of characteristic $p$ the problem is in $\ModpL/\Poly$. 2.We show an exponential lower bound for expressing the Raz-Yehudayoff polynomial as t…
We introduce the polynomial coefficient matrix and identify maximum rank of this matrix under variable substitution as a complexity measure for multivariate polynomials. We use our techniques to prove super-polynomial lower bounds against several classes of non-multilinear arithmetic circuits. In particular, we obtain the following results : As our main result, we prove that any homogeneous depth-3 circuit for computing the product of $d$ matrices of dimension $n \times n$ requires $Ω(n^{d-1}/2^d)$ size. This improves the lower bounds by Nisan and Wigderson(1995) when $d=ω(1)$. There is an exp…
In recent years, a very exciting and promising method for proving lower bounds for arithmetic circuits has been proposed. This method combines the method of {\it depth reduction} developed in the works of Agrawal-Vinay [AV08], Koiran [Koi12] and Tavenas [Tav13], and the use of the shifted partial derivative complexity measure developed in the works of Kayal [Kay12] and Gupta et al [GKKS13a]. These results inspired a flurry of other beautiful results and strong lower bounds for various classes of arithmetic circuits, in particular a recent work of Kayal et al [KSS13] showing superpolynomial low…
Powerful skew arithmetic circuits are introduced. These are skew arithmetic circuits with variables, where input gates can be labelled with powers $x^n$ for binary encoded numbers $n$. It is shown that polynomial identity testing for powerful skew arithmetic circuits belongs to $\mathsf{coRNC}^2$, which generalizes a corresponding result for (standard) skew circuits. Two applications of this result are presented: (i) Equivalence of higher-dimensional straight-line programs can be tested in $\mathsf{coRNC}^2$; this result is even new in the one-dimensional case, where the straight-line programs…
A rational number can be naturally presented by an arithmetic computation (AC): a sequence of elementary arithmetic operations starting from a fixed constant, say 1. The asymptotic complexity issues of such a representation are studied e.g. in the framework of the algebraic complexity theory over arbitrary field. Here we study a related problem of the complexity of performing arithmetic operations and computing elementary predicates, e.g. ``='' or ``>'', on rational numbers given by AC. In the first place, we prove that AC can be efficiently simulated by the exact semidefinite programming (SDP…
In this paper, we study the structure of set-multilinear arithmetic circuits and set-multilinear branching programs with the aim of showing lower bound results. We define some natural restrictions of these models for which we are able to show lower bound results. Some of our results extend existing lower bounds, while others are new and raise open questions. More specifically, our main results are the following: (1) We observe that set-multilinear arithmetic circuits can be transformed into shallow set-multilinear circuits efficiently, similar to depth reduction results of [VSBR83,RY08] for mo…
Algorithmic tools for graphs of small treewidth are used to address questions in complexity theory. For both arithmetic and Boolean circuits, it is shown that any circuit of size $n^{O(1)}$ and treewidth $O(\log^i n)$ can be simulated by a circuit of width $O(\log^{i+1} n)$ and size $n^c$, where $c = O(1)$, if $i=0$, and $c=O(\log \log n)$ otherwise. For our main construction, we prove that multiplicatively disjoint arithmetic circuits of size $n^{O(1)}$ and treewidth $k$ can be simulated by bounded fan-in arithmetic formulas of depth $O(k^2\log n)$. From this we derive the analogous statement…
An efficient randomized polynomial identity test for noncommutative polynomials given by noncommutative arithmetic circuits remains an open problem. The main bottleneck to applying known techniques is that a noncommutative circuit of size $s$ can compute a polynomial of degree exponential in $s$ with a double-exponential number of nonzero monomials. In this paper, we report some progress by dealing with two natural subcases (both allow for polynomials of exponential degree and a double exponential number of monomials): (1) We consider \emph{$+$-regular} noncommutative circuits: these are homog…
In this paper, we consider bounded width circuits and nondeterministic circuits in three somewhat new directions. In the first part of this paper, we mainly consider bounded width circuits. The main purpose of this part is to prove that there is a Boolean function $f$ which cannot be computed by any nondeterministic circuit of size $O(n)$ and width $o(n)$. To the best of our knowledge, this is the first result on the lower bound of (nonuniform) bounded width circuits computing an explicit Boolean function, even for deterministic circuits. Actually, we prove a more generalized lower bound. Our …
We study the class $\textrm{AC}^0$ of functions computed by constant-depth polynomial-size arithmetic circuits of unbounded fan-in addition and multiplication gates. No model-theoretic characterization for arithmetic circuit classes is known so far. Inspired by Immerman's characterization of the Boolean class $\textrm{AC}^0$, we remedy this situation and develop such a characterization of $\textrm{AC}^0$. Our characterization can be interpreted as follows: Functions in $\textrm{AC}^0$ are exactly those functions counting winning strategies in first-order model checking games. A consequence of …
Arithmetic circuits (AC) are circuits over the real numbers with 0/1-valued input variables whose gates compute the sum or the product of their inputs. Positive AC -- that is, AC representing non-negative functions -- subsume many interesting probabilistic models such as probabilistic sentential decision diagram (PSDD) or sum-product network (SPN) on indicator variables. Efficient algorithms for many operations useful in probabilistic reasoning on these models critically depend on imposing structural restrictions to the underlying AC. Generally, adding structural restrictions yields new tracta…
Assuming the Generalised Riemann Hypothesis (GRH), we show that for all k, there exist polynomials with coefficients in $\MA$ having no arithmetic circuits of size O(n^k) over the complex field (allowing any complex constant). We also build a family of polynomials that can be evaluated in AM having no arithmetic circuits of size O(n^k). Then we investigate the link between fixed-polynomial size circuit bounds in the Boolean and arithmetic settings. In characteristic zero, it is proved that $\NP \not\subset \size(n^k)$, or $\MA \subset \size(n^k)$, or NP=MA imply lower bounds on the circuit siz…
This survey presents a necessarily incomplete (and biased) overview of results at the intersection of arithmetic circuit complexity, structured matrices and deep learning. Recently there has been some research activity in replacing unstructured weight matrices in neural networks by structured ones (with the aim of reducing the size of the corresponding deep learning models). Most of this work has been experimental and in this survey, we formalize the research question and show how a recent work that combines arithmetic circuit complexity, structured matrices and deep learning essentially answe…
Recurrent Graph Neural Networks and Arithmetic Circuits
Institut für Theoretische Informatik, Leibniz Universität Hannover, Germany（汉诺威莱布尼茨大学理论计算机研究所）；School of Computing Science, University of Glasgow, UK（格拉斯哥大学计算机科学学院）
In a sequence of seminal results in the 80's, Kaltofen showed that the complexity class VP is closed under taking factors. A natural question in this context is to understand if other natural classes of multivariate polynomials, for instance, arithmetic formulas, algebraic branching programs, bounded depth arithmetic circuits or the class VNP, are closed under taking factors. In this paper, we show that all factors of degree at most $\log^a n$ of polynomials with poly(n) size depth $k$ circuits have poly(n) size circuits of depth at most $O(k + a)$. This partially answers a question of Shpilka…
In recent years, there has been a flurry of activity towards proving lower bounds for homogeneous depth-4 arithmetic circuits, which has brought us very close to statements that are known to imply $\textsf{VP} \neq \textsf{VNP}$. It is open if these techniques can go beyond homogeneity, and in this paper we make some progress in this direction by considering depth-4 circuits of low algebraic rank, which are a natural extension of homogeneous depth-4 circuits. A depth-4 circuit is a representation of an $N$-variate, degree-$n$ polynomial $P$ as \[ P = \sum_{i = 1}^T Q_{i1}\cdot Q_{i2}\cdot \cdo…
We give improved hitting sets for two special cases of Read-once Oblivious Arithmetic Branching Programs (ROABP). First is the case of an ROABP with known order of the variables. The best previously known hitting set for this case had size $(nw)^{O(\log n)}$ where $n$ is the number of variables and $w$ is the width of the ROABP. Even for a constant-width ROABP, nothing better than a quasi-polynomial bound was known. We improve the hitting-set size for the known-order case to $n^{O(\log w)}$. In particular, this gives the first polynomial-size hitting set for constant-width ROABP (known-order).…
Recently, Forbes, Kumar and Saptharishi [CCC, 2016] proved that there exists an explicit $d^{O(1)}$-variate and degree $d$ polynomial $P_{d}\in VNP$ such that if any depth four circuit $C$ of bounded formal degree $d$ which computes a polynomial of bounded individual degree $O(1)$, that is functionally equivalent to $P_d$, then $C$ must have size $2^{Ω(\sqrt{d}\log{d})}$. The motivation for their work comes from Boolean Circuit Complexity. Based on a characterization for $ACC^0$ circuits by Yao [FOCS, 1985] and Beigel and Tarui [CC, 1994], Forbes, Kumar and Saptharishi [CCC, 2016] observed tha…
Institute for Theoretical Computer Science（理论计算机科学研究所）；Leibniz University Hanover（汉诺威莱布尼茨大学）；School of Computer Science（计算机科学学院）；University of Sheffield（谢菲尔德大学）
We prove super-polynomial lower bounds for low-depth arithmetic circuits using the shifted partials measure [Gupta-Kamath-Kayal-Saptharishi, CCC 2013], [Kayal, ECCC 2012] and the affine projections of partials measure [Garg-Kayal-Saha, FOCS 2020], [Kayal-Nair-Saha, STACS 2016]. The recent breakthrough work of Limaye, Srinivasan and Tavenas [FOCS 2021] proved these lower bounds by proving lower bounds for low-depth set-multilinear circuits. An interesting aspect of our proof is that it does not require conversion of a circuit to a set-multilinear circuit, nor does it involve a random restrictio…
We study the problem of polynomial identity testing (PIT) for depth 2 arithmetic circuits over matrix algebra. We show that identity testing of depth 3 (Sigma-Pi-Sigma) arithmetic circuits over a field F is polynomial time equivalent to identity testing of depth 2 (Pi-Sigma) arithmetic circuits over U_2(F), the algebra of upper-triangular 2 x 2 matrices with entries from F. Such a connection is a bit surprising since we also show that, as computational models, Pi-Sigma circuits over U_2(F) are strictly `weaker' than Sigma-Pi-Sigma circuits over F. The equivalence further shows that PIT of dept…
We show that if we can design poly($s$)-time hitting-sets for $Σ\wedge^aΣΠ^{O(\log s)}$ circuits of size $s$, where $a=ω(1)$ is arbitrarily small and the number of variables, or arity $n$, is $O(\log s)$, then we can derandomize blackbox PIT for general circuits in quasipolynomial time. This also establishes that either E$\not\subseteq$\#P/poly or that VP$\ne$VNP. In fact, we show that one only needs a poly($s$)-time hitting-set against individual-degree $a'=ω(1)$ polynomials that are computable by a size-$s$ arity-$(\log s)$ $ΣΠΣ$ circuit (note: $Π$ fanin may be $s$). Alternatively, we claim …
Agrawal and Vinay [AV08] showed how any polynomial size arithmetic circuit can be thought of as a depth four arithmetic circuit of subexponential size. The resulting circuit size in this simulation was more carefully analyzed by Korian [Koiran] and subsequently by Tavenas [Tav13]. We provide a simple proof of this chain of results. We then abstract the main ingredient to apply it to formulas and constant depth circuits, and show more structured depth reductions for them. In an apriori surprising result, Raz [Raz10] showed that for any $n$ and $d$, such that $ ω(1) \leq d \leq O\left(\frac{\log…
A read-once oblivious arithmetic branching program (ROABP) is an arithmetic branching program (ABP) where each variable occurs in at most one layer. We give the first polynomial time whitebox identity test for a polynomial computed by a sum of constantly many ROABPs. We also give a corresponding blackbox algorithm with quasi-polynomial time complexity $n^{O(\log n)}$. In both the cases, our time complexity is double exponential in the number of ROABPs. ROABPs are a generalization of set-multilinear depth-$3$ circuits. The prior results for the sum of constantly many set-multilinear depth-$3$ c…
Computer Science Department, University of California, Los Angeles（加州大学洛杉矶分校计算机科学系）
Tractable Boolean and arithmetic circuits have been studied extensively in AI for over two decades now. These circuits were initially proposed as "compiled objects," meant to facilitate logical and probabilistic reasoning, as they permit various types of inference to be performed in linear-time and a feed-forward fashion like neural networks. In more recent years, the role of tractable circuits has significantly expanded as they became a computational and semantical backbone for some approaches that aim to integrate knowledge, reasoning and learning. In this article, we review the foundations …
In this note, we give a short, simple and almost completely self contained proof of a classical result of Kaltofen [Kal86, Kal87, Kal89] which shows that if an $n$ variate degree $d$ polynomial $f$ can be computed by an arithmetic circuit of size $s$, then each of its factors can be computed by an arithmetic circuit of size at most $\textsf{poly}\left(s, n, d\right)$. However, unlike Kaltofen's argument, our proof does not directly give an efficient algorithm for computing the circuits for the factors of $f$.
A polynomial identity testing algorithm must determine whether an input polynomial (given for instance by an arithmetic circuit) is identically equal to 0. In this paper, we show that a deterministic black-box identity testing algorithm for (high-degree) univariate polynomials would imply a lower bound on the arithmetic complexity of the permanent. The lower bounds that are known to follow from derandomization of (low-degree) multivariate identity testing are weaker. To obtain our lower bound it would be sufficient to derandomize identity testing for polynomials of a very specific norm: sums o…
In the present paper we show a dichotomy theorem for the complexity of polynomial evaluation. We associate to each graph H a polynomial that encodes all graphs of a fixed size homomorphic to H. We show that this family is computable by arithmetic circuits in constant depth if H has a loop or no edge and that it is hard otherwise (i.e., complete for VNP, the arithmetic class related to #P). We also demonstrate the hardness over the rational field of cut eliminator, a polynomial defined by Bürgisser which is known to be neither VP nor VNP-complete in the field of two elements, if VP is not equal…
In this paper we study arithmetic computations in the nonassociative, and noncommutative free polynomial ring $\mathbb{F}\{x_1,x_2,\ldots,x_n\}$. Prior to this work, nonassociative arithmetic computation was considered by Hrubes, Wigderson, and Yehudayoff [HWY10], and they showed lower bounds and proved completeness results. We consider Polynomial Identity Testing (PIT) and polynomial factorization over $\mathbb{F}\{x_1,x_2,\ldots,x_n\}$ and show the following results. (1) Given an arithmetic circuit $C$ of size $s$ computing a polynomial $f\in \mathbb{F} \{x_1,x_2,\ldots,x_n\}$ of degree $d$,…
We extend the well known characterization of $\vpws$ as the class of polynomials computed by polynomial size arithmetic branching programs to other complexity classes. In order to do so we add additional memory to the computation of branching programs to make them more expressive. We show that allowing different types of memory in branching programs increases the computational power even for constant width programs. In particular, this leads to very natural and robust characterizations of $\vp$ and $\vnp$ by branching programs with memory.
Motivated by the fundamental lower bounds questions in proof complexity, we initiate the study of matrix identities as hard instances for strong proof systems. A matrix identity of $d \times d$ matrices over a field $\mathbb{F}$, is a non-commutative polynomial $f(x_1,\ldots,x_n)$ over $\mathbb{F}$ such that $f$ vanishes on every $d \times d$ matrix assignment to its variables. We focus on arithmetic proofs, which are proofs of polynomial identities operating with arithmetic circuits and whose axioms are the polynomial-ring axioms (these proofs serve as an algebraic analogue of the Extended Fr…
Tropical circuits are circuits with Min and Plus, or Max and Plus operations as gates. Their importance stems from their intimate relation to dynamic programming algorithms. The power of tropical circuits lies somewhere between that of monotone boolean circuits and monotone arithmetic circuits. In this paper we present some lower bounds arguments for tropical circuits, and hence, for dynamic programs.
Arithmetic complexity is considered simpler to understand than Boolean complexity, namely computing Boolean functions via logical gates. And indeed, we seem to have significantly more lower bound techniques and results in arithmetic complexity than in Boolean complexity. Despite many successes and rapid progress, however, challenges like proving super-polynomial lower bounds on circuit or formula size for explicit polynomials, or super-linear lower bounds on explicit 3-dimensional tensors, remain elusive. At the same time, we have plenty more "barrier results" for failing to prove basic lower …
In this paper, we study the computational complexity of the commutative determinant polynomial computed by a class of set-multilinear circuits which we call regular set-multilinear circuits. Regular set-multilinear circuits are commutative circuits with a restriction on the order in which they can compute polynomials. A regular circuit can be seen as the commutative analogue of the ordered circuit defined by Hrubes,Wigderson and Yehudayoff [HWY10]. We show that if the commutative determinant polynomial has small representation in the sum of constantly many regular set-multilinear circuits, the…
A Quadratic Lower Bound for Noncommutative Circuits
The Institute of Mathematical Sciences, Chennai, India（金奈数学科学研究所）；Homi Bhabha National Institute, Training School Complex,Anushakti Nagar, Mumbai, India（霍米·巴巴国立学院）
Let $C$ be a depth-3 arithmetic circuit of size at most $s$, computing a polynomial $ f \in \mathbb{F}[x_1,\ldots, x_n] $ (where $\mathbb{F}$ = $\mathbb{Q}$ or $\mathbb{C}$) and the fan-in of the product gates of $C$ is bounded by $d$. We give a deterministic polynomial identity testing algorithm to check whether $f\equiv 0$ or not in time $ 2^d \text{ poly}(n,s) $.
