The theorem states that: Every Boolean function can be $ε-approximated$ by a Disjunctive Normal Form (DNF) of size $O_ε(2^{n}/\log{n})$. This paper will demonstrate this theorem in detail by showing how this theorem is generated and proving its correctness. We will also dive into some specific Boolean functions and explore how these Boolean functions can be approximated by a DNF whose size is within the universal bound $O_ε(2^{n}/\log{n})$. The Boolean functions we interested in are: Parity Function: the parity function can be $ε-approximated$ by a DNF of width $(1 - 2ε)n$ and size $2^{(1 - 2ε)n}$. Furthermore, we will explore the lower bounds on the DNF's size and width. Majority Function: for every constant $1/2 < ε< 1$, there is a DNF of size $2^{O(\sqrt{n})}$ that can $ε-approximated$ the Majority Function on n bits. Monotone Functions: every monotone function f can be $ε-approximated$ by a DNF g of size $2^{n - Ωε(n)}$ satisfying $g(x) \le f(x)$ for all x.
Size and Depth Separation in Approximating Benign Functions with Neural Networks
Weizmann Institute of Science（魏茨曼科学研究所）；Worcester Polytechnic Institute（伍斯特理工学院）；University of Toronto（多伦多大学）；IAS（高等研究院）
When studying the expressive power of neural networks , a main challenge is to understand how the size and depth of the network affect its ability to approximate real functions. However, not all functions are interesting from a practical viewpoint: functions of interest usually have a polynomially-bounded Lipschitz constant, and can be computed efficiently. We call functions that satisfy these conditions "benign", and explore the benefits of size and depth for approximation of benign functions with ReLU networks. As we show, this problem is more challenging than the corresponding problem for n…
Improved Approximation of Linear Threshold Functions
We prove two main results on how arbitrary linear threshold functions $f(x) = \sign(w\cdot x - θ)$ over the $n$ -dimensional Boolean hypercube can be approximated by simple threshold functions. Our first result shows that every $n$ -variable threshold function $f$ is $\eps$ -close to a threshold function depending only on $\Inf(f)^2 \cdot \poly(1/\eps)$ many variables, where $\Inf(f)$ denotes the total influence or average sensitivity of $f.$ This is an exponential sharpening of Friedgut's well-known theorem \cite{Friedgut:98} , which states that every Boolean function $f$ is $\eps$ -close to …
