Traditional orthogonal range problems allow queries over a static set of points, each with some value. Dynamic variants allow points to be added or removed, one at a time. To support more powerful updates, we introduce the Grid Range class of data structure problems over integer arrays in one or more dimensions. These problems allow range updates (such as filling all cells in a range with a constant) and queries (such as finding the sum or maximum of values in a range). In this work, we consider these operations along with updates that replace each cell in a range with the minimum, maximum, or sum of its existing value, and a constant. In one dimension, it is known that segment trees can be leveraged to facilitate any $n$ of these operations in $\tilde{O}(n)$ time overall. Other than a few specific cases, until now, higher dimensional variants have been largely unexplored.
We show that no truly subquadratic time algorithm can support certain pairs of these updates simultaneously without falsifying several popular conjectures. On the positive side, we show that truly subquadratic algorithms can be obtained for variants induced by other subsets. We provide two approaches to designing such algorithms that can be generalised to online and higher dimensional settings. First, we give almost-tight $\tilde{O}(n^{3/2})$ time algorithms for single-update variants where the update operation distributes over the query operation. Second, for other variants, we provide a general framework for reducing to instances with a special geometry. Using this, we show that $O(m^{3/2-ε})$ time algorithms for counting paths and walks of length 2 and 3 between vertex pairs in sparse graphs imply truly subquadratic data structures for certain variants; to this end, we give an $\tilde{O}(m^{(4ω-1)/(2ω+1)}) = O(m^{1.478})$ time algorithm for counting simple 3-paths between vertex pairs.
Multidimensional packing problems generalize the classical packing problems such as Bin Packing, Multiprocessor Scheduling by allowing the jobs to be $d$-dimensional vectors. While the approximability of the scalar problems is well understood, there has been a significant gap between the approximation algorithms and the hardness results for the multidimensional variants. In this paper, we close this gap by giving almost tight hardness results for these problems. 1. We show that Vector Bin Packing has no polynomial time $Ω( \log d)$ factor asymptotic approximation algorithm when $d$ is a large …
We study the uniform query reliability problem, which asks, for a fixed Boolean query Q, given an instance I, how many subinstances of I satisfy Q. Equivalently, this is a restricted case of Boolean query evaluation on tuple-independent probabilistic databases where all facts must have probability 1/2. We focus on graph signatures, and on queries closed under homomorphisms. We show that for any such query that is unbounded, i.e., not equivalent to a union of conjunctive queries, the uniform reliability problem is #P-hard. This recaptures the hardness, e.g., of s-t connectedness, which counts h…
In this paper, aiming at securing range query, top-k query, and skyline query in tiered sensor networks, we propose the Secure Range Query (SRQ), Secure Top-$k$ Query (STQ), and Secure Skyline Query (SSQ) schemes, respectively. In particular, SRQ, by using our proposed \emph{prime aggregation} technique, has the lowest communication overhead among prior works, while STQ and SSQ, to our knowledge, are the first proposals in tiered sensor networks for securing top-$k$ and skyline queries, respectively. Moreover, the relatively unexplored issue of the security impact of sensor node compromises on…
Traditional indexing techniques commonly employed in da\-ta\-ba\-se systems perform poorly on multidimensional array scientific data. Bitmap indices are widely used in commercial databases for processing complex queries, due to their effective use of bit-wise operations and space-efficiency. However, bitmap indices apply natively to relational or linearized datasets, which is especially notable in binned or compressed indices. We propose a new method for multidimensional array indexing that overcomes the dimensionality-induced inefficiencies. The hierarchical indexing method is based on $n$-di…
We show an essentially tight bound on the number of adaptively chosen statistical queries that a computationally efficient algorithm can answer accurately given $n$ samples from an unknown distribution. A statistical query asks for the expectation of a predicate over the underlying distribution, and an answer to a statistical query is accurate if it is "close" to the correct expectation over the distribution. This question was recently studied by Dwork et al., who showed how to answer $\tildeΩ(n^2)$ queries efficiently, and also by Hardt and Ullman, who showed that answering $\tilde{O}(n^3)$ q…
We consider the problem of representing multidimensional data where the domain of each dimension is organized hierarchically, and the queries require summary information at a different node in the hierarchy of each dimension. This is the typical case of OLAP databases. A basic approach is to represent each hierarchy as a one-dimensional line and recast the queries as multidimensional range queries. This approach can be implemented compactly by generalizing to more dimensions the $k^2$-treap, a compact representation of two-dimensional points that allows for efficient summarization queries alon…
Despite much study, the computational complexity of differential privacy remains poorly understood. In this paper we consider the computational complexity of accurately answering a family $Q$ of statistical queries over a data universe $X$ under differential privacy. A statistical query on a dataset $D \in X^n$ asks "what fraction of the elements of $D$ satisfy a given predicate $p$ on $X$?" Dwork et al. (STOC'09) and Boneh and Zhandry (CRYPTO'14) showed that if both $Q$ and $X$ are of polynomial size, then there is an efficient differentially private algorithm that accurately answers all the …
Jumbled indexing is the problem of indexing a text $T$ for queries that ask whether there is a substring of $T$ matching a pattern represented as a Parikh vector, i.e., the vector of frequency counts for each character. Jumbled indexing has garnered a lot of interest in the last four years. There is a naive algorithm that preprocesses all answers in $O(n^2|Σ|)$ time allowing quick queries afterwards, and there is another naive algorithm that requires no preprocessing but has $O(n\log|Σ|)$ query time. Despite a tremendous amount of effort there has been little improvement over these running tim…
In this paper we present several novel efficient techniques and multidimensional data structures which can improve the decision making process in many domains. We consider online range aggregation, range selection and range weighted median queries; for most of them, the presented data structures and techniques can provide answers in polylogarithmic time. The presented results have applications in many business and economic scenarios, some of which are described in detail in the paper.
A distance labeling scheme is an assignment of bit-labels to the vertices of an undirected, unweighted graph such that the distance between any pair of vertices can be decoded solely from their labels. An important class of distance labeling schemes is that of hub labelings, where a node $v \in G$ stores its distance to the so-called hubs $S_v \subseteq V$, chosen so that for any $u,v \in V$ there is $w \in S_u \cap S_v$ belonging to some shortest $uv$ path. Notice that for most existing graph classes, the best distance labelling constructions existing use at some point a hub labeling scheme a…
We consider an ordinal tree $T$ on $n$ nodes, with each node assigned a $d$-dimensional weight vector $\pnt{w} \in \{1,2,\ldots,n\}^d,$ where $d \in \mathbb{N}$ is a constant. We study path queries as generalizations of well-known {\textit{orthogonal range queries}}, with one of the dimensions being tree topology rather than a linear order. Since in our definitions $d$ only represents the number of dimensions of the weight vector without taking the tree topology into account, a path query in a tree with $d$-dimensional weight vectors generalize the corresponding $(d+1)$-dimensional orthogonal …
We introduce a new technique for the efficient management of large sequences of multidimensional data, which takes advantage of regularities that arise in real-world datasets and supports different types of aggregation queries. More importantly, our representation is flexible in the sense that the relevant dimensions and queries may be used to guide the construction process, easily providing a space-time tradeoff depending on the relevant queries in the domain. We provide two alternative representations for sequences of multidimensional data and describe the techniques to efficiently store the…
Updating and querying on a range is a classical algorithmic problem with a multitude of applications. The Segment Tree data structure is particularly notable in handling the range query and update operations. A Segment Tree divides the range into disjoint segments and merges them together to perform range queries and range updates elegantly. Although this data structure is remarkably potent for 1-dimensional problems, it falls short in higher dimensions. Lazy Propagation enables the operations to be computed in $O(logn)$ time in a single dimension. However, the concept of lazy propagation coul…
We consider the following class of submodular k-multiway partitioning problems: (Sub-$k$-MP) $\min \sum_{i=1}^k f(S_i): S_1 \uplus S_2 \uplus \cdots \uplus S_k = V \mbox{ and } S_i \neq \emptyset \mbox{ for all }i\in [k]$. Here $f$ is a non-negative submodular function, and $\uplus$ denotes the union of disjoint sets. Hence the goal is to partition $V$ into $k$ non-empty sets $S_1,S_2,\ldots,S_k$ such that $\sum_{i=1}^k f(S_i)$ is minimized. These problems were introduced by Zhao et al. partly motivated by applications to network reliability analysis, VLSI design, hypergraph cut, and other par…
We consider the task of enumerating and counting answers to $k$-ary conjunctive queries against relational databases that may be updated by inserting or deleting tuples. We exhibit a new notion of q-hierarchical conjunctive queries and show that these can be maintained efficiently in the following sense. During a linear time preprocessing phase, we can build a data structure that enables constant delay enumeration of the query results; and when the database is updated, we can update the data structure and restart the enumeration phase within constant time. For the special case of self-join fre…
Data stored in a data warehouse are inherently multidimensional, but most data-pruning techniques (such as iceberg and top-k queries) are unidimensional. However, analysts need to issue multidimensional queries. For example, an analyst may need to select not just the most profitable stores or--separately--the most profitable products, but simultaneous sets of stores and products fulfilling some profitability constraints. To fill this need, we propose a new operator, the diamond dice. Because of the interaction between dimensions, the computation of diamonds is challenging. We present the first…
There has been significant success in designing highly efficient algorithms for distance and shortest-path queries in recent years; many of the state-of-the-art algorithms use the hub labeling framework. In this paper, we study the approximability of the Hub Labeling problem. We prove a hardness of $Ω(\log n)$ for Hub Labeling, matching known approximation guarantees. The hardness result applies to graphs that have multiple shortest paths between some pairs of vertices. No hardness of approximation results were known previously. Then, we focus on graphs that have a unique shortest path between…
We consider the problem of privately answering queries defined on databases which are collections of points belonging to some metric space. We give simple, computationally efficient algorithms for answering distance queries defined over an arbitrary metric. Distance queries are specified by points in the metric space, and ask for the average distance from the query point to the points contained in the database, according to the specified metric. Our algorithms run efficiently in the database size and the dimension of the space, and operate in both the online query release setting, and the offl…
Given query access to a monotone function $f\colon\{0,1\}^n\to\{0,1\}$ with certificate complexity $C(f)$ and an input $x^{\star}$, we design an algorithm that outputs a size-$C(f)$ subset of $x^{\star}$ certifying the value of $f(x^{\star})$. Our algorithm makes $O(C(f) \cdot \log n)$ queries to $f$, which matches the information-theoretic lower bound for this problem and resolves the concrete open question posed in the STOC '22 paper of Blanc, Koch, Lange, and Tan [BKLT22]. We extend this result to an algorithm that finds a size-$2C(f)$ certificate for a real-valued monotone function with $O…
In the SetDisjointness problem, a collection of $m$ sets $S_1,S_2,...,S_m$ from some universe $U$ is preprocessed in order to answer queries on the emptiness of the intersection of some two query sets from the collection. In the SetIntersection variant, all the elements in the intersection of the query sets are required to be reported. These are two fundamental problems that were considered in several papers from both the upper bound and lower bound perspective. Several conditional lower bounds for these problems were proven for the tradeoff between preprocessing and query time or the tradeoff…
Given a $d$-dimensional array $A$, an update operation adds a given constant $C$ to each element within a continuous sub-array of $A$. A query operation computes the sum of all the elements within a continuous sub-array of $A$. The one-dimensional update and query handling problem has been studied intensively and is usually solved using segment trees with lazy propagation technique. In this paper, we present a new algorithm incorporating Binary Indexed Trees and Inclusion-Exclusion Principle to accomplish the same task. We extend the algorithm to update and query sub-matrices of matrices (two-…
Hardness of High-Dimensional Linear Classification
TU Dortmund, Germany（图恩多特大学，德国）；University of Utah, UT, USA（犹他大学，美国）；TU Dortmund - Center for Data Science & Simulation (DoDaS)（图恩多特大学-数据科学与模拟中心（DoDaS））
Let P be a set of n points in R^2. Given a rectangle Q = [α_1, α_2] x [β_1, β_2], a range skyline query returns the maxima of the points in P \cap Q. An important variant is the so-called top-open queries, where Q is a 3-sided rectangle whose upper edge is grounded at y = \infty (that is, β_2 = \infty). These queries are crucial in numerous database applications. In internal memory, extensive research has been devoted to designing data structures that can answer such queries efficiently. In contrast, currently there is no clear understanding about their exact complexities in external memory. T…
In recent years much effort has been concentrated towards achieving polynomial time lower bounds on algorithms for solving various well-known problems. A useful technique for showing such lower bounds is to prove them conditionally based on well-studied hardness assumptions such as 3SUM, APSP, SETH, etc. This line of research helps to obtain a better understanding of the complexity inside P. A related question asks to prove conditional space lower bounds on data structures that are constructed to solve certain algorithmic tasks after an initial preprocessing stage. This question received littl…
University of California, Los Angeles（加州大学洛杉矶分校）；University of Trento（特伦托大学）
Circuit representations are becoming the lingua franca to express and reason about tractable generative and discriminative models. In this paper, we show how complex inference scenarios for these models that commonly arise in machine learning -- from computing the expectations of decision tree ensembles to information-theoretic divergences of deep mixture models -- can be represented in terms of tractable modular operations over circuits. Specifically, we characterize the tractability of a vocabulary of simple transformations -- sums, products, quotients, powers, logarithms, and exponentials -…
We consider the following natural generalization of Binary Search: in a given undirected, positively weighted graph, one vertex is a target. The algorithm's task is to identify the target by adaptively querying vertices. In response to querying a node $q$, the algorithm learns either that $q$ is the target, or is given an edge out of $q$ that lies on a shortest path from $q$ to the target. We study this problem in a general noisy model in which each query independently receives a correct answer with probability $p > \frac{1}{2}$ (a known constant), and an (adversarial) incorrect one with proba…
The classic string indexing problem is to preprocess a string S into a compact data structure that supports efficient pattern matching queries. Typical queries include existential queries (decide if the pattern occurs in S), reporting queries (return all positions where the pattern occurs), and counting queries (return the number of occurrences of the pattern). In this paper we consider a variant of string indexing, where the goal is to compactly represent the string such that given two patterns P1 and P2 and a gap range [α,β] we can quickly find the consecutive occurrences of P1 and P2 with d…
Query Complexity of Densest Sub(hyper)graph Problems
Pacific Northwest National Laboratory（太平洋西北国家实验室）；School of Computing, University of Utah（犹他大学计算机学院）；Department of Computer Science and Engineering, Texas A&M University（德克萨斯农工大学计算机科学工程系）
IBM Research Almaden（IBM阿尔马登研究院）；Harvard University School of Engineering and Applied Sciences（哈佛大学工程与应用科学学院）；Center for Research on Computation and Society（计算与社会研究中心）
We show that, under a standard hardness assumption, there is no computationally efficient algorithm that given $n$ samples from an unknown distribution can give valid answers to $n^{3+o(1)}$ adaptively chosen statistical queries. A statistical query asks for the expectation of a predicate over the underlying distribution, and an answer to a statistical query is valid if it is "close" to the correct expectation over the distribution. Our result stands in stark contrast to the well known fact that exponentially many statistical queries can be answered validly and efficiently if the queries are c…
We present a new technique for efficiently removing almost all short cycles in a graph without unintentionally removing its triangles. Consequently, triangle finding problems do not become easy even in almost $k$-cycle free graphs, for any constant $k\geq 4$. Triangle finding is at the base of many conditional lower bounds in P, mainly for distance computation problems, and the existence of many $4$- or $5$-cycles in a worst-case instance had been the obstacle towards resolving major open questions. Hardness of approximation: Are there distance oracles with $m^{1+o(1)}$ preprocessing time and …
The suffix array $SA[1..n]$ of a text $T$ of length $n$ is a permutation of $\{1,\ldots,n\}$ describing the lexicographical ordering of suffixes of $T$, and it is considered to be among of the most important data structures in string algorithms, with dozens of applications in data compression, bioinformatics, and information retrieval. One of the biggest drawbacks of the suffix array is that it is very difficult to maintain under text updates: even a single character substitution can completely change the contents of the suffix array. Thus, the suffix array of a dynamic text is modelled using …
An important question in the study of constraint satisfaction problems (CSP) is understanding how the graph or hypergraph describing the incidence structure of the constraints influences the complexity of the problem. For binary CSP instances (i.e., where each constraint involves only two variables), the situation is well understood: the complexity of the problem essentially depends on the treewidth of the graph of the constraints. However, this is not the correct answer if constraints with unbounded number of variables are allowed, and in particular, for CSP instances arising from query evalu…
Local moments are used for local regression, to compute statistical measures such as sums, averages, and standard deviations, and to approximate probability distributions. We consider the case where the data source is a very large I/O array of size n and we want to compute the first N local moments, for some constant N. Without precomputation, this requires O(n) time. We develop a sequence of algorithms of increasing sophistication that use precomputation and additional buffer space to speed up queries. The simpler algorithms partition the I/O array into consecutive ranges called bins, and the…
This paper presents a polynomial-time $1/2$-approximation algorithm for maximizing nonnegative $k$-submodular functions. This improves upon the previous $\max\{1/3, 1/(1+a)\}$-approximation by Ward and Živný~(SODA'14), where $a=\max\{1, \sqrt{(k-1)/4}\}$. We also show that for monotone $k$-submodular functions there is a polynomial-time $k/(2k-1)$-approximation algorithm while for any $\varepsilon>0$ a $((k+1)/2k+\varepsilon)$-approximation algorithm for maximizing monotone $k$-submodular functions would require exponentially many queries. In particular, our hardness result implies that our al…
In this paper, we consider lower bounds on the query complexity for testing CSPs in the bounded-degree model. First, for any ``symmetric'' predicate $P:{0,1}^{k} \to {0,1}$ except \equ where $k\geq 3$, we show that every (randomized) algorithm that distinguishes satisfiable instances of CSP(P) from instances $(|P^{-1}(0)|/2^k-ε)$-far from satisfiability requires $Ω(n^{1/2+δ})$ queries where $n$ is the number of variables and $δ>0$ is a constant that depends on $P$ and $ε$. This breaks a natural lower bound $Ω(n^{1/2})$, which is obtained by the birthday paradox. We also show that every one-sid…
In the densest subgraph problem, given a weighted undirected graph $G(V,E,w)$, with non-negative edge weights, we are asked to find a subset of nodes $S\subseteq V$ that maximizes the degree density $w(S)/|S|$, where $w(S)$ is the sum of the edge weights induced by $S$. This problem is a well studied problem, known as the {\em densest subgraph problem}, and is solvable in polynomial time. But what happens when the edge weights are negative? Is the problem still solvable in polynomial time? Also, why should we care about the densest subgraph problem in the presence of negative weights? In this …
