We propose a new and easily-realizable distributed hash table (DHT) peer-to-peer structure, incorporating a random caching strategy that allows for {\em polylogarithmic search time} while having only a {\em constant cache} size. We also show that a very large class of deterministic caching strategies, which covers almost all previously proposed DHT systems, can not achieve polylog search time with constant cache size. In general, the new scheme is the first known DHT structure with the following highly-desired properties: (a) Random caching strategy with constant cache size; (b) Average search time of $O(log^{2}(N))$; (c) Guaranteed search time of $O(log^{3}(N))$; (d) Truly local cache dynamics with constant overhead for node deletions and additions; (e) Self-organization from any initial network state towards the desired structure; and (f) Allows a seamless means for various trade-offs, e.g., search speed or anonymity at the expense of larger cache size.
The following paper presents various methods and implementation techniques used to harvest metadata efficiently from a Kademlia Distributed Hashtable (DHT) as used in the BitTorrent P2P network to build an index of publicly available files in the BitTorrent ecosystem. The indexer design makes various tradeoffs between throughput and fairness towards other DHT nodes while also being scaleable in a distributed environment.
Distributed Hash Tables (DHT) are a common architecture for decentralized applications and, therefore, would be suited for privacy-aware applications. However, currently existing DHTs allow every peer to access any index. To build privacy-aware applications, we need to control this access. In this paper, we present k-rAC, a privacy-aware fine-grained AC for DHTs. For authentication, we present three different mechanisms based on public-key cryptography, zero-knowledge-proofs, and cryptographic hashes. For authorization, we use distributed AC lists. The security of our approach is based on k-re…
Network size is a fundamental statistic for a peer-to-peer system but is generally considered to contain too little information to be useful. However, most existing work only considers the metric by itself and does not explore what features could be extracted from this seem- ingly trivial metric. In this paper, we show that Fourier transform allows us to extract frequency features from such time series data, which can further be used to characterize user behaviors and detect system anoma- lies in a peer-to-peer system automatically without needing to resort to visual comparisons. By using the …
We present and analyze a simple and general scheme to build a churn (fault)-tolerant structured Peer-to-Peer (P2P) network. Our scheme shows how to "convert" a static network into a dynamic distributed hash table(DHT)-based P2P network such that all the good properties of the static network are guaranteed with high probability (w.h.p). Applying our scheme to a cube-connected cycles network, for example, yields a $O(\log N)$ degree connected network, in which every search succeeds in $O(\log N)$ hops w.h.p., using $O(\log N)$ messages, where $N$ is the expected stable network size. Our scheme h…
Distributed Ledger Technologies (DLT) and Decentralized File Storages (DFS) are becoming increasingly used to create common, decentralized and trustless infrastructures where participants interact and collaborate in Peer-to-Peer interactions. A prominent use case is represented by decentralized data marketplaces, where users are consumers and providers at the same time, and trustless interactions are required. However, data in DLTs and DFS are usually unstructured and there are no efficient mechanisms to query a certain type of data for the search in the market. In this paper, we propose the u…
Random hashing is a standard method to balance loads among nodes in Peer-to-Peer networks. However, hashing destroys locality properties of object keys, the critical properties to many applications, more specifically, those that require range searching. To preserve a key order while keeping loads balanced, Ganesan, Bawa and Garcia-Molina proposed a load-balancing algorithm that supports both object insertion and deletion that guarantees a ratio of 4.237 between the maximum and minimum loads among nodes in the network using constant amortized costs. However, their algorithm is not straightforwa…
The Interplanetary Filesystem (IPFS) is a distributed data storage service frequently used by blockchain applications and for sharing content in a censorship-resistant manner. Data is distributed within an open set of peers using a Kademlia-based distributed hash table (DHT). In this paper, we study the structure of the resulting overlay network, as it significantly influences the robustness and performance of IPFS. We monitor and systematically crawl IPFS' DHT towards mapping the IPFS overlay network. Our measurements found an average of 44474 nodes at every given time. At least 52.19% of the…
In this paper, we consider the IoT data discovery problem in very large and growing scale networks. Through analysis, examples, and experimental studies, we show the importance of peer-to-peer, unstructured routing for IoT data discovery and point out the space efficiency issue that has been overlooked in keyword-based routing algorithms in unstructured networks. Specifically, as the first in the field, this paper investigates routing table designs and various compression techniques to support effective and space-efficient IoT data discovery routing. Novel summarization algorithms, including a…
The InterPlanetary File System (IPFS) is a peer-to-peer distributed file system that seeks to connect all computing devices with the same system of files. In some ways, IPFS is similar to the Web, but IPFS could be seen as a single BitTorrent swarm, exchanging objects within one Git repository. In other words, IPFS provides a high throughput content-addressed block storage model, with content-addressed hyper links. This forms a generalized Merkle DAG, a data structure upon which one can build versioned file systems, blockchains, and even a Permanent Web. IPFS combines a distributed hashtable, …
This paper proposes a new microblogging architecture based on peer-to-peer networks overlays. The proposed platform is comprised of three mostly independent overlay networks. The first provides distributed user registration and authentication and is based on the Bitcoin protocol. The second one is a Distributed Hash Table (DHT) overlay network providing key/value storage for user resources and tracker location for the third network. The last network is a collection of possibly disjoint "swarms" of followers, based on the Bittorrent protocol, which can be used for efficient near-instant notific…
A key performance metric in blockchains is the latency between when a transaction is broadcast and when it is confirmed (the so-called, confirmation latency). While improvements in consensus techniques can lead to lower confirmation latency, a fundamental lower bound on confirmation latency is the propagation latency of messages through the underlying peer-to-peer (p2p) network (inBitcoin, the propagation latency is several tens of seconds). The de facto p2p protocol used by Bitcoin and other blockchains is based on random connectivity: each node connects to a random subset of nodes. The induc…
Peer-to-Peer (P2P) networks as distributed solutions are used in a variety of applications. Based on the type of routing for queries among their nodes, they are classified into three groups: structured, unstructured and small-world P2P networks. Each of these categories has its own applications and benefits. Structured networks by using Distributed Hash Tables (DHT) can forward request search queries more efficiently. These networks usually organize a specific topology and make a geometrical shape. A circular topology is a prevalent design which was first introduced by Chord. In this paper, we…
