It is now cost-effective to outsource large dataset and perform query over the cloud. However, in this scenario, there exist serious security and privacy issues that sensitive information contained in the dataset can be leaked. The most effective way to address that is to encrypt the data before outsourcing. Nevertheless, it remains a grand challenge to process queries in ciphertext efficiently. In this work, we shall focus on solving one representative query task, namely dynamic skyline query, in a secure manner over the cloud. However, it is difficult to be performed on encrypted data as its dynamic domination criteria require both subtraction and comparison, which cannot be directly supported by a single encryption scheme efficiently. To this end, we present a novel framework called SCALE. It works by transforming traditional dynamic skyline domination into pure comparisons. The whole process can be completed in single-round interaction between user and the cloud. We theoretically prove that the outsourced database, query requests, and returned results are all kept secret under our model. Moreover, we also present an efficient strategy for dynamic insertion and deletion of stored records. Empirical study over a series of datasets demonstrates that our framework improves the efficiency of query processing by nearly three orders of magnitude compared to the state-of-the-art.
To make intelligent decisions over complex data by discovering a set of interesting options is something that has become very important for users of modern applications. Consequently, researchers are studying new techniques to overcome limitations of traditional ways of querying data from databases as top-k queries and skyline queries. Over the past few years new methods have been developed as Flexible Skylines, Regret Minimization and Skyline ordering/ranking. The aim of this survey is to describe these techniques and some their possible variants comparing them and explaining how they improve…
The most common archetypes to identify relevant information in large datasets and find the bestoptions according to some preferences or user criteria, are the top-k queries (ranking method based ona score function defined over the records attributes) and skyline queries (based on Pareto dominance oftuples). Despite their large diffusion, both approaches have their pros and cons. In this survey paper, a comparison is made between these methods and the Flexible Skylines, which is a framework that combines the ranking and skyline approaches using the novel concept ofF-dominanceto a set of monoton…
Outsourcing data and computation to cloud server provides a cost-effective way to support large scale data storage and query processing. However, due to security and privacy concerns, sensitive data (e.g., medical records) need to be protected from the cloud server and other unauthorized users. One approach is to outsource encrypted data to the cloud server and have the cloud server perform query processing on the encrypted data only. It remains a challenging task to support various queries over encrypted data in a secure and efficient way such that the cloud server does not gain any knowledge…
This paper presents a new application for multi-dimensional Skyline query. The idea presented in this paper can be used to find best shopping malls based on users requirements. A web-based application was used to simulate the problem and proposed solution. Also, a mathematical definition was developed to define the problem and show how multi-dimensional Skyline query can be used to solve complex problems, such as, finding shopping malls using multiple different criteria. The idea of this paper can be used in other fields, where different criteria should be considered.
The problem of selecting the most representative tuples from a dataset has led to the development of powerful tools, among which Skyline and Ranking (or Top-k) queries stand out for their ability to support the optimization of multiple criteria in the query process. This paper surveys the remarkable efforts made towards the extension of the aforementioned tools to overcome their limitations, respectively the explosion of the output result and the difficulty of query formulation. Moreover, we explore the application of these state-of-the-art techniques as preference-based query frameworks, prop…
Given a graph, and a set of query vertices (subset of the vertices), the dynamic skyline query problem returns a subset of data vertices (other than query vertices) which are not dominated by other data vertices based on certain distance measure. In this paper, we study the dynamic skyline query problem on uncertain graphs (DySky). The input to this problem is an uncertain graph, a subset of its nodes as query vertices, and the goal here is to return all the data vertices which are not dominated by others. We employ two distance measures in uncertain graphs, namely, \emph{Majority Distance}, a…
Nowadays, as the quantity of data to process began to rise, so did the need for a method to discern what pieces of information could be useful for the user; in response, researchers focused their efforts on improving the already existing ranking methods or creating new ones starting from them. This survey will be presented a small list of some of the most known and/or most recent solutions proposed, with some possible applications for them, concerning a state of the art restricted to around the last ten years, comparing their performance with the traditional one top-k and skyline queries.
Living in the Information Age allows almost everyone have access to a large amount of information and options to choose from in order to fulfill their needs. In many cases, the amount of information available and the rate of change may hide the optimal and truly desired solution. This reveals the need of a mechanism that will highlight the best options to choose among every possible scenario. Based on this the skyline query was proposed which is a decision support mechanism, that retrieves the valuefor- money options of a dataset by identifying the objects that present the optimal combination …
Skyline computation is an increasingly popular query, with broad applicability in domains such as healthcare, travel and finance. Given the recent trend to outsource databases and query evaluation, and due to the proprietary and sometimes highly sensitivity nature of the data (e.g., in healthcare), it is essential to evaluate skylines on encrypted datasets. Several research efforts acknowledged the importance of secure skyline computation, but existing solutions suffer from at least one of the following shortcomings: (i) they only provide ad-hoc security; (ii) they are prohibitively expensive;…
$k$ nearest neighbor ($k$NN) queries and skyline queries are important operators on multi-dimensional data points. Given a query point, $k$NN query returns the $k$ nearest neighbors based on a scoring function such as a weighted sum of the attributes, which requires predefined attribute weights (or preferences). Skyline query returns all possible nearest neighbors for any monotonic scoring functions without requiring attribute weights but the number of returned points can be prohibitively large. We observe that both $k$NN and skyline are inflexible and cannot be easily customized. In this pape…
Distributed Continuous Range-Skyline Query Monitoring over the Internet of Mobile Things
National Chiao Tung University（国立交通大学）；Electronic Engineering Polytechnic Institute of Surabaya (PENS)（泗水电子工程学院）；National Taipei University of Technology（台北科技大学）
The task of similarity search in multimedia databases is usually accomplished by range or k nearest neighbor queries. However, the expressing power of these "single-example" queries fails when the user's delicate query intent is not available as a single example. Recently, the well-known skyline operator was reused in metric similarity search as a "multi-example" query type. When applied on a multi-dimensional database (i.e., on a multi-attribute table), the traditional skyline operator selects all database objects that are not dominated by other objects. The metric skyline query adopts the sk…
The datasets available nowadays are very rich and complex, but how do we reach the information we are looking for? In this survey, two different approaches to query a dataset are analyzed and algorithms for each type are explained. Specifically, the TA and NRA have been analyzed for the Top-K query and the Basic Block Nested Loops has been examined for the skyline query. Moreover, it's explained the core idea behind the Prioritized and Flexible skyline. In the end, the pros and cons of each type of analyzed query have been evaluated based on different criteria.
The well-known benefits of cloud computing have spurred the popularity of database service outsourcing, where one can resort to the cloud to conveniently store and query databases. Coming with such popular trend is the threat to data privacy, as the cloud gains access to the databases and queries which may contain sensitive information, like medical or financial data. A large body of work has been presented for querying encrypted databases, which has been mostly focused on secure keyword search. In this paper, we instead focus on the support for secure skyline query processing over encrypted o…
Nowadays, efficient and effective processing over massive stream data has attracted much attention from the database community, which are useful in many real applications such as sensor data monitoring, network intrusion detection, and so on. In practice, due to the malfunction of sensing devices or imperfect data collection techniques, real-world stream data may often contain missing or incomplete data attributes. In this paper, we will formalize and tackle a novel and important problem, named skyline query over incomplete data stream (Sky-iDS), which retrieves skyline objects (in the presenc…
Multi-criteria decision making has been made possible with the advent of skyline queries. However, processing such queries for high dimensional datasets remains a time consuming task. Real-time applications are thus infeasible, especially for non-indexed skyline techniques where the datasets arrive online. In this paper, we propose a caching mechanism that uses the semantics of previous skyline queries to improve the processing time of a new query. In addition to exact queries, utilizing such special semantics allow accelerating related queries. We achieve this by generating partial result set…
Many web databases are "hidden" behind proprietary search interfaces that enforce the top-$k$ output constraint, i.e., each query returns at most $k$ of all matching tuples, preferentially selected and returned according to a proprietary ranking function. In this paper, we initiate research into the novel problem of skyline discovery over top-$k$ hidden web databases. Since skyline tuples provide critical insights into the database and include the top-ranked tuple for every possible ranking function following the monotonic order of attribute values, skyline discovery from a hidden web database…
Given a set of multidimensional points, the skyline operator returns a set of potentially interesting points from such a dataset. This popular operator filters out a set of tuples that are not dominated by other ones, reducing the size of a possibly large initial dataset. However, with the dataset dimensions growing, the query result of the skyline operator can have a high cardinality that could overwhelm the final user looking at the result, yielding the very opposite goal. Moreover, it doesn't allow any user to express their preferences over the attributes, in order to control the dataset ca…
With the advancement of technology, the data generated in our lives is getting faster and faster, and the amount of data that various applications need to process becomes extremely huge. Therefore, we need to put more effort into analyzing data and extracting valuable information. Cloud computing used to be a good technology to solve a large number of data analysis problems. However, in the era of the popularity of the Internet of Things (IoT), transmitting sensing data back to the cloud for centralized data analysis will consume a lot of wireless communication and network transmission costs. …
Multi-objective optimization is the problem of optimizing simultaneously multiple objective functions and several techniques exist to deal with this problem. This paper aims to present the main methods that can be used to solve this issue in the context of relational databases. In particular, this work examines Top-k query to get the k best result from a dataset and Skyline query that provides a more general overview of the best results. We also discuss Flexible-skyline, a new method designed to improve upon the previous techniques, mitigating their shortcomings. For each method, we describe t…
Adaptive Probabilistic Skyline Analytics for Mobile Edge-Cloud Systems via State-Aware Deep Reinforcement Learning
National Chung Cheng University（中正大学）；Advanced Institute of Manufacturing with High-tech Innovations (AIM-HI), National Chung Cheng University（中正大学先进制造与高科技创新研究院）
In this paper we study skyline queries in the distributed computational model, where we have $s$ remote sites and a central coordinator (the query node); each site holds a piece of data, and the coordinator wants to compute the skyline of the union of the $s$ datasets. The computation is in terms of rounds, and the goal is to minimize both the total communication cost and the round cost. Viewing data objects as points in the Euclidean space, we consider both the horizontal data partition case where each site holds a subset of points, and the vertical data partition case where each site holds o…
Understanding the influence of a product is crucially important for making informed business decisions. This paper introduces a new type of skyline queries, called uncertain reverse skyline, for measuring the influence of a probabilistic product in uncertain data settings. More specifically, given a dataset of probabilistic products P and a set of customers C, an uncertain reverse skyline of a probabilistic product q retrieves all customers c in C which include q as one of their preferred products. We present efficient pruning ideas and techniques for processing the uncertain reverse skyline q…
Assisting end users to identify desired results from a large dataset is an important problem for multi-criteria decision making. To address this problem, top-k and skyline queries have been widely adopted, but they both have inherent drawbacks, i.e., the user either has to provide a specific utility function or faces many results. The k-regret minimization query is proposed, which integrates the merits of top-k and skyline queries. Due to the NP-hardness of the problem, the k-regret minimization query is time consuming and the greedy framework is widely adopted. However, formal theoretical ana…
Motivated by cloud security concerns, there is an increasing interest in database systems that can store and support queries over encrypted data. A common architecture for such systems is to use a trusted component such as a cryptographic co-processor for query processing that is used to securely decrypt data and perform computations in plaintext. The trusted component has limited memory, so most of the (input and intermediate) data is kept encrypted in an untrusted storage and moved to the trusted component on ``demand.'' In this setting, even with strong encryption, the data access pattern f…
Smart city has been consider the wave of the future and the route recommendation in networks is a fundamental problem in it. Most existing approaches for the shortest route problem consider that there is only one kind of cost in networks. However, there always are several kinds of cost in networks and users prefer to select an optimal route under the global consideration of these kinds of cost. In this paper, we study the problem of finding the optimal route in the multi-cost networks. We prove this problem is NP-hard and the existing index techniques cannot be used to this problem. We propose…
People and machines are collecting data at an unprecedented rate. Despite this newfound abundance of data, progress has been slow in sharing it for open science, business, and other data-intensive endeavors. Many such efforts are stymied by privacy concerns and regulatory compliance issues. For example, many hospitals are interested in pooling their medical records for research, but none may disclose arbitrary patient records to researchers or other healthcare providers. In this context we propose the Private Data Network (PDN), a federated database for querying over the collective data of mut…
Spatial queries like range queries, nearest neighbor, circular range queries etc. are the most widely used queries in the location-based applications. Building secure and efficient solutions for these queries in the cloud computing framework is critical and has been an area of active research. This paper focuses on the problem of Secure Circular Range Queries (SCRQ), where client submits an encrypted query (consisting of a center point and radius of the circle) and the cloud (storing encrypted data points) has to return the points lying inside the circle. The existing solutions for this proble…
The database community, at least for the last decade, has been grappling with querying encrypted data, which would enable secure database as a service solutions. A recent breakthrough in the cryptographic community (in 2009) related to fully homomorphic encryption (FHE) showed that arbitrary computation on encrypted data is possible. Successful adoption of FHE for query processing is, however, still a distant dream, and numerous challenges have to be addressed. One challenge is how to perform algebraic query processing of encrypted data, where we produce encrypted intermediate results and oper…
Various cryptographic techniques are used in outsourced database systems to ensure data privacy while allowing for efficient querying. This work proposes a definition and components of a new secure and efficient outsourced database system, which answers various types of queries, with different privacy guarantees in different security models. This work starts with the survey of five order-revealing encryption schemes that can be used directly in many database indices and five range query protocols with various security / efficiency tradeoffs. The survey systematizes the state-of-the-art range q…
Outsourcing data into the cloud becomes popular thanks to the pay-as-you-go paradigm. However, such practice raises privacy concerns. The conventional way to achieve data privacy is to encrypt sensitive data before outsourcing. When data are encrypted, a trade-off must be achieved between security and efficient query processing. Existing solutions that adopt multiple encryption schemes induce a heavy overhead in terms of data storage and query performance, and are not suited for cloud data warehouses. In this paper, we propose an efficient additive encryption scheme (S4) based on Shamir's secr…
Over the past years several works have proposed access control models for XML data where only read-access rights over non-recursive DTDs are considered. A few amount of works have studied the access rights for updates. In this paper, we present a general model for specifying access control on XML data in the presence of update operations of W3C XQuery Update Facility. Our approach for enforcing such updates specifications is based on the notion of query rewriting where each update operation defined over arbitrary DTD (recursive or not) is rewritten to a safe one in order to be evaluated only o…
Most state-of-the art approaches for securing XML documents allow users to access data only through authorized views defined by annotating an XML grammar (e.g. DTD) with a collection of XPath expressions. To prevent improper disclosure of confidential information, user queries posed on these views need to be rewritten into equivalent queries on the underlying documents. This rewriting enables us to avoid the overhead of view materialization and maintenance. A major concern here is that query rewriting for recursive XML views is still an open problem. To overcome this problem, some works have b…
A private data federation is a set of autonomous databases that share a unified query interface offering in-situ evaluation of SQL queries over the union of the sensitive data of its members. Owing to privacy concerns, these systems do not have a trusted data collector that can see all their data and their member databases cannot learn about individual records of other engines. Federations currently achieve this goal by evaluating queries obliviously using secure multiparty computation. This hides the intermediate result cardinality of each query operator by exhaustively padding it. With casca…
While the security of the cloud remains a concern, a common practice is to encrypt data before outsourcing them for utilization. One key challenging issue is how to efficiently perform queries over the ciphertext. Conventional crypto-based solutions, e.g. partially/fully homomorphic encryption and searchable encryption, suffer from low performance, poor expressiveness and weak compatibility. An alternative method that utilizes hardware-assisted trusted execution environment, i.e., Intel SGX, has emerged recently. On one hand, such work lacks of supporting scalable access control over multiple …
Despite exciting progress on cryptography, secure and efficient query processing over outsourced data remains an open challenge. We develop a communication-efficient and information-theoretically secure system, entitled Obscure for aggregation queries with conjunctive or disjunctive predicates, using secret-sharing. Obscure is strongly secure (i.e., secure regardless of the computational-capabilities of an adversary) and prevents the network, as well as, the (adversarial) servers to learn the user's queries, results, or the database. In addition, Obscure provides additional security features, …
With the wide deployment of public cloud computing infrastructures, using clouds to host data query services has become an appealing solution for the advantages on scalability and cost-saving. However, some data might be sensitive that the data owner does not want to move to the cloud unless the data confidentiality and query privacy are guaranteed. On the other hand, a secured query service should still provide efficient query processing and significantly reduce the in-house workload to fully realize the benefits of cloud computing. We propose the RASP data perturbation method to provide secu…
A closer integration of machine learning and relational databases has gained steam in recent years due to the fact that the training data to many ML tasks is the results of a relational query (most often, a join-select query). In a federated setting, this poses an additional challenge, that the tables are held by different parties as their private data, and the parties would like to train the model without having to use a trusted third party. Existing work has only considered the case where the training data is stored in a flat table that has been vertically partitioned, which corresponds to a…
Treasure Data is processing millions of distributed SQL queries every day on the cloud. Upgrading the query engine service at this scale is challenging because we need to migrate all of the production queries of the customers to a new version while preserving the correctness and performance of the data processing pipelines. To ensure the quality of the query engines, we utilize our query logs to build customer-specific benchmarks and replay these queries with real customer data in a secure pre-production environment. To simulate millions of queries, we need effective minimization of test query…
While working in collaborative team elsewhere sometimes the federated (huge) data are from heterogeneous cloud vendors. It is not only about the data privacy concern but also about how can those federated data can be querying from cloud directly in fast and securely way. Previous solution offered hybrid cloud between public and trusted private cloud. Another previous solution used encryption on MapReduce framework. But the challenge is we are working on heterogeneous clouds. In this paper, we present a novel technique for querying with privacy concern. Since we take execution time into account…
In this paper, we consider secure outsourced growing databases that support view-based query answering. These databases allow untrusted servers to privately maintain a materialized view, such that they can use only the materialized view to process query requests instead of accessing the original data from which the view was derived. To tackle this, we devise a novel view-based secure outsourced growing database framework, Incshrink. The key features of this solution are: (i) Incshrink maintains the view using incremental MPC operators which eliminates the need for a trusted third party upfront…
Cloud business intelligence is an increasingly popular choice to deliver decision support capabilities via elastic, pay-per-use resources. However, data security issues are one of the top concerns when dealing with sensitive data. In this pa-per, we propose a novel approach for securing cloud data warehouses by flexible verifiable secret sharing, fVSS. Secret sharing encrypts and distributes data over several cloud ser-vice providers, thus enforcing data privacy and availability. fVSS addresses four shortcomings in existing secret sharing-based approaches. First, it allows refreshing the data …
Background: Genomic data have been collected by different institutions and companies and need to be shared for broader use. In a cross-site genomic data sharing system, a secure and transparent access control audit module plays an essential role in ensuring the accountability. The 2018 iDASH competition first track provides us with an opportunity to design efficient logging and querying system for cross-site genomic dataset access audit. We designed a blockchain-based log system which can provide a light-weight and widely compatible module for existing blockchain platforms. The submitted solut…
Genomic data are becoming increasingly valuable as we develop methods to utilize the information at scale and gain a greater understanding of how genetic information relates to biological function. Advances in synthetic biology and the decreased cost of sequencing are increasing the amount of privately held genomic data. As the quantity and value of private genomic data grows, so does the incentive to acquire and protect such data, which creates a need to store and process these data securely. We present an algorithm for the Secure Interrogation of Genomic DataBases (SIG-DB). The SIG-DB algori…
Differential privacy offers a formal privacy guarantee for individuals, but many deployments of differentially private systems require a trusted third party (the data curator). We propose DuetSGX, a system that uses secure hardware (Intel's SGX) to eliminate the need for a trusted data curator. Data owners submit encrypted data that can be decrypted only within a secure enclave running the DuetSGX system, ensuring that sensitive data is never available to the data curator. Analysts submit queries written in the Duet language, which is specifically designed for verifying that programs satisfy d…
Despite extensive research on cryptography, secure and efficient query processing over outsourced data remains an open challenge. This paper continues along with the emerging trend in secure data processing that recognizes that the entire dataset may not be sensitive, and hence, non-sensitivity of data can be exploited to overcome limitations of existing encryption-based approaches. We, first, provide a new security definition, entitled partitioned data security for guaranteeing that the joint processing of non-sensitive data (in cleartext) and sensitive data (in encrypted form) does not lead …
Despite extensive research on cryptography, secure and efficient query processing over outsourced data remains an open challenge. This paper continues along the emerging trend in secure data processing that recognizes that the entire dataset may not be sensitive, and hence, non-sensitivity of data can be exploited to overcome limitations of existing encryption-based approaches. We propose a new secure approach, entitled query binning (QB) that allows non-sensitive parts of the data to be outsourced in clear-text while guaranteeing that no information is leaked by the joint processing of non-se…
A Web Service Management System (WSMS) can be well-thought-out as a consistent and a secure way of managing the web services. Web Service has become a quintessential part of the web world, managing and sharing the resources of the business it is associated with. In this paper, we focus on the query optimization aspect of handling the "natural language" query, queried to the WSMS. The map-select-composite operations are piloted to select specific web services. The main aftermath of our research is ensued in an algorithm which uses cost-based as well as heuristic based approach for query optimiz…
We present a relational MPC framework for secure collaborative analytics on private data with no information leakage. Our work targets challenging use cases where data owners may not have private resources to participate in the computation, thus, they need to securely outsource the data analysis to untrusted third parties. We define a set of oblivious operators, explain the secure primitives they rely on, and analyze their costs in terms of operations and inter-party communication. We show how these operators can be composed to form end-to-end oblivious queries, and we introduce logical and ph…
