Online Automated Privacy Policy Generators (APPGs) are tools used by app developers to quickly create app privacy policies which are required by privacy regulations to be incorporated to each mobile app. The creation of these tools brings convenience to app developers; however, the quality of these tools puts developers and stakeholders at legal risk. In this paper, we conduct an empirical study to assess the quality of online APPGs. We analyze the completeness of privacy policies, determine what categories and items should be covered in a complete privacy policy, and conduct APPG assessment with boilerplate apps. The results of assessment show that due to the lack of static or dynamic analysis of app's behavior, developers may encounter two types of issues caused by APPGs. First, the generated policies could be incomplete because they do not cover all the essential items required by a privacy policy. Second, some generated privacy policies contain unnecessary personal information collection or arbitrary commitments inconsistent with user input. Ultimately, the defects of APPGs may potentially lead to serious legal issues. We hope that the results and insights developed in this paper can motivate the healthy and ethical development of APPGs towards generating a more complete, accurate, and robust privacy policy.
Privacy analysis is critical but also a time-consuming and tedious task. We present a formalization which eases designing and auditing high-level privacy properties of software architectures. It is incorporated into a larger policy analysis and verification framework and enables the assessment of commonly accepted data protection goals of privacy. The formalization is based on static taint analysis and makes flow and processing of privacy-critical data explicit, globally as well as on the level of individual data subjects. Formally, we show equivalence to traditional label-based information fl…
Mounting privacy legislation calls for the preservation of privacy in speech technology, though solutions are gravely lacking. While evaluation campaigns are long-proven tools to drive progress, the need to consider a privacy adversary implies that traditional approaches to evaluation must be adapted to the assessment of privacy and privacy preservation solutions. This paper presents the first step in this direction: metrics. We introduce the zero evidence biometric recognition assessment (ZEBRA) framework and propose two new privacy metrics. They measure the average level of privacy preservat…
Privacy risk assessments aim to analyze and quantify the privacy risks associated with new systems. As such, they are critically important in ensuring that adequate privacy protections are built in. However, current methods to quantify privacy risk rely heavily on experienced analysts picking the "correct" risk level on e.g. a five-point scale. In this paper, we argue that a more scientific quantification of privacy risk increases accuracy and reliability and can thus make it easier to build privacy-friendly systems. We discuss how the impact and likelihood of privacy violations can be decompo…
We consider the privacy implications of public release of a de-identified dataset of Opal card transactions. The data was recently published at https://opendata.transport.nsw.gov.au/dataset/opal-tap-on-and-tap-off. It consists of tap-on and tap-off counts for NSW's four modes of public transport, collected over two separate week-long periods. The data has been further treated to improve privacy by removing small counts, aggregating some stops and routes, and perturbing the counts. This is a summary of our findings.
The rapid spread of COVID-19 has made manual contact tracing difficult. Thus, various public health authorities have experimented with automatic contact tracing using mobile applications (or "apps"). These apps, however, have raised security and privacy concerns. In this paper, we propose an automated security and privacy assessment tool, COVIDGUARDIAN, which combines identification and analysis of Personal Identification Information (PII), static program analysis and data flow analysis, to determine security and privacy weaknesses. Furthermore, in light of our findings, we undertake a user st…
Privacy and data protection have become more and more important in recent years since an increasing number of enterprises and startups are harvesting personal data as a part of their business model. One central requirement of the GDPR is the implementation of a data protection impact assessment for privacy critical systems. However, the law does not dictate or recommend the use of any particular framework. In this paper we compare different data protection impact assessment frameworks. We have developed a comparison and evaluation methodology and applied this to three popular impact assessment…
Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
In this paper, we develop a user-centric privacy framework for quantitatively assessing the exposure of personal information in open settings. Our formalization addresses key-challenges posed by such open settings, such as the unstructured dissemination of heterogeneous information and the necessity of user- and context-dependent privacy requirements. We propose a new definition of information sensitivity derived from our formalization of privacy requirements, and, as a sanity check, show that hard non-disclosure guarantees are impossible to achieve in open settings. After that, we provide an …
Genome-wide association studies (GWAS) identify correlations between the genetic variants and an observable characteristic such as a disease. Previous works presented privacy-preserving distributed algorithms for a federation of genome data holders that spans multiple institutional and legislative domains to securely compute GWAS results. However, these algorithms have limited applicability, since they still require a centralized instance to decide whether GWAS results can be safely disclosed, which is in violation to privacy regulations, such as GDPR. In this work, we introduce GenDPR, a dist…
Social Virtual Reality based Learning Environments (VRLEs) such as vSocial render instructional content in a three-dimensional immersive computer experience for training youth with learning impediments. There are limited prior works that explored attack vulnerability in VR technology, and hence there is a need for systematic frameworks to quantify risks corresponding to security, privacy, and safety (SPS) threats. The SPS threats can adversely impact the educational user experience and hinder delivery of VRLE content. In this paper, we propose a novel risk assessment framework that utilizes at…
Information security and privacy in the healthcare sector is an issue of growing importance. The adoption of digital patient records, increased regulation, provider consolidation and the increasing need for information exchange between patients, providers and payers, all point towards the need for better information security. We critically survey the literature on information security and privacy in healthcare, published in information systems journals as well as many other related disciplines including health informatics, public health, law, medicine, and the trade press and industry reports
The security of many cryptographic constructions depends on random number generators for providing unpredictable keys, nonces, initialization vectors and other parameters. Modern operating systems implement cryptographic pseudo-random number generators (PRNGs) to fulfill this need. Performance counters and other system parameters are often used as a low-entropy source to initialize (seed) the generators. We perform an experiment to analyze all performance counters in standard installation of Microsoft Windows 7 operating system, and assess their suitability as entropy sources. Besides selectin…
A Practical Framework for Evaluating Medical AI Security: Reproducible Assessment of Jailbreaking and Privacy Vulnerabilities Across Clinical Specialties
Web-based chatbots provide website owners with the benefits of increased sales, immediate response to their customers, and insight into customer behaviour. While Web-based chatbots are getting popular, they have not received much scrutiny from security researchers. The benefits to owners come at the cost of users' privacy and security. Vulnerabilities, such as tracking cookies and third-party domains, can be hidden in the chatbot's iFrame script. This paper presents a large-scale analysis of five Web-based chatbots among the top 1-million Alexa websites. Through our crawler tool, we identify t…
University of Marburg（马尔堡大学）；Technical University of Darmstadt（达姆施塔特工业大学）；Georgia Institute of Technology（佐治亚理工学院）；Manipal Institute of Technology（马尼帕尔理工学院）
Clinical NLP tasks such as mental health assessment from text, must take social constraints into account - the performance maximization must be constrained by the utmost importance of guaranteeing privacy of user data. Consumer protection regulations, such as GDPR, generally handle privacy by restricting data availability, such as requiring to limit user data to 'what is necessary' for a given purpose. In this work, we reason that providing stricter formal privacy guarantees, while increasing the volume of user data in the model, in most cases increases benefit for all parties involved, especi…
The proliferation of speech technologies and rising privacy legislation calls for the development of privacy preservation solutions for speech applications. These are essential since speech signals convey a wealth of rich, personal and potentially sensitive information. Anonymisation, the focus of the recent VoicePrivacy initiative, is one strategy to protect speaker identity information. Pseudonymisation solutions aim not only to mask the speaker identity and preserve the linguistic content, quality and naturalness, as is the goal of anonymisation, but also to preserve voice distinctiveness. …
Next-generation wireless networks like 5G promise faster speed, shorter latency, and the ability to connect more devices. Such benefits are set to make drastic changes to the future society, empowering smart cities, enabling autonomous cars, enhancing business processes, changing consumer behaviors, etc. In the financial industry, banks evaluate the deployment of Internet of Things (IoT) technologies and edge computing for better customer engagement, e.g., mobile branches on a vehicle, micro-ATM, self-service digital panel, etc. One of the trends is breaking down monolithic business applicatio…
Although there are privacy-enhancing tools designed to protect users' online privacy, it is surprising to see a lack of user-centric solutions allowing privacy control based on the joint assessment of privacy risks and benefits, due to data disclosure to multiple platforms. In this paper, we propose a conceptual framework to fill the gap: aiming at the user-centric privacy protection, we show the framework can not only assess privacy risks in using online services but also the added values earned from data disclosure. Through following a human-in-the-loop approach, it is expected the framework…
Sungkyunkwan University（成均馆大学）；Samsung Medical Center（三星医疗中心）
As deep learning have been applied in a clinical context, privacy concerns have increased because of the collection and processing of a large amount of personal data. Recently, federated learning (FL) has been suggested to protect personal privacy because it does not centralize data during the training phase. In this study, we assessed the reliability and performance of FL on benchmark datasets including MNIST and MIMIC-III. In addition, we attempted to verify FL on datasets that simulated a realistic clinical data distribution. We implemented FL that uses a client and server architecture and …
Shanghai Jiao Tong University（上海交通大学）；The University of Edinburgh（爱丁堡大学）；Sony AI（索尼人工智能研究院）
To prevent unintentional data leakage, research community has resorted to data generators that can produce differentially private data for model training. However, for the sake of the data privacy, existing solutions suffer from either expensive training cost or poor generalization performance. Therefore, we raise the question whether training efficiency and privacy can be achieved simultaneously. In this work, we for the first time identify that dataset condensation (DC) which is originally designed for improving training efficiency is also a better solution to replace the traditional data ge…
University of Wisconsin（威斯康星大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
The EU General Data Protection Regulation (GDPR) is one of the most demanding and comprehensive privacy regulations of all time. A year after it went into effect, we study its impact on the landscape of privacy policies online. We conduct the first longitudinal, in-depth, and at-scale assessment of privacy policies before and after the GDPR. We gauge the complete consumption cycle of these policies, from the first user impressions until the compliance assessment. We create a diverse corpus of two sets of 6,278 unique English-language privacy policies from inside and outside the EU, covering th…
PriEval-Protect: A Unified Framework for Privacy Evaluation and Protection in Healthcare Systems
National Institute of Applied Science and Technology INSAT（国立应用科学与技术学院）；Efrei Research Lab, Efrei, Paris-Panthéon-Assas University（巴黎-潘特翁-阿萨斯大学埃弗雷伊研究实验室）；LARIA Research Lab, National School Of Computer Science ENSI（国立计算机科学学院拉里亚研究实验室）
Big data analysis poses the dual problem of privacy preservation and utility, i.e., how accurate data analyses remain after transforming original data in order to protect the privacy of the individuals that the data is about - and whether they are accurate enough to be meaningful. In this paper, we thus investigate across several datasets whether different methods of generating fully synthetic data vary in their utility a priori (when the specific analyses to be performed on the data are not known yet), how closely their results conform to analyses on original data a posteriori, and whether th…
As machine learning expanding application, there are more and more unignorable privacy and safety issues. Especially inference attacks against Machine Learning models allow adversaries to infer sensitive information about the target model, such as training data, model parameters, etc. Inference attacks can lead to serious consequences, including violating individuals privacy, compromising the intellectual property of the owner of the machine learning model. As far as concerned, researchers have studied and analyzed in depth several types of inference attacks, albeit in isolation, but there is …
Privacy regulation laws, such as GDPR, impose transparency and security as design pillars for data processing algorithms. In this context, federated learning is one of the most influential frameworks for privacy-preserving distributed machine learning, achieving astounding results in many natural language processing and computer vision tasks. Several federated learning frameworks employ differential privacy to prevent private data leakage to unauthorized parties and malicious attackers. Many studies, however, highlight the vulnerabilities of standard federated learning to poisoning and inferen…
Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents
University of Notre Dame（圣约翰大学）；Northeastern University（东北大学）；Virginia Tech（弗吉尼亚理工大学）；University of Washington（华盛顿大学）；Johns Hopkins University（约翰霍普金斯大学）
Federated learning (FL) is getting increased attention for processing sensitive, distributed datasets common to domains such as healthcare. Instead of directly training classification models on these datasets, recent works have considered training data generators capable of synthesising a new dataset which is not protected by any privacy restrictions. Thus, the synthetic data can be made available to anyone, which enables further evaluation of machine learning architectures and research questions off-site. As an additional layer of privacy-preservation, differential privacy can be introduced i…
Today, online privacy is the domain of regulatory measures and privacy-enhancing technologies. Transparency in the form of external and public assessments has been proposed for improving privacy and security because it exposes otherwise hidden deficiencies. Previous work has studied privacy attitudes and behavior of consumers. However, little is known on how organizations react to measures that employ public "naming and shaming" as an incentive for improvement. We performed the first study on this aspect by conducting a qualitative survey with 152 German health insurers. We scanned their websi…
