For complex segmentation tasks, the achievable accuracy of fully automated systems is inherently limited. Specifically, when a precise segmentation result is desired for a small amount of given data sets, semi-automatic methods exhibit a clear benefit for the user. The optimization of human computer interaction (HCI) is an essential part of interactive image segmentation. Nevertheless, publications introducing novel interactive segmentation systems (ISS) often lack an objective comparison of HCI aspects. It is demonstrated, that even when the underlying segmentation algorithm is the same throughout interactive prototypes, their user experience may vary substantially. As a result, users prefer simple interfaces as well as a considerable degree of freedom to control each iterative step of the segmentation. In this article, an objective method for the comparison of ISS is proposed, based on extensive user studies. A summative qualitative content analysis is conducted via abstraction of visual and verbal feedback given by the participants. A direct assessment of the segmentation system is executed by the users via the system usability scale (SUS) and AttrakDiff-2 questionnaires. Furthermore, an approximation of the findings regarding usability aspects in those studies is introduced, conducted solely from the system-measurable user actions during their usage of interactive segmentation prototypes. The prediction of all questionnaire results has an average relative error of 8.9%, which is close to the expected precision of the questionnaire results themselves. This automated evaluation scheme may significantly reduce the resources necessary to investigate each variation of a prototype's user interface (UI) features and segmentation methodologies.
High resolution computed tomography (HRCT) is the most important imaging modality for interstitial lung diseases, where the radiologists are interested in identifying certain patterns, and their volumetric and regional distribution. The use of machine learning can assist the radiologists with both these tasks by performing semantic segmentation. In this paper, we propose an interactive annotation-tool for semantic segmentation that assists the radiologist in labeling CT scans. The annotation tool is evaluated by six radiologists and radiology residents classifying healthy lung and reticular pa…
This work designs, evaluates, and improves a proposed search engine interface for Visually Impaired VI users to efficiently perform web search activities. Our conceptual modeling technique is based on Formal Concept Analysis FCA that is used for data analysis. This approach highlights the hierarchized approach to represent the discovered concepts. It is combined with context interactive navigation in an interface which is called interactive search engine (InteractSE). This interface aims to reduce the time and effort required by the VI users to browse search results. InteractSE was evaluated b…
Interactive tools like user interfaces help democratize data access for end-users by hiding underlying programming details and exposing the necessary widget interface to users. Since customized interfaces are costly to build, automated interface generation is desirable. SQL is the dominant way to analyze data and there already exists logs to analyze data. Previous work proposed a syntactic approach to analyze structural changes in SQL query logs and automatically generates a set of widgets to express the changes. However, they do not consider layout usability and the sequential order of querie…
Pennsylvania State University（宾夕法尼亚州立大学）；Zhejiang University（浙江大学）
Providing explanations for deep neural networks (DNNs) is essential for their use in domains wherein the interpretability of decisions is a critical prerequisite. Despite the plethora of work on interpreting DNNs, most existing solutions offer interpretability in an ad hoc, one-shot, and static manner, without accounting for the perception, understanding, or response of end-users, resulting in their poor usability in practice. In this paper, we argue that DNN interpretability should be implemented as the interactions between users and models. We present i-Algebra, a first-of-its-kind interacti…
We present SummaryLens, a concept and prototype for a mobile tool that leverages automated text summarization to enable users to quickly scan and summarize physical text documents. We further combine this with a text-to-speech system to read out the summary on demand. With this concept, we propose and explore a concrete application case of bringing ongoing progress in AI and Natural Language Processing to a broad audience with interactive use cases in everyday life. Based on our implemented features, we describe a set of potential usage scenarios and benefits, including support for low-vision,…
Assessing the Usability of GutGPT: A Simulation Study of an AI Clinical Decision Support System for Gastrointestinal Bleeding Risk
Yale University（耶鲁大学）；CUNY Baruch College（纽约市立大学巴鲁克学院）；Yale School of Medicine（耶鲁医学院）；University of California, Berkeley（加州大学伯克利分校）；Cornell University（康奈尔大学）
In this paper we study a brand new topic of interactive image captioning with human in the loop. Different from automated image captioning where a given test image is the sole input in the inference stage, we have access to both the test image and a sequence of (incomplete) user-input sentences in the interactive scenario. We formulate the problem as Visually Conditioned Sentence Completion (VCSC). For VCSC, we propose asynchronous bidirectional decoding for image caption completion (ABD-Cap). With ABD-Cap as the core module, we build iCap, a web-based interactive image captioning system capab…
To foster usefulness and accountability of machine learning (ML), it is essential to explain a model's decisions in addition to evaluating its performance. Accordingly, the field of explainable artificial intelligence (XAI) has resurfaced as a topic of active research, offering approaches to address the "how" and "why" of automated decision-making. Within this domain, counterfactual explanations (CFEs) have gained considerable traction as a psychologically grounded approach to generate post-hoc explanations. To do so, CFEs highlight what changes to a model's input would have changed its predic…
Nowadays many software development frameworks implement Behavior-Driven Development (BDD) as a mean of automating the test of interactive systems under construction. Automated testing helps to simulate user's action on the User Interface and therefore check if the system behaves properly and in accordance to Scenarios that describe functional requirements. However, most of tools supporting BDD requires that tests should be written using low-level events and components that only exist when the system is already implemented. As a consequence of such low-level of abstraction, BDD tests can hardly…
Ownership is the concept of tracking aliases and mutations to data, useful for both memory safety and system design. The Rust programming language implements ownership via the borrow checker, a static analyzer that extends the core type system. The borrow checker is a notorious learning barrier for new Rust users. In this paper, I focus on the gap between understanding ownership in theory versus its implementation in the borrow checker. As a sound and incomplete analysis, compiler errors may arise from either ownership-unsound behavior or limitations of the analyzer. Understanding this distinc…
The majority of blindness is preventable, and is located in developing countries. While mHealth applications for retinal imaging in combination with affordable smartphone lens adaptors are a step towards better eye care access, the expert knowledge and additional hardware needed are often unavailable in developing countries. Eye screening apps without lens adaptors exist, but we do not know much about the experience of guiding users to take medical eye images. Additionally, when an AI based diagnosis is provided, trust plays an important role in ensuring in the adoption. This work addresses fa…
Georgia Institute of Technology（佐治亚理工学院）；Mailchimp
With the widespread use of toxic language online, platforms are increasingly using automated systems that leverage advances in natural language processing to automatically flag and remove toxic comments. However, most automated systems -- when detecting and moderating toxic language -- do not provide feedback to their users, let alone provide an avenue of recourse for these users to make actionable changes. We present our work, RECAST, an interactive, open-sourced web tool for visualizing these models' toxic predictions, while providing alternative suggestions for flagged toxic language. Our w…
Usability Analysis of Configurator User Interfaces with Multimodal Large Language Models
Graz University of Technology（格拉茨工业大学）；University of Klagenfurt（克拉根福大学）；isn – innovation service network GmbH（isn创新服务网络有限公司）；Morgendigital GmbH（Morgendigital有限公司）
Programming environments such as Snap, Scratch, and Processing engage learners by allowing them to create programming artifacts such as apps and games, with visual and interactive output. Learning programming with such a media-focused context has been shown to increase retention and success rate. However, assessing these visual, interactive projects requires time and laborious manual effort, and it is therefore difficult to offer automated or real-time feedback to students as they work. In this paper, we introduce SnapCheck, a dynamic testing framework for Snap that enables instructors to auth…
Despite recent advances in the field of explainable artificial intelligence systems, a concrete quantitative measure for evaluating the usability of such systems is nonexistent. Ensuring the success of an explanatory interface in interacting with users requires a cyclic, symbiotic relationship between human and artificial intelligence. We, therefore, propose explanatory efficacy, a novel metric for evaluating the strength of the cyclic relationship the interface exhibits. Furthermore, in a user study, we evaluated the perceived affect and workload and recorded the EEG signals of our participan…
Interactive Augmented Reality-enabled Outdoor Scene Visualization For Enhanced Real-time Disaster Response
Harokopio University of Athens（哈罗基皮奥雅典大学）；Archimedes, Athena Research Center（阿基米德，雅典研究中心）；Kingston University（金斯顿大学）；University of Western Macedonia（西马其顿大学）
Digital prototyping and evaluation using 3D modeling and digital human models are becoming more practical for customizing products to the preference of a user. However, the 3D modeling is less accessible to casual users, and digital human models suffer from insufficient body data and less intuitive illustration on how people use the product or how it accommodates to their body. Recently, VR-supported 'Do It Yourself' design has achieved real-time ergonomic evaluation with users themselves by capturing their poses, however, it lacks reliability and quality of design. In this paper, we explore a…
MemoCare: An Interactive Multimodal Mobile System for Automated Cognitive Screening
Lausanne University Hospital（洛桑大学医院）；Faculty of Biology and Medicine, University of Lausanne（洛桑大学生物与医学学院）；VNU University of Engineering and Technology（越南国立大学工程技术大学）；VinUni-Illinois Smart Health Center, VinUniversity（文大-伊利诺伊智能健康中心，文大）；University of Science, VNU-HCM（胡志明市越南国立大学科学大学）；Hanoi Medical University（河内医科大学）；National Geriatric Hospital（国家老年病医院）；Military Hospital 175（第175军医院）；School of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City（胡志明市医药大学医学院）；International University, VNU-HCM（胡志明市越南国立大学国际大学）；Vietnam National University Ho Chi Minh City（胡志明市越南国立大学）
Detecting and analyzing potential anomalous performances in cloud computing systems is essential for avoiding losses to customers and ensuring the efficient operation of the systems. To this end, a variety of automated techniques have been developed to identify anomalies in cloud computing performance. These techniques are usually adopted to track the performance metrics of the system (e.g., CPU, memory, and disk I/O), represented by a multivariate time series. However, given the complex characteristics of cloud computing data, the effectiveness of these automated methods is affected. Thus, su…
Interactive Machine Teaching (IMT) systems allow non-experts to easily create Machine Learning (ML) models. However, existing vision-based IMT systems either ignore annotations on the objects of interest or require users to annotate in a post-hoc manner. Without the annotations on objects, the model may misinterpret the objects using unrelated features. Post-hoc annotations cause additional workload, which diminishes the usability of the overall model building process. In this paper, we develop LookHere, which integrates in-situ object annotations into vision-based IMT. LookHere exploits users…
Recent graphical interfaces offer direct manipulation mechanisms for authoring visualizations, but are largely restricted to static output. To author interactive visualizations, users must instead turn to textual specification; but, such approaches impose a higher technical burden than their graphical counterparts. To bridge this gap, we introduce interaction design by demonstration: a novel method for authoring interaction techniques via direct manipulation. Users perform an interaction (e.g., button clicks, drags, or key presses) directly on the visualization they are editing. The system int…
Tabular visualization techniques integrate visual representations with tabular data to avoid additional cognitive load caused by splitting users' attention. However, most of the existing studies focus on simple flat tables instead of hierarchical tables, whose complex structure limits the expressiveness of visualization results and affects users' efficiency in visualization construction. We present HiTailor, a technique for presenting and exploring hierarchical tables. HiTailor constructs an abstract model, which defines row/column headings as biclustering and hierarchical structures. Based on…
