Mobile access to information is a considerable problem for many users, especially to information found on the Web. In this paper, we explore how a voice-controlled service, accessible by telephone, could support mobile users' needs for refinding specific information previously found on the Web. We outline challenges in creating such a service and describe architectural and user interfaces issues discovered in an exploratory prototype we built called WebContext.
We also present the results of a study, motivated by our experience with WebContext, to explore what people remember about information that they are trying to refind and how they express information refinding requests in a collaborative conversation. As part of the study, we examine how end-usercreated Web page annotations can be used to help support mobile information re-finding. We observed the use of URLs, page titles, and descriptions of page contents to help identify waypoints in the search process. Furthermore, we observed that the annotations were utilized extensively, indicating that explicitly added context by the user can play an important role in re-finding.
Web pages contain a large variety of information, but are largely designed for use by graphical web browsers. Mobile access to web-based information often requires presenting HTML web pages using channels that are limited in their graphical capabilities such as small-screens or audio-only interfaces. Content transcoding and annotations have been explored as methods for intelligently presenting HTML documents. Much of this work has focused on transcoding for small-screen devices such as are found on PDAs and cell phones. Here, we focus on the use of annotations and transcoding for presenting HT…
Voice input has been tremendously improving the user experience of mobile devices by freeing our hands from typing on the small screen. Speech recognition is the key technology that powers voice input, and it is usually outsourced to the cloud for the best performance. However, the cloud might compromise users' privacy by identifying their identities by voice, learning their sensitive input content via speech recognition, and then profiling the mobile users based on the content. In this paper, we design an intermediate between users and the cloud, named VoiceMask, to sanitize users' voice data…
The iPhone was introduced only a decade ago in 2007 but has fundamentally changed the way we interact with online information. Mobile devices differ radically from classic command-based and point-and-click user interfaces, now allowing for gesture-based interaction using fine-grained touch and swipe signals. Due to the rapid growth in the use of voice-controlled intelligent personal assistants on mobile devices, such as Microsoft's Cortana, Google Now, and Apple's Siri, mobile devices have become personal, allowing us to be online all the time, and assist us in any task, both in work and in ou…
We report a user study of over four months on the non-voice usage of mobile phones by teens from an underserved urban community in the USA where a community-wide, open-access Wi-Fi network exists. We instrumented the phones to record quantitative information regarding their usage and location in a privacy-respecting manner. We conducted focus group meetings and interviewed participants regularly for qualitative data. We present our findings on what applications our participants used and how their usage changed over time. The findings highlight the challenges to evaluating the usability of mobi…
Supporting voice commands in applications presents significant benefits to users. However, adding such support to existing GUI-based web apps is effort-consuming with a high learning barrier, as shown in our formative study, due to the lack of unified support for creating multimodal interfaces. We present Geno---a developer tool for adding the voice input modality to existing web apps without requiring significant NLP expertise. Geno provides a high-level workflow for developers to specify functionalities to be supported by voice (intents), create language models for detecting intents and the …
Automatic voice-controlled systems have changed the way humans interact with a computer. Voice or speech recognition systems allow a user to make a hands-free request to the computer, which in turn processes the request and serves the user with appropriate responses. After years of research and developments in machine learning and artificial intelligence, today voice-controlled technologies have become more efficient and are widely applied in many domains to enable and improve human-to-human and human-to-computer interactions. The state-of-the-art e-commerce applications with the help of web t…
Traditionally, the efficiency and effectiveness of search systems have both been of great interest to the information retrieval community. However, an in-depth analysis of the interaction between the response latency and users' subjective search experience in the mobile setting has been missing so far. To address this gap, we conduct a controlled study that aims to reveal how response latency affects mobile web search. Our preliminary results indicate that mobile web search users are four times more tolerant to response latency reported for desktop web search users. However, when exceeding a c…
The trend towards mobile devices usage has put more than ever the Web as a ubiquitous platform where users perform all kind of tasks. In some cases, users access the Web with 'native' mobile applications developed for well-known sites, such as LinkedIn, Facebook, Twitter, etc. These native applications might offer further (e.g. location-based) functionalities to their users in comparison with their corresponding Web sites, because they were developed with mobile features in mind. However, most Web applications have not this native mobile counterpart and users access them using browsers in the …
Voice authentication is drawing increasing attention and becomes an attractive alternative to passwords for mobile authentication. Recent advances in mobile technology further accelerate the adoption of voice biometrics in an array of diverse mobile applications. However, recent studies show that voice authentication is vulnerable to replay attacks, where an adversary can spoof a voice authentication system using a pre-recorded voice sample collected from the victim. In this paper, we propose VoiceLive, a liveness detection system for both text-dependent and text-independent voice authenticati…
With the tremendous growth in the information communication sector, the mobile phones have become the prime information communication devices. The convergence of traditional telephony with the modern web enabled communication in the mobile devices has made the communication much effective and simpler. As mobile phones are becoming the crucial source of accessing the contents of the World Wide Web which was originally designed for personal computers, has opened up a new challenge of accommodating the web contents in to the smaller mobile devices. This paper proposes an approach towards building…
The broad proliferation of mobile devices in recent years has drastically changed the means of accessing the World Wide Web. Describing a shift away from the desktop computer era for content consumption, predictions indicate that the main access of web-based content will come from mobile devices. Concurrently, the manner of content presentation has changed as well; web artifacts are allowing for richer media and higher levels of user interaction which is enabled through increasing access networks speeds. This article provides an overview of more than two years of high level web page characteri…
This is the preprint version of our paper on JOMS. In this paper, two mHealth applications are introduced, which can be employed as the terminals of bigdata based health service to collect information for electronic medical records (EMRs). The first one is a hybrid system for improving the user experience in the hyperbaric oxygen chamber by 3D stereoscopic virtual reality glasses and immersive perception. Several HMDs have been tested and compared. The second application is a voice interactive serious game as a likely solution for providing assistive rehabilitation tool for therapists. The rec…
Web technology underpins many interactive mobile applications. However, energy-efficient mobile web interactions is an outstanding challenge. Given the increasing diversity and complexity of mobile hardware, any practical optimization scheme must work for a wide range of users, mobile platforms and web workloads. This paper presents CAMEL , a novel energy optimization system for mobile web interactions. CAMEL leverages machine learning techniques to develop a smart, adaptive scheme to judiciously trade performance for reduced power consumption. Unlike prior work, C AMEL directly models how a g…
This paper examines some of the potential challenges associated with enabling a seamless web experience on underpowered mobile devices such as Google Glass from the perspective of web content providers, device, and the network. We conducted experiments to study the impact of webpage complexity, individual web components and different application layer protocols while accessing webpages on the performance of Glass browser, by measuring webpage load time, temperature variation and power consumption and compare it to a smartphone. Our findings suggest that (a) performance of Glass compared to a s…
We contribute MobileVisFixer, a new method to make visualizations more mobile-friendly. Although mobile devices have become the primary means of accessing information on the web, many existing visualizations are not optimized for small screens and can lead to a frustrating user experience. Currently, practitioners and researchers have to engage in a tedious and time-consuming process to ensure that their designs scale to screens of different sizes, and existing toolkits and libraries provide little support in diagnosing and repairing issues. To address this challenge, MobileVisFixer automates …
IIHR – Hydroscience & Engineering, University of Iowa（爱荷华大学IIHR水科学与工程研究所）；Civil and Environmental Engineering, University of Iowa（爱荷华大学土木与环境工程学院）
COVID-19 pandemic elucidated that knowledge systems will be instrumental in cases where accurate information needs to be communicated to a substantial group of people with different backgrounds and technological resources. However, several challenges and obstacles hold back the wide adoption of virtual assistants by public health departments and organizations. This paper presents the Instant Expert, an open-source semantic web framework to build and integrate voice-enabled smart assistants (i.e. chatbots) for any web platform regardless of the underlying domain and technology. The component al…
Displays based on organic light-emitting diode (OLED) technology are appearing on many mobile devices. Unlike liquid crystal displays (LCD), OLED displays consume dramatically different power for showing different colors. In particular, OLED displays are inefficient for showing bright colors. This has made them undesirable for mobile devices because much of the web content is of bright colors. To tackle this problem, we present the motivational studies, design, and realization of Chameleon, a color adaptive web browser that renders web pages with power-optimized color schemes under user-suppli…
This paper describes the ongoing development of a conversational interaction concept that allows visually impaired users to easily create and edit text documents on mobile devices using mainly voice input. In order to verify the concept, a prototype app was developed and tested for both iOS and Android systems, based on the natural-language understanding (NLU) platform Google Dialogflow. The app and interaction concept were repeatedly tested by users with and without visual impairments. Based on their feedback, the concept was continuously refined, adapted and improved on both mobile platforms…
While Alexa can perform over 100,000 skills on paper, its capability covers only a fraction of what is possible on the web. To reach the full potential of an assistant, it is desirable that individuals can create skills to automate their personal web browsing routines. Many seemingly simple routines, however, such as monitoring COVID-19 stats for their hometown, detecting changes in their child's grades online, or sending personally-addressed messages to a group, cannot be automated without conventional programming concepts such as conditional and iterative evaluation. This paper presents VASH…
In this paper, we discuss the physical web projects based on network proximity for Smart Cities. In general, the Physical Web is an approach for connecting any physical object to the web. With this approach, we can navigate and control physical objects in the world surrounding mobile devices. Alternatively, we can execute services on mobile devices, depending on the surrounding physical objects. Technically, there are different ways to enumerate physical objects. In this paper, we will target the models based on the wireless proximity.
Aiming at increasing system simplicity and flexibility, an audio evoked based system was developed by integrating simplified headphone and user-friendly software design. This paper describes a Hindi Speech Actuated Computer Interface for Web search (HSACIWS), which accepts spoken queries in Hindi language and provides the search result on the screen. This system recognizes spoken queries by large vocabulary continuous speech recognition (LVCSR), retrieves relevant document by text retrieval, and provides the search result on the Web by the integration of the Web and the voice systems. The LVCS…
The ability to estimate the current affective statuses of web users has considerable potential for the realization of user-centric services in the society. However, in real-world web services, it is difficult to determine the type of data to be used for such estimation, as well as collecting the ground truths of such affective statuses. We propose a novel method of such estimation based on the combined use of user web search queries and mobile sensor data. The system was deployed in our product server stack, and a large-scale data analysis with more than 11,000,000 users was conducted. Interes…
Phishing is an online fraudulent technique, which aims to steal sensitive information such as usernames, passwords and online banking details from its victims. To prevent this, anti-phishing education needs to be considered. This research focuses on examining the effectiveness of mobile game based learning compared to traditional online learning to thwart phishing threats. Therefore, a mobile game prototype was developed based on the design introduced by Arachchilage and Cole [3]. The game design aimed to enhance avoidance behaviour through motivation to thwart phishing threats. A website deve…
Dyson School of Design Engineering（戴森设计工程学院）；Imperial College London（伦敦帝国理工学院）
PlugSonic is a suite of web- and mobile-based applications for the curation and experience of binaural interactive soundscapes and sonic narratives. It was developed as part of the PLUGGY EU project (Pluggable Social Platform for Heritage Awareness and Participation) and consists of two main applications: PlugSonic Sample, to edit and apply audio effects, and PlugSonic Soundscape, to create and experience binaural soundscapes. The audio processing within PlugSonic is based on the Web Audio API and the 3D Tune-In Toolkit, while the exploration of soundscapes in a physical space is obtained usin…
Riga Technical University（里加技术大学）；University of Central Lancashire（中央兰开夏大学）
asya is a mobile application that consists of deep learning models which analyze spectra of a human voice and do noise detection, speaker diarization, gender detection, tempo estimation, and classification of emotions using only voice. All models are language agnostic and capable of running in real-time. Our speaker diarization models have accuracy over 95% on the test data set. These models can be applied for a variety of areas like customer service improvement, sales effective conversations, psychology and couples therapy.
