As part of formative and summative assessments in programming courses, students work on developing programming artifacts following a given specification. These artifacts are evaluated by the teachers. At the end of this evaluation, the students receive feedback and marks. Providing feedback on programming artifacts is time demanding and could make feedback to arrive too late for it to be effective for the students' learning. We propose to combine software testing with peer feedback which has been praised for offering a timely and effective learning activity with program testing.
In this paper we report on the development of a Web platform for peer feedback on programming artifacts through program testing. We discuss the development process of our peer-testing platform informed by teachers and students.
Web testing has long been recognized as a notoriously difficult task. Even nowadays, web testing still heavily relies on manual efforts while automated web testing is far from achieving human-level performance. Key challenges in web testing include dynamic content update and deep bugs hiding under complicated user interactions and specific input values, which can only be triggered by certain action sequences in the huge search space. In this paper, we propose WebExplor, an automatic end-to-end web testing framework, to achieve an adaptive exploration of web applications. WebExplor adopts curio…
To improve the reliability and efficiency of Web Software, the Testing Team should be creative and innovative. The experience and intuition of Tester also matters a lot and most often the destructive nature of Tester brings reliable software to the user. Actually, Testing is the responsibility of everybody who is involved in the Project.
Specification-based regression testing of web services is an important activity which verifies the quality of web services. A major problem in web services is that only provider has the source code and both user and broker only have the XML based specification. So from the perspective of user and broker, specification based regression testing of web services is needed. The existing techniques are code based. Due to the dynamic behavior of web services, web services undergo maintenance and evolution process rapidly. Retesting of web services is required in order to verify the impact of changes.…
WebCQ: Cooperative Multi-Agent Deep Reinforcement Learning for Scalable Web GUI Testing
Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology（可信自主系统研究院，南方科技大学）；Department of Computer Science and Engineering, Southern University of Science and Technology（计算机科学与工程系，南方科技大学）
Security testing aims at verifying that the software meets its security properties. In modern Web systems, however, this often entails the verification of the outputs generated when exercising the system with a very large set of inputs. Full automation is thus required to lower costs and increase the effectiveness of security testing. Unfortunately, to achieve such automation, in addition to strategies for automatically deriving test inputs, we need to address the oracle problem, which refers to the challenge, given an input for a system, of distinguishing correct from incorrect behavior. In t…
Security testing verifies that the data and the resources of software systems are protected from attackers. Unfortunately, it suffers from the oracle problem, which refers to the challenge, given an input for a system, of distinguishing correct from incorrect behavior. In many situations where potential vulnerabilities are tested, a test oracle may not exist, or it might be impractical due to the many inputs for which specific oracles have to be defined. In this paper, we propose a metamorphic testing approach that alleviates the oracle problem in security testing. It enables engineers to spec…
JavaScript has become one of the most widely used languages for Web development. However, it is challenging to ensure the correctness and reliability of Web applications written in JavaScript, due to their dynamic and event-driven features. A variety of dynamic analysis techniques for JavaScript Web applications have been proposed, but they are limited in either coverage or scalability. In this paper, we propose a model-based automated approach to achieve high code coverage in a reasonable amount of time via testing with longer event sequences. We implement our approach as the tool LJS, and pe…
In the context of End-to-End testing of web applications, automated exploration techniques (a.k.a. crawling) are widely used to infer state-based models of the site under test. These models, in which states represent features of the web application and transitions represent reachability relationships, can be used for several model-based testing tasks, such as test case generation. However, current exploration techniques often lead to models containing many near-duplicate states, i.e., states representing slightly different pages that are in fact instances of the same feature. This has a negati…
Context: Selenium is claimed to be the most popular software test automation tool. Past academic works have mainly neglected testing tools in favor of more methodological topics. Objective: We investigated the performance of web-testing tools, to provide empirical evidence supporting choices in software test tool selection and configuration. Method: We used 4*5 factorial design to study 20 different configurations for testing a web-store. We studied 5 programming language bindings (C#, Java, Python, and Ruby for Selenium, while Watir supports Ruby only) and 4 browsers (Google Chrome, Internet …
HxAgent: Iterative Agent Planning for End-to-End Web Application Testing
University of Science, Ho Chi Minh city, Vietnam（胡志明市科学大学）；Vietnam National University, Ho Chi Minh city, Vietnam（胡志明市越南国立大学）；University of Information Technology, Ho Chi Minh city, Vietnam（胡志明市信息技术大学）；University of Texas at Dallas, USA（美国德克萨斯大学达拉斯分校）
Modern web programming involves coordinating interactions between browser clients and a server. Typically, the interactions in web-based distributed systems are informally described, making it hard to ensure correctness, especially communication safety, i.e. all endpoints progress without type errors or deadlocks, conforming to a specified protocol. We present STScript, a toolchain that generates TypeScript APIs for communication-safe web development over WebSockets, and RouST, a new session type theory that supports multiparty communications with routing mechanisms. STScript provides develope…
We present TAO, a software testing tool performing automated test and oracle generation based on a semantic approach. TAO entangles grammar-based test generation with automated semantics evaluation using a denotational semantics framework. We show how TAO can be incorporated with the Selenium automation tool for automated web testing, and how TAO can be further extended to support automated delta debugging, where a failing web test script can be systematically reduced based on grammar-directed strategies. A real-life parking website is adopted throughout the paper to demonstrate the effectivit…
Technical University of Munich（慕尼黑工业大学）；Università degli Studi di Napoli Federico II（那不勒斯费德里科二世大学）；University of St. Gallen（圣加仑大学）；Università della Svizzera italiana（瑞士意大利语区大学）
PDPM Indian Institute of Information Technology, Design and Manufacturing Jabalpur（PDPM印度信息技术设计与制造学院贾巴尔普尔）
Reducing the effort required to make changes in web services is one of the primary goals in web service projects maintenance and evolution. Normally, functional and non-functional testing of a web service is performed by testing the operations specified in its WSDL. The regression testing is performed by identifying the changes made thereafter to the web service code and the WSDL. In this thesis, we present a tool-supported approach to perform efficient regression testing of web services. By representing a web service as a directed graph of WSDL elements, we identify and gathers the changed po…
Background: Research software is software developed by and/or used by researchers, across a wide variety of domains, to perform their research. Because of the complexity of research software, developers cannot conduct exhaustive testing. As a result, researchers have lower confidence in the correctness of the output of the software. Peer code review, a standard software engineering practice, has helped address this problem in other types of software. Aims: Peer code review is less prevalent in research software than it is in other types of software. In addition, the literature does not contain…
Towards Automated Page Object Generation for Web Testing using Large Language Models
Technical University of Munich（慕尼黑工业大学）；Università degli Studi di Genova（热那亚大学）；University of St. Gallen and USI（圣加仑大学和瑞士意大利语联邦理工学院）；fortiss GmbH（fortiss有限公司）
The HTML5 <canvas> is used to display high quality graphics in web applications such as web games (i.e., <canvas> games). However, automatically testing <canvas> games is not possible with existing web testing techniques and tools, and manual testing is laborious. Many widely used web testing tools rely on the Document Object Model (DOM) to drive web test automation, but the contents of the <canvas> are not represented in the DOM. The main alternative approach, snapshot testing, involves comparing oracle snapshot images with test-time snapshot images using an image similarity metric to catch v…
A Survey on Web Application Testing: Over a Decade of Evolution
School of Computer Science and Engineering, Macau University of Science and Technology（澳门科技大学计算机科学与工程学院）；Macau University of Science and Technology Zhuhai MUST Science and Technology Research Institute（澳门科技大学珠海MUST科学研究院）；School of Artificial Intelligence (Institute of Artificial Intelligence), Beihang University（北京航空航天大学人工智能学院（人工智能研究院））；School of Computer Science, University of Nottingham Ningbo China（诺丁汉大学中国宁波校区计算机学院）；School of Computer Science, University of Tokyo（东京大学计算机学院）；Department of Electrical and Computer Engineering, University of Alberta（阿尔伯塔大学电气与计算机工程系）
Context: Most approaches to automated white-box testing consider the client side and the server side of a web application in isolation from each other. Such testers lack a whole-program perspective on the web application under test. Inquiry: We hypothesise that an additional whole-program perspective would enable the tester to discover which server side errors can be triggered by an actual end user accessing the application through the client, and which ones can only be triggered in hypothetical scenarios. Approach: In this paper, we explore the idea of employing such a whole-program perspecti…
LLM-Assisted Model-Based GUI Testing for Vue.js Web Applications
Macau University of Science and Technology（澳门科技大学）；Beihang University（北京航空航天大学）；Macau University of Science and Technology Zhuhai MUST Science and Technology Research Institute（澳门科技大学珠海MUST科学研究院）；University of Nottingham Ningbo China（宁波诺丁汉大学）；Dalian Maritime University（大连海事大学）；University of Tokyo（东京大学）；University of Alberta（阿尔伯塔大学）
ResTest: Targeted Coverage of Residual Not-Covered Code Using Large Language Models for Web GUI Testing
Key Lab of HCST (PKU), MOE（北京大学）；SCS（北京大学）；Peking University, Beijing, China（中国科学院软件研究所；中国科学院大学）；Key Lab of HCST (PKU) & NERC of SE, MOE（北京通明湖信息技术应用创新中心）；Peking University, Beijing, China（复旦大学高级计算系统研究所）；Institute of Software Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China；Beijing Tongming Lake Information Technology Application Innovation Center (TLAIC), China；Fudan University Institute of Systems for Advanced Computing, China
Understanding Automated Web GUI Testing: An Empirical Study Across Exploration Strategies and State Abstractions
SCS, Peking University（北京大学计算科学研究所）；Key Lab of HCST (PKU), MOE（北京大学智能技术与系统国家重点实验室）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；National Engineering Research Center of Software Engineering, MOE（软件工程国家工程研究中心）
In the context of a large software testing company, we have deployed the model-based testing (MBT) approach to take the company's test automation practices to higher levels of maturity /and capability. We have chosen, from a set of open-source/commercial MBT tools, an open-source tool named GraphWalker, and have pragmatically used MBT for end-to-end test automation of several large web and mobile applications under test. The MBT approach has provided, so far in our project, various tangible and intangible benefits in terms of improved test coverage (number of paths tested), improved test-desig…
An Online A/B Testing Decision Support System for Web Usability Assessment Based on a Linguistic Decision-making Methodology: Case of Study a Virtual Learning Environment
To automatically test web applications, crawling-based techniques are usually adopted to mine the behavior models, explore the state spaces or detect the violated invariants of the applications. However, in existing crawlers, rules for identifying the topics of input text fields, such as login ids, passwords, emails, dates and phone numbers, have to be manually configured. Moreover, the rules for one application are very often not suitable for another. In addition, when several rules conflict and match an input text field to more than one topics, it can be difficult to determine which rule sug…
We present Quickstrom, a property-based testing system for acceptance testing of interactive applications. Using Quickstrom, programmers can specify the behaviour of web applications as properties in our testing-oriented dialect of Linear Temporal Logic (LTL) called QuickLTL, and then automatically test their application against the given specification with hundreds of automatically generated interactions. QuickLTL extends existing finite variants of LTL for the testing use-case, determining likely outcomes from partial traces whose minimum length is itself determined by the LTL formula. This …
Generation-based fuzzing is a software testing approach which is able to discover different types of bugs and vulnerabilities in software. It is, however, known to be very time consuming to design and fine tune classical fuzzers to achieve acceptable coverage, even for small-scale software systems. To address this issue, we investigate a machine learning-based approach to fuzz testing in which we outline a family of test-case generators based on Recurrent Neural Networks (RNNs) and train those on readily available datasets with a minimum of human fine tuning. The proposed generators do, in con…
