The Diversity of Gamification Evaluation in the Software Engineering Education and Industry: Trends, Comparisons and Gaps
Rodrigo Henrique Barbosa Monteiro, Maurício Ronny de Almeida Souza, Sandro Ronaldo Bezerra Oliveira, Carlos dos Santos Portela, Cesar Elias de Cristo Lobato
备注：
43rd International Conference on Software Engineering (ICSE 2021), JSEET - Joint Track on Software Engineering Education and Training
Gamification has been used to motivate and engage participants in software engineering education and practice activities. There is a significant demand for empirical studies for the understanding of the impacts and efficacy of gamification. However, the lack of standard procedures and models for the evaluation of gamification is a challenge for the design, comparison, and report of results related to the assessment of gamification approaches and its effects. The goal of this study is to identify models and strategies for the evaluation of gamification reported in the literature. To achieve this goal, we conducted a systematic mapping study to investigate strategies for the evaluation of gamification in the context of software engineering. We selected 100 primary studies on gamification in software engineering (from 2011 to 2020). We categorized the studies regarding the presence of evaluation procedures or models for the evaluation of gamification, the purpose of the evaluation, the criteria used, the type of data, instruments, and procedures for data analysis. Our results show that 64 studies report procedures for the evaluation of gamification. However, only three studies actually propose evaluation models for gamification. We observed that the evaluation of gamification focuses on two aspects: the evaluation of the gamification strategy itself, related to the user experience and perceptions; and the evaluation of the outcomes and effects of gamification on its users and context. The most recurring criteria for the evaluation are 'engagement', 'motivation', 'satisfaction', and 'performance'. Finally, the evaluation of gamification requires a mix of subjective and objective inputs, and qualitative and quantitative data analysis approaches. Depending of the focus of the evaluation (the strategy or the outcomes), there is a predominance of a type of data and analysis.
Gamification is the use of game elements such as points, leaderboards, and badges in a non-game context to encourage a desired behavior from individuals interacting with an environment. Recently, gamification has found its way into software engineering contexts as a means to promote certain activities to practitioners. Previous studies investigated the use of gamification to promote the adoption of a variety of tools and practices, however, these studies were either performed in an educational environment or in small to medium-sized teams of developers in the industry. We performed a large-sca…
Context: Gamification is an emerging subject that has been applied in different areas, bringing contributions to different types of activities. Objective: This paper aims to characterize how gamification has been adopted in non-educational contexts of software engineering (SE) activities. Method: We performed a Systematic Mapping of the literature obtained from relevant databases of the area. The searches retrieved 2640 studies (published up to January 2020), of which 548 were duplicates, 82 were selected after applying the inclusion and exclusion criteria, and 21 were included via the backwar…
We see an explosive global labour demand in the Software Industry, and higher education institutions play a crucial role in supplying the industry with professionals with relevant education. Existing literature identifies a gap between what software engineering education teaches students and what the software industry demands. Using our open-sourced Job Market AnalyseR (JMAR) text-analysis tool, we compared keywords from higher education course syllabi and job posts to investigate the knowledge gap from a technology-focused departure point. We present a trend analysis of technology in job post…
Software engineering educators are continually challenged by rapidly evolving concepts, technologies, and industry demands. Due to the omnipresence of software in a digitalized society, higher education institutions (HEIs) have to educate the students such that they learn how to learn, and that they are equipped with a profound basic knowledge and with latest knowledge about modern software and system development. Since industry demands change constantly, HEIs are challenged in meeting such current and future demands in a timely manner. This paper analyzes the current state of practice in soft…
The teaching-learning process may require specific pedagogical approaches to establish a relationship with industry practices. Recently, some studies investigated the educators' perspectives and the undergraduate courses curriculum to identify potential weaknesses and solutions for the software testing teaching process. However, it is still unclear how the practitioners evaluate the acquisition of knowledge about software testing in undergraduate courses. This study carried out an expert survey with 68 newly graduated practitioners to determine what the industry expects from them and what they…
According to different reports, many recent software engineering graduates often face difficulties when beginning their professional careers, due to misalignment of the skills learnt in their university education with what is needed in industry. To address that need, many studies have been conducted to align software engineering education with industry needs. To synthesize that body of knowledge, we present in this paper a systematic literature review (SLR) which summarizes the findings of 33 studies in this area. By doing a meta-analysis of all those studies and using data from 12 countries a…
The Department of Homeland Security in the United States estimates that 90% of software vulnerabilities can be traced back to defects in design and software coding. The financial impact of these vulnerabilities has been shown to exceed 380 million USD in industrial control systems alone. Since software developers write software, they also introduce these vulnerabilities into the source code. However, secure coding guidelines exist to prevent software developers from writing vulnerable code. This study focuses on the human factor, the software developer, and secure coding, in particular secure …
According to a recent survey with more than 4000 software developers, less than half of developers can spot security holes. As a result, software products present a low-security quality expressed by vulnerabilities that can be exploited by cyber-criminals. This lack of quality and security is particularly dangerous if the software which contains the vulnerabilities is deployed in critical infrastructures. Serious games, and in particular, Capture-the-Flag(CTF) events, have shown promising results in improving secure coding awareness of software developers in the industry. The challenges in the…
Context: Software engineering researchers and practitioners rely on empirical evidence from the field. Thus, education of software engineers must include strong and applied education in empirical research methods. For most students, the master's thesis is the last, but also most applied form of this education in their studies. Problem: Especially thesis work in collaboration with industry requires that concerns of stakeholders from academia and practice are carefully balanced. It is possible, yet difficult to do high-impact empirical work within the timeframe of a typical thesis. In particular…
Over the last years, the software industry has adopted several DevOps technologies related to practices such as continuous integration and continuous delivery. The high demand for DevOps practitioners requires non-trivial adjustments in traditional software engineering courses and educational methodologies. This work presents an interview study with 14 DevOps educators from different universities and countries, aiming to identify the main challenges and recommendations for DevOps teaching. Our study identified 83 challenges, 185 recommendations, and several association links and conflicts betw…
Generative AI Assistants in Software Development Education: A vision for integrating Generative AI into educational practice, not instinctively defending against it
Newcastle University（纽卡斯尔大学）
针对生成式 AI 编程助手带来的行业变革，本文通过访谈从业者了解实践与挑战，提出将 GAI 融入软件开发教育的愿景及教学建议。
Software Engineering as an industry is highly diverse in terms of development methods and practices. Practitioners employ a myriad of methods and tend to further tailor them by e.g. omitting some practices or rules. This diversity in development methods poses a challenge for software engineering education, creating a gap between education and industry. General theories such as the Essence Theory of Software Engineering can help bridge this gap by presenting software engineering students with higher-level frameworks upon which to build an understanding of software engineering methods and practi…
Modern society is diverse, multicultural, and multifaceted. Because of these characteristics, we are currently observing an increase in the debates about equity, diversity, and inclusion in different areas, especially because several groups of individuals are underrepresented in many environments. In computer science and software engineering, it seems counter-intuitive that these areas, which are responsible for creating technological solutions and systems for billions of users around the world, do not reflect the diversity of the society to which it serves. In trying to solve this diversity c…
Previous research has demonstrated that preparing students for life in software engineering is not a trivial task. Authentic learning experiences are challenging to provide, and there are gaps between what students have done at the university and what they are expected to master when getting into the industry after graduation. To address this challenge, we present a novel way of teaching industry-relevant skills in a university-led internal software startup called Software Development Academy (SDA). In addition to describing the SDA concept in detail, we have investigated what educational aspe…
Context: With the rising complexity and scale of software systems, there is an ever-increasing demand for sophisticated and cost-effective software testing. To meet such a demand, there is a need for a highly-skilled software testing work-force (test engineers) in the industry. To address that need, many university educators worldwide have included software-testing education in their software engineering (SE) or computer science (CS) programs. Objective: Our objective in this paper is to summarize the body of experience and knowledge in the area of software-testing education to benefit the rea…
In industry as well as education as well as academics we see a growing need for knowledge on how to apply machine learning in software applications. With the educational programme ICT & AI at Fontys UAS we had to find an answer to the question: "How should we educate software engineers to become AI engineers?" This paper describes our educational programme, the open source tools we use, and the literature it is based on. After three years of experience, we present our lessons learned for both educational institutions and software engineers in practice.
Software architecture education remains challenging for instructors, students, and software industry professionals. Several initiatives have been proposed to mitigate the inherent challenges, including games, supporting tools, collaborative courses, and hands-on projects. Case-based learning has been introduced in software architecture, and its benefits are recognized. However, choosing the right cases that cover the stated learning objectives and developing learning activities to achieve high-order learning are also challenging. The main goal of this paper is to present a case-based learning …
