A/B experimentation is a known technique for data-driven product development and has demonstrated its value in web-facing businesses. With the digitalisation of the automotive industry, the focus in the industry is shifting towards software. For automotive embedded software to continuously improve, A/B experimentation is considered an important technique. However, the adoption of such a technique is not without challenge. In this paper, we present an architecture to enable A/B testing in automotive embedded software. The design addresses challenges that are unique to the automotive industry in a systematic fashion. Going from hypothesis to practice, our architecture was also applied in practice for running online experiments on a considerable scale. Furthermore, a case study approach was used to compare our proposal with state-of-practice in the automotive industry. We found our architecture design to be relevant and applicable in the efforts of adopting continuous A/B experiments in automotive embedded software.
The automotive industry has transitioned from being an electro-mechanical to a software-intensive industry. A current high-end production vehicle contains 100 million+ lines of code surpassing modern airplanes, the Large Hadron Collider, the Android OS, and Facebook's front-end software, in code size by a huge margin. Today, software companies worldwide, including Apple, Google, Huawei, Baidu, and Sony are reportedly working to bring their vehicles to the road. This paper ventures into the automotive software landscape in open source, providing the first glimpse into this multi-disciplinary in…
This paper describes first results from the AutoMoDe (Automotive Model-Based Development) project. The overall goal of the project is to develop an integrated methodology for model-based development of automotive control software, based on problem-specific design notations with an explicit formal foundation. Based on the existing AutoFOCUS framework, a tool prototype is being developed in order to illustrate and validate the key elements of our approach.
In recent years, the role and the importance of software in the automotive domain have changed dramatically. Being able to systematically evaluate and manage software quality is becoming even more crucial. In practice, however, we still find a largely static approach for measuring software quality based on a predefined list of complexity metrics with static thresholds to fulfill. We propose using a more flexible framework instead, which systematically derives measures and evaluation rules based on the goals and context of a development project.
Background: Volvo Cars is pioneering an agile transformation on a large scale in the automotive industry. Social psychological aspects of automotive software development are an under-researched area in general. Few studies on team maturity or group dynamics can be found specifically in the automotive software engineering domain. Objective: This study is intended as an initial step to fill that gap by investigating the connection between issues and problem reports and team maturity. Method: We conducted a quantitative study with 84 participants from 14 teams and qualitatively validated the resu…
In order to increase the ability to build complex, software-intensive systems, as well as to decrease time-to-market for new functionality, automotive companies aim to scale agile methods beyond individual teams. This is challenging, given the specifics of automotive systems that are often safety-critical and consist of software, hardware, and mechanical components. This paper investigates the concrete reasons for scaling agility beyond teams, the strategies that support such scaling, and foreseeable implications that such a drastic organizational change will entail. The investigation is based…
A/B testing is gaining attention in the automotive sector as a promising tool to measure causal effects from software changes. Different from the web-facing businesses, where A/B testing has been well-established, the automotive domain often suffers from limited eligible users to participate in online experiments. To address this shortcoming, we present a method for designing balanced control and treatment groups so that sound conclusions can be drawn from experiments with considerably small sample sizes. While the Balance Match Weighted method has been used in other domains such as medicine, …
Within the growing domain of software engineering in the automotive sector, the number of used tools, processes, methods and languages has increased distinctly in the past years. To be able to choose proper methods for particular development use cases, factors like the intended use, key-features and possible limitations have to be evaluated. This requires a taxonomy that aids the decision making. An analysis of the main existing taxonomies revealed two major deficiencies: the lack of the automotive focus and the limitation to particular engineering method types. To face this, a graphical taxon…
Modern cyber-physical systems (CPS) have a close inter-dependence between software and physical components. Automotive embedded systems are typical CPS, as physical chips, sensors and actuators are physical components and software embedded within are the cyber components. The current stage of embedded systems is highly complex in architecture design for both software and hardware. It is common in industrial practice that high level control algorithm development and low level code implementation on hardware platforms are developed separately with limited shared information. However, software co…
Randomised field experiments, such as A/B testing, have long been the gold standard for evaluating software changes. In the automotive domain, running randomised field experiments is not always desired, possible, or even ethical. In the face of such limitations, we develop a framework BOAT (Bayesian causal modelling for ObvservAtional Testing), utilising observational studies in combination with Bayesian causal inference, in order to understand real-world impacts from complex automotive software updates and help software development organisations arrive at causal conclusions. In this study, we…
Randomised field experiments, such as A/B testing, have long been the gold standard for evaluating the value that new software brings to customers. However, running randomised field experiments is not always desired, possible or even ethical in the development of automotive embedded software. In the face of such restrictions, we propose the use of the Bayesian propensity score matching technique for causal inference of observational studies in the automotive domain. In this paper, we present a method based on the Bayesian propensity score matching framework, applied in the unique setting of au…
As the automotive industry focuses its attention more and more towards the software functionality of vehicles, techniques to deliver new software value at a fast pace are needed. Continuous Experimentation, a practice coming from the web-based systems world, is one of such techniques. It enables researchers and developers to use real-world data to verify their hypothesis and steer the software evolution based on performances and user preferences, reducing the reliance on simulations and guesswork. Several challenges prevent the verbatim adoption of this practice on automotive cyber-physical sy…
We present models for automotive software that capture quantitative and qualitative aspects of software systems and the underlying hardware architecture. In particular, we consider different levels of computing power. These range from controllers up to the cloud. We present a modeling approach for software deployment taking different automotive requirements such as criticality, latency, memory, computational resources, and communication into account. Our models capture automotive software and hardware system configurations and can serve as digital twins that are digital counterparts of (usuall…
As the modern vehicle becomes more software-defined, it is beginning to take significant effort to avoid serious regression in software design. This is because automotive software architects rely largely upon manual review of code to spot deviations from specified design principles. Such an approach is both inefficient and prone to error. In recent days, neural language models pre-trained on source code are beginning to be used for automating a variety of programming tasks. In this work, we extend the application of such a Programming Language Model (PLM) to automate the assessment of design c…
The most important way to achieve higher performance in computer systems is through heterogeneous computing, i.e., by adopting hardware platforms containing more than one type of processor, such as CPUs, GPUs, and FPGAs. Several types of algorithms can be executed significantly faster on a heterogeneous platform. However, migrating CPU-executable software to other types of execution platforms poses a number of challenges to software engineering. Significant efforts are required in such type of migration, particularly for re-architecting and re-implementing the software. Further, optimizing it …
Over the past five years, there has been considerable growth and established value in the practice of modeling automotive software requirements. Much of this growth has been centered on requirements of software associated with the established functional areas of an automobile, such as those associated with powertrain, chassis, body, safety and infotainment. This paper makes a case for modeling four additional attributes that are increasingly important as vehicles become information conduits: security, privacy, usability, and reliability. These four attributes are important in creating specific…
The size of a software artifact influences the software quality and impacts the development process. In industry, when software size exceeds certain thresholds, memory errors accumulate and development tools might not be able to cope anymore, resulting in a lengthy program start up times, failing builds, or memory problems at unpredictable times. Thus, foreseeing critical growth in software modules meets a high demand in industrial practice. Predicting the time when the size grows to the level where maintenance is needed prevents unexpected efforts and helps to spot problematic artifacts befor…
Randomized field experiments are the gold standard for evaluating the impact of software changes on customers. In the online domain, randomization has been the main tool to ensure exchangeability. However, due to the different deployment conditions and the high dependence on the surrounding environment, designing experiments for automotive software needs to consider a higher number of restricted variables to ensure conditional exchangeability. In this paper, we show how at Volvo Cars we utilize causal graphical models to design experiments and explicitly communicate the assumptions of experime…
Functional dependencies and feature interactions in automotive software systems are a major source of erroneous and deficient behavior. To overcome these problems, many approaches exist that focus on modeling these functional dependencies in early stages of system design. However, there are only few empirical studies that report on the extent of such dependencies in industrial software systems and how they are considered in an industrial development context. In this paper, we analyze the functional architecture of a real automotive software system with the aim to assess the extent, awareness a…
In this paper, using of automotive use cases as benchmarks for real-time system design has been proposed. The use cases are described in a format supported by AMALTHEA platform, which is a model based open source development environment for automotive multi-core systems. An example of a simple Electronic Control Unit has been analysed and presented with enough details to reconstruct this system in any format. For researchers willing to use AMALTHEA file format directly, an appropriate parser has been developed and offered. An example of applying this parser and benchmark for optimising makespa…
In domains like automotive, safety-critical features are increasingly realized by software. Some features might even require fail-operational behavior, so that they must be provided even in the presence of random hardware failures. A new fault-tolerant SW/HW architecture for electric vehicles provides inherent safety capabilities that enable fail-operational features. In this paper we introduce a formal model of this architecture and an approach to calculate valid deployments of mixed-critical software-components to the execution nodes, while ensuring fail-operational behavior of certain compo…
University of Gothenburg（哥德堡大学）；RISE Research Institutes of Sweden（瑞典RISE研究院）；Zenseact AB（Zenseact公司）；Kognic AB（Kognic公司）
Software that contains machine learning algorithms is an integral part of automotive perception, for example, in driving automation systems. The development of such software, specifically the training and validation of the machine learning components, require large annotated datasets. An industry of data and annotation services has emerged to serve the development of such data-intensive automotive software components. Wide-spread difficulties to specify data and annotation needs challenge collaborations between OEMs (Original Equipment Manufacturers) and their suppliers of software components,…
Development of quality assured software-intensive systems, such as automotive embedded systems, is an increasing challenge as the complexity of these systems significantly increases. EAST-ADL is an architecture description language developed to specify automotive embedded system architectures at multiple abstraction levels in the development of safety-critical automotive products. In this paper, we propose an architecture-based verification technique which enhances the model-based development process supported by EAST-ADL by adapting model-checking to EAST-ADL specifications. We employ UPPAAL …
Automotive software development requires engineers to test their systems to detect violations of both functional and drivability requirements. Functional requirements define the functionality of the automotive software. Drivability requirements refer to the driver's perception of the interactions with the vehicle; for example, they typically require limiting the acceleration and jerk perceived by the driver within given thresholds. While functional requirements are extensively considered by the research literature, drivability requirements garner less attention. This industrial paper describes…
Intensive testing using model-based approaches is the standard way of demonstrating the correctness of automotive software. Unfortunately, state-of-the-art techniques leave a crucial and labor intensive task to the test engineer: identifying bugs in failing tests. Our contribution is a model-based classification algorithm for failing tests that assists the engineer when identifying bugs. It consists of three steps. (i) Fault localization replays the test on the model to identify the moment when the two diverge. (ii) Fault explanation then computes the reason for the divergence. The reason is a…
The use of machine learning (ML) is on the rise in many sectors of software development, and automotive software development is no different. In particular, Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) are two areas where ML plays a significant role. In automotive development, safety is a critical objective, and the emergence of standards such as ISO 26262 has helped focus industry practices to address safety in a systematic and consistent way. Unfortunately, these standards were not designed to accommodate technologies such as ML or the type of functionality t…
The required knowledge and skills that should be provided to the novice developer, designing and testing the safety critical device in automotive industry using Hardware-in-the-Loop (HiL), are presented in the paper. They should be available to the student finishing the MSc level of Electrical Engineering or Mechatronics that aims to seek the employment in the automotive industry. The development process is presented in short, together with the brief explanations of phases, which include some typical examples of written text in the documentation (requirements, test cases, etc.). The process fo…
Deep Neural Networks (DNN) will emerge as a cornerstone in automotive software engineering. However, developing systems with DNNs introduces novel challenges for safety assessments. This paper reviews the state-of-the-art in verification and validation of safety-critical systems that rely on machine learning. Furthermore, we report from a workshop series on DNNs for perception with automotive experts in Sweden, confirming that ISO 26262 largely contravenes the nature of DNNs. We recommend aerospace-to-automotive knowledge transfer and systems-based safety approaches, e.g., safety cage architec…
The goal of this technical report is to give the details of a real world existing sub-system in the automotive industry. It is produced to be used for reproduction of the same experiment if other researchers are interested in. Hence, it would be possible to compare the results of our published studies with the results of similar tools. The data is collected for the purpose of applying metaheuristic optimization approaches. The case study based on these data shows that metaheuristic optimization approaches can find efficient solutions for multiple quality attributes while fulfilling given const…
There is an increasing need for organizations to collaborate with internal and external partners on a global scale for creating software-based products and services. Many aspects and risks need to be addressed when setting up such global collaborations. Different types of collaborations such as engineering collaborations or innovation-focused collaborations need to be considered. Further aspects such as cultural and social aspects, coordination, infrastructure, organizational change process, and communication issues need to be examined. Although there are already experiences available with res…
Modeling the logical architecture is an often underestimated development step to gain an early insight into the fundamental functional properties of an automotive system. An architectural description supports developers in making design decisions for further development steps like the refinement towards a software architecture or the partition of logical functions on ECUs and buses. However, due to the large size and complexity of the system and hence the logical architecture, a good notation, method, and tooling is necessary. In this paper, we show how the logical architectures can be modeled…
Recently, an increasingly growing number of companies is focusing on achieving self-driving systems towards SAE level 3 and higher. Such systems will have much more complex capabilities than today's advanced driver assistance systems (ADAS) like adaptive cruise control and lane-keeping assistance. For complex software systems in the Web-application domain, the logical successor for Continuous Integration and Deployment (CI/CD) is known as Continuous Experimentation (CE), where product owners jointly with engineers systematically run A/B experiments on possible new features to get quantifiable …
The technological advancements of recent years have steadily increased the complexity of vehicle-internal software systems, and the ongoing development towards autonomous driving will further aggravate this situation. This is leading to a level of complexity that is pushing the limits of existing vehicle software architectures and system designs. By changing the software structure to a service-based architecture, companies in other domains successfully managed the rising complexity and created a more agile and future-oriented development process. This paper presents a case-study investigating …
The scope of automotive functions has grown from a single-vehicle as an entity to multiple vehicles working together as an entity, referred to as cooperative driving. The current automotive safety standard, ISO 26262, is designed for single vehicles. With the increasing number of cooperative driving capable vehicles on the road, it is now imperative to systematically assess the functional safety of architectures of these vehicles. Many methods are proposed to assess architectures with respect to different quality attributes in the software architecture domain, but to the best of our knowledge,…
