Highly automated systems are being increasingly used in our day-to-day life. A great number of these system are also safety-critical, meaning that failures in them could result in loss of lives or damage to the environment. Examples of these systems are cars, airplanes, and health monitoring systems. Is it possible to anticipate possible errors and eliminate them to the extent that we could launch these systems into our everyday lives? The answer is -Yes! RISE is the coordinator of the VALU3S project where state-of-the-art methods and tools are used to verify and validate the safety and security of automated systems.
A need for cost effective and secure verification and validation of automated systems
Today, several different methods and tools are used in the verification and validation (V&V) of safety-critical automated systems. The V&V is often a costly and time-consuming process as these systems need to be verified and validated with respect to many requirements such as functionality, safety and security. Moreover, we see an increasing complexity and connectivity in modern systems that makes it necessary to conduct increasingly thorough testing before they are released on the market. In the end, we want these systems to be safe and secure for the end users. These issues have led to the need for developing workflows that reduce the cost of V&V processes, which are less time-consuming while still contributing to safer and more secure systems.
The VALUES Project with RISE as the coordinator
The need for more cost effective workflows of V&V processes is the reason for the creation of the VALU3S Project that brings together a consortium with partners from 10 different countries, amounting to a mix of 25 industrial partners, 6 leading research institutes, and 10 universities. In the project, 13 use cases within 6 different domains with specific safety, security and privacy requirements, will be studied in detail. With decades of knowledge and research in the field of dependable computer systems, RISE has the coordinating role of the VALU3S project.
“As we have a broad and deep knowledge as a developer of technical solutions, we also contribute to the work of an improved V&V workflow with our knowledge on “Fault and Attack Injection”, explains Behrooz Sangchoolie, researcher in the Department of Electrification and Reliability at RISE Research Institutes of Sweden.
The Fault and Attack Injection method
Fault injection is an established method used for measurement, test, and assessment of dependable computer systems. Fault and attack injection accelerate the occurrence of faults and attacks in computer systems to evaluate the system’s dependability attributes such as reliability, availability, safety&security, integrity, etc. The inclusion of fault injection as a highly recommended assessment method in the ISO 26262 standard [1] for functional safety of road vehicles demonstrates the increasing importance of experimentally validating error handling techniques in safety-critical computer systems.
To build dependable systems, some form of redundancy is used in the system to detect and recover from faults and attacks. The redundancy could be of different forms such as extra hardware, software, information or time.
“One way to validate the fault tolerance of the redundant system is to inject faults and attacks to execute and evaluate the safety and security mechanisms that ensure that the product can deliver correctly according to its specification”, says Behrooz.
Fig 1: The fault injection results could be illustrated using FIND analyser, i.e., a fault and attack injection database.
Two types of techniques investigated by RISE
Several different techniques for fault and attack injection have been studied in the past. These techniques could be classified into simulation-based and physical techniques [2]. Simulation-based techniques inject faults into hardware/software models as opposed to physical techniques which uses actual physical systems or prototypes.
“In the VALU3S project, RISE will be focusing on two types of simulation-based fault and attack injection techniques i.e., model-implemented fault and attack injection and simulation-based fault and attack injection at system-level. These types of injections are specifically useful when the hardware components of the system under test are not available” says Behrooz.
a) Model-implemented fault and attack injection
In this technique, faults and attacks are injected into MATLAB Simulink [3] models of the systems under test. These models are commonly used in e.g., the automotive and the aerospace domains. Here, the injections are performed using MODIFI [4], a model-implemented fault and attack injector developed at the unit of Dependable Transport Systems at RISE. This type of fault and attack injection is useful for early dependability evaluation of software developed as models. MODIFI facilitates the injection of hardware faults such as bit-flip and stuck-at, offset and oscillation, as well as cybersecurity attacks such as replay, jamming, and intercept.
Fig 2: MODIFI’s graphical user interface.
b) Simulation-based fault and attack injection at system-level
In this technique, faults and attacks are injected at the system-level, into various driver assistance functions modelled in the SUMO [5] and CARLA [6] simulators. SUMO is an urban mobility simulator, whereas CARLA is an open-source simulator for autonomous driving research. Examples of the functions modelled in these simulators are adaptive cruise control and lane keeping aid. The technique facilitates the evaluation of safety-critical functions in a safe environment before they are being tested on an actual vehicle in a real traffic situation.
Objective: the reduction of number of test cases
As discussed, the VALU3S project aims at the design, development and evaluation of verification and validation (V&V) methods and tools that reduce the time and cost of V&V processes with respect to requirements such as safety and cybersecurity. To this end, RISE studies the reduction of error space, also known as error space pruning [7][8].
“The error space is the representation of all faults and attacks that the system under test could face, some of which could influence the system negatively, hence violating system safety and security. Here, the error space pruning refers to elimination of the faults and attacks that, if introduced into the target system, would not influence the systems safety and security. The exclusion of these faults and attacks results in the reduction of the number of test cases that are needed when verifying and validating automated systems”, says Behrooz.
Two ways of analysing a system: before or after injection of faults and attacks
Pre-injection analysis and post-injection analysis are two techniques that will be further investigated by RISE to reduce the error space through the knowledge we obtained from the target system prior and after conducting the injections, respectively.
Pre-injection analysis contributes to the pruning of the error space through detailed study of the system under test, prior to injection of faults and attacks. Through this study, faults and attacks in parts of the system that are inherently protected could be removed from the error space.
“Since one’s knowledge of the system under test may not be complete prior to conducting injections, faults and attacks should be injected into other parts of the systems that are potentially sensitive and could violate system safety and security”, says Behrooz.
The results obtained from the injections could later be evaluated using post-injection analysis techniques, which contributes to one’s knowledge of the target system leading to additional pruning of the error space for future injections.
References
[1] “ISO 26262:2018 Road vehicles – functional safety,” ISO, Standard, 2018.
[2] P. Folkesson, S. Svensson and J. Karlsson, “A comparison of simulation based and scan chain implemented fault injection,” Digest of Papers. Twenty-Eighth Annual International Symposium on Fault-Tolerant Computing (Cat. No.98CB36224), Munich, Germany, 1998, pp. 284-293, doi: 10.1109/FTCS.1998.689479.
[3] MathWorks, “MATLAB Simulink”, https://mathworks.com/products/simulink.html, accessed: 2020-11-16.
[4] R. Svenningsson, J. Vinter, H. Eriksson, and M. Törngren, “MODIFI: A model-implemented fault injection tool,” in Proceedings of the 29th International Conference on Computer Safety, Reliability, and Security, ser. SAFECOMP’10. Berlin, Heidelberg: Springer-Verlag, 2010, pp. 210–222.
[5] Eclipse Foundation, “SUMO Urban Mobility simulator”, https://www.eclipse.org/sumo/, accessed: 2020-11-16.
[6] CARLA team, “CARLA open-source simulator for autonomous driving research”, https://carla.org/, accessed: 2020-11-16.
[7] B. Sangchoolie, K. Pattabiraman and J. Karlsson, “One Bit is (Not) Enough: An Empirical Study of the Impact of Single and Multiple Bit-Flip Errors,” 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Denver, CO, 2017, pp. 97-108, doi: 10.1109/DSN.2017.30.
[8] B. Sangchoolie, R. Johansson and J. Karlsson, “Light-Weight Techniques for Improving the Controllability and Efficiency of ISA-Level Fault Injection Tools,” 2017 IEEE 22nd Pacific Rim International Symposium on Dependable Computing (PRDC), Christchurch, 2017, pp. 68-77, doi: 10.1109/PRDC.2017.18.
