An application of KLEE to aerospace industrial software
Juan Francisco Garc´ıa, Daniel Jurjo, Fernando Mac´ıas, Jos´e F. Morales, Alessandra Gorla
IMDEA Software Institute
June 10, 2021
IMDEA MFoC June 10, 2021 1 / 17
Motivation
Testing is a critical process in aerospace software.
Unit test generation is currently done manually, in a trial and error process.
Aerospace Software Nature
Loops are bounded.
1 f o r ( i = 0 ; i <= 2 ; i ++) {
2 . . .
3 }
4
IMDEA MFoC June 10, 2021 3 / 17
Aerospace Software Nature
The decision variables are mostly:
Boolean.
Floating point, usually normalized in a range.
1 f o r ( i = 0 ; i <= 2 ; i ++) {
2 i f ( v a l i d i t y [ i ] ){
3 i f ( s e n s o r [ i ]>t h r e s h o l d ){
4 s t a t u s [ i ]=1
5 }
6 }
7 }
8
The company approach
Build Simulink
Model
Verify using physics
simu- lations
Generate C code from the
model
Make tests inputs by
“ramp- ing”
Run the tests inputs on the model
Run tests inputs on the code
Check I/O con- formance with model
IMDEA MFoC June 10, 2021 5 / 17
The company approach
Model is assumed to be correct if it passes series of tests designed from the physical/engineering perspective.
There is a lack of specification.
The company approach
IMDEA MFoC June 10, 2021 7 / 17
Our approach
Build Simulink
Model
Verify using physics
simu- lations
Generate C code from the
model
Parse C and generate symbolic calls
Run KLEE
Get outputs
from KLEE
tests
Check I/O con- formance with model
Simulink and C code conformance
Make state symbolic, get pairs of values for state and input (S,I). It is needed to reach stateS in the model to test the input I.
IMDEA MFoC June 10, 2021 9 / 17
Simulink and C code conformance
Make state symbolic, get pairs of values for state and input (S,I).
It is needed to reach stateS in the model to test the input I.
Results
Experiment Company Approach Our approach Tests Decision
Coverage1 Tests Decision Coverage1
Exp. 1 6000 78% 54 100%
Exp. 2 34 100% 6 100%
Exp. 3 10 100% 2 100%
1Measured with Simulink Coverage Tool
IMDEA MFoC June 10, 2021 10 / 17
Benefits of our approach
An equivalent process that is automatic.
Reduce human bias. Reduce number of tests.
Assure path coverage instead of decision coverage.
Benefits of our approach
An equivalent process that is automatic.
Reduce human bias.
Reduce number of tests.
Assure path coverage instead of decision coverage.
IMDEA MFoC June 10, 2021 11 / 17
Benefits of our approach
An equivalent process that is automatic.
Reduce human bias.
Reduce number of tests.
Assure path coverage instead of decision coverage.
Benefits of our approach
An equivalent process that is automatic.
Reduce human bias.
Reduce number of tests.
Assure path coverage instead of decision coverage.
IMDEA MFoC June 10, 2021 11 / 17
Challenge: Floating Point
KLEE does not support floating point. The alternatives we have considered are:
KLEE-Float.
Fuzzing.
Combining KLEE with Fuzzers.
Work in progress: libFuzzer
Why libFuzzer?
Function oriented.
Structure-aware fuzzing. Coverage-guided.
Custom mutation and recombination function.
IMDEA MFoC June 10, 2021 13 / 17
Work in progress: libFuzzer
Why libFuzzer?
Function oriented.
Structure-aware fuzzing.
Coverage-guided.
Custom mutation and recombination function.
Work in progress: libFuzzer
Why libFuzzer?
Function oriented.
Structure-aware fuzzing.
Coverage-guided.
Custom mutation and recombination function.
IMDEA MFoC June 10, 2021 13 / 17
Work in progress: libFuzzer
Why libFuzzer?
Function oriented.
Structure-aware fuzzing.
Coverage-guided.
Custom mutation and recombination function.
Work in progress: libFuzzer
It does not generate a test for each path.
1 f o r ( i = 0 ; i <= 2 ; i ++) {
2 i f ( b o o l e a n [ i ] ){
3 . . .
4 }
5 }
6
IMDEA MFoC June 10, 2021 14 / 17
Work in progress: libFuzzer
It does not generate a test for each path.
It provides meaningful values for floating point.
1 f o r ( i = 0 ; i <= 2 ; i ++) {
2 i f ( f p v a l u e [ i ]>t h r e s h o l d ){
3 . . .
4 }
5 }
6
Work in progress: KLEE-Float
It works well on the simpler models.
It suffers of path explosion on more complex ones.
IMDEA MFoC June 10, 2021 15 / 17
Future lines
Evaluate KLEE/KLEE-Float+fuzz with different configurations.
Memory-based equivalence relation for better oracles.
Perform symbolic execution of the hardware constants.
Questions
IMDEA MFoC June 10, 2021 17 / 17
