In a previous article I responded to the question Why test an app that is connected to avionics? In this second part, I will talk about a test proposal by merging OWASP  security requirements and DO-178C  safety test requirements.
Where to test what
After three “fails” looking for test methods, I asked my research director Adriana Cházaro, how could I test this App? And she came with a wonderful idea: Why don’t you include in your test suite security and safety-critical requirements.
This is explained in Figure 1, where the security requirements are tested on the front end (App) and the safety-critical on the back end (avionics). DO-178C clearly defines the safety testing requirements for avionics. but, what about the security requirements ones? This is explained below.
The fastest response to my questions
This time, I went to a bookstore here in the city (Querétaro) looking for a good Cybersecurity resource, and guess what? I found a good book: “Seguridad en aplicaciones Web Java” written by José Manuel Ortega .
The response was in the book introduction, where it explains its objectives: “to share with the read security aspects when developing Java apps in alignment to OWASP” . This may be the response I have been in search of for years. Let’s investigate on OWASP!
Tests to run on avionics apps covering cybersecurity requirements
During the OWASP investigation, I found my second Eureka, the Mobile Application Security Verification Standard (MASVS) , with the following goals:
- Provide requirements for software architects and developers seeking to develop secure mobile applications
- Offer an industry standard that can be tested against in mobile app security reviews
- Clarify the role of software protection mechanisms in mobile security and provide requirements to verify their effectiveness
- Provide specific recommendations as to what level of security is recommended for different use-cases
MASVS has three verification levels:
Resiliency Against Reverse Engineering and Tampering
|A mobile app that achieves MASVS-L1 adheres to mobile application security best practices.||This level is appropriate for applications that handle sensitive data, such as mobile banking.||MASVS-R is applicable to apps that handle highly sensitive data and may serve as a means of protecting intellectual property or tamper-proofing an app.|
As this research is focused on Airborne Systems, L2 tests were considered.
What tests can be run on avionics apps covering DO-178C requirements?
Being our “Airborne app” a safety-critical system, it is important to identify the level of testing :
|E||No effect||No software testing needed|
|D||Minor||High-Level Requirements Review |
High-Level Requirements Testing
|C||Major||Same as D, plus|
Review the Design and Code
Low-Level Requirements Testing
Data Control and Coupling
Source Statement Coverage
|B||Hazardous||Same as C, plus |
Testing on target (real) hardware Decision Condition Coverage
Independence of Code Coverage
|A||Catastrophic||Same as B, plus|
modified Decision Condition Coverage
To determine if your airborne system is A or E, you need to perform an assessment per ARP4755  where the criticality is inherited from the “parent” system and is also defined by the effect that module can have on the passengers if it fails.
- The Navigation module of a Flight Management System is considered B, and the Navigation is also considered B on the Aircraft
- In the case of an Engine (FADEC) module, it considered Level A
As this research simulates a navigation system, level B tests were considered.
An app to have fun
Once we identified what security and safety requirements should be verified, I decided to develop this very simple app to simulate data from an ILS (Instrument Landing System)  and VORs (VHF Omnidirectional Frequency)  based on measurements collected from real avionics modules from the UNAQ Lab:
The Y Axis represents the LOC (Localizer), the X the GS (Glide Slope) and the Blue BOX markers beacons from the ILS.
This app may be not as fancy as the commercial ones, and in addition to funny development times, it helps to exercise the DO-178C and OWASP test suite:
|Type of requirements||Standard||Testing level required||Requirements to covered by testing|
|Safety||DO-178C||B||Independent Testing from DevelopmentDecision Condition CoverageReview the Design and CodeHigh and Low-Level Requirements TestingData Control and CouplingSource statement coverage|
|Security||OWASP MASVS||L2||V1: Architecture, Design and Threat Modeling Requirements|
V2: Data Storage and Privacy Requirements
V3: Cryptography Requirements
V4: Authentication and Session Management Requirements
V5: Network Communication Requirements
V6: Platform Interaction Requirements
V7: Code Quality and Build Setting Requirements
V8: Resilience Requirements
In the following blog post I will explain the architecture of this app. You will get examples on how the requirements were traced to test cases and some of the tests performed.
Remark: This article is just a reference for developers and does not represent formal compliance to requirements. Formal Stage Of Involvement (SOI) with a certification authority is required to certify any avionics-related module.