Testing

We use Off-platform PC based testing.
This involves a test build of the software with the real interfaces replaced by virtual interfaces which can be accessed by Python scripts.
The test build uses the MINGW C++ compiler to build the software with virtual interfaces into a dynamic link library.
The Python library “Ctypes” allows Python access to the virtual interfaces.
Each Python script performs a series of tests on a particular function reporting its results in a HTML report.
A Python master script runs all the individual scripts reporting the summary result of each individual script in a summary HTML report which has hyperlinks to the detailed test results for each individual script.

You can view the summary HTML report for our showcase project here .

Advantages

1. The test suite can be run automatically with the click of a button yielding the reports and results in minutes (usually less than a minute).
2. There is no hardware in the loop, so the tests are repeatable, i.e. will always yield the same result from the same build.
3. The testing does not rely on hardware in the loop test equipment which is expensive with maintenance issues can itself be a source of uncertainty.

Disadvantages

There are some disadvantages and these need to be mitigated:

1. The software is tested on a different platform with a different compiler from the operational platform.
2. The actual interfaces (as distinct from the virtual interfaces) are not tested.

Mitigations

1. C++ can be ambiguous in that it can be interpreted differently by different compilers if it is badly written. This is mitigated by using a static analysis tool. Also the bit lengths of the types can vary between compilers. This is mitigated by defining a common set of types.
2. The real (as opposed to virtual) interfaces can be assumed to stay working once they are found to be working.