We use Off-platform PC based testing.
This involves a test build of the software with the real interfaces replaced by virtual interfaces which can be accessed by Python scripts.
The test build uses the MINGW C++ compiler to build the software with virtual interfaces into a dynamic link library.
The Python library “Ctypes” allows Python access to the virtual interfaces.
Each Python script performs a series of tests on a particular function reporting its results in a HTML report.
A Python master script runs all the individual scripts reporting the summary result of each individual script in a summary HTML report which has hyperlinks to the detailed test results for each individual script.
You can view the summary HTML report for our showcase project here .
- The test suite can be run automatically with the click of a button yielding the reports and results in minutes (usually less than a minute).
- There is no hardware in the loop, so the tests are repeatable, i.e. will always yield the same result from the same build.
- The testing does not rely on hardware in the loop test equipment which is expensive with maintenance issues can itself be a source of uncertainty.
There are some disadvantages and these need to be mitigated:
- The software is tested on a different platform with a different compiler from the operational platform.
- The actual interfaces (as distinct from the virtual interfaces) are not tested.
- C++ can be ambiguous in that it can be interpreted differently by different compilers if it is badly written. This is mitigated by using a static analysis tool. Also the bit lengths of the types can vary between compilers. This is mitigated by defining a common set of types.
- The real (as opposed to virtual) interfaces can be assumed to stay working once they are found to be working.