Long-lived projects like Parabix typically develop their own testing frameworks.

Testing and Version Control

Parabix testing scripts and data files are kept under version control with Gitlab.

• Scripts and data files are called test artifacts.
• Test scripts and data are valuable and should be preserved.
• Tests can be updated and improved over time.

Parabix Unit Tests

• Custom unit tests are developed in Parabix to test bitstream manipulation features.
• For example, consider the EveryNth(bitstream, n) operation.
  • Goal is to filter the 1 bits of a stream, selecting only every nth such bit.
  • EveryNth unit tests
  • Parabix unit test framework: include lib

Parabix Integration Tests

Many Parabix applications have end-to-end integration tests to demonstrate the full functionality.

• The icgrep application uses the greptest.py script, together with test data in the greptest.xml file.
  • The greptest.py script actually is capable of automatically making additional versions of tests using the --tests_per_grepcase and --random_flag_count options.
  • Most test cases are functional in nature to show the specific matched lines for a given regular expression.
  • Tests for all kinds of regular expression features are used.
  • Some tests also deal with important boundary conditions.
    • Files which are exactly 4096 bytes (the operating system page size), for testing buffer overflow issues.
    • Files which have no line terminator on the last line.
• The Parabix XML tests use the XML Conformance Test Suite
  • In general, seeking out standards-compliant test suites is important to develop interoperable software.

Parabix Regression Testing

• Parabix testing is set up using the CMake ctest system.
• The entire set of tests is rerun with each make check operation.
• The top-level CMakeLists.txt file implements the logic of Parabix regression testing.
• In general, regression testing is very valuable to ensure that software contineus to pass tests!