Executable Testing / Test Data

Introduction

An executable test can be run with several wrappers, e.g. with valgrind, memcheck or a simple time-measurement. Each wrapper run can then be validated (called testers), e.g. with file-diffs or -greps. This can be configured in CMake:

AddTest(
    NAME GroundWaterFlowProcess
    PATH ${ExternalData_SOURCE_ROOT}/Elliptic/quad_20x10_GroundWaterFlow
    EXECUTABLE ogs                                                    # optional
    EXECUTABLE_ARGS quad_20x10_GroundWaterFlow.prj
    WRAPPER time                                                      # optional
    TESTER diff                                                       # optional
    DIFF_DATA quad_20x10_constMat0.mesh.vtu quad_20x10_left_right.gml # optional
)

Tests are then run with make ctest or for more verbose output with ctest -VV (you may also use other ctest options). If the checker has some errors they are displayed.

The functionality is very flexible and more wrappers and checker can be added later on. e.g. for running some statistics on output files and comparing them with statistics from reference files.

Test Data

Test data is stored with git lfs in Tests/Data. Generated test output files should be found in [build-dir]/Tests/Data.

In the OGS-cli outputting to [build-dir]/Tests/Data is already handled (via the -o parameter). For other executables you have to implement this, e.g. a with parameter specifying the output directory.

In code BaseLib::BuildInfo::data_path (from BuildInfo.h) references the data source directory and BaseLib::BuildInfo::data_binary_path references the data output directory.

For adding new data files make sure you have setup git lfs already. Then simply commit the new files as usual. For pushing you need to have setup an account on our own GitLab server as the lfs files are stored there (due to bandwidth limitations on GitHub). When asked for GitLab credentials on pushing use your GitLab account name (should be the same as the GitHub account name) and your created GitLab personal access token (see the GitLab Setup page).

Check this in-depth tutorial to learn more about git lfs.