Developing Programmers .com

We all know that we should write tests for each new feature we add to a program, to ensure that the feature works correctly. In the “Extreme Programming” methodology, tests are considered vital enough that tests for new code are written before the code itself.

Proper unit testing software will make writing tests as painless as possible and will also automate the process of running your tests. This way you can easily find out whether a new change has broken any previous features, re-introduced any old bugs or generally changed the programs behaviour from what was expected. Testing eliminates a wide range of preventable mistakes from occurring (but sadly not all of them).

At work we don’t have a formal testing process so I use a trivial shell script to run a series of tests for me. Each test is set up in a different “test*” sub-directory, which contains an executable and possibly a small amount of source code to produce the executable. (I use “qmake” in my build process so compiling all the sub-directories is trivial).

Some of my test executables are stub main() functions which link to the rest of my work, calling selected functions and printing out the test results to see if they’re as expected. Other test executables are simply shell scripts that run my primary executable with particular settings and input files to see what happens.

Either way you can run a test executable on its own to observe the output, or you can run my “runtests” shell script which loops through all the tests, comparing actual output against expected output, as stored in files (using ./testfoo > testfoo.expect).

“Real” unit testing is more thorough and formal but this works for me to get some basic testing done in a low overhead workplace.

This is the source for the “runtests” script:

    #!/bin/sh
    # Unit Testing On The Cheap
    #
    # Copyright Sarah George, 2005
    #
    # This program is free software; you can redistribute it and/or modify
    # it under the terms of the GNU General Public License as published by
    # the Free Software Foundation; either version 2 of the License, or
    # (at your option) any later version.
    #
    # This program is distributed in the hope that it will be useful,
    # but WITHOUT ANY WARRANTY; without even the implied warranty of
    # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    # GNU General Public License for more details.
    #
    # The GPL license can be found at http://www.gnu.org

    abort()
    {
            echo $*
            exit 1;
    }

    TOTAL=0
    PASSED=0
    FAILED=0

    echo Test results as of `date`
    echo ""

    for tst in test*
    do
            if [ -d $tst ]
            then
                    TOTAL=$[ $TOTAL + 1 ]

                    THISPASS=1
                    MSG=""

                    (cd $tst && ./$tst > $tst.actual) || ( THISPASS=0; MSG="$tst failed to run" )
                    if [ $THISPASS == 1 ]
                    then
                            if ! diff -q $tst/$tst.expect $tst/$tst.actual > /dev/null
                            then
                                    THISPASS=0
                                    MSG="Output didn't match .expect file"
                            fi
                    fi

                    if [ $THISPASS == 1 ]
                    then
                            echo "PASS: $tst. $MSG"
                            PASSED=$[ $PASSED + 1 ]
                    else
                            echo "FAIL: $tst. $MSG"
                            FAILED=$[ $FAILED + 1 ]
                    fi
            fi
    done
    echo ""
    echo "$PASSED passed, $FAILED failed, $TOTAL total tests"
    echo ""

    if [ $FAILED != 0 ]
    then
            abort "ALERT! Not all tests passed!" 
    fi