HOWTO handle external (blackbox) tests
======================================

Quick and dirty howto on how we do "blackbox" tests, the kind of tests
that are run against the mtn executable as a whole, and live in the
test/func/ directory.  Very early draft.  Feel free to improve.

See also the "tester.txt" and "testsuite.txt" files, which
respectively describe the basic testing framework's API, and the
monotone-specific addons.


Running tests:
--------------

-  Starting in the monotone main dir. After having './configure'd monotone you 
   can do 'make testers' to create the needed test infrastructure, such as
   the 'tester' program that runs the tests, the 'unit_tester' binary for the
   unit tests and some auxiliary scripts.

-  Execute './run_func_tests' to run the functional tests, './run_unit_tests'
   for the C++ unit tests and './run_tester_tests' for tests of the test framework
   itself. Each script accepts the following options and arguments:

    '-l'    : lists the names and numbers of all available tests in the suite
    '<n>'   : runs only test number n from the suite, negative n counts from the end
    'foobar': runs tests with "foobar" in the name (it's actually a regex)
    '-d'    : keeps the work files for post-test debugging in test/work;
              summary of test logs are saved under test/work/{func,unit,tester}.log,
              detailed test logs are available under
              test/work/{func,unit,tester}/<testname>/tester.log


Creating new functional tests:
------------------------------

-  Copy and paste is your friend :)
-  Make a new directory test/func/<testname>
-  Create a new file __driver__.lua inside this directory
-  Sometimes you need to canonicalize things


Template for a functional test:
-------------------------------

-- Initialize our workspace
mtn_setup()

-- run monotone
-- we want return value 0
-- we want to save the standard output in file "stdout"
-- we want to ignore the standard error
check(mtn("ls", "unknown"), 0, true, false)

-- we want "mtn foobar" to work, but since we know it doesn't
-- we tell the test program that this is expected to fail
xfail_if(true, mtn("foobar"), 0)


Debugging
---------

compile with -O0
run the offending command under gdb with something like gdb --args mtn ...
put a breakpoint on 'sanity::invar<TAB> (note the leading single quote)

