HOWTO handle external (blackbox) tests
======================================

Quick and dirty howto on how we do "blackbox" tests, the kind of tests
that are run against the mtn executable as a whole, and live in the
tests/ directory.  Very early draft.  Feel free to improve.

See also the "tester.txt" and "testsuite.txt" files, which
respectively describe the basic testing framework's API, and the
monotone-specific addons.

Running tests:
--------------
-  Starting in the monotone main dir. After having './configure'd monotone you 
   can do 'make tester' to create 'tester', the program that runs the tests.
   Running './testsuite.lua' (or './tester testsuite.lua') will run all tests.
-  'testsuite.lua -l' lists the names and numbers of all available tests
-  'testsuite.lua <n>' runs only test number n, negative n counts from the end
-  'testsuite.lua foobar' runs tests with "foobar" in the name
   (it's actually a regex)
-  option -d will keep the tester_dir files for post-test debugging
-  option -h is your friend :)
-  summary of test logs in tester_dir/tester.log
-  details of test logs in tester_dir/<testname>/tester.log

- './run_tester_tests --help' describes how to run the tester progam's tests
- './run_unit_tests   --help' describes how to run the c++ unit tests 
- './run_lua_tests    --help' describes how to run the lua test suite

Creating tests:
---------------
-  Copy and paste is your friend :)
-  Make a new directory tests/<testname>
-  Add a line for <testname> at the end of testsuite.lua
-  TODO: need more here...
-  Sometimes you need to canonicalize things

Template for a test (name tests/<testname>/__driver__.lua:
------------------------------------------------

-- Initialize our workspace
mtn_setup()

-- run monotone
-- we want return value 0
-- we want to save the standard output in file "stdout"
-- we want to ignore the standard error
check(mtn("ls", "unknown"), 0, true, false)

-- we want "mtn foobar" to work, but since we know it doesn't
-- we tell the test program that this is expected to fail
xfail_if(true, mtn("foobar"), 0)

Debugging
---------

compile with -O0
run the offending command under gdb with something like gdb --args mtn ...
put a breakpoint on 'sanity::invar<TAB> (note the leading single quote)

