= TBC: Test by Convention =
TBC provides two features:
- It attempts to compile and run all tests, even if some do not
compile or run.
- Aspiring to the write-it-once principle, tests following
conventions require a lot less boilerplate.
Inspired by test-based development, our aim is not to displace
existing tools such as QuickCheck and HUnit but to make a project's
tests more useful when things are in an inconsistent state. TBC is
also useful for supporting 'what-if' experiments, aiding program
comprehension.
TBC is presently alpha. It has proven useful to the authors but is
embryonic in many ways.
The directory 'Sample/' contains a sample project following TBC
conventions. We suggest running 'tbc' in that directory as a quick way
of understanding what it can do.
== Conventions ==
Tests live in $PROJECT/Tests. Test-specific hierarchical modules can
also exist there.
Within a test file, lines beginning with _ should _ when evaluated:
"exception_" / throw an exception.
"hunit_" / be HUnit tests: TBC applies "runTestTT . test" to them.
"ok_" / not throw an exception.
"prop_" / be QuickCheck tests: TBC applies "Test.QuickCheck.quickCheck" to them.
"test_" / be boolean tests (of type Bool or IO Bool or IO () with a
print statement or ...): TBC runs them and expects the final line of
output to be "True".
The tests themselves must ensure the test framework (QuickCheck, etc.)
is in scope, i.e. be executable as they are in GHCi.
The 'tbc' executable will search upwards for a '.cabal' file and
assumes the 'dist/' directory is in the same place.
== Invocation ==
There are two ways to use TBC:
- as a Cabal 'test' hook. This is not recommended as packages that do
this cannot be uploaded to hackage. Perhaps changes in Cabal will
make this more useful.
- invoking the 'tbc' executable.
The 'tbc' executable will search upwards in the directory hierarchy
for a .cabal file, and then go looking for tests in the current
directory and its children.
Presently it is best to run 'tbc' in the directory containing the
tests (or a subdirectory of it).
The Sample/ directory contains a sample project. After installing TBC:
$ cd Sample
$ runghc Setup configure
$ tbc
$ cd Tests
$ tbc
== Gotchas ==
Say:
tbc -v
for a verbose session.
It seems that the parts of Cabal that TBC depends on change
frequently. TBC has only been tested with Cabal-1.10.1.0 in the
Haskell Platform 2011.2.0.1.
It is probably not difficult to make it work with other Cabals. Mostly
this involves copying out the guts of some of the Cabal modules.
Conventions must retain the full name of the test, otherwise we run
into some nasty lexical issues, e.g. to avoid:
hunit_prop_blah =tName equals= prop_blah
We only support GHCi on *NIX (including OS X) presently.
This:
$ tbc
>> GHCi died. Output <<
GHCi, version 7.0.3: http://www.haskell.org/ghc/ :? for help
TBC: fd:4: hFlush: resource vanished (Broken pipe)
probably means that you need to reconfigure your project as one of
libraries it depends on has a different hash but not a new version
number. (For example, Cabal calls QuickCheck-2.4.0.1 the
overly-specific QuickCheck-2.4.0.1-5898cfb0116fc1d6ecef04b3f053c323.)
== Changes ==
0.0.1
Initial release
0.0.2
The tbc binary exits with status 1 if anything goes wrong, 0 otherwise
Fixed GHCi scoping issue (say ":l *filename" instead of ":l filename" - perhaps new with GHC 7.
Andy Morris: support for Bird tracks
Works with the Haskell Platform 2011.2.0.1
- GHC 7.0.3
- QuickCheck 2
