notmuch/performance-test
David Bremner a3137c61e1 perf-test: add corpus size to output, compact I/O stats
Austin suggested a while ago that the corpus size be printed in the
header. In the end it seems the corpus will be fixed per test script,
so this suggestion indeed makes sense.

The tabbing was wrapping on my usual 80 column terminal, so I joined
the input and output columns together.
2012-12-14 22:23:55 -04:00
..
download test: initial performance testing infrastructure 2012-11-26 08:39:21 -04:00
.gitignore test: initial performance testing infrastructure 2012-11-26 08:39:21 -04:00
basic test: initial performance testing infrastructure 2012-11-26 08:39:21 -04:00
Makefile test: initial performance testing infrastructure 2012-11-26 08:39:21 -04:00
Makefile.local test: initial performance testing infrastructure 2012-11-26 08:39:21 -04:00
notmuch-perf-test test: initial performance testing infrastructure 2012-11-26 08:39:21 -04:00
perf-test-lib.sh perf-test: add corpus size to output, compact I/O stats 2012-12-14 22:23:55 -04:00
README performance-test/README: update instructions for fetching corpus. 2012-11-29 09:28:03 -04:00
version.sh test: initial performance testing infrastructure 2012-11-26 08:39:21 -04:00

Pre-requisites
--------------

In addition to having notmuch, you need:

- gpg
- gnu tar
- gnu time
- xz. Some speedup can be gotten by installing "pixz", but this is
  probably only worthwhile if you are debugging the tests.

Getting set up to run tests:
----------------------------

First, you need to get the corpus.  If you don't already have the gpg
key for David Bremner, run

   % gpg --search 'david@tethera.net'

This should get you a key with fingerprint

    815B 6398 2A79 F8E7 C727  86C4 762B 57BB 7842 06AD

(the last 8 digits are printed as the "key id").

To fetch the actual corpus it should work to run

   % make download-corpus

In case that fails or is too slow, check

   http://notmuchmail.org/corpus

for a list of mirrors.

Running tests
-------------

The easiest way to run performance tests is to say "make perf-test", (or
simply run the notmuch-perf-test script). Either command will run all
available performance tests.

Alternately, you can run a specific subset of tests by simply invoking
one of the executable scripts in this directory, (such as ./basic).

Writing tests
-------------

Have a look at "basic" for an example.

add_email_corpus takes arguments "--small" and "--medium" for when you
want smaller subsets of the corpus to check.

time_done does the cleanup; comment it out or define "$debug" to leave
the temporary files around.

Currently there is no option processing (e.g. --debug) in the
performance tests.