notmuch/performance-test/README

Pre-requisites
--------------

In addition to having notmuch, you need:

- gpg
- gnu tar
- gnu time
- xz. Some speedup can be gotten by installing "pixz", but this is
  probably only worthwhile if you are debugging the tests.

Getting set up to run tests:
----------------------------

First, you need to get the corpus.  If you don't already have the gpg
key for David Bremner, run

   % gpg --search 'david@tethera.net'

This should get you a key with fingerprint

    815B 6398 2A79 F8E7 C727  86C4 762B 57BB 7842 06AD

(the last 8 digits are printed as the "key id").

To fetch the actual corpus it should work to run

   % make download-corpus

In case that fails or is too slow, check

   http://notmuchmail.org/corpus

for a list of mirrors.

Running tests
-------------

The easiest way to run performance tests is to say "make time-test", (or
simply run the notmuch-time-test script). Either command will run all
available performance tests.

Alternately, you can run a specific subset of tests by simply invoking
one of the executable scripts in this directory, (such as ./basic).
Each test script supports the following arguments

--small / --medium / --large	Choose corpus size.
--debug				Enable debugging. In particular don't delete
				temporary directories.

Writing tests
-------------

Have a look at "T01-dump-restore" for an example. Sourcing
"perf-test-lib.sh" is mandatory.  Utility functions include

- 'add_email_corpus' unpacks a set of messages and adds them to the database.
- 'cache_database': makes a snapshot of the current database
- 'uncache_database': forces the next 'add_email_corpus' to rebuild the
  database.
- 'time_start' unpacks the mail corpus and calls notmuch new if it
   cannot find a cache of the appropriate corpus.
- 'time_done' does the cleanup; comment it out or pass --debug to the
  script to leave the temporary files around.

Scripts are run in the order specified in notmuch-perf-test. In the
future this order might be chosen automatically so please follow the
convention of starting the name with 'T' followed by two digits to
specify the order.