debian/notmuch
1
0
Fork 0
mirror of https://git.notmuchmail.org/git/notmuch synced 2024-12-01 07:04:10 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
notmuch/performance-test
History
David Bremner f873790b6f perf-test: update corpus signature 
In the decade (!) since this corpus was last updated, the keyserver
network is essentially dead, and I have migrated gpg keys. Bump the
version number as a clean way of switching signatures. Also update the
instructions to suggest using "--locate-external-key" to download the
public key. By default this uses WKD, which is now supported for my
UID.
2023-07-09 12:14:29 -03:00
..
download perf-test: update corpus signature 2023-07-09 12:14:29 -03:00
.gitignore Use rooted paths in .gitignore files 2017-08-18 19:42:35 -03:00
M00-new.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
M01-dump-restore.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
M02-show.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
M03-search.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
M04-reply.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
M05-reindex.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
M06-insert.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
Makefile test: initial performance testing infrastructure 2012-11-26 08:39:21 -04:00
Makefile.local emacs: Use makefile-gmake-mode in Makefile*s 2020-08-09 21:14:36 -03:00
notmuch-memory-test test: copyright information updates 2016-05-19 07:56:58 -03:00
notmuch-time-test test: copyright information updates 2016-05-19 07:56:58 -03:00
perf-test-lib.sh perf-test: allow running test_emacs from performance test suite. 2022-02-25 08:38:03 -04:00
README perf-test: update corpus signature 2023-07-09 12:14:29 -03:00
T00-new.sh perf-test/T00-new: fix backup creation. 2021-05-19 08:54:44 -03:00
T01-dump-restore.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
T02-tag.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
T03-reindex.sh perf-test: add test for reindexing after removal/renaming msgs 2021-05-23 08:22:31 -03:00
T04-thread-subquery.sh perf-tests: #!/bin/bash -> #!/usr/bin/env bash 2019-06-07 07:59:56 -03:00
T05-ruby.sh perf-test: add ruby test 2021-07-18 16:50:27 -03:00
T06-emacs.sh perf-test/emacs: test showing longish threads 2022-07-14 04:50:33 -03:00
T07-git.sh perf-test: add tests notmuch-git 2022-07-16 20:27:40 -04:00
version.sh perf-test: update corpus signature 2023-07-09 12:14:29 -03:00

README 

Performance Tests
-----------------

This directory contains two kinds of performance tests: time tests,
and memory tests. The former use gnu time, and the latter use
valgrind.

Pre-requisites
--------------

In addition to having notmuch, you need:

- gpg
- gnu tar
- gnu time (for the time tests)
- xz. Some speedup can be gotten by installing "pixz", but this is
  probably only worthwhile if you are debugging the tests.
- valgrind (for the memory tests)
- perf (optional, for more fine-grained timing)

Getting set up to run tests:
----------------------------

First, you need to get the corpus.  If you don't already have the gpg
key for David Bremner, run

   % gpg --locate-external-key 'david@tethera.net'

This should get you a key with fingerprint

    7A18 807F 100A 4570 C596  8420 7E4E 65C8 720B 706B

(the last 8 digits are printed as the "key id").

To fetch the actual corpus it should work to run

   % make download-corpus

In case that fails or is too slow, check

   https://notmuchmail.org/corpus

for a list of mirrors.

Running tests
-------------

The easiest way to run performance tests is to say "make perf-test".
This will run all time and memory tests.  Be aware that the memory
tests are quite time consuming when run on the full corpus, and that
depending on your interests it may be more sensible to run "make
time-test" or "make memory-test".  You can also invoke one of the
scripts notmuch-time-test or notmuch-memory-test or run a more
specific subset of tests by simply invoking one of the executable
scripts in this directory, (such as ./T00-new).  Each test script
supports the following arguments

--small / --medium / --large	Choose corpus size.
--debug				Enable debugging. In particular don't delete
                                temporary directories.
--perf                          Run perf record in place of /usr/bin/time. Perf output can be
                                found in a log directory.
--call-graph {fp,lbr,dwarf}     Call graph option for perf record. Default is 'lbr'.

When using the make targets, you can pass arguments to all test
scripts by defining the make variable OPTIONS.

Log Directory
-------------

The memory tests, and the time tests when option '--perf' is given
save their output in a directory named as follows

     log.$test_name-$corpus_size-$timestamp

These directories are removed by "make clean".

Writing tests
-------------

Have a look at "T01-dump-restore" for an example time test and
"M00-new" for an example memory test. In both cases sourcing
"perf-test-lib.sh" is mandatory.

Basics:

- '(time|memory)_start' unpacks the mail corpus and calls notmuch new if it
   cannot find a cache of the appropriate corpus.
- '(time|memory)_run' runs the command under time or valgrind. Currently
  "memory_run" does not support i/o redirection in the command.
- '(time|memory)_done' does the cleanup; comment it out or pass --debug to the
  script to leave the temporary files around.

Utility functions include

- 'add_email_corpus' unpacks a set of messages and tags
- 'cache_database': makes a snapshot of the current database
- 'uncache_database': forces the next '(time|memory)_start' to rebuild the
  database.

Scripts are run in the order specified in notmuch-perf-test. In the
future this order might be chosen automatically so please follow the
convention of starting the name with 'T' or 'M' followed by two digits
to specify the order.