notmuch/test/notmuch-test

#!/bin/bash

# Run tests
#
# Copyright (c) 2005 Junio C Hamano
#
# Adapted from a Makefile to a shell script by Carl Worth (2010)

if [ ${BASH_VERSINFO[0]} -lt 4 ]; then
    echo "Error: The notmuch test suite requires a bash version >= 4.0"
    echo "due to use of associative arrays within the test suite."
    echo "Please try again with a newer bash (or help us fix the"
    echo "test suite to be more portable). Thanks."
    exit 1
fi

cd $(dirname "$0")

TESTS="basic new search search-output search-by-folder json thread-naming raw reply dump-restore uuencode thread-order author-order from-guessing long-id encoding emacs maildir-sync"

# Clean up any results from a previous run
rm -r test-results >/dev/null 2>/dev/null

# Run the tests
for test in $TESTS; do
	./$test "$@"
done

# Report results
./aggregate-results.sh test-results/*

# Clean up
rm -r test-results corpus.mail
test: Emit a friendly error message if run with bash < 4.0. The bash code in the test suite is using associative arrays which were only added to bash as of release 4.0. If the test suite is run with an older bash, we now immediately error out and explain the situation, (instead of emitting confusing error messages and failing dozens of tests, which is what happened before this change). 2010-10-28 21:07:42 +02:00			`#!/bin/bash`
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is. 2010-01-23 19:36:39 +01:00
test: Fix test suite to integrate with our non-recursive Makefile system. This avoids "make test" emitting messages from three (3!) recursive invocations of make. We change the invocations of the tests themselves to occur directly from the shell script rather than having the shell script invoke make again and using wildcards in the Makefile. 2010-09-17 21:10:54 +02:00			`# Run tests`
			`#`
			`# Copyright (c) 2005 Junio C Hamano`
			`#`
			`# Adapted from a Makefile to a shell script by Carl Worth (2010)`

test: Emit a friendly error message if run with bash < 4.0. The bash code in the test suite is using associative arrays which were only added to bash as of release 4.0. If the test suite is run with an older bash, we now immediately error out and explain the situation, (instead of emitting confusing error messages and failing dozens of tests, which is what happened before this change). 2010-10-28 21:07:42 +02:00			`if [ ${BASH_VERSINFO[0]} -lt 4 ]; then`
			`echo "Error: The notmuch test suite requires a bash version >= 4.0"`
			`echo "due to use of associative arrays within the test suite."`
			`echo "Please try again with a newer bash (or help us fix the"`
			`echo "test suite to be more portable). Thanks."`
			`exit 1`
			`fi`

test: Fix test suite to integrate with our non-recursive Makefile system. This avoids "make test" emitting messages from three (3!) recursive invocations of make. We change the invocations of the tests themselves to occur directly from the shell script rather than having the shell script invoke make again and using wildcards in the Makefile. 2010-09-17 21:10:54 +02:00			`cd $(dirname "$0")`

test: Add new tests for folder-based searching. This is a new feature which is not implemente yet, so these tests mostly fail currently. A subsequent commit will add the feature and cause these tests to start passing. These tests verify that we can search for containing folders of mail files by word or by phrase and that the search terms are updated correctly when directories are renamed. 2011-01-15 23:05:10 +01:00			`TESTS="basic new search search-output search-by-folder json thread-naming raw reply dump-restore uuencode thread-order author-order from-guessing long-id encoding emacs maildir-sync"`
test: Rename all tests to get rid of the ugly numbers in file names. The numbers were meaningless, and they made it hard to find a file of interest. Instead, we get the ordering we want by adding an explicit list of tests to run to the notmuch-test script. 2010-09-18 00:58:39 +02:00
test: Fix test suite to integrate with our non-recursive Makefile system. This avoids "make test" emitting messages from three (3!) recursive invocations of make. We change the invocations of the tests themselves to occur directly from the shell script rather than having the shell script invoke make again and using wildcards in the Makefile. 2010-09-17 21:10:54 +02:00			`# Clean up any results from a previous run`
Revert "test: Break on test script (or other) error" This reverts commit f22a7ec1e28d1264cf9d67d78796b8ab22e09a35. Interrupting the test suite due to an actual bug in a test script would be just fine, but interrupting the run of the entire test suite at the first test failure is unacceptable. 2010-12-08 00:28:44 +01:00			`rm -r test-results >/dev/null 2>/dev/null`
test: Fix test suite to integrate with our non-recursive Makefile system. This avoids "make test" emitting messages from three (3!) recursive invocations of make. We change the invocations of the tests themselves to occur directly from the shell script rather than having the shell script invoke make again and using wildcards in the Makefile. 2010-09-17 21:10:54 +02:00
			`# Run the tests`
test: Rename all tests to get rid of the ugly numbers in file names. The numbers were meaningless, and they made it hard to find a file of interest. Instead, we get the ordering we want by adding an explicit list of tests to run to the notmuch-test script. 2010-09-18 00:58:39 +02:00			`for test in $TESTS; do`
test: Fix notmuch-test to pass command-line arguments to sub-scripts. The is useful for things like "notmuch-test --valgrind", etc. 2010-09-20 22:45:29 +02:00			`./$test "$@"`
test: Fix test suite to integrate with our non-recursive Makefile system. This avoids "make test" emitting messages from three (3!) recursive invocations of make. We change the invocations of the tests themselves to occur directly from the shell script rather than having the shell script invoke make again and using wildcards in the Makefile. 2010-09-17 21:10:54 +02:00			`done`

			`# Report results`
test: Rename all tests to get rid of the ugly numbers in file names. The numbers were meaningless, and they made it hard to find a file of interest. Instead, we get the ordering we want by adding an explicit list of tests to run to the notmuch-test script. 2010-09-18 00:58:39 +02:00			`./aggregate-results.sh test-results/*`
test: Fix test suite to integrate with our non-recursive Makefile system. This avoids "make test" emitting messages from three (3!) recursive invocations of make. We change the invocations of the tests themselves to occur directly from the shell script rather than having the shell script invoke make again and using wildcards in the Makefile. 2010-09-17 21:10:54 +02:00
			`# Clean up`
test: Fix the search and dump-restore tests to operator on non-empty mail store. We do this with a new add_email_corpus function that establishes a mail store with 50 messages from the notmuch mailing list. 2010-09-21 01:40:35 +02:00			`rm -r test-results corpus.mail`