Commit graph

1880 commits

Author SHA1 Message Date
Carl Worth
b9eac48c22 notmuch: Add a new "notmuch config" command for querying configuration.
So far, we implement only "notmuch config get". It won't be too much
work from here to also implement "notmuch config set".
2010-10-27 13:02:17 -07:00
Carl Worth
6c65fcd9aa TODO: Note idea for a new "notmuch compose"
Which would also allow the recently added test of sending an email
message with the emacs interface to be a little more honest about the
From address.
2010-10-27 10:48:40 -07:00
Carl Worth
102c57c825 test: Add test that emacs interface actually sends mail.
Rather than *reall* sending mail here, we instead have a new test
program, smtp-dummy which implements (a small piece of) the
server-side SMTP protocol and saves a mail message to the filename
provided. This gives us reasonable test coverage of a large chunk of
the notmuch+emacs code base (down to talking to an SMTP server with
the final mail contents).
2010-10-27 10:42:46 -07:00
Carl Worth
f30200a429 test: Set alternate HOME during tests.
We set the HOME environment variable to the test directory to avoid
the tests relying on any configuration files from the test author's
own home directory, (such as ${HOME}/.emacs or similar).
2010-10-27 10:40:44 -07:00
Carl Worth
fad0c3b00b test: Fix false failure from the "available tests" test.
We recently added a new sub-directory below test, so we have to
blacklist it explicitly in this test.
2010-10-22 18:03:39 -07:00
Carl Worth
14a2227f77 notmuch search: Fix to handle failure of notmuch_query_search_threads.
Now that notmuch_query_search_threads can return NULL, (for example,
due to a Xapian exception), we need to handle that case (rather than
just segfault). It's simple enough to just return a non-zero exit
code.
2010-10-22 18:03:39 -07:00
Carl Worth
f6cb896bc4 lib: Fix notmuch_query_search_threads to return NULL on any Xapian exception.
Previously, if the underlying search_messages hit an exception and returned
NULL, this function would ignore that and return a non-NULL, (but empty)
threads object. Fix this to properly propagate the error.
2010-10-22 17:56:58 -07:00
Carl Worth
4606ea60a7 emacs: Fix quoting of Message-Id to fix test case of Id containing ".."
If Xapian sees unquoted ".." as in id:123..456 then it thinks that's a
range specification. We avoid this problem by instead passing
id:"123..456" to Xapian.
2010-10-22 17:41:40 -07:00
Carl Worth
4c998189b7 test: Add test demonstrating failure in emacs interface when Message-Id has ..
Thanks to Jameson Rollins for pointing out this bug
(id:87y6g7zr6q.fsf@servo.finestructure.net).
2010-10-22 17:38:38 -07:00
Carl Worth
a9849caadf test: Add tests for adding/removing tags within emacs interface
Exercising both the notmuch-search and notmuch-show views.
2010-10-22 17:17:00 -07:00
Carl Worth
a8aa437bbc test: Add simple tests for navigating notmuch-hello and notmuch-search views
We simulate the act of selecting the "inbox" saved search from
notmuch-hello and the act of selecting a desired thread from the
notmuch-search results.

The test for the navigation of notmuch-hello is currently marked as
BROKEN since its output is in the opposite order compared to the
'(notmuch-search "tag:inbox")' test. This question of ordering is a
currently open issue on the notmuch mailing list, so we'll let the
test suite reflect that for now.

Finally, this commit also abstracts some common emacs lisp code,
(waiting for the current buffer's process to complete), into a new
notmuch-test-wait function that is made available to anything calling
test_emacs.
2010-10-22 17:02:07 -07:00
Carl Worth
6307f306fd test: Add a new test_expect_equal_failure
Which allows us to have a known-broken test that would otherwise use
test_expect_equal.
2010-10-22 17:01:34 -07:00
Carl Worth
4bb1b1603c test: Add tests for emacs notmuch-search and notmuch-show functions.
Moving the expected output into individual files (rather than inline)
to keep the test script much easier to read.
2010-10-22 16:58:35 -07:00
Carl Worth
ca956552bd emacs: Remove the joke from the first line of the notmuch-hello view.
Overuse just makes the joke unfunny.
2010-10-22 12:12:22 -07:00
Carl Worth
1375d71ff4 test: Add the most rudimentary testing of the emacs interface.
So far, this is doing nothing more than adding a corpus of email and
ensuring that the `notmuch-hello' function produces the desired
output.
2010-10-22 12:10:15 -07:00
Carl Worth
209e756cd4 test: Add a new test_emacs function to test-lib.sh
This should be quite handy for doing automated testing of the
emacs-based functionality in notmuch. This function invokes emacs with
the necessary command-line arguments, (to run in batch mode with no
local initialization, to load the notmuch code from the source
directory, and to ensure an 80-column width).
2010-10-22 12:09:56 -07:00
Carl Worth
b97a763dc1 test: Fix add_email_corpus function to be quiet.
This simply avoids some unneeded noise in the "make test" output.
2010-10-22 12:04:41 -07:00
Carl Worth
1a17faf48d emacs: Fix notmuch-hello to not break when given a very narrow window.
Simply ensure that some subtractions never result in a negative
number, (since emacs complains when asked to create a string with a
negative length).
2010-10-22 12:03:34 -07:00
Carl Worth
f117d80d9b TODO: Add some notes to fix some recently noted problems.
Just don't want to forget about things that people have pointed out.
2010-10-12 17:40:17 -07:00
Carl Worth
31a5e5a125 emacs: Fix bug when parsing a subject cotaining: \[[0-9/]\]
That is, a subject with a bracketed set of digits (and optionally a
slash), for example "[2010]" would cause the emacs code to misparse
the search results. Fix this by tweaking the regular expression.
2010-09-23 13:21:03 -07:00
Carl Worth
8071c5cd64 lib: Fix "make install"
This has been broken since the addition of the test sub-directory to our
non-recursive make system.
2010-09-21 09:09:01 -07:00
Carl Worth
9c7fd320c9 test/README: Document add_email_corpus (and add_message/generate_message)
While adding the documentation here for add_email_corpus I noticed
that the other email-adding functions in test-lib.sh were not yet
documented here, so add all of that documentation.
2010-09-20 16:41:31 -07:00
Carl Worth
5497b01c27 test: Fix the search and dump-restore tests to operator on non-empty mail store.
We do this with a new add_email_corpus function that establishes a
mail store with 50 messages from the notmuch mailing list.
2010-09-20 16:40:35 -07:00
Carl Worth
d805866ec5 test: Add a corpus of email messages to be used in testing.
This is simply 50 messages from the early history of the notmuch mailing
list, (fetched from the public archives).
2010-09-20 16:37:53 -07:00
Carl Worth
ba9f9efc9a test: Remove useless NOTMUCH variable (in favor of simply "notmuch")
When the NOTMUCH variable was originally invented it was used as an
explicit path to the notmuch binary being tested. Today, the test
suite sets the PATH variable instead, so the NOTMUCH variable always
has a value of simply "notmuch".

We simplifying that by using the constant value rather than the
continual variable reference.
2010-09-20 16:15:08 -07:00
Carl Worth
129a4417e3 test: Remove some dead code in test-lib.sh
These assignements weren't being used at all and were just confusing me,
(the real assignments happen later on in the file).
2010-09-20 16:09:20 -07:00
Carl Worth
2b3a219bc9 test: Fix PATH-checking test to work with --valgrind
The --valgrind option munges the PATH variable, so un-munge it before
testing that we have PATH pointing to the source directory.
2010-09-20 15:07:26 -07:00
Carl Worth
029830c1f1 lib: Fix use-after-free bug.
Thanks to the new git-based test suite, it's easy to run the whole
test suite in valgrind, (simply "make test OPTIONS="--valgrind"), and
doing so showed this obvious use-after-free bug, (triggered by the
thread-order tests).
2010-09-20 22:01:52 +00:00
Carl Worth
341e2bd86e test: Fix to actually report errors (!).
A bug in the results-aggregation code was causing the test suite to report
"all tests passed" even when there were failures, (as long as there were
also no "broken" tests). Fix this.
2010-09-20 14:39:40 -07:00
Carl Worth
49a8cb9de3 test: Avoid printing "Testing Testing ..."
The test suite isn't auditioning for a roadie position, after all.
2010-09-20 14:39:06 -07:00
Carl Worth
4813ee41d6 test: Print section names, and rename all test sections
Now that we can usefully pass section names via the NOTMUCH_SKIP_TESTS
environment variable, it's useful to actually print those names out
for the user. Then, since we're now printing these names, let's use
nicer names, (not excessively long but also not using abbreviations
like "msg").
2010-09-20 14:38:56 -07:00
Carl Worth
169639e606 test: Make the --valgrind option useful, (and drop --verbose).
In order for --valgrind to be useful, we drop noisy additional output of
all of the commands being executed in verbose mode. This makes --verbose
alone quite useless, so we don't document it any more.

Also, add a zlib valgrind suppression that was showing up frequently in the
test suite.
2010-09-20 14:28:13 -07:00
Carl Worth
029a105da3 test/README: Update to become notmuch-specific rather than git-specific
This file was obviously describing the git test suite previously, and
would have been very hard to understand in the context of the notmuch
test suite. HOpefully it's easier to follow now.
2010-09-20 14:06:38 -07:00
Carl Worth
16a6301ade test: Rename GIT_SKIP_TESTS to NOTMUCH_SKIP_TESTS
By scanning test-lib.sh for occurrences of "git" or "GIT", I found
that most of those are internal things, (like the GIT_TEST_TEE_STARTED
variable). But GIT_SKIP_TESTS is part of the user-interface to the
test suite, so we rename it to reference notmuch rather than git.

Also, the GIT_TRACE warning is git-specific, so we drop that as well.
2010-09-20 13:54:57 -07:00
Carl Worth
8369ed31d0 test: Fix notmuch-test to pass command-line arguments to sub-scripts.
The is useful for things like "notmuch-test --valgrind", etc.
2010-09-20 13:45:29 -07:00
Carl Worth
60c599036e test: Fix test suite so that --valgrind option works.
The output is ugly, and we need a better suppressions file, but this
is at least a start.
2010-09-20 13:44:32 -07:00
Carl Worth
265de5006f test: Add test to ensure that all available test scripts are run
Since we are now using an explicit list of tests to run in
notmuch-test we need to be careful that we don't add a new file of
tests and then forget to add it to the list.
2010-09-20 12:41:10 -07:00
Carl Worth
7263aa9ec3 test: Rename all tests to get rid of the ugly numbers in file names.
The numbers were meaningless, and they made it hard to find a file of interest.

Instead, we get the ordering we want by adding an explicit list of
tests to run to the notmuch-test script.
2010-09-17 16:01:42 -07:00
Carl Worth
51cd69feb1 test: Remove basic testing of broken, fixed, and skipped tests.
These were interfering with the aggregate statistics reported at the
end of the test-suite run. (Always reporting 1 broken, 1 fixed, and 1
skipped). The correct way to test the test-suite itself would be to
run the test suite externally for these cases, capture the expected
result, and then report that as a PASS test.

But, really, there's almost no value in these tests anyway. It's
almost to the level of testing that 'if false; exit 1; fi' returns
1. That is, there are so many ways that the test suite could be broken
internally, that these minor tests don't really help.
2010-09-17 15:30:17 -07:00
Carl Worth
82155c3d96 test: Fix ordering of tests in t0000-basic.sh
The basic notmuch tests depend on the test-suite harness working, so
test it first.
2010-09-17 15:28:53 -07:00
Carl Worth
c92ad8bf6a test: Rework test-suite input to avoid ulti-command strings
The original git test suite works by concatenating many commands into
a very long string (each separated by &&). This is painful to work
with since it prevents the editor from helping by parsing the shell
script, indenting, colorizing, etc.

Instead, we switch this back to something like the original notmuch
test suite, and add two new functions to test-lib.sh
(test_begin_subtest and test_expect_equal) to support these.

This also fixes the test suite to once again display the diff when a
test fails to generate the expected input.
2010-09-17 15:25:39 -07:00
Carl Worth
222926abe1 test: Cleanup the test output
This makes the new, git-derived test suite report results in a manner
similar to the original notmuch test suite.

Notable changes include:

  * No more initial '*' on every line
  * Only colorize a single word
  * Don't print useless test numbers
  * Use "PASS" in place of "ok"
  * Begin sentences with a capital letter
  * Print test descriptions for each block
  * Separate each block of tests with a blank line
  * Don't summarize counts between each block
2010-09-17 14:08:36 -07:00
Carl Worth
e6ba2c63c9 test: Fix test suite to integrate with our non-recursive Makefile system.
This avoids "make test" emitting messages from three (3!) recursive
invocations of make. We change the invocations of the tests themselves
to occur directly from the shell script rather than having the shell
script invoke make again and using wildcards in the Makefile.
2010-09-17 12:16:10 -07:00
Michal Sojka
7a72999fad test: Set all times to UTC
In order to have repeatable test suite, all times in messages are set
to UTC time zone to match the time zone (TZ variable) set in
test-lib.sh.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2010-09-16 15:56:44 -07:00
Michal Sojka
5398e6966a Convert the actual tests to the new framework
The changes are:
- The notmuch-test was split into several files (t000?-*.sh).
- Removed helper functions which were moved to test-lib.sh
- Replaced every printf with test_expect_success.
- Test commands chained with && (test-lib.sh doesn't use "set -e" in
  order to complete the test suite even if something fails)
- Many variables such as ${MAIL_DIR} were properly quoted as they
  contain spaces.
- Changed quoting patterns in add_message and generate_message (single
  quotes are already used by the test framework).
- ${TEST_DIR} replaced by ${PWD}

QUICK HOWTO:

To run the whole test suite
    make

To run only a single test
   ./t0001-new.sh

To stop on the first error
   ./t0001-new.sh -i
then mail store and database can be inspected in
"trash directory.t0001-new"

To see the output of tests
   ./t0001-new.sh -v

To not remove trash directory at the end:
   ./t0001-new.sh -d

To run all tests verbosely:
   make GIT_TEST_OPTS="-v"

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2010-09-16 15:56:44 -07:00
Michal Sojka
04d454f582 test: Update helper functions
Modify the helper functions to work with git-based test suite i.e.
1) Quote arguments where it is necessary.
2) Do not use $NOTMUCH. It is equal to "notmuch" since $PATH is set to
   the build tree.
3) Modify pass_if_equal to fit into the git-based test suite.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2010-09-16 15:56:44 -07:00
Michal Sojka
223987bace Update test framework for use with notmuch
This removes Git specific things from the test-lib.sh and adds helper
functions for notmuch taken from Carl's notmuch-test script. README is
also slightly modified to reflect the current state.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2010-09-16 15:56:44 -07:00
Michal Sojka
0083854b12 Copy test framework from Git
Git uses a simple and yet powerful test framework, written in shell.
The framework is easy to use for both users and developers so I think
it would help if it is used in notmuch as well.

This is a copy of Git's test framework from commit
b6b0afdc30e066788592ca07c9a6c6936c68cc11 in git repository.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2010-09-16 15:56:44 -07:00
Carl Worth
4e77148a4b emacs: Allow '|' to operate on multiple messages (by means of prefix argument).
We extend the '|' command so that passing a prefix argument, (for
example, "C-u |"), causes it to pipe all open messages in the current
thread rather than just the single, current message.
2010-09-16 15:52:12 -07:00
Carl Worth
9f0accb6aa notmuch show: Add documentation of --format=mbox
Shame on me for adding a feature without documenting it at the same
time.
2010-09-16 14:47:32 -07:00