debian/notmuch
1
0
Fork 0
mirror of https://git.notmuchmail.org/git/notmuch synced 2024-12-27 20:01:42 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
notmuch/test/notmuch-test

1115 lines
44 KiB
Text
Raw Normal View History

notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
#!/bin/bash
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
set -e
test: Comment why we need to set TZ
2010-04-26 12:12:13 +02:00
# Messages contain time/date values with time zone and notmuch
# displays them converted to the local time zone. We need to set fixed
# timezone here so that the output of the tests is always the same
# without regard to the time zone of where the test suite is run.
test: Set fixed time zone When the test suite is run in a different time zone that where Carl lives, some tests may fail depending on the time when the test suite is run. For example, just now I get: Search for all messages ("*"):... FAIL --- test-031.expected 2010-04-23 09:33:47.898634822 +0200 +++ test-031.output 2010-04-23 09:33:47.898634822 +0200 @@ -1,5 +1,5 @@ -thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Test message #6 (inbox unread) -thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Test message #14 (inbox unread) +thread:XXX 2001-01-06 [1/1] Notmuch Test Suite; Test message #6 (inbox unread) +thread:XXX 2001-01-06 [1/1] Notmuch Test Suite; Test message #14 (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; body search (inbox unread) thread:XXX 2000-01-01 [1/1] searchbyfrom; search by from (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (inbox unread) By setting a fixed time zone in the test script, these problems should be eliminated. Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2010-04-23 09:36:45 +02:00
export TZ=UTC+8
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
find_notmuch_binary ()
{
dir=$1
while [ -n "$dir" ]; do
bin=$dir/notmuch
if [ -x $bin ]; then
echo $bin
return
fi
dir=$(dirname $dir)
if [ "$dir" = "/" ]; then
break
fi
done
echo notmuch
}
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime_amount=0
increment_mtime ()
{
dir=$1
increment_mtime_amount=$((increment_mtime_amount + 1))
touch -d "+${increment_mtime_amount} seconds" $dir
}
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
# Generate a new message in the mail directory, with a unique message
# ID and subject. The message is not added to the index.
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
#
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
# After this function returns, the filename of the generated message
# is available as $gen_msg_filename and the message ID is available as
# $gen_msg_id .
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
#
# This function supports named parameters with the bash syntax for
# assigning a value to an associative array ([name]=value). The
# supported parameters are:
#
# [dir]=directory/of/choice
#
# Generate the message in directory 'directory/of/choice' within
# the mail store. The directory will be created if necessary.
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
#
# [body]=text
#
# Text to use as the body of the email message
#
# '[from]="Some User <user@example.com>"'
# '[to]="Some User <user@example.com>"'
# '[subject]="Subject of email message"'
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
# '[date]="RFC 822 Date"'
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
#
# Values for email headers. If not provided, default values will
# be generated instead.
#
# '[cc]="Some User <user@example.com>"'
notmuch-test: Add "notmuch reply" test for reply-to support. This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
2010-02-04 20:52:08 +01:00
# [reply-to]=some-address
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
# [in-reply-to]=<message-id>
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
# '[header]=full header line, including keyword'
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
#
# Additional values for email headers. If these are not provided
# then the relevant headers will simply not appear in the
# message.
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
#
# '[id]=<message-id>'
#
# Controls the message-id of the created message.
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
gen_msg_cnt=0
gen_msg_filename=""
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
gen_msg_id=""
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
generate_message ()
{
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
# This is our (bash-specific) magic for doing named parameters
local -A template="($@)"
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
local additional_headers
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
if [ -z "${template[id]}" ]; then
gen_msg_cnt=$((gen_msg_cnt + 1))
gen_msg_name=msg-$(printf "%03d" $gen_msg_cnt)
gen_msg_id="${gen_msg_name}@notmuch-test-suite"
else
gen_msg_name="msg-${template[id]}"
gen_msg_id="${template[id]}"
fi
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
if [ -z "${template[dir]}" ]; then
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
gen_msg_filename="${MAIL_DIR}/$gen_msg_name"
else
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
gen_msg_filename="${MAIL_DIR}/${template[dir]}/$gen_msg_name"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
mkdir -p $(dirname $gen_msg_filename)
fi
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
if [ -z "${template[body]}" ]; then
test: Improve grammar in test-message subjects. Before codifying these in any expected test output, I'd like them to at least resemble English.
2010-04-22 22:21:15 +02:00
template[body]="This is just a test message (#${gen_msg_cnt})"
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
fi
if [ -z "${template[from]}" ]; then
template[from]="Notmuch Test Suite <test_suite@notmuchmail.org>"
fi
if [ -z "${template[to]}" ]; then
template[to]="Notmuch Test Suite <test_suite@notmuchmail.org>"
fi
if [ -z "${template[subject]}" ]; then
test: Remove test-message filenames from generated messages The filenames aren't predictable (including the current directory) nor stable from one run to the next (including the PID). This makes it hard to predict the output from a search command that returns such a message (such as "*"). The original goal was simply to ensure that each generated message was distinguishable somehow. So just use the message counter instead.
2010-04-15 23:01:54 +02:00
template[subject]="Test message #${gen_msg_cnt}"
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
fi
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
if [ -z "${template[date]}" ]; then
test: Use an older date for the generated messages. Using a date in the current year makes the test suite fragile since the search output will include a date of "January 05" for now, but will start doing "2010-01-05" in the future.
2010-04-15 23:09:07 +02:00
template[date]="Tue, 05 Jan 2001 15:43:57 -0800"
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
fi
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
additional_headers=""
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
if [ ! -z "${template[header]}" ]; then
additional_headers="${template[header]}
${additional_headers}"
fi
notmuch-test: Add "notmuch reply" test for reply-to support. This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
2010-02-04 20:52:08 +01:00
if [ ! -z "${template[reply-to]}" ]; then
additional_headers="Reply-To: ${template[reply-to]}
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
${additional_headers}"
fi
if [ ! -z "${template[in-reply-to]}" ]; then
additional_headers="In-Reply-To: ${template[in-reply-to]}
${additional_headers}"
fi
notmuch-test: Add "notmuch reply" test for reply-to support. This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
2010-02-04 20:52:08 +01:00
if [ ! -z "${template[cc]}" ]; then
additional_headers="Cc: ${template[cc]}
${additional_headers}"
fi
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
cat <<EOF >$gen_msg_filename
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
From: ${template[from]}
To: ${template[to]}
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
Message-Id: <${gen_msg_id}>
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
Subject: ${template[subject]}
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
Date: ${template[date]}
notmuch-test: Allow custom headers when generating messages This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
2010-02-04 17:39:23 +01:00
${additional_headers}
${template[body]}
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
EOF
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
# Ensure that the mtime of the containing directory is updated
increment_mtime $(dirname ${gen_msg_filename})
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
}
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
# Generate a new message and add it to the index.
#
# All of the arguments and return values supported by generate_message
test: Add tests for naming threads with changing subjects. We recently added a feature to name threads based on the messages that actually matched the search, (as opposed to simply the oldest or newest message in the thread whether it matched or not). So add tests for that, and (surprise, surprise!) the feature does not entirely work.
2010-04-22 00:50:33 +02:00
# are also supported here, so see that function for details.
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
add_message ()
{
generate_message "$@"
$NOTMUCH new > /dev/null
}
make test: Actually count and report on failures. Hurrah---no more manual verification of that PASS column. This means that "make test" can actually be a useful part of the release process now, (since it will exit with non-zero status if there are any failures).
2010-04-16 00:18:30 +02:00
tests=0
test_failures=0
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
pass_if_equal ()
Add actual testing to notmuch-test We verify that each command creates output exactly as expected (after ignoring variable output such as the report of how long an operation took).
2010-02-03 01:24:53 +01:00
{
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$1
Add actual testing to notmuch-test We verify that each command creates output exactly as expected (after ignoring variable output such as the report of how long an operation took).
2010-02-03 01:24:53 +01:00
expected=$2
make test: Actually count and report on failures. Hurrah---no more manual verification of that PASS column. This means that "make test" can actually be a useful part of the release process now, (since it will exit with non-zero status if there are any failures).
2010-04-16 00:18:30 +02:00
tests=$((tests + 1))
Add actual testing to notmuch-test We verify that each command creates output exactly as expected (after ignoring variable output such as the report of how long an operation took).
2010-02-03 01:24:53 +01:00
if [ "$output" = "$expected" ]; then
echo " PASS"
else
echo " FAIL"
test: When a test fails, show diff only (save complete output to a file) We're starting to get test output that's fairly long, so it's much kinder to just show a diff rather than displaying the complete expected and actual output. To allow the user to investigate things after the fact, we save the expected and actual output to files named test-${test_number}.expected and test-${test_number}.output .
2010-04-22 22:22:08 +02:00
testname=test-$(printf "%03d" $tests)
echo "$expected" > $testname.expected
echo "$output" > $testname.output
diff -u $testname.expected $testname.output || true
make test: Actually count and report on failures. Hurrah---no more manual verification of that PASS column. This means that "make test" can actually be a useful part of the release process now, (since it will exit with non-zero status if there are any failures).
2010-04-16 00:18:30 +02:00
test_failures=$((test_failures + 1))
Add actual testing to notmuch-test We verify that each command creates output exactly as expected (after ignoring variable output such as the report of how long an operation took).
2010-02-03 01:24:53 +01:00
fi
}
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
TEST_DIR=$(pwd)/test.$$
MAIL_DIR=${TEST_DIR}/mail
export NOTMUCH_CONFIG=${TEST_DIR}/notmuch-config
NOTMUCH=$(find_notmuch_binary $(pwd))
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
NOTMUCH_NEW ()
{
$NOTMUCH new | grep -v -E -e '^Processed [0-9]*( total)? file|Found [0-9]* total file'
}
notmuch_search_sanitize ()
{
First tests for JSON output and UTF-8 in mail body and subject The test suite doesn't yet cover --format=json output nor UTF-8 in subject or body. This patch starts with test cases for 'search --format=json' and 'show --format=json'. Furthermore, it has test cases for a search for a UTF-8 string in a mail body for a UTF-8 string in a mail subject. Finally, it has a test case for --format=json with UTF-8 messages, demonstrating the fix in 1267697893-sup-4538@sam.mediasupervision.de. Reviewed-by: Carl Worth <cworth@cworth.org> Updated tests to current implementation of the test suite. These tests demonstrate a bug in the current implementation of "notmuch show --format=json", (timestamp output is changed depending on current timezone).
2010-04-13 18:37:57 +02:00
sed -r -e 's/("?thread"?: ?)("?)................("?)/\1\2XXX\3/'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
}
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
NOTMUCH_SHOW_FILENAME_SQUELCH='s,filename:.*/mail,filename:/XXX/mail,'
notmuch_show_sanitize ()
{
sed -e "$NOTMUCH_SHOW_FILENAME_SQUELCH"
}
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm -rf ${TEST_DIR}
mkdir ${TEST_DIR}
cd ${TEST_DIR}
mkdir ${MAIL_DIR}
cat <<EOF > ${NOTMUCH_CONFIG}
[database]
path=${MAIL_DIR}
[user]
name=Notmuch Test Suite
primary_email=test_suite@notmuchmail.org
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
other_email=test_suite_other@notmuchmail.org;test_suite@otherdomain.org
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
EOF
Add actual testing to notmuch-test We verify that each command creates output exactly as expected (after ignoring variable output such as the report of how long an operation took).
2010-02-03 01:24:53 +01:00
printf "Testing \"notmuch new\" in several variations:\n"
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " No new messages...\t\t\t\t"
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "No new mail."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Single new message...\t\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
generate_message
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Multiple new messages...\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
generate_message
generate_message
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 2 new messages to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " No new messages (non-empty DB)...\t\t"
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "No new mail."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " New directories...\t\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm -rf ${MAIL_DIR}/* ${MAIL_DIR}/.notmuch
mkdir ${MAIL_DIR}/def
mkdir ${MAIL_DIR}/ghi
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
generate_message [dir]=def
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Alternate inode order...\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm -rf ${MAIL_DIR}/.notmuch
mv ${MAIL_DIR}/ghi ${MAIL_DIR}/abc
rm ${MAIL_DIR}/def/*
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
generate_message [dir]=abc
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Message moved in...\t\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm -rf ${MAIL_DIR}/* ${MAIL_DIR}/.notmuch
generate_message
tmp_msg_filename=tmp/$gen_msg_filename
mkdir -p $(dirname $tmp_msg_filename)
mv $gen_msg_filename $tmp_msg_filename
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
$NOTMUCH new > /dev/null
mv $tmp_msg_filename $gen_msg_filename
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Renamed message...\t\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
generate_message
$NOTMUCH new > /dev/null
mv $gen_msg_filename ${gen_msg_filename}-renamed
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "No new mail. Detected 1 file rename."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Deleted message...\t\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm ${gen_msg_filename}-renamed
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "No new mail. Removed 1 message."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Renamed directory...\t\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
generate_message [dir]=dir
generate_message [dir]=dir
generate_message [dir]=dir
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
Add actual testing to notmuch-test We verify that each command creates output exactly as expected (after ignoring variable output such as the report of how long an operation took).
2010-02-03 01:24:53 +01:00
$NOTMUCH new > /dev/null
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
mv ${MAIL_DIR}/dir ${MAIL_DIR}/dir-renamed
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "No new mail. Detected 3 file renames."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Deleted directory...\t\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm -rf ${MAIL_DIR}/dir-renamed
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "No new mail. Removed 3 messages."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " New directory (at end of list)...\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
generate_message [dir]=zzz
generate_message [dir]=zzz
generate_message [dir]=zzz
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 3 new messages to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Deleted directory (end of list)...\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm -rf ${MAIL_DIR}/zzz
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "No new mail. Removed 3 messages."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " New symlink to directory...\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm -rf ${MAIL_DIR}/.notmuch
mv ${MAIL_DIR} ${TEST_DIR}/actual_maildir
mkdir ${MAIL_DIR}
ln -s ${TEST_DIR}/actual_maildir ${MAIL_DIR}/symlink
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " New symlink to a file...\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
generate_message
external_msg_filename=${TEST_DIR}/external/$(basename $gen_msg_filename)
mkdir -p $(dirname $external_msg_filename)
mv $gen_msg_filename $external_msg_filename
ln -s $external_msg_filename $gen_msg_filename
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " New two-level directory...\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
notmuch-test: Use named-parameters for generate_message function This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
2010-02-04 02:24:01 +01:00
generate_message [dir]=two/levels
generate_message [dir]=two/levels
generate_message [dir]=two/levels
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 3 new messages to the database."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Deleted two-level directory...\t\t\t"
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
rm -rf ${MAIL_DIR}/two
notmuch-test: Eliminate sleeps to speed up test suite run We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
2010-02-04 20:31:01 +01:00
increment_mtime ${MAIL_DIR}
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "No new mail. Removed 3 messages."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
printf "\nTesting \"notmuch search\" in several variations:\n"
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search body...\t\t\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="body search"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [body]=bodysearchtest
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search bodysearchtest | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; body search (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by from:...\t\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by from"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [from]=searchbyfrom
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search from:searchbyfrom | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] searchbyfrom; search by from (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by to:...\t\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by to"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [to]=searchbyto
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search to:searchbyto | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by subject:...\t\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message [subject]=subjectsearchtest '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search subject:subjectsearchtest | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; subjectsearchtest (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by id:...\t\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by id"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search id:${gen_msg_id} | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by id (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by tag:...\t\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by tag"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
$NOTMUCH tag +searchbytag id:${gen_msg_id}
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search tag:searchbytag | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by tag (inbox searchbytag unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by thread:...\t\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by thread"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
thread_id=$($NOTMUCH search id:${gen_msg_id} | sed -e 's/thread:\([a-f0-9]*\).*/\1/')
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search thread:${thread_id} | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by thread (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search body (phrase)...\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="body search (phrase)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="body search (phrase)"'
test: Add some negative results for the phrase searches. These results have all the same terms as the target phrase, but not in the expected order. They are designed to ensure that we actually test phrase searches. And as it turns out, we're not currently quoting the search terms properly, so the phrase-search tests now fail with this commit.
2010-03-11 18:52:08 +01:00
add_message '[subject]="negative result"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="This phrase should not match the body search"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search '\"body search (phrase)\"' | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; body search (phrase) (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by from: (address)...\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by from (address)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [from]=searchbyfrom@example.com
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search from:searchbyfrom@example.com | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] searchbyfrom@example.com; search by from (address) (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by from: (name)...\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by from (name)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[from]="Search By From Name <test@example.com>"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search from:'Search By From Name' | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Search By From Name; search by from (name) (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by to: (address)...\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by to (address)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [to]=searchbyto@example.com
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search to:searchbyto@example.com | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (address) (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by to: (name)...\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="search by to (name)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[to]="Search By To Name <test@example.com>"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search to:'Search By To Name' | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (name) (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search by subject: (phrase)...\t\t\t"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
add_message '[subject]="subject search test (phrase)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
test: Add some negative results for the phrase searches. These results have all the same terms as the target phrase, but not in the expected order. They are designed to ensure that we actually test phrase searches. And as it turns out, we're not currently quoting the search terms properly, so the phrase-search tests now fail with this commit.
2010-03-11 18:52:08 +01:00
add_message '[subject]="this phrase should not match the subject search test"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search 'subject:\"subject search test (phrase)\"' | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; subject search test (phrase) (inbox unread)"
notmuch-test: Add several tests of "notmuch search" These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
2010-02-05 19:00:04 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Search for all messages (\"*\"):...\t\t"
test: Add a test for "notmuch search '*'" This feature was added recently and should have gotten a new test at the time. As this test demonstrates, the code is broken, ("notmuch search '*' returns bogus dates of the Unix epoch for any threads where the term "and" does not appear in any messages).
2010-04-15 23:06:10 +02:00
output=$($NOTMUCH search '*' | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Test message #6 (inbox unread)
thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Test message #14 (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; body search (inbox unread)
thread:XXX 2000-01-01 [1/1] searchbyfrom; search by from (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; subjectsearchtest (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by id (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by tag (inbox searchbytag unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by thread (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; body search (phrase) (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; negative result (inbox unread)
thread:XXX 2000-01-01 [1/1] searchbyfrom@example.com; search by from (address) (inbox unread)
thread:XXX 2000-01-01 [1/1] Search By From Name; search by from (name) (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (address) (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (name) (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; subject search test (phrase) (inbox unread)
thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; this phrase should not match the subject search test (inbox unread)"
test: Put the json tests into their own section. Not that the sections actually mean anything yet, but it makes for clean output.
2010-04-23 02:17:11 +02:00
printf " Search body (utf-8):...\t\t\t"
add_message '[subject]="utf8-message-body-subject"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="message body utf8: bödý"'
output=$($NOTMUCH search 'bödý' | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-message-body-subject (inbox unread)"
printf "\nTesting --format=json output:\n"
First tests for JSON output and UTF-8 in mail body and subject The test suite doesn't yet cover --format=json output nor UTF-8 in subject or body. This patch starts with test cases for 'search --format=json' and 'show --format=json'. Furthermore, it has test cases for a search for a UTF-8 string in a mail body for a UTF-8 string in a mail subject. Finally, it has a test case for --format=json with UTF-8 messages, demonstrating the fix in 1267697893-sup-4538@sam.mediasupervision.de. Reviewed-by: Carl Worth <cworth@cworth.org> Updated tests to current implementation of the test suite. These tests demonstrate a bug in the current implementation of "notmuch show --format=json", (timestamp output is changed depending on current timezone).
2010-04-13 18:37:57 +02:00
printf " Show message: json...\t\t\t\t"
add_message '[subject]="json-show-subject"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="json-show-message"'
output=$($NOTMUCH show --format=json 'json-show-message')
pass_if_equal "$output" '[[[{"id": "'${gen_msg_id}'", "match": true, "filename": "'${gen_msg_filename}'", "timestamp": 946728000, "date_relative": "2000-01-01", "tags": ["inbox","unread"], "headers": {"Subject": "json-show-subject", "From": "Notmuch Test Suite <test_suite@notmuchmail.org>", "To": "Notmuch Test Suite <test_suite@notmuchmail.org>", "Cc": "", "Bcc": "", "Date": "Sat, 01 Jan 2000 12:00:00 -0000"}, "body": [{"id": 1, "content-type": "text/plain", "content": "json-show-message\n"}]}, []]]]'
printf " Search message: json...\t\t\t"
add_message '[subject]="json-search-subject"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="json-search-message"'
output=$($NOTMUCH search --format=json 'json-search-message' | notmuch_search_sanitize)
pass_if_equal "$output" '[{"thread": "XXX",
"timestamp": 946728000,
"matched": 1,
"total": 1,
"authors": "Notmuch Test Suite",
"subject": "json-search-subject",
"tags": ["inbox", "unread"]}]'
printf " Search by subject (utf-8):...\t\t\t"
add_message [subject]=utf8-sübjéct '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
output=$($NOTMUCH search subject:utf8-sübjéct | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread)"
printf " Show message: json, utf-8...\t\t\t"
add_message '[subject]="json-show-utf8-body-sübjéct"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="jsön-show-méssage"'
output=$($NOTMUCH show --format=json 'jsön-show-méssage')
pass_if_equal "$output" '[[[{"id": "'${gen_msg_id}'", "match": true, "filename": "'${gen_msg_filename}'", "timestamp": 946728000, "date_relative": "2000-01-01", "tags": ["inbox","unread"], "headers": {"Subject": "json-show-utf8-body-sübjéct", "From": "Notmuch Test Suite <test_suite@notmuchmail.org>", "To": "Notmuch Test Suite <test_suite@notmuchmail.org>", "Cc": "", "Bcc": "", "Date": "Sat, 01 Jan 2000 12:00:00 -0000"}, "body": [{"id": 1, "content-type": "text/plain", "content": "jsön-show-méssage\n"}]}, []]]]'
printf " Search message: json, utf-8...\t\t\t"
add_message '[subject]="json-search-utf8-body-sübjéct"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="jsön-search-méssage"'
output=$($NOTMUCH search --format=json 'jsön-search-méssage' | notmuch_search_sanitize)
pass_if_equal "$output" '[{"thread": "XXX",
"timestamp": 946728000,
"matched": 1,
"total": 1,
"authors": "Notmuch Test Suite",
"subject": "json-search-utf8-body-sübjéct",
"tags": ["inbox", "unread"]}]'
test: Add tests for naming threads with changing subjects. We recently added a feature to name threads based on the messages that actually matched the search, (as opposed to simply the oldest or newest message in the thread whether it matched or not). So add tests for that, and (surprise, surprise!) the feature does not entirely work.
2010-04-22 00:50:33 +02:00
printf "\nTesting naming of threads with changing subject:\n"
add_message '[subject]="thread-naming: Initial thread subject"' \
'[date]="Fri, 05 Jan 2001 15:43:56 -0800"'
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
first=${gen_msg_cnt}
test: Add tests for naming threads with changing subjects. We recently added a feature to name threads based on the messages that actually matched the search, (as opposed to simply the oldest or newest message in the thread whether it matched or not). So add tests for that, and (surprise, surprise!) the feature does not entirely work.
2010-04-22 00:50:33 +02:00
parent=${gen_msg_id}
add_message '[subject]="thread-naming: Older changed subject"' \
'[date]="Sat, 06 Jan 2001 15:43:56 -0800"' \
"[in-reply-to]=\<$parent\>"
add_message '[subject]="thread-naming: Newer changed subject"' \
'[date]="Sun, 07 Jan 2001 15:43:56 -0800"' \
"[in-reply-to]=\<$parent\>"
add_message '[subject]="thread-naming: Final thread subject"' \
'[date]="Mon, 08 Jan 2001 15:43:56 -0800"' \
"[in-reply-to]=\<$parent\>"
final=${gen_msg_id}
printf " Initial thread name (oldest-first search)...\t"
output=$($NOTMUCH search --sort=oldest-first thread-naming and tag:inbox | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-05 [4/4] Notmuch Test Suite; thread-naming: Initial thread subject (inbox unread)"
printf " Initial thread name (newest-first search)...\t"
output=$($NOTMUCH search --sort=newest-first thread-naming and tag:inbox | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-08 [4/4] Notmuch Test Suite; thread-naming: Final thread subject (inbox unread)"
# Remove oldest and newest messages from search results
$NOTMUCH tag -inbox id:$parent or id:$final
printf " Changed thread name (oldest-first search)...\t"
output=$($NOTMUCH search --sort=oldest-first thread-naming and tag:inbox | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-06 [2/4] Notmuch Test Suite; thread-naming: Older changed subject (inbox unread)"
printf " Changed thread name (newest-first search)...\t"
output=$($NOTMUCH search --sort=newest-first thread-naming and tag:inbox | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-07 [2/4] Notmuch Test Suite; thread-naming: Newer changed subject (inbox unread)"
printf " Ignore added reply prefix (Re:)...\t\t"
add_message '[subject]="Re: thread-naming: Initial thread subject"' \
'[date]="Tue, 09 Jan 2001 15:43:45 -0800"' \
"[in-reply-to]=\<$parent\>"
output=$($NOTMUCH search --sort=newest-first thread-naming and tag:inbox | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-09 [3/5] Notmuch Test Suite; thread-naming: Initial thread subject (inbox unread)"
printf " Ignore added reply prefix (Aw:)...\t\t"
add_message '[subject]="Aw: thread-naming: Initial thread subject"' \
'[date]="Wed, 10 Jan 2001 15:43:45 -0800"' \
"[in-reply-to]=\<$parent\>"
output=$($NOTMUCH search --sort=newest-first thread-naming and tag:inbox | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-10 [4/6] Notmuch Test Suite; thread-naming: Initial thread subject (inbox unread)"
printf " Ignore added reply prefix (Vs:)...\t\t"
add_message '[subject]="Vs: thread-naming: Initial thread subject"' \
'[date]="Thu, 11 Jan 2001 15:43:45 -0800"' \
"[in-reply-to]=\<$parent\>"
output=$($NOTMUCH search --sort=newest-first thread-naming and tag:inbox | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-11 [5/7] Notmuch Test Suite; thread-naming: Initial thread subject (inbox unread)"
printf " Ignore added reply prefix (Sv:)...\t\t"
add_message '[subject]="Sv: thread-naming: Initial thread subject"' \
'[date]="Fri, 12 Jan 2001 15:43:45 -0800"' \
"[in-reply-to]=\<$parent\>"
output=$($NOTMUCH search --sort=newest-first thread-naming and tag:inbox | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2001-01-12 [6/8] Notmuch Test Suite; thread-naming: Initial thread subject (inbox unread)"
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
printf " Test order of messages in \"notmuch show\"\t"
output=$($NOTMUCH show thread-naming | notmuch_show_sanitize)
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
pass_if_equal "$output" " message{ id:msg-$(printf "%03d" $first)@notmuch-test-suite depth:0 match:1 filename:/XXX/mail/msg-$(printf "%03d" $first)
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
header{
Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (unread)
Subject: thread-naming: Initial thread subject
From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: Notmuch Test Suite <test_suite@notmuchmail.org>
Date: Fri, 05 Jan 2001 15:43:56 -0800
header}
body{
part{ ID: 1, Content-type: text/plain
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
This is just a test message (#$first)
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
part}
body}
message}
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
message{ id:msg-$(printf "%03d" $((first + 1)))@notmuch-test-suite depth:1 match:1 filename:/XXX/mail/msg-$(printf "%03d" $((first + 1)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
header{
Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-06) (inbox unread)
Subject: thread-naming: Older changed subject
From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: Notmuch Test Suite <test_suite@notmuchmail.org>
Date: Sat, 06 Jan 2001 15:43:56 -0800
header}
body{
part{ ID: 1, Content-type: text/plain
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
This is just a test message (#$((first + 1)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
part}
body}
message}
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
message{ id:msg-$(printf "%03d" $((first + 2)))@notmuch-test-suite depth:1 match:1 filename:/XXX/mail/msg-$(printf "%03d" $((first + 2)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
header{
Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-07) (inbox unread)
Subject: thread-naming: Newer changed subject
From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: Notmuch Test Suite <test_suite@notmuchmail.org>
Date: Sun, 07 Jan 2001 15:43:56 -0800
header}
body{
part{ ID: 1, Content-type: text/plain
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
This is just a test message (#$((first + 2)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
part}
body}
message}
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
message{ id:msg-$(printf "%03d" $((first + 3)))@notmuch-test-suite depth:1 match:1 filename:/XXX/mail/msg-$(printf "%03d" $((first + 3)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
header{
Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-08) (unread)
Subject: thread-naming: Final thread subject
From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: Notmuch Test Suite <test_suite@notmuchmail.org>
Date: Mon, 08 Jan 2001 15:43:56 -0800
header}
body{
part{ ID: 1, Content-type: text/plain
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
This is just a test message (#$((first + 3)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
part}
body}
message}
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
message{ id:msg-$(printf "%03d" $((first + 4)))@notmuch-test-suite depth:1 match:1 filename:/XXX/mail/msg-$(printf "%03d" $((first + 4)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
header{
Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-09) (inbox unread)
Subject: Re: thread-naming: Initial thread subject
From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: Notmuch Test Suite <test_suite@notmuchmail.org>
Date: Tue, 09 Jan 2001 15:43:45 -0800
header}
body{
part{ ID: 1, Content-type: text/plain
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
This is just a test message (#$((first + 4)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
part}
body}
message}
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
message{ id:msg-$(printf "%03d" $((first + 5)))@notmuch-test-suite depth:1 match:1 filename:/XXX/mail/msg-$(printf "%03d" $((first + 5)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
header{
Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-10) (inbox unread)
Subject: Aw: thread-naming: Initial thread subject
From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: Notmuch Test Suite <test_suite@notmuchmail.org>
Date: Wed, 10 Jan 2001 15:43:45 -0800
header}
body{
part{ ID: 1, Content-type: text/plain
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
This is just a test message (#$((first + 5)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
part}
body}
message}
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
message{ id:msg-$(printf "%03d" $((first + 6)))@notmuch-test-suite depth:1 match:1 filename:/XXX/mail/msg-$(printf "%03d" $((first + 6)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
header{
Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-11) (inbox unread)
Subject: Vs: thread-naming: Initial thread subject
From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: Notmuch Test Suite <test_suite@notmuchmail.org>
Date: Thu, 11 Jan 2001 15:43:45 -0800
header}
body{
part{ ID: 1, Content-type: text/plain
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
This is just a test message (#$((first + 6)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
part}
body}
message}
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
message{ id:msg-$(printf "%03d" $((first + 7)))@notmuch-test-suite depth:1 match:1 filename:/XXX/mail/msg-$(printf "%03d" $((first + 7)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
header{
Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-12) (inbox unread)
Subject: Sv: thread-naming: Initial thread subject
From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: Notmuch Test Suite <test_suite@notmuchmail.org>
Date: Fri, 12 Jan 2001 15:43:45 -0800
header}
body{
part{ ID: 1, Content-type: text/plain
test: Make existing "notmuch show" test more resilient If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
2010-04-23 02:08:08 +02:00
This is just a test message (#$((first + 7)))
test: Add a test to demonstrate message-sorting regression. In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
2010-04-22 22:26:37 +02:00
part}
body}
message}"
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
printf "\nTesting \"notmuch reply\" in several variations:\n"
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Basic reply...\t\t\t\t\t"
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
add_message '[from]="Sender <sender@example.com>"' \
[to]=test_suite@notmuchmail.org \
[subject]=notmuch-reply-test \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="basic reply test"'
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
test: Fix reply tests to track insignificant change in output. The recent fix to properly decode encoded headers made the expected output of "notmuch reply" differ by a single space, (previously, there were two spaces before the References: value and now there is just one). Fix the test suite so that these are all noted as correct results again.
2010-04-15 02:17:50 +02:00
References: <${gen_msg_id}>
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
notmuch-test: Add "notmuch reply" test for reply-to support. This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
2010-02-04 20:52:08 +01:00
> basic reply test"
notmuch-test: Add a basic test of "notmuch reply" Simply ensuring that the reply template is formatted as expected.
2010-02-04 20:33:56 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Multiple recipients...\t\t\t\t"
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
add_message '[from]="Sender <sender@example.com>"' \
'[to]="test_suite@notmuchmail.org, Someone Else <someone@example.com>"' \
[subject]=notmuch-reply-test \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="Multiple recipients"'
notmuch-test: Test "notmuch reply" with multiple recipients The feature tested here is that we reply to both the sender and to others addresses on the To: line of the original message, but that we don't reply to our own address.
2010-02-04 21:03:30 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
notmuch-test: Test "notmuch reply" with multiple recipients The feature tested here is that we reply to both the sender and to others addresses on the To: line of the original message, but that we don't reply to our own address.
2010-02-04 21:03:30 +01:00
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, Someone Else <someone@example.com>
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
test: Fix reply tests to track insignificant change in output. The recent fix to properly decode encoded headers made the expected output of "notmuch reply" differ by a single space, (previously, there were two spaces before the References: value and now there is just one). Fix the test suite so that these are all noted as correct results again.
2010-04-15 02:17:50 +02:00
References: <${gen_msg_id}>
notmuch-test: Test "notmuch reply" with multiple recipients The feature tested here is that we reply to both the sender and to others addresses on the To: line of the original message, but that we don't reply to our own address.
2010-02-04 21:03:30 +01:00
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> Multiple recipients"
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Reply with CC...\t\t\t\t"
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
add_message '[from]="Sender <sender@example.com>"' \
[to]=test_suite@notmuchmail.org \
'[cc]="Other Parties <cc@example.com>"' \
[subject]=notmuch-reply-test \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="reply with CC"'
notmuch-test: Add test for "notmuch reply" with a CC header. A simple test to help round out the set of tested features.
2010-02-04 20:55:57 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
notmuch-test: Add test for "notmuch reply" with a CC header. A simple test to help round out the set of tested features.
2010-02-04 20:55:57 +01:00
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>
Cc: Other Parties <cc@example.com>
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
test: Fix reply tests to track insignificant change in output. The recent fix to properly decode encoded headers made the expected output of "notmuch reply" differ by a single space, (previously, there were two spaces before the References: value and now there is just one). Fix the test suite so that these are all noted as correct results again.
2010-04-15 02:17:50 +02:00
References: <${gen_msg_id}>
notmuch-test: Add test for "notmuch reply" with a CC header. A simple test to help round out the set of tested features.
2010-02-04 20:55:57 +01:00
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> reply with CC"
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Reply from alternate address...\t\t"
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
add_message '[from]="Sender <sender@example.com>"' \
[to]=test_suite_other@notmuchmail.org \
[subject]=notmuch-reply-test \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="reply from alternate address"'
notmuch-test: Test "notmuch reply" from alternate address. When reply to a message addresses to an address configured in the other_email setting in the configuration file, the reply should use that address in the From header. Test this.
2010-02-04 20:39:31 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite_other@notmuchmail.org>
notmuch-test: Test "notmuch reply" from alternate address. When reply to a message addresses to an address configured in the other_email setting in the configuration file, the reply should use that address in the From header. Test this.
2010-02-04 20:39:31 +01:00
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
test: Fix reply tests to track insignificant change in output. The recent fix to properly decode encoded headers made the expected output of "notmuch reply" differ by a single space, (previously, there were two spaces before the References: value and now there is just one). Fix the test suite so that these are all noted as correct results again.
2010-04-15 02:17:50 +02:00
References: <${gen_msg_id}>
notmuch-test: Test "notmuch reply" from alternate address. When reply to a message addresses to an address configured in the other_email setting in the configuration file, the reply should use that address in the From header. Test this.
2010-02-04 20:39:31 +01:00
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
notmuch-test: Add "notmuch reply" test for reply-to support. This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
2010-02-04 20:52:08 +01:00
> reply from alternate address"
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Support for Reply-To...\t\t\t"
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
add_message '[from]="Sender <sender@example.com>"' \
[to]=test_suite@notmuchmail.org \
[subject]=notmuch-reply-test \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="support for reply-to"' \
'[reply-to]="Sender <elsewhere@example.com>"'
notmuch-test: Add "notmuch reply" test for reply-to support. This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
2010-02-04 20:52:08 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
notmuch-test: Add "notmuch reply" test for reply-to support. This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
2010-02-04 20:52:08 +01:00
Subject: Re: notmuch-reply-test
To: Sender <elsewhere@example.com>
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
test: Fix reply tests to track insignificant change in output. The recent fix to properly decode encoded headers made the expected output of "notmuch reply" differ by a single space, (previously, there were two spaces before the References: value and now there is just one). Fix the test suite so that these are all noted as correct results again.
2010-04-15 02:17:50 +02:00
References: <${gen_msg_id}>
notmuch-test: Add "notmuch reply" test for reply-to support. This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
2010-02-04 20:52:08 +01:00
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> support for reply-to"
notmuch-test: Test "notmuch reply" from alternate address. When reply to a message addresses to an address configured in the other_email setting in the configuration file, the reply should use that address in the From header. Test this.
2010-02-04 20:39:31 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Un-munging Reply-To...\t\t\t\t"
notmuch-test: Add a new add_message function. The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
2010-02-05 02:34:48 +01:00
add_message '[from]="Sender <sender@example.com>"' \
'[to]="Some List <list@example.com>"' \
[subject]=notmuch-reply-test \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="Un-munging Reply-To"' \
'[reply-to]="Evil Munging List <list@example.com>"'
notmuch-test: Test "notmuch reply" with a reply-to-munged mail This is a test for the recently added feature where we detect that the reply-to address already exists in the To: or Cc: header so will already be replied to. In this case we want to include the From: address in our reply, (where, otherwise we would use the Reply-To address *instead* of the address in the From header).
2010-02-04 21:08:13 +01:00
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
notmuch-test: Test "notmuch reply" with a reply-to-munged mail This is a test for the recently added feature where we detect that the reply-to address already exists in the To: or Cc: header so will already be replied to. In this case we want to include the From: address in our reply, (where, otherwise we would use the Reply-To address *instead* of the address in the From header).
2010-02-04 21:08:13 +01:00
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, Some List <list@example.com>
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
test: Fix reply tests to track insignificant change in output. The recent fix to properly decode encoded headers made the expected output of "notmuch reply" differ by a single space, (previously, there were two spaces before the References: value and now there is just one). Fix the test suite so that these are all noted as correct results again.
2010-04-15 02:17:50 +02:00
References: <${gen_msg_id}>
notmuch-test: Test "notmuch reply" with a reply-to-munged mail This is a test for the recently added feature where we detect that the reply-to address already exists in the To: or Cc: header so will already be replied to. In this case we want to include the From: address in our reply, (where, otherwise we would use the Reply-To address *instead* of the address in the From header).
2010-02-04 21:08:13 +01:00
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> Un-munging Reply-To"
notmuch-test: Add test to verify that uuencoded data is not indexed. As recently promised, no new features are being accepted to notmuch without corresponding new tests for the test suite.
2010-02-05 19:24:51 +01:00
printf "\nTesting handling of uuencoded data:\n"
add_message [subject]=uuencodetest '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' \
'[body]="This message is used to ensure that notmuch correctly handles a
message containing a block of uuencoded data. First, we have a marker
this content beforeuudata . Then we beging the uunencoded data itself:
begin 644 bogus-uuencoded-data
M0123456789012345678901234567890123456789012345678901234567890
MOBVIOUSLY, THIS IS NOT ANY SORT OF USEFUL UUNECODED DATA.
MINSTEAD THIS IS JUST A WAY TO ENSURE THAT THIS BLOCK OF DATA
MIS CORRECTLY IGNORED WHEN NOTMUCH CREATES ITS INDEX. SO WE
MINCLUDE A DURINGUUDATA MARKER THAT SHOULD NOT RESULT IN ANY
MSEARCH RESULT.
\`
end
Finally, we have our afteruudata marker as well."'
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Ensure content before uu data is indexed...\t"
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search beforeuudata | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; uuencodetest (inbox unread)"
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Ensure uu data is not indexed...\t\t"
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search DURINGUUDATA | notmuch_search_sanitize)
pass_if_equal "$output" ""
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Ensure content after uu data is indexed...\t"
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search afteruudata | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; uuencodetest (inbox unread)"
notmuch-test: Add test to verify that uuencoded data is not indexed. As recently promised, no new features are being accepted to notmuch without corresponding new tests for the test suite.
2010-02-05 19:24:51 +01:00
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
printf "\nTesting \"notmuch dump\" and \"notmuch restore\":\n"
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Dumping all tags...\t\t\t\t"
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
$NOTMUCH dump dump.expected
test: Unify all tests to use the pass_if_equal function. Previously some tests (dump/restore) were doing ad-hoc verification of values and their own printing of PASS/FAIL, etc. This made it impossible to count test pass/fail rates in a single place. The only reason these tests were written that way was because the old execute_expecting function only worked if one could directly test the stdout output of a notmuch command. The recent switch to pass_if_equal means that all tests can use it.
2010-04-16 00:09:21 +02:00
pass_if_equal "$?" "0"
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Clearing all tags...\t\t\t\t"
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
sed -e 's/(\([^(]*\))$/()/' < dump.expected > clear.expected
$NOTMUCH restore clear.expected
$NOTMUCH dump clear.actual
test: Unify all tests to use the pass_if_equal function. Previously some tests (dump/restore) were doing ad-hoc verification of values and their own printing of PASS/FAIL, etc. This made it impossible to count test pass/fail rates in a single place. The only reason these tests were written that way was because the old execute_expecting function only worked if one could directly test the stdout output of a notmuch command. The recent switch to pass_if_equal means that all tests can use it.
2010-04-16 00:09:21 +02:00
pass_if_equal "$(< clear.actual)" "$(< clear.expected)"
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Restoring original tags...\t\t\t"
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
$NOTMUCH restore dump.expected
$NOTMUCH dump dump.actual
test: Unify all tests to use the pass_if_equal function. Previously some tests (dump/restore) were doing ad-hoc verification of values and their own printing of PASS/FAIL, etc. This made it impossible to count test pass/fail rates in a single place. The only reason these tests were written that way was because the old execute_expecting function only worked if one could directly test the stdout output of a notmuch command. The recent switch to pass_if_equal means that all tests can use it.
2010-04-16 00:09:21 +02:00
pass_if_equal "$(< dump.actual)" "$(< dump.expected)"
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
test: Align the PASS/FAIL column. Just to make things a little more pleasant to look at.
2010-04-16 00:06:00 +02:00
printf " Restore with nothing to do...\t\t\t"
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
$NOTMUCH restore dump.expected
test: Unify all tests to use the pass_if_equal function. Previously some tests (dump/restore) were doing ad-hoc verification of values and their own printing of PASS/FAIL, etc. This made it impossible to count test pass/fail rates in a single place. The only reason these tests were written that way was because the old execute_expecting function only worked if one could directly test the stdout output of a notmuch command. The recent switch to pass_if_equal means that all tests can use it.
2010-04-16 00:09:21 +02:00
pass_if_equal "$?" "0"
notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore" Our test-suite coverage keeps improving, (slowly).
2010-02-06 02:28:25 +01:00
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
printf "\nTesting threading when messages received out of order:\n"
printf " Adding initial child message...\t\t"
generate_message [body]=foo '[in-reply-to]=\<parent-id\>' [subject]=brokenthreadtest '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
printf " Searching returns the message...\t\t"
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search foo | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; brokenthreadtest (inbox unread)"
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
printf " Adding second child message...\t\t\t"
generate_message [body]=foo '[in-reply-to]=\<parent-id\>' [subject]=brokenthreadtest '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
printf " Searching returns both messages in one thread..."
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search foo | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [2/2] Notmuch Test Suite; brokenthreadtest (inbox unread)"
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
printf " Adding parent message...\t\t\t"
generate_message [body]=foo [id]=parent-id [subject]=brokenthreadtest '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
printf " Searching returns all three messages in one thread..."
test: Disentangle execution of notmuch from the examination of the results. The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
2010-04-15 22:44:00 +02:00
output=$($NOTMUCH search foo | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [3/3] Notmuch Test Suite; brokenthreadtest (inbox unread)"
test: Add new tests for out-of-order messages. These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
2010-03-13 22:27:57 +01:00
Add tests for author name reordering in search results This should be required for all patches :-) Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-24 20:20:56 +02:00
printf "\nTesting author reordering;\n"
printf " Adding parent message...\t\t\t"
generate_message [body]=findme [id]=new-parent-id [subject]=author-reorder-threadtest '[from]="User <user@example.com>"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
printf " Adding initial child message...\t\t"
generate_message [body]=findme '[in-reply-to]=\<new-parent-id\>' [subject]=author-reorder-threadtest '[from]="User1 <user1@example.com>"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
printf " Adding second child message...\t\t\t"
generate_message [body]=findme '[in-reply-to]=\<new-parent-id\>' [subject]=author-reorder-threadtest '[from]="User2 <user2@example.com>"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
output=$(NOTMUCH_NEW)
pass_if_equal "$output" "Added 1 new message to the database."
printf " Searching when all three messages match...\t"
output=$($NOTMUCH search findme | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [3/3] User, User1, User2; author-reorder-threadtest (inbox unread)"
printf " Searching when two messages match...\t\t"
output=$($NOTMUCH search User1 or User2 | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [2/3] User1, User2| User; author-reorder-threadtest (inbox unread)"
printf " Searching when only one message matches...\t"
output=$($NOTMUCH search User2 | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/3] User2| User, User1; author-reorder-threadtest (inbox unread)"
printf " Searching when only first message matches...\t"
output=$($NOTMUCH search User | notmuch_search_sanitize)
pass_if_equal "$output" "thread:XXX 2000-01-01 [1/3] User| User1, User2; author-reorder-threadtest (inbox unread)"
test: Exercise magic-from guessing with a single configured address Immediately after releasing 0.3 we learned that the magic-from-guessing code could hang in an infinite loop in some cases. The bug occurred only when the user had configured only a primary email addresss and no other email addresses. The test suite wasn't previously covering this case, so address this shortcoming.
2010-04-27 19:13:04 +02:00
printf "\nTesting From line heuristics (with multiple configured addresses):\n"
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
printf " Magic from guessing (nothing to go on)...\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
fix expected test result to include Bcc lines this test actually tests behavior that I consider as broken. The Bcc should be to the same address as used in the From line, otherwise we are creating a potential information leak as email that is related to one email account (say, work) is copied to a different account Signed-off-by: Dirk Hohndel <hohndel@infradead.org> Reviewed-by: Carl Worth <cworth@cworth.org> These tests don't actually pass yet, since the feature being tested has not been merged. But gettting these tests in first will let us more easily test that the feature actually works, (and will help us ensure we don't forget the feature before the next release).
2010-04-21 23:55:58 +02:00
Bcc: test_suite@notmuchmail.org
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
printf " Magic from guessing (Envelope-to:)...\t\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[header]="Envelope-To: test_suite_other@notmuchmail.org"' \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite_other@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
fix expected test result to include Bcc lines this test actually tests behavior that I consider as broken. The Bcc should be to the same address as used in the From line, otherwise we are creating a potential information leak as email that is related to one email account (say, work) is copied to a different account Signed-off-by: Dirk Hohndel <hohndel@infradead.org> Reviewed-by: Carl Worth <cworth@cworth.org> These tests don't actually pass yet, since the feature being tested has not been merged. But gettting these tests in first will let us more easily test that the feature actually works, (and will help us ensure we don't forget the feature before the next release).
2010-04-21 23:55:58 +02:00
Bcc: test_suite@notmuchmail.org
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
printf " Magic from guessing (X-Original-To:)...\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[header]="X-Original-To: test_suite_other@notmuchmail.org"' \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite_other@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
fix expected test result to include Bcc lines this test actually tests behavior that I consider as broken. The Bcc should be to the same address as used in the From line, otherwise we are creating a potential information leak as email that is related to one email account (say, work) is copied to a different account Signed-off-by: Dirk Hohndel <hohndel@infradead.org> Reviewed-by: Carl Worth <cworth@cworth.org> These tests don't actually pass yet, since the feature being tested has not been merged. But gettting these tests in first will let us more easily test that the feature actually works, (and will help us ensure we don't forget the feature before the next release).
2010-04-21 23:55:58 +02:00
Bcc: test_suite@notmuchmail.org
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
printf " Magic from guessing (Received: .. for ..)...\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[header]="Received: from mail.example.com (mail.example.com [1.1.1.1])\
by mail.notmuchmail.org (some MTA) with ESMTP id 12345678\
for <test_suite_other@notmuchmail.org>; Sat, 10 Apr 2010 07:54:51 -0400 (EDT)"' \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite_other@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
fix expected test result to include Bcc lines this test actually tests behavior that I consider as broken. The Bcc should be to the same address as used in the From line, otherwise we are creating a potential information leak as email that is related to one email account (say, work) is copied to a different account Signed-off-by: Dirk Hohndel <hohndel@infradead.org> Reviewed-by: Carl Worth <cworth@cworth.org> These tests don't actually pass yet, since the feature being tested has not been merged. But gettting these tests in first will let us more easily test that the feature actually works, (and will help us ensure we don't forget the feature before the next release).
2010-04-21 23:55:58 +02:00
Bcc: test_suite@notmuchmail.org
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
printf " Magic from guessing (Received: domain)...\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[header]="Received: from mail.example.com (mail.example.com [1.1.1.1])\
by mail.otherdomain.org (some MTA) with ESMTP id 12345678\
Sat, 10 Apr 2010 07:54:51 -0400 (EDT)"' \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@otherdomain.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
fix expected test result to include Bcc lines this test actually tests behavior that I consider as broken. The Bcc should be to the same address as used in the From line, otherwise we are creating a potential information leak as email that is related to one email account (say, work) is copied to a different account Signed-off-by: Dirk Hohndel <hohndel@infradead.org> Reviewed-by: Carl Worth <cworth@cworth.org> These tests don't actually pass yet, since the feature being tested has not been merged. But gettting these tests in first will let us more easily test that the feature actually works, (and will help us ensure we don't forget the feature before the next release).
2010-04-21 23:55:58 +02:00
Bcc: test_suite@notmuchmail.org
add From guessing tests to test suite right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
2010-04-21 23:55:57 +02:00
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
test: Exercise magic-from guessing with a single configured address Immediately after releasing 0.3 we learned that the magic-from-guessing code could hang in an infinite loop in some cases. The bug occurred only when the user had configured only a primary email addresss and no other email addresses. The test suite wasn't previously covering this case, so address this shortcoming.
2010-04-27 19:13:04 +02:00
printf "\nTesting From line heuristics (with single configured address):\n"
sed -i -e 's/^other_email.*//' ${NOTMUCH_CONFIG}
printf " Magic from guessing (nothing to go on)...\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
printf " Magic from guessing (Envelope-to:)...\t\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[header]="Envelope-To: test_suite_other@notmuchmail.org"' \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
printf " Magic from guessing (X-Original-To:)...\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[header]="X-Original-To: test_suite_other@notmuchmail.org"' \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
printf " Magic from guessing (Received: .. for ..)...\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[header]="Received: from mail.example.com (mail.example.com [1.1.1.1])\
by mail.notmuchmail.org (some MTA) with ESMTP id 12345678\
for <test_suite_other@notmuchmail.org>; Sat, 10 Apr 2010 07:54:51 -0400 (EDT)"' \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
printf " Magic from guessing (Received: domain)...\t"
add_message '[from]="Sender <sender@example.com>"' \
[to]=mailinglist@notmuchmail.org \
[subject]=notmuch-reply-test \
'[header]="Received: from mail.example.com (mail.example.com [1.1.1.1])\
by mail.otherdomain.org (some MTA) with ESMTP id 12345678\
Sat, 10 Apr 2010 07:54:51 -0400 (EDT)"' \
'[date]="Tue, 05 Jan 2010 15:43:56 -0800"' \
'[body]="from guessing test"'
output=$($NOTMUCH reply id:${gen_msg_id})
pass_if_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
Subject: Re: notmuch-reply-test
To: Sender <sender@example.com>, mailinglist@notmuchmail.org
Bcc: test_suite@notmuchmail.org
In-Reply-To: <${gen_msg_id}>
References: <${gen_msg_id}>
On Tue, 05 Jan 2010 15:43:56 -0800, Sender <sender@example.com> wrote:
> from guessing test"
make test: Actually count and report on failures. Hurrah---no more manual verification of that PASS column. This means that "make test" can actually be a useful part of the release process now, (since it will exit with non-zero status if there are any failures).
2010-04-16 00:18:30 +02:00
echo ""
echo "Notmuch test suite complete."
Add a very rough beginning of a test suite. This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
2010-01-23 19:36:39 +01:00
make test: Actually count and report on failures. Hurrah---no more manual verification of that PASS column. This means that "make test" can actually be a useful part of the release process now, (since it will exit with non-zero status if there are any failures).
2010-04-16 00:18:30 +02:00
if [ "$test_failures" = "0" ]; then
echo "All $tests tests passed."
rm -rf ${TEST_DIR}
else
echo "$test_failures/$tests tests failed. The failures can be investigated in:"
echo "${TEST_DIR}"
fi
echo ""
exit $test_failures
Reference in a new issue Copy permalink