Previously, the test framework generated a variable name for each
external prereq as a poor man's associative array. Unfortunately,
prereqs names may not be legal variable names, leading to
unintelligible bash errors like
test_missing_external_prereq_emacsclient.emacs24_=t: command not found
Using proper associative arrays to track prereqs, in addition to being
much cleaner than generating variable names and using grep to
carefully construct unique string lists, removes restrictions on
prereq names.
Previously, if a test script aborted (e.g., because it passed too few
arguments to a test function), the test driver loop would simply
continue on to the next test script and the final results would
declare that everything passed (except that the test count would look
suspiciously low, but maybe you just misremembered how many tests
there were).
Now, if a test script exits with a non-zero status and did not produce
a final results file, we propagate that failure out of the driver loop
immediately.
To keep this simple, this patch removes the PID from the test-results
file name. This PID was inherited from the git test system and seems
unnecessary, since the file name already includes the name of the test
script and the test-results directory is created anew for each run.
And require that if TEST_EMACS is specified, so is TEST_EMACSCLIENT.
Previously, the test framework always used "emacsclient", even if the
Emacs in use was overridden by TEST_EMACS. This causes problems if
both Emacs 23 and Emacs 24 are installed, the Emacs 23 emacsclient is
the system default, but TEST_EMACS is set to emacs24. Specifically,
with an Emacs 24 server and an Emacs 23 client, emacs tests that run
very quickly may produce no output from emacsclient, causing the test
to fail.
The Emacs server uses a very simple line-oriented protocol in which
the client sends a request to evaluate an expression and the server
sends a request to print the result of evaluation. Prior to Emacs bzr
commit 107565 on March 11th, 2012 (released in Emacs 24.1), if
multiple commands were sent to the emacsclient between when it sent
the evaluation command and when it entered its receive loop, it would
only process the first response command, ignoring the rest of the
received buffer. This wasn't a problem with the Emacs 23 server
because it sent only the command to print the evaluation result.
However, the Emacs 24 server first sends an unprompted command
specifying the PID of the Emacs server, then processes the evaluation
request, then sends the command to print the result. If the
evaluation is fast enough, it can send both of these commands before
emacsclient enters the receive loop. Hence, if an Emacs 24 server is
used with an Emacs 23 emacsclient, it may miss the response printing
command, ultimately causing intermittent notmuch test failures.
Previously, many tests in emacs-subject-to-filename didn't quote the
$output argument to test_expect_equal. As a result, if $output was
empty, test_expect_equal would be passed only one argument and would
abort the entire test script. By quoting the argument, we ensure
test_expect_equal will always receive two arguments.
We now test for user ignore patterns before attempting to determine if
a directory entry is itself a directory. As a result, we no longer
abort for broken symlinks if the user has explicitly ignored them.
This fixes the test added in the previous patch. It also slightly
changes the debug output checked by another test of ignores.
The macro with-current-notmuch-show-message executes command
`notmuch show --format=raw id:...` which just outputs the contents
of the mail file verbatim (into temporary buffer). In case e.g. utf-8
locale is used the temporary buffer has buffer-file-coding-system as
utf-8. In this case Emacs converts the data to multibyte format, guessing
that input is in utf-8.
However, the "raw" (MIME) message may contain octet data in any other
8bit format, and as no (MIME-)content spesific handling to the message
is done at this point, conversion to other formats may lose information.
By setting coding-system-for-read 'no-conversion drops the conversion part
and makes this handle input as notmuch-get-bodypart-internal() does.
This marks the broken test in previous change fixed.
Previously, we would treat multi-message mboxes as one giant email,
which, besides the obvious incorrect indexing, often led to
out-of-memory errors for archival mboxes. Now we explicitly reject
multi-message mboxes. For historical reasons, we retain support for
single-message mboxes, but official deprecate this behavior.
This test is currently broken. Note that its brokenness cascades and
causes the next test to fail as well (because notmuch incorrectly
indexes the mbox file).
There are currently 45 TESTS scripts. 36 of those load
test-lib.sh using '. ./test-lib.sh' and 9 '. test-lib.sh'.
In latter case test-lib.sh is first searched from directories
in PATH (posix) and then from current directory (bash feature).
Changed the 9 files to execute '. ./test-lib.sh'. The test-lib.sh
should never be loaded from directory in PATH.
Previously, this would simply indicate that the grep failed without
any indication of the Emacs output it failed on. Now we take
advantage of the test framework's handling of stdout to display the
incorrect Emacs output if the test fails.
It seems we have never tested the case that restore --accumulate
actually adds tags. I noticed this when I started optimizing and no
tests failed.
The bracketing with "restore --input=dump.expected" are to make sure
we start in a known state, and we leave the database in a known state
for the next test.
The test designed to exercise Emacs' rendering of HTML emails
containing images inadvertently assumed w3m was available under Emacs
23. The real point of this test was to check that Emacs 24's shr
renderer didn't crash when given img tags, so use shr if it's
available, html2text otherwise (which is built in), and do only a
simple sanity check of the result.
This regexp agrees with Xapian query syntax much more closely, though
we specifically disallow various cases that would be confusing in the
context of an email body (e.g., punctuation at the end of an id: link
is not considered part of the id: link because it's probably part of
the surrounding text).
In particular, this handles id: links that are not surrounded by
quotes much better, which stash is much more likely to generate now
that we don't quote id's that don't need to be quoted. It also
handles quoted id: links better.
We update the buttonization test to reflect the new pattern.
This matches the current behavior of the buttonizer, so it passes, but
many of these cases are not what you'd want (and some of them aren't
even valid Xapian queries). The next patch will fix the handling of
these cases and update the test.
Over time, maintaining this very long regex has become irritating,
especially when resolving conflicts.
This patch replaces the call to sed with multiple extra arguments to
find. Since each test binary is now on it's own line, this should
make resolving conflicts easier.
Test the date/time parser module directly, independent of notmuch,
using the parse-time test tool.
Credits to Michal Sojka <sojkam1@fel.cvut.cz> for writing most of the
tests.
Add a smoke testing tool to support testing the date/time parser
module directly and independent of the rest of notmuch.
Credits to Michal Sojka <sojkam1@fel.cvut.cz> for the stdin parsing
idea and consequent massive improvement in testability.
Currently, we only properly escape stashed id queries, but there are
other places where the Emacs UI constructs queries for boolean terms.
Since this escaping function is meant to be used in other places, it
avoids escaping strings that don't need escaping.
This disallows adding empty tags, since nothing but confusion follows
in their wake, and disallows adding tags that begin with "-" because
they are also confusing, the tag "-" is impossible to remove using the
CLI, and because the syntax for removing such tags conflicts with long
argument syntax.
This does not place any restrictions on what tags can be removed, as
that would make it difficult for people who have the misfortune of
already having malformed tags to remove these tags.
Although messages are created in a particular order, it seems that
when they are created on a tmpfs, they do not always come back in the
same order, leading to the same files being ignored but being output
in a different order. This causes the test to fail because the outputs
being compared are the same.
Fix the failures by sorting the output of notmuch --debug and
comparing this to a hand-sorted version of its output.
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
The use of --background option (instead of shell '&') ensures that
smtp-dummy is listening its server socket until execution of shell
script can continue, thus the client will always have socket where
to connect.
smtp-dummy outputs smtp_dummy_pid variable in shell assignment format;
eval'ing that output makes that variable available for the shell.
As the smtp-dummy instance is no longer child process of the script
the SIGKILL signal sent to it will ensure it is going away in case
the mail sender fails to connect to smtp-dummy.
When shell executes background process using '&' the scheduling of
that new process is arbitrary. It could be that smtp-dummy doesn't
get execution time to listen() it's server socket until some other
process attempts to connect() to it. The --background option in
smtp-dummy makes it to go background *after* it started to listen
its server socket.
When --background option is used, the line "smtp_dummy_pid='<pid>'"
is printed to stdout from where shell can eval it.
Demonstrates that *every* file/directory which matches one of the values
in 'new.ignore' will be ignored, independent of its depth/location in
the mail store.
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Obviates the need to create a 'NOTMUCH_NEW' clone which runs
'notmuch new --debug'. This will be used in a later patch.
Doesn't cause any issues for other tests.
* test/emacs:
- New subtest "notmuch-show: collapse all messages in thread":
`notmuch-show-open-or-close-all' with prefix arg ("C-u M-RET")
collapses all messages in thread.
- New subtest "notmuch-show: uncollapse all messages in thread":
`notmuch-show-open-or-close-all' without prefix arg ("M-RET")
uncollapses all messages in thread.
* test/emacs:
- New subtest "notmuch-show: show message headers":
Setting `notmuch-message-headers-visible' to t causes all headers
defined in `notmuch-message-headers' to be shown.
- New subtest "notmuch-show: hide message headers":
Setting `notmuch-message-headers-visible' to nil causes all headers
defined in `notmuch-message-headers' to be hidden.
("Subject:" may be an exception; See the use of `headers-start' in
`notmuch-show-insert-msg')
- New subtest "notmuch-show: hide message headers (w/ notmuch-show-toggle-headers)":
Setting `notmuch-message-headers-visible' to t causes all headers
defined in `notmuch-message-headers' to be shown, but they can be
hidden for the current message by running `notmuch-show-toggle-headers'.
This requires changing the contents of the crypto tests, as one thread
that was marked read by the earlier tests in test/emacs is no longer
marked read.
This moves tests for:
- 09d19ac "test: emacs: toggle eliding of non-matching messages in
`notmuch-show'", which should have actually read: "test: emacs:
toggle processing of cryptographic MIME parts in `notmuch-show'".
See commit 19ec74c5.
- 5ea1dbe "test: emacs: toggle eliding of non-matching messages in
`notmuch-show'"
- 345faab "test: emacs: toggle thread content indentation in
`notmuch-show'"
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Since $TEST_DIRECTORY is an absolute path, any filenames generated
with it will be complete paths. Only use the basename to generate
suffixes for filenames.
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Most Emacs tests end with a call to (test-output), which saves the
buffer to a filed called OUTPUT. Previously, if the test code failed
with an exception before this call, the test framework would then
compare against the OUTPUT file from the last Emacs test, resulting in
confusing diffs.
This requires one tweak to an emacs test that made two calls to
test_emacs and expected an OUTPUT file from the first call. We simply
reverse the order of the test_emacs calls.
