The add_email_corpus test utility includes logic that tries to re-use
an index of the corpus if available. This was seemingly done as an
optimization, so that every test that uses the corpus didn't have to
create it's own index of the corpus. However, this has the perverse
side effect of entangling tests together, and breaks parallelization.
Forcing each test to do it's own index does increase the overall time
of the test slightly (~6%), but this will be more than made up for in
the next patch that introduces paraellization.
The typical use case for gpg is that if you control a secret key, you
mark it with "ultimate" ownertrust.
The opaque --import-ownertrust mechanism is GnuPG's standard mechanism
to set up ultimate ownertrust (the ":6:" means "ultimate", for
whatever reason).
We adjust the test suite to match this change, inverting the sense of
one test: since the default is now that the user ID of the suite's own
key is valid, we change the test to make sure that the user ID is not
emitted when it is *not* valid.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
The user ID on the self-test is a little bit clunky-looking. It also
may end up showing up elsewhere in the test suite. Centralizing the
user ID in one place should make it easier to handle if it ever
changes, and should make tests easier to read.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
note that "notmuch-show for message with invalid From" is still broken
in T310-emacs.sh. It would be good to debug what's going on there and
try to get it fixed!
signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
If a test has added a GnuPG homedir, it may well want to know the
fingerprint. This saves us from having to redefine this magic string
in multiple places when more tests eventually use the GnuPG homedir.
This way, one can build for a different Ruby than $PATH/ruby
(e. g. different versions, or Ruby in other paths).
Signed-off-by: Thomas Schneider <qsx@chaotikum.eu>
the command-line interface for indexing (reindex, new, insert) used
--try-decrypt; and the configuration records used index.try_decrypt.
But by comparison with "show" and "reply", there doesn't seem to be
any reason for the "try" prefix.
This changeset adjusts the command-line interface and the
configuration interface.
For the moment, i've left indexopts_{set,get}_try_decrypt alone. The
subsequent changeset will address those.
In 8e7fb88237 Jani replaced the use of $(pwd -P) to find the
TEST_DIRECTORY in order to better support out of tree
builds. Unfortunately the performance-tests need a different value for
the variable and were thus broken.
This commit splits out the setting of this variable for the two sets
of tests. Performance tests still don't work out of tree, because
the handling of the downloaded corpus needs to be updated.
Subsequent patches may want to send GNU-style --long-arguments to
notmuch new in the test suite, in particular when invoking
emacs_fcc_message. This changeset makes that possible.
The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
running each test when the current working directory is not the test
subdirectory. Tests with failures would leave their TMP_DIRECTORY
directory behind for debugging, and repeated out-of-tree test runs
would have old temp directories. (This lead to e.g. T310-emacs.sh
hanging because emacs would prompt for overwriting files.)
We remove the likely anyway defunct --root test option while at it,
just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
Changed "" quotes to '' as we're not supposed to dynamically
alter python program (via shell $variable expansion).
Added space to python program to match general python style.
Replaced $* with 'idiomatic' "$@" to serve as better example.
In [1], Vladimir Panteleev observed that the In-Reply-To and
References headers could be wrapped in the 'default' output format of
notmuch-reply, depending on the version of Emacs creating the
message. In my own experiments notmuch-reply sometimes wraps headers
with only one message-id if that message-id is long enough. However it
happens, this causes the previous approach using grep to fail.
Since I found the proposed unwrapping shell fragment in [1] a bit hard
to follow, I decided to write a little python script instead. Then
Tomi suggested a slight generalization of my script, and here we are.
[1] id:20170817175145.3204-7-notmuch@thecybershadow.net
json.tool does not sort or otherwise normalize the order of JSON keys
in its output, which can result in test failures on some test systems.
Instead, use a one-line Python script passed to the interpreter
directly on its command line. Use sort_keys=True for json.dump to
ensure the key order is normalized. The script works with both Python
2 and 3.
* test/test-lib.sh: Update test_expect_equal_json.
New function at_exit_function registers given function to be called
at script termination.
Functions so registered are called in the reverse order of their
registration; no arguments are passed.
Function is called only once; re-adding with function name already
registered will remove previous registration.
New function rm_exit_function can be used to remove registration.
Modules (and possibly test-lib.sh functions) in future commits will
register such functions.
Unify the subtests by requiring test_begin_subtest before
test_expect_code. (Similar change for test_expect_success has already
been done.)
This increases clarity in the test scripts by having a separate line
for the start of the subtest with the heading, and makes it possible
to simplify the test infrastructure by making all subtests similar.
Unify the subtests by requiring test_begin_subtest before
test_expect_success. (Similar change for test_expect_code will
follow.)
This increases clarity in the test scripts by having a separate line
for the start of the subtest with the heading, and makes it possible
to simplify the test infrastructure by making all subtests similar.
The only place where we use the implicit prereq check is T000-basic.sh
where we check that it works. It's an added complication that we don't
use. Remove it.
The test_have_prereq function can still be used for the same effect in
subtests that use test_begin_subtest. For now, this will make it
impossible to have prereqs in one-line subtests that don't require
test_begin_subtest. This will be fixed in follow-up work.
Known broken tests are, well, known broken. Do not print the result
diff for them unless V=1 is specified. Now that the test description
is printed also when known broken tests fail, the user can also skip
to running the individual failing tests.
We already use this directory for dtach sockets, so it makes sense to
put gnupg sockets there as well. There doesn't seem to be a clean way
to put a fully functional socket in a different location than
GNUPGHOME.
This reverts commit e7b88e8b0a.
It turns out that this does not work well in environments without a
running systemd (or some other provider of /run/user)
Instead of just having the first filename for the message, list all
duplicate filenames of the message as a list in the formatted
outputs. This bumps the format version to 3.
$NOTMUCH_PYTHON is sourced from sh.config, configured by
./configure and stated to be used as:
"Name of python command to use in configure and the test suite."
This enables the shortened socket pathes in /run or equivalent. The
explicit call to gpgconf is needed for nonstandard GNUPGHOME settings.
(amended according to id:m2fujatr4k.fsf@guru.guru-group.fi)
GnuPG 2.1.16 is now injecting the full issuer fingerprint in its
signatures, which makes them about 32 octets larger when
ascii-armored.
This change in size means that the size of the MIME parts will vary
depending on the version of gpg that the user has installed. at any
rate, the signature part should be non-zero (this is true for
basically any MIME part), so we just test for that instead of an exact
size.
Moved the 2 basename(1) executions to the test failure branch in
test_expect_equal_file ().
The output of basename(1) executions in function test_expect_equal_file ()
are only used when tests fails -- when all tests pass these 2 basename(1)
executions are no longer done at all.
In case of the test script is to be relaunced under valgrind, or --tee
is requested, use the $BASH shell variable to locate the command
interpreter. The $SHELL variable is re-set by non-interactive shells
so in case the shell uses some other shell (e.g. zsh) for interactive
use these bash scripts continue to work.
We largely use the corpus under test/corpus for
testing. Unfortunately, many of our tests have grown to depend on
having exactly this set of messages, making it hard to add new message
files for testing specific cases.
We do use a lot of add_message from within the tests, but it's not
possible to use that for adding broken messages, and adding several
messages at once can get unwieldy.
Move the basic corpus under tests/corpora/default, and make it
possible to add new, independent corpora along its side. This means
tons of renames with a few tweaks to add_email_corpus function in
test-lib.sh to let tests specify which corpus to use.
