If either the moreutils or GNU parallel utility are available, run all
tests in parallel. On my eight core machine this makes for a ~x7
speed-up in the full test suite (1m24s -> 12s).
The design of the test suite makes this parallelization trivial.
The add_email_corpus test utility includes logic that tries to re-use
an index of the corpus if available. This was seemingly done as an
optimization, so that every test that uses the corpus didn't have to
create it's own index of the corpus. However, this has the perverse
side effect of entangling tests together, and breaks parallelization.
Forcing each test to do it's own index does increase the overall time
of the test slightly (~6%), but this will be more than made up for in
the next patch that introduces paraellization.
The typical use case for gpg is that if you control a secret key, you
mark it with "ultimate" ownertrust.
The opaque --import-ownertrust mechanism is GnuPG's standard mechanism
to set up ultimate ownertrust (the ":6:" means "ultimate", for
whatever reason).
We adjust the test suite to match this change, inverting the sense of
one test: since the default is now that the user ID of the suite's own
key is valid, we change the test to make sure that the user ID is not
emitted when it is *not* valid.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
The user ID on the self-test is a little bit clunky-looking. It also
may end up showing up elsewhere in the test suite. Centralizing the
user ID in one place should make it easier to handle if it ever
changes, and should make tests easier to read.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
This is a subtle difference, but the output of notmuch shouldn't ever
change based on ownertrust itself -- notmuch is intended to show valid
User IDs, and to avoid showing invalid User IDs.
It so happens that setting ownertrust of a key to ultimate sets all
associated user IDs to "full" validity, so the test is correct, but
just misnamed.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Previously if the input was exactly a multiple of the internal buffer
size, notmuch would attempt to fwrite nothing to stdout, but still
expected fwrite to return 1, causing a failure that looked like this:
$ notmuch show --format=raw id:87o96f1cya.fsf@codeaurora.org
...entire message shown as expected..
Error: Write failed
$ echo $?
1
To fix the problem don't call fwrite at all when there's nothing to
write.
Amended by db: add some tests of message sizes likely to cause this
problem.
This drops "file" from mime_node_context and just uses a local
variable. It also uses the new gzip aware utility routines recently
added to util/gmime-extra.c. The use of gzopen / gzfile in addition is
a bit icky, but the choice is between that, and providing yet another
readline implimentation that understands GMime streams.
Rather than storing the lower level stdio FILE object, we store a
GMime stream. This allows both transparent decompression, and passing
the stream into GMime for parsing. As a side effect, we can let GMime
close the underlying OS stream (indeed, that stream isn't visible here
anymore).
This change is enough to get notmuch-{new,search} working, but there is still
some work required for notmuch-show, to be done in a following commit.
note that "notmuch-show for message with invalid From" is still broken
in T310-emacs.sh. It would be good to debug what's going on there and
try to get it fixed!
signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Note that we do keep ignoring the gpg_path configuration option,
though, to avoid breakage of existing installations. It is ignored
like any other unknown configuration option, but we at least document
that it is ignored so that people who find it in their legacy configs
can know that it's safe to drop.
signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Unsigned encrypted mail shows up with a weird empty signature list.
If we successfully decrypted and there was no signature in it, we
should just not show a sigstatus at all.
The documentation for g_mime_decrypt_result_get_signatures says:
a GMimeSignatureList or NULL if the stream was not signed.
If a test has added a GnuPG homedir, it may well want to know the
fingerprint. This saves us from having to redefine this magic string
in multiple places when more tests eventually use the GnuPG homedir.
The new `body:` field (in Xapian terms) or prefix (in slightly
sloppier notmuch) terms allows matching terms that occur only in the
body.
Unprefixed query terms should continue to match anywhere (header or
body) in the message.
This follows a suggestion of Olly Betts to use the facility (since
Xapian 1.0.4) to add the same field with multiple prefixes. The double
indexing of previous versions is thus replaced with a query time
expension of unprefixed query terms to the various prefixed
equivalent.
Reindexing will be needed for 'body:' searches to work correctly;
otherwise they will also match messages where the term occur in
headers (demonstrated by the new tests in T530-upgrade.sh)
Add test of forwarding messages from within emacs.
The first test checks that a references header is properly
added to the new message. The second test checks that the
send-hook of the forwarding message adds a forwarded-tag
to the original message.
Thanks to plujon for pointing out this problem on IRC. The underlying
issue is that the quotes are stripped before the field processors get
the query string, and the heuristic for putting them back is not quite
right.
The exact error messages returned by regerror() aren't standardized;
relying on them isn't portable. Thus, add a a prefix to make clear that
the subsequent message is a regexp parsing error, and only look for this
prefix in the test suite, ignoring the rest of the message.
POSIX doesn't specify the flushing behaviour of the STDOUT stream, so
it's invalid to assume a particular order between the stdout and stderr
output. The current test breaks on musl due to this.
I can't figure out how checking the sign of a bool ever worked. The
following program demonstrates the problem (i.e. for me it prints 1).
#include <stdio.h>
#include <stdbool.h>
int main(int argc, char **argv) {
bool x;
x = -1;
printf("x = %d\n", x);
}
This seems to be mandated by the C99 standard 6.3.1.2.
When generating a reply message, if the user was the originator and
only recipient of the original message, include the user as a
recipient of the reply.
b31e44c678 introduced message-id-parse
as a new binary created by the test suite. It shows up as something
additional to git, but git ought to know to ignore it.
As reported by Sean Whitton, there are mailers (in particular the
Debian Bug Tracking System) that have sensible In-Reply-To headers,
but un-useful-for-notmuch References (in particular with the BTS, the
oldest reference is last). I looked at a sample of about 200K
messages, and only about 0.5% these had something other than a single
message-id in In-Reply-To. On this basis, if we see a single
message-id in In-Reply-To, consider that as authoritative.
The idea is that if a message-id parses with this function, the MUA
generating it was probably sane, and in particular it's probably safe
to use the result as a parent from In-Reply-to.
In a future commit, we will start trusting In-Reply-To's when they
look sane (i.e. a single message-id). Modify these tests so they will
keep passing (i.e. keep choosing References) when that happens.
The current scheme of choosing the replyto (i.e. the default parent
for threading purposes) does not work well for mailers that put
the oldest Reference last.
We (finally) implement the XXX comment. It requires a bit of care not
to reparent all of the possible toplevel messages.
_notmuch_messages_has_next is not ready to be a public function yet,
since it punts on the mset case. We know in the one case it is called,
the notmuch_messages_t is just a regular list / iterator.
This is mainly to lay out the structure of the final code. The problem
isn't really solved yet, although some very simple cases are
better (hence the fixed test). We need two passes through the messages
because we need to be careful not to re-parent too many messages and
end up without any toplevel messages.
For non-root messages, this should not should anything currently, as
the messages are already added in date order. In the future we will
add some non-root messages in a second pass out of order and the
sorting will be useful. It does fix the order of multiple
root-messages (although it is overkill for that).
This documents the bug discussed at
id:87efgmmysi.fsf@len.workgroup
The underlying issue is that the reply to a ghost (missing) message is
falsely classified as a root message in _resolve_thread_relationships.
There are two pairs of tests; in each case the the first test is
simpler / more robust, but also easier to fool.
There are 3 threads here, two synthetic, and one anonymized one using
data from Gregor. They test various aspects of thread
ordering/construction in the presence of replies to ghost messages.
This clarifies that the breakage seen with Xapian 1.4.6 does not have
to do with "funny" tags.
This test is "known broken", but only with xapian 1.4.6, and there's
curently no convenient way to mark that.
Messages that contain Windows-1252 are frequently mislabeled as ISO
8859-1, which may result in non-printable characters when displaying
the message. The test asserts that such characters (in this case
curved quotes) are displayed correctly.
Add fancy new feature, which makes "notmuch show" capable of actually
indexing messages that it just decrypted.
This enables a workflow where messages can come in in the background
and be indexed using "--decrypt=auto". But when showing an encrypted
message for the first time, it gets automatically indexed.
This is something of a departure for "notmuch show" -- in particular,
because it requires read/write access to the database. However, this
might be a common use case -- people get mail delivered and indexed in
the background, but only want access to their secret key to happen
when they're directly interacting with notmuch itself.
In such a scenario, they couldn't search newly-delivered, encrypted
messages, but they could search for them once they've read them.
Documentation of this new feature also uses a table form, similar to
that found in the description of index.decrypt in notmuch-config(1).
A notmuch UI that wants to facilitate this workflow while also
offering an interactive search interface might instead make use of
these additional commands while the user is at the console:
Count received encrypted messages (if > 0, there are some things we
haven't yet tried to index, and therefore can't yet search):
notmuch count tag:encrypted and \
not property:index.decryption=success and \
not property:index.decryption=failure
Reindex those messages:
notmuch reindex --try-decrypt=true tag:encrypted and \
not property:index.decryption=success and \
not property:index.decryption=failure
