This is similar to the case of toplevel messages. Currently everything
is cached, so no database access is necessary. This might change in
the future, but it should not crash in either case.
The goal of this abstraction was to save space. But that failed as
the result actually was that four trivial lines got replace with 15
fairly complicated lines. The opposite of what it was supposed to
do.
Also it made it harder to come up with the fix in the previous commit;
simply grepping for the relevant symbols did not work because they get
constructed at run-time instead of appearing in the source file.
Starting with Emacs 27 undeclared variables in evaluated interactive
code uses lexical scope. This includes code passed with '--eval' as
we do in the Emacs tests, which also happen to assume dynamic scope.
- This can affect variables defined by libraries that we use. We
let-bind such variables to change the behavior of functions which we
then call with these bindings in effect. If these libraries are not
loaded beforehand, then the bindings are lexical and fail to have
the effect we intended.
At this time only 'smtpmail' has to be loaded explicitly (for the
variables let-bound in emacs_deliver_message and emacs_fcc_message).
'message' doesn't have to be loaded explicitly, because loading
'notmuch' (in 'run_emacs') already takes care of that, indirectly.
- Our own testing-only variables also have to be declared explicitly.
We should have done that anyway, but because of how and where these
variables are used it was very easy to overlook that (i.e. it isn't
something the byte-compiler ever looks at). Not so in Emacs 27
anymore; here this oversight caused four tests to fail.
The numeric values of these variables get incremented by functions
that we add to hooks that are run by many tests, not just the tests
where we actually inspect the value and therefore take care to let-
bind the values to 0 before we begin. The global values therefore
have to be numeric values as well. I have chosen -100 instead of 0
as the default in case someone writes a test that inspects the value
but forgets to let-bind the value. I hope that the unusual negative
value that one is going to see in such a case will help debugging
the issue.
The API docs promise to handle relative filenames, but the code did
not do it.
Also check for files outside the mail root, as implied by the API
description.
This fixes the bug reported at
id:87sgdqo0rz.fsf@tethera.net
Xapian currently succeeds to begin/end a transaction on a closed database,
or at least does not throw an exception. Make the test robust against
this changing.
The original generic handler had an extra '%s' in the format
string. Update tests that failed to catch this because the template to
print status strings checked 'stat', which was not set.
This is actually one of the few potentially useful things you can do
with a message belonging to a closed database, since in principle you
could re-open the database.
It's not very nice to return FALSE for an error, so provide
notmuch_message_get_flag_st as a migration path.
Bump LIBNOTMUCH_MINOR_VERSION because the API is extended.
Instead of printing the same static string for each test, can replace
the assert with something simpler (or at least easier to integrate
into the test suite).
These are less crucial since we stopped generating new database
versions and relied primarily on features. They also rely on a
pre-generated v1 database which happens to be chert format. This
backend is not supported by Xapian 1.5.
Also drop the tool gen-testdb.sh, which is currently broken, due to
changes in the testing infrastructure.
I haven't traced the code path as exhaustively for the SMIME test, but
the expiry date in question is larger then representable in a signed
32 bit integer.
This is a simple hack to enable out-of-tree builds, a concern raised
by Tomi in id:m24kzjib9a.fsf@guru.guru-group.fi
This change at least enables "make check" to complete without error,
but I'm sure it could be improved. I am not expert enough in
setuptools to know how.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Amended by db per id:87d06usa31.fsf@powell.devork.be
json_check_nodes.py exists in source tree, not in out of tree
build tree. Added -B to the execution so source tree is not
"polluted" by a .pyc file when json_check_nodes.py is executed.
When creating run_emacs.sh make it load .elc files from out of
tree build tree, not from source tree if such files existed.
If existed, those may be outdated, or even created by some other
emacs than the one that was used to build .elc files in out of
tree build dir.
This change means we can support "notmuch show --decrypt=true" for
S/MIME encrypted messages, resolving several outstanding broken tests,
including all the remaining S/MIME protected header examples.
We do not yet handle indexing the cleartext of S/MIME encrypted
messages, though.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
When composing a reply, no one wants to see this line in the proposed
message:
Non-text part: application/pkcs7-mime
So we hide it, the same way we hide PGP/MIME cruft.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Until we did PKCS#7 unwrapping, no leaf MIME part could have a child.
Now, we treat the unwrapped MIME part as the child of the PKCS#7
SignedData object. So in that case, we want to show it instead of
deliberately omitting the content.
This fixes the test of the protected subject in
id:smime-onepart-signed@protected-headers.example.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Unwrap a PKCS#7 SignedData part unconditionally when the cli is
traversing the MIME tree, and return it as a "child" of what would
otherwise be a leaf in the tree.
Unfortunately, this also breaks the JSON output. We will fix that
next.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
When we are indexing, we should treat SignedData parts the same way
that we treat a multipart object, indexing the wrapped part as a
distinct MIME object.
Unfortunately, this means doing some sort of cryptographic
verification whose results we throw away, because GMime doesn't offer
us any way to unwrap without doing signature verification.
I've opened https://github.com/jstedfast/gmime/issues/67 to request
the capability from GMime but for now, we'll just accept the
additional performance hit.
As we do this indexing, we also apply the "signed" tag, by analogy
with how we handle multipart/signed messages. These days, that kind
of change should probably be done with a property instead, but that's
a different set of changes. This one is just for consistency.
Note that we are currently *only* handling signedData parts, which are
basically clearsigned messages. PKCS#7 parts can also be
envelopedData and authEnvelopedData (which are effectively encryption
layers), and compressedData (which afaict isn't implemented anywhere,
i've never encountered it). We're laying the groundwork for indexing
these other S/MIME types here, but we're only dealing with signedData
for now.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
When checking cryptographic signatures, Notmuch relies on GMime to
tell it whether the certificate that signs a message has a valid User
ID or not.
If the User ID is not valid, then notmuch does not report the signer's
User ID to the user. This means that the consumer of notmuch's
cryptographic summary of a message (or of its protected headers) can
be confident in relaying the reported identity to the user.
However, some versions of GMime before 3.2.7 cannot report Certificate
validity for X.509 certificates. This is resolved upstream in GMime
at https://github.com/jstedfast/gmime/pull/90.
We adapt to this by marking tests of reported User IDs for
S/MIME-signed messages as known-broken if GMime is older than 3.2.7
and has not been patched.
If GMime >= 3.2.7 and certificate validity still doesn't work for
X.509 certs, then there has likely been a regression in GMime and we
should fail early, during ./configure.
To break out these specific User ID checks from other checks, i had to
split some tests into two parts, and reuse $output across the two
subtests.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Several functions in test/test-lib.sh used variable names that are
also used outside of those functions (e.g. $output and $expected are
used in many of the test scripts), but they are not expected to
communicate via those variables.
We mark those variables "local" within test-lib.sh so that they do not
get clobbered when used outside test-lib.
We also move the local variable declarations to beginning of each
function, to avoid weird gotchas with local variable declarations as
described in https://tldp.org/LDP/abs/html/localvar.html.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
strncmp looks for a prefix that matches, which is very much not what
we want here. This fixes the bug reported by Franz Fellner in
id:1588595993-ner-8.651@TPL520
In id:1588595993-ner-8.651@TPL520 Franz Fellner reported that tags
starting with 'attachment' are removed by 'notmuch reindex'. This is
probably related to the use of STRNCMP_LITERAL in
_notmuch_message_remove_indexed_terms.
GPGME has a strange failure mode when it is in offline mode, and/or
when certificates don't have any CRLs: in particular, it refuses to
accept the validity of any certificate other than a "root" cert.
This can be worked around by setting the `disable-crl-checks`
configuration variable for gpgsm.
I've reported this to the GPGME upstream at
https://dev.gnupg.org/T4883, but I have no idea how it will be
resolved. In the meantime, we'll just work around it.
Note that this fixes the test for verification of
id:smime-multipart-signed@protected-headers.example, because
multipart/signed messages are already handled correctly (one-part
PKCS#7 messages will get fixed later).
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Add a simple S/MIME SignedData message, taken from an upcoming draft
of
https://datatracker.ietf.org/doc/draft-autocrypt-lamps-protected-headers/
RFC 8551 describes a SignedData, a one-part clearsigned object that is
more resistant to common patterns of MTA message munging than
multipart/signed (but has the downside that it is only readable by
clients that implement S/MIME).
To make sure sure notmuch can handle this kind of object, we want to
know a few things:
Already working:
- Is the content of the SignedData object indexed? It actually is
right now because of dumb luck -- i think we're indexing the raw
CMS object and it happens to contain the cleartext of the message
in a way that we can consume it before passing it on to Xapian.
- Are we accidentally indexing the embedded PKCS#7 certificates? We
don't want to, and for some reason I don't understand, our indexing
is actually skipping the embedded certificates already. That's
good!
Still need fixing:
- do we know the MIME type of the embedded part?
- do we know that the message is signed?
- can notmuch-show read its content?
- can notmuch-show indicate the signature validity?
- can notmuch-reply properly quote and attribute content?
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
This test does exactly what it says on the tin. It expects JSON data
to be parseable by Python, at least.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
These tests describe some simple behavior we would expect to work if
we were to correctly index the cleartext of encrypted S/MIME messages
(PKCS#7 envelopedData).
Of course, they don't currently pass, so we mark them known-broken.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
