Add a simple S/MIME SignedData message, taken from an upcoming draft
of
https://datatracker.ietf.org/doc/draft-autocrypt-lamps-protected-headers/
RFC 8551 describes a SignedData, a one-part clearsigned object that is
more resistant to common patterns of MTA message munging than
multipart/signed (but has the downside that it is only readable by
clients that implement S/MIME).
To make sure sure notmuch can handle this kind of object, we want to
know a few things:
Already working:
- Is the content of the SignedData object indexed? It actually is
right now because of dumb luck -- i think we're indexing the raw
CMS object and it happens to contain the cleartext of the message
in a way that we can consume it before passing it on to Xapian.
- Are we accidentally indexing the embedded PKCS#7 certificates? We
don't want to, and for some reason I don't understand, our indexing
is actually skipping the embedded certificates already. That's
good!
Still need fixing:
- do we know the MIME type of the embedded part?
- do we know that the message is signed?
- can notmuch-show read its content?
- can notmuch-show indicate the signature validity?
- can notmuch-reply properly quote and attribute content?
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
This test does exactly what it says on the tin. It expects JSON data
to be parseable by Python, at least.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
These tests describe some simple behavior we would expect to work if
we were to correctly index the cleartext of encrypted S/MIME messages
(PKCS#7 envelopedData).
Of course, they don't currently pass, so we mark them known-broken.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
When consuming a signed+encrypted S/MIME message generated by emacs,
we expect to see the same cryptographic properties for the message as
a whole. This is not done correctly yet, so the test is marked as
known broken.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
These sample messages are taken directly from the Protected Headers
draft:
https://www.ietf.org/id/draft-autocrypt-lamps-protected-headers-02.html
Note that this commit doesn't strictly pass the common git pre-commit
hook due to introducing some trailing whitespace. That's just the
nature of the corpus, though. We should have that trailing
whitespace, so I've made this commit with --no-verify.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
This is taken from the same Internet Draft that test/smime/ca.crt
comes from. See that draft for more details.
https://www.ietf.org/id/draft-dkg-lamps-samples-02.html#name-pkcs12-object-for-bob
We don't use it yet, but it will be used to decrypt other messages in
the test suite.
Note that we include it here with an empty passphrase, rather than
with the passphrase "bob" that it is supplied with in the I-D. The
underlying cryptographic material is the same, but this way we can
import cleanly into gpgsm without having a passphrase set on it (gpgsm
converts an empty-string passphrase into no passphrase at all on
import).
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Without this fix, we couldn't run both add_gnupg_home and
add_gpgsm_home in the same test script.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
No functional change.
We no longer need to identify the key and cert to mml-mode when
sending an S/MIME message, so making a copy of key+cert.pem to
test_suite.pem is superfluous. Get rid of the extra file.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
The documentation for message mode clearly states that EasyPG (which
uses GnuPG) is the default and recommended way to use S/MIME with
mml-secure:
[0] https://www.gnu.org/software/emacs/manual/html_node/message/Using-S_002fMIME.html
To ensure that this mode works, we just need to import the secret key
in question into gpgsm in addition to the public key. gpgsm should be
able pick the right keys+certificates to use based on To/From headers,
so we don't have to specify anything manually in the #secure mml tag.
The import process from the OpenSSL-preferred form (cert+secretkey) is
rather ugly, because gpgsm wants to see a PKCS#12 object when
importing secret keys.
Note that EasyPG generates the more modern Content-Type:
application/pkcs7-signature instead of application/x-pkcs7-signature
for the detached signature.
We are also obliged to manually set gpgsm's include-certs setting to 1
because gpgsm defaults to send "everything but the root cert". In our
weird test case, the certificate we're using is self-signed, so it
*is* the root cert, which means that gpgsm doesn't include it by
default. Setting it to 1 forces inclusion of the signer's cert, which
satisfies openssl's smime subcommand. See https://dev.gnupg.org/T4878
for more details.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
This CA is useful for test suites and the like, but is not an
actually-secure CA, because its secret key material is also published.
I plan to use it for its intended purpose in the notmuch test suite.
It was copied from this Internet Draft:
https://tools.ietf.org/id/draft-dkg-lamps-samples-01.html#name-certificate-authority-certi
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
GnuPG's gpgsm, like gpg, should always be used with --batch when it is
invoked in a non-interactive environment.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
It turns out that putting inline functions in C header files is not a
good idea, and can cause linking problems if the compiler decides not
to inline them. In principle this is solvable by using a "static
inline" declaration, but this potentially makes a copy in every
compilation unit. Since we don't actually care about the performance
of this function, just use a non-inline function.
Starting with Emacs 27 the old `cl' implementation is finally
considered obsolete. Previously its use was strongly discouraged
at run-time but one was still allowed to use it at compile-time.
For the most part the transition is very simple and boils down to
adding the "cl-" prefix to some symbols. A few replacements do not
follow that simple pattern; e.g. `first' is replaced with `car',
even though the alias `cl-first' exists, because the latter is not
idiomatic emacs-lisp.
In a few cases we start using `pcase-let' or `pcase-lambda' instead
of renaming e.g. `first' to `car'. That way we can remind the reader
of the meaning of the various parts of the data that is being
deconstructed.
An obsolete `lexical-let' and a `lexical-let*' are replaced with their
regular variants `let' and `let*' even though we do not at the same
time enable `lexical-binding' for that file. That is the right thing
to do because it does not actually make a difference in those cases
whether lexical bindings are used or not, and because this should be
enabled in a separate commit.
We need to explicitly depend on the `cl-lib' package because Emacs
24.1 and 24.2 lack that library. When using these releases we end
up using the backport from GNU Elpa.
We need to explicitly require the `pcase' library because
`pcase-dolist' was not autoloaded until Emacs 25.1.
This test extracts values from a (key,value) map where multiple entries
can have the same key, and the entries are sorted by key, but not by
value. The test incorrectly assumes that the values will be sorted as
well, so sort the output.
Xapian 1.4 is over 3 years old now (1.4.0 released 2016-06-24),
and 1.2 has been deprecated in Notmuch version 0.27 (2018-06-13).
Xapian 1.4 supports compaction, field processors and retry locking;
conditionals checking compaction and field processors were removed
but user may want to disable retry locking at configure time so it
is kept.
Context: I am compiling notmuch on OpenBSD which has a rather old zlib
1.2.3. It seems that the behaviour of gzgets() changed slightly between
this version and more recent versions, but the manual does not reflect
that change. Note that zlib's manual:
- does not specify which error code (Z_OK or Z_STREAM_END) is set when
EOF is reached,
- does not indicate the meaning of Z_STREAM_END after gzgets(), but
based on its meaning as a possible return value of inflate(), I would
guess that it means EOF.
amended by db: tidy commit message
This is the non-deprecated way to use completing-read. Additionally
the old use was broken when using ivy for completing-read. For user's
using completing-read-default they won't see the default URL now, but
if they hit enter it will be visited. Alternatively they can select
it with M-n.
From the completing-read documentation for initial-input:
This feature is deprecated--it is best to pass nil for INITIAL-INPUT
and supply the default value DEF instead. The user can yank the
default value into the minibuffer easily using M-n.
Additionally collection is now all urls, rather than all but the
first. I'm not sure why "(cdr urls)" was previously done.
This is like notmuch-search-filter-by-tag, but creates a new search
rather than filtering the current search. We add this to
notmuch-common-keymap since this can be used by many contexts. We bind
to the key "t", which is the same key used by
notmuch-search-filter-by-tag in notmuch-search-mode-map. This is done
intentionally since the keybinding for notmuch-search-mode-map can be
seen as a specialization of creating a new search.
This change was motivated for use in "notmuch-hello". It is a more
convenient way to search a tag than expanding the list of all tags. I
also noticed many saved searches people use are simply tags.
'qsx' reported a bug on #notmuch with notmuch-dump and large stored
queries. This test will pass (on my machine) if the value of `repeat'
is made smaller.
Reported-By: Thomas Schneider <qsx@chaotikum.eu>
Add x and X binds to notmuch-tree for functionally that we have in
notmuch-show.
The notmuch-tree-quit binding is somewhat redundant, since it is
handled by notmuch-bury-or-kill-this-buffer which is bound to q.
Signed-off-by: William Casarin <jb55@jb55.com>
This functions removes some duplicate logic between
notmuch-tree-{next,prev}-matching-message
We do this because we will be adding some additional logic similar to
the notmuch-show-next-open-message function, and it will help if this
logic is all in one place.
Signed-off-by: William Casarin <jb55@jb55.com>
This function captures some common logic when jumping to matching
messages in notmuch-tree mode.
We also add a new return value (t or nil), that indicates if there was
a next matching message in the thread to show.
Signed-off-by: William Casarin <jb55@jb55.com>
nmbug and notmuch-report are developer tools. It's 2018, and all
developers should have python3 available.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Ship a new debian package for the notmuch2 CFFI-based Python interface
to notmuch.
Unlike the notmuch python module, the new notmuch2 module is no longer
arch-independent, because it builds and ships a shared object in
addition to the python code.
This patch encourages new downstream development to rely on notmuch2
instead of on notmuch, to get the benefits of the new module.
I welcome any suggested improvements to this packaging, but it appears
to me to be sufficient to get "import notmuch2" to work and do some
basic tests.
Since doxygen 1.8.16, MSCGEN_PATH and PERL_PATH are obsolete:
MSCGEN_PATH:
873e0ccfbe
PERL_PATH:
6d1535c38f
I don't think that the notmuch builds ever depended on them in the
first place, and including them in the default config yields the
following two warnings:
```
doxygen ./doc/doxygen.cfg
warning: Tag 'PERL_PATH' at line 267 of file './doc/doxygen.cfg' has become obsolete.
To avoid this warning please remove this line from your configuration file or upgrade it using "doxygen -u"
warning: Tag 'MSCGEN_PATH' at line 272 of file './doc/doxygen.cfg' has become obsolete.
To avoid this warning please remove this line from your configuration file or upgrade it using "doxygen -u"
```
Remove them to avoid the warnings.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
The -D flag to install (used in the Makefile) is GNU-specific and does
not work on BSD distributions (i.e. macOS). Likewise with the xargs -r
flag. These changes use portable alternatives to these flags while
preserving the exact behavior.
Without this change, we see the following warning when compiling the
elisp:
```
EMACS emacs/notmuch-crypto.elc
In end of data:
emacs/notmuch-crypto.el:266:1:Warning: the function
‘notmuch-show-get-message-id’ is not known to be defined.
```
Thanks to Örjan Ekeberg and David Edmondson for their followup about
this.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Avoid a memory leak in the notmuch command line.
gmime_multipart_encrypted_decrypt returns a GMimeObject marked by
GMime as "transfer full", so we are supposed to clean up after it.
When parsing a message, notmuch would leak one GMimeObject part per
multipart/encrypted MIME layer. We clean it up by analogy with
cleaning up the signature list associated with a MIME node.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
When walking the MIME tree, we might need to extract a new MIME
object. Thus far, we've only done it when decrypting
multipart/encrypted messages, but PKCS#7 (RFC 8551, S/MIME) has
several other transformations that warrant a comparable form of
unwrapping.
Make this member re-usable for PKCS#7 unwrappings as well as
multipart/encrypted decryptions.
This change is just a naming change, it has no effect on function.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Saved searches in notmuch-hello and notmuch-jump can specify whether
to use search mode or tree mode. This adds an option for them to
specify unthreaded mode.
We have shortcuts S and Z to let the user switch to Search view and
Tree view with the current search. Add U to let the user switch to
unthreaded view from the current search, and ensure that S and Z
switch from unthreaded to search and tree veiew respectively.
Tree mode allows the user to choose whether to use the split screen
displaying just the current message or a full screen displaying the
entire thread. As unthreaded mode is quite different in use the user
may want a different customisation for this mode.
It is likely that the user will want a different line format for
unthreaded mode from tree mode; in particular the thread structure
graphics are unnecessary in unthreaded mode.
Add a new customisable variable and set it to something sensible.
This commit introduces a new 'unthreaded' search mode where each
matching message is shown on a separate line. It shares almost all of
its code with tree view. Subsequent commits will allow it to diverge
slightly in appearance.
This adds a --unthreaded option to notmuch show to tell it to return
the matching messages in an unthreaded order (so just by date).
To make it easier for users, in particular for notmuch-tree.el, we
output each message with the same "nesting" as if it were an entire
thread in its own right.
amended by db: s/status= /status = /
