If you're going to store the cleartext index of an encrypted message,
in most situations you might just as well store the session key.
Doing this storage has efficiency and recoverability advantages.
Combined with a schedule of regular OpenPGP subkey rotation and
destruction, this can also offer security benefits, like "deletable
e-mail", which is the store-and-forward analog to "forward secrecy".
But wait, i hear you saying, i have a special need to store cleartext
indexes but it's really bad for me to store session keys! Maybe
(let's imagine) i get lots of e-mails with incriminating photos
attached, and i want to be able to search for them by the text in the
e-mail, but i don't want someone with access to the index to be
actually able to see the photos themselves.
Fret not, the next patch in this series will support your wacky
uncommon use case.
There are some situations where the user wants to get rid of the
cleartext index of a message. For example, if they're indexing
encrypted messages normally, but suddenly they run across a message
that they really don't want any trace of in their index.
In that case, the natural thing to do is:
notmuch reindex --decrypt=false id:whatever@example.biz
But of course, clearing the cleartext index without clearing the
stashed session key is just silly. So we do the expected thing and
also destroy any stashed session keys while we're destroying the index
of the cleartext.
Note that stashed session keys are stored in the xapian database, but
xapian does not currently allow safe deletion (see
https://trac.xapian.org/ticket/742).
As a workaround, after removing session keys and cleartext material
from the database, the user probably should do something like "notmuch
compact" to try to purge whatever recoverable data is left in the
xapian freelist. This problem really needs to be addressed within
xapian, though, if we want it fixed right.
The new "auto" decryption policy is not only good for "notmuch show"
and "notmuch reindex". It's also useful for indexing messages --
there's no good reason to not try to go ahead and index the cleartext
of a message that we have a stashed session key for.
This change updates the defaults and tunes the test suite to make sure
that they have taken effect.
When showing a message, if the user doesn't specify --decrypt= at all,
but a stashed session key is known to notmuch, notmuch should just go
ahead and try to decrypt the message with the session key (without
bothering the user for access to their asymmetric secret key).
The user can disable this at the command line with --decrypt=false if
they really don't want to look at the e-mail that they've asked
notmuch to show them.
and of course, "notmuch show --decrypt" still works for accessing the
user's secret keys if necessary.
If the user doesn't specify --decrypt= at all, but a stashed session
key is known to notmuch, when replying to an encrypted message,
notmuch should just go ahead and decrypt.
The user can disable this at the command line with --decrypt=false,
though it's not clear why they would ever want to do that.
This new automatic decryption policy should make it possible to
decrypt messages that we have stashed session keys for, without
incurring a call to the user's asymmetric keys.
Future patches in this series will introduce new policies; this merely
readies the way for them.
We also convert --try-decrypt to a keyword argument instead of a boolean.
the command-line interface for indexing (reindex, new, insert) used
--try-decrypt; and the configuration records used index.try_decrypt.
But by comparison with "show" and "reply", there doesn't seem to be
any reason for the "try" prefix.
This changeset adjusts the command-line interface and the
configuration interface.
For the moment, i've left indexopts_{set,get}_try_decrypt alone. The
subsequent changeset will address those.
If the version of GMime we're building against doesn't support session
key extraction or re-use, mark the tests that rely on session key
capabilities as known-broken.
This should resolve test suite failures on ubuntu trusty and debian
jessie and earlier, which have GMime 2.6.20 -- session key support was
introduced in GMime 2.6.21.
If you've got a notmuch dump that includes stashed session keys for
every decrypted message, and you've got your message archive, you
should be able to get back to the same index that you had before.
Here we add a simple test that give some flavor of how that works.
In 8e7fb88237 Jani replaced the use of $(pwd -P) to find the
TEST_DIRECTORY in order to better support out of tree
builds. Unfortunately the performance-tests need a different value for
the variable and were thus broken.
This commit splits out the setting of this variable for the two sets
of tests. Performance tests still don't work out of tree, because
the handling of the downloaded corpus needs to be updated.
backup_database() and restore_database() used to store the backups in
the test specific temporary directory, through the current working
directory being there. Commit 8e7fb88237 ("test: use source and
build paths in test-lib-common.sh") started using a test specific
backup directories under the build tree test directory. This was in
error. Switch back to the old location, but using paths to the
location instead of relying on current working directory.
Reported by Daniel Kahn Gillmor <dkg@fifthhorseman.net>.
Enable override of the index.try_decrypt setting on a per-run basis
when invoking "notmuch reindex". This allows the possibility of (for
example) an emacs keybinding that adds the cleartext of the currently
shown decrypted message to the index, making it searchable in the
future.
It also enables one-time indexing of all messages matching some query,
like so:
notmuch reindex tag:encrypted and\
not property:index.decryption=success and\
from:alice@example.org
We also update the documentation and tab completion, and add a few
more tests.
Enable override of the index.try_decrypt setting on a per-message
basis when invoking "notmuch insert".
We also update the documentation and tab completion, and add more tests.
Enable override of the index.try_decrypt setting during "notmuch new"
on a per-invocation basis.
We update the documentation and tab completion, and also add a test.
Subsequent patches may want to send GNU-style --long-arguments to
notmuch new in the test suite, in particular when invoking
emacs_fcc_message. This changeset makes that possible.
The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
running each test when the current working directory is not the test
subdirectory. Tests with failures would leave their TMP_DIRECTORY
directory behind for debugging, and repeated out-of-tree test runs
would have old temp directories. (This lead to e.g. T310-emacs.sh
hanging because emacs would prompt for overwriting files.)
We remove the likely anyway defunct --root test option while at it,
just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
Turns out round trip tests didn't really round trip anything. Broken
by yours truly in 971cdc72cd ("test: make it possible to have
multiple corpora"). Ooops.
C99 stdbool turned 18 this year. There really is no reason to use our
own, except in the library interface for backward
compatibility. Convert the cli and test binaries to stdbool.
Several changes at once, just to not have to change the same lines
several times over:
- Use designated initializers to initialize opt desc arrays.
- Only initialize the needed fields.
- Remove arg_id (short options) as unused.
- Replace opt_type and output_var with several type safe output
variables, where the output variable being non-NULL determines the
type. Introduce checks to ensure only one is set. The downside is
some waste of const space per argument; this could be saved by
retaining opt_type and using a union, but that's still pretty
verbose.
- Fix some variables due to the type safety. Mostly a good thing, but
leads to some enums being changed to ints. This is pedantically
correct, but somewhat annoying. We could also cast, but that defeats
the purpose a bit.
- Terminate the opt desc arrays using {}.
The output variable type safety and the ability to add new fields for
just some output types or arguments are the big wins. For example, if
we wanted to add a variable to set when the argument is present, we
could do so for just the arguments that need it.
Beauty is in the eye of the beholder, but I think this looks nice when
defining the arguments, and reduces some of the verbosity we have
there.
This imports a message with ISO-8859-2 encoded characters, then opens
the database using the python bindings. We peek through all mesage
parts, afterwards print the message id.
Signed-off-by: Florian Klink <flokli@flokli.de>
Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Commit a7964c86d1 ("emacs: Sanitize authors and subjects in search
and show") added sanitization of header information for display. Do
the same for reply subjects.
This fixes the long-standing annoying artefact of certain versions of
mailman using tab as folding whitespace, leading to tabs in reply
subjects.
This is a logical followup to "lib: index the content type of
signature parts", which will make it easier to record the message
structure of all messages.
