debian/notmuch
1
0
Fork 0
mirror of https://git.notmuchmail.org/git/notmuch synced 2024-11-21 18:38:08 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
notmuch/test/T357-index-decryption.sh

259 lines
9.3 KiB
Bash
Raw Normal View History

cli/new: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting during "notmuch new" on a per-invocation basis. We update the documentation and tab completion, and also add a test.
2017-10-21 04:25:46 +02:00
#!/usr/bin/env bash
# TODO: test index.decryption=failed
test_description='indexing decrypted mail'
. $(dirname "$0")/test-lib.sh || exit 1
##################################################
add_gnupg_home
# get key fingerprint
FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10)
# create a test encrypted message
test_begin_subtest 'emacs delivery of encrypted message'
test_expect_success \
'emacs_fcc_message \
"test encrypted message for cleartext index 001" \
"This is a test encrypted message with a wumpus.\n" \
"(mml-secure-message-encrypt)"'
test_begin_subtest "search for unindexed cleartext"
output=$(notmuch search wumpus)
expected=''
test_expect_equal \
"$output" \
"$expected"
# create a test encrypted message that is indexed in the clear
test_begin_subtest 'emacs delivery of encrypted message'
test_expect_success \
indexing: Change from try_decrypt to decrypt the command-line interface for indexing (reindex, new, insert) used --try-decrypt; and the configuration records used index.try_decrypt. But by comparison with "show" and "reply", there doesn't seem to be any reason for the "try" prefix. This changeset adjusts the command-line interface and the configuration interface. For the moment, i've left indexopts_{set,get}_try_decrypt alone. The subsequent changeset will address those.
2017-12-08 07:23:50 +01:00
'emacs_fcc_message --decrypt=true \
cli/new: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting during "notmuch new" on a per-invocation basis. We update the documentation and tab completion, and also add a test.
2017-10-21 04:25:46 +02:00
"test encrypted message for cleartext index 002" \
"This is a test encrypted message with a wumpus.\n" \
"(mml-secure-message-encrypt)"'
test_begin_subtest "emacs delivery of encrypted message, indexed cleartext"
output=$(notmuch search wumpus)
expected='thread:0000000000000002 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (encrypted inbox)'
test_expect_equal \
"$output" \
"$expected"
# and the same search, but by property ($expected is untouched):
test_begin_subtest "emacs search by property for one message"
output=$(notmuch search property:index.decryption=success)
test_expect_equal \
"$output" \
"$expected"
cli/insert: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-message basis when invoking "notmuch insert". We also update the documentation and tab completion, and add more tests.
2017-10-21 04:25:47 +02:00
test_begin_subtest "message should go away after deletion"
# cache the message in an env var and remove it:
fname=$(notmuch search --output=files wumpus)
contents="$(notmuch show --format=raw wumpus)"
rm -f "$fname"
notmuch new
output=$(notmuch search wumpus)
expected=''
test_expect_equal \
"$output" \
"$expected"
# try reinserting it without decryption, should stay the same:
test_begin_subtest "message cleartext not present after insert"
notmuch insert --folder=sent <<<"$contents"
output=$(notmuch search wumpus)
test_expect_equal \
"$output" \
"$expected"
# try reinserting it with decryption, should appear again, but now we
# have two copies of the message:
lib: convert notmuch decryption policy to an enum Future patches in this series will introduce new policies; this merely readies the way for them. We also convert --try-decrypt to a keyword argument instead of a boolean.
2017-12-08 07:23:52 +01:00
test_begin_subtest "message cleartext is present after reinserting with --decrypt=true"
notmuch insert --folder=sent --decrypt=true <<<"$contents"
cli/insert: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-message basis when invoking "notmuch insert". We also update the documentation and tab completion, and add more tests.
2017-10-21 04:25:47 +02:00
output=$(notmuch search wumpus)
expected='thread:0000000000000003 2000-01-01 [1/1(2)] Notmuch Test Suite; test encrypted message for cleartext index 002 (encrypted inbox unread)'
test_expect_equal \
"$output" \
"$expected"
# remove all copies
test_begin_subtest "delete all copies of the message"
mid="$(notmuch search --output=messages wumpus)"
rm -f $(notmuch search --output=files wumpus)
notmuch new
output=$(notmuch search "id:$mid")
expected=''
test_expect_equal \
"$output" \
"$expected"
# try inserting it with decryption, should appear as a single copy
# (note: i think thread id skips 4 because of duplicate message-id
# insertion, above)
lib: convert notmuch decryption policy to an enum Future patches in this series will introduce new policies; this merely readies the way for them. We also convert --try-decrypt to a keyword argument instead of a boolean.
2017-12-08 07:23:52 +01:00
test_begin_subtest "message cleartext is present with insert --decrypt=true"
notmuch insert --folder=sent --decrypt=true <<<"$contents"
cli/insert: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-message basis when invoking "notmuch insert". We also update the documentation and tab completion, and add more tests.
2017-10-21 04:25:47 +02:00
output=$(notmuch search wumpus)
expected='thread:0000000000000005 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (encrypted inbox unread)'
test_expect_equal \
"$output" \
"$expected"
cli/reindex: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-run basis when invoking "notmuch reindex". This allows the possibility of (for example) an emacs keybinding that adds the cleartext of the currently shown decrypted message to the index, making it searchable in the future. It also enables one-time indexing of all messages matching some query, like so: notmuch reindex tag:encrypted and\ not property:index.decryption=success and\ from:alice@example.org We also update the documentation and tab completion, and add a few more tests.
2017-10-21 04:25:48 +02:00
# add a tag to all messages to ensure that it stays after reindexing
test_begin_subtest 'tagging all messages'
test_expect_success 'notmuch tag +blarney "encrypted message"'
test_begin_subtest "verify that tags have not changed"
output=$(notmuch search tag:blarney)
expected='thread:0000000000000001 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox)
thread:0000000000000005 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox unread)'
test_expect_equal \
"$output" \
"$expected"
indexing: Change from try_decrypt to decrypt the command-line interface for indexing (reindex, new, insert) used --try-decrypt; and the configuration records used index.try_decrypt. But by comparison with "show" and "reply", there doesn't seem to be any reason for the "try" prefix. This changeset adjusts the command-line interface and the configuration interface. For the moment, i've left indexopts_{set,get}_try_decrypt alone. The subsequent changeset will address those.
2017-12-08 07:23:50 +01:00
# see if first message shows up after reindexing with --decrypt=true (same $expected, untouched):
cli/reindex: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-run basis when invoking "notmuch reindex". This allows the possibility of (for example) an emacs keybinding that adds the cleartext of the currently shown decrypted message to the index, making it searchable in the future. It also enables one-time indexing of all messages matching some query, like so: notmuch reindex tag:encrypted and\ not property:index.decryption=success and\ from:alice@example.org We also update the documentation and tab completion, and add a few more tests.
2017-10-21 04:25:48 +02:00
test_begin_subtest 'reindex old messages'
indexing: Change from try_decrypt to decrypt the command-line interface for indexing (reindex, new, insert) used --try-decrypt; and the configuration records used index.try_decrypt. But by comparison with "show" and "reply", there doesn't seem to be any reason for the "try" prefix. This changeset adjusts the command-line interface and the configuration interface. For the moment, i've left indexopts_{set,get}_try_decrypt alone. The subsequent changeset will address those.
2017-12-08 07:23:50 +01:00
test_expect_success 'notmuch reindex --decrypt=true tag:encrypted and not property:index.decryption=success'
cli/reindex: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-run basis when invoking "notmuch reindex". This allows the possibility of (for example) an emacs keybinding that adds the cleartext of the currently shown decrypted message to the index, making it searchable in the future. It also enables one-time indexing of all messages matching some query, like so: notmuch reindex tag:encrypted and\ not property:index.decryption=success and\ from:alice@example.org We also update the documentation and tab completion, and add a few more tests.
2017-10-21 04:25:48 +02:00
test_begin_subtest "reindexed encrypted message, including cleartext"
output=$(notmuch search wumpus)
test_expect_equal \
"$output" \
"$expected"
# and the same search, but by property ($expected is untouched):
test_begin_subtest "emacs search by property for both messages"
output=$(notmuch search property:index.decryption=success)
test_expect_equal \
"$output" \
"$expected"
# try to remove cleartext indexing
test_begin_subtest 'reindex without cleartext'
test_expect_success 'notmuch reindex tag:encrypted and property:index.decryption=success'
test_begin_subtest "reindexed encrypted messages, without cleartext"
output=$(notmuch search wumpus)
expected=''
test_expect_equal \
"$output" \
"$expected"
crypto: new decryption policy "auto" This new automatic decryption policy should make it possible to decrypt messages that we have stashed session keys for, without incurring a call to the user's asymmetric keys.
2017-12-08 07:23:53 +01:00
# ensure no session keys are present:
test_begin_subtest 'reindex using only session keys'
cli/new, insert, reindex: change index.decrypt to "auto" by default The new "auto" decryption policy is not only good for "notmuch show" and "notmuch reindex". It's also useful for indexing messages -- there's no good reason to not try to go ahead and index the cleartext of a message that we have a stashed session key for. This change updates the defaults and tunes the test suite to make sure that they have taken effect.
2017-12-08 07:23:59 +01:00
test_expect_success 'notmuch reindex tag:encrypted and property:index.decryption=success'
crypto: new decryption policy "auto" This new automatic decryption policy should make it possible to decrypt messages that we have stashed session keys for, without incurring a call to the user's asymmetric keys.
2017-12-08 07:23:53 +01:00
test_begin_subtest "reindexed encrypted messages, decrypting only with session keys"
output=$(notmuch search wumpus)
expected=''
test_expect_equal \
"$output" \
"$expected"
cli/reindex: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-run basis when invoking "notmuch reindex". This allows the possibility of (for example) an emacs keybinding that adds the cleartext of the currently shown decrypted message to the index, making it searchable in the future. It also enables one-time indexing of all messages matching some query, like so: notmuch reindex tag:encrypted and\ not property:index.decryption=success and\ from:alice@example.org We also update the documentation and tab completion, and add a few more tests.
2017-10-21 04:25:48 +02:00
# and the same search, but by property ($expected is untouched):
test_begin_subtest "emacs search by property with both messages unindexed"
output=$(notmuch search property:index.decryption=success)
test_expect_equal \
"$output" \
"$expected"
# ensure that the tags remain even when we are dropping the cleartext.
test_begin_subtest "verify that tags remain without cleartext"
output=$(notmuch search tag:blarney)
expected='thread:0000000000000001 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox)
thread:0000000000000005 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox unread)'
test_expect_equal \
"$output" \
"$expected"
crypto: Test restore of cleartext index from stashed session keys If you've got a notmuch dump that includes stashed session keys for every decrypted message, and you've got your message archive, you should be able to get back to the same index that you had before. Here we add a simple test that give some flavor of how that works.
2017-11-30 09:59:31 +01:00
add_email_corpus crypto
test_begin_subtest "indexing message fails when secret key not available"
lib: convert notmuch decryption policy to an enum Future patches in this series will introduce new policies; this merely readies the way for them. We also convert --try-decrypt to a keyword argument instead of a boolean.
2017-12-08 07:23:52 +01:00
notmuch reindex --decrypt=true id:simple-encrypted@crypto.notmuchmail.org
crypto: Test restore of cleartext index from stashed session keys If you've got a notmuch dump that includes stashed session keys for every decrypted message, and you've got your message archive, you should be able to get back to the same index that you had before. Here we add a simple test that give some flavor of how that works.
2017-11-30 09:59:31 +01:00
output=$(notmuch dump )
expected='#notmuch-dump batch-tag:3 config,properties,tags
+encrypted +inbox +unread -- id:simple-encrypted@crypto.notmuchmail.org
#= simple-encrypted@crypto.notmuchmail.org index.decryption=failure'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "cannot find cleartext index"
output=$(notmuch search sekrit)
expected=''
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "cleartext index recovery on reindexing with stashed session keys"
notmuch restore <<EOF
#notmuch-dump batch-tag:3 config,properties,tags
#= simple-encrypted@crypto.notmuchmail.org session-key=9%3AFC09987F5F927CC0CC0EE80A96E4C5BBF4A499818FB591207705DFDDD6112CF9
EOF
cli/new, insert, reindex: change index.decrypt to "auto" by default The new "auto" decryption policy is not only good for "notmuch show" and "notmuch reindex". It's also useful for indexing messages -- there's no good reason to not try to go ahead and index the cleartext of a message that we have a stashed session key for. This change updates the defaults and tunes the test suite to make sure that they have taken effect.
2017-12-08 07:23:59 +01:00
notmuch reindex id:simple-encrypted@crypto.notmuchmail.org
crypto: Test restore of cleartext index from stashed session keys If you've got a notmuch dump that includes stashed session keys for every decrypted message, and you've got your message archive, you should be able to get back to the same index that you had before. Here we add a simple test that give some flavor of how that works.
2017-11-30 09:59:31 +01:00
output=$(notmuch search sekrit)
expected='thread:0000000000000001 2016-12-22 [1/1] Daniel Kahn Gillmor; encrypted message (encrypted inbox unread)'
test: session keys are known broken without session key support If the version of GMime we're building against doesn't support session key extraction or re-use, mark the tests that rely on session key capabilities as known-broken. This should resolve test suite failures on ubuntu trusty and debian jessie and earlier, which have GMime 2.6.20 -- session key support was introduced in GMime 2.6.21.
2017-12-05 19:40:27 +01:00
if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
test_subtest_known_broken
fi
crypto: Test restore of cleartext index from stashed session keys If you've got a notmuch dump that includes stashed session keys for every decrypted message, and you've got your message archive, you should be able to get back to the same index that you had before. Here we add a simple test that give some flavor of how that works.
2017-11-30 09:59:31 +01:00
test_expect_equal \
"$output" \
"$expected"
cli/reply: use decryption policy "auto" by default. If the user doesn't specify --decrypt= at all, but a stashed session key is known to notmuch, when replying to an encrypted message, notmuch should just go ahead and decrypt. The user can disable this at the command line with --decrypt=false, though it's not clear why they would ever want to do that.
2017-12-08 07:23:54 +01:00
test_begin_subtest "notmuch reply should show cleartext if session key is present"
output=$(notmuch reply id:simple-encrypted@crypto.notmuchmail.org | grep '^>')
expected='> This is a top sekrit message.'
if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
test_subtest_known_broken
fi
test_expect_equal \
"$output" \
"$expected"
cli/show: use decryption policy "auto" by default. When showing a message, if the user doesn't specify --decrypt= at all, but a stashed session key is known to notmuch, notmuch should just go ahead and try to decrypt the message with the session key (without bothering the user for access to their asymmetric secret key). The user can disable this at the command line with --decrypt=false if they really don't want to look at the e-mail that they've asked notmuch to show them. and of course, "notmuch show --decrypt" still works for accessing the user's secret keys if necessary.
2017-12-08 07:23:55 +01:00
test_begin_subtest "notmuch show should show cleartext if session key is present"
output=$(notmuch show id:simple-encrypted@crypto.notmuchmail.org | awk '/^\014part}/{ f=0 }; { if (f) { print $0 } } /^\014part{ ID: 3/{ f=1 }')
expected='This is a top sekrit message.'
if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
test_subtest_known_broken
fi
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "notmuch show should show nothing if decryption is explicitly disallowed"
output=$(notmuch show --decrypt=false id:simple-encrypted@crypto.notmuchmail.org | awk '/^\014part}/{ f=0 }; { if (f) { print $0 } } /^\014part{ ID: 3/{ f=1 }')
expected='Non-text part: application/octet-stream'
test_expect_equal \
"$output" \
"$expected"
cli/reindex: destroy stashed session keys when --decrypt=false There are some situations where the user wants to get rid of the cleartext index of a message. For example, if they're indexing encrypted messages normally, but suddenly they run across a message that they really don't want any trace of in their index. In that case, the natural thing to do is: notmuch reindex --decrypt=false id:whatever@example.biz But of course, clearing the cleartext index without clearing the stashed session key is just silly. So we do the expected thing and also destroy any stashed session keys while we're destroying the index of the cleartext. Note that stashed session keys are stored in the xapian database, but xapian does not currently allow safe deletion (see https://trac.xapian.org/ticket/742). As a workaround, after removing session keys and cleartext material from the database, the user probably should do something like "notmuch compact" to try to purge whatever recoverable data is left in the xapian freelist. This problem really needs to be addressed within xapian, though, if we want it fixed right.
2017-12-08 07:24:00 +01:00
test_begin_subtest "purging stashed session keys should lose access to the cleartext"
notmuch reindex --decrypt=false id:simple-encrypted@crypto.notmuchmail.org
output=$(notmuch search sekrit)
expected=''
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "and cleartext should be unrecoverable now that there are no stashed session keys"
notmuch dump
notmuch reindex --decrypt=true id:simple-encrypted@crypto.notmuchmail.org
output=$(notmuch search sekrit)
expected=''
test_expect_equal \
"$output" \
"$expected"
cli/reindex: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-run basis when invoking "notmuch reindex". This allows the possibility of (for example) an emacs keybinding that adds the cleartext of the currently shown decrypted message to the index, making it searchable in the future. It also enables one-time indexing of all messages matching some query, like so: notmuch reindex tag:encrypted and\ not property:index.decryption=success and\ from:alice@example.org We also update the documentation and tab completion, and add a few more tests.
2017-10-21 04:25:48 +02:00
# TODO: test removal of a message from the message store between
# indexing and reindexing.
cli/insert: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-message basis when invoking "notmuch insert". We also update the documentation and tab completion, and add more tests.
2017-10-21 04:25:47 +02:00
cli/reindex: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-run basis when invoking "notmuch reindex". This allows the possibility of (for example) an emacs keybinding that adds the cleartext of the currently shown decrypted message to the index, making it searchable in the future. It also enables one-time indexing of all messages matching some query, like so: notmuch reindex tag:encrypted and\ not property:index.decryption=success and\ from:alice@example.org We also update the documentation and tab completion, and add a few more tests.
2017-10-21 04:25:48 +02:00
# TODO: insert the same message into the message store twice, index,
# remove one of them from the message store, and then reindex.
# reindexing should return a failure but the message should still be
# present? -- or what should the semantics be if you ask to reindex a
# message whose underlying files have been renamed or moved or
# removed?
cli/insert: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting on a per-message basis when invoking "notmuch insert". We also update the documentation and tab completion, and add more tests.
2017-10-21 04:25:47 +02:00
cli/new: add --try-decrypt=(true|false) Enable override of the index.try_decrypt setting during "notmuch new" on a per-invocation basis. We update the documentation and tab completion, and also add a test.
2017-10-21 04:25:46 +02:00
test_done
Reference in a new issue Copy permalink