Commit graph

5890 commits

Author SHA1 Message Date
Jani Nikula
1b91884296 test: fix test database backup/restore location
backup_database() and restore_database() used to store the backups in
the test specific temporary directory, through the current working
directory being there. Commit 8e7fb88237 ("test: use source and
build paths in test-lib-common.sh") started using a test specific
backup directories under the build tree test directory. This was in
error. Switch back to the old location, but using paths to the
location instead of relying on current working directory.

Reported by Daniel Kahn Gillmor <dkg@fifthhorseman.net>.
2017-10-29 09:30:30 -07:00
Daniel Kahn Gillmor
9479baf6f1 NEWS: cleartext indexing
give a short summary of cleartext indexing of encrypted e-mail
messages.
2017-10-22 14:02:00 -03:00
Jani Nikula
61c346c286 NEWS: test suite out-of-tree builds
Out-of-tree builds now work and supersede --root option.
2017-10-22 11:03:21 -03:00
Daniel Kahn Gillmor
8116891744 cli/reindex: add --try-decrypt=(true|false)
Enable override of the index.try_decrypt setting on a per-run basis
when invoking "notmuch reindex".  This allows the possibility of (for
example) an emacs keybinding that adds the cleartext of the currently
shown decrypted message to the index, making it searchable in the
future.

It also enables one-time indexing of all messages matching some query,
like so:

    notmuch reindex tag:encrypted and\
       not property:index.decryption=success and\
       from:alice@example.org

We also update the documentation and tab completion, and add a few
more tests.
2017-10-21 19:58:52 -03:00
Daniel Kahn Gillmor
c5356b9ed5 cli/insert: add --try-decrypt=(true|false)
Enable override of the index.try_decrypt setting on a per-message
basis when invoking "notmuch insert".

We also update the documentation and tab completion, and add more tests.
2017-10-21 19:58:43 -03:00
Daniel Kahn Gillmor
35456d4b0c cli/new: add --try-decrypt=(true|false)
Enable override of the index.try_decrypt setting during "notmuch new"
on a per-invocation basis.

We update the documentation and tab completion, and also add a test.
2017-10-21 19:58:32 -03:00
Daniel Kahn Gillmor
92f318abe4 tests: emacs_fcc_message: allow passing --arguments to notmuch new
Subsequent patches may want to send GNU-style --long-arguments to
notmuch new in the test suite, in particular when invoking
emacs_fcc_message.  This changeset makes that possible.
2017-10-21 19:58:23 -03:00
Daniel Kahn Gillmor
5bdb8549f6 cli: set up shared command-line arguments for indexing
We have an indexopts structure for manipulating indexing in different
ways, but we also have three command-line invocations that can trigger
indexing: new, insert, and reindex.

This changeset prepares a common parser that these subcommands can
share.

Note: if the deprecated crypto.gpg_path configuration option is set to
anything other than "gpg", we ignore it (and print a warning on
stderr, if built against gmime < 3.0).

At the moment, it's just --try-decrypt, but others will likely follow.
2017-10-21 19:58:11 -03:00
Daniel Kahn Gillmor
d0da7a0a1c config: define new option index.try_decrypt
By default, notmuch won't try to decrypt on indexing.  With this
patch, we make it possible to indicate a per-database preference using
the config variable "index.try_decrypt", which by default will be
false.

At indexing time, the database needs some way to know its internal
defaults for how to index encrypted parts.  It shouldn't be contingent
on an external config file (since that can't be retrieved from the
database object itself), so we store it in the database.

This behaves similarly to the query.* configurations, which are also
stored in the database itself, so we're not introducing any new
dependencies by requiring that it be stored in the database.
2017-10-21 19:54:33 -03:00
Daniel Kahn Gillmor
886c0b1666 config: test whether an item is stored in the database by name
QUERY_STRING was only used in two places, both to test whether a
variable should be stored in (or retrieved from) the database.

Since other configuration variables might be stored in the database in
the future, consolidate that test into a single function.

We also document that these configuration options should not be placed
in the config file.
2017-10-21 19:54:24 -03:00
Daniel Kahn Gillmor
4dfcc8c9b2 crypto: index encrypted parts when indexopts try_decrypt is set.
If we see index options that ask us to decrypt when indexing a
message, and we encounter an encrypted part, we'll try to descend into
it.

If we can decrypt, we add the property index.decryption=success.

If we can't decrypt (or recognize the encrypted type of mail), we add
the property index.decryption=failure.

Note that a single message may have both values of the
"index.decryption" property: "success" and "failure".  For example,
consider a message that includes multiple layers of encryption.  If we
manage to decrypt the outer layer ("index.decryption=success"), but
fail on the inner layer ("index.decryption=failure").

Because of the property name, this will be automatically cleared (and
possibly re-set) during re-indexing.  This means it will subsequently
correspond to the actual semantics of the stored index.
2017-10-21 19:53:19 -03:00
Daniel Kahn Gillmor
0bb05ff693 reindex: drop all properties named with prefix "index."
This allows us to create new properties that will be automatically set
during indexing, and cleared during re-indexing, just by choice of
property name.
2017-10-21 19:53:08 -03:00
Daniel Kahn Gillmor
6575b7eb31 doc: add notmuch-properties(7)
We will want a user-facing place to record details about the use of
notmuch properties shortly.  This establishes a new manual page for
that purpose.
2017-10-21 19:52:55 -03:00
Daniel Kahn Gillmor
20ff9de24d index: implement notmuch_indexopts_t with try_decrypt
This is currently mostly a wrapper around _notmuch_crypto_t that keeps
its internals private and doesn't expose any of the GMime API.
However, non-crypto indexing options might also be added later
(e.g. filters or other transformations).
2017-10-21 19:52:47 -03:00
Jani Nikula
d6929040a4 build: don't copy the test directory for out-of-tree builds
It should now work without.
2017-10-21 16:34:55 -03:00
Jani Nikula
d7929d61a2 test: use source path in add_gnupg_home
Make a distinction between source and build directories.
2017-10-21 16:34:04 -03:00
Jani Nikula
d455c8b74a test: remove --root option and fix TMP_DIRECTORY cleanup
The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
running each test when the current working directory is not the test
subdirectory. Tests with failures would leave their TMP_DIRECTORY
directory behind for debugging, and repeated out-of-tree test runs
would have old temp directories. (This lead to e.g. T310-emacs.sh
hanging because emacs would prompt for overwriting files.)

We remove the likely anyway defunct --root test option while at it,
just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
2017-10-21 11:32:20 -03:00
Jani Nikula
409d877364 test: use source and build paths in T590-thread-breakage.sh
Make a distinction between source and build directories.
2017-10-20 20:50:17 -03:00
Jani Nikula
3c23780396 test: use source and build paths in T480-hex-escaping.sh
Make a distinction between source and build directories.
2017-10-20 20:50:03 -03:00
Jani Nikula
1ab72b13cd test: use source and build paths in T4?0-emacs-*.sh
Make a distinction between source and build directories.
2017-10-20 20:49:50 -03:00
Jani Nikula
e216d9b7ab test: use source and build paths in T380-atomicity.sh
Make a distinction between source and build directories.
2017-10-20 20:49:37 -03:00
Jani Nikula
0364a6d74b test: use source and build paths in T360-symbol-hiding.sh
Make a distinction between source and build directories.
2017-10-20 20:49:26 -03:00
Jani Nikula
bc349ad316 test: use source and build paths in T355-smime.sh
Make a distinction between source and build directories.
2017-10-20 20:49:18 -03:00
Jani Nikula
f6b098b1a6 test: use source and build paths in T310-emacs.sh
Make a distinction between source and build directories.
2017-10-20 19:56:13 -03:00
Jani Nikula
b079133ca8 test: use source and build paths in T260-thread-order.sh
Make a distinction between source and build directories.
2017-10-20 19:55:57 -03:00
Jani Nikula
32c088b524 test: use source and build paths in T160-json.sh and T170-sexp.sh
Make a distinction between source and build directories.
2017-10-20 19:55:49 -03:00
Jani Nikula
fc12f6f07b test: use source and build paths in T000-basic.sh
Make a distinction between source and build directories.
2017-10-20 19:55:41 -03:00
Jani Nikula
a8cd1ec26b test: use source and build paths in notmuch-test
Make a distinction between source and build directories.
2017-10-20 19:55:31 -03:00
Jani Nikula
8dfa116010 test: mkdir and cd to $TMP_DIRECTORY instead of $test for testing
$TMP_DIRECTORY is a full path, while $test is not.
2017-10-20 19:55:10 -03:00
Jani Nikula
8e7fb88237 test: use source and build paths in test-lib-common.sh
Make a distinction between source and build directories.

The expectation is that anyone sourcing test-lib-common.sh has sourced
export-dirs.sh.
2017-10-20 19:54:43 -03:00
Jani Nikula
2e16b05152 perf-test: use source and build paths in perf-test-lib.sh
Make a distinction between source and build directories.
2017-10-20 19:54:22 -03:00
Jani Nikula
a9894fce06 test: use source and build paths in test-lib.sh
Make a distinction between source and build directories.
2017-10-20 19:54:08 -03:00
Jani Nikula
1f3c114f08 test: add a script to figure out source and build directories
Expect the tests to be run somewhere in the build directory
hierarchy. Allow for source and build directories to be passed through
the environment.
2017-10-20 19:53:55 -03:00
Jani Nikula
e1bd4f2363 test: check for notmuch binary in test-lib.sh
Move notmuch executable check into common code. Redundant for
notmuch-test runs, but works also for individual tests.
2017-10-20 19:53:31 -03:00
Jani Nikula
fd36ae9f51 perf-test: use $(dirname "$0") for sourcing perf-test-lib.sh
Don't assume the tests are always run from within the source tree.
2017-10-20 19:53:14 -03:00
Jani Nikula
a863de1e43 test: use $(dirname "$0") for sourcing test-lib.sh
Don't assume the tests are always run from within the source tree.
2017-10-20 19:52:49 -03:00
Jani Nikula
0dd58ea7e7 build: use NOTMUCH_SRCDIR for referencing notmuch-test
Don't assume the tests are always run from within the source tree.
2017-10-20 19:52:24 -03:00
Jani Nikula
1c2152ce73 test: use corpora/default not corpus in T480-hex-escaping.sh
Turns out round trip tests didn't really round trip anything. Broken
by yours truly in 971cdc72cd ("test: make it possible to have
multiple corpora"). Ooops.
2017-10-20 19:52:14 -03:00
Daniel Kahn Gillmor
0b9e1a2472 properties: add notmuch_message_remove_all_properties_with_prefix()
Subsequent patches will introduce a convention that properties whose
name starts with "index." will be stripped (and possibly re-added)
during re-indexing.  This patch lays the groundwork for doing that.
2017-10-20 07:58:43 -03:00
Daniel Kahn Gillmor
a18bbf7f15 crypto: make shared crypto code behave library-like
If we're going to reuse the crypto code across both the library and
the client, then it needs to report error states properly and not
write to stderr.
2017-10-20 07:58:20 -03:00
Daniel Kahn Gillmor
197d67959b crypto: move into libnotmuch_util
This prepares us for using the crypto object in both libnotmuch and
the client.
2017-10-20 07:58:10 -03:00
Daniel Kahn Gillmor
6499fce391 doc: make SEE ALSO references one-per-line
This will make future diffs cleaner, make it easier to keep them
alphabetical, and make it easier to scan and search the documentation
sources.
2017-10-18 22:36:39 -03:00
Daniel Kahn Gillmor
5fb0f5f113 doc: fix documentation typos and copy/paste-os 2017-10-18 22:36:09 -03:00
Jani Nikula
ab8b40f7e3 cli: make notmuch count --exclude a boolean argument
Commit 0f314c0c99 ("cli: convert notmuch_bool_t to stdbool")
over-eagerly converted EXCLUDE_TRUE and EXCLUDE_FALSE to EXCLUDE_true
and EXCLUDE_false in notmuch-count.c. We could just fix the case back,
but convert the option to an actual boolean argument instead.

We've used a keyword argument rather than a boolean argument for
notmuch count --exclude for five years, since commit 785c1e497f
("cli: move count to the new --exclude=(true|false|flag) naming
scheme."), "to allow future options to be added more easily". I think
we can conclude future options aren't coming any time soon.
2017-10-15 19:22:40 -03:00
Daniel Kahn Gillmor
bcd6229d26 gmime-extra: drop compat layer for g_mime_multipart_encrypted_decrypt
In practice, we're going to see this function invoked differently
depending on which gmime we build against.  The compatibility layer
forces our code into the lowest-common-denominator -- unable to make
use of new features even when built against a newer version.

Dropping the compatibility layer paves the way for clearer use of
features from GMime 3.0 in future commits.
2017-10-14 10:57:06 -03:00
Daniel Kahn Gillmor
ee5df7d7db tests: prepare for more crypto tests (using add_gnupg_home)
Move add_gnupg_home to test-lib.sh to prepare it for reuse.
2017-10-14 10:53:42 -03:00
Daniel Kahn Gillmor
30e33dfdd1 gmime-extra: remove duplicate GMimeAddressType typedef
"typedef GMimeAddressType GMimeRecipientType" is already present
further down in the compatibility wrapper (with other typedefs).  We
don't need it twice.
2017-10-12 22:12:32 -03:00
Daniel Kahn Gillmor
bfd669bb2e crypto: _notmuch_crypto_cleanup should return void
There's no chance that _notmuch_crypto_cleanup() will ever return
anything other than 0, and no one ever checks its return value anyway.
So make it return void instead of int.
2017-10-12 22:11:55 -03:00
Daniel Kahn Gillmor
3e9857a0b9 crypto: drop pretense of notmuch_crypto_context_t
notmuch_crypto_context_t was introduced (i think) as some sort of
abstraction layer to make notmuch somewhat independent of GMime.  But
it isn't even useful for GMime 3.0 or later -- we can drop the
pretense that it's some sort of abstraction in this case, and just
call it what it is, GMimeCryptoContext, which is useful for building
against older versions of GMime.

This also renames _notmuch_crypto_get_context() to
_notmuch_crypto_get_gmime_context().
2017-10-12 22:11:33 -03:00
Daniel Kahn Gillmor
88f2a72ef1 crypto: rename notmuch_crypto_t to _notmuch_crypto_t
The notmuch_crypto_t struct isn't used externally, and we have no
plans to explicitly export it.  Prefix its name (and associated
functions) with _ to make that intent clear.
2017-10-12 22:11:25 -03:00