notmuch-show --verify will now also process S/MIME multiparts if
encountered. Requires gmime-2.6 and gpgsm.
Based on work by Jameson Graef Rollins <jrollins@finestructure.net>.
Currently we key the address hash table with the case sensitive "name
<address>". Switch to case insensitive keying with just address, and
store the case sensitive name and address in linked lists. This will
be helpful in adding support for different deduplication schemes in
the future.
There will be a slight performance penalty for the current full case
sensitive name + address deduplication, but this is simpler as a whole
when other deduplication schemes are added, and I expect the schemes
to be added to become more popular than the current default.
Aparet from the possible performance penalty, the only user visible
change should be the change in the output ordering for
--output=count. The order is not guaranteed (and is based on hash
table traversal) currently anyway, so this should be of no
consequence.
It's becoming a maintenance burden to do anything things with the
crypto glue code twice, once for 2.4 and once for 2.6. I don't have
any 2.4 version available to test on my development machine anymore,
so the 2.4 specific code paths are likely not very well tested.
The function notmuch_exit_if_unmatched_db_uuid is split from
notmuch_process_shared_options because it needs an open notmuch
database.
There are two exceptional cases in uuid handling.
1) notmuch config and notmuch setup don't currently open the database,
so it doesn't make sense to check the UUID.
2) notmuch compact opens the database inside the library, so we either
need to open the database just to check uuid, or change the API.
In particular this fixes a recently encountered bug where the
"--config" argument to "notmuch setup" is silently ignored, which the
unpleasant consequence of overwriting the users config file.
Unfortunately it seems trickier to support --config globally
The non-trivial changes are in notmuch.c; most of the other changes
consists of blindly inserting two lines into every subcommand.
Previously we set up a way for the top level notmuch command to choose
which gpg binary was invoked by libgmime. In this commit we add the
(mostly boilerplate) code to allow the notmuch-config command to read
and write this path, and use it in the appropriate struct.
Update tests for new default variable
GMIME takes a path to gpg, but we hardcode that path. In this commit
we set up argument passing and option storage to allow this path to
specified in the top level notmuch command.
This moves address-related functionality from search command to the
new address command. The implementation shares almost all code and
some command line options.
Options --offset and --limit were intentionally not included in the
address command, because they refer to messages numbers, which users
do not see in the output. This could confuse users because, for
example, they could see more addresses in the output that what was
specified with --limit. This functionality can be correctly
reimplemented for address subcommand later.
Also useless values of --exclude flag were not included in the address
command.
This was inspired by a patch from Jani Nikula.
The main goal is to support gzipped output for future internal
calls (e.g. from notmuch-new) to notmuch_database_dump.
The additional dependency is not very heavy since xapian already pulls
in zlib.
We want the dump to be "atomic", in the sense that after running the
dump file is either present and complete, or not present. This avoids
certain classes of mishaps involving overwriting a good backup with a
bad or partial one.
These queries will match exactly the set of messages currently in the
thread, even if more messages later arrive. Two queries are provided:
one for matched messages and one for unmatched messages.
This can be used to fix race conditions with tagging threads from
search results. While tagging based on a thread: query can affect
messages that arrived after the search, tagging based on stable
queries affects only the messages the user was shown in the search UI.
Since we want clients to be able to depend on the presence of these
queries, this ushers in schema version 2.
Previously, the CLI would print a deprecation warning if a client
requested any format version other than the current one. However, if
we add fields that are backwards-compatible, but want clients to be
able to depend on, we need to bump the version, but that doesn't make
the older version deprecated.
Hence, separate out the "minimum active" version and only print a
warning for requests below this version number.
For my client, the largest bottleneck for displaying large threads is
exporting each html part individually since by default notmuch will not
show the json parts. For large threads there can be quite a few parts and
each must be exported and decoded one by one. Also, I then have to deal
with all the crazy charsets which I can do through a library but is a
pain.
Therefore, this patch adds an --include-html option that causes the
text/html parts to be included as part of the output of show.
diff man/man1/notmuch-show.1
The notmuch insert command reads a message from standard input,
writes it to a Maildir folder, and then incorporates the message into
the notmuch database. Essentially it moves the functionality of
notmuch-deliver into notmuch.
Though it could be used as an alternative to notmuch new, the reason
I want this is to allow my notmuch frontend to add postponed or sent
messages to the mail store and notmuch database, without resorting to
another tool (e.g. notmuch-deliver) nor directly modifying the maildir.
While looked good on paper, its attempted use caused confusion, complexity,
and potential for information leak when passed through wrapper scripts.
For slimmer code and to lessen demand for maintenance/support the set of
commits which added top level --stderr= option is now reverted.
With this option all writes to stderr are redirected to the specified
FILE (or to stdout on case FILE is '-'). This is immediately useful
in emacs interface as some of its exec intefaces do not provide
separation of stdout and stderr.
This allows specifying config file as a top level argument to notmuch,
and generally makes it possible to override config file options in
main(), without having to touch the subcommands.
If the config file does not exist, one will be created for the notmuch
main command and setup and help subcommands. Help is special in this
regard; the config is created just to avoid errors about missing
config, but it will not be saved.
This also makes notmuch config the talloc context for subcommands.
We now have a notmuch_config_is_new() function to query whether a
config was created or not. Change the notmuch_config_open() is_new
parameter into boolean create_new to determine whether the function
should create a new config if one doesn't exist. This reduces the
complexity of the API.
Keep track of whether the config is newly created, and add
notmuch_config_is_new() accessor function to query this.
This is to support anyone with a config handle to check this, instead
of just whoever called notmuch_config_open().
Currently there is a period of pain whenever we make
backward-incompatible changes to the structured output format, which
discourages not only backward-incompatible improvements to the format,
but also backwards-compatible additions that may not be "perfect". In
the end, these problems limit experimentation and innovation.
This series of patches introduces a way for CLI callers to request a
specific format version on the command line and to determine if the
CLI does not supported the requested version (and perhaps present a
useful diagnostic to the user). Since the caller requests a format
version, it's also possible for the CLI to support multiple
incompatible versions simultaneously, unlike the alternate approach of
including version information in the output.
This patch lays the groundwork by introducing a versioning convention,
standard exit codes, and a utility function to check the requested
version and produce standardized diagnostic messages and exit
statuses.
All the structured output functions in notmuch-reply and notmuch-show
are renamed to a generic name (as they do not contain any json-specific
code anyway). This patch is a preparation to actually using the new
S-Expression sprinter in notmuch-reply and notmuch-show.
There are several levels of function calls between where we create the
sprinter and the call to the part formatter in show_message. This
feeds the sprinter through all of them and into the part formatters.
This associates an sprinter constructor with each show format and uses
this to construct the appropriate sprinter. Currently nothing is done
with this sprinter, but the following patches will weave it through
the layers of notmuch show.
This option allows the caller to suppress the output of the bodies of
the messages. Currently this is only implemented for format=json.
This is used by notmuch-pick.el (although not needed) because it gives
a speed-up of at least a factor of a two (and in some cases a speed up
of more than a factor of 8); moreover it reduces the memory usage in
emacs hugely.
All formats except Json can output empty messages for non
entire-thread, but in Json format we output "null" to keep the other
elements (e.g. the replies to the omitted message) in the correct
place.
This has the affect of lazily creating the crypto contexts only when
needed. This removes code duplication from notmuch-show and
notmuch-reply, and should speed up these functions considerably if the
crypto flags are provided but the messages don't have any
cryptographic parts.
Use this flag rather than depend on the existence of an initialized
gpgctx, to determine whether we should verify a multipart/signed. We
will be moving to create the ctx lazily, so we don't want to depend on
it being previously initialized if it's not needed.
This new structure, notmuch_crypto_t, keeps all relevant crypto
contexts and parameters together, and will make it easier to pass the
stuff around and clean it up. The name of the crypto context inside
this new struct will change, to reflect that it is actually a GPG
context, which is a sub type of Crypto context. There are other types
of Crypto contexts (Pkcs7 in particular, which we hope to support) so
we want to be clear.
The new crypto.c contains functions to return the proper context from
the struct for a given protocol (and initialize it if needed), and to
cleanup a struct by releasing the crypto contexts.
gmime 2.4 defines GMimeCipherContext, while 2.6 defines
GMimeCryptoContext. typedef them both to notmuch_crypto_context_t to
cover this discrepancy and remove a bunch of #ifdefs.
This moves notmuch show to the --exclude=(true|false) naming
scheme. When exclude=false show returns all threads that match
including those that only match in an excluded message. The excluded
messages are flagged.
When exclude=true the behaviour depends on whether --entire-thread is
set. If it is not set then show only returns the messages which match
and are not excluded. If it is set then show returns all messages in
the threads that match in a non-excluded message, flagging the excluded
messages in these threads. The rationale is that it is awkward to use
a thread with some missing messages.
Previously, show and reply had separate implementations of decoding
and printing text parts. Now both use show's implementation, which
was more complete. Show's implementation has been extended with an
option to add reply quoting to the extracted part (this is implemented
as a named flag to avoid naked booleans, even though it's the only
flag it can take).
This new JSON format for replies includes headers generated for a
reply message as well as the headers of the original message. Using
this data, a client can intelligently create a reply. For example, the
emacs client will be able to create replies with quoted HTML parts by
parsing the HTML parts.
Formatter errors are propagated to the exit status of notmuch show.
This isn't used by the JSON or text formatters, but it will be useful
for the raw format, which is pickier.
A new configuration key 'new.ignore' is used to determine which
files and directories user wants not to be scanned as new mails.
Mark the corresponding test as no longer broken.
This work merges my previous attempts and Andreas Amann's work
in id:"ylp7hi23mw8.fsf@tyndall.ie"
Use notmuch_bool_t instead of int for entire_thread, raw, and decrypt
boolean fields in notmuch_show_params_t. No functional changes.
Signed-off-by: Jani Nikula <jani@nikula.org>
This callback is the gateway to the new mime_node_t-based formatters.
This maintains backwards compatibility so the formatters can be
transitioned one at a time. Once all formatters are converted, the
formatter structure can be reduced to only message_set_{start,sep,end}
and part, most of show_message can be deleted, and all of
show-message.c can be deleted.
This makes the part numbers readily accessible to formatters.
Hierarchical part numbering would be a more natural and efficient fit
for MIME and may be the way to go in the future, but depth-first
numbering maintains compatibility with what we currently do.
All other config-related functions and args include the section
title in their name, so for the sake of consistency, mirror that.
Also, the "auto"matic part is a given, so that was dropped.
There are lots of API changes in gmime 2.6 crypto handling. By adding
preprocessor directives, it is however possible to add gmime 2.6 compatibility
while preserving compatibility with gmime 2.4 too.
This is mostly based on id:"8762i8hrb9.fsf@bookbinder.fernseed.info".
This was tested against both gmime 2.6.4 and 2.4.31. With gmime 2.4.31, the
crypto tests all work fine (as expected). With gmime 2.6.4, one crypto test is
currently broken (signature verification with signer key unavailable), most
likely because of a bug in gmime which will hopefully be fixed in a future
version.
