Prepare for further future unification by making the code similar. The
only functional change is that errors in mime_node_open() also break
execution in default reply format.
There's quite a bit of duplication, and some consequent deviation,
between the various notmuch reply format code paths. Perform the query
and message iteration in common code, and make the format specific
functions operate on single messages.
There should be no functional changes.
Many of the external links found in the notmuch source can be resolved
using https instead of http. This changeset addresses as many as i
could find, without touching the e-mail corpus or expected outputs
found in tests.
In case of notmuch reply and notmuch show --part=N it is required that
search terms match to one message. If match count was != 1, error
message "Error: search term did not match precisely one message"
was too vague to explain what happened.
By appending (matched <num> messages) to the error message it
makes the problem more understandable (e.g when <num> is '0'
user reckons the query had a typo in it).
Current documentation and comments in the code do not correspond to
the actual code and tests in the test suite ("Un-munging Reply-To" in
T230-reply-to-sender.sh). Fix it.
Per RFC 2183, the values for Content-Disposition values are not
case-sensitive. While at it, use the gmime function for getting at the
disposition string instead of referencing the field directly.
This fixes attachment display and quoting in notmuch show and reply,
respectively.
I think it would be no real problem to cut and paste the gdb based
error message test from count to the other clients modified here, but
I'm not currently convinced it's worth the trouble since the code path
being tested is almost the the same, and the tests are relatively
heavyweight.
The function notmuch_exit_if_unmatched_db_uuid is split from
notmuch_process_shared_options because it needs an open notmuch
database.
There are two exceptional cases in uuid handling.
1) notmuch config and notmuch setup don't currently open the database,
so it doesn't make sense to check the UUID.
2) notmuch compact opens the database inside the library, so we either
need to open the database just to check uuid, or change the API.
Unfortunately it seems trickier to support --config globally
The non-trivial changes are in notmuch.c; most of the other changes
consists of blindly inserting two lines into every subcommand.
Previously we set up a way for the top level notmuch command to choose
which gpg binary was invoked by libgmime. In this commit we add the
(mostly boilerplate) code to allow the notmuch-config command to read
and write this path, and use it in the appropriate struct.
Update tests for new default variable
GMIME takes a path to gpg, but we hardcode that path. In this commit
we set up argument passing and option storage to allow this path to
specified in the top level notmuch command.
Apart from the status codes for format mismatches, the non-zero exit
status codes have been arbitrary. Make the cli consistently return
either EXIT_SUCCESS or EXIT_FAILURE.
For my client, the largest bottleneck for displaying large threads is
exporting each html part individually since by default notmuch will not
show the json parts. For large threads there can be quite a few parts and
each must be exported and decoded one by one. Also, I then have to deal
with all the crazy charsets which I can do through a library but is a
pain.
Therefore, this patch adds an --include-html option that causes the
text/html parts to be included as part of the output of show.
diff man/man1/notmuch-show.1
Previously, reply's default text format used an odd mix of RFC 2045
MIME encoding for the reply template's body and some made-up RFC
2822-like UTF-8 format for the headers. The intent was to present the
headers to the user in a nice, un-encoded format, but this assumed
that whatever ultimately sent the email would RFC 2047-encode the
headers, while at the same time the body was already RFC 2045 encoded,
so it assumed that whatever sent the email would *not* re-encode the
body.
This can be fixed by either producing a fully decoded UTF-8 reply
template, or a fully encoded MIME-compliant RFC 2822 message. This
patch does the latter because it is
a) Well-defined by RFC 2822 and MIME (while any UTF-8 format would be
ad hoc).
b) Ready to be piped to sendmail. The point of the text format is to
be minimal, so a user should be able to pop up the template in
whatever editor they want, edit it, and push it to sendmail.
c) Consistent with frontend capabilities. If a frontend has the
smarts to RFC 2047 encode the headers before sending the mail, it
probably has the smarts to RFC 2047 decode them before presenting
the template to a user for editing.
Also, as far as I know, nothing automated consumes the reply text
format, so changing this should not cause serious problems. (And if
anything does still consume this format, it probably gets these
encoding issues wrong anyway.)
Previously, the References header code seemed to assume
notmuch_message_get_header would return NULL if the header was not
present, but it actually returns "". As a result of this, it was
inserting an unnecessary space when concatenating an empty or missing
original references header with the new reference.
This shows up in only two tests because the text reply format later
passes the whole reply template through g_mime_filter_headers, which
has the side effect of stripping out this extra space.
This allows specifying config file as a top level argument to notmuch,
and generally makes it possible to override config file options in
main(), without having to touch the subcommands.
If the config file does not exist, one will be created for the notmuch
main command and setup and help subcommands. Help is special in this
regard; the config is created just to avoid errors about missing
config, but it will not be saved.
This also makes notmuch config the talloc context for subcommands.
We now have a notmuch_config_is_new() function to query whether a
config was created or not. Change the notmuch_config_open() is_new
parameter into boolean create_new to determine whether the function
should create a new config if one doesn't exist. This reduces the
complexity of the API.
All the structured output functions in notmuch-reply and notmuch-show
are renamed to a generic name (as they do not contain any json-specific
code anyway). This patch is a preparation to actually using the new
S-Expression sprinter in notmuch-reply and notmuch-show.
There are several levels of function calls between where we create the
sprinter and the call to the part formatter in show_message. This
feeds the sprinter through all of them and into the part formatters.
This option allows the caller to suppress the output of the bodies of
the messages. Currently this is only implemented for format=json.
This is used by notmuch-pick.el (although not needed) because it gives
a speed-up of at least a factor of a two (and in some cases a speed up
of more than a factor of 8); moreover it reduces the memory usage in
emacs hugely.
This has the affect of lazily creating the crypto contexts only when
needed. This removes code duplication from notmuch-show and
notmuch-reply, and should speed up these functions considerably if the
crypto flags are provided but the messages don't have any
cryptographic parts.
Use this flag rather than depend on the existence of an initialized
gpgctx, to determine whether we should verify a multipart/signed. We
will be moving to create the ctx lazily, so we don't want to depend on
it being previously initialized if it's not needed.
Add another fallback header Delivered-To for guessing the user's from
address for notmuch reply before using the Received
headers. Apparently some MTAs use Delivered-To instead of
X-Original-To (which already exists as a fallback).
Reported-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Signed-off-by: Jani Nikula <jani@nikula.org>
Get rid of user address matching code duplication in
guess_from_received_header() by using the new address matching
helpers.
No functional changes.
Signed-off-by: Jani Nikula <jani@nikula.org>
Add a multi-purpose address_match() function for matching strings
against user's configured primary and other email addresses. Add thin
wrappers user_address_in_string() and string_in_user_address() for
ease of use, and also convert existing address_is_users() to wrapper
for the same.
No functional changes.
Signed-off-by: Jani Nikula <jani@nikula.org>
It has been a long-standing issue that notmuch_database_open doesn't
return any indication of why it failed. This patch changes its
prototype to return a notmuch_status_t and set an out-argument to the
database itself, like other functions that return both a status and an
object.
In the interest of atomicity, this also updates every use in the CLI
so that notmuch still compiles. Since this patch does not update the
bindings, the Python bindings test fails.
This makes more logical sense, since it makes the recursive printer
responsible for the entire reply body and lets it start at the root of
the MIME tree instead of the first child. (We could move reply header
creation in there, too, but if we ever support proper reply to
multiple messages, we'll want just one set of reply headers computed
from the entire message set and many bodies.)
Previously, show and reply had separate implementations of decoding
and printing text parts. Now both use show's implementation, which
was more complete. Show's implementation has been extended with an
option to add reply quoting to the extracted part (this is implemented
as a named flag to avoid naked booleans, even though it's the only
flag it can take).
This new JSON format for replies includes headers generated for a
reply message as well as the headers of the original message. Using
this data, a client can intelligently create a reply. For example, the
emacs client will be able to create replies with quoted HTML parts by
parsing the HTML parts.
Use notmuch_bool_t instead of int for entire_thread, raw, and decrypt
boolean fields in notmuch_show_params_t. No functional changes.
Signed-off-by: Jani Nikula <jani@nikula.org>
This callback is the gateway to the new mime_node_t-based formatters.
This maintains backwards compatibility so the formatters can be
transitioned one at a time. Once all formatters are converted, the
formatter structure can be reduced to only message_set_{start,sep,end}
and part, most of show_message can be deleted, and all of
show-message.c can be deleted.
There are lots of API changes in gmime 2.6 crypto handling. By adding
preprocessor directives, it is however possible to add gmime 2.6 compatibility
while preserving compatibility with gmime 2.4 too.
This is mostly based on id:"8762i8hrb9.fsf@bookbinder.fernseed.info".
This was tested against both gmime 2.6.4 and 2.4.31. With gmime 2.4.31, the
crypto tests all work fine (as expected). With gmime 2.6.4, one crypto test is
currently broken (signature verification with signer key unavailable), most
likely because of a bug in gmime which will hopefully be fixed in a future
version.
Add new option --reply-to=(all|sender) to "notmuch reply" to select whether
to reply to all (sender and all recipients), or just sender. Reply to all
remains the default.
Credits to Mark Walters <markwalters1009@gmail.com> for his similar earlier
work where I picked up the basic idea of handling reply-to-sender in
add_recipients_from_message(). All bugs are mine, though.
Signed-off-by: Jani Nikula <jani@nikula.org>
Slightly refactor "notmuch reply" recipient and user from address scanning
functions in preparation for reply-to-sender feature.
Add support for not adding recipients at all (just scan for user from
address), and returning the number of recipients added.
No externally visible functional changes.
Signed-off-by: Jani Nikula <jani@nikula.org>
Use the new notmuch argument parser to handle arguments in "notmuch
reply". There should be no functional changes.
Signed-off-by: Jani Nikula <jani@nikula.org>
notmuch_show_params_t params is only initialized partially in
notmuch_reply_command(). The only field that is used uninitialized is
params.decrypt. It is usually non-zero, making "notmuch reply" on encrypted
messages work by coincidence.
Initialize params properly, and set params.decrypt as needed.
Signed-off-by: Jani Nikula <jani@nikula.org>
previously we deleted the subcommand name from argv before passing to
the subcommand. In this version, the deletion is done in the actual
subcommands. Although this causes some duplication of code, it allows
us to be more flexible about how we parse command line arguments in
the subcommand, including possibly using off-the-shelf routines like
getopt_long that expect the name of the command in argv[0].
This new function takes a GMimeMessage as input, and outputs the
formatted headers. This allows for message/rfc822 parts to be
formatted on output in a similar way to full messages (see previous
patch that overhauls the multipart test for more info).
Various typo fixes in comments within the source code.
Signed-off-by: Pieter Praet <pieter@praet.org>
Edited-by: Carl Worth <cworth@cworth.org> Restricted to just
source-code comments, (and fixed fix of "descriptios" to "descriptors"
rather than "descriptions").
Before the change, the last loop in guess_from_received_header()
did not reset domain and tld variables to NULL. This leads to
double free error in some cases and possibly other bugs.
Our use of GMimeSession was unneeded boilerplate, and we weren't doing
anything with it. This simplifies and clarifies that assumption.
If we want to do anything fancier later, the examples in the gmime
source are a reasonable source to work from in defining a new
GMimeSession derivative.
Since GMimeSession is going away in GMime 2.6, though, i don't
recommend using it.
This adds support for decrypting PGP/MIME-encrypted parts to
notmuch-show and notmuch-reply. The --decrypt option implies
--verify. Once decryption (and possibly signature verification) is
done, a new part_encstatus formatter is emitted, the part_sigstatus
formatter is emitted, and the entire multipart/encrypted part is
replaced by the contents of the encrypted part.
At the moment only a json part_encstatus formatting function is
available, even though decryption is done for all formats. Emacs
support to follow.
This is primarily for notmuch-show, although the functionality is
added to show-message. Once signatures are processed a new
part_sigstatus formatter is emitted, and the entire multipart/signed
part is replaced with the contents of the signed part.
At the moment only a json part_sigstatus formatting function is
available. Emacs support to follow.
The original work for this patch was done by
Daniel Kahn Gillmor <dkg@fifthhorseman.net>
whose help with this functionality I greatly appreciate.
Future improvements (eg. crypto support) will require adding new part
header. By breaking up the output of part headers from the output of
part content, we can easily out new part headers with new formatting
functions.
After the last patch to eliminate some redundant code paths in
reply_part, the reply_part_content function was only being called
once. Disolving the function and integrating its contents into the
reply_part function makes things a little simpler, and frees up some
name space that will be needed in the next patch.
Outputting of single MIME parts is moved to an option of notmuch show,
instead of being handled in it's own sub-command. The recent rework
of multipart mime allowed for this change but consolidating part
handling into a single recursive function (show_message_part) that
includes formatting. This allows for far simpler handling single
output of a single part, including formatting.
A new field "part_sep" is added to the notmuch_show_format structure,
to be used for part separation. This is cleaner than the "first"
argument that was being passed around to the part arguments, and
allows the function that handles overall part output formatting
(show_message_part) to directly handle when outputting the separator.
Various show_message* functions require formatting functions, which
were previously being passed individually as arguments. Since we will
need to be needing to passing in more formatting function in the
future (ie. for crypto support), we here modify things so that we just
pass in the entire format structure. This will make things much
simpler down the line as we need to pass in new format functions.
We move the show_format structure into notmuch-client.c as
notmuch_show_format. This also affects notmuch-reply.c, so we create
a mostly-empty format_reply to pass the reply_part function to
show_message_body.
Previously, notmuch show flattened all output, losing information
about the nesting of the MIME hierarchy. Now, the output is properly
nested, (both in the --format=text and --format=json output), so that
clients can analyze the original MIME structure.
Internally, this required splitting the final closing delimiter out of
the various show_part functions and putting it into a new
show_part_end function instead. Also, the show_part function now
accepts a new "first" argument that is set not only for the first MIME
part of a message, but also for each first MIME part within a series
of multipart parts. This "first" argument controls the omission of a
preceding comma when printing a part (for json).
Many thanks to David Edmondson <dme@dme.org> for originally
identifying the lack of nesting in the json output and submitting an
early implementation of this feature. Thanks as well to Jameson Graef
Rollins <jrollins@finestructure.net> for carefully shepherding David's
patches through a remarkably long review process, patiently explaining
them, and providing a cleaned up series that led to this final
implementation. Jameson also provided the new emacs code here.
These were introduced as a side-effect of commit
b9eac48c22 (shame on me for doing
side-effect commits like that!).
For me, at least, compilation is now warning-free.
Notmuch reply should not be Bcc'ing the sender by default. This is
not the appropriate way to save copies of sent mail (which should
probably be handled by an Fcc header[*]) and it doesn't give the user the
option to not be bcc'd. This is really something that should be
handled by the reader UI. For instance, emacs message-mode can easily
be configured to add Bcc's if the user wishes.
[*] Carl Worth: The FCC header is now in place by default in the emacs
user-interface (and tested in the test suite) so the Bcc is ready to
be eliminated.
i is already used in a for loop at this point, so using i here again
broke notmuch-reply (it would just hang). Use j instead of i here.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
We want to be able to correctly guess the best From: header to use when
replying to emails. This is what we are looking at now:
1 is one of the users' mail addresses in the To: or Cc: header
2 check for an Envelope-to: header
3 check for an X-Original-To: header
4 check for a (for <email@add.res>) clause in Received: headers
5 check for the domain part of known email addresses in the
'by' part of Received headers
6 fall back to the primary email address
This patch changes the algorithm for steps 2-5 of this process. Prior to
this patch we had a first attempt to implement only step 5 - but this
broke in many email setups where mail delivery to the local machine added
additional Received: lines.
Steps 2-4 are new, step 5 now analyzes the concatenated Received: header
(this was in the previous patch) to do this analysis.
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
This seems a rather unlikely case, (replying to a message that
disappears out from under us half way through the reply), but
notmuch_message_get_header is documented to return NULL in error
cases, so we might as well deal sanely with that (rather than just
crashing).
When headers contain non-ASCII characters, they are encoded according
to rfc2047. Nomtuch reply command emits the headers in the encoded
form, which makes them hard to read by humans who compose the reply.
For example instead of "Subject: Re: Rozlučka" one currently sees
"Subject: Re: =?iso-8859-2?q?Rozlu=E8ka?=".
This patch adds a new GMime filter which is used to decode headers to
UTF-8 and uses this filter when notmuch reply outputs headers.
Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
The previous code made too many assumptions about the (sadly not
standardized) format of the Received headers. This version should
be more robust to deal with different variations.
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
When replying to a message notmuch tries to pick the correct From
address by looking which one of a user's configured email addresses
were included in To or Cc headers of the email that is being replied to.
If none of the users email addresses are in the To or Cc headers we now
try to guess from the first (chronologically, last) Received header
which domain this email was received in and therefore which of the
email addresses to use in a reply
If that fails we still use the primary email as From email
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
We rename 'has_more' to 'valid' so that it can function whether
iterating in a forward or reverse direction. We also rename
'advance' to 'move_to_next' to setup parallel naming with
the proposed functions 'move_to_first', 'move_to_last', and
'move_to_previous'.
This function detects whether the address in the Reply-To header
already appears in either To or Cc. So give it a name that reflects
what it does (reply_to_header_is_redundant) rather than the old name
which described one possible use of the function, (as a simple
heuristic for detecting whether a mailing list had applied reply-to
munging).
Apparently, GMime doesn't want to create a valid address list object
for an empty string. That's annoying, but it's easy enough to test for
the empty string and avoid the problem.
This change was already recommended in a comment in the original
implementation of this patch. If someone really wants to support
un-munging in the case of To: and Reply-To: having the same address
but different case, then they can provide a portable approach for
that.
Some mailing lists engage in the evil practice of changing the Reply-To
header so that replies from all mailers go to the list by default, at
the expense of not responding to the person who actually sent the
message. When this is detected, we reply to `From' and remove the
duplicate response to the mailing list. Consider a reply to the
following message.
From: Some User <some.user@example.com>
To: Sample users list <sample-users@sample.org>
Reply-To: Sample users list <sample-users@sample.org>
Prior to this patch, `notmuch reply' produces
To: Sample users list <sample-users@sample.org>,
Sample users list <sample-users@sample.org>
and after the patch,
To: Some User <some.user@example.com>,
Sample users list <sample-users@sample.org>
Signed-off-by: Jed Brown <jed@59A2.org>
This code was already duplicated. We move it to a new, shared
add_recipients_from_message function, in preparation for more
sophisticated mailing list logic.
Signed-off-by: Jed Brown <jed@59A2.org>
Pass the message through the charset filter so that we can view
messages wrote in different charset encoding.
Signed-off-by: Kan-Ru Chen <kanru@kanru.info>
This command only generates References, To, and Cc headers.
The purpose is primarily for use in
git send-email --notmuch id:<MESSAGE-ID>
to get proper threading and address the relevant parties. Hooks for
other SCMs may come later.
Signed-off-by: Jed Brown <jed@59A2.org>
This factors actual generation of the reply out of notmuch_reply_command
into notmuch_reply_format_default(), in preparation for other --format=
options.
Signed-off-by: Jed Brown <jed@59A2.org>
The domain is alway case insensitive, but in principle the username is
case sensitive. Few systems actually enforce this so I think a good
default is to treat the entire address as case insensitive, it will
eliminate a lot of superfluous self-addressed messages and reply from
the correct address in these cases.
Signed-off-by: Jed Brown <jed@59A2.org>
We only rarely need to actually open the database for writing, but we
always create a Xapian::WritableDatabase. This has the effect of
preventing searches and like whilst updating the index.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Acked-by: Carl Worth <cworth@cworth.org>
