All formats except Json can output empty messages for non
entire-thread, but in Json format we output "null" to keep the other
elements (e.g. the replies to the omitted message) in the correct
place.
This has the affect of lazily creating the crypto contexts only when
needed. This removes code duplication from notmuch-show and
notmuch-reply, and should speed up these functions considerably if the
crypto flags are provided but the messages don't have any
cryptographic parts.
Use this flag rather than depend on the existence of an initialized
gpgctx, to determine whether we should verify a multipart/signed. We
will be moving to create the ctx lazily, so we don't want to depend on
it being previously initialized if it's not needed.
It has been a long-standing issue that notmuch_database_open doesn't
return any indication of why it failed. This patch changes its
prototype to return a notmuch_status_t and set an out-argument to the
database itself, like other functions that return both a status and an
object.
In the interest of atomicity, this also updates every use in the CLI
so that notmuch still compiles. Since this patch does not update the
bindings, the Python bindings test fails.
This moves notmuch show to the --exclude=(true|false) naming
scheme. When exclude=false show returns all threads that match
including those that only match in an excluded message. The excluded
messages are flagged.
When exclude=true the behaviour depends on whether --entire-thread is
set. If it is not set then show only returns the messages which match
and are not excluded. If it is set then show returns all messages in
the threads that match in a non-excluded message, flagging the excluded
messages in these threads. The rationale is that it is awkward to use
a thread with some missing messages.
Previously, show and reply had separate implementations of decoding
and printing text parts. Now both use show's implementation, which
was more complete. Show's implementation has been extended with an
option to add reply quoting to the extracted part (this is implemented
as a named flag to avoid naked booleans, even though it's the only
flag it can take).
This new JSON format for replies includes headers generated for a
reply message as well as the headers of the original message. Using
this data, a client can intelligently create a reply. For example, the
emacs client will be able to create replies with quoted HTML parts by
parsing the HTML parts.
This is fully compatible for root and leaf parts, but now has proper
support for interior parts. This requires some design decisions that
were guided by what I would want if I were to save a part.
Specifically:
- Leaf parts are printed without headers and with transfer decoding.
This is what makes sense for saving attachments. (Furthermore, the
transfer decoding is necessary since, without the headers, the
caller would not be able to interpret non-transfer-decoded output.)
- Message parts are printed with their message headers, but without
enclosing part headers. This is what makes sense for saving a
message as a whole (which is a message part) and for saving attached
messages. This is symmetric for whole messages and for attached
messages, though we special-case the whole message for performance
reasons (and corner-case correctness reasons: given malformed input,
GMime may not be able to reproduce it from the parsed
representation).
- Multipart parts are printed with their headers and all child parts.
It's not clear what the best thing to do for multipart is, but this
was the most natural to implement and can be justified because such
parts can't be interpreted without their headers.
As an added benefit, we can move the special-case code for part 0 into
the raw formatter.
Formatter errors are propagated to the exit status of notmuch show.
This isn't used by the JSON or text formatters, but it will be useful
for the raw format, which is pickier.
In all cases of notmuch count/search/show where the results returned
cannot reflect the exclude flag return just the matched not-excluded
results. If the caller wishes to have all the matched results (i.e.,
including the excluded ones) they should call with the
--no-exclude option.
The relevant cases are
count: both threads and messages
search: all cases except the summary view
show: mbox format
This adds the excludes to notmuch-show.c. We do not exclude when only
a single message (or part) is requested. notmuch-show will output the
exclude information when either text or json format is requested. As
this changes the output from notmuch-show it breaks many tests (in a
trivial and expected fashion).
This has three ramifications:
- Blank To and Cc headers are no longer output for messages.
- Dates are now canonicalized for messages, which means they always
have a day of the week and GMT is printed +0000 (never -0000)
- Invalid From message headers are handled slightly differently, since
they get parsed by GMime now instead of notmuch.
Previously, top-level message headers were printed as Subject, From,
To, Date, while embedded message headers were printed From, To,
Subject, Date. This makes both cases use the former order and updates
the tests accordingly.
As before, this is all code movement and a smidgen of glue. This
moves the existing JSON formatter code into one self-recursive
function, but doesn't change any of the logic to take advantage of the
new structure.
In general, "leafs" of the JSON structure are left in helper functions
(most of them untouched), so that it's easy to see the overall
structure of the format from the main recursive function.
The last lines of notmuch_show_command() function were
unreachable. Fix it by using a variable for return value.
Signed-off-by: Jani Nikula <jani@nikula.org>
Use the new notmuch argument parser to handle arguments in "notmuch
show". There are three minor functional changes:
1) Also set params.raw = TRUE when defaulting to raw format when part
is requested but format is not specified. This was a bug, and
--part=0 without --format=raw did not work previously.
2) Set params.decrypt = FALSE if crypto context creation fails.
3) Only use the parameters for the last --format if specified multiple
times. Previously this could have resulted in a non-working mixture
of parameters.
Signed-off-by: Jani Nikula <jani@nikula.org>
Use notmuch_bool_t instead of int for entire_thread, raw, and decrypt
boolean fields in notmuch_show_params_t. No functional changes.
Signed-off-by: Jani Nikula <jani@nikula.org>
This makes the text formatter take advantage of the new code
structure. The previously duplicated header logic is now unified,
several things that we used to compute repeatedly across different
callbacks are now computed once, and the code is simpler overall and
32% shorter.
Unifying the header logic causes this to format some dates slightly
differently, so the two affected test cases are updated.
This is all code movement and a smidgen of glue. This moves the
existing text formatter code into one self-recursive function, but
doesn't change any of the logic. The next patch will actually take
advantage of what the new structure has to offer.
Note that this patch retains format_headers_message_part_text because
it is also used by the raw format.
This callback is the gateway to the new mime_node_t-based formatters.
This maintains backwards compatibility so the formatters can be
transitioned one at a time. Once all formatters are converted, the
formatter structure can be reduced to only message_set_{start,sep,end}
and part, most of show_message can be deleted, and all of
show-message.c can be deleted.
Previously, top-level message headers were printed as Subject, From,
To, Date, while embedded message headers were printed From, To,
Subject, Date. This makes both cases use the former order and updates
the tests accordingly.
Strangely, the raw format also uses this function, so this also fixes
the two raw format tests affected by this change.
JSON does not support hex literals (0x..) so numbers must be formatted
as %d instead of %x.
Currently, the possible values for the gmime error code are 1 (expired
signature), 2 (no public key), 4 (expired key) and 8 (revoked key).
The other possible value is 16 (unsupported algorithm) but obviously
it is much more rare. If this happens, the current code will add
'"errors": 10'. This is valid JSON (it looks like a decimal number)
but it is incorrect (should be 16, not 10).
Since this is just an issue in the JSON encoder, no changes are needed
on the Emacs side (or in other UIs using the JSON output).
There are lots of API changes in gmime 2.6 crypto handling. By adding
preprocessor directives, it is however possible to add gmime 2.6 compatibility
while preserving compatibility with gmime 2.4 too.
This is mostly based on id:"8762i8hrb9.fsf@bookbinder.fernseed.info".
This was tested against both gmime 2.6.4 and 2.4.31. With gmime 2.4.31, the
crypto tests all work fine (as expected). With gmime 2.6.4, one crypto test is
currently broken (signature verification with signer key unavailable), most
likely because of a bug in gmime which will hopefully be fixed in a future
version.
For showing a message in raw format, rather than silently succeeding
when a read or a write fails (or, probably, looping if a read fails),
try to print an error message and exit with a non-zero status.
This silences one of the buildbot warnings about unused results. While
my libc lacks the declarations that trigger these warnings, this can
be tested by adding the following to notmuch.h:
__attribute__((warn_unused_result))
size_t fwrite(const void *ptr, size_t size, size_t nmemb, FILE *stream);
In my test case added g_object_unref(charset_filter) reduces memory
consumption over 90% when 'notmuch show --format=text "*"' is
executed (~11000 messages, RES ~330M -> ~25M).
g_object_unref() releases the memory of the InternetAddressList object
returned by internet_address_list_parse_string() -- when last (only)
reference is released, internet_address_list_finalize() will do cleanup.
previously we deleted the subcommand name from argv before passing to
the subcommand. In this version, the deletion is done in the actual
subcommands. Although this causes some duplication of code, it allows
us to be more flexible about how we parse command line arguments in
the subcommand, including possibly using off-the-shelf routines like
getopt_long that expect the name of the command in argv[0].
This was a minor oversite in checking of part type when outputing
content raw. This was causing gmime was to throw an exception to
stderr.
Unfortunately the gmime exception was not being caught by notmuch, or
the test suite. I'm not sure if notmuch should have done anything in
this case, but certainly the test suite should be capable of detecting
that something unexpected was output to stderr.
This new function takes a GMimeMessage as input, and outputs the
formatted headers. This allows for message/rfc822 parts to be
formatted on output in a similar way to full messages (see previous
patch that overhauls the multipart test for more info).
Before the change, notmuch show output had filename only for
parts with "Content-Disposition: attachment". But parts with
inline disposition may have filename as well.
The patch makes notmuch show always output filename if available,
independent of Content-Disposition. Both JSON and text output
formats are changed.
Also, the patch adds Content-id to text output format of notmuch
show.
The main goal of these changes is to have filenames on Emacs
buttons for inline attachments. In particular, this is very
helpful for inline patches.
Note: text format changes may require updates in clients that use
it. The changes are:
* text part header format changed from:
^Lpart{ ID: 2, Content-type: text/x-diff
to:
^Lpart{ ID: 2, Filename: cool-feature.patch, Content-type: text/x-diff
* attachment format changed from:
^Lattachment{ ID: 4, Content-type: application/octet-stream
Attachment: data.tar.bz2 (application/octet-stream)
Non-text part: application/octet-stream
^Lattachment}
to:
^Lattachment{ ID: 4, Filename: data.tar.bz2, Content-type: application/octet-stream
Non-text part: application/octet-stream
^Lattachment}
Our use of GMimeSession was unneeded boilerplate, and we weren't doing
anything with it. This simplifies and clarifies that assumption.
If we want to do anything fancier later, the examples in the gmime
source are a reasonable source to work from in defining a new
GMimeSession derivative.
Since GMimeSession is going away in GMime 2.6, though, i don't
recommend using it.
GMime has a nasty habit of taking ownership by default of any FILE*
handed to it va g_mime_stream_file_new. Specifically it will close the
FILE* when the stream is destroyed---even though GMime didn't open the
file itself.
To avoid this bad behavior, we have to carefully set_owner(FALSE)
after calling g_mime_stream_file_new. In the format_part_content_text
function, since commit d92146d3a6 we've
been calling g_mime_stream_file_new unconditionally, but only calling
g_mime_stream_file_set_owner(FALSE) conditionally.
This led to the FILE* being closed early when notmuch show output was
redirected to a file.
Fixing this fixes the test-suite cases that broke with the previous
commit, (which added redirected "notmuch show" calls to the test suite
to expose this bug).
Edited-by: Carl Worth <cworth@cworth.org> with a new commit message to
explain the bug and fix.
