In addition to being the Right Thing to do, this noticeably improves
the time taken to display the first page of search results, since it's
roughly an order of magnitude faster than the JSON parser.
Interestingly, it does *not* significantly improve the time to
completely fill a large search buffer because for large search
buffers, the cost of creating author invisibility overlays and
inserting text (which slows down with more overlays) dominates.
However, the time required to display the first page of results is
generally more important to the user experience.
Previously, search started the async notmuch process directly. Now,
it uses `notmuch-start-notmuch'. This simplifies the process sentinel
a bit and means that we no longer have to worry about errors
interleaved with the JSON output.
We also update the tests of Emacs error handling, since the error
output is now separated from the search results buffer.
This unifies the part button actions and the underlying part action
functions into single interactive command that simply applies to the
part containing point using the just-added part p-list text property
instead of button properties. Since all part actions can be performed
by applying the appropriate mm function to an mm-handle, this patch
abstracts out the creation of mm handles, making the implementations
of the part commands trivial. This also eliminates our special
handling for part save in favor of using the appropriate mm function.
This necessarily modifies the way we handle the default part button
action, but in a way that does not change the meaning of the
notmuch-show-part-button-default-action defcustom.
Since these commands are no longer specific to buttons, this patch
eliminates the extra metadata stored with each button. This also
eliminates one rather special-purpose macro for a collection of
general purpose part handling utilities.
We recently switched to popping up a buffer to report CLI errors, but
this was too intrusive, especially for transient errors and especially
since we made fewer things ignore errors. This patch changes this to
display a basic error message in the minibuffer (using Emacs' usual
error handling path) and, if there are additional details, to log
these to a separate error buffer and reference the error buffer from
the minibuffer message. This is more in line with how Emacs typically
handles errors, but makes the details available to the user without
flooding them with the details.
Given this split, we pare down the basic message and make it more
user-friendly, and also make the verbose message even more detailed
(and more debugging-oriented).
* test/emacs:
- Rename subtests "{Add,Remove} tag from notmuch-show view" to
"notmuch-show: {add,remove} single tag {to,from} single message"
to be consistent with the following tests.
- New subtest "notmuch-show: add multiple tags to single message":
`notmuch-show-add-tag' ("+") can add multiple tags to a message.
- New subtest "notmuch-show: remove multiple tags from single message":
`notmuch-show-remove-tag' ("-") can remove multiple tags from a message.
The macro with-current-notmuch-show-message executes command
`notmuch show --format=raw id:...` which just outputs the contents
of the mail file verbatim (into temporary buffer). In case e.g. utf-8
locale is used the temporary buffer has buffer-file-coding-system as
utf-8. In this case Emacs converts the data to multibyte format, guessing
that input is in utf-8.
However, the "raw" (MIME) message may contain octet data in any other
8bit format, and as no (MIME-)content spesific handling to the message
is done at this point, conversion to other formats may lose information.
By setting coding-system-for-read 'no-conversion drops the conversion part
and makes this handle input as notmuch-get-bodypart-internal() does.
This marks the broken test in previous change fixed.
There are currently 45 TESTS scripts. 36 of those load
test-lib.sh using '. ./test-lib.sh' and 9 '. test-lib.sh'.
In latter case test-lib.sh is first searched from directories
in PATH (posix) and then from current directory (bash feature).
Changed the 9 files to execute '. ./test-lib.sh'. The test-lib.sh
should never be loaded from directory in PATH.
Previously, this would simply indicate that the grep failed without
any indication of the Emacs output it failed on. Now we take
advantage of the test framework's handling of stdout to display the
incorrect Emacs output if the test fails.
The test designed to exercise Emacs' rendering of HTML emails
containing images inadvertently assumed w3m was available under Emacs
23. The real point of this test was to check that Emacs 24's shr
renderer didn't crash when given img tags, so use shr if it's
available, html2text otherwise (which is built in), and do only a
simple sanity check of the result.
Currently, we only properly escape stashed id queries, but there are
other places where the Emacs UI constructs queries for boolean terms.
Since this escaping function is meant to be used in other places, it
avoids escaping strings that don't need escaping.
* test/emacs:
- New subtest "notmuch-show: collapse all messages in thread":
`notmuch-show-open-or-close-all' with prefix arg ("C-u M-RET")
collapses all messages in thread.
- New subtest "notmuch-show: uncollapse all messages in thread":
`notmuch-show-open-or-close-all' without prefix arg ("M-RET")
uncollapses all messages in thread.
* test/emacs:
- New subtest "notmuch-show: show message headers":
Setting `notmuch-message-headers-visible' to t causes all headers
defined in `notmuch-message-headers' to be shown.
- New subtest "notmuch-show: hide message headers":
Setting `notmuch-message-headers-visible' to nil causes all headers
defined in `notmuch-message-headers' to be hidden.
("Subject:" may be an exception; See the use of `headers-start' in
`notmuch-show-insert-msg')
- New subtest "notmuch-show: hide message headers (w/ notmuch-show-toggle-headers)":
Setting `notmuch-message-headers-visible' to t causes all headers
defined in `notmuch-message-headers' to be shown, but they can be
hidden for the current message by running `notmuch-show-toggle-headers'.
This requires changing the contents of the crypto tests, as one thread
that was marked read by the earlier tests in test/emacs is no longer
marked read.
This moves tests for:
- 09d19ac "test: emacs: toggle eliding of non-matching messages in
`notmuch-show'", which should have actually read: "test: emacs:
toggle processing of cryptographic MIME parts in `notmuch-show'".
See commit 19ec74c5.
- 5ea1dbe "test: emacs: toggle eliding of non-matching messages in
`notmuch-show'"
- 345faab "test: emacs: toggle thread content indentation in
`notmuch-show'"
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Most Emacs tests end with a call to (test-output), which saves the
buffer to a filed called OUTPUT. Previously, if the test code failed
with an exception before this call, the test framework would then
compare against the OUTPUT file from the last Emacs test, resulting in
confusing diffs.
This requires one tweak to an emacs test that made two calls to
test_emacs and expected an OUTPUT file from the first call. We simply
reverse the order of the test_emacs calls.
The version of message.el in emacs24 omits the charset=us-ascii,
causing the current version of this test to fail. With this patch, we
accept either option. According to RFC 2046, they are semantically
equivalent.
The JSON format eliminates the complex escaping issues that have
plagued the text search format. This uses the incremental JSON parser
so that, like the text parser, it can output search results
incrementally.
This slows down the parser by about ~4X, but puts us in a good
position to optimize either by improving the JSON parser (evidence
suggests this can reduce the overhead to ~40% over the text format) or
by switching to S-expressions (evidence suggests this will more than
double performance over the text parser). [1]
This also fixes the incremental search parsing test.
This has one minor side-effect on search result formatting.
Previously, the date field was always padded to a fixed width of 12
characters because of how the text parser's regexp was written. The
JSON format doesn't do this. We could pad it out in Emacs before
formatting it, but, since all of the other fields are variable width,
we instead fix notmuch-search-result-format to take the variable-width
field and pad it out. For users who have customized this variable,
we'll mention in the NEWS how to fix this slight format change.
[1] id:"20110720205007.GB21316@mit.edu"
This advises the search process filter to make it process one
character at a time in order to test the pessimal case for incremental
search output parsing.
The text parser fails this test because it gets tricked into thinking
a parenthetical remark in a subject is the tag list.
The output of the HTML reply test in the emacs suite can vary
depending on which HTML renderers are installed on the machine running
the tests. The renderer that is always available is emacs's builtin
html2text function. In order to get consistency, force the test to use
html2text even if other renderers are available.
Quote non-text parts nicely by displaying them with mm-display-part
before calling message-cite-original to quote them. HTML-only emails
can now be quoted correctly. We re-use some code from notmuch-show
(notmuch-show-mm-display-part-inline), which has been moved to
notmuch-lib.el.
Mark the test for this feature as not broken.
With the latest reply infrastructure, we should be able to nicely
quote HTML-only emails. But currently emacs quotes the raw HTML
instead of parsing it first. This commit adds a test for this case.
This test currently marked as broken.
This patch removes trailing spaces in notmuch-hello view.
A side effect of this change is that tag/query buttons no longer
include a space at the end. This means that pressing RET when the
point is at the first character after the tag/query button no longer
works (note that this is the standard behavior for buttons). We may
change this behavior in the future (without adding trailing spaces
back) if people would find this change inconvenient.
In the new reply code, the References header gets inserted by
message.el using a function called message-shorten-references. Unlike
all the other header-inserting functions, it doesn't put a newline
after the header, causing the next header to end up on the same
line. In our case, this header happened to be User-Agent, so it's hard
to notice. This is probably a bug in message.el, but we need to work
around it.
This fixes the problem by wrapping message-shorten-references in a
function that inserts a newline after if necessary. This should
protect against the message.el bug being fixed in the future.
By default, emacs hides the User-Agent and References headers when
composing mail. This is a good thing for users, but a bad thing for
testing, since we can create ugly or invalid headers and not have it
show up in the tests.
By setting message-hidden-headers to an empty list, we force emacs to
show all the headers, so we can check that they're correct. Users
won't see this, but it will let us catch future bugs.
As a side-effect, this breaks all the reply tests, since there is a
bug with the References and User-Agent headers, fixed in the next commit.
Bug 1: Replying from alternate addresses
----------------------------------------
The reply code was inconsistent in its use of symbols and strings for
header names being passed to message.el functions. This caused the
From header to be lookup up incorrectly, causing an additional From
header to be added with the user's primary address instead of the
correct alternate address.
This is fixed by using symbols everywhere, i.e. never using strings
for header names when interacting with message.el.
This change also removes our use of `mail-header`, since we don't use
it anywhere else, and using assq makes it clear how the header lists
are expected to work.
Bug 2: Duplicate headers in emacs 23.2
--------------------------------------
The message.el code in emacs 23.2 assumes that header names will
always be passed as symbols, so our use of strings caused
problems. The symptom was that on 23.2 (and presumably on earlier
versions) the reply message would end up with two of some headers.
Converting everything to symbols also fixes this issue.
Since the recent reply changes were pushed, there has been a bug that
causes emacs to always reply from the primary address, even if the
JSON or default CLI reply output uses an alternate address.
This adds two tests to the emacs test library based on the two "Reply
form..." tests in the reply test library. One is currently marked
broken.
This adds a lib function to turn a message ID into a properly escaped
message ID query and uses this function wherever we previously
hand-constructed ID queries. Wherever this new function is used,
documentation has been clarified to refer to "id: queries" instead of
"message IDs".
This fixes the broken test introduced by the previous patch.
Use the new JSON reply format to create replies in emacs. Quote HTML
parts nicely by using mm-display-part to turn them into displayable
text, then quoting them with message-cite-original. This is very
useful for users who regularly receive HTML-only email.
Use message-mode's message-cite-original function to create the
quoted body for reply messages. In order to make this act like the
existing notmuch defaults, you will need to set the following in
your emacs configuration:
message-citation-line-format "On %a, %d %b %Y, %f wrote:"
message-citation-line-function 'message-insert-formatted-citation-line
The tests have been updated to reflect the (ugly) emacs default.
This has three ramifications:
- Blank To and Cc headers are no longer output for messages.
- Dates are now canonicalized for messages, which means they always
have a day of the week and GMT is printed +0000 (never -0000)
- Invalid From message headers are handled slightly differently, since
they get parsed by GMime now instead of notmuch.
Consensus seems to be that people prefer that refreshing show buffers
retains state by default, rather than resetting it by default. This
turns out to be the case in the code, as well. In fact, there's even
a test for this that's been marked broken for several months, which
this patch finally gets to mark as fixed.
* emacs/notmuch-show.el
(notmuch-show-stash-mlarchive-link-alist):
New defcustom of type `alist' (key = name, value = URI),
containing Mailing List Archive URI's for searching by Message-Id.
(notmuch-show-stash-mlarchive-link-default):
New defcustom, default MLA to use when `notmuch-show-stash-mlarchive-link'
received no user input whatsoever. Available choices are generated using
the contents of `notmuch-show-stash-mlarchive-link-alist'.
(notmuch-show-stash-map):
Added keybinds "l" and "L" for `notmuch-show-stash-mlarchive-link'
respectively `notmuch-show-stash-mlarchive-link-and-go'.
(notmuch-show-stash-mlarchive-link):
New function, stashes a URI pointing to the current message at one
of the MLAs configured in `notmuch-show-stash-mlarchive-link-alist'.
Prompts user with `completing-read' if not provided with an MLA key.
(notmuch-show-stash-mlarchive-link-and-go):
New function, uses `notmuch-show-stash-mlarchive-link' to
stash a URI, and then visits it using the browser configured
in `browse-url-browser-function'.
Based on original work [1] by David Edmondson <dme@dme.org>.
[1] id:"1327397873-20596-1-git-send-email-dme@dme.org"
`notmuch-show-stash-mlarchive-link' stashes a URI pointing to the current message
at one of the MLAs configured in `notmuch-show-stash-mlarchive-link-alist'.
Marked as "broken": fixed in next commit.
After the recent tagging operations changes, functions bound to "+"
and "-" in notmuch-search and notmuch-show views always read input
from the minibuffer. Use kbd macros instead of calling them directly.
Emacs message-mode uses certain text strings to indicate how to attach
files to outgoing mail. If these are present in the text of an email,
and a user is tricked into replying to the message, the user’s files
could be exposed.
Edited-by: Pieter Praet <pieter@praet.org>: Rebased to release branch.
The test is broken at this time; the next commit will introduce a fix.
Edited-by: Pieter Praet <pieter@praet.org>:
Rebased to release branch, moved expected output into the actual test,
and fixed "Fcc:" line.
Used emacs (whitespace-cleanup) function to "cleanup blank problems"
in test files where that could be done without breaking tests;
test/emacs was partially, and test/multipart was fully reverted.
