Notmuch cli provides all structured data previously provided
in json format now in s-expression format, rendering all current
json functionality obsolete.
This switches `notmuch-mua-reply' and `notmuch-query-get-threads' to
the S-exp format. These were the last two uses of the JSON format in
the Emacs frontend.
This is just like `notmuch-call-notmuch-json', but parses S-expression
output. Note that, also like `notmuch-call-notmuch-json', this
doesn't consider trailing data to be an error, which may or may not be
what we want in the long run.
This removes the v command, since we now have much nicer part commands,
and deprecates the underlying notmuch-show-view-all-mime-parts. This
also means that people who try using the old unprefixed 'v' command on
a part button will no longer be greeted by ALL of their parts popping
up.
Previously the query string for piping a message to a command was
"Pipe message to command: " regardless of whether the function was
called with a prefix argument (which pipes all open messages to the
command). This patch modifies the `interactive' command to reflect
this.
This adds the actual code to do the lazy insertion of hidden parts.
We use a memory inefficient but simple method: when we come to insert
the part if it is hidden we just store all of the arguments to the
part insertion function as a button property. This means when we want
to show the part we can just resume where we left off.
One thing is that we can't tell if a lazy part will produce text until
we try to render it so when unhiding a part we check to see if it
rendered; if not we invoke the default part handler (e.g. an external
viewer).
Also, we would like to insert the lazy part at the start of the line
after the part button. But if this line has some text properties
(e.g. the colours for a following message header) then the lazy part
gets these properties. Thus we start at the end of the part button
line, insert a newline, insert the lazy part, and then delete the
extra newline at the end of the part.
Previously, whether a part was hidden or shown was recorded in the
invisibility/visibility of the part overlay. Since we are going to
have lazily rendered parts with no overlay store the hidden/shown
state in the part button itself.
Additionally, in preparation for the invisible part handling move the
actual hiding of the hidden parts to insert-bodypart from
create-part-overlays.
Finally, we will need to know whether a part-insertion has done
anything (it won't if the invisible part cannot be displayed by emacs)
so we slightly rejig the code order in
notmuch-show-toggle-part-invisibility to make it easier for the
function to set an appropriate return value.
Previously each of the part insertion handlers inserted the part
button themselves. Move this up into
notmuch-show-insert-bodypart. Since a small number of the handlers
modify the button (the encryption/signature ones) we need to pass the
header button as an argument into the individual part insertion
handlers. However, the declared-type argument was only used for the
text for the part buttons we can now omit it.
The patch is large but mostly simple. The only things of note are that
we let the text/plain handler applies notmuch-wash to the whole part
including the part button. In particular, notmuch-wash removes leading
blank lines from a text/plain part, but since the button is counted as
part of the part this does not happen with text/plain buttons that
have a button. This is probably a bug in notmuch-wash but changing it
does make several tests fail (that rely on this blank line) so, for
the moment, keep the old behaviour.
Earlier patches have moved the handling of wash fake inline patch
parts to insert-bodypart so we can drop the function
notmuch-show-insert-part-inline-patch-fake-part
Occasionally, when the user killed the search buffer when the CLI
process was still running, Emacs would run the
notmuch-start-notmuch-sentinel sentinel twice. The first call would
process and delete the error output file and the second would fail
with an "Opening input file: no such file or directory, ..." error
when attempting to access the error file.
Emacs isn't supposed to run the sentinel twice. The reason it does is
rather subtle (and probably a bug in Emacs):
1) When the user kills the search buffer, Emacs invokes
kill_buffer_processes, which sends a SIGHUP to notmuch, but doesn't do
anything else. Meanwhile, suppose the notmuch search process has
printed some more output, but Emacs hasn't consumed it yet (this is
critical and is why this error only happens sometimes).
2) Emacs gets a SIGCHLD from the dying notmuch process, which invokes
handle_child_signal, which sets the new process status, but can't do
anything else because it's a signal handler.
3) Emacs returns to its idle loop, which calls status_notify, which
sees that the notmuch process has a new status. This is where things
get interesting.
3.1) Emacs guarantees that it will run process filters on any
unconsumed output before running the process sentinel, so
status_notify calls read_process_output, which consumes the final
output and calls notmuch-search-process-filter.
3.1.1) notmuch-search-process-filter checks if the search buffer is
still alive and, since it's not, it calls delete-process.
3.1.1.1) delete-process correctly sees that the process is already
dead and doesn't try to send another signal, *but* it still modifies
the status to "killed". To deal with the new status, it calls
status_notify. Dun dun dun. We've seen this function before.
3.1.1.1.1) The *recursive* status_notify invocation sees that the
process has a new status and doesn't have any more output to consume,
so it invokes our sentinel and returns.
3.2) The outer status_notify call (which we're still in) is now done
flushing pending process output, so it *also* invokes our sentinel.
This patch addresses this problem at step 3.1.1, where the filter
calls delete-process, since this is a strange and redundant thing to
do anyway.
Previously, when the user killed the search buffer before the CLI
search process had completed, we would report the signal sent by Emacs
to kill the CLI to the user as an error. Fix this by only reporting
error exits if the process buffer is still live. We still report
stderr output regardless in case stderr output was relevant to why the
user killed the search buffer (such as a wrapper script being stuck).
This commit adds an extra button at the end of the search entries that
allows deleting that individual search from the history. A short
confirmation («y» or «n») is made before taking action.
The button to clear the recent searches in notmuch-hello is easy to
press accidentally while moving around the, clearing potentially
useful searches with no way of recovering them.
Previously, we simply called pushnew to add :notmuch-part to the
front-sticky and rear-nonsticky text property lists. This works if
these are nil or lists, but they can also have the value t, meaning
that all properties are front-sticky/rear-nonsticky. In this case,
pushnew will signal an error because t is not a list. We never set
these properties to t ourselves, but since we apply these property
changes over arbitrary renderer output, we have to deal with this
possibility.
Several function docstrings refer to behaviour in docstrings that is
really controlled by notmuch-archive-tags. Add cross references, and
replace hardcoding.
These functions refer to default values of variables, but it seems
less confusing and less likely to get out of date to just allow the
user to follow the help cross-reference links.
I found several places where a setq is immediately followed by a let
or a let*. This seems to be the pessimal combination, with the
implicit scope of the setq combined with the extra indentation of the let.
I combined these cases into a single let* which I think is easier to read.
I can't see any benefit to the funcall, and it looks like the result
of cut-and-paste from some code that actually used a variable for the
function to call.
In addition to being the Right Thing to do, this noticeably improves
the time taken to display the first page of search results, since it's
roughly an order of magnitude faster than the JSON parser.
Interestingly, it does *not* significantly improve the time to
completely fill a large search buffer because for large search
buffers, the cost of creating author invisibility overlays and
inserting text (which slows down with more overlays) dominates.
However, the time required to display the first page of results is
generally more important to the user experience.
This provides the same interface as the streaming JSON parser, but
reads S-expressions incrementally. The only difference is that the
`notmuch-sexp-parse-partial-list' helper does not handle interleaved
error messages (since we now have the ability to separate these out at
the invocation level), so it no longer takes an error function and
does not need to do the horrible resynchronization that the JSON
parser had to.
Some implementation improvements have been made over the JSON parser.
This uses a vector instead of a list for the parser data structure,
since this allows faster access to elements (and modern versions of
Emacs handle storage of small vectors efficiently). Private functions
follow the "prefix--name" convention. And the implementation is much
simpler overall because S-expressions are much easier to parse.
Previously, search started the async notmuch process directly. Now,
it uses `notmuch-start-notmuch'. This simplifies the process sentinel
a bit and means that we no longer have to worry about errors
interleaved with the JSON output.
We also update the tests of Emacs error handling, since the error
output is now separated from the search results buffer.
This provides a new notmuch-lib utility to start an asynchronous
notmuch process that handles redirecting of stderr and checking of the
exit status. This is similar to `notmuch-call-notmuch-json', but for
asynchronous processes (and it leaves output processing to the
caller).
Since the part commands are no longer tied to a button, but can be
applied with point anywhere within a part, bind the part commands
keymap to "." everywhere in the show buffer. This lets you save or
view parts without having to navigate to the part button, and is
particularly useful for parts that have no button.
This removes the un-prefixed MIME part commands from the part button
keymap, but that's okay because those clashed in annoying ways with
show buffer bindings like "s" for search. RET on part buttons is
unaffected, which is the most important part button binding.
This unifies the part button actions and the underlying part action
functions into single interactive command that simply applies to the
part containing point using the just-added part p-list text property
instead of button properties. Since all part actions can be performed
by applying the appropriate mm function to an mm-handle, this patch
abstracts out the creation of mm handles, making the implementations
of the part commands trivial. This also eliminates our special
handling for part save in favor of using the appropriate mm function.
This necessarily modifies the way we handle the default part button
action, but in a way that does not change the meaning of the
notmuch-show-part-button-default-action defcustom.
Since these commands are no longer specific to buttons, this patch
eliminates the extra metadata stored with each button. This also
eliminates one rather special-purpose macro for a collection of
general purpose part handling utilities.
This is similar to what we already do with the message p-list, though
we apply the part's text property to the whole part's text, in
contrast with the message p-list, which is (rather obscurely) only
applied to the first character.
Previously, we lost any text properties applied to part buttons or
wash buttons when they were toggled because `insert' directly copies
the text properties of the string being inserted. Fix this by
capturing the properties applied to the button beforehand and
re-applying them after inserting the new text.
For such a simple regexp, this was broken in a very complicated way.
The intent was to strip the newline (and potentially other whitespace)
off the end of the error string so there wasn't an extra newline in
the error signal. However, the regexp was deeply dependent on the
active syntax table and the subtleties of $. We didn't notice this
because all notmuch major modes put ?\n in the whitespace class, which
makes this behaved as intended: the "\\s " matches all newlines, but
by matching the newline character, causes the $ *not* to match
*except* where it matched the empty string at the very end of the
string, which was not followed by a newline.
However, if the syntax table declares ?\n to be non-whitespace
(lisp-mode declares it as endcomment, and is likely to be the mode
you're in when testing functions), then this regexp behaves completely
differently, matching trailing spaces at the end of every line within
the string.
The solution is to say what we mean for whitespace *and* to switch
from $ to \', which matches only the end of the string, rather than
the end of each line. Both are necessary or this will strip away
interior newlines, which is not what we want.
Previously, notmuch-show-view-part overrode the function binding of
mm-show-part to redirect it to notmuch-show-save-part to get notmuch's
default file name handling in case mm-display-part decided to fall
back to saving the part. In addition to being messy, this depended on
the now-deprecated dynamic binding behavior of flet.
This patch removes the mm-show-part override in favor of passing the
file name in to mm-show-part the way it expects, so we get its default
file name handling. It's not clear why we didn't do this before;
mm-show-part has supported default file names since at least Emacs
23.1.
Previously, we simply byte compiled each Elisp source file
independently. This is actually the wrong thing to do and can lead to
issues with macros and performance issues with substitutions because
1) when the byte compiler encounters a (require 'x) form, it will load
x.elc in preference to x.el, even if x.el is newer, and as a result
may load old macro and substitution definitions and 2) if we update a
macro or substitution definition in one file, we currently won't
re-compile other files that depend on the file containing the
definition.
This patch addresses these problems by computing make dependency rules
from the (require 'x) forms in the Elisp source files, which we inject
into make's dependency database.
Currently mime parts are basically handled based on their mime-type
with the exception of application/octet-stream parts. Deal with these
parts at the top level (notmuch-show-insert-bodypart).
This is needed later in the series as we need to put in a part button
for each part (which means knowing its mime type) while deferring the
actual insertion of the part.
Apparently Emacs provides a function to stringify errors properly.
Use this in the search sentinel where we have to do our own error
messaging, rather than assuming the first error argument will be the
descriptive string.
When compiling notmuch-tag.el there is a compile warning:
notmuch-tag.el:27:1:Warning: cl package required at runtime
Since we have decided to allow runtime use of cl we suppress this
warning by adding a tail comment to the file.
The support for emacs version 22 has not worked at least since
September 2011 when I attempted to use it. I expanded the support in
id:yf6ippgtbn0.fsf@taco2.nixu.fi but that was not enough and then I
found it easier to switch to emacs 23.
In case one wants to resurrect emacs 22 (or earlier!) support, pick
the changes from the patch email referenced above.
Added a customizable variable notmuch-address-selection-function
and the function with the same name to provide a way for user to
change the function called to do address selection.
By default the functionality is exactly the same as it has been so
far; completing-read is called with the same parameters as before.
Setting equivalent lambda expression in place of using
notmuch-address-selection-function function is done as follows:
(setq notmuch-address-selection-function
(lambda (prompt collection initial-input)
(completing-read prompt collection nil nil initial-input)))
For example drop-in replacement with ido-completing-read can be done
easily as an one alternative to the default.
Some (declare-function ...) definitions were drifted away from the
actual (defun ...)'s. To find the drifts and to verify changes
the following command line was used:
$ emacs --batch -L emacs --eval '(check-declare-directory "emacs")'
From a show buffer, notmuch-bbdb/snarf-from imports the sender into
bbdb. notmuch-bbdb/snarf-to imports all recipients. Newly imported
contacts are reported in the minibuffer / Messages buffer.
Both functions use the BBDB parser to recognize email address formats.
This modifies notmuch hello to use the new count --batch
functionality. It should give exactly the same results as before but
under many conditions it should be much faster. In particular it is
much faster for remote use.
The code is a little ugly as it has to do some working out of the
query when asking the query and some when dealing with the result.
However, the code path is exactly the same in both local and remote
use.
This has two benefits: unified error handling, and avoiding tramp's
hooking into shell-command-string.
This seems to be a fix for id:874nguxbvq.fsf@tu-dortmund.de
This patch extracts the rendering of tags in notmuch-show to
the notmuch-tag file.
This file introduces a `notmuch-tag-formats' variable that associates
each tag to a particular format. This variable can be customized
thanks to the work of Austin Clements. For example,
'(("unread" (propertize tag 'face '(:foreground "red")))
("flagged" (notmuch-tag-format-image tag "star.svg")))
associates a red foreground to the "unread" tag and a star picture to
the "flagged" tag.
Signed-off-by: Damien Cassou <damien.cassou@gmail.com>
This improves notmuch-combine-face-text-property to support both
applying faces to strings and to support combining the given face
under existing faces, rather than over.
