This converts notmuch-help to use map-keymap for all keymap traversal.
This generally cleans up and simplifies construction of keymap
documentation, and also makes notmuch-help support anything that can
be in a keymap, including more esoteric stuff like multiple
inheritance.
This unifies the various refresh and poll-and-refresh functions we
have for different modes. Now all modes bind "=" and "G" (except
show, which doesn't bind "G" for some reason) to
`notmuch-refresh-this-buffer' and
`notmuch-poll-and-refresh-this-buffer', respectively.
Since notmuch-hello doesn't need this any more, we can remove this
hack. This also eliminates `notmuch-search-quit', so now all modes
bind "q" to `notmuch-kill-this-buffer'.
Previously, if `notmuch-search' was called interactively (bound to "s"
in search and show, but not hello), it would always use newest-first.
However, `notmuch-hello-search' (bound to "s" in hello) and
`notmuch-hello-widget-search` would call it with the user-configured
sort order. This inconsistency seems unintentional, so change
`notmuch-search' to use the user-configured sort order when called
interactively.
Occasionally, when the user killed the search buffer when the CLI
process was still running, Emacs would run the
notmuch-start-notmuch-sentinel sentinel twice. The first call would
process and delete the error output file and the second would fail
with an "Opening input file: no such file or directory, ..." error
when attempting to access the error file.
Emacs isn't supposed to run the sentinel twice. The reason it does is
rather subtle (and probably a bug in Emacs):
1) When the user kills the search buffer, Emacs invokes
kill_buffer_processes, which sends a SIGHUP to notmuch, but doesn't do
anything else. Meanwhile, suppose the notmuch search process has
printed some more output, but Emacs hasn't consumed it yet (this is
critical and is why this error only happens sometimes).
2) Emacs gets a SIGCHLD from the dying notmuch process, which invokes
handle_child_signal, which sets the new process status, but can't do
anything else because it's a signal handler.
3) Emacs returns to its idle loop, which calls status_notify, which
sees that the notmuch process has a new status. This is where things
get interesting.
3.1) Emacs guarantees that it will run process filters on any
unconsumed output before running the process sentinel, so
status_notify calls read_process_output, which consumes the final
output and calls notmuch-search-process-filter.
3.1.1) notmuch-search-process-filter checks if the search buffer is
still alive and, since it's not, it calls delete-process.
3.1.1.1) delete-process correctly sees that the process is already
dead and doesn't try to send another signal, *but* it still modifies
the status to "killed". To deal with the new status, it calls
status_notify. Dun dun dun. We've seen this function before.
3.1.1.1.1) The *recursive* status_notify invocation sees that the
process has a new status and doesn't have any more output to consume,
so it invokes our sentinel and returns.
3.2) The outer status_notify call (which we're still in) is now done
flushing pending process output, so it *also* invokes our sentinel.
This patch addresses this problem at step 3.1.1, where the filter
calls delete-process, since this is a strange and redundant thing to
do anyway.
Several function docstrings refer to behaviour in docstrings that is
really controlled by notmuch-archive-tags. Add cross references, and
replace hardcoding.
I found several places where a setq is immediately followed by a let
or a let*. This seems to be the pessimal combination, with the
implicit scope of the setq combined with the extra indentation of the let.
I combined these cases into a single let* which I think is easier to read.
I can't see any benefit to the funcall, and it looks like the result
of cut-and-paste from some code that actually used a variable for the
function to call.
In addition to being the Right Thing to do, this noticeably improves
the time taken to display the first page of search results, since it's
roughly an order of magnitude faster than the JSON parser.
Interestingly, it does *not* significantly improve the time to
completely fill a large search buffer because for large search
buffers, the cost of creating author invisibility overlays and
inserting text (which slows down with more overlays) dominates.
However, the time required to display the first page of results is
generally more important to the user experience.
Previously, search started the async notmuch process directly. Now,
it uses `notmuch-start-notmuch'. This simplifies the process sentinel
a bit and means that we no longer have to worry about errors
interleaved with the JSON output.
We also update the tests of Emacs error handling, since the error
output is now separated from the search results buffer.
Apparently Emacs provides a function to stringify errors properly.
Use this in the search sentinel where we have to do our own error
messaging, rather than assuming the first error argument will be the
descriptive string.
This patch extracts the rendering of tags in notmuch-show to
the notmuch-tag file.
This file introduces a `notmuch-tag-formats' variable that associates
each tag to a particular format. This variable can be customized
thanks to the work of Austin Clements. For example,
'(("unread" (propertize tag 'face '(:foreground "red")))
("flagged" (notmuch-tag-format-image tag "star.svg")))
associates a red foreground to the "unread" tag and a star picture to
the "flagged" tag.
Signed-off-by: Damien Cassou <damien.cassou@gmail.com>
We recently switched to popping up a buffer to report CLI errors, but
this was too intrusive, especially for transient errors and especially
since we made fewer things ignore errors. This patch changes this to
display a basic error message in the minibuffer (using Emacs' usual
error handling path) and, if there are additional details, to log
these to a separate error buffer and reference the error buffer from
the minibuffer message. This is more in line with how Emacs typically
handles errors, but makes the details available to the user without
flooding them with the details.
Given this split, we pare down the basic message and make it more
user-friendly, and also make the verbose message even more detailed
(and more debugging-oriented).
This slightly changes the output of an existing test since we now
report non-zero exits with a pop-up buffer instead of at the end of
the search results.
This patch just renames the internal variables for the JSON parser now
it is no longer specific to search mode. It also fixes up the white
space after the previous patch. There should be no functional changes.
This patch splits out the incremental json parser into its own
function.
It moves the main logic of the parser to happen inside the parse
buffer rather than inside the results buffer, but makes sure all
results and all errors are displayed in the results buffer.
It also changes the local parser variables from being buffer
local to the results buffer to being buffer local to the parse buffer,
and sets them up automatically so the caller does not need to.
Finally to keep the diff small this patch does not fix the whitespace,
nor complete the code movement (these are done in subsequent patches)
but it should contain all the functional changes.
Since archiving a thread can now be a user customized set of tag
changes, make reversing this easier. Allow a prefix argument to
notmuch-search-archive-thread to reverse the archiving, similar to the
unarchiving in notmuch-show-archive-message.
Add support for customization of the tag changes that are applied when
a message or a thread is archived. Instead of hard-coded removal of
the "inbox" tag, the user can now specify a list of tag changes to
perform.
* emacs/notmuch.el (notmuch-search-mode):
`notmuch-search-tag-all' currently uses the current query string
instead of `notmuch-search-find-thread-id-region-search', which
might cause a race condition.
The recent change to use json for notmuch-search.el introduced a bug
in the code for keeping position on refresh. The problem is a
comparison between (plist-get result :thread) and a thread-id returned
by notmuch-search-find-thread-id: the latter is prefixed with
"thread:"
We fix this by adding an option to notmuch-search-find-thread-id to
return the bare thread-id. It appears that notmuch-search-refresh-view
is the only caller of notmuch-search that supplies a thread-id so this
change should be safe (but could theoretically break users .emacs
functions).
In commit 5d0883e the function notmuch-search-next-thread was changed.
In particular it only goes to the next message if there is a next
message. This breaks notmuch-show-archive-thread-then-next. Fix this
by going to the "next" message whenever we are on a current message.
At this point, the only remaining functions that don't support
multi-line search result formats are the thread navigation functions.
This patch fixes that by rewriting them in terms of
notmuch-search-result-{beginning,end}.
This changes the behavior of notmuch-search-previous-thread slightly
so that if point isn't at the beginning of a result, it first moves
point to the beginning of the result.
Previously we ignored any notmuch-search-result-format customizations
for tag formatting because we needed to be able to parse back in the
result line and update the tags in place. We no longer do either of
these things, so we can allow customization of this format.
(Coincidentally, previously we still allowed too much customization of
the tags format, since moving it earlier on the line or removing it
from the line would interfere with the tagging mechanism. There is
now no problem with doing such things.)
Since the result object contains everything that the other text
properties recorded, we can remove the other text properties and
simply look in the plist of the appropriate result object.
This simplifies the traversal of regions of results and eliminates the
need for save-excursions (which tend to get in the way of maintaining
point when we make changes to the buffer). It also fixes some strange
corner cases in the old line-based code where results that bordered
the region but were not included in it could be affected by region
commands. Coincidentally, this also essentially enables multi-line
search result formats; the only remaining non-multi-line-capable
functions are notmuch-search-{next,previous}-thread, which are only
used for interactive navigation.
Now that we keep the full thread result object, we can refresh a
result after any changes by simply deleting and reconstructing the
result line from scratch.
A convenient side-effect of this wholesale replacement is that search
now re-applies notmuch-search-line-faces when tags change.
Previously, tag-based search result highlighting was done by creating
an overlay over each search result. However, overlays have annoying
front- and rear-advancement semantics that make it difficult to
manipulate text at their boundaries, which the next patch will do.
They also have performance problems (creating an overlay is linear in
the number of overlays between point and the new overlay, making
highlighting a search buffer quadratic in the number of results).
Text properties have neither problem. However, text properties make
it more difficult to apply multiple faces since, unlike with overlays,
a given character can only have a single 'face text property. Hence,
we introduce a utility function that combines faces into any existing
'face text properties.
Using this utility function, it's straightforward to apply all of the
appropriate tag faces in notmuch-search-color-line.
The JSON format eliminates the complex escaping issues that have
plagued the text search format. This uses the incremental JSON parser
so that, like the text parser, it can output search results
incrementally.
This slows down the parser by about ~4X, but puts us in a good
position to optimize either by improving the JSON parser (evidence
suggests this can reduce the overhead to ~40% over the text format) or
by switching to S-expressions (evidence suggests this will more than
double performance over the text parser). [1]
This also fixes the incremental search parsing test.
This has one minor side-effect on search result formatting.
Previously, the date field was always padded to a fixed width of 12
characters because of how the text parser's regexp was written. The
JSON format doesn't do this. We could pad it out in Emacs before
formatting it, but, since all of the other fields are variable width,
we instead fix notmuch-search-result-format to take the variable-width
field and pad it out. For users who have customized this variable,
we'll mention in the NEWS how to fix this slight format change.
[1] id:"20110720205007.GB21316@mit.edu"
Rather than passing lots of arguments and then further passing those
to `notmuch-search-insert-field', pass a plist containing all of the
search result information. This plist is compatible with the JSON
format search results.
This is a simpler place to do this, since we can avoid any point
motion and hence any save-excursions in
`notmuch-search-process-filter', which in turn lets us put all of the
search-target logic outside of any save-excursions.
`notmuch-search-show-{result,error}' are now responsible for their own
point motion.
`notmuch-search-process-filter' could use some reindentation after
this, but we're about to rewrite it entirely, so we won't bother.
This removes the last bit of direct output from the parsing function.
With the parser now responsible solely for parsing, we can swap it out
for another parser.
Previously, much of the display of search lines was done in the same
function that parsed the CLI's output. Now the parsing function only
parses, and notmuch-search-show-result fully inserts the search result
in the search buffer.
The main change here is to modify argument parsing so as to not force
tag-changes to be a list, and to let notmuch-tag handle prompting the
user when required. doc strings are also updated and cleaned up.
Tagging functions are used in notmuch.el, notmuch-show.el, and
notmuch-message.el. There are enough common functions for tagging
that it makes sense to put them all in their own library.
No code is modified, just moved around.
A previous patch [0] replaced blank subject lines with '[No Subject]'
in search and show mode. Apparently this was needed to circumvent
some bug in the printing code, but there was no need for it search or
show, and it is definitely not desirable, so we undo it here (a revert
is no longer feasible). We should not be modifying strings in the
original message without good reason, or without a clear indication
that we are doing so, neither of which apply in this case. For
further discussion see [0].
[0] id:"1327918561-16245-3-git-send-email-dme@dme.org"
No functional change here. The help message previously referred to
the "delete" tag, but "deleted" is now preferred, so hopefully this
will reduce any potential confusion.
The new message exclude functionality will hide tags that only exist
on excluded messages. However, one might very well want to manually
modify excluded tags. This makes sure tags from excluded messages are
always available in tab completion.
In X, Emacs distinguishes the tab key, which produces a 'tab event;
from C-i, which produces a ?\t event. However, in a terminal, these
are indistinguishable and only produce a ?\t event. In order to
simplify things, Emacs automatically translates from 'tab to ?\t (see
"Function key translations" in M-x describe-bindings), so functions
only need to be bound to ?\t to work in all situations.
Previously, the search tab completion code usedq (kbd "<tab>"), which
produced the event sequence [tab], which only matched the 'tab event
and hence only worked in X. This patch changes it to (kbd "TAB"),
which matches the general ?\t event and works in all situations.
This is a small change to make notmuch.el ignore excluded matches. In
the future it could do something better like add a button for
rerunning the search with the excludes (particularly if nothing
matches with the excludes) or having them invisible and allowing the
visibility to be toggled.
When notmuch-search-line-faces is used to set background color in search
results, the highlight of the current line is not always displayed
correctly. This patch fixes that by increasing the priority property of
the highlight overlay.
Re-work the existing crypto switch toggle to be based on a persistant
buffer-local variable.
To allow this, modify `notmuch-show-refresh-view' to erase and re-draw
in the current buffer rather than killing the current buffer and
creating a new one. (This will also allow more per-buffer behaviour in
future patches.)
Add a binding ('$') to toggle crypto processing of the current buffer
and remove the prefix argument approach that achieves a similar
result.
Add default value to notmuch-search-line-faces to show "unread"
messages in bold, and "flagged" messages in blue, to have some visual
indication of important messages in search results. This should be
helpful for new users.
"unread" tag is quite obvious, and handled specially both in the lib
and emacs ui. "flagged" is synced to maildir F flag in the lib. If one
syncs the maildir to IMAP, this also translates to corresponding IMAP
flag. (This is "starred" in GMail and Android.)
Signed-off-by: Jani Nikula <jani@nikula.org>
This makes the argument names more consistent and clear. The
following functions changed: `notmuch-tag',
`notmuch-search-tag-thread', `notmuch-search-tag-region' and
`notmuch-search-tag-all'.
Since `notmuch-tag' is a non-interactive function and hence is meant
to be invoked programmatically, it should accept zero tags. Also, the
tagging operations (bound to "*", "+", "-") would accept empty input
without an error.
The tag syntax check in `notmuch-tag' function was too strict and did
not allow nmbug tags with "::". Since the check is done for all
tagging operations in Emacs UI, this basically means that no nmbug
tags can be changed. The patch relaxes the tag syntax check to allow
any tag names that do not include whitespace characters.
Some tag-related operations accept a single tag without prefix
(`notmuch-select-tag-with-completion'), others accept multiple tags
prefixed with '+' or '-' (`notmuch-read-tag-changes'). Before the
change, both functions used a single default minibuffer history. This
is inconvenient because you have to skip options with incompatible
format when going through the history. The patch adds separate
history lists for the two functions. Note that functions that accept
the same input format (e.g. "+", "-", "*") share the history list as
before.
Before the change, "+" and "-" tagging operations in notmuch-search
view accepted only a single tag. The patch makes them use the
recently added `notmuch-read-tag-changes' function (renamed
`notmuch-select-tags-with-completion'), which allows to enter multiple
tags with "+" and "-" prefixes. So after the change, "+" and "-"
bindings in notmuch-search view allow to both add and remove multiple
tags. The only difference between "+" and "-" is the minibuffer
initial input ("+" and "-" respectively).
Before the change, tag format validation was done in
`notmuch-search-operate-all' function only. The patch moves it down
to `notmuch-tag', so that all users of that function get input
validation.
There are two ways to do search in Emacs UI: search widget in
notmuch-hello buffer and `notmuch-search' function bound to "s".
Before the change, these search mechanisms used different history
lists. The patch makes notmuch-hello search use the same history list
as `notmuch-search' function.
To allow for expansion whilst keeping everything tidy and organized,
move all defcustom/defface variables to the following subgroups,
defined in notmuch-lib.el:
- Hello
- Search
- Show
- Send
- Crypto
- Hooks
- External Commands
- Appearance
As an added benefit, defcustom keyword args are now consistently
ordered as they appear @ defcustom's docstring (OCD much?).
Proper defgroup docstrings and various other improvements
by courtesy of Austin Clements.
As suggested by j4ni in #notmuch, rename
`notmuch-jump-to-recent-buffer' as `notmuch-cycle-notmuch-buffers' and
have it behave accordingly.
Consider `message-mode' buffers to be of interest.
Provide reply to sender counterparts to the search and show reply
functions. Add key binding 'R' to reply to sender, while keeping 'r' as
reply to all, both in search and show views.
Signed-off-by: Jani Nikula <jani@nikula.org>
`notmuch-hello-mode' inherits the keymap for widgets, which confused
`notmuch-substitute-command-keys'. Fix the confusion.
Simplify `notmuch-substitute-command-keys' a little to make it easier
to read.
With the default configuration ('space' moves through the messages
matching the search and back to the results index at the end) it's
unnecessary to signal an error when the last message has been read, as
this is the common case.
Moreover, it's very annoying when `debug-on-error' is t.
Examples in documentation for `notmuch-search-line-faces' had an extra
quote, e.g.:
'(\"unread\" . '(:foreground \"green\"))
Which resulted in values like:
(\"unread\" quote (:foreground \"green\"))
And tons of "Invalid face reference: quote" errors in the messages
buffer.
From a Carl Worth idea: add a function which will select the most
recently used notmuch buffer (search, show or hello). If no recent
buffer is found, run `notmuch'.
It is expected that the user will global bind this command to a key
sequence.
Support nil value for notmuch-poll-script to run "notmuch new" instead of
an external script, and make this the new default. "notmuch new" is run
using the configured notmuch-command.
This allows taking better advantage of the "notmuch new" hooks from emacs
without intermediate scripts.
Signed-off-by: Jani Nikula <jani@nikula.org>
The process-lines function calls the notmuch binary. The location of
the binary may have been customized by the user, so it is better to
use the customized location rather than allowing the process-lines
function to search the user's PATH for the binary.
There's no reason to record undo information for read-only,
programmatically-constructed buffers. The undo list just chews up
memory keeping track of our calls to insert.
Buffer redisplay requires traversing the buffer's invisibility spec
for every part of the display that has an 'invisible text or overlay
property. Previously, the search buffer's invisibility spec list
contained roughly one entry for each search result. As a result,
redisplay took O(NM) time where N is the number of visible lines and M
is the total number of results. On a slow computer, this is enough to
make even buffer motion noticeably slow. Worse, during a search
operation, redisplay is triggered for each search result (even if
there are no visible buffer changes), so search was quadratic
(O(NM^2)) in the number of search results.
This change switches to using a single element buffer invisibility
spec. To un-hide authors, instead of removing an entry from the
invisibility spec, it simply removes the invisibility overlay from
those authors.
I tested using a query with 6633 results on a 9 year old machine.
Before this patch, Emacs took 70 seconds to fill the search buffer;
toward the end of the search, Emacs consumed 10-20x as much CPU as
notmuch; and moving point in the buffer took about a second. With
this patch, the same query takes 40 seconds, Emacs consumes ~3x the
CPU of notmuch by the end, and there's no noticeable lag to moving
point. (There's still some source of non-linearity, because Emacs and
notmuch consume roughly the same amount of CPU early in the search.)
Emacs 23.2 queries by default about killing existing processes. This
is annoying when one wants to interrupt long search with 'q' key.
Disable this behavior for notmuch.
Various typo fixes in documentation within the code that can be made
available to the user, (emacs function help strings, "notmuch help"
output, notmuch man page, etc.).
Signed-off-by: Pieter Praet <pieter@praet.org>
Edited-by: Carl Worth <cworth@cworth.org> Restricted to just
documentation and fixed fix of "comman" to "common" rather than
"command".
A new emacs configuration variable "notmuch-crypto-process-mime"
controls the processing of PGP/MIME signatures and encrypted parts.
When this is set true, notmuch-query will use the notmuch show
--decrypt flag to decrypt encrypted messages and/or calculate the
sigstatus of signed messages. If sigstatus is available, notmuch-show
will place a specially color-coded header at the begining of the
signed message.
Also included is the ability to switch decryption/verification on/off
on the fly, which is bound to M-RET in notmuch-search-mode.
This patch adds hooks that are run before/after messages are tagged
From the emacs interface. In order to implement this and to avoid
having hooks parse all the arguments to the notmuch binary again, I
created a `notmuch-tag' function that other modules should use instead
of running (notmuch-call-notmuch-process "tag" ...) directly.
With the previous commit, unexpected output before or between search results
would be displayed. However, trailing junk from the "notmuch search" output
would still be silently swallowed.
The most common case for an error message from "notmuch search" would be
an invalid command-line, and in that case, there would be no search results
and the trailing error message would get swallowed.
We fix the process sentinel to check for leftover data and add it to the
final buffer. We also add a test case to ensure this works.
This fixes the recently-added emacs-large-search-buffer test. This is
as simple as saving any trailing input and then pre-prepending it on
the next call.
MAny thanks to Thomas Schwinge <thomas@schwinge.name> for tracking
down this problem and contributing a preliminary version of this fix.
Rather than silently swallowing unexpected output, the emacs interface will now
display it. This will allow error messages to actually arrive at the emacs
interface (though not in an especially pretty way). This also allows for easier
investigation of the inadvertent swallowing of search results that span page
boundaries (as demonstrated by the recent added emacs-large-search-buffer test).
The page-boundary bug has been present since a commit from 2009-11-24:
93af7b5745
Many thanks to Thomas Schwinge for tracking that bug down and
contributing the test for it.
Incremental search does not match strings that span a
visible/invisible boundary. This results in failure to correctly
isearch for authors in `notmuch-search' mode if the name of the author
is split between the visible and invisible components of the authors
string. To avoid this, attempt to truncate the visible component of
the authors string on a boundary between authors, such that the
entirety of an author's name is either visible or invisible.
This add a "stash-map" for search-mode, just like in show-mode, and
adds one function, bound to "i" to stash the thread-id of the current
selected thread.
Couldn't think of the correct way to stash other thread info, so I
didn't add any other stash functions for now.
kill-this-buffer appears to be a function intended specifically for
use in the menu bar, and causes problem killing notmuch buffers when
multiple frames have been used. This patch replaces kill-this-buffer
with notmuch-kill-this-buffer, which in turn just simply calls
(kill-buffer (current-buffer)).
