This fixes races in thread-local and global tagging in notmuch-search
(e.g., "+", "-", "a", "*", etc.). Previously, these would modify tags
of new messages that arrived after the search. Now they only operate
on the messages that were in the threads when the search was
performed. This prevents surprises like archiving messages that
arrived in a thread after the search results were shown.
This eliminates `notmuch-search-find-thread-id-region(-search)'
because these functions strongly encouraged racy usage.
This fixes the two broken tests added by the previous patch.
Previously, this was in notmuch.el, but all of the other notmuch call
wrappers were in notmuch-lib.el. Move `notmuch-call-notmuch-process'
to live with its friends. This happens to fix a missing dependency
from notmuch-tag.el, which required notmuch-lib, but not notmuch.
We want to load notmuch-tree when notmuch is loaded, so include it as
a require in notmuch.el. To avoid circular dependency we need to move
one keybinding from notmuch-tree.el to notmuch.el: it makes sense for
it to be defined there anyway.
Since tree is now loaded by default there is no need to print a
message when it is loaded.
notmuch-help is in notmuch.el not notmuch-lib.el and this is
incovenient for the way pick/tree uses it. I think lib makes more
sense anyway so move it there.
Authors and subjects can contain embedded, encoded control characters
like "\n" and "\t" that mess up display. Transform control characters
into spaces everywhere we display them in search and show.
This is similar to the previous commit, but applies to search.
Search is somewhat more complicated because its tagging operations can
also apply to a region. Hence, this lifts interactive prompting into
a helper function. This also takes advantage of the new ability to
provide a prompt to distinguish tagging a single thread from tagging a
region of threads.
The calling convention for `notmuch-tag' changed in commit 97aa3c06 to
take a list of tag changes instead of a &rest argument, but the call
from `notmuch-search-tag-all' still passed a &rest argument. This
happened to work for interactive calls because tag-changes would be
nil, so the `apply' call would pass only the query string to
`notmuch-tag' and simply omit the &optional tag-changes argument.
Currently notmuch-show looks at the prefix-arg directly via
current-prefix-arg. This changes it to use the interactive
specification.
One test (for elide-toggle functionality) set the prefix arg
directly. Update this test to set the new argument directly.
This improves the function documentation for many interactive
commands, either by improving their documentation string where the
improvement also makes sense for programmatic use or by adding a
'notmuch-doc property where it doesn't.
For nearly all commands that support a prefix argument, this adds a
'notmuch-prefix-doc property to document their prefixed behavior This
omits prefix documentation for a few commands where I thought the
prefixed behavior was too obscure (or too complex to fit in one line).
Traditionally, function documentation strings are intended primarily
for programmers, rather than users. They're written from the
perspective of calling the function, not interactively invoking it.
They're only ever displayed along with the function prototype (and
often refer to argument names). And built-in help commands like
`describe-bindings' show the name of the command, not its
documentation.
The notmuch help system is like `describe-bindings', but tries to be
more user-friendly by displaying documentation strings, rather than
Elisp command names. For most commands, this is fine, but for some
the "programmer description" is inappropriate for interactive use.
This is particularly noticeable for commands that take an optional
prefix argument.
This patch adds support for two symbol properties: notmuch-doc and
notmuch-prefix-doc, which let a command override its interactive
documentation and provide separate documentation for its prefixed
invocation. If notmuch-prefix-doc is present, we add an extra line to
the help giving the prefixed key sequence along with the documentation
for the prefixed command.
In the recent changes for search order handling the default-value of
notmuch-search-oldest-first was used. However, default-value needs a
symbol so the symbol-name needs to be quoted.
This missing quote was causing strange sort-orders in some cases.
The only user-visible effect of this should be that "G" now works in
show mode (previously it was unbound for no apparent reason).
This shared keymap gives us one place to put global commands, which
both forces us to think about what commands should be global, and
ensures their bindings can't diverge (like the missing "G" in show).
This converts notmuch-help to use map-keymap for all keymap traversal.
This generally cleans up and simplifies construction of keymap
documentation, and also makes notmuch-help support anything that can
be in a keymap, including more esoteric stuff like multiple
inheritance.
This unifies the various refresh and poll-and-refresh functions we
have for different modes. Now all modes bind "=" and "G" (except
show, which doesn't bind "G" for some reason) to
`notmuch-refresh-this-buffer' and
`notmuch-poll-and-refresh-this-buffer', respectively.
Since notmuch-hello doesn't need this any more, we can remove this
hack. This also eliminates `notmuch-search-quit', so now all modes
bind "q" to `notmuch-kill-this-buffer'.
Previously, if `notmuch-search' was called interactively (bound to "s"
in search and show, but not hello), it would always use newest-first.
However, `notmuch-hello-search' (bound to "s" in hello) and
`notmuch-hello-widget-search` would call it with the user-configured
sort order. This inconsistency seems unintentional, so change
`notmuch-search' to use the user-configured sort order when called
interactively.
Occasionally, when the user killed the search buffer when the CLI
process was still running, Emacs would run the
notmuch-start-notmuch-sentinel sentinel twice. The first call would
process and delete the error output file and the second would fail
with an "Opening input file: no such file or directory, ..." error
when attempting to access the error file.
Emacs isn't supposed to run the sentinel twice. The reason it does is
rather subtle (and probably a bug in Emacs):
1) When the user kills the search buffer, Emacs invokes
kill_buffer_processes, which sends a SIGHUP to notmuch, but doesn't do
anything else. Meanwhile, suppose the notmuch search process has
printed some more output, but Emacs hasn't consumed it yet (this is
critical and is why this error only happens sometimes).
2) Emacs gets a SIGCHLD from the dying notmuch process, which invokes
handle_child_signal, which sets the new process status, but can't do
anything else because it's a signal handler.
3) Emacs returns to its idle loop, which calls status_notify, which
sees that the notmuch process has a new status. This is where things
get interesting.
3.1) Emacs guarantees that it will run process filters on any
unconsumed output before running the process sentinel, so
status_notify calls read_process_output, which consumes the final
output and calls notmuch-search-process-filter.
3.1.1) notmuch-search-process-filter checks if the search buffer is
still alive and, since it's not, it calls delete-process.
3.1.1.1) delete-process correctly sees that the process is already
dead and doesn't try to send another signal, *but* it still modifies
the status to "killed". To deal with the new status, it calls
status_notify. Dun dun dun. We've seen this function before.
3.1.1.1.1) The *recursive* status_notify invocation sees that the
process has a new status and doesn't have any more output to consume,
so it invokes our sentinel and returns.
3.2) The outer status_notify call (which we're still in) is now done
flushing pending process output, so it *also* invokes our sentinel.
This patch addresses this problem at step 3.1.1, where the filter
calls delete-process, since this is a strange and redundant thing to
do anyway.
Several function docstrings refer to behaviour in docstrings that is
really controlled by notmuch-archive-tags. Add cross references, and
replace hardcoding.
I found several places where a setq is immediately followed by a let
or a let*. This seems to be the pessimal combination, with the
implicit scope of the setq combined with the extra indentation of the let.
I combined these cases into a single let* which I think is easier to read.
I can't see any benefit to the funcall, and it looks like the result
of cut-and-paste from some code that actually used a variable for the
function to call.
In addition to being the Right Thing to do, this noticeably improves
the time taken to display the first page of search results, since it's
roughly an order of magnitude faster than the JSON parser.
Interestingly, it does *not* significantly improve the time to
completely fill a large search buffer because for large search
buffers, the cost of creating author invisibility overlays and
inserting text (which slows down with more overlays) dominates.
However, the time required to display the first page of results is
generally more important to the user experience.
Previously, search started the async notmuch process directly. Now,
it uses `notmuch-start-notmuch'. This simplifies the process sentinel
a bit and means that we no longer have to worry about errors
interleaved with the JSON output.
We also update the tests of Emacs error handling, since the error
output is now separated from the search results buffer.
Apparently Emacs provides a function to stringify errors properly.
Use this in the search sentinel where we have to do our own error
messaging, rather than assuming the first error argument will be the
descriptive string.
This patch extracts the rendering of tags in notmuch-show to
the notmuch-tag file.
This file introduces a `notmuch-tag-formats' variable that associates
each tag to a particular format. This variable can be customized
thanks to the work of Austin Clements. For example,
'(("unread" (propertize tag 'face '(:foreground "red")))
("flagged" (notmuch-tag-format-image tag "star.svg")))
associates a red foreground to the "unread" tag and a star picture to
the "flagged" tag.
Signed-off-by: Damien Cassou <damien.cassou@gmail.com>
We recently switched to popping up a buffer to report CLI errors, but
this was too intrusive, especially for transient errors and especially
since we made fewer things ignore errors. This patch changes this to
display a basic error message in the minibuffer (using Emacs' usual
error handling path) and, if there are additional details, to log
these to a separate error buffer and reference the error buffer from
the minibuffer message. This is more in line with how Emacs typically
handles errors, but makes the details available to the user without
flooding them with the details.
Given this split, we pare down the basic message and make it more
user-friendly, and also make the verbose message even more detailed
(and more debugging-oriented).
This slightly changes the output of an existing test since we now
report non-zero exits with a pop-up buffer instead of at the end of
the search results.
This patch just renames the internal variables for the JSON parser now
it is no longer specific to search mode. It also fixes up the white
space after the previous patch. There should be no functional changes.
This patch splits out the incremental json parser into its own
function.
It moves the main logic of the parser to happen inside the parse
buffer rather than inside the results buffer, but makes sure all
results and all errors are displayed in the results buffer.
It also changes the local parser variables from being buffer
local to the results buffer to being buffer local to the parse buffer,
and sets them up automatically so the caller does not need to.
Finally to keep the diff small this patch does not fix the whitespace,
nor complete the code movement (these are done in subsequent patches)
but it should contain all the functional changes.
Since archiving a thread can now be a user customized set of tag
changes, make reversing this easier. Allow a prefix argument to
notmuch-search-archive-thread to reverse the archiving, similar to the
unarchiving in notmuch-show-archive-message.
Add support for customization of the tag changes that are applied when
a message or a thread is archived. Instead of hard-coded removal of
the "inbox" tag, the user can now specify a list of tag changes to
perform.
* emacs/notmuch.el (notmuch-search-mode):
`notmuch-search-tag-all' currently uses the current query string
instead of `notmuch-search-find-thread-id-region-search', which
might cause a race condition.
The recent change to use json for notmuch-search.el introduced a bug
in the code for keeping position on refresh. The problem is a
comparison between (plist-get result :thread) and a thread-id returned
by notmuch-search-find-thread-id: the latter is prefixed with
"thread:"
We fix this by adding an option to notmuch-search-find-thread-id to
return the bare thread-id. It appears that notmuch-search-refresh-view
is the only caller of notmuch-search that supplies a thread-id so this
change should be safe (but could theoretically break users .emacs
functions).
In commit 5d0883e the function notmuch-search-next-thread was changed.
In particular it only goes to the next message if there is a next
message. This breaks notmuch-show-archive-thread-then-next. Fix this
by going to the "next" message whenever we are on a current message.
At this point, the only remaining functions that don't support
multi-line search result formats are the thread navigation functions.
This patch fixes that by rewriting them in terms of
notmuch-search-result-{beginning,end}.
This changes the behavior of notmuch-search-previous-thread slightly
so that if point isn't at the beginning of a result, it first moves
point to the beginning of the result.
Previously we ignored any notmuch-search-result-format customizations
for tag formatting because we needed to be able to parse back in the
result line and update the tags in place. We no longer do either of
these things, so we can allow customization of this format.
(Coincidentally, previously we still allowed too much customization of
the tags format, since moving it earlier on the line or removing it
from the line would interfere with the tagging mechanism. There is
now no problem with doing such things.)
Since the result object contains everything that the other text
properties recorded, we can remove the other text properties and
simply look in the plist of the appropriate result object.
This simplifies the traversal of regions of results and eliminates the
need for save-excursions (which tend to get in the way of maintaining
point when we make changes to the buffer). It also fixes some strange
corner cases in the old line-based code where results that bordered
the region but were not included in it could be affected by region
commands. Coincidentally, this also essentially enables multi-line
search result formats; the only remaining non-multi-line-capable
functions are notmuch-search-{next,previous}-thread, which are only
used for interactive navigation.
Now that we keep the full thread result object, we can refresh a
result after any changes by simply deleting and reconstructing the
result line from scratch.
A convenient side-effect of this wholesale replacement is that search
now re-applies notmuch-search-line-faces when tags change.
Previously, tag-based search result highlighting was done by creating
an overlay over each search result. However, overlays have annoying
front- and rear-advancement semantics that make it difficult to
manipulate text at their boundaries, which the next patch will do.
They also have performance problems (creating an overlay is linear in
the number of overlays between point and the new overlay, making
highlighting a search buffer quadratic in the number of results).
Text properties have neither problem. However, text properties make
it more difficult to apply multiple faces since, unlike with overlays,
a given character can only have a single 'face text property. Hence,
we introduce a utility function that combines faces into any existing
'face text properties.
Using this utility function, it's straightforward to apply all of the
appropriate tag faces in notmuch-search-color-line.
The JSON format eliminates the complex escaping issues that have
plagued the text search format. This uses the incremental JSON parser
so that, like the text parser, it can output search results
incrementally.
This slows down the parser by about ~4X, but puts us in a good
position to optimize either by improving the JSON parser (evidence
suggests this can reduce the overhead to ~40% over the text format) or
by switching to S-expressions (evidence suggests this will more than
double performance over the text parser). [1]
This also fixes the incremental search parsing test.
This has one minor side-effect on search result formatting.
Previously, the date field was always padded to a fixed width of 12
characters because of how the text parser's regexp was written. The
JSON format doesn't do this. We could pad it out in Emacs before
formatting it, but, since all of the other fields are variable width,
we instead fix notmuch-search-result-format to take the variable-width
field and pad it out. For users who have customized this variable,
we'll mention in the NEWS how to fix this slight format change.
[1] id:"20110720205007.GB21316@mit.edu"
Rather than passing lots of arguments and then further passing those
to `notmuch-search-insert-field', pass a plist containing all of the
search result information. This plist is compatible with the JSON
format search results.
This is a simpler place to do this, since we can avoid any point
motion and hence any save-excursions in
`notmuch-search-process-filter', which in turn lets us put all of the
search-target logic outside of any save-excursions.
`notmuch-search-show-{result,error}' are now responsible for their own
point motion.
`notmuch-search-process-filter' could use some reindentation after
this, but we're about to rewrite it entirely, so we won't bother.
This removes the last bit of direct output from the parsing function.
With the parser now responsible solely for parsing, we can swap it out
for another parser.
Previously, much of the display of search lines was done in the same
function that parsed the CLI's output. Now the parsing function only
parses, and notmuch-search-show-result fully inserts the search result
in the search buffer.
The main change here is to modify argument parsing so as to not force
tag-changes to be a list, and to let notmuch-tag handle prompting the
user when required. doc strings are also updated and cleaned up.
Tagging functions are used in notmuch.el, notmuch-show.el, and
notmuch-message.el. There are enough common functions for tagging
that it makes sense to put them all in their own library.
No code is modified, just moved around.
A previous patch [0] replaced blank subject lines with '[No Subject]'
in search and show mode. Apparently this was needed to circumvent
some bug in the printing code, but there was no need for it search or
show, and it is definitely not desirable, so we undo it here (a revert
is no longer feasible). We should not be modifying strings in the
original message without good reason, or without a clear indication
that we are doing so, neither of which apply in this case. For
further discussion see [0].
[0] id:"1327918561-16245-3-git-send-email-dme@dme.org"
No functional change here. The help message previously referred to
the "delete" tag, but "deleted" is now preferred, so hopefully this
will reduce any potential confusion.
The new message exclude functionality will hide tags that only exist
on excluded messages. However, one might very well want to manually
modify excluded tags. This makes sure tags from excluded messages are
always available in tab completion.
In X, Emacs distinguishes the tab key, which produces a 'tab event;
from C-i, which produces a ?\t event. However, in a terminal, these
are indistinguishable and only produce a ?\t event. In order to
simplify things, Emacs automatically translates from 'tab to ?\t (see
"Function key translations" in M-x describe-bindings), so functions
only need to be bound to ?\t to work in all situations.
Previously, the search tab completion code usedq (kbd "<tab>"), which
produced the event sequence [tab], which only matched the 'tab event
and hence only worked in X. This patch changes it to (kbd "TAB"),
which matches the general ?\t event and works in all situations.
This is a small change to make notmuch.el ignore excluded matches. In
the future it could do something better like add a button for
rerunning the search with the excludes (particularly if nothing
matches with the excludes) or having them invisible and allowing the
visibility to be toggled.
When notmuch-search-line-faces is used to set background color in search
results, the highlight of the current line is not always displayed
correctly. This patch fixes that by increasing the priority property of
the highlight overlay.
Re-work the existing crypto switch toggle to be based on a persistant
buffer-local variable.
To allow this, modify `notmuch-show-refresh-view' to erase and re-draw
in the current buffer rather than killing the current buffer and
creating a new one. (This will also allow more per-buffer behaviour in
future patches.)
Add a binding ('$') to toggle crypto processing of the current buffer
and remove the prefix argument approach that achieves a similar
result.
Add default value to notmuch-search-line-faces to show "unread"
messages in bold, and "flagged" messages in blue, to have some visual
indication of important messages in search results. This should be
helpful for new users.
"unread" tag is quite obvious, and handled specially both in the lib
and emacs ui. "flagged" is synced to maildir F flag in the lib. If one
syncs the maildir to IMAP, this also translates to corresponding IMAP
flag. (This is "starred" in GMail and Android.)
Signed-off-by: Jani Nikula <jani@nikula.org>
This makes the argument names more consistent and clear. The
following functions changed: `notmuch-tag',
`notmuch-search-tag-thread', `notmuch-search-tag-region' and
`notmuch-search-tag-all'.
Since `notmuch-tag' is a non-interactive function and hence is meant
to be invoked programmatically, it should accept zero tags. Also, the
tagging operations (bound to "*", "+", "-") would accept empty input
without an error.
The tag syntax check in `notmuch-tag' function was too strict and did
not allow nmbug tags with "::". Since the check is done for all
tagging operations in Emacs UI, this basically means that no nmbug
tags can be changed. The patch relaxes the tag syntax check to allow
any tag names that do not include whitespace characters.
Some tag-related operations accept a single tag without prefix
(`notmuch-select-tag-with-completion'), others accept multiple tags
prefixed with '+' or '-' (`notmuch-read-tag-changes'). Before the
change, both functions used a single default minibuffer history. This
is inconvenient because you have to skip options with incompatible
format when going through the history. The patch adds separate
history lists for the two functions. Note that functions that accept
the same input format (e.g. "+", "-", "*") share the history list as
before.
Before the change, "+" and "-" tagging operations in notmuch-search
view accepted only a single tag. The patch makes them use the
recently added `notmuch-read-tag-changes' function (renamed
`notmuch-select-tags-with-completion'), which allows to enter multiple
tags with "+" and "-" prefixes. So after the change, "+" and "-"
bindings in notmuch-search view allow to both add and remove multiple
tags. The only difference between "+" and "-" is the minibuffer
initial input ("+" and "-" respectively).
Before the change, tag format validation was done in
`notmuch-search-operate-all' function only. The patch moves it down
to `notmuch-tag', so that all users of that function get input
validation.
There are two ways to do search in Emacs UI: search widget in
notmuch-hello buffer and `notmuch-search' function bound to "s".
Before the change, these search mechanisms used different history
lists. The patch makes notmuch-hello search use the same history list
as `notmuch-search' function.
To allow for expansion whilst keeping everything tidy and organized,
move all defcustom/defface variables to the following subgroups,
defined in notmuch-lib.el:
- Hello
- Search
- Show
- Send
- Crypto
- Hooks
- External Commands
- Appearance
As an added benefit, defcustom keyword args are now consistently
ordered as they appear @ defcustom's docstring (OCD much?).
Proper defgroup docstrings and various other improvements
by courtesy of Austin Clements.
As suggested by j4ni in #notmuch, rename
`notmuch-jump-to-recent-buffer' as `notmuch-cycle-notmuch-buffers' and
have it behave accordingly.
Consider `message-mode' buffers to be of interest.
Provide reply to sender counterparts to the search and show reply
functions. Add key binding 'R' to reply to sender, while keeping 'r' as
reply to all, both in search and show views.
Signed-off-by: Jani Nikula <jani@nikula.org>
`notmuch-hello-mode' inherits the keymap for widgets, which confused
`notmuch-substitute-command-keys'. Fix the confusion.
Simplify `notmuch-substitute-command-keys' a little to make it easier
to read.
With the default configuration ('space' moves through the messages
matching the search and back to the results index at the end) it's
unnecessary to signal an error when the last message has been read, as
this is the common case.
Moreover, it's very annoying when `debug-on-error' is t.
Examples in documentation for `notmuch-search-line-faces' had an extra
quote, e.g.:
'(\"unread\" . '(:foreground \"green\"))
Which resulted in values like:
(\"unread\" quote (:foreground \"green\"))
And tons of "Invalid face reference: quote" errors in the messages
buffer.
From a Carl Worth idea: add a function which will select the most
recently used notmuch buffer (search, show or hello). If no recent
buffer is found, run `notmuch'.
It is expected that the user will global bind this command to a key
sequence.
Support nil value for notmuch-poll-script to run "notmuch new" instead of
an external script, and make this the new default. "notmuch new" is run
using the configured notmuch-command.
This allows taking better advantage of the "notmuch new" hooks from emacs
without intermediate scripts.
Signed-off-by: Jani Nikula <jani@nikula.org>
The process-lines function calls the notmuch binary. The location of
the binary may have been customized by the user, so it is better to
use the customized location rather than allowing the process-lines
function to search the user's PATH for the binary.
There's no reason to record undo information for read-only,
programmatically-constructed buffers. The undo list just chews up
memory keeping track of our calls to insert.
