This patch just renames the internal variables for the JSON parser now
it is no longer specific to search mode. It also fixes up the white
space after the previous patch. There should be no functional changes.
This patch splits out the incremental json parser into its own
function.
It moves the main logic of the parser to happen inside the parse
buffer rather than inside the results buffer, but makes sure all
results and all errors are displayed in the results buffer.
It also changes the local parser variables from being buffer
local to the results buffer to being buffer local to the parse buffer,
and sets them up automatically so the caller does not need to.
Finally to keep the diff small this patch does not fix the whitespace,
nor complete the code movement (these are done in subsequent patches)
but it should contain all the functional changes.
Since archiving a thread can now be a user customized set of tag
changes, make reversing this easier. Allow a prefix argument to
notmuch-search-archive-thread to reverse the archiving, similar to the
unarchiving in notmuch-show-archive-message.
Add support for customization of the tag changes that are applied when
a message or a thread is archived. Instead of hard-coded removal of
the "inbox" tag, the user can now specify a list of tag changes to
perform.
* emacs/notmuch.el (notmuch-search-mode):
`notmuch-search-tag-all' currently uses the current query string
instead of `notmuch-search-find-thread-id-region-search', which
might cause a race condition.
The recent change to use json for notmuch-search.el introduced a bug
in the code for keeping position on refresh. The problem is a
comparison between (plist-get result :thread) and a thread-id returned
by notmuch-search-find-thread-id: the latter is prefixed with
"thread:"
We fix this by adding an option to notmuch-search-find-thread-id to
return the bare thread-id. It appears that notmuch-search-refresh-view
is the only caller of notmuch-search that supplies a thread-id so this
change should be safe (but could theoretically break users .emacs
functions).
In commit 5d0883e the function notmuch-search-next-thread was changed.
In particular it only goes to the next message if there is a next
message. This breaks notmuch-show-archive-thread-then-next. Fix this
by going to the "next" message whenever we are on a current message.
At this point, the only remaining functions that don't support
multi-line search result formats are the thread navigation functions.
This patch fixes that by rewriting them in terms of
notmuch-search-result-{beginning,end}.
This changes the behavior of notmuch-search-previous-thread slightly
so that if point isn't at the beginning of a result, it first moves
point to the beginning of the result.
Previously we ignored any notmuch-search-result-format customizations
for tag formatting because we needed to be able to parse back in the
result line and update the tags in place. We no longer do either of
these things, so we can allow customization of this format.
(Coincidentally, previously we still allowed too much customization of
the tags format, since moving it earlier on the line or removing it
from the line would interfere with the tagging mechanism. There is
now no problem with doing such things.)
Since the result object contains everything that the other text
properties recorded, we can remove the other text properties and
simply look in the plist of the appropriate result object.
This simplifies the traversal of regions of results and eliminates the
need for save-excursions (which tend to get in the way of maintaining
point when we make changes to the buffer). It also fixes some strange
corner cases in the old line-based code where results that bordered
the region but were not included in it could be affected by region
commands. Coincidentally, this also essentially enables multi-line
search result formats; the only remaining non-multi-line-capable
functions are notmuch-search-{next,previous}-thread, which are only
used for interactive navigation.
Now that we keep the full thread result object, we can refresh a
result after any changes by simply deleting and reconstructing the
result line from scratch.
A convenient side-effect of this wholesale replacement is that search
now re-applies notmuch-search-line-faces when tags change.
Previously, tag-based search result highlighting was done by creating
an overlay over each search result. However, overlays have annoying
front- and rear-advancement semantics that make it difficult to
manipulate text at their boundaries, which the next patch will do.
They also have performance problems (creating an overlay is linear in
the number of overlays between point and the new overlay, making
highlighting a search buffer quadratic in the number of results).
Text properties have neither problem. However, text properties make
it more difficult to apply multiple faces since, unlike with overlays,
a given character can only have a single 'face text property. Hence,
we introduce a utility function that combines faces into any existing
'face text properties.
Using this utility function, it's straightforward to apply all of the
appropriate tag faces in notmuch-search-color-line.
The JSON format eliminates the complex escaping issues that have
plagued the text search format. This uses the incremental JSON parser
so that, like the text parser, it can output search results
incrementally.
This slows down the parser by about ~4X, but puts us in a good
position to optimize either by improving the JSON parser (evidence
suggests this can reduce the overhead to ~40% over the text format) or
by switching to S-expressions (evidence suggests this will more than
double performance over the text parser). [1]
This also fixes the incremental search parsing test.
This has one minor side-effect on search result formatting.
Previously, the date field was always padded to a fixed width of 12
characters because of how the text parser's regexp was written. The
JSON format doesn't do this. We could pad it out in Emacs before
formatting it, but, since all of the other fields are variable width,
we instead fix notmuch-search-result-format to take the variable-width
field and pad it out. For users who have customized this variable,
we'll mention in the NEWS how to fix this slight format change.
[1] id:"20110720205007.GB21316@mit.edu"
Rather than passing lots of arguments and then further passing those
to `notmuch-search-insert-field', pass a plist containing all of the
search result information. This plist is compatible with the JSON
format search results.
This is a simpler place to do this, since we can avoid any point
motion and hence any save-excursions in
`notmuch-search-process-filter', which in turn lets us put all of the
search-target logic outside of any save-excursions.
`notmuch-search-show-{result,error}' are now responsible for their own
point motion.
`notmuch-search-process-filter' could use some reindentation after
this, but we're about to rewrite it entirely, so we won't bother.
This removes the last bit of direct output from the parsing function.
With the parser now responsible solely for parsing, we can swap it out
for another parser.
Previously, much of the display of search lines was done in the same
function that parsed the CLI's output. Now the parsing function only
parses, and notmuch-search-show-result fully inserts the search result
in the search buffer.
The main change here is to modify argument parsing so as to not force
tag-changes to be a list, and to let notmuch-tag handle prompting the
user when required. doc strings are also updated and cleaned up.
Tagging functions are used in notmuch.el, notmuch-show.el, and
notmuch-message.el. There are enough common functions for tagging
that it makes sense to put them all in their own library.
No code is modified, just moved around.
A previous patch [0] replaced blank subject lines with '[No Subject]'
in search and show mode. Apparently this was needed to circumvent
some bug in the printing code, but there was no need for it search or
show, and it is definitely not desirable, so we undo it here (a revert
is no longer feasible). We should not be modifying strings in the
original message without good reason, or without a clear indication
that we are doing so, neither of which apply in this case. For
further discussion see [0].
[0] id:"1327918561-16245-3-git-send-email-dme@dme.org"
No functional change here. The help message previously referred to
the "delete" tag, but "deleted" is now preferred, so hopefully this
will reduce any potential confusion.
The new message exclude functionality will hide tags that only exist
on excluded messages. However, one might very well want to manually
modify excluded tags. This makes sure tags from excluded messages are
always available in tab completion.
In X, Emacs distinguishes the tab key, which produces a 'tab event;
from C-i, which produces a ?\t event. However, in a terminal, these
are indistinguishable and only produce a ?\t event. In order to
simplify things, Emacs automatically translates from 'tab to ?\t (see
"Function key translations" in M-x describe-bindings), so functions
only need to be bound to ?\t to work in all situations.
Previously, the search tab completion code usedq (kbd "<tab>"), which
produced the event sequence [tab], which only matched the 'tab event
and hence only worked in X. This patch changes it to (kbd "TAB"),
which matches the general ?\t event and works in all situations.
This is a small change to make notmuch.el ignore excluded matches. In
the future it could do something better like add a button for
rerunning the search with the excludes (particularly if nothing
matches with the excludes) or having them invisible and allowing the
visibility to be toggled.
When notmuch-search-line-faces is used to set background color in search
results, the highlight of the current line is not always displayed
correctly. This patch fixes that by increasing the priority property of
the highlight overlay.
Re-work the existing crypto switch toggle to be based on a persistant
buffer-local variable.
To allow this, modify `notmuch-show-refresh-view' to erase and re-draw
in the current buffer rather than killing the current buffer and
creating a new one. (This will also allow more per-buffer behaviour in
future patches.)
Add a binding ('$') to toggle crypto processing of the current buffer
and remove the prefix argument approach that achieves a similar
result.
Add default value to notmuch-search-line-faces to show "unread"
messages in bold, and "flagged" messages in blue, to have some visual
indication of important messages in search results. This should be
helpful for new users.
"unread" tag is quite obvious, and handled specially both in the lib
and emacs ui. "flagged" is synced to maildir F flag in the lib. If one
syncs the maildir to IMAP, this also translates to corresponding IMAP
flag. (This is "starred" in GMail and Android.)
Signed-off-by: Jani Nikula <jani@nikula.org>
This makes the argument names more consistent and clear. The
following functions changed: `notmuch-tag',
`notmuch-search-tag-thread', `notmuch-search-tag-region' and
`notmuch-search-tag-all'.
Since `notmuch-tag' is a non-interactive function and hence is meant
to be invoked programmatically, it should accept zero tags. Also, the
tagging operations (bound to "*", "+", "-") would accept empty input
without an error.
The tag syntax check in `notmuch-tag' function was too strict and did
not allow nmbug tags with "::". Since the check is done for all
tagging operations in Emacs UI, this basically means that no nmbug
tags can be changed. The patch relaxes the tag syntax check to allow
any tag names that do not include whitespace characters.
Some tag-related operations accept a single tag without prefix
(`notmuch-select-tag-with-completion'), others accept multiple tags
prefixed with '+' or '-' (`notmuch-read-tag-changes'). Before the
change, both functions used a single default minibuffer history. This
is inconvenient because you have to skip options with incompatible
format when going through the history. The patch adds separate
history lists for the two functions. Note that functions that accept
the same input format (e.g. "+", "-", "*") share the history list as
before.
Before the change, "+" and "-" tagging operations in notmuch-search
view accepted only a single tag. The patch makes them use the
recently added `notmuch-read-tag-changes' function (renamed
`notmuch-select-tags-with-completion'), which allows to enter multiple
tags with "+" and "-" prefixes. So after the change, "+" and "-"
bindings in notmuch-search view allow to both add and remove multiple
tags. The only difference between "+" and "-" is the minibuffer
initial input ("+" and "-" respectively).
Before the change, tag format validation was done in
`notmuch-search-operate-all' function only. The patch moves it down
to `notmuch-tag', so that all users of that function get input
validation.
There are two ways to do search in Emacs UI: search widget in
notmuch-hello buffer and `notmuch-search' function bound to "s".
Before the change, these search mechanisms used different history
lists. The patch makes notmuch-hello search use the same history list
as `notmuch-search' function.
To allow for expansion whilst keeping everything tidy and organized,
move all defcustom/defface variables to the following subgroups,
defined in notmuch-lib.el:
- Hello
- Search
- Show
- Send
- Crypto
- Hooks
- External Commands
- Appearance
As an added benefit, defcustom keyword args are now consistently
ordered as they appear @ defcustom's docstring (OCD much?).
Proper defgroup docstrings and various other improvements
by courtesy of Austin Clements.
As suggested by j4ni in #notmuch, rename
`notmuch-jump-to-recent-buffer' as `notmuch-cycle-notmuch-buffers' and
have it behave accordingly.
Consider `message-mode' buffers to be of interest.
Provide reply to sender counterparts to the search and show reply
functions. Add key binding 'R' to reply to sender, while keeping 'r' as
reply to all, both in search and show views.
Signed-off-by: Jani Nikula <jani@nikula.org>
`notmuch-hello-mode' inherits the keymap for widgets, which confused
`notmuch-substitute-command-keys'. Fix the confusion.
Simplify `notmuch-substitute-command-keys' a little to make it easier
to read.
With the default configuration ('space' moves through the messages
matching the search and back to the results index at the end) it's
unnecessary to signal an error when the last message has been read, as
this is the common case.
Moreover, it's very annoying when `debug-on-error' is t.
Examples in documentation for `notmuch-search-line-faces' had an extra
quote, e.g.:
'(\"unread\" . '(:foreground \"green\"))
Which resulted in values like:
(\"unread\" quote (:foreground \"green\"))
And tons of "Invalid face reference: quote" errors in the messages
buffer.
From a Carl Worth idea: add a function which will select the most
recently used notmuch buffer (search, show or hello). If no recent
buffer is found, run `notmuch'.
It is expected that the user will global bind this command to a key
sequence.
Support nil value for notmuch-poll-script to run "notmuch new" instead of
an external script, and make this the new default. "notmuch new" is run
using the configured notmuch-command.
This allows taking better advantage of the "notmuch new" hooks from emacs
without intermediate scripts.
Signed-off-by: Jani Nikula <jani@nikula.org>
The process-lines function calls the notmuch binary. The location of
the binary may have been customized by the user, so it is better to
use the customized location rather than allowing the process-lines
function to search the user's PATH for the binary.
There's no reason to record undo information for read-only,
programmatically-constructed buffers. The undo list just chews up
memory keeping track of our calls to insert.
Buffer redisplay requires traversing the buffer's invisibility spec
for every part of the display that has an 'invisible text or overlay
property. Previously, the search buffer's invisibility spec list
contained roughly one entry for each search result. As a result,
redisplay took O(NM) time where N is the number of visible lines and M
is the total number of results. On a slow computer, this is enough to
make even buffer motion noticeably slow. Worse, during a search
operation, redisplay is triggered for each search result (even if
there are no visible buffer changes), so search was quadratic
(O(NM^2)) in the number of search results.
This change switches to using a single element buffer invisibility
spec. To un-hide authors, instead of removing an entry from the
invisibility spec, it simply removes the invisibility overlay from
those authors.
I tested using a query with 6633 results on a 9 year old machine.
Before this patch, Emacs took 70 seconds to fill the search buffer;
toward the end of the search, Emacs consumed 10-20x as much CPU as
notmuch; and moving point in the buffer took about a second. With
this patch, the same query takes 40 seconds, Emacs consumes ~3x the
CPU of notmuch by the end, and there's no noticeable lag to moving
point. (There's still some source of non-linearity, because Emacs and
notmuch consume roughly the same amount of CPU early in the search.)
Emacs 23.2 queries by default about killing existing processes. This
is annoying when one wants to interrupt long search with 'q' key.
Disable this behavior for notmuch.
Various typo fixes in documentation within the code that can be made
available to the user, (emacs function help strings, "notmuch help"
output, notmuch man page, etc.).
Signed-off-by: Pieter Praet <pieter@praet.org>
Edited-by: Carl Worth <cworth@cworth.org> Restricted to just
documentation and fixed fix of "comman" to "common" rather than
"command".
A new emacs configuration variable "notmuch-crypto-process-mime"
controls the processing of PGP/MIME signatures and encrypted parts.
When this is set true, notmuch-query will use the notmuch show
--decrypt flag to decrypt encrypted messages and/or calculate the
sigstatus of signed messages. If sigstatus is available, notmuch-show
will place a specially color-coded header at the begining of the
signed message.
Also included is the ability to switch decryption/verification on/off
on the fly, which is bound to M-RET in notmuch-search-mode.
This patch adds hooks that are run before/after messages are tagged
From the emacs interface. In order to implement this and to avoid
having hooks parse all the arguments to the notmuch binary again, I
created a `notmuch-tag' function that other modules should use instead
of running (notmuch-call-notmuch-process "tag" ...) directly.
With the previous commit, unexpected output before or between search results
would be displayed. However, trailing junk from the "notmuch search" output
would still be silently swallowed.
The most common case for an error message from "notmuch search" would be
an invalid command-line, and in that case, there would be no search results
and the trailing error message would get swallowed.
We fix the process sentinel to check for leftover data and add it to the
final buffer. We also add a test case to ensure this works.
This fixes the recently-added emacs-large-search-buffer test. This is
as simple as saving any trailing input and then pre-prepending it on
the next call.
MAny thanks to Thomas Schwinge <thomas@schwinge.name> for tracking
down this problem and contributing a preliminary version of this fix.
Rather than silently swallowing unexpected output, the emacs interface will now
display it. This will allow error messages to actually arrive at the emacs
interface (though not in an especially pretty way). This also allows for easier
investigation of the inadvertent swallowing of search results that span page
boundaries (as demonstrated by the recent added emacs-large-search-buffer test).
The page-boundary bug has been present since a commit from 2009-11-24:
93af7b5745
Many thanks to Thomas Schwinge for tracking that bug down and
contributing the test for it.
Incremental search does not match strings that span a
visible/invisible boundary. This results in failure to correctly
isearch for authors in `notmuch-search' mode if the name of the author
is split between the visible and invisible components of the authors
string. To avoid this, attempt to truncate the visible component of
the authors string on a boundary between authors, such that the
entirety of an author's name is either visible or invisible.
This add a "stash-map" for search-mode, just like in show-mode, and
adds one function, bound to "i" to stash the thread-id of the current
selected thread.
Couldn't think of the correct way to stash other thread info, so I
didn't add any other stash functions for now.
kill-this-buffer appears to be a function intended specifically for
use in the menu bar, and causes problem killing notmuch buffers when
multiple frames have been used. This patch replaces kill-this-buffer
with notmuch-kill-this-buffer, which in turn just simply calls
(kill-buffer (current-buffer)).
Re-write `notmuch-search-color-line', with the following improvements:
- create overlays only if they will be needed,
- merge the properties specified for a tag on top of any matching a
previous tag.
Remove them from non-top-level entry points, (such as the functions to
set notmuch modes and the deprecated notmuch-folder function). And add
one to the notmuch-hello function. Also, add missing documentation
string to notmuch-hello.
Yet another case of "how could this have possibly worked before?!".
I guess we were just getting very lucky with the emacs lisp calling
conventions and what happens with extra arguments, but, ick! Much
better now.
As the emacs compiler warns, the goto-line function is only intended for
interactive use. Instead use the approach recommended in the goto-line
documentation to avoid this.
The GNU Emacs Lisp Reference Manual section D.1 says:
> * Please don't require the cl package of Common Lisp extensions at
> run time. Use of this package is optional, and it is not part of
> the standard Emacs namespace. If your package loads cl at run time,
> that could cause name clashes for users who don't use that package.
>
> However, there is no problem with using the cl package at compile
> time, with (eval-when-compile (require 'cl)). That's sufficient for
> using the macros in the cl package, because the compiler expands
> them before generating the byte-code.
Follow this advice, requiring the following changes where `cl' was
used at runtime:
- replace `rassoc-if' in `notmuch-search-buffer-title' with the `loop'
macro and inline code. At the same time find the longest prefix
which matches the query rather than simply the last,
- replace `union', `intersection' and `set-difference' in
`notmuch-show-add-tag' and `notmuch-show-remove-tag' with local code
to calculate the result of adding and removing a list of tags from
another list of tags.
That is, a subject with a bracketed set of digits (and optionally a
slash), for example "[2010]" would cause the emacs code to misparse
the search results. Fix this by tweaking the regular expression.
Rather than discarding authors when truncated to fit the defined
column width, mark the text beyond the end of the column as invisible
and allow `isearch' to be used over the text so hidden.
This allows us to retain the compact display whilst enabling a user to
find the elided text.
Add face declarations for the date, count, matching author and subject
columns in search mode and apply those faces when building the search
mode display.
Approved-by: Jameson Rollins <jrollins@finestructure.net>
In search mode some messages don't match the search criteria. Show
their authors names with a different face - generally darker than
those that do match.
Add a (require 'notmuch-message) to notmuch.el. This is for functions that
specifically target message mode (and, in the future, notmuch-message
mode).
The complete-string matching of commit
f2ebe3ac44
defeats the substitution of partial search
strings when the user manually types a
long search string that just happens to
partially match a saved search.
For example, typing "tag:inbox and not tag:foo"
should result in "[inbox] and not tag:foo" but
this has been broken since that commit.
As a compromise between this feature and what the
commit was trying to achieve, we now reverse the
saved-searches list before looking for a match.
This happens to work for me, but won't necessarily
work in general.
What we really want is the longest match, but rassoc-if
just gives us the first match. All of this is just about
creating slightly nice search-buffer names. So if anyone
really cares about making the names *even* nicer, then
they could improve this further.
I happen to have a lot of saved searches that are variants of the
tag:inbox search, (such as "tag:inbox and tag:notmuch"). The logic for
these was always matching inbox first, resulting in "[ inbox ] and
tag:notmuch" rather than "notmuch" as desired.
Anchor the regular expression on both ends to make it look harder for
the better match.
We are asserting that the new notmuch-hello implementation, (available
by just calling `notmuch') is just as easy to use as the old
notmuch-folder. So let's remove what's now a largely redundant
implementation.
To make this transition easier, we are still supporting the
notmuch-folders variable name, and we still provide `notmuch-folder'
as an alias which can be invoked to get the new notmuch-hello
functionality.
