The JSON format eliminates the complex escaping issues that have
plagued the text search format. This uses the incremental JSON parser
so that, like the text parser, it can output search results
incrementally.
This slows down the parser by about ~4X, but puts us in a good
position to optimize either by improving the JSON parser (evidence
suggests this can reduce the overhead to ~40% over the text format) or
by switching to S-expressions (evidence suggests this will more than
double performance over the text parser). [1]
This also fixes the incremental search parsing test.
This has one minor side-effect on search result formatting.
Previously, the date field was always padded to a fixed width of 12
characters because of how the text parser's regexp was written. The
JSON format doesn't do this. We could pad it out in Emacs before
formatting it, but, since all of the other fields are variable width,
we instead fix notmuch-search-result-format to take the variable-width
field and pad it out. For users who have customized this variable,
we'll mention in the NEWS how to fix this slight format change.
[1] id:"20110720205007.GB21316@mit.edu"
Rather than passing lots of arguments and then further passing those
to `notmuch-search-insert-field', pass a plist containing all of the
search result information. This plist is compatible with the JSON
format search results.
This is a simpler place to do this, since we can avoid any point
motion and hence any save-excursions in
`notmuch-search-process-filter', which in turn lets us put all of the
search-target logic outside of any save-excursions.
`notmuch-search-show-{result,error}' are now responsible for their own
point motion.
`notmuch-search-process-filter' could use some reindentation after
this, but we're about to rewrite it entirely, so we won't bother.
This removes the last bit of direct output from the parsing function.
With the parser now responsible solely for parsing, we can swap it out
for another parser.
Previously, much of the display of search lines was done in the same
function that parsed the CLI's output. Now the parsing function only
parses, and notmuch-search-show-result fully inserts the search result
in the search buffer.
The main change here is to modify argument parsing so as to not force
tag-changes to be a list, and to let notmuch-tag handle prompting the
user when required. doc strings are also updated and cleaned up.
Tagging functions are used in notmuch.el, notmuch-show.el, and
notmuch-message.el. There are enough common functions for tagging
that it makes sense to put them all in their own library.
No code is modified, just moved around.
A previous patch [0] replaced blank subject lines with '[No Subject]'
in search and show mode. Apparently this was needed to circumvent
some bug in the printing code, but there was no need for it search or
show, and it is definitely not desirable, so we undo it here (a revert
is no longer feasible). We should not be modifying strings in the
original message without good reason, or without a clear indication
that we are doing so, neither of which apply in this case. For
further discussion see [0].
[0] id:"1327918561-16245-3-git-send-email-dme@dme.org"
No functional change here. The help message previously referred to
the "delete" tag, but "deleted" is now preferred, so hopefully this
will reduce any potential confusion.
The new message exclude functionality will hide tags that only exist
on excluded messages. However, one might very well want to manually
modify excluded tags. This makes sure tags from excluded messages are
always available in tab completion.
In X, Emacs distinguishes the tab key, which produces a 'tab event;
from C-i, which produces a ?\t event. However, in a terminal, these
are indistinguishable and only produce a ?\t event. In order to
simplify things, Emacs automatically translates from 'tab to ?\t (see
"Function key translations" in M-x describe-bindings), so functions
only need to be bound to ?\t to work in all situations.
Previously, the search tab completion code usedq (kbd "<tab>"), which
produced the event sequence [tab], which only matched the 'tab event
and hence only worked in X. This patch changes it to (kbd "TAB"),
which matches the general ?\t event and works in all situations.
This is a small change to make notmuch.el ignore excluded matches. In
the future it could do something better like add a button for
rerunning the search with the excludes (particularly if nothing
matches with the excludes) or having them invisible and allowing the
visibility to be toggled.
When notmuch-search-line-faces is used to set background color in search
results, the highlight of the current line is not always displayed
correctly. This patch fixes that by increasing the priority property of
the highlight overlay.
Re-work the existing crypto switch toggle to be based on a persistant
buffer-local variable.
To allow this, modify `notmuch-show-refresh-view' to erase and re-draw
in the current buffer rather than killing the current buffer and
creating a new one. (This will also allow more per-buffer behaviour in
future patches.)
Add a binding ('$') to toggle crypto processing of the current buffer
and remove the prefix argument approach that achieves a similar
result.
Add default value to notmuch-search-line-faces to show "unread"
messages in bold, and "flagged" messages in blue, to have some visual
indication of important messages in search results. This should be
helpful for new users.
"unread" tag is quite obvious, and handled specially both in the lib
and emacs ui. "flagged" is synced to maildir F flag in the lib. If one
syncs the maildir to IMAP, this also translates to corresponding IMAP
flag. (This is "starred" in GMail and Android.)
Signed-off-by: Jani Nikula <jani@nikula.org>
This makes the argument names more consistent and clear. The
following functions changed: `notmuch-tag',
`notmuch-search-tag-thread', `notmuch-search-tag-region' and
`notmuch-search-tag-all'.
Since `notmuch-tag' is a non-interactive function and hence is meant
to be invoked programmatically, it should accept zero tags. Also, the
tagging operations (bound to "*", "+", "-") would accept empty input
without an error.
The tag syntax check in `notmuch-tag' function was too strict and did
not allow nmbug tags with "::". Since the check is done for all
tagging operations in Emacs UI, this basically means that no nmbug
tags can be changed. The patch relaxes the tag syntax check to allow
any tag names that do not include whitespace characters.
Some tag-related operations accept a single tag without prefix
(`notmuch-select-tag-with-completion'), others accept multiple tags
prefixed with '+' or '-' (`notmuch-read-tag-changes'). Before the
change, both functions used a single default minibuffer history. This
is inconvenient because you have to skip options with incompatible
format when going through the history. The patch adds separate
history lists for the two functions. Note that functions that accept
the same input format (e.g. "+", "-", "*") share the history list as
before.
Before the change, "+" and "-" tagging operations in notmuch-search
view accepted only a single tag. The patch makes them use the
recently added `notmuch-read-tag-changes' function (renamed
`notmuch-select-tags-with-completion'), which allows to enter multiple
tags with "+" and "-" prefixes. So after the change, "+" and "-"
bindings in notmuch-search view allow to both add and remove multiple
tags. The only difference between "+" and "-" is the minibuffer
initial input ("+" and "-" respectively).
Before the change, tag format validation was done in
`notmuch-search-operate-all' function only. The patch moves it down
to `notmuch-tag', so that all users of that function get input
validation.
There are two ways to do search in Emacs UI: search widget in
notmuch-hello buffer and `notmuch-search' function bound to "s".
Before the change, these search mechanisms used different history
lists. The patch makes notmuch-hello search use the same history list
as `notmuch-search' function.
To allow for expansion whilst keeping everything tidy and organized,
move all defcustom/defface variables to the following subgroups,
defined in notmuch-lib.el:
- Hello
- Search
- Show
- Send
- Crypto
- Hooks
- External Commands
- Appearance
As an added benefit, defcustom keyword args are now consistently
ordered as they appear @ defcustom's docstring (OCD much?).
Proper defgroup docstrings and various other improvements
by courtesy of Austin Clements.
As suggested by j4ni in #notmuch, rename
`notmuch-jump-to-recent-buffer' as `notmuch-cycle-notmuch-buffers' and
have it behave accordingly.
Consider `message-mode' buffers to be of interest.
Provide reply to sender counterparts to the search and show reply
functions. Add key binding 'R' to reply to sender, while keeping 'r' as
reply to all, both in search and show views.
Signed-off-by: Jani Nikula <jani@nikula.org>
`notmuch-hello-mode' inherits the keymap for widgets, which confused
`notmuch-substitute-command-keys'. Fix the confusion.
Simplify `notmuch-substitute-command-keys' a little to make it easier
to read.
With the default configuration ('space' moves through the messages
matching the search and back to the results index at the end) it's
unnecessary to signal an error when the last message has been read, as
this is the common case.
Moreover, it's very annoying when `debug-on-error' is t.
Examples in documentation for `notmuch-search-line-faces' had an extra
quote, e.g.:
'(\"unread\" . '(:foreground \"green\"))
Which resulted in values like:
(\"unread\" quote (:foreground \"green\"))
And tons of "Invalid face reference: quote" errors in the messages
buffer.
From a Carl Worth idea: add a function which will select the most
recently used notmuch buffer (search, show or hello). If no recent
buffer is found, run `notmuch'.
It is expected that the user will global bind this command to a key
sequence.
Support nil value for notmuch-poll-script to run "notmuch new" instead of
an external script, and make this the new default. "notmuch new" is run
using the configured notmuch-command.
This allows taking better advantage of the "notmuch new" hooks from emacs
without intermediate scripts.
Signed-off-by: Jani Nikula <jani@nikula.org>
The process-lines function calls the notmuch binary. The location of
the binary may have been customized by the user, so it is better to
use the customized location rather than allowing the process-lines
function to search the user's PATH for the binary.
There's no reason to record undo information for read-only,
programmatically-constructed buffers. The undo list just chews up
memory keeping track of our calls to insert.
Buffer redisplay requires traversing the buffer's invisibility spec
for every part of the display that has an 'invisible text or overlay
property. Previously, the search buffer's invisibility spec list
contained roughly one entry for each search result. As a result,
redisplay took O(NM) time where N is the number of visible lines and M
is the total number of results. On a slow computer, this is enough to
make even buffer motion noticeably slow. Worse, during a search
operation, redisplay is triggered for each search result (even if
there are no visible buffer changes), so search was quadratic
(O(NM^2)) in the number of search results.
This change switches to using a single element buffer invisibility
spec. To un-hide authors, instead of removing an entry from the
invisibility spec, it simply removes the invisibility overlay from
those authors.
I tested using a query with 6633 results on a 9 year old machine.
Before this patch, Emacs took 70 seconds to fill the search buffer;
toward the end of the search, Emacs consumed 10-20x as much CPU as
notmuch; and moving point in the buffer took about a second. With
this patch, the same query takes 40 seconds, Emacs consumes ~3x the
CPU of notmuch by the end, and there's no noticeable lag to moving
point. (There's still some source of non-linearity, because Emacs and
notmuch consume roughly the same amount of CPU early in the search.)
Emacs 23.2 queries by default about killing existing processes. This
is annoying when one wants to interrupt long search with 'q' key.
Disable this behavior for notmuch.
Various typo fixes in documentation within the code that can be made
available to the user, (emacs function help strings, "notmuch help"
output, notmuch man page, etc.).
Signed-off-by: Pieter Praet <pieter@praet.org>
Edited-by: Carl Worth <cworth@cworth.org> Restricted to just
documentation and fixed fix of "comman" to "common" rather than
"command".
A new emacs configuration variable "notmuch-crypto-process-mime"
controls the processing of PGP/MIME signatures and encrypted parts.
When this is set true, notmuch-query will use the notmuch show
--decrypt flag to decrypt encrypted messages and/or calculate the
sigstatus of signed messages. If sigstatus is available, notmuch-show
will place a specially color-coded header at the begining of the
signed message.
Also included is the ability to switch decryption/verification on/off
on the fly, which is bound to M-RET in notmuch-search-mode.
This patch adds hooks that are run before/after messages are tagged
From the emacs interface. In order to implement this and to avoid
having hooks parse all the arguments to the notmuch binary again, I
created a `notmuch-tag' function that other modules should use instead
of running (notmuch-call-notmuch-process "tag" ...) directly.
With the previous commit, unexpected output before or between search results
would be displayed. However, trailing junk from the "notmuch search" output
would still be silently swallowed.
The most common case for an error message from "notmuch search" would be
an invalid command-line, and in that case, there would be no search results
and the trailing error message would get swallowed.
We fix the process sentinel to check for leftover data and add it to the
final buffer. We also add a test case to ensure this works.
This fixes the recently-added emacs-large-search-buffer test. This is
as simple as saving any trailing input and then pre-prepending it on
the next call.
MAny thanks to Thomas Schwinge <thomas@schwinge.name> for tracking
down this problem and contributing a preliminary version of this fix.
Rather than silently swallowing unexpected output, the emacs interface will now
display it. This will allow error messages to actually arrive at the emacs
interface (though not in an especially pretty way). This also allows for easier
investigation of the inadvertent swallowing of search results that span page
boundaries (as demonstrated by the recent added emacs-large-search-buffer test).
The page-boundary bug has been present since a commit from 2009-11-24:
93af7b5745
Many thanks to Thomas Schwinge for tracking that bug down and
contributing the test for it.
Incremental search does not match strings that span a
visible/invisible boundary. This results in failure to correctly
isearch for authors in `notmuch-search' mode if the name of the author
is split between the visible and invisible components of the authors
string. To avoid this, attempt to truncate the visible component of
the authors string on a boundary between authors, such that the
entirety of an author's name is either visible or invisible.
This add a "stash-map" for search-mode, just like in show-mode, and
adds one function, bound to "i" to stash the thread-id of the current
selected thread.
Couldn't think of the correct way to stash other thread info, so I
didn't add any other stash functions for now.
kill-this-buffer appears to be a function intended specifically for
use in the menu bar, and causes problem killing notmuch buffers when
multiple frames have been used. This patch replaces kill-this-buffer
with notmuch-kill-this-buffer, which in turn just simply calls
(kill-buffer (current-buffer)).
Re-write `notmuch-search-color-line', with the following improvements:
- create overlays only if they will be needed,
- merge the properties specified for a tag on top of any matching a
previous tag.
Remove them from non-top-level entry points, (such as the functions to
set notmuch modes and the deprecated notmuch-folder function). And add
one to the notmuch-hello function. Also, add missing documentation
string to notmuch-hello.
Yet another case of "how could this have possibly worked before?!".
I guess we were just getting very lucky with the emacs lisp calling
conventions and what happens with extra arguments, but, ick! Much
better now.
As the emacs compiler warns, the goto-line function is only intended for
interactive use. Instead use the approach recommended in the goto-line
documentation to avoid this.
The GNU Emacs Lisp Reference Manual section D.1 says:
> * Please don't require the cl package of Common Lisp extensions at
> run time. Use of this package is optional, and it is not part of
> the standard Emacs namespace. If your package loads cl at run time,
> that could cause name clashes for users who don't use that package.
>
> However, there is no problem with using the cl package at compile
> time, with (eval-when-compile (require 'cl)). That's sufficient for
> using the macros in the cl package, because the compiler expands
> them before generating the byte-code.
Follow this advice, requiring the following changes where `cl' was
used at runtime:
- replace `rassoc-if' in `notmuch-search-buffer-title' with the `loop'
macro and inline code. At the same time find the longest prefix
which matches the query rather than simply the last,
- replace `union', `intersection' and `set-difference' in
`notmuch-show-add-tag' and `notmuch-show-remove-tag' with local code
to calculate the result of adding and removing a list of tags from
another list of tags.
That is, a subject with a bracketed set of digits (and optionally a
slash), for example "[2010]" would cause the emacs code to misparse
the search results. Fix this by tweaking the regular expression.
Rather than discarding authors when truncated to fit the defined
column width, mark the text beyond the end of the column as invisible
and allow `isearch' to be used over the text so hidden.
This allows us to retain the compact display whilst enabling a user to
find the elided text.
Add face declarations for the date, count, matching author and subject
columns in search mode and apply those faces when building the search
mode display.
Approved-by: Jameson Rollins <jrollins@finestructure.net>
In search mode some messages don't match the search criteria. Show
their authors names with a different face - generally darker than
those that do match.
Add a (require 'notmuch-message) to notmuch.el. This is for functions that
specifically target message mode (and, in the future, notmuch-message
mode).
The complete-string matching of commit
f2ebe3ac44
defeats the substitution of partial search
strings when the user manually types a
long search string that just happens to
partially match a saved search.
For example, typing "tag:inbox and not tag:foo"
should result in "[inbox] and not tag:foo" but
this has been broken since that commit.
As a compromise between this feature and what the
commit was trying to achieve, we now reverse the
saved-searches list before looking for a match.
This happens to work for me, but won't necessarily
work in general.
What we really want is the longest match, but rassoc-if
just gives us the first match. All of this is just about
creating slightly nice search-buffer names. So if anyone
really cares about making the names *even* nicer, then
they could improve this further.
I happen to have a lot of saved searches that are variants of the
tag:inbox search, (such as "tag:inbox and tag:notmuch"). The logic for
these was always matching inbox first, resulting in "[ inbox ] and
tag:notmuch" rather than "notmuch" as desired.
Anchor the regular expression on both ends to make it look harder for
the better match.
We are asserting that the new notmuch-hello implementation, (available
by just calling `notmuch') is just as easy to use as the old
notmuch-folder. So let's remove what's now a largely redundant
implementation.
To make this transition easier, we are still supporting the
notmuch-folders variable name, and we still provide `notmuch-folder'
as an alias which can be invoked to get the new notmuch-hello
functionality.
We use this function to abstract away the common 3-step process for
looking for a value for the saved-searches variable:
1. Look at the notmuch-saved-searches variable itself
2. Look at the notmuch-folders vaiable
3. Use a default value
We were already using this logic (open-coded) in notmuch-hello, but
notmuch.el was accessing notmuch-folders directly for the clever name
selection of search buffers.
Require notmuch-maildir-fcc and also install it.
Rename all jkr/* functions to notmuch-maildir-fcc-*
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Reviewed-by: Carl Worth <cworth@cworth.org>
This variable was moved from notmuch.el to notmuch-lib.el some time
ago, but the declaration in notmuch.el was left around. Clean that up.
Reviewed-by: Carl Worth <cworth@cworth.org>
The notmuch-hello functionality is now sufficiently useful that we
want to make it the default view of notmuch for new users. This also
effectively hides the "hello" name from the user, so we'll be free to
change that in the implementation if necessary.
This change also shuffles the requires between notmuch.el and
notmuch-hello.el. This fixes things so that our documented (require
'notmuch) is sufficient for getting the notmuch-hello functionality.
Finally, the shuffling caused the notmuch-search-oldest-first variable
from one file to the other. While doing that, give this variable the
defcustom treatment for easier customization.
Define a new `mail-user-agent' (`notmuch-user-agent') and use it by
default. Re-arrange various routines that send mail to use this
(compose, reply, forward). Insert a `User-Agent:' header by default.
This is the real commit for this functionality this time. The
previous attempt to merge this code:
commit 57926bc7b0
was botched (by Carl Worth, not David) to include only the Makefile
change. So the build was broken until this commit that actually adds
the new file.
Fix missing argumen in declaration of notmuch-search function and add
a definition of notmuch-search-continuation to avoid warning about
assignment to a free variable.
This is based on the prototype that Carl Worth described in the TODO
file. It provides a search bar as well as support for recent searches,
saved searches, and a list of all tags in the database (as well as the
number of messages with each tag).
The width of the authors field in search output was previously
specified in two places:
- `notmuch-search-authors-width': the limit beyond which the authors
names are truncated,
- `notmuch-search-result-format': the layout of the search results.
Changing the configuration of one of these may have required the user
to know about and adapt the other accordingly. This led to confusion.
Instead, remove `notmuch-search-authors-width' and perform truncation
based on the relevant field in `notmuch-search-result-format'.
Approved-By: Jameson Rollins <jrollins@finestructure.net>
With defcustom the user can easily find this variable (and its
documentation) within "M-x customize-group" "notmuch" (though finding
*that* is still tricky).
The new name of notmuch-poll-script is also easier to remember, (for
me at least).
Emacs scoping rules strongly encourage us to have fully-namespaced
function names. A prefix like "notmuch-search" is a pretty ugly
namespace name, but it's what we have for now.
The new functions first check if an external poll script has been defined in
the variable 'notmuch-external-refresh-script and if yes, runs that script
before executing the existing refresh function (which is bound to '=')
This can be used to have 'G' mimic the mutt behavior of polling an external
mail server - or if the mail polling is already automatic, it can trigger
the call to notmuch new and any necessary automatic tagging of new email.
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
As the user has already defined aliases for certain searches in
notmuch-folders, search buffer names that use these aliases will
be easier to identify.
This patch helps in customizing search result display similar to
mutt's index_format. The customization is done by defining an alist as
below:
(setq notmuch-search-result-format '(("date" . "%s ")
("authors" . "%-40s ")
("subject" . "%s ")))
The supported keywords are date, count, authors, subject and tags.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@gmail.com>
Signed-off-by: David Edmondson <dme@dme.org>
