Fix notmuch-describe-key crashing for the following two cases
1. format-kbd-macro cannot deal with keys like [(32 . 126)], switch to
use key-description instead.
2. if a function in the current keymap is not bounded, it will crash
the whole process. We check if it is bounded and silently skip it to
avoid crashing.
On some platforms (e.g. macOS), it is necessary to add a real sentinel
process for the error buffer used by `notmuch-start-notmuch' rather
than a no-op sentinel.
We also expand tab completion for it, update the emacs bindings, and
update T350, T357, and T450 to match.
Make use of the bool-to-keyword backward-compatibility feature.
make-process is a new function introduced in Emacs 25, which provides
greater control over process creation. Crucially, it allows
separately redirecting stderr directly to a buffer, which allows us to
avoid needing to use the shell to redirect to a temporary file in
order to correctly distinguish stdout and stderr.
* notmuch-lib.el: Use make-process when it is available; fall back to
the previous method when not.
Load subprocess error output to a string in the callers, and propagate
the error messages as a string parameter instead of a path to file
names.
Required to be able to avoid using temporary files for subprocess
error output.
* notmuch-lib.el: Update notmuch-check-async-exit-status,
notmuch-check-exit-status: accept an err parameter instead of
err-file; shift the responsibility of loading error messages from
files up the call stack.
We now have several customizable options for address completion. There
is a customize group notmuch-address but it only contains one of these
options. Add all the others, and make it part of the notmuch customize
group.
Some of the recent changes to the emacs code have used functions
introduced in emacs 24. The functions used are read-char-choice and
setq-local. This changeset adds a file notmuch-compat.el which
contains compatibility functions so that it should work on emacs
23.
Note, since these functions are taken almost unchanged from the emacs
source they are copyright the Free Software Foundation, and the header
in the file reflects that.
We can't use c-u = as some of the notmuch-show refresh function
already uses that. This is a global keybinding for a relatively
infrequent function but M-= seems unlikely to cause conflicts.
notmuch-refresh-all-buffers calls each buffer's major mode specific
refresh function using the generic notmuch-refresh-this-buffer function.
Since the earlier changesets have stopped the refresh functions from
forcing the buffers to be redisplayed this can refresh buffers that
are not currently displayed without disturbing the user. This is very
useful for silent async background updating the emacs display when new
mail is fetched.
Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
This function prints diagnostic information in the event of an
error. However, one of the callers has an optional :stdin-string
keyword argument. This causes the error printing routine to error
itself.
Rather than reach notmuch-check-exit-status about the possible keyword
arguments (currently only one but could be more in the future) this
commit just tells notmuch-check-exit-status how to print non-string arguments.
Many of the external links found in the notmuch source can be resolved
using https instead of http. This changeset addresses as many as i
could find, without touching the e-mail corpus or expected outputs
found in tests.
`notmuch--get-bodypart-raw' previously assumed that all non-binary MIME
parts could be successfully read by assuming that they were UTF-8
encoded. This was demonstrated to be wrong, specifically when a part was
marked as ISO8859-1 and included accented characters (which were
incorrectly rendered as a result).
Rather than assuming UTF-8, attempt to use the part's declared charset
when reading it, falling back to US-ASCII if the declared charset is
unknown, unsupported or invalid.
Currently the preference for which sub-part of a multipart/alternative
part is shown is global. Allow to the user to override the settings on a
per-message basis by providing the ability to call a function that has
access to the message to return the discouraged type list.
The original approach is retained as the default.
`notmuch-get-bodypart-text' assumed that it is always possible to
acquire text/* parts via the sexp output format. This is not true if the
part in question has a content type of application/octet-stream but is
being interpreted as text/* based on the extension of the part filename.
Rework `notmuch-get-bodypart-text' to use the raw output format to
address this and make the implementation common with that of
`notmuch-get-bodypart-binary'.
Previously poll called from emacs would fail silently. This makes it
return a useful error message.
In the non-deprecated case of notmuch new and appropriate hooks, it
uses notmuch-call-notmuch-process which gives an error and
additionally puts the stdout/stderr etc in the *Notmuch errors*
buffer.
In the deprecated case of a custom poll script it only returns an
error message.
Commit based on a bug report, and a potential fix, by Ketil Malde.
Currently, notmuch has an address completion mechanism that requires
external command to provide completion candidates. This commit adds a
completion mechanism inspired by https://github.com/tjim/nevermore,
which is implemented in Emacs lisp only.
The preexisting address completion mechanism, activated by pressing
TAB on To/Cc lines, is extended to use the new mechanism when
notmuch-address-command to 'internal, which is the new default.
The core of the new mechanism is the function notmuch-address-harvest,
which collects the completion candidates from the notmuch database and
stores them in notmuch-address-completions variable. The address
harvesting can run either synchronously (same as with the previous
mechanism) or asynchronously. When the user presses TAB for the first
time, synchronous harvesting limited to user entered text is performed.
If the entered text is reasonably long, this operation is relatively
fast. Then, asynchronous harvesting over the full database is triggered.
This operation may take long time (minutes on rotating disk). After it
finishes, no harvesting is normally performed again and subsequent
completion requests use the harvested data cached in memory. Completion
cache is updated after 24 hours.
Note that this commit restores (different) completion functionality for
users when the user used external command named "notmuch-addresses",
i.e. the old default. The result will be that the user will use
the new mechanism instead of this command. I believe that many users may
not even recognize this because the new mechanism works the same as
http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps also
as other commands suggested at
http://notmuchmail.org/emacstips/#address_completion.
[This feature was significantly improved by David Bremner and Mark Walters]
Now that we have `notmuch-emacs-version' defined in notmuch emacs MUA
use that as a part of User-Agent: header to provide more accurate
version information when sending emails.
In case some incomplete installation of notmuch emacs MUA is used and
`notmuch-emacs-version' is defined as "unknown" then fall back to ask
version info from cli (as it used to be before this commit).
Requiring notmuch-version[.elc] and if that is missing setting
"fallback" notmuch-emacs-version (to "unknown") was moved from
notmuch.el to notmuch-lib.el as notmuch-mua.el (which provides
User-Agent: information) require's the latter.
Unibyte strings are meant for representing binary data. In practice,
using unibyte versus multibyte strings affects *almost* nothing. It
does happen to matter if we use the binary data in an image descriptor
(which is, helpfully, not documented anywhere and getting it wrong
results in opaque errors like "Not a PNG image: <giant binary spew
that is, in fact, a PNG image>").
`notmuch-get-bodypart-content' could do two very different things,
depending on conditions: for text/* parts other than text/html, it
would return the part content as a multibyte Lisp string *after*
charset conversion, while for other parts (including text/html), it
would return binary part content without charset conversion.
This commit completes the split of `notmuch-get-bodypart-content' into
two different and explicit APIs: `notmuch-get-bodypart-binary' and
`notmuch-get-bodypart-text'. It updates all callers to use one or the
other depending on what's appropriate.
The new function, `notmuch-get-bodypart-binary', replaces
`notmuch-get-bodypart-internal'. Whereas the old function was really
meant for internal use in `notmuch-get-bodypart-content', it was used
in a few other places. Since the difference between
`notmuch-get-bodypart-content' and `notmuch-get-bodypart-internal' was
unclear, these other uses were always confusing and potentially
inconsistent. The new call clearly requests the part as undecoded
binary.
This is step 1 of 2 in separating `notmuch-get-bodypart-content' into
two APIs for retrieving either undecoded binary or decoded text.
This introduces notmuch-jump, which is like a user-friendly,
user-configurable global prefix map for saved searches. This provides
a non-modal and much faster way to access saved searches than
notmuch-hello.
A user configures shortcut keys in notmuch-saved-searches, which are
immediately accessible from anywhere in Notmuch under the "j" key (for
"jump"). When the user hits "j", the minibuffer immediately shows a
helpful table of bindings reminiscent of a completions buffer.
This code is a combination of work from myself (originally,
"notmuch-go"), David Edmondson, and modifications from Mark Walters.
notmuch-poll-script has long since been deprecated in favor of
post-new hooks, but this wasn't obvious from the documentation.
Update the documentation to make this clear. Since
notmuch-poll-script could, to some extend, be used to control the path
of the notmuch binary and that use is now clearly discouraged, promote
notmuch-command to a real defcustom instead of just a variable.
The recent changes for saved searches introduced a bug when notmuch
was loaded after the saved search was defined. This was caused by a
utility function not being defined when the defcustom was loaded.
Fix this by moving some code around: the defcustom is moved into
notmuch-hello (which is a more natural place anyway), and the utility
functions are moved before the defcustom in notmuch-hello. We are
rather constrained as the defcustom for saved searches is the first
variable in the notmuch-hello customize window; to avoid moving this
customize the defcustom needs to be the first defcustom in
notmuch-hello, and the utility functions come before that.
This patch also renames one of the utility functions from
notmuch--saved-searches-to-plist to
notmuch-hello--saved-searches-to-plist (as it is purely local to
notmuch-hello) and corrects a couple of typo/spelling mistakes pointed
out by Tomi.
This adds a sort-order option to saved-searches, stores it in the
saved-search buttons (widgets), and uses the stored value when the
button is pressed.
Storing the sort-order in the widget was suggested by Jani in
id:4c3876274126985683e888641b29cf18142a5eb8.1391771337.git.jani@nikula.org.
Make the defcustom for notmuch-saved-searches use the new plist
format. It should still work with oldstyle saved-searches but will
write the newstyle form.
Previously, the term escaper used a blacklist of characters that
needed escaping. This blacklist turned out to be somewhat incomplete;
for example, it did not contain non-whitespace ASCII control
characters or Unicode "fancy quotes", both of which do require the
term to be escaped.
Switch to a whitelist of characters that are definitely safe to leave
unquoted. This fixes the broken test introduced by the previous
patch.
This combines our two face combining functions into one, easy to use
function with a much shorter name: `notmuch-apply-face'. This
function takes the full set of arguments that
`notmuch-combine-face-text-property' took, but takes them in a more
convenient order and provides smarter defaults that make the function
easy to use on both strings and buffers.
When "notmuch config" is called with the name of an empty or
unconfigured setting, it prints nothing (not even a new line).
Previously, `notmuch-config-get' assumed it would always print a
newline. As a result, when `notmuch-config-get' was called with the
name of an empty of unconfigured setting, it would attempt to
(substring "" 0 -1) to strip the newline, which would fail with a
(args-out-of-range "" 0 -1) exception.
Fix this by only stripping the newline if there actually is one.
Imitated from "Enabling advice" in Emacs lisp manual...
ad-disable-advice by itself only changes the enable flag for a
piece of advice. To make the change take effect in the
advised definition, the advice needs to be activated again.
remaps are a rather unusual keymap consisting of "first key" 'remap
and then "second-key" the remapped-function. Thus we do the
documentation for it separately.
To support key remapping in emacs help we need to know the base keymap
when looking at the remapping. keep track of this while we recurse
down the sub-keymaps in help.
The actual documentation function notmuch-describe-keymap was getting
rather complicated so split out the code for a single key into its own
function notmuch-describe-key.
If the user (or a mode) overrides a keybinding from the common keymap
in one of the modes then both help lines appear in the help screen
even though only one of them is applicable.
Fix this by checking if we already have that key binding. We do this
by constructing an list of (key . docstring) pairs so it is easy to
check if we have already had that binding. Then the actual print help
routine changes these pairs into strings "key \t docstring"
The routines that construct the help page in notmuch-lib rely on
match-data being preserved across some fairly complicated code. This
is currently valid but will not be when this series is finished. Thus
place everything between the string-match and replace-match inside a
save-match-data.
A standard way to unset a key binding is local-unset-key which is equivalent to
(define-key (current-local-map) key nil)
Currently notmuch-help gives an error and fails if a user has done this.
To fix this we only add a help line if the binding is non-nil.
The functions referred to in the documentation for this variable were
replaced by the unified `notmuch-poll-and-refresh-this-buffer' in
21474f0e. Update the documentation to reflect the new function.
(Unfortunately, it's difficult to first demonstrate this problem with
a known-broken test because modern Linux kernels have argument length
limits in the megabytes, which makes Emacs really slow!)
This adds support for passing a string to write to notmuch's stdin to
`notmuch-call-notmuch-process' and `notmuch-call-notmuch-sexp'. Since
this makes both interfaces a little more complicated, it also unifies
their documentation better.
Previously, this was in notmuch.el, but all of the other notmuch call
wrappers were in notmuch-lib.el. Move `notmuch-call-notmuch-process'
to live with its friends. This happens to fix a missing dependency
from notmuch-tag.el, which required notmuch-lib, but not notmuch.
notmuch-help is in notmuch.el not notmuch-lib.el and this is
incovenient for the way pick/tree uses it. I think lib makes more
sense anyway so move it there.
Authors and subjects can contain embedded, encoded control characters
like "\n" and "\t" that mess up display. Transform control characters
into spaces everywhere we display them in search and show.
The only user-visible effect of this should be that "G" now works in
show mode (previously it was unbound for no apparent reason).
This shared keymap gives us one place to put global commands, which
both forces us to think about what commands should be global, and
ensures their bindings can't diverge (like the missing "G" in show).
This unifies the various refresh and poll-and-refresh functions we
have for different modes. Now all modes bind "=" and "G" (except
show, which doesn't bind "G" for some reason) to
`notmuch-refresh-this-buffer' and
`notmuch-poll-and-refresh-this-buffer', respectively.
Notmuch cli provides all structured data previously provided
in json format now in s-expression format, rendering all current
json functionality obsolete.
This is just like `notmuch-call-notmuch-json', but parses S-expression
output. Note that, also like `notmuch-call-notmuch-json', this
doesn't consider trailing data to be an error, which may or may not be
what we want in the long run.
Previously, when the user killed the search buffer before the CLI
search process had completed, we would report the signal sent by Emacs
to kill the CLI to the user as an error. Fix this by only reporting
error exits if the process buffer is still live. We still report
stderr output regardless in case stderr output was relevant to why the
user killed the search buffer (such as a wrapper script being stuck).
This provides a new notmuch-lib utility to start an asynchronous
notmuch process that handles redirecting of stderr and checking of the
exit status. This is similar to `notmuch-call-notmuch-json', but for
asynchronous processes (and it leaves output processing to the
caller).
This is similar to what we already do with the message p-list, though
we apply the part's text property to the whole part's text, in
contrast with the message p-list, which is (rather obscurely) only
applied to the first character.
For such a simple regexp, this was broken in a very complicated way.
The intent was to strip the newline (and potentially other whitespace)
off the end of the error string so there wasn't an extra newline in
the error signal. However, the regexp was deeply dependent on the
active syntax table and the subtleties of $. We didn't notice this
because all notmuch major modes put ?\n in the whitespace class, which
makes this behaved as intended: the "\\s " matches all newlines, but
by matching the newline character, causes the $ *not* to match
*except* where it matched the empty string at the very end of the
string, which was not followed by a newline.
However, if the syntax table declares ?\n to be non-whitespace
(lisp-mode declares it as endcomment, and is likely to be the mode
you're in when testing functions), then this regexp behaves completely
differently, matching trailing spaces at the end of every line within
the string.
The solution is to say what we mean for whitespace *and* to switch
from $ to \', which matches only the end of the string, rather than
the end of each line. Both are necessary or this will strip away
interior newlines, which is not what we want.
The support for emacs version 22 has not worked at least since
September 2011 when I attempted to use it. I expanded the support in
id:yf6ippgtbn0.fsf@taco2.nixu.fi but that was not enough and then I
found it easier to switch to emacs 23.
In case one wants to resurrect emacs 22 (or earlier!) support, pick
the changes from the patch email referenced above.
This has two benefits: unified error handling, and avoiding tramp's
hooking into shell-command-string.
This seems to be a fix for id:874nguxbvq.fsf@tu-dortmund.de
This improves notmuch-combine-face-text-property to support both
applying faces to strings and to support combining the given face
under existing faces, rather than over.
Previously, notmuch-combine-face-text-property assumed that any
existing face properties of the modified text were already in face
list form. This was true as long as it was the only function
manipulating faces (since it always produced a list form face), but if
anything else has manipulated the face, it was more likely to be
either a face name or a face plist. It also didn't correctly handle
face lists as arguments, even though the doc string claimed it did.
This patch fixes notmuch-combine-face-text-property to handle all face
forms correctly by canonicalizing both the argument face and the
existing faces into list form. This also means we can set the face to
a simpler non-list form if there's no existing face.
Emacs has two button type objects: widgets (as used for saved searches
in notmuch-hello) and buttons as used by parts/citations and id links
in notmuch-show. These two behave subtly differently when clicked with
the mouse: widgets select the window clicked before running the
action, buttons do not.
This patch makes all of these behave the same: clicking always selects
the clicked window. It does this by defining a notmuch-button-type
supertype that the other notmuch buttons can inherit from. This
supertype binds the mouse-action to select the window and then
activate the button.
We recently switched to popping up a buffer to report CLI errors, but
this was too intrusive, especially for transient errors and especially
since we made fewer things ignore errors. This patch changes this to
display a basic error message in the minibuffer (using Emacs' usual
error handling path) and, if there are additional details, to log
these to a separate error buffer and reference the error buffer from
the minibuffer message. This is more in line with how Emacs typically
handles errors, but makes the details available to the user without
flooding them with the details.
Given this split, we pare down the basic message and make it more
user-friendly, and also make the verbose message even more detailed
(and more debugging-oriented).
This slightly changes the output of an existing test since we now
report non-zero exits with a pop-up buffer instead of at the end of
the search results.
This checks for non-zero exit status from JSON CLI calls and pops up
an error buffer with stderr and stdout. A consequence of this is that
show and reply now handle errors, rather than ignoring them.
This provides library functions for unified handling of errors from
the notmuch CLI. Follow-up patches will convert some scattered error
handling to use this and add error handling where we currently ignore
errors.
Previously, if the input stream consisted only of an error message,
notmuch-json-begin-compound would signal a (wrong-type-argument
number-or-marker-p nil) error when reaching the end of the error
message. This happened because notmuch-json-scan-to-value would think
that it reached a value and put the parser into the 'value state.
Even after notmuch-json-begin-compound signaled the syntax error, the
parser would remain in this state and when the resynchronization logic
reached the end of the buffer, the parser would fail because the
'value state indicates that characters are available.
This fixes this problem by restoring the parser's previous state if it
encounters a syntax error.
Remove notmuch-folders which has been deprecated since
commit a466921760
Author: Carl Worth <cworth@cworth.org>
Date: Mon Apr 26 22:42:07 2010 -0700
emacs: Rip out all of the notmuch-folder code.
This lets us simplify the notmuch-saved-searches code slightly.
Currently, we only properly escape stashed id queries, but there are
other places where the Emacs UI constructs queries for boolean terms.
Since this escaping function is meant to be used in other places, it
avoids escaping strings that don't need escaping.
Emacs 24's mm-shr HTML email renderer fails to load gnus-art before
referencing gnus-inhibit-images, resulting in a void-variable error
when notmuch attempts to render an HTML email with inline images.
This works around this bug by advising mm-shr to load gnus-art.
mm-shr is the only function outside of gnus-art itself that references
gnus-inhibit-images, so this workaround should be correct. If this
ever changes, hopefully they will have fixed this bug upstream first.
This fixes the "Rendering HTML mail with images" test for Emacs 24.
Add support for customization of the tag changes that are applied when
a message or a thread is archived. Instead of hard-coded removal of
the "inbox" tag, the user can now specify a list of tag changes to
perform.
Previously, the Emacs byte compiler produced the warning
the function `remove-if-not' might not be defined at runtime.
because we only required cl at compile-time (not runtime). This fixes
this warning by requiring cl at runtime, ensuring that the definition
of remove-if-not is available.
Previously, tag-based search result highlighting was done by creating
an overlay over each search result. However, overlays have annoying
front- and rear-advancement semantics that make it difficult to
manipulate text at their boundaries, which the next patch will do.
They also have performance problems (creating an overlay is linear in
the number of overlays between point and the new overlay, making
highlighting a search buffer quadratic in the number of results).
Text properties have neither problem. However, text properties make
it more difficult to apply multiple faces since, unlike with overlays,
a given character can only have a single 'face text property. Hence,
we introduce a utility function that combines faces into any existing
'face text properties.
Using this utility function, it's straightforward to apply all of the
appropriate tag faces in notmuch-search-color-line.
This parser is designed to read streaming JSON whose structure is
known to the caller. Like a typical JSON parsing interface, it
provides a function to read a complete JSON value from the input.
However, it extends this with an additional function that
requires the next value in the input to be a compound value and
descends into it, allowing its elements to be read one at a time
or further descended into. Both functions can return 'retry to
indicate that not enough input is available.
The parser supports efficient partial parsing, so there's no need to
frame the input for correctness or performance.
The bulk of the parsing is still done by Emacs' json.el, so any
improvements or optimizations to that will benefit the incremental
parser as well.
Currently only descending into JSON lists is supported because that's
all we need, but support for descending into JSON objects can be added
in the future.
It was decided in the thread starting at [0] that it is okay for
notmuch to use 'cl runtime functions. However, by default, these
produce byte compiler warnings. This suppresses those using
file-local variables.
[0] id:"m262g864dz.fsf@wal122.wireless-pennnet.upenn.edu"
When mail message is read from emacs, the message structure
obtained may contain parts which have content included
(`text/plain` for example) and other parts where content is not
included (`text/html` for example).
In case content is included, the string is already available in
emacs' internal format and therefore mm-... functions should not
attempt to do further decoding for the data in temp buffer
provided for it.
Currently when reply buffer is created,
notmuch-mm-display-part-inline () is used to provided quoted reply
content. This change makes the mm-... functions called by it use
'gnus-decoded as charset whenever the content is already available.
File .../emacs-23.3/lisp/gnus/mm-uu.el mentions:
"`gnus-decoded' is a fake charset, which means no further decoding."
Quote non-text parts nicely by displaying them with mm-display-part
before calling message-cite-original to quote them. HTML-only emails
can now be quoted correctly. We re-use some code from notmuch-show
(notmuch-show-mm-display-part-inline), which has been moved to
notmuch-lib.el.
Mark the test for this feature as not broken.
Bug 1: Replying from alternate addresses
----------------------------------------
The reply code was inconsistent in its use of symbols and strings for
header names being passed to message.el functions. This caused the
From header to be lookup up incorrectly, causing an additional From
header to be added with the user's primary address instead of the
correct alternate address.
This is fixed by using symbols everywhere, i.e. never using strings
for header names when interacting with message.el.
This change also removes our use of `mail-header`, since we don't use
it anywhere else, and using assq makes it clear how the header lists
are expected to work.
Bug 2: Duplicate headers in emacs 23.2
--------------------------------------
The message.el code in emacs 23.2 assumes that header names will
always be passed as symbols, so our use of strings caused
problems. The symptom was that on 23.2 (and presumably on earlier
versions) the reply message would end up with two of some headers.
Converting everything to symbols also fixes this issue.
Previously, this function took an argument called "message-id", even
though it was a general query, rather than a message ID. This changes
it to "query".
This adds a lib function to turn a message ID into a properly escaped
message ID query and uses this function wherever we previously
hand-constructed ID queries. Wherever this new function is used,
documentation has been clarified to refer to "id: queries" instead of
"message IDs".
This fixes the broken test introduced by the previous patch.
The function notmuch-match-content-type was comparing content types
case sensitively. Fix it so it tests case insensitively.
This fixes a bug where emacs would not include any body when replying
to a message with content-type TEXT/PLAIN.
Use the new JSON reply format to create replies in emacs. Quote HTML
parts nicely by using mm-display-part to turn them into displayable
text, then quoting them with message-cite-original. This is very
useful for users who regularly receive HTML-only email.
Use message-mode's message-cite-original function to create the
quoted body for reply messages. In order to make this act like the
existing notmuch defaults, you will need to set the following in
your emacs configuration:
message-citation-line-format "On %a, %d %b %Y, %f wrote:"
message-citation-line-function 'message-insert-formatted-citation-line
The tests have been updated to reflect the (ugly) emacs default.
