Previously, some parts of upgrade didn't report progress and for
others it was possible for the progress meter to restart at 0 part way
through the upgrade because each stage was reported separately.
Fix this by computing the total amount of work that needs to be done
up-front and updating completed work monotonically.
Rather than potentially making multiple passes over the same type of
data in the database, reorganize upgrade around each type of data that
may be upgraded. This eliminates code duplication, will make
multi-version upgrades faster, and will let us improve progress
reporting.
Previously, we had database version information hard-coded in the
upgrade code. Slightly re-organize the upgrade process around the set
of new database features to be enabled by the upgrade.
Previously, the upgrade was organized as two passes -- an upgrade
pass, and a separate cleanup pass -- so the database was always in a
valid state. This change substantially simplifies this code by
performing the upgrade in a transaction and combining both passes in
to one. This 1) eliminates a lot of duplicate code between the
passes, 2) speeds up the upgrade process, 3) makes progress reporting
more accurate, 4) eliminates the potential for stale data if the
upgrade is interrupted during the cleanup pass, and 5) makes it easier
to reason about the safety of the upgrade code.
Previously, our database schema was versioned by a single number.
Each database schema change had to occur "atomically" in Notmuch's
development history: before some commit, Notmuch used version N, after
that commit, it used version N+1. Hence, each new schema version
could introduce only one change, the task of developing a schema
change fell on a single person, and it all had to happen and be
perfect in a single commit series. This made introducing a new schema
version hard. We've seen only two schema changes in the history of
Notmuch.
This commit introduces database schema version 3; hopefully the last
schema version we'll need for a while. With this version, we switch
from a single version number to "features": a set of named,
independent aspects of the database schema.
Features should make backwards compatibility easier. For many things,
it should be easy to support databases both with and without a
feature, which will allow us to make upgrades optional and will enable
"unstable" features that can be developed and tested over time.
Features also make forwards compatibility easier. The features
recorded in a database include "compatibility flags," which can
indicate to an older version of Notmuch when it must support a given
feature to open the database for read or for write. This lets us
replace the old vague "I don't recognize this version, so something
might go wrong, but I promise to try my best" warnings upon opening a
database with an unknown version with precise errors. If a database
is safe to open for read/write despite unknown features, an older
version will know that and issue no message at all. If the database
is not safe to open for read/write because of unknown features, an
older version will know that, too, and can tell the user exactly which
required features it lacks support for.
The version number has always been pretty meaningless to the user and
it's about to become even more meaningless with the introduction of
"features". Hopefully, the database will remain on version 3 for some
time to come; however, the introduction of new features over time in
version 3 will necessitate upgrades within version 3. It would be
confusing if we always tell the user they've been "upgraded to version
3". If the user wants to know what's new, they should read the news.
The completion script depends on bash-completion 1.90 or later, with
_init_completion function. If that's not present, for some reason, the
completion currently fails with an ugly message, messing up user's
command line:
$ notmuch -bash: _init_completion: command not found
It's better to just not complete
According to RFC2822 References and In-Reply-To headers are supposed
to contain one or more Message-IDs, however older RFC822 allowed
almost any content. When both References and In-Reply-To headers ends
with something else that a Message-ID (see e.g. [1]), the thread
structure presented by notmuch is incorrect. The reason is that
notmuch treats this case as if the email contained no "replyto"
information (see _notmuch_database_link_message_to_parents).
This patch changes the parse_references() function to return the last
valid Message-ID encountered rather than NULL resulting from the last
hunk of text not being the Message-ID.
[1] https://lkml.org/lkml/headers/2014/5/19/864
Fix byte compiler warning "Warning: the function `window-body-width'
is not known to be defined." by moving our compatibility wrapper
before its use and simplify the definition to a defalias for the old
name of the function.
Because of limitations in the C type system, we can't a strtok_len
that can work on both const string and non-const strings. The C
library solves this by taking a const char* and returning a char*
in functions like this (e.g., strchr), but that's not const-safe.
Solve it by introducing strtok_len_c, a version of strtok_len for
const strings.
Previously the build system was generating automatic header
dependencies for test sources, but only smtp-dummy was in SRCS, so
only its dependencies were being included. Add all of the test
sources to SRCS so that the root Makefile.local includes their
dependencies.
This should help new users off to a better start with the addition of
more sensible saved searches and default shortcut keys. Most existing
users have probably customized this variable and won't be affected.
This introduces notmuch-jump, which is like a user-friendly,
user-configurable global prefix map for saved searches. This provides
a non-modal and much faster way to access saved searches than
notmuch-hello.
A user configures shortcut keys in notmuch-saved-searches, which are
immediately accessible from anywhere in Notmuch under the "j" key (for
"jump"). When the user hits "j", the minibuffer immediately shows a
helpful table of bindings reminiscent of a completions buffer.
This code is a combination of work from myself (originally,
"notmuch-go"), David Edmondson, and modifications from Mark Walters.
Previously, we invalidated stored message metadata in
_notmuch_message_add_term and _notmuch_message_remove_term, but not in
_notmuch_message_gen_terms. This doesn't currently result in any bugs
because of our limited uses of _notmuch_message_gen_terms, but it may
could cause trouble in the future.
notmuch-poll-script has long since been deprecated in favor of
post-new hooks, but this wasn't obvious from the documentation.
Update the documentation to make this clear. Since
notmuch-poll-script could, to some extend, be used to control the path
of the notmuch binary and that use is now clearly discouraged, promote
notmuch-command to a real defcustom instead of just a variable.
Dump currently only takes the read lock. Xapian can cope with some
changes while maintaining a read snapshot but with more changes it
fails. Currently notmuch just gives a xapian error.
To avoid this we take the write lock when dumping. This prevents other
notmuch processes from modifying the xapian database preventing this
error.
Discussion with Olly on irc indicates that this is currently the best
solution: in xapian trunk there may be better possibilities using
snapshots but they need to make it to a release and propogate out to
users before we can switch approach.
Finally, this breaks one use case: pipelines of the form
notmuch dump | ... | notmuch restore
According to Olly this is already very fragile: it will only work on
small databases. One of the tests relies on this behaviour so fix that
to store the dump rather than use a pipe.
If we don't have an upstream, there is nothing to merge, so nothing is
unmerged. This avoids errors like:
$ nmbug status
error: No upstream configured for branch 'master'
error: No upstream configured for branch 'master'
fatal: ambiguous argument '@{upstream}': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
'git rev-parse @{upstream}' exited with nonzero value
You might not have an upstream if you're only using nmbug locally to
version-control your tags.
These tests deliver all possible (single-root) four-message threads in
all possible orders and check that notmuch successfully links them
into threads. These tests supersede and replace the previous and much
less thorough "T260-thread-order" tests.
There are two variants of the test: one delivers messages that
reference only their immediate parent and the other delivers messages
that reference all of their parents. The latter test is currently
known-broken.
tree overrides notmuch-show-get-prop so that it can use many of the
utility function directly. Now that tree is in mainline the version
from tree can be moved to show and the original overridden show
version dropped.
Sometimes we want to catch Git errors and handle them, instead of
dying with an error message. This lower-level version of git() allows
us to get the error status when we want it.
This adds the current query as a "default value" to
notmuch-read-qeury. The default value is available via a down-arrow as
opposed to history which is available from the up arrow.
Note if a user presses return in the minibuffer this value is not
returned.
The implementation is simple but notmuch-read-query could be called
via notmuch-search/notmuch-tree etc from any buffer so it makes sense
to put the decision of how to extract the current query in
notmuch-read-query rather than in each of the callers.
Our repository [1] has a post-update hook that rebuilds the status
page after each push. Since that may happen several times a day, we
might as well show the build time (as well as the date) in the footer.
The trailing 'Z' is the ISO 8601 designator for UTC. Now that we're
showing times, it's nice to be explicit about the timezone we're
using.
The rename from date -> datetime gives us backward-compatibility for
folks that *do* only want the date. We keep the old date formatting
to support those folks.
[1]: http://nmbug.tethera.net/git/nmbug-tags.git
Rather than splitting this context into header-only and footer-only
groups, just dump it all in a shared dict. This will make it easier
to eventually split the header/footer templates out of this script
(e.g. if we want to load them from the config file).
It's useful reference information, but anyone who wants it will look
for and find it. We don't need this front-and-center. Follow the
pattern set by our header template with a triple-quoted string.
The gray <hr> styling is less agressive. IE uses 'color' for drawing
the rule, while Gecko and Opera use the border or 'background-color'
[1].
[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=239386
Prefer a docstring to a header comment so we can use it as the
ArgumentParser description (formatted with 'nmbug-status --help').
Script readers still have it near the top of the file. Since it's a
docstring, use PEP 257's summary-line-and-body format [1].
[1]: http://legacy.python.org/dev/peps/pep-0257/#multi-line-docstrings
notmuch-search-archive-thread moves to the next line after tagging. In
the normal case this makes sense, but if the region is active, it tags
the whole region and then it doesn't really. Thus only move to the
next line if region is not active.
The yyyy-mm-dd (actually \d\d\d\d-\d\d-\d\d) for a bit more restrictive
(and self-documentative) than the \w\w\w\w-... that used to be there and
UNRELEASED so that developers can test the latest NEWS converted to mdwn
format before submitting NEWS patches.
Add a function for updating seen messages to the
post-command-hook. This function calls a customizable (by eg
defcustom) function with parameters the start and end of the current
window and that function can decide what to mark read based on that
and the current point.
Since this is in the post-command-hook it should get called after most
user actions (exceptions include user resizing the window) so it
should be possible to make sure the seen status gets updated whether
the user uses notmuch commands like next-message or normal emacs
commands like scroll-up.
It removes all of the old mark read/seen points but introduces a
simple example function that just marks the current message read if it
is open. This function has one small subtlety: it makes sure it
doesn't mark the same message read twice (in the same instance of the
same buffer); otherwise the post-command-hook makes it impossible for
a user to manually mark a message unread.
This fixes the current bugs (imo) that closed messages can be marked
read, and that opening a closed message does not mark it read.
Another advantage of using the post-command-hook any programmatic use
with point passing through a message will not mark it read.
The unread/read changes will use the post-command-hook. test_emacs
does not call the post-command-hook. This adds a notmuch-test-progn
which takes a list of commands as argument and executes them in turn
but runs the post-command-hook after each one.
The caller can batch operations (ie to stop post-command-hook from
being interleaved) by wrapping the batch of operations inside a progn.
We also explicitly run the post-command-hook before getting the output
from a test; this makes sense as this will be a place the user would
be seeing the information.
As noted in devel/STYLE, every private library function should start
with _notmuch. This patch corrects function naming that did not adhere
to this style in lib/notmuch-private.h. In particular, the old function
names that now begin with _notmuch are
notmuch_sha1_of_file
notmuch_sha1_of_string
notmuch_message_file_close
notmuch_message_file_get_header
notmuch_message_file_open
notmuch_message_get_author
notmuch_message_set_author
Signed-off-by: Charles Celerier <cceleri@cs.stanford.edu>
