Commit graph

5565 commits

Author SHA1 Message Date
David Bremner
ae27403641 debian: re-enable atomicity test on arm64
according to

   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=757454

the kernel problem which caused this test to fail should be
fixed now
2014-09-20 09:51:48 +02:00
David Bremner
56c48dc82d debian: bump SONAME
Note that this is one of those cases where an ABI change is not
obvious from the symbols file. Several previously void functions now
have return codes.
2014-09-16 21:08:54 +02:00
David Bremner
f6ce18fae9 python: bump SONAME
This should have happened in commit 6754ad9f9, but oops.

This was not caught by our test suite because it uses an installed
notmuch library of it cannot find the just built one.
2014-09-16 20:50:57 +02:00
Peter Wang
ee3ccccd25 cli: refactor insert
Change insert_message into write_message and move its responsibilities
for indexing the message into the main function, to simplify the control
flow.
2014-09-16 20:18:50 +02:00
Peter Wang
034102303e ruby: handle return status of database close
Throw an exception if notmuch_database_destroy fails.
2014-09-16 20:17:15 +02:00
Peter Wang
d81fc4b42d go: add return status to database close method
Add return status to the Database.Close() method that calls
notmuch_database_destroy.
2014-09-16 20:17:06 +02:00
Peter Wang
ea90d8e043 python: handle return status of database close and destroy
Throw an exception if notmuch_database_close or notmuch_database_destroy
fail.
2014-09-16 20:16:52 +02:00
Peter Wang
6754ad9f9e lib: bump soname
Adding return values to notmuch_database_close and
notmuch_database_destroy may require bumping the soname.
2014-09-16 20:16:31 +02:00
David Bremner
c34d6bad0f test: simplify T360-symbol-hiding, use nm instead of objdump
After yet another variation in objdump output caused this test to fail
(on a Debian port, no less), I decided whatever putative benefit we
get from looking at the object files instead of the library isn't
worth the maintenence headache.

This version uses nm -P. nm -P should be portable, and fixed format.
It purposely doesn't use the -D argument, since that is non-POSIX and
nm on GNU/Linux seems do the right thing without it.

It still won't work out of the box on e.g. Mac OS/X. I think the right
thing to do there is to move some more configuration information into
sh.config.
2014-09-13 08:49:50 +02:00
Gaute Hope
2c9e120e0a notmuch_thread_get_authors: document match grouping with |
as stated in thread.cc:115

/* Construct an authors string from matched_authors_array and
 * authors_array. The string contains matched authors first, then
 * non-matched authors (with the two groups separated by '|'). Within
 * each group, authors are listed in date order. */

this is, however, not reflected in the public API documentation in
notmuch.h:970. This patch a paragraph explaining how | separates the
group of authors of messages matching the query and those of messages
that do not, but are still contained in the thread.
2014-09-13 08:43:35 +02:00
Mark Walters
5c4f6ed99b emacs: jump: sort-order bugfix
default-value needs its argument to be quoted.

Slightly strangely default-value of 't or nil is 't or nil
respectively so the code

(default-value notmuch-search-oldest-first)

just gives the current value of notmuch-search-oldest-first rather
than intended default-value of this variable.

The symptom is that if you are in a search buffer and use notmuch jump
to run a saved search which does not have an explicitly set sort order
then the sort order of the saved-search is inherited from the current
search buffer rather than being the default search order.

Thanks to Jani for finding the bug.
2014-09-07 20:02:16 +02:00
Austin Clements
3d39d346d7 cli: Be more helpful when .notmuch-config does not exist
Previously, if the user ran any subcommand that required a
configuration (e.g., notmuch new) but didn't have a configuration,
notmuch would give the rather un-friendly and un-actionable message

  Error reading configuration file .notmuch-config: No such file or directory

Since this condition is expected for new users, this patch adds
specific handling for the file-not-found case to give a message that
is friendly and actionable.
2014-09-07 20:01:01 +02:00
Tomi Ollila
ef5e66ae8e doc: 'rm -f' potential doxygen temporary output file
Some (older) Doxygen versions do not create such a temporary file.
2014-09-02 08:43:20 -07:00
Austin Clements
cca05ac10e lib: Fix endless upgrade problem
48db8c8 introduced a disagreement between when
notmuch_database_needs_upgrade returned TRUE and when
notmuch_database_upgrade actually performed an upgrade.  As a result,
if a database had a version less than 3, but no new features were
required, notmuch new would call notmuch_database_upgrade to perform
an upgrade, but notmuch_database_upgrade would return immediately
without updating the database version.  Hence, the next notmuch new
would do the same, and so on.

Fix this by ensuring that the upgrade-required logic is identical
between the two.
2014-09-01 23:06:51 -07:00
Austin Clements
658a00e7c8 lib: Update doc of notmuch_database_{needs_upgrade,upgrade}
Clients are no longer required to call these functions after opening a
database in read/write mode (which is good, because almost none of
them do!).
2014-08-30 11:42:07 -07:00
Austin Clements
ec573cd54f lib: Return an error from operations that require an upgrade
Previously, there was no protection against a caller invoking an
operation on an old database version that would effectively corrupt
the database by treating it like a newer version.

According to notmuch.h, any caller that opens the database in
read/write mode is supposed to check if the database needs upgrading
and perform an upgrade if it does.  This would protect against this,
but nobody (even the CLI) actually does this.

However, with features, it's easy to protect against incompatible
operations on a fine-grained basis.  This lightweight change allows
callers to safely operate on old database versions, while preventing
specific operations that would corrupt the database with an
informative error message.
2014-08-30 11:39:41 -07:00
Austin Clements
5dbfed4a73 lib: Support empty header values in database
Commit 567bcbc2 introduced support for storing various headers in
document values.  However, doing so in a backwards-compatible way
meant that genuinely empty header values could not be distinguished
from the old behavior of not storing the headers at all, so these
required parsing the original message.

Now that we have database features, new databases can declare that all
messages have header values, so if we have this feature flag, we can
use the stored header value even if it's the empty string.

This requires slight cleanup to notmuch_message_get_header, since the
code previously couldn't distinguish between empty headers and headers
that are never stored in the database (previously this distinction
didn't matter).
2014-08-30 11:37:33 -07:00
Austin Clements
02fec226fc lib: Report progress for combined upgrade operation
Previously, some parts of upgrade didn't report progress and for
others it was possible for the progress meter to restart at 0 part way
through the upgrade because each stage was reported separately.

Fix this by computing the total amount of work that needs to be done
up-front and updating completed work monotonically.
2014-08-30 11:36:08 -07:00
Austin Clements
e0635bd003 lib: Reorganize upgrade around document types
Rather than potentially making multiple passes over the same type of
data in the database, reorganize upgrade around each type of data that
may be upgraded.  This eliminates code duplication, will make
multi-version upgrades faster, and will let us improve progress
reporting.
2014-08-30 11:24:11 -07:00
Austin Clements
48db8c8b60 lib: Use database features to drive upgrade
Previously, we had database version information hard-coded in the
upgrade code.  Slightly re-organize the upgrade process around the set
of new database features to be enabled by the upgrade.
2014-08-30 11:21:48 -07:00
Austin Clements
4a38588488 lib: Simplify upgrade code using a transaction
Previously, the upgrade was organized as two passes -- an upgrade
pass, and a separate cleanup pass -- so the database was always in a
valid state.  This change substantially simplifies this code by
performing the upgrade in a transaction and combining both passes in
to one.  This 1) eliminates a lot of duplicate code between the
passes, 2) speeds up the upgrade process, 3) makes progress reporting
more accurate, 4) eliminates the potential for stale data if the
upgrade is interrupted during the cleanup pass, and 5) makes it easier
to reason about the safety of the upgrade code.
2014-08-30 10:45:36 -07:00
Austin Clements
7876bd72af test: Tests for future version and unknown feature handling 2014-08-30 10:44:17 -07:00
Austin Clements
d06adc52e0 test: Tool to build DB with specific version and features
This will let us test basic version and feature handling.
2014-08-30 10:43:46 -07:00
Austin Clements
8363c90531 lib: Database version 3: Introduce fine-grained "features"
Previously, our database schema was versioned by a single number.
Each database schema change had to occur "atomically" in Notmuch's
development history: before some commit, Notmuch used version N, after
that commit, it used version N+1.  Hence, each new schema version
could introduce only one change, the task of developing a schema
change fell on a single person, and it all had to happen and be
perfect in a single commit series.  This made introducing a new schema
version hard.  We've seen only two schema changes in the history of
Notmuch.

This commit introduces database schema version 3; hopefully the last
schema version we'll need for a while.  With this version, we switch
from a single version number to "features": a set of named,
independent aspects of the database schema.

Features should make backwards compatibility easier.  For many things,
it should be easy to support databases both with and without a
feature, which will allow us to make upgrades optional and will enable
"unstable" features that can be developed and tested over time.

Features also make forwards compatibility easier.  The features
recorded in a database include "compatibility flags," which can
indicate to an older version of Notmuch when it must support a given
feature to open the database for read or for write.  This lets us
replace the old vague "I don't recognize this version, so something
might go wrong, but I promise to try my best" warnings upon opening a
database with an unknown version with precise errors.  If a database
is safe to open for read/write despite unknown features, an older
version will know that and issue no message at all.  If the database
is not safe to open for read/write because of unknown features, an
older version will know that, too, and can tell the user exactly which
required features it lacks support for.
2014-08-30 10:42:08 -07:00
Austin Clements
344e4c65a4 new: Don't report version after upgrade
The version number has always been pretty meaningless to the user and
it's about to become even more meaningless with the introduction of
"features".  Hopefully, the database will remain on version 3 for some
time to come; however, the introduction of new features over time in
version 3 will necessitate upgrades within version 3.  It would be
confusing if we always tell the user they've been "upgraded to version
3".  If the user wants to know what's new, they should read the news.
2014-08-30 10:40:41 -07:00
Jani Nikula
523f06a0ad completion: fail silently if _init_completion is not found
The completion script depends on bash-completion 1.90 or later, with
_init_completion function. If that's not present, for some reason, the
completion currently fails with an ugly message, messing up user's
command line:

$ notmuch -bash: _init_completion: command not found

It's better to just not complete
2014-08-22 17:02:41 -07:00
Michal Sojka
028c56061e Make parsing of References and In-Reply-To header less error prone
According to RFC2822 References and In-Reply-To headers are supposed
to contain one or more Message-IDs, however older RFC822 allowed
almost any content. When both References and In-Reply-To headers ends
with something else that a Message-ID (see e.g. [1]), the thread
structure presented by notmuch is incorrect. The reason is that
notmuch treats this case as if the email contained no "replyto"
information (see _notmuch_database_link_message_to_parents).

This patch changes the parse_references() function to return the last
valid Message-ID encountered rather than NULL resulting from the last
hunk of text not being the Message-ID.

[1] https://lkml.org/lkml/headers/2014/5/19/864
2014-08-16 17:45:16 -07:00
Michal Sojka
61993923b4 Add test for incorrect threading of messages
This happens when there is some garbage after the last Message-ID in
the References header. See for example
https://lkml.org/lkml/headers/2014/5/19/864.
2014-08-16 17:45:07 -07:00
Austin Clements
c1845bf0a4 emacs: Improved compatibility for window-body-width in Emacs < 24
Fix byte compiler warning "Warning: the function `window-body-width'
is not known to be defined." by moving our compatibility wrapper
before its use and simplify the definition to a defalias for the old
name of the function.
2014-08-16 17:44:47 -07:00
David Bremner
f92342cb76 uploaded to Debian unstable
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQGcBAABCAAGBQJT5kVjAAoJEPIClx2kp54sdH4L+gL7hQAh8Sq/IkEydzxa6esG
 d6emMEyoZq9gfvj2wuWWF+lJ4ik6XrL0wc9KXtzyeFKAVMwql+wdx198+uhjjG1t
 SJ9a962EiqXCPfhskrmQhzWAw0e5gQxz8pvIPR0kUZ+4sMZjjhan3l5f2Hux1zti
 YnkJ13k8KTmMUWb80GHMGl2gq6D8k44Bi2Cc7QYf2czWn5Jx66AaQmj5zTl9+J+m
 RRz7dpumUidLxvxcQrKDeFEfq5AYOZZaqdwcW+sUeqP+n8XNqXZ5HkVgspaMg+Te
 EtioFwbXj0yCy5GktQz33HdQTJBU0EqpS6bhmXNkfkI7IS+jplMXFsAFNmB1vf6K
 UcMd8T5MBMWONk98MH7d/VRfvRp0uC4msIQR3WOxELGKVMwuSQPyjbiMrajzEATB
 UX/Z4cgKqroJpX45FIx7ga/2nsvJxwQITsV8hpIZwycoch34cCR14yBF1KPgds4V
 hp67wy6GGW/5b0b29wjXsxb6pU7kIC4wvgVCTdQs1Q==
 =r7jR
 -----END PGP SIGNATURE-----

Merge tag 'debian/0.18.1-2'

uploaded to Debian unstable
2014-08-09 13:21:57 -03:00
David Bremner
01c8bf89a4 debian: re-enable atomicity tests on armhf
Commit a33ec9c seems to have fixed the problem on the armhf
porterbox (harris.debian.org).
2014-08-09 12:02:30 -03:00
David Bremner
81d437723b debian: build with emacs24 by default, disable gdb on arm64
From wookey@debian.org

     id:20140808012130.GT7605@stoneboat.aleph1.co.uk

Fixes for the port in progress of debian to arm64.
2014-08-09 09:42:04 -03:00
Tomi Ollila
748798511d notmuch-config: talloc_strdup MAILDIR and NAME environment variables
When defined -- the pointer is soon given to talloc_free() which
expects it to be allocated by talloc.
2014-08-07 21:12:27 -03:00
Austin Clements
9f0f30f7de util: Const version of strtok_len
Because of limitations in the C type system, we can't a strtok_len
that can work on both const string and non-const strings.  The C
library solves this by taking a const char* and returning a char*
in functions like this (e.g., strchr), but that's not const-safe.
Solve it by introducing strtok_len_c, a version of strtok_len for
const strings.
2014-08-06 09:56:36 -03:00
Austin Clements
c95a398deb test: Include generated dependencies for test sources
Previously the build system was generating automatic header
dependencies for test sources, but only smtp-dummy was in SRCS, so
only its dependencies were being included.  Add all of the test
sources to SRCS so that the root Makefile.local includes their
dependencies.
2014-08-06 09:56:25 -03:00
Austin Clements
26b6470db3 new: Report and abort on upgrade failure
Previously the return status of notmuch_database_upgrade went
completely unchecked.
2014-08-06 09:56:12 -03:00
Austin Clements
0c1292051e lib: Improve documentation of _notmuch_message_create_for_message_id
Clarify the state of the returned message when
_notmuch_message_create_for_message_id returns
NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND.
2014-08-05 08:14:15 -03:00
Austin Clements
e501a16e71 emacs: Expand default saved searches and add shortcut keys
This should help new users off to a better start with the addition of
more sensible saved searches and default shortcut keys.  Most existing
users have probably customized this variable and won't be affected.
2014-08-05 08:07:52 -03:00
Austin Clements
3c1ad5bfa0 emacs: Introduce notmuch-jump: shortcut keys to saved searches
This introduces notmuch-jump, which is like a user-friendly,
user-configurable global prefix map for saved searches.  This provides
a non-modal and much faster way to access saved searches than
notmuch-hello.

A user configures shortcut keys in notmuch-saved-searches, which are
immediately accessible from anywhere in Notmuch under the "j" key (for
"jump").  When the user hits "j", the minibuffer immediately shows a
helpful table of bindings reminiscent of a completions buffer.

This code is a combination of work from myself (originally,
"notmuch-go"), David Edmondson, and modifications from Mark Walters.
2014-08-05 08:07:27 -03:00
Austin Clements
36a59b466a travis: Only notify IRC on success if the previous build failed
Without this, Travis is rather spammy.  Travis will continue to notify
the IRC channel on each build failure, which seems desirable.
2014-08-05 08:03:05 -03:00
Austin Clements
1d652c8719 lib: Fix slight misinformation in the database schema doc
The database schema documentation made it sound like each mail
document had exactly one on-disk message file, which hasn't been true
for a long time.
2014-08-04 18:58:11 -03:00
Austin Clements
30de720ba0 lib: Invalidate message metadata in _notmuch_message_gen_terms
Previously, we invalidated stored message metadata in
_notmuch_message_add_term and _notmuch_message_remove_term, but not in
_notmuch_message_gen_terms.  This doesn't currently result in any bugs
because of our limited uses of _notmuch_message_gen_terms, but it may
could cause trouble in the future.
2014-08-04 18:57:55 -03:00
Mark Oteiza
6c9721a407 config: read database.path from $MAILDIR if set
Try to read the config parameter database.path from $MAILDIR before
falling back to $HOME/mail
2014-07-31 07:13:38 -03:00
Mark Oteiza
488b261964 config: read user.name from $NAME if set
Try to read the config parameter user.name from $NAME before taking the
user name from /etc/passwd.
2014-07-31 07:13:14 -03:00
Austin Clements
a5a47dab87 util: Make string-util.h C++-compatible 2014-07-31 07:11:25 -03:00
Austin Clements
8251d639b2 emacs: Clarify that notmuch-poll-script is deprecated
notmuch-poll-script has long since been deprecated in favor of
post-new hooks, but this wasn't obvious from the documentation.
Update the documentation to make this clear.  Since
notmuch-poll-script could, to some extend, be used to control the path
of the notmuch binary and that use is now clearly discouraged, promote
notmuch-command to a real defcustom instead of just a variable.
2014-07-31 07:10:47 -03:00
Mark Walters
d25c729825 emacs: show: make return value of notmuch-show-get-prop explicit
This makes the fact the notmuch-show-get-prop returns nil if the major
mode is neither show not tree explicit.
2014-07-30 18:25:58 -03:00
Mark Walters
bbbdf0478e dump: make dump take Xapian write lock
Dump currently only takes the read lock. Xapian can cope with some
changes while maintaining a read snapshot but with more changes it
fails. Currently notmuch just gives a xapian error.

To avoid this we take the write lock when dumping. This prevents other
notmuch processes from modifying the xapian database preventing this
error.

Discussion with Olly on irc indicates that this is currently the best
solution: in xapian trunk there may be better possibilities using
snapshots but they need to make it to a release and propogate out to
users before we can switch approach.

Finally, this breaks one use case: pipelines of the form

notmuch dump | ... | notmuch restore

According to Olly this is already very fragile: it will only work on
small databases. One of the tests relies on this behaviour so fix that
to store the dump rather than use a pipe.
2014-07-16 19:33:10 -03:00
W. Trevor King
567aab405b nmbug: Handle missing @upstream in is_unmerged
If we don't have an upstream, there is nothing to merge, so nothing is
unmerged.  This avoids errors like:

  $ nmbug status
  error: No upstream configured for branch 'master'
  error: No upstream configured for branch 'master'
  fatal: ambiguous argument '@{upstream}': unknown revision or path not in the working tree.
  Use '--' to separate paths from revisions, like this:
  'git <command> [<revision>...] -- [<file>...]'
  'git rev-parse @{upstream}' exited with nonzero value

You might not have an upstream if you're only using nmbug locally to
version-control your tags.
2014-07-16 19:31:28 -03:00
Austin Clements
c2bbe9eb6c test: Test thread linking in all possible delivery orders
These tests deliver all possible (single-root) four-message threads in
all possible orders and check that notmuch successfully links them
into threads.  These tests supersede and replace the previous and much
less thorough "T260-thread-order" tests.

There are two variants of the test: one delivers messages that
reference only their immediate parent and the other delivers messages
that reference all of their parents.  The latter test is currently
known-broken.
2014-07-16 07:08:02 -03:00