Commit graph

2321 commits

Author SHA1 Message Date
Carl Worth
5a4532860c notmuch-test: Eliminate sleeps to speed up test suite run
We were sleeping merely to ensure that our updates to the mail store
would result in the mtime of the appropriate directories being
updated. We make the test suite run faster by not sleeping, but
instead explicitly updating the mtime of the directory to a future
time with touch.

We're careful to ensure that the time is not merely in the future
compared to the current time, but also later than any previous update
to the same directory mtime.
2010-02-04 12:10:42 -08:00
Carl Worth
a821ba5737 notmuch-test: Allow custom headers when generating messages
This provides the control that future tests will need, (for example,
adding a CC field to ensure proper handling with "notmuch reply",
etc.)
2010-02-04 08:44:05 -08:00
Carl Worth
0d67c52f4d notmuch-test: Use named-parameters for generate_message function
This makes the test suite bash-specific, but that's not much of
an issue for me, (if somebody else would prefer some other language
then they can rewrite the test suite and maintain it).

The advantage here is that we'll now be able to easily generate
custom messages for testing operations that depend on the message
content, (such as "notmuch reply", etc.).
2010-02-04 08:44:05 -08:00
Carl Worth
8d37b2f982 Add actual testing to notmuch-test
We verify that each command creates output exactly as expected (after
ignoring variable output such as the report of how long an operation
took).
2010-02-04 08:44:05 -08:00
martin f. krafft
c2904e9842 Add suggestion to vim-addon-manager.
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-26 09:52:58 +13:00
Carl Worth
a2d919eb2f Add a very rough beginning of a test suite.
This notmuch-test script simply runs a few different notmuch operations,
(things that I found were useful while testing the rename-support code).

It's not useful as a test suite yet, since it doesn't actually check
the results of any operation, (the user of the suite has to know what
the results should be and must manually verify them. So there's no
integration with the build system yet, (no "make test" target).

But I didn't want to lose what I had so far, so here it is.
2010-01-24 07:36:39 +13:00
Carl Worth
4e051f8d95 notmuch.desktop: Clarify that this is the emacs interface to notmuch.
And the name is notmuch not not much.
2010-01-24 07:30:08 +13:00
Jeffrey C. Ollie
1bf121924d Add install target for notmuch.desktop file.
Add an install target that uses desktop-file-install to install the
desktop file in the appropriate location.  The location of the install
can be modified by changing the desktop_dir variable.

Signed-off-by: Jeffrey C. Ollie <jeff@ocjtech.us>
2010-01-24 07:29:54 +13:00
Carl Worth
c25bc03dc6 Fix misspelling of DT_UNKNOWN.
How foolish of me to advertise the fact that I pushed a commit without
compiling it first...
2010-01-23 22:45:23 +13:00
Carl Worth
184625ac9b README: Tighten up the text a bit.
As Keith pointed out, (with a humorous citation from Mark Twain),
the two uses of "very" added nothing to the description. Also,
"large collection of email" was repeated uselessly.
2010-01-23 19:00:25 +13:00
Carl Worth
344c48a47d Add some comments to document the recently-fixed handling of d_type.
The fix was subtle, (requiring less code than originally expected), so
it behooves us to document it well.
2010-01-23 18:58:30 +13:00
Geo Carncross
c5416b6f1b notmuch new: Fix to work on filesystems returning DT_UNKNOWN
Such as reiserfs or xfs. This has been broken since the merge of
support for rename and deletion of files from the mail store.

Here's the original justification for the patch:

A review of notmuch-new.c shows three uses of ->d_type:

Near line 153, in _entries_resemble_maildir() we can simply allow for
DT_UNKNOWN. This would fail if people have MH-style folders which have
three folders called "new" "cur" and "tmp", but that seems unlikely, in
which case the "tmp" folder would simply not be scanned.

Near line 273 in add_files_recursive() we have another check. If
DT_UNKNOWN, we fall through, then add_files_recursive() does a stat
almost immediately, returning with success if the path isn't a
directory.

Thus, the fallback is already written.

Finally, near line 343, in add_files_recursive() (a long function) we
have another check. Here we can simply treat DT_UNKNOWN as DT_LNK, since
the logic for the stat() results are the same.
2010-01-23 18:52:30 +13:00
martin f. krafft
b416d7e6b9 builddep on dh must be versioned due to overrides
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:58:32 +13:00
martin f. krafft
6e4be17ab8 Install vim plugin with vim-addons
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:58:19 +13:00
martin f. krafft
9a66a8f877 add myself to uploaders
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:40:35 +13:00
martin f. krafft
cd9778c967 put cworth as maintainer per his own request
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:40:17 +13:00
martin f. krafft
814d2887ae Build-depend on emacs23
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:19:16 +13:00
martin f. krafft
38f1bf18c3 add ITP bug number to changelog entry about upload
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:09:00 +13:00
martin f. krafft
76ba495696 no need to create /usr/share dirs
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:08:08 +13:00
martin f. krafft
1551c35925 Put debian/* under separate copyright.
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:07:41 +13:00
martin f. krafft
cf72677e34 Fix suggestions/enhancements for vim+emacs
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:06:09 +13:00
martin f. krafft
61209b5d65 Start new changelog stanza pending upload
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 14:05:45 +13:00
martin f. krafft
9cef26de47 add Vcs-Browser field
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-21 13:59:40 +13:00
martin f. krafft
0ea5f3fc0e Merge branch 'upstream' 2010-01-21 13:58:55 +13:00
martin f. krafft
1ef33800df Install zsh completion file
According to the Debian zsh maintainer Clint Adams, this is the first
time that a package installs its own completer into zsh. Part of the
reason this is not usually done is because zsh does not provide a stable
API.

We agreed to try it, given that notmuch is expected to change quite
a bit initially. If there are problems or the completer goes stable,
we'll move it into the upstream zsh repository.

Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-14 18:12:13 +13:00
Carl Worth
c340c1bd11 notmuch new: Print upgrade progress report as a percentage.
Previously we were printing a number of messages upgraded so far. The
original motivation for this was to accurately reflect the fact that
there are two passes, (so each message is processed twice and it's not
accurate to represent with a single count). But as it turns out, the
second pass takes zero time (relatively speaking) so we're still not
accounting for it.

If nothing else, the percentage-based reporting makes for a cleaner
API for the progress_notify function.
2010-01-09 17:38:23 -08:00
Carl Worth
ccf2e0cc42 lib: Add non-content terms with a WDF value of 0.
The WDF is the "within-document frequency" value for a particular
term. It's intended to provide an indication of how frequent a term is
within a document, (for use in computing relevance). Xapian's term
generator already computes WDF values when we use that, (which we do
for indexing all mail content).

We don't use the term generator when adding single terms for things
that don't actually appear in the mail document, (such as tags, the
filename, etc.). In this case, the WDF value for these terms doesn't
matter much.

But Xapian's flint backend can be more efficient with changes to terms
that don't affect the document "length". So there's a performance
advantage for manipulating tags (with the flint backend) if the WDF of
these terms is 0.
2010-01-09 11:18:27 -08:00
Carl Worth
45b1856782 lib: Explicitly set BoolWeight when searching.
All notmuch searches currently sort by value (either date or message
ID) so it's just wasted effort for Xapian to compute relevance values
for each result. We now explicitly tell Xapian that we're uninterested
in the relevance values.
2010-01-09 11:16:40 -08:00
Carl Worth
d12801c8b4 lib: Split the database upgrade into two phases for safer operation.
The first phase copies data from the old format to the new format
without deleting anything. This allows an old notmuch to still use the
database if the upgrade process gets interrupted. The second phase
performs the deletion (after updating the database version number). If
the second phase is interrupted, there will be some unused data in the
database, but it shouldn't cause any actual harm.
2010-01-09 11:13:12 -08:00
martin f. krafft
994e46845d Add build-dependency on emacs
Signed-off-by: martin f. krafft <madduck@debian.org>
2010-01-09 11:50:12 +13:00
Carl Worth
5fe5e802ab lib: Delete stale timestamp documents during database upgrade.
Once we move the timestamp to the new directory document, we don't
need the old one anymore.
2010-01-08 09:57:09 -08:00
Carl Worth
c485c51585 notmuch new: Don't prevent database upgrade from being interrupted.
Our signal handler is designed to quickly flush out changes and then
exit. But if a database upgrade is in progress when the user
interrupts, then we just want to immediately abort. We could do
something fancy like add a return value to our progress_notify
function to allow it to tell the upgrade process to abort. But it's
actually much cleaner and robust to delay the installation of our
signal handler so that the default abort happens on SIGINT.
2010-01-08 08:45:16 -08:00
Carl Worth
1c86b48329 notmuch new: Fix progress notification on database upgrade.
This was firing continuously rather than just once per second as
intended.
2010-01-07 21:24:44 -08:00
Carl Worth
e307e990c9 notmuch new: Automatically upgrade the database if necessary.
This takes advantage of the recently added library support to detect
if the database needs to be upgraded and then automatically performs
that upgrade, (with a nice progress report).
2010-01-07 18:30:32 -08:00
Carl Worth
909f52bd8c lib: Implement versioning in the database and provide upgrade function.
The recent support for renames in the database is our first time
(since notmuch has had more than a single user) that we have a
database format change. To support smooth upgrades we now encode a
database format version number in the Xapian metadata.

Going forward notmuch will emit a warning if used to read from a
database with a newer version than it natively supports, and will
refuse to write to a database with a newer version.

The library also provides functions to query the database format
version:

	notmuch_database_get_version

to ask if notmuch wants a newer version than that:

	notmuch_database_needs_upgrade

and a function to actually perform that upgrade:

	notmuch_database_upgrade
2010-01-07 18:26:31 -08:00
Carl Worth
21f8fd6967 notmuch new: Fix deletion support to recurse on removed directories.
Previously, when notmuch detected that a directory had been deleted it
was only removing files immediately in that directory. We now
correctly recurse to also remove any directories (and files, etc.)
within sub-directories, etc.
2010-01-07 18:20:28 -08:00
Carl Worth
cb8e4bc9c0 TODO: Add a couple of ideas that came up during recent coding.
The notmuch_query_count_messages functions duplicates a lot of code
undesirably.
2010-01-07 18:17:38 -08:00
Carl Worth
807aef93d3 Prefer READ_ONLY consistently over READONLY.
Previously we had NOTMUCH_DATABASE_MODE_READ_ONLY but
NOTMUCH_STATUS_READONLY_DATABASE which was ugly and confusing. Rename
the latter to NOTMUCH_STATUS_READ_ONLY_DATABASE for consistency.
2010-01-07 10:29:05 -08:00
Carl Worth
f93b7218c3 lib: Consolidate checks for read-only database.
Previously, many checks were deep in the library just before a cast
operation. These have now been replaced with internal errors and new
checks have instead been added at the beginning of all top-levelentry
points requiring a read-write database.

The new checks now also use a single function for checking and
printing the error message. This will give us a convenient location to
extend the check, (such as based on database version as well).
2010-01-07 10:19:44 -08:00
Carl Worth
6ed606c19e lib: Clarify internal documentation of _notmuch_database_filename_to_direntry
The original wording made it sound like this function was just doing
some string manipulation. But this function actually creates new
directory documents as a side effect. So make that explicit in its
documentation.
2010-01-07 09:31:58 -08:00
Carl Worth
a274848f95 notmuch_message_get_filename: Support old-style filename storage.
When a notmuch database is upgraded to the new database format, (to
support file rename and deletion), any message documents corresponding
to deleted files will not currently be upgraded. This means that a
search matching these documents will find no filenames in the expected
place.

Go ahead and return the filename as originally stored, (rather than
aborting with an internal error), in this case.
2010-01-07 09:22:34 -08:00
Carl Worth
1a38cb841c notmuch new: Never ask the database for any names from a new directory.
When we know that we are adding a new directory to the database, (and
we therefore are using inode rather than strcmp-based sorting of the
filenames), then we *never* want to see any names from the
database. If we get any names that could only make us inadvertently
remove files that we just added.

Since it's not obvious from the Xapian documentation whether new terms
being added as part of new documents will appear in the in-progress
all-terms iteration we are using, (and this might differ based on
Xapian backend and also might differ based on how many new directories
are added and whether a flush threshold is reached).

For all of these reasons, we play it safe and use NULL rather than a
real notmuch_filenames_t iterator in this case to avoid any problem.
2010-01-06 14:35:56 -08:00
Carl Worth
957ae198e7 lib: Treat NULL as a valid (and empty) notmuch_filenames_t iterator.
This will be convenient to avoid some special-casing in higher-level
code.
2010-01-06 14:35:11 -08:00
Carl Worth
7d8271dd9d notmuch new: Fix bug resulting in file removal on initial build of database.
The bug here was that we would see that the database did not know
anything about a directory so would get results from the filesystem in
inode rather than strcmp order.

However, we wouldn't actually ask for the list of files from the
database until after recursing into the sub-directories. So by the
time we traverse the filenames looking for deletions, the database
*does* have entries and we end up detecting erroneous deletions
because our filename list from the filesystem isn't in strcmp order.

So ask for the list of names from the database before doing any
additions to avoid this problem.
2010-01-06 13:54:39 -08:00
Carl Worth
59c09623c8 notmuch new: Fix to detect deletions of names at the end of the list.
Previously we only scanned the list of filenames in the filesystem and
detected a deletion whenever that scan skipped a name that existed in
the database. That much was fine, but we *also* need to continue
walking the list of names from the database when the filesystem list
is exhausted.

Without this, removing the last file or directory within any
particular directory would go undetected.
2010-01-06 13:26:47 -08:00
Carl Worth
39e81ca431 notmuch new: Fix regression preventing addition of symlinked mail files.
As described in the previous commit message, we introduced multiple
symlink-based regressions in commit
3df737bc4addfce71c647792ee668725e5221a98

Here, we fix the case of symlinks to regular files by doing an extra
stat of any DT_LNK files to determine if they do, in fact, link to
regular files.
2010-01-06 10:48:43 -08:00
Carl Worth
49f09958df notmuch new: Fix regression preventing recursion through symlinks.
In commit 3df737bc4addfce71c647792ee668725e5221a98 we switched from
using stat() to using the d_type field in the result of scandir() to
determine whether a filename is a regular file or a directory. This
change introduced a regression in that the recursion would no longer
traverse through a symlink to a directory. (Since stat() would resolve
the symlink but with scandir() we see a distinct DT_LNK value in
d_type).

We fix this for directories by allowing both DT_DIR and DT_LNK values
to recurse, and then downgrading the existing not-a-directory check
within the recursion to not be an error. We also add a new
not-a-directory check outside the recursion that is an error.
2010-01-06 10:32:06 -08:00
Carl Worth
bd72d95bac Fix typo in comment.
The difference between "now" and "not" ends up being fairly dramatic.
2010-01-06 10:32:06 -08:00
Carl Worth
9d4d7963a1 notmuch new: Print counts of deleted and renamed messages.
It's nice to be able to see a report indicating that the recently
added support for detecting file rename and deletion is working.
2010-01-06 10:32:06 -08:00
Carl Worth
4b418343f6 lib: Indicate whether notmuch_database_remove_message removed anything.
Similar to the return value of notmuch_database_add_message, we now
enhance the return value of notmuch_database_remove_message to
indicate whether the message document was entirely removed (SUCCESS)
or whether only this filename was removed and the document exists
under other filenamed (DUPLICATE_MESSAGE_ID).
2010-01-06 10:32:06 -08:00