This gives a rather decent reduction in number of seeks required when
reading a Maildir that isn't in pagecache.
Most filesystems give some locality on disk based on inode numbers.
In ext[234] this is the inode tables, in XFS groups of sequential inode
numbers are together on disk and the most significant bits indicate
allocation group (i.e inode 1,000,000 is always after inode 1,000).
With this patch, we read in the whole directory, sort by inode number
before stat()ing the contents.
Ideally, directory is sequential and then we make one scan through the
file system stat()ing.
Since the universe is not ideal, we'll probably seek during reading the
directory and a fair bit while reading the inodes themselves.
However... with readahead, and stat()ing in inode order, we should be
in the best place possible to hit the cache.
In a (not very good) benchmark of "how long does it take to find the first
15,000 messages in my Maildir after 'echo 3 > /proc/sys/vm/drop_caches'",
this patch consistently cut at least 8 seconds off the scan time.
Without patch: 50 seconds
With patch: 38-42 seconds.
(I did this in a previous maildir reading project and saw large improvements too)
By installing a signal handler for SIGINT we can ensure that no work
that is already complete will be lost if the user interrupts a
"notmuch new" run with Control-C.
I recently discovered that mb2md has the annoying bug of creating
files with mtime of 0, and notmuch then promptly ignored them,
(thinking that its timestamps initialized to 0 were just as new).
We fix notmuch to not exclude messages based on a database timestamp
of 0.
Leaving this variable uninitialized caused notmuch to display a random
number while counting files for the new database.
Signed-off-by: Keith Packard <keithp@keithp.com>
Now that the client sources are alone here in their own directory,
(with all the library sources down inside the lib directory), we can
break the client up into multiple files without mixing the files up.
The hope is that these smaller files will be easier to manage and
maintain.
