A new item from IRC discussion, (speeding up "notmuch restore"), as
well as a bug I just hit myself, (content from citations is not being
indexed).
While here, notce that several items have recently been completed ('?'
now displays documentation, not function names; we have a search
binding from notmush-show-mode; and "notmuch new" responds to SIGINT
by flushing). Finally, the item regarding optimizing chunky searching
is irrelevant since we dropped chunky searching in favor of the much
better streamed searching.
Since we need to do this for portability, (some systems don't have a
strndup function), we might as well do it unconditionally. There's
almost no disadvantage to doing so, and this has the advantages of not
requiring a configure-time check nor having two different
implementations, one of which would often be less tested.
This way, the user gets a steady (but bursty) stream of reults. We
double the chunk size each time since each successive chunk has to
redo work from all previous chunks.
Of course, the overall time is thereby slower, as the price we pay for
increased responsiveness. With a search returning about 17000 thread
results I measured a total time of 48.8 seconds before this change and
58.4 seconds afterwards.
The rudimentary aspect here is that the date ranges are specified with
UNIX timestamp values (number of seconds since 1970-01-01 UTC). One
thing that can help here is using the date program to determins
timestamps, such as:
$(date +%s -d 2009-10-01)..$(date +%s)
Long-term, we'll probably need to do our own query parsing to be able
to support directly-specified dates and also relative expressions like
"since:'2 months ago'".
I had these notes sitting in an uncommitted file that was cluttering
up my "git status" output. This cleans that up, and also shares the
ideas with the wider community.
This note was described in the previous commit message, but mistakenly
not committed:
The note about making "notmuch setup" faster is now rewritten to apply
to "notmuch new" since "notmuch setup" no longer does any mail
indexing.
We recently added support for "notmuch reply" and also made (most of)
the hidden components self documenting.
The note about making "notmuch setup" faster is now rewritten to apply
to "notmuch new" since "notmuch setup" no longer does any mail
indexing.
A recent "notmuch restore" command took *forever* for me. Obviously,
we need to fix the underlying performance bug in Xapian, but in the
meantime, a progress indicator would help.
The magic space bar is nice, but sometimes there's a message with a
long attachment that I just want to skip, but still consider the
message marked as read.
I had noticed several times earlier that having a talloc context
passed in would make things more convenient. I'm not exercising
that convenience yet, but the context is there now, (and there's
one fewer item on our TODO list).
The timestamp stuff we'll want to do soon, since it's a database
change, (though not a major one---at worst a handful of stale
timestamp documents would be left in the database).
"notmuch tag" is implemented now and seems to work great (and fast).
As for the race condition, as noted in the description we're removing
it's not exposed directly in the API, but only in a client that
allows for looping over search results and removing the inbox tag
from all of them. But then, that's exactly what the "notmuch tag"
command does. So, as discussed, we've now documented that command
to highlight the issue. Problem resolved, (as well as we can).
Interstingly, it's our simple "notmuch" client that's going to be the
most difficult to fix. There's just not as much information preserved
in the textual representation from "notmuch search" as there is in the
objects returned from notmuch_query_search_threads.
The archive-thread race condition doesn't even exist now because there's
no command for modifying tags at the level of a thread (just individual
messages).
This means that the restore operation will now properly pick up the
removal of tags indicated by the tag just not being present in the
dump file.
We added a few new public functions in order to support this:
notmuch_message_freeze
notmuch_message_remove_all_tags
notmuch_message_thaw
