I had previously thought Xapian only offered an estimate for the
number of results that might match a search. But Olly let me know
that we can easily ask for Xapian to provide the exact count.
We can't fit all the authors and the various changed subjects in 80
columns, but it would be great is isearch could still find these, (and
automatically expand the hidden content as necessary).
It was noted today in IRC that libnotmuch is not yet careful about
wrapping all Xapian calls with try/catch blocks to print nicer error
messages. It seems it would be natural to audit that at the same time
as doing the symbol-hiding work.
One of these is a bad bug I noticed this morning, (archiving messages I had
never read when going through a search of "tag:inbox and tag:to-me" and
hitting space bar).
The other ideas came from recent conversations with Dirk and Eric.
Recent coding around the "*" feature suggests some improvements that
we could make, (some of which might push us into writing a custom
query parser rather than using the one that exists in Xapian).
The recent fix to handle utf8 in the JSON output is the kind of bug
I'd never like to see again, (so that I'd like the test suite to be
helping us track that).
When trying to restore the current position, if the "current" thread
no longer appears in the buffer, then '=' moves to the current line
instead. When near the end of the buffer, the "current" line should
be counted as the number of lines from the end.
I just tried (and failed) to write a test for the recent magic
inference of phrase searches. That's a feature that makes me *really*
uncomfortable to not have an automated test. But I believe the
proposed modularization of the test suite should reduce some quoting
nightmares, so will hopefully make this easier.
There's been a lot of good work done, and we've been doing a generally
poor job of noticing when some of the tasks we've completed were
already on our TODO list.
So here's a careful scan, removing all items I could find that have
already been done.
We've been using --output in IRC and on the mailing list for a while,
(someone had the good sense to point out that --for would defeat
command-line completion since it's a prefix of the proposed --format).
Similar to the way thread-viewing was broken after a thread was
archived, (and recently fixed), tag manipulation has also been broken
when the thread no longer matches the current search.
This also means that the behavior of '+' and '-' are now different
than that of '*'. The '+' and '-' bindings now return to the previous
behavior old affecting all messages in the thread, (and not simply
those matching the search).
I actually prefer this behavior, since otherwise a '-' operation on a
thread might not actually remove the tag from the thread, (since it
could operate on a subset of the thread and not hit all messages with
the given tag).
So I'd now like to fix '*' to be consistent with '+' and '-', for
which we add an item to TODO.
This would provide support for "muted" threads, as well as allowing for negative
filtering based on messages not matched by the original search, (but present in
threads that do have at least one matched message).
A new item from IRC discussion, (speeding up "notmuch restore"), as
well as a bug I just hit myself, (content from citations is not being
indexed).
While here, notce that several items have recently been completed ('?'
now displays documentation, not function names; we have a search
binding from notmush-show-mode; and "notmuch new" responds to SIGINT
by flushing). Finally, the item regarding optimizing chunky searching
is irrelevant since we dropped chunky searching in favor of the much
better streamed searching.
Since we need to do this for portability, (some systems don't have a
strndup function), we might as well do it unconditionally. There's
almost no disadvantage to doing so, and this has the advantages of not
requiring a configure-time check nor having two different
implementations, one of which would often be less tested.
This way, the user gets a steady (but bursty) stream of reults. We
double the chunk size each time since each successive chunk has to
redo work from all previous chunks.
Of course, the overall time is thereby slower, as the price we pay for
increased responsiveness. With a search returning about 17000 thread
results I measured a total time of 48.8 seconds before this change and
58.4 seconds afterwards.
The rudimentary aspect here is that the date ranges are specified with
UNIX timestamp values (number of seconds since 1970-01-01 UTC). One
thing that can help here is using the date program to determins
timestamps, such as:
$(date +%s -d 2009-10-01)..$(date +%s)
Long-term, we'll probably need to do our own query parsing to be able
to support directly-specified dates and also relative expressions like
"since:'2 months ago'".
I had these notes sitting in an uncommitted file that was cluttering
up my "git status" output. This cleans that up, and also shares the
ideas with the wider community.
This note was described in the previous commit message, but mistakenly
not committed:
The note about making "notmuch setup" faster is now rewritten to apply
to "notmuch new" since "notmuch setup" no longer does any mail
indexing.
We recently added support for "notmuch reply" and also made (most of)
the hidden components self documenting.
The note about making "notmuch setup" faster is now rewritten to apply
to "notmuch new" since "notmuch setup" no longer does any mail
indexing.
A recent "notmuch restore" command took *forever* for me. Obviously,
we need to fix the underlying performance bug in Xapian, but in the
meantime, a progress indicator would help.
