notmuch/TODO

Fix the things that are causing the most pain to new users
----------------------------------------------------------
1. A new import is tagging all messages as "inbox" -- total pain

2. Allow an easy way to get tags from directory names (if the user has them)

3. Fix Xapian defect #250 so tagging is fast.

Emacs interface (notmuch.el)
----------------------------
Enhance '+' and '-' in the search view to operate on an entire region
if set.

Fix '*' to work by simply calling '+' or '-' on a region consisting of
the entire buffer.

Add a global keybinding table for notmuch, and then view-specific
tables that add to it.

Add a command to archive all threads in a search view.
	
Add a '|' binding from the search view.

When a thread has been entirely read, start out by closing all
messages except those that matched the search terms.

Add support for choosing from one of the user's configured email
addresses for the From line.

Make 'notmuch-show-pipe-message have a private history.

Add support for a delete keybinding that adds a "deleted" tag to the
current message/thread and make searches not return deleted messages
by default, (unless the user asks explicitly for deleted messags in
the search query).

Add support to "mute" a thread (add a "muted" tag and then don't
display threads in searches by default where any message of the thread
has the "muted" tag).

Fix i-search to open up invisible citations as necessary.

Portability
-----------
Fix configure script to test each compiler warning we want to use.

Completion
----------
Fix bash completion to complete multiple search options (both --first
and *then* --max-threads), and also complete value for --sort=
(oldest-first or newest-first).

notmuch command-line tool
-------------------------
Implement "notmuch search --exclude-threads=<search-terms>" to allow
for excluding muted threads, (and any other negative, thread-based
filtering that the user wants to do).

Fix "notmuch show" so that the UI doesn't fail to show a thread that
is visible in a search buffer, but happens to no longer match the
current search. (Perhaps add a --matching=<secondary-search-terms>
option (or similar) to "notmuch show".)

Teach "notmuch search" to return many different kinds of results. Some
ideas:

	notmuch search --output=threads	# Default if no --output is given
	notmuch search --output=messages
	notmuch search --output=tags
	notmuch search --output=addresses
	notmuch search --output=terms

Add a "--format" option to "notmuch search", (something printf-like
for selecting what gets printed).

Add a "--count-only" (or so?) option to "notmuch search" for returning
the count of search results.

Add documented syntax for searching all threads/messages.

Give "notmuch restore" some progress indicator. Until we get the
Xapian bugs fixed that are making this operation slow, we really need
to let the user know that things are still moving.

Fix "notmuch restore" to operate in a single pass much like "notmuch
dump" does, rather than doing N searches into the database, each
matching 1/N messages.

Add a "-f <filename>" option to select an alternate configuration
file.

Fix notmuch.c to call add_timestamp/get_timestamp with path names
relative to the database path. (Otherwise, moving the database to a
new directory will result in notmuch creating new timestamp documents
and leaving stale ones behind.)

Fix notmuch.c to use a DIR prefix for directory timestamps, (the idea
being that it can then add other non-directory timestamps such as for
noting how far back in the past mail has been indexed, and whether it
needs to re-tag messages based on a theoretical "auto-tags"
configuration file).

Make "notmuch new" notice when a mail directory has gone more than a
month without receiving new mail and use that to trigger the printing
of the note that the user might want to mark the directory read-only.

Also make "notmuch new" optionally able to just mark those month-old
directories read-only on its own. (Could conflict with low-volume
lists such as announce lists if they are setup to deliver to their own
maildirs.)

Allow configuration for filename patterns that should be ignored when
indexing.

notmuch library
---------------
Index content from citations, please.

Provide a sane syntax for date ranges. First, we don't want to require
both endpoints to be specified. For example it would be nice to be
able to say things like "since:2009-01-1" or "until:2009-01-1" and
have the other endpoint be implicit. Second we'd like to support
relative specifications of time such as "since:'2 months ago'". To do
any of this we're probably going to need to break down an write our
own parser for the query string rather than using Xapian's QueryParser
class.

Make failure to read a file (such as a permissions problem) a warning
rather than an error (should be similar to the existing warning for a
non-mail file).

Add support for files that are moved or deleted (which obviously need
to be handled differently).

Actually compile and install a libnotmuch shared library.

Fix to use the *last* Message-ID header if multiple such headers are
encountered, (I noticed this is one thing that kept me from seeing the
same message-ID values as sup).

Add support for the user to specify custom headers to be indexed.

Add support for configuring "virtual tags" which are a tuple of
(tag-name, search-specification). The database is responsible for
ensuring that the virtual tag is always consistent.

Indicate to the user if two files with the same message ID have
content that is actually different in some interesting way. Perhaps
notmuch initially sees all changes as interesting, and quickly learns
from the user which changes are not interesting (such as the very
common mailing-list footer).

Fix notmuch_query_count_messages to share code with
notmuch_query_search_messages rather than duplicating code. (And
consider renaming it as well.)

Provide a mechanism for doing automatic address completion based on
notmuch searches. Here was one proposal made in IRC:

	<cworth> I guess all it would really have to be would be a way
	         to configure a series of searches to try in turn,
	         (presenting ambiguities at a given single level, and
	         advancing to the next level only if one level
	         returned no matches).
	<cworth> So then I might have a series that looks like this:
	<cworth> notmuch search --output=address_from tag:address_book_alias
	<cworth> notmuch search --output=address_to tag:sent
	<cworth> notmuch search --output=address_from
	<cworth> I think I might like that quite a bit.
	<cworth> And then we have a story for an address book for
	         non-emacs users.

Provide a ~me Xapian synonym for all of the user's configured email
addresses.

General
-------
Audit everything for dealing with out-of-memory (and drop xutil.c).

Write a test suite.

Achieve 100% test coverage with the test suite.

Investigate why the notmuch database is slightly larger than the sup
database for the same corpus of email.

Xapian
------
Fix defect #250

	replace_document should make minimal changes to database file
	http://trac.xapian.org/ticket/250

	It looks like it's going to be easy to fix. Here's the file to
	change:

		xapian-core/backends/flint/flint_database.cc

	And look for:

	  // FIXME - in the case where there is overlap between the new
	  // termlist and the old termlist, it would be better to compare the
	  // two lists, and make the minimum set of modifications required.
	  // This would lead to smaller changesets for replication, and
	  // probably be faster overall

	So I think this might be as easy as just walking over two
	sorted lists looking for differences.

	Note that this is in the currently default "flint" backend,
	but the Xapian folks are probably more interested in fixing
	the in-development "chert" backend. So the patch to get
	upstreamed there will probably also fix:

		xapian-core/backends/chert/chert_database.cc

	(I'm hoping the fix will be the same---an identical comment
	exists there.)

	Also, if you want to experiment with the chert backend,
	compile current Xapian source and run notmuch with
	XAPIAN_PREFER_CHERT=1. I haven't tried that yet, but there are
	claims that a chert database can be 40% smaller than an
	equivalent flint database.

Report this bug:

	"tag:foo and tag:bar and -tag:deleted" goes insane

	This seems to be triggered by a Boolean operator next to a
	token starting with a non-word character---suddenly all the
	Boolean operators get treated as literal tokens)