This function uses Xapian's Compactor machinery to compact the notmuch
database. The compacted database is built in a temporary directory and
later moved into place while the original uncompacted database is
preserved.
Signed-off-by: Ben Gamari <bgamari.foss@gmail.com>
notmuch_message_get_header started returning some headers straight
from the database in 567bcbc, but this comment explicitly claimed all
headers were read from the message file.
Add NOTMUCH_EXCLUDE_FLAG to notmuch_exclude_t so that it can
cover all four values of search --exclude in the cli.
Previously the way to avoid any message being marked excluded was to
pass in an empty list of excluded tags: since we now have an explicit
option we might as well honour it.
The enum is in a slightly strange order as the existing FALSE/TRUE
options correspond to the new
NOTMUCH_EXCLUDE_FLAG/NOTMUCH_EXCLUDE_TRUE options so this means we do
not need to bump the version number.
Indeed, an example of this is that the cli count and show still use
FALSE/TRUE and still work.
Previously, getting the list of all messages in a thread required
recursively traversing the thread's message hierarchy, which was both
difficult and resulted in messages being out of order. This adds a
public function to retrieve an iterator over all of the messages in a
thread in oldest-first order.
Using the new support from _notmuch_directory_create, this makes
notmuch_database_get_directory a read-only operation that simply
returns the directory object if it exists or NULL otherwise. This
also means that notmuch_database_get_directory can work on read-only
databases.
This change breaks the directory mtime workaround in notmuch-new.c by
fixing the exact issue it was working around. This permits mtime
update races to prevent scans of changed directories, which
non-deterministically breaks a few tests. The next patch fixes this.
Previously, notmuch_database_get_directory had no way to indicate how
it had failed. This changes its prototype to return a status code and
set an out-argument to the retrieved directory, like similar functions
in the library API. This does *not* change its currently broken
behavior of creating directory objects when they don't exist, but it
does document it and paves the way for fixing this. Also, it can now
check for a read-only database and return
NOTMUCH_STATUS_READ_ONLY_DATABASE instead of crashing.
In the interest of atomicity, this also updates calls from the CLI so
that notmuch still compiles.
This is the notmuch_database_create equivalent of the previous change.
In this case, there were places where errors were not being propagated
correctly in notmuch_database_create or in calls to it. These have
been fixed, using the new status value.
It has been a long-standing issue that notmuch_database_open doesn't
return any indication of why it failed. This patch changes its
prototype to return a notmuch_status_t and set an out-argument to the
database itself, like other functions that return both a status and an
object.
In the interest of atomicity, this also updates every use in the CLI
so that notmuch still compiles. Since this patch does not update the
bindings, the Python bindings test fails.
Formerly notmuch_database_close closed the xapian database and
destroyed the talloc structure associated with the notmuch database
object. Split notmuch_database_close into notmuch_database_close and
notmuch_database_destroy.
This makes it possible for long running programs to close the xapian
database and thus release the lock associated with it without
destroying the data structures obtained from it.
This also makes the api more consistent since every other data
structure has a destructor function.
The comments in notmuch.h are a courtesy of Austin Clements.
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
Add the NOTMUCH_MESSAGE_FLAG_EXCLUDED flag to
notmuch_query_search_threads. Implemented by inspecting the tags
directly in _notmuch_thread_create/_thread_add_message rather than as
a Xapian query for speed reasons.
Note notmuch_thread_get_matched_messages now returns the number of
non-excluded matching messages. This API is not totally desirable but
fixing it means breaking binary compatibility so we delay that.
Add a flag NOTMUCH_MESSAGE_FLAG_EXCLUDED which is set by
notmuch_query_search_messages for excluded messages. Also add an
option omit_excluded_messages to the search that we do not want the
excludes at all.
This exclude flag will be added to notmuch_query_search threads in the
next patch.
This is useful for tags like "deleted" and "spam" that people
generally want to exclude from query results. These exclusions will
be overridden if a tag is explicitly mentioned in a query.
Add function notmuch_query_count_threads() to get the number of threads
matching a search. This is done by performing a search and figuring out the
number of unique thread IDs in the matching messages, a significantly
heavier operation than notmuch_query_count_messages().
Signed-off-by: Jani Nikula <jani@nikula.org>
Previously, the functions notmuch_database_find_message() and
notmuch_database_find_message_by_filename() functions did not properly
report error condition to the library user.
For more information, read the thread on the notmuch mailing list
starting with my mail "id:871uv2unfd.fsf@gmail.com"
Make these functions accept a pointer to 'notmuch_message_t' as argument
and return notmuch_status_t which may be used to check for any error
condition.
restore: Modify for the new notmuch_database_find_message()
new: Modify for the new notmuch_database_find_message_by_filename()
State up front that these functions may add a filename to an existing
message or remove only a filename (and not the message), respectively.
Previously, this key information was buried in return value
documentation or in "notes", which made it seem secondary to these
functions' semantics.
notmuch_database_find_message_by_filename is mostly stolen from
notmuch_database_remove_message, so this patch also vastly simplfies
the latter using the former.
This API is also useful in its own right and will be used in a later
patch for eager maildir flag synchronization.
notmuch_database_t now keeps a nesting count and we only start a
transaction or commit for the outermost atomic section.
Introduces a new error, NOTMUCH_STATUS_UNBALANCED_ATOMIC.
Various typo fixes in comments within the source code.
Signed-off-by: Pieter Praet <pieter@praet.org>
Edited-by: Carl Worth <cworth@cworth.org> Restricted to just
source-code comments, (and fixed fix of "descriptios" to "descriptors"
rather than "descriptions").
This is to prevent notmuch from destroying any information the user
has encoded as flags in the maildir filename. Tests are also added to
the test suite to verify the documented behavior.
Some people use notmuch with non-maildir files, (for example, email
messages in MH format, or else cool things like using sluk[*] to suck
down feeds into a format that notmuch can index).
To better support uses like that, don't do any renaming for files that
are not in a directory named either "new" or "cur".
[*] https://github.com/krl/sluk/
I had originally hoped for better semantics, such as doing nothing in
non-maildir directories, and preserving unknown maildir flags that
happen to be present.
We could still do those things, of course, but for now, remove them
from the documentation since the implementation does not do these
things yet.
Previously the documentation of notmuch_message_maildir_flags_to_tags
suggested that the presence of a flag would cause tags to be added,
(or in the case of "unread", removed). But the case of absent maildir
flags was not explicitly described.
What we actually want, is that for supported flags, the absence of the
flag in all messages causes the corresponding tag to be removed,
(or in the case of "unread", added). So document that explicitly.
This is the case recently added to the test suite as a failing test,
(so we'll need to do bug fixing before the documentation is honest
here).
This augments the existing notmuch_message_get_filename by allowing
the caller access to all filenames in the case of multiple files for a
single message.
To support this, we split the iterator (notmuch_filenames_t) away from
the list storage (notmuch_filename_list_t) where previously these were
a single object (notmuch_filenames_t). Then, whenever the user asks
for a file or filename, the message object lazily creates a complete
notmuch_filename_list_t and then:
For notmuch_message_get_filename, returns the first filename
in the list.
For notmuch_message_get_filenames, creates and returns a new
iterator for the filename list.
This rather ugly hack was recently obviated by the removal of the
notmuch_database_set_maildir_sync function. Now, clients must make
explicit calls to do any syncrhonization between maildir flags and
tags. So the library no longer needs to worry about doing inconsistent
synchronization while a message is only partially added.
Instead of having an API for setting a library-wide flag for
synchronization (notmuch_database_set_maildir_sync) we instead
implement maildir synchronization with two new library functions:
notmuch_message_maildir_flags_to_tags
and notmuch_message_tags_to_maildir_flags
These functions are nicely documented here, (though the implementation
does not quite match the documentation yet---as plainly evidenced by
the current results of the test suite).
This adds group [maildir] and key 'synchronize_flags' to the
configuration file. Its value enables (true) or diables (false) the
synchronization between notmuch tags and maildir flags. By default,
the synchronization is disabled.
This patch allows bi-directional synchronization between maildir
flags and certain tags. The flag-to-tag mapping is defined by flag2tag
array.
The synchronization works this way:
1) Whenever notmuch new is executed, the following happens:
o New messages are tagged with configured new_tags.
o For new or renamed messages with maildir info present in the file
name, the tags defined in flag2tag are either added or removed
depending on the flags from the file name.
2) Whenever notmuch tag (or notmuch restore) is executed, a new set of
flags based on the tags is constructed for every message and a new
file name is prepared based on the old file name but with the new
flags. If the flags differs and the old message was in 'new'
directory then this is replaced with 'cur' in the new file name. If
the new and old file names differ, the file is renamed and notmuch
database is updated accordingly.
The rename happens before the database is updated. In case of crash
between rename and database update, the next run of notmuch new
brings the database in sync with the mail store again.
Previously, if the underlying search_messages hit an exception and returned
NULL, this function would ignore that and return a non-NULL, (but empty)
threads object. Fix this to properly propagate the error.
Our current approach is for top-level entry poitns in the library
to have try/catch blocks that catch any Xapian exception and print
a message. Add a few missing blocks and fix up the documentation.
Previously, we always sorted the returned results by some string value,
(newest-to-oldest by default), however in some cases (as when applying
tags to a search result) we are not interested in any special order.
This introduces a NOTMUCH_SORT_UNSORTED value that does just that. It is
not used at the moment anywhere in the code.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Reviewed-by: Carl Worth <cworth@cworth.org>:
The original proposal for having different open modes used the name
WRITABLE. I didn't like that name, (easy to misspell as WRITEABLE even
for native English speakers). So we renamed it to READ_WRITE
immediately, but apparently some of the documentation held the old
name for a while.
This seems like a generally useful thing to support, (but the previous
support through an empty string was not convenient for some users,
(such as the command-line client).
For the case of adding a file that already exist, (with the same
filename). In this case, nothing will happen to the database, but
that wasn't clear before.
Commit cd467caf renamed notmuch_query_search to notmuch_query_search_messages.
Commit 1ba3d46f created notmuch_query_search_threads. We better keep the docs
of notmuch_query_create consistent with those changes.
Signed-off-by: Fernando Carrijo <fcarrijo@yahoo.com.br>
Edited-by: Carl Worth to explicitly list the full name of each
function being referenced.
We rename 'has_more' to 'valid' so that it can function whether
iterating in a forward or reverse direction. We also rename
'advance' to 'move_to_next' to setup parallel naming with
the proposed functions 'move_to_first', 'move_to_last', and
'move_to_previous'.
Previously we were printing a number of messages upgraded so far. The
original motivation for this was to accurately reflect the fact that
there are two passes, (so each message is processed twice and it's not
accurate to represent with a single count). But as it turns out, the
second pass takes zero time (relatively speaking) so we're still not
accounting for it.
If nothing else, the percentage-based reporting makes for a cleaner
API for the progress_notify function.
The recent support for renames in the database is our first time
(since notmuch has had more than a single user) that we have a
database format change. To support smooth upgrades we now encode a
database format version number in the Xapian metadata.
Going forward notmuch will emit a warning if used to read from a
database with a newer version than it natively supports, and will
refuse to write to a database with a newer version.
The library also provides functions to query the database format
version:
notmuch_database_get_version
to ask if notmuch wants a newer version than that:
notmuch_database_needs_upgrade
and a function to actually perform that upgrade:
notmuch_database_upgrade
Previously we had NOTMUCH_DATABASE_MODE_READ_ONLY but
NOTMUCH_STATUS_READONLY_DATABASE which was ugly and confusing. Rename
the latter to NOTMUCH_STATUS_READ_ONLY_DATABASE for consistency.
Previously, many checks were deep in the library just before a cast
operation. These have now been replaced with internal errors and new
checks have instead been added at the beginning of all top-levelentry
points requiring a read-write database.
The new checks now also use a single function for checking and
printing the error message. This will give us a convenient location to
extend the check, (such as based on database version as well).
