debian/notmuch
1
0
Fork 0
mirror of https://git.notmuchmail.org/git/notmuch synced 2024-11-24 11:58:10 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
notmuch/lib/directory.cc

266 lines
7.3 KiB
C++
Raw Normal View History

lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
/* directory.cc - Results of directory-based searches from a notmuch database
*
* Copyright © 2009 Carl Worth
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see http://www.gnu.org/licenses/ .
*
* Author: Carl Worth <cworth@cworth.org>
*/
#include "notmuch-private.h"
#include "database-private.h"
/* Create an iterator to iterate over the basenames of files (or
* directories) that all share a common parent directory.
*
* The code here is general enough to be reused for any case of
* iterating over the non-prefixed portion of terms sharing a common
* prefix.
*/
lib: Add some missing static qualifiers. These various functions and data are all used only locally, so should be marked static. Ensuring we get these right will avoid us accidentally leaking unintended symbols through the library interface.
2010-11-02 05:58:43 +01:00
static notmuch_filenames_t *
lib: Add new implementation of notmuch_filenames_t The new implementation is simply a talloc-based list of strings. The former support (a list of database terms with a common prefix) is implemented by simply pre-iterating over the terms and populating the list. This should provide no performance disadvantage as callers of thigns like notmuch_directory_get_child_files are very likely to always iterate over all filenames anyway. This new implementation of notmuch_filenames_t is in preparation for adding API to query all of the filenames for a single message.
2010-11-11 08:26:31 +01:00
_create_filenames_for_terms_with_prefix (void *ctx,
notmuch_database_t *notmuch,
const char *prefix)
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
{
Implement an internal generic string list and use it. This replaces the guts of the filename list and tag list, making those interfaces simple iterators over the generic string list. The directory, message filename, and tags-related code now build generic string lists and then wraps them in specific iterators. The real wins come in later patches, when we use these for even more generic functionality. As a nice side-effect, this also eliminates the annoying dependency on GList in the tag list.
2010-12-09 01:26:05 +01:00
notmuch_string_list_t *filename_list;
lib: Add new implementation of notmuch_filenames_t The new implementation is simply a talloc-based list of strings. The former support (a list of database terms with a common prefix) is implemented by simply pre-iterating over the terms and populating the list. This should provide no performance disadvantage as callers of thigns like notmuch_directory_get_child_files are very likely to always iterate over all filenames anyway. This new implementation of notmuch_filenames_t is in preparation for adding API to query all of the filenames for a single message.
2010-11-11 08:26:31 +01:00
Xapian::TermIterator i, end;
int prefix_len = strlen (prefix);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
Implement an internal generic string list and use it. This replaces the guts of the filename list and tag list, making those interfaces simple iterators over the generic string list. The directory, message filename, and tags-related code now build generic string lists and then wraps them in specific iterators. The real wins come in later patches, when we use these for even more generic functionality. As a nice side-effect, this also eliminates the annoying dependency on GList in the tag list.
2010-12-09 01:26:05 +01:00
filename_list = _notmuch_string_list_create (ctx);
lib: Add new, public notmuch_message_get_filenames This augments the existing notmuch_message_get_filename by allowing the caller access to all filenames in the case of multiple files for a single message. To support this, we split the iterator (notmuch_filenames_t) away from the list storage (notmuch_filename_list_t) where previously these were a single object (notmuch_filenames_t). Then, whenever the user asks for a file or filename, the message object lazily creates a complete notmuch_filename_list_t and then: For notmuch_message_get_filename, returns the first filename in the list. For notmuch_message_get_filenames, creates and returns a new iterator for the filename list.
2010-11-11 09:07:24 +01:00
if (unlikely (filename_list == NULL))
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
return NULL;
lib: Add new implementation of notmuch_filenames_t The new implementation is simply a talloc-based list of strings. The former support (a list of database terms with a common prefix) is implemented by simply pre-iterating over the terms and populating the list. This should provide no performance disadvantage as callers of thigns like notmuch_directory_get_child_files are very likely to always iterate over all filenames anyway. This new implementation of notmuch_filenames_t is in preparation for adding API to query all of the filenames for a single message.
2010-11-11 08:26:31 +01:00
end = notmuch->xapian_db->allterms_end (prefix);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
lib: Add new implementation of notmuch_filenames_t The new implementation is simply a talloc-based list of strings. The former support (a list of database terms with a common prefix) is implemented by simply pre-iterating over the terms and populating the list. This should provide no performance disadvantage as callers of thigns like notmuch_directory_get_child_files are very likely to always iterate over all filenames anyway. This new implementation of notmuch_filenames_t is in preparation for adding API to query all of the filenames for a single message.
2010-11-11 08:26:31 +01:00
for (i = notmuch->xapian_db->allterms_begin (prefix); i != end; i++)
{
std::string term = *i;
lib: Treat NULL as a valid (and empty) notmuch_filenames_t iterator. This will be convenient to avoid some special-casing in higher-level code.
2010-01-06 23:35:11 +01:00
Implement an internal generic string list and use it. This replaces the guts of the filename list and tag list, making those interfaces simple iterators over the generic string list. The directory, message filename, and tags-related code now build generic string lists and then wraps them in specific iterators. The real wins come in later patches, when we use these for even more generic functionality. As a nice side-effect, this also eliminates the annoying dependency on GList in the tag list.
2010-12-09 01:26:05 +01:00
_notmuch_string_list_append (filename_list, term.c_str () + prefix_len);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
}
lib: Add new, public notmuch_message_get_filenames This augments the existing notmuch_message_get_filename by allowing the caller access to all filenames in the case of multiple files for a single message. To support this, we split the iterator (notmuch_filenames_t) away from the list storage (notmuch_filename_list_t) where previously these were a single object (notmuch_filenames_t). Then, whenever the user asks for a file or filename, the message object lazily creates a complete notmuch_filename_list_t and then: For notmuch_message_get_filename, returns the first filename in the list. For notmuch_message_get_filenames, creates and returns a new iterator for the filename list.
2010-11-11 09:07:24 +01:00
return _notmuch_filenames_create (ctx, filename_list);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
}
struct _notmuch_directory {
notmuch_database_t *notmuch;
Xapian::docid document_id;
Xapian::Document doc;
time_t mtime;
};
/* We end up having to call the destructor explicitly because we had
* to use "placement new" in order to initialize C++ objects within a
* block that we allocated with talloc. So C++ is making talloc
* slightly less simple to use, (we wouldn't need
* talloc_set_destructor at all otherwise).
*/
static int
_notmuch_directory_destructor (notmuch_directory_t *directory)
{
directory->doc.~Document ();
return 0;
}
static notmuch_private_status_t
find_directory_document (notmuch_database_t *notmuch,
const char *db_path,
Xapian::Document *document)
{
notmuch_private_status_t status;
Xapian::docid doc_id;
status = _notmuch_database_find_unique_doc_id (notmuch, "directory",
db_path, &doc_id);
if (status) {
*document = Xapian::Document ();
return status;
}
*document = notmuch->xapian_db->get_document (doc_id);
return NOTMUCH_PRIVATE_STATUS_SUCCESS;
}
notmuch_directory_t *
_notmuch_directory_create (notmuch_database_t *notmuch,
const char *path,
notmuch_status_t *status_ret)
{
Xapian::WritableDatabase *db;
notmuch_directory_t *directory;
notmuch_private_status_t private_status;
const char *db_path;
*status_ret = NOTMUCH_STATUS_SUCCESS;
path = _notmuch_database_relative_path (notmuch, path);
lib: Consolidate checks for read-only database. Previously, many checks were deep in the library just before a cast operation. These have now been replaced with internal errors and new checks have instead been added at the beginning of all top-levelentry points requiring a read-write database. The new checks now also use a single function for checking and printing the error message. This will give us a convenient location to extend the check, (such as based on database version as well).
2010-01-07 19:19:44 +01:00
if (notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY)
INTERNAL_ERROR ("Failure to ensure database is writable");
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
db = static_cast <Xapian::WritableDatabase *> (notmuch->xapian_db);
directory = talloc (notmuch, notmuch_directory_t);
if (unlikely (directory == NULL))
return NULL;
directory->notmuch = notmuch;
/* "placement new"---not actually allocating memory */
new (&directory->doc) Xapian::Document;
talloc_set_destructor (directory, _notmuch_directory_destructor);
db_path = _notmuch_database_get_directory_db_path (path);
try {
Xapian::TermIterator i, end;
private_status = find_directory_document (notmuch, db_path,
&directory->doc);
directory->document_id = directory->doc.get_docid ();
if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
void *local = talloc_new (directory);
const char *parent, *basename;
Xapian::docid parent_id;
char *term = talloc_asprintf (local, "%s%s",
_find_prefix ("directory"), db_path);
lib: Add non-content terms with a WDF value of 0. The WDF is the "within-document frequency" value for a particular term. It's intended to provide an indication of how frequent a term is within a document, (for use in computing relevance). Xapian's term generator already computes WDF values when we use that, (which we do for indexing all mail content). We don't use the term generator when adding single terms for things that don't actually appear in the mail document, (such as tags, the filename, etc.). In this case, the WDF value for these terms doesn't matter much. But Xapian's flint backend can be more efficient with changes to terms that don't affect the document "length". So there's a performance advantage for manipulating tags (with the flint backend) if the WDF of these terms is 0.
2010-01-09 20:18:27 +01:00
directory->doc.add_term (term, 0);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
directory->doc.set_data (path);
_notmuch_database_split_path (local, path, &parent, &basename);
_notmuch_database_find_directory_id (notmuch, parent, &parent_id);
if (basename) {
term = talloc_asprintf (local, "%s%u:%s",
_find_prefix ("directory-direntry"),
parent_id, basename);
lib: Add non-content terms with a WDF value of 0. The WDF is the "within-document frequency" value for a particular term. It's intended to provide an indication of how frequent a term is within a document, (for use in computing relevance). Xapian's term generator already computes WDF values when we use that, (which we do for indexing all mail content). We don't use the term generator when adding single terms for things that don't actually appear in the mail document, (such as tags, the filename, etc.). In this case, the WDF value for these terms doesn't matter much. But Xapian's flint backend can be more efficient with changes to terms that don't affect the document "length". So there's a performance advantage for manipulating tags (with the flint backend) if the WDF of these terms is 0.
2010-01-09 20:18:27 +01:00
directory->doc.add_term (term, 0);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
}
directory->doc.add_value (NOTMUCH_VALUE_TIMESTAMP,
Xapian::sortable_serialise (0));
Avoid database corruption by not adding partially-constructed mail documents. Previously we were using Xapian's add_document to allocate document ID values for notmuch_message_t objects. This had the drawback of adding a partially constructed mail document to the database. If notmuch was subsequently interrupted before fully populating this document, then later runs would be quite confused when seeing the partial documents. There are reports from the wild of people hitting internal errors of the form "Message ... has no thread ID" for example, (which is currently an unrecoverable error). We fix this by manually allocating document IDs without adding documents. With this change, we never call Xapian's add_document method, but only replace_document with either the current document ID of a message or a new one that we have allocated.
2010-06-04 19:16:53 +02:00
directory->document_id = _notmuch_database_generate_doc_id (notmuch);
db->replace_document (directory->document_id, directory->doc);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
talloc_free (local);
}
directory->mtime = Xapian::sortable_unserialise (
directory->doc.get_value (NOTMUCH_VALUE_TIMESTAMP));
} catch (const Xapian::Error &error) {
fprintf (stderr,
"A Xapian exception occurred creating a directory: %s.\n",
error.get_msg().c_str());
notmuch->exception_reported = TRUE;
notmuch_directory_destroy (directory);
*status_ret = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
return NULL;
}
if (db_path != path)
free ((char *) db_path);
return directory;
}
unsigned int
_notmuch_directory_get_document_id (notmuch_directory_t *directory)
{
return directory->document_id;
}
notmuch_status_t
notmuch_directory_set_mtime (notmuch_directory_t *directory,
time_t mtime)
{
notmuch_database_t *notmuch = directory->notmuch;
Xapian::WritableDatabase *db;
lib: Consolidate checks for read-only database. Previously, many checks were deep in the library just before a cast operation. These have now been replaced with internal errors and new checks have instead been added at the beginning of all top-levelentry points requiring a read-write database. The new checks now also use a single function for checking and printing the error message. This will give us a convenient location to extend the check, (such as based on database version as well).
2010-01-07 19:19:44 +01:00
notmuch_status_t status;
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
lib: Consolidate checks for read-only database. Previously, many checks were deep in the library just before a cast operation. These have now been replaced with internal errors and new checks have instead been added at the beginning of all top-levelentry points requiring a read-write database. The new checks now also use a single function for checking and printing the error message. This will give us a convenient location to extend the check, (such as based on database version as well).
2010-01-07 19:19:44 +01:00
status = _notmuch_database_ensure_writable (notmuch);
if (status)
return status;
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
db = static_cast <Xapian::WritableDatabase *> (notmuch->xapian_db);
try {
directory->doc.add_value (NOTMUCH_VALUE_TIMESTAMP,
Xapian::sortable_serialise (mtime));
db->replace_document (directory->document_id, directory->doc);
} catch (const Xapian::Error &error) {
fprintf (stderr,
"A Xapian exception occurred setting directory mtime: %s.\n",
error.get_msg().c_str());
notmuch->exception_reported = TRUE;
return NOTMUCH_STATUS_XAPIAN_EXCEPTION;
}
return NOTMUCH_STATUS_SUCCESS;
}
time_t
notmuch_directory_get_mtime (notmuch_directory_t *directory)
{
return directory->mtime;
}
notmuch_filenames_t *
notmuch_directory_get_child_files (notmuch_directory_t *directory)
{
char *term;
notmuch_filenames_t *child_files;
term = talloc_asprintf (directory, "%s%u:",
_find_prefix ("file-direntry"),
directory->document_id);
lib: Add new implementation of notmuch_filenames_t The new implementation is simply a talloc-based list of strings. The former support (a list of database terms with a common prefix) is implemented by simply pre-iterating over the terms and populating the list. This should provide no performance disadvantage as callers of thigns like notmuch_directory_get_child_files are very likely to always iterate over all filenames anyway. This new implementation of notmuch_filenames_t is in preparation for adding API to query all of the filenames for a single message.
2010-11-11 08:26:31 +01:00
child_files = _create_filenames_for_terms_with_prefix (directory,
directory->notmuch,
term);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
talloc_free (term);
return child_files;
}
notmuch_filenames_t *
notmuch_directory_get_child_directories (notmuch_directory_t *directory)
{
char *term;
notmuch_filenames_t *child_directories;
term = talloc_asprintf (directory, "%s%u:",
_find_prefix ("directory-direntry"),
directory->document_id);
lib: Add new implementation of notmuch_filenames_t The new implementation is simply a talloc-based list of strings. The former support (a list of database terms with a common prefix) is implemented by simply pre-iterating over the terms and populating the list. This should provide no performance disadvantage as callers of thigns like notmuch_directory_get_child_files are very likely to always iterate over all filenames anyway. This new implementation of notmuch_filenames_t is in preparation for adding API to query all of the filenames for a single message.
2010-11-11 08:26:31 +01:00
child_directories = _create_filenames_for_terms_with_prefix (directory,
directory->notmuch, term);
lib: Implement new notmuch_directory_t API. This new directory ojbect provides all the infrastructure needed to detect when files or directories are deleted or renamed. There's still code needed on top of this (within "notmuch new") to actually do that detection.
2010-01-05 22:29:23 +01:00
talloc_free (term);
return child_directories;
}
void
notmuch_directory_destroy (notmuch_directory_t *directory)
{
talloc_free (directory);
}
Reference in a new issue Copy permalink