Allow query debugging to be enabled at run-time by setting the
NOTMUCH_DEBUG_QUERY environment variable to a non-empty string.
Previously, enabling query debugging required recompiling, but parsed
queries are often useful for tracking down bugs in situations where
recompiling is inconvenient.
Formerly it was possible for *message_ret to be left
uninitialized. The documentation however clearly states that "[o]n any
failure or when the message is not found, this function initializes
'*message' to NULL".
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
Formerly the xapian database object was deleted and closed in its
destructor once the object was garbage collected. Explicitly call
close() so that the database and the associated lock is released
immediately.
The comment is a courtesy of Austin Clements.
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
Add the NOTMUCH_MESSAGE_FLAG_EXCLUDED flag to
notmuch_query_search_threads. Implemented by inspecting the tags
directly in _notmuch_thread_create/_thread_add_message rather than as
a Xapian query for speed reasons.
Note notmuch_thread_get_matched_messages now returns the number of
non-excluded matching messages. This API is not totally desirable but
fixing it means breaking binary compatibility so we delay that.
Add a flag NOTMUCH_MESSAGE_FLAG_EXCLUDED which is set by
notmuch_query_search_messages for excluded messages. Also add an
option omit_excluded_messages to the search that we do not want the
excludes at all.
This exclude flag will be added to notmuch_query_search threads in the
next patch.
This fixes a bug that didn't allow to search for non-ASCII words such
parts. The code here was copied from show_text_part_content(), because
the show command already does the needed conversion when showing the
message.
Previously opening a notmuch database in read write mode that has been
locked resulted in the notmuch_database_open function executing
notmuch_database_close as a cleanup function. notmuch_database_close
failed to check whether the xapian database has in fact been created.
Add a check whether the xapian database object has actually been
created before trying to call its flush method.
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
Previously, we manually "free"d various pointers in
notmuch_database_open. Use a local talloc context instead to simplify
cleanup and eliminate various NULL pointer initializations and
conditionals.
In the error-handling paths of notmuch_database_open, we call
notmuch_database_close, which "delete"s several objects referenced by
the notmuch_database_t object. However, some of these pointers may be
uninitialized, resulting in undefined behavior. Hence, allocate the
notmuch_database_t with talloc_zero to make sure these pointers are
NULL so that "delete"ing them is harmless.
This is useful for tags like "deleted" and "spam" that people
generally want to exclude from query results. These exclusions will
be overridden if a tag is explicitly mentioned in a query.
lib/messages.c: In function ‘notmuch_messages_move_to_next’:
lib/messages.c:131:2: warning: ISO C forbids ‘return’ with expression, in function returning void [-pedantic]
Signed-off-by: Jani Nikula <jani@nikula.org>
As reported in
id:"CAEbOPGyuHnz4BPtDutnTPUHcP3eYcRCRkXhYoJR43RUMw671+g@mail.gmail.com"
sometimes gmime tries to access a NULL pointer, e.g. g_mime_iconv_open()
tries to access iconv_cache that is NULL if g_mime_init() is not called.
This causes notmuch to segfault when calling gmime functions.
Calling g_mime_init() initializes iconv_cache and others variables needed
by gmime, making sure they are initialized when notmuch calls gmime
functions.
Test marked fix by db.
It appears to be an oversight that encrypted parts were indexed
previously. The terms generated from encrypted parts are meaningless
and do nothing but add bloat to the database. It is not worth
indexing the encrypted content, just as it's not worth indexing the
signatures in signed parts.
Commit 567bcbc2 introduced two new values for each message (content of the
"From" and "Subject" headers), but the comments about the database schema had
not been updated accordingly.
For some reason, on my machine, the link is picking up
/usr/lib/libutil.so instead of util/libutil.a. This causes there to be
undefined symbols in libnotmuch, making it unuseable. This patch causes
the link to fail instead.
Add function notmuch_query_count_threads() to get the number of threads
matching a search. This is done by performing a search and figuring out the
number of unique thread IDs in the matching messages, a significantly
heavier operation than notmuch_query_count_messages().
Signed-off-by: Jani Nikula <jani@nikula.org>
This is a rebase and cleanup of Istvan Marko's patch from
id:m3pqnj2j7a.fsf@zsu.kismala.com
Search retrieves these headers for every message in the search
results. Previously, this required opening and parsing every message
file. Storing them directly in the database significantly reduces IO
and computation, speeding up search by between 50% and 10X.
Taking full advantage of this requires a database rebuild, but it will
fall back to the old behavior for messages that do not have headers
stored in the database.
Apparently the method was renamed in Xapian 1.1.0 but the old method
name will stay around for a while. It seems better to stick with the
old name to make notmuch compile with older versions of Xapian, at
least for now.
We keep the lib/xutil.c version. As a consequence, also factor out
_internal_error and associated macros. It might be overkill to make a
new file error_util.c for this, but _internal_error does not really
belong in database.cc.
Based on discussions with amdragon, tschwinge, and others on IRC, I concluded that
1) symbol versioning was probably overkill for libnotmuch
2) It was also probably GNU ld specific
3) Most importantly, nobody could tell me on short notice how exactly it works.
So since the change to the notmuch_database_find_message breaks the
previous ABI, we need to bump the SONAME.
Previously, the functions notmuch_database_find_message() and
notmuch_database_find_message_by_filename() functions did not properly
report error condition to the library user.
For more information, read the thread on the notmuch mailing list
starting with my mail "id:871uv2unfd.fsf@gmail.com"
Make these functions accept a pointer to 'notmuch_message_t' as argument
and return notmuch_status_t which may be used to check for any error
condition.
restore: Modify for the new notmuch_database_find_message()
new: Modify for the new notmuch_database_find_message_by_filename()
State up front that these functions may add a filename to an existing
message or remove only a filename (and not the message), respectively.
Previously, this key information was buried in return value
documentation or in "notes", which made it seem secondary to these
functions' semantics.
Adding a message may involve changes to multiple database documents,
and thus needs to be done in a transaction. This makes add_message
(and, I think, the whole library) atomicity-safe: library callers only
needs to use atomic sections if they needs atomicity across multiple
library calls.
notmuch_database_find_message_by_filename is mostly stolen from
notmuch_database_remove_message, so this patch also vastly simplfies
the latter using the former.
This API is also useful in its own right and will be used in a later
patch for eager maildir flag synchronization.
Previously, notmuch_database_remove_message would remove the message
file name, sync the change to the message document, re-find the
message document, and then delete it if there were no more file names.
An interruption after sync'ing would result in a file-name-less,
permanently un-removable zombie message that would produce errors and
odd results in searches. We could wrap this in an atomic section, but
it's much simpler to eliminate the round-about approach and just
delete the message document instead of sync'ing it if we removed the
last filename.
notmuch_database_t now keeps a nesting count and we only start a
transaction or commit for the outermost atomic section.
Introduces a new error, NOTMUCH_STATUS_UNBALANCED_ATOMIC.
Previously, this function would synchronize the folder list even if
removing the file name failed. Now it returns immediately if removing
the file name fails.
If the configure script detects missing getline and/or getdelim
symbols, then notmuch will use it's own versions. This patch, based on
id:"87k49v12i5.fsf@pc44es141.cs.uni-magdeburg.de" by Matthias
Guedemann, adds the symbols to notmuch.sym as well so they are
properly exported from the library.
OpenBSD nm apparently doesn't support --defined.
The awk condition is based on the assumption that all defined symbols
have some hex number in the first column.
Thanks to Matthias Guedemann reporting the problem, and an earlier
version of this patch.
Unfortunately Robin Green's patch 52e4dedf9a was lost when I created
gen-version-script.sh. This merges his changes manually into that
script. It turns out tabs seem not needed in version script
files, so I simplified a bit and removed the printf.
Thanks to Alexander Botero-Lowry for help and testing.
If the notmuch.sym target does not explicitly depend on $(libnotmuch_modules),
gen-version-script.sh may be run before all the .o files are created, for
example when doing a parallel build on a machine with many cores.
Conflicts:
lib/Makefile.local
The conflicts are from three kinds of commits not merged into release:
- typo fixes
- removal of debug output
- fix for CLEAN rule
That were never merged into the release branch.
The lack of such exporting seems to cause problems catching
exceptions, as suggested by
http://gcc.gnu.org/wiki/Visibility
This manifested in the symbol-hiding test failing when notmuch was
compile with gcc 4.4.5. On i386, this further manifested as notmuch
new failing to run (crashing with an uncaught exception on first run).
Add removal of all ZXFOLDER terms to removal of all XFOLDER terms for
each message filename removal.
The existing filename-list reindexing will put all the needed terms
back in. Test search-folder-coherence now passes.
Signed-off-by:Mark Anderson <ma.skies@gmail.com>
(cherry picked from commit 8a856e5c38)
