Previously we were using Xapian's add_document to allocate document ID
values for notmuch_message_t objects. This had the drawback of adding
a partially constructed mail document to the database. If notmuch was
subsequently interrupted before fully populating this document, then
later runs would be quite confused when seeing the partial documents.
There are reports from the wild of people hitting internal errors of
the form "Message ... has no thread ID" for example, (which is
currently an unrecoverable error).
We fix this by manually allocating document IDs without adding
documents. With this change, we never call Xapian's add_document
method, but only replace_document with either the current document ID
of a message or a new one that we have allocated.
Admittedly, an author name ending in ',' guarantees this is spam, and
indeed this was triggered by a spam email, but that doesn't mean we
shouldn't handle this case correctly.
We now check that there is actually a component of the name (presumably
the first name) after the comma in the author name.
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
Just before releasing 0.3 we received reports of crashes that were
bisected to the commit adding thread-author moving. Sure enough,
valgrind pointed to buffer overruns in _thread_move_matched_author.
Rather than trying to make sense of all the by strncpy, strchr, +1,
and +2 of that code, I reimplemented thread-author ordering with a
pair of hash tables and an array.
Valgrind is at least happy now on the test cases it was complaining
about previously.
With this patch the Received: header becomes special in the way
we treat headers - this is the only header for which we concatenate
all the instances we find (instead of just returning the first one).
This will be used in the From guessing code for replies as we need to
be able to walk ALL of the Received: headers in a message to have a
good chance to guess which mailbox this email was delivered to.
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
This patch only addresses the typical Outlook/Exchange case
where we have "Last, First" <first.last@company.com> or
"Last, First MI" <first.mi.last@company.com>.
In the future we should be more fexible as to the formats
we recognize, but for now we address this one as it is the
Exchange default setting and therefore the most common one.
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
When displaying threads as result of a search it makes sense to list those
authors first who match the search. The matching authors are separated from the
non-matching ones with a '|' instead of a ','
Imagine the default "+inbox" query. Those mails in the thread that
match the query are actually "new" (whatever that means). And some
people seem to think that it would be much better to see those author
names first. For example, imagine a long and drawn out thread that once
was started by me; you have long read the older part of the thread and
removed the inbox tag. Whenever a new email comes in on this thread,
prior to this patch the author column in the search display will first show
"Dirk Hohndel" - I think it should first show the actual author(s) of the new
mail(s).
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
message->authors contains the author's name (as we want to print it)
get / set methods are declared in notmuch-private.h
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
Our current approach is for top-level entry poitns in the library
to have try/catch blocks that catch any Xapian exception and print
a message. Add a few missing blocks and fix up the documentation.
Sebastian Spaeth reported [*] a segfault within libnotmuch when
running notmuch operations while an asyncronous offlineimap job had
removed some files from the mail store. Avoid this by handling all
cases where notmuch_message_get_header could return NULL.
[*] See message id:87d3xqti3o.fsf@SSpaeth.de on notmuch@notmuchmail.org
Previously, we always sorted the returned results by some string value,
(newest-to-oldest by default), however in some cases (as when applying
tags to a search result) we are not interested in any special order.
This introduces a NOTMUCH_SORT_UNSORTED value that does just that. It is
not used at the moment anywhere in the code.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The thread-naming feature depends on the matched messages being passed
down in a precise order, (the order of the top-level search). We fix
the feature by passing that sort order down.
We know that matched messages are always added in order, so we can
always just grab the subject from the first message. This is the same
approach that was used previously in _thread_add_message. That is, the
recent feature of renaming a thread based on the subject of the
"first" matched message is as simple as moving the subject assignment
from _thread_add_message to _thread_add_matched_message.
At the moment all threads are named based on the name of the first message
in the thread. However, this can cause problems if people either start
new threads by replying-all (as unfortunately, many out there do) or
change the subject of their mails to reflect a shift in a thread on a
list.
This patch names threads based on (a) matches for the query, and (b) the
search order. If the search order is oldest-first (as in the default
inbox) it chooses the oldest matching message as the subject. If the
search order is newest-first it chooses the newest one.
Reply prefixes ("Re: ", "Aw: ", "Sv: ", "Vs: ") are ignored
(case-insensitively) so a Re: won't change the subject.
Note that this adds a "sort" argument to _notmuch_thread_create and
_thread_add_matched_message, so that when constructing the thread we can
be aware of the sort order.
Signed-off-by: Jesse Rosenthal <jrosenthal@jhu.edu>
When constructing a thread, we usually run a nested query to find all
messages in the thread that match the original search string. However,
we need to have special-case handling of an original search string of
"*" now that that is a supported means of specifying all messages.
The special-case ends up bein quite simple---we do less work, (just
skipping the nested search since we know that all messages must
match). I had been wanting to write this identical code to more
efficiently handle "notmuch search thread:<foo>" which was previously
running two identical searches. So that case is now more efficient as
well.
This encodes the library version into the library, where the linking
binary can pick it up, and the linker can even enforce mismatches in
the minor release, (such as linking a binary against version 1.2 and
then attempting to run it against version 1.1).
I'm not sure which system Aaron used, but on the machine I have access
to, (Darwin 8.11.0), the -shared and -dylib_install_name options are
not recognized. Instead I use -dynamic_lib and -install_name as
documented here:
http://www.finkproject.org/doc/porting/shared.php
This patch adds a configure check for OS X (actually Darwin),
and sets up the Makefiles to build a proper shared library on
that platform.
Signed-off-by: Aaron Ecay <aaronecay@gmail.com>
notmuch previously unconditionally checked mime parts for various
properties, but not for NULL, which is the case if libgmime encounters
an empty mime part.
Upon encounter of an empty mime part, the following is printed to
stderr (the second line due to my patch):
(process:17197): gmime-CRITICAL **: g_mime_message_get_mime_part: assertion `GMIME_IS_MESSAGE (message)' failed
Warning: Not indexing empty mime part.
This is probably a bug that should get addressed in libgmime, but for
not, my patch is an acceptable workaround.
Signed-off-by: martin f. krafft <madduck@madduck.net>
Reviewed-by: Carl Worth <cworth@cworth.org>:
The original proposal for having different open modes used the name
WRITABLE. I didn't like that name, (easy to misspell as WRITEABLE even
for native English speakers). So we renamed it to READ_WRITE
immediately, but apparently some of the documentation held the old
name for a while.
Previously, we were only adding the reference terms for cases where
the referenced message did not yet exist in the database. For thread
presentation, it's useful to have the connection information provided
by the references, even when the messages are present. So add this
term unconditionally.
This function was recently modified, (to include a metadata lookup for
a message's thread ID before looking for parent/child thread IDs), but
the documentation wasn't updated. Fix that.
There are two primary cases in this function, (the message exists in
the database or it does not). Previously the code for these two cases
was split and intermingled with goto-spaghetti connections.
This allows us to thread messages even when we receive them out of
order, or never receive the root.
The thread ids for messages that aren't present but are referred to are
stored as metadata in the database and then retrieved if we ever get
that message.
When determining the thread id for a message we also check for this
metadata so that we can thread descendants of a message together before
we receive it.
Edited by Carl Worth <cworth@cworth.org>: Split this portion of the
commit from the earlier-applied portion adding test cases.
This seems like a generally useful thing to support, (but the previous
support through an empty string was not convenient for some users,
(such as the command-line client).
fix notmuch_message_file_get_header to always return the first instance
of the header you are looking for
Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
With the original quiet function, there's an actual purpose (hiding
excessively long compiler command lines so that warnings and errors
from the compiler can be seen).
But with things like quiet_symlink there's nothing quieter. In fact
"SYMLINK" is longer than "ln -sf". So all this is doing is hiding the
actual command from the user for no real benefit.
The only actual reason we implemented the quiet_* functions was to be
able to neatly right-align the command name and left-align the arguments.
Let's give up on that, and just left-align everything, simplifying the
Makefiles considerably. Now, the only instances of a captialized command
name in the output is if there's some actually shortening of the command
itself.
We add a magic line to the beginning of each Makefile.local file to
help the editor know that it should use makefile mode for editing the
file, (even though the filename isn't exactly "Makefile").
Edited-by: Carl Worth <cworth@cworth.org>: Expand treatment from
emacs/Makefile.local to each instance of Makefile.local.
The idea here is to allow a new user of notmuch to be able to run
notmuch immediately after compiling, (without having to install
the shared library first). This also ensures that the test suite
tests the locally compiled library, and not whatever installled
version of the library the dynamic linker happens to find.
When I wanted to create a debian package from the current master, make
install failed because of non-existent include directory. This patch
fixes this minor issue.
We had a fairly ugly violation of modularity with the top-level
Makefile.local isntalling everything, (even when the build commands
for the library were down in lib/Makefile.local).
For the case of adding a file that already exist, (with the same
filename). In this case, nothing will happen to the database, but
that wasn't clear before.
We were previously maintaining two lists of the child Makefile
fragments---one for the includes and another for the dependencies. So,
of course, they drifted and the dependency list wasn't up to date.
We fix this by adding a single subdirs variable, and then using GNU
Makefile substitution to generate both the include and the dependency
lists.
Some side effect of this change caused the '=' assignment of the dir
variable to not work anymore. I'm not sure why that is, but using ':='
makes sense here and fixes the problem.
Commit cd467caf renamed notmuch_query_search to notmuch_query_search_messages.
Commit 1ba3d46f created notmuch_query_search_threads. We better keep the docs
of notmuch_query_create consistent with those changes.
Signed-off-by: Fernando Carrijo <fcarrijo@yahoo.com.br>
Edited-by: Carl Worth to explicitly list the full name of each
function being referenced.
We rename 'has_more' to 'valid' so that it can function whether
iterating in a forward or reverse direction. We also rename
'advance' to 'move_to_next' to setup parallel naming with
the proposed functions 'move_to_first', 'move_to_last', and
'move_to_previous'.
Thanks to Michal Sojka <sojkam1@fel.cvut.cz> for pointing out the
correct fix, which I verified in the freely-available WG14/N1124 draft
(from the C99 working group) which is available here:
http://www.open-std.org/JTC1/SC22/wg14/www/docs/n1124.pdf
The sequential identifiers have the advantage of being guaranteed to
be unique (until we overflow a 64-bit unsigned integer), and also take
up half as much space in the "notmuch search" output (16 columns
rather than 32).
This change also has the side effect of fixing a bug where notmuch
could block on /dev/random at startup (waiting for some entropy to
appear). This bug was hit hard by the test suite, (which could easily
exhaust the available entropy on common systems---resulting in large
delays of the test suite).
If we had external users of this filter then they might expect some of
these macros to exist. But since this is just internal, that's just
unneeded noise.
With modern MIME attachments, we're already avoiding indexing the
attachments. But for old-school uuencoded data in the mail, we have
been directly indexing the encoded data as terms, (which is not useful
at all---nobody will ever ytry to search based on the seemingly random
uuencoded data).
Additionally, indexing a modestly large uuencoded file seems to make
Xapian go insane, (consuming *lots* of memory).
We fix both problems by detecting uuencoded content and not performing
any indexing of it.
Previously we were printing a number of messages upgraded so far. The
original motivation for this was to accurately reflect the fact that
there are two passes, (so each message is processed twice and it's not
accurate to represent with a single count). But as it turns out, the
second pass takes zero time (relatively speaking) so we're still not
accounting for it.
If nothing else, the percentage-based reporting makes for a cleaner
API for the progress_notify function.
The WDF is the "within-document frequency" value for a particular
term. It's intended to provide an indication of how frequent a term is
within a document, (for use in computing relevance). Xapian's term
generator already computes WDF values when we use that, (which we do
for indexing all mail content).
We don't use the term generator when adding single terms for things
that don't actually appear in the mail document, (such as tags, the
filename, etc.). In this case, the WDF value for these terms doesn't
matter much.
But Xapian's flint backend can be more efficient with changes to terms
that don't affect the document "length". So there's a performance
advantage for manipulating tags (with the flint backend) if the WDF of
these terms is 0.
All notmuch searches currently sort by value (either date or message
ID) so it's just wasted effort for Xapian to compute relevance values
for each result. We now explicitly tell Xapian that we're uninterested
in the relevance values.
The first phase copies data from the old format to the new format
without deleting anything. This allows an old notmuch to still use the
database if the upgrade process gets interrupted. The second phase
performs the deletion (after updating the database version number). If
the second phase is interrupted, there will be some unused data in the
database, but it shouldn't cause any actual harm.
The recent support for renames in the database is our first time
(since notmuch has had more than a single user) that we have a
database format change. To support smooth upgrades we now encode a
database format version number in the Xapian metadata.
Going forward notmuch will emit a warning if used to read from a
database with a newer version than it natively supports, and will
refuse to write to a database with a newer version.
The library also provides functions to query the database format
version:
notmuch_database_get_version
to ask if notmuch wants a newer version than that:
notmuch_database_needs_upgrade
and a function to actually perform that upgrade:
notmuch_database_upgrade
Previously we had NOTMUCH_DATABASE_MODE_READ_ONLY but
NOTMUCH_STATUS_READONLY_DATABASE which was ugly and confusing. Rename
the latter to NOTMUCH_STATUS_READ_ONLY_DATABASE for consistency.
Previously, many checks were deep in the library just before a cast
operation. These have now been replaced with internal errors and new
checks have instead been added at the beginning of all top-levelentry
points requiring a read-write database.
The new checks now also use a single function for checking and
printing the error message. This will give us a convenient location to
extend the check, (such as based on database version as well).
The original wording made it sound like this function was just doing
some string manipulation. But this function actually creates new
directory documents as a side effect. So make that explicit in its
documentation.
When a notmuch database is upgraded to the new database format, (to
support file rename and deletion), any message documents corresponding
to deleted files will not currently be upgraded. This means that a
search matching these documents will find no filenames in the expected
place.
Go ahead and return the filename as originally stored, (rather than
aborting with an internal error), in this case.
Similar to the return value of notmuch_database_add_message, we now
enhance the return value of notmuch_database_remove_message to
indicate whether the message document was entirely removed (SUCCESS)
or whether only this filename was removed and the document exists
under other filenamed (DUPLICATE_MESSAGE_ID).
Previously, adding a filename with the same message ID as an existing
message would do nothing. But we recently fixed this to instead add
the new filename to the existing message document. So update the
documentation to match now.
In the presentation we often omit citations and signatures, but this
is not content that should be omitted from the index, (especially
when the citation detection is wrong---see cases where a line
beginning with "From" is corrupted to ">From" by mail processing
tools).
This new directory ojbect provides all the infrastructure needed to
detect when files or directories are deleted or renamed. There's still
code needed on top of this (within "notmuch new") to actually do that
detection.
This commit contains my changes to the API proposed by Keith. Nothing
is dramatically different. There are minor things like changing
notmuch_files_t to notmuch_filenames_t and then various things needed
for completeness as noticed while implementing this, (such as
notmuch_directory_destroy and notmuch_directory_set_mtime).
This will allow applications to support the removal of messages, (such
as when a file is deleted from the mail store). No removal support is
provided yet in commands such as "notmuch new".
The existing find_doc_ids function is convenient when the caller
doesn't want to be bothered constructing a term. But when the caller
*does* have the term already, that interface is just wasteful. So we
export a lower-level interface that maps a pre-constructed term to a
document-ID iterators.
The code to map a filename to a direntry is something that we're going
to want in a future _remove_message function, so put it in a new
function _notmuch_database_filename_to_direntry .
The library interface is unchanged so far, (still just
notmuch_database_add_message), but internally, the old
_set_filename function is now _add_filename instead.
Instead of storing the complete message filename in the data portion
of a mail document we now store a 'direntry' term that contains the
document ID of a directory document and also the basename of the
message filename within that directory. This will allow us to easily
store multple filenames for a single message, and will also allow us
to find mail documents for files that previously existed in a
directory but that have since been deleted.
Some pending commits want the _split_path functionality separate from
mapping a directory to a document ID. The split_path function now
returns the basename as well as the directory name.
We're planning to have mail documents refer to directory documents for
the path of the containing directory. To support this, we need the
path in the data, (since the path in the 'directory' term can be
irretrievable as it will be the SHA1 sum of the path for a very long
path).
We'll soon have mail documents referring to their parent directory's
directory documents, so we'll need access to _find_parent_id in files
such as message.cc.
Storing the document ID of the parent of each directory document will
allow us to find all child-directory documents for a given directory
document. We will need this in order to detect directories that have
been removed from the mail store, (though we aren't yet doing this).
The recent change from storing absolute paths to relative paths means
that new directory documents will already be created, (and the old
ones will just linger stale in the database). Given that, we might as
well put a clean name on the term in the new documents, (and no real
flag day is needed).
We were already storing relative mail filenames, so this is consistent
with that. Additionally, it means that directory documents remain
valid even if the database is relocated within its containing
filesystem.
We'll soon be having multiple entry points that accept a filename
path, so we want common code for getting a relative path from a
potentially absolute path.
And fix the initialization such that the private enum will always have
distinct values from the public enum even if we similarly miss the
addition of a new public value in the future.
The function _notmuch_message_add_thread_id has been removed
from the private interface of notmuch. There's no reason for
one to keep a declaration of its prototype in the code base.
Also, lets update a commentary that referenced that function
and escaped from previous scrutiny.
Signed-off-by: Fernando Carrijo <fcarrijo@yahoo.com.br>
Since we need to do this for portability, (some systems don't have a
strndup function), we might as well do it unconditionally. There's
almost no disadvantage to doing so, and this has the advantages of not
requiring a configure-time check nor having two different
implementations, one of which would often be less tested.
We carefully noted the fact that we had locally allocated the string
here, but then we neglected to free it. Switch to talloc instead
which makes it easier to get the behavior we want. It's simpler since
we can just call talloc_free unconditionally, without having to track
the state of whether we allocated the storage for name or not.
This error was tirggered with a debugging build via:
make CXXFLAGS="-DDEBUG"
and reported by David Bremner. The actual error is that I'm an
idiot that doesn't know how to use strcmp's return value. Of
course, the strcmp interface scores a negative 7 on Rusty Russell
ranking of bad interfaces:
http://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html
As per Carl's request, this patch corrects the only value defined under
the notmuch_message_flag_t enum typedef to match the name of the type.
Signed-off-by: Bart Trojanowski <bart@jukie.net>
If Xapian threw an exception on notmuch_query_count_messages the count
variable could be used uninitialized. Initialize count to solve the
problem.
Signed-off-by: Jeffrey C. Ollie <jeff@ocjtech.us>
When _notmuch_thread_create() is given a query string, it can return more
messages than just those matching the query. To distinguish those that
matched the query expression, the MATCHING_SEARCH flag is set
appropriately.
Signed-off-by: Bart Trojanowski <bart@jukie.net>
This patch allows for different flags, internal to notmuch, to be set on a
message object. The patch does not define any such flags, just the
facilities to manage these flags.
Signed-off-by: Bart Trojanowski <bart@jukie.net>
This patch adds a new function that can be used to collect a list of
unique tags from a list of messages. 'notmuch search-tags' uses the
function to get a list of tags from messages matching a search-term,
but it has the potential to be used elsewhere so we put it in the lib.
Signed-off-by: Jan Janak <jan@ryngle.com>
This patch adds a new function called notmuch_database_get_all_tags
which can be used to obtain a list of all tags from the database
(in other words, the list contains all tags from all messages). The
function produces an alphabetically sorted list.
To add support for the new function, we rip the guts off of
notmuch_message_get_tags and put them in a new generic function
called _notmuch_convert_tags. The generic function takes a
Xapian::TermIterator as argument and uses the iterator to find tags.
This makes the function usable with different Xapian objects.
Function notmuch_message_get_tags is then reimplemented to call the
generic function with message->doc.termlist_begin() as argument.
Similarly, we implement notmuch_message_database_get_all_tags, the
function calls the generic function with db->xapian_db->allterms_begin()
as argument.
Finally, notmuch_database_get_all_tags is exported through
lib/notmuch.h
Signed-off-by: Jan Janak <jan@ryngle.com>
Xapian provides an interator-based interface to all search results.
So it was natural to make notmuch_messages_t be iterator-based as
well. Which we did originally.
But we ran into a problem when we added two APIs, (_get_replies and
_get_toplevel_messages), that want to return a messages iterator
that's *not* based on a Xapian search result. My original compromise
was to use notmuch_message_list_t as the basis for all returned
messages iterators in the public interface.
This had the problem of introducing extra latency at the beginning
of a search for messages, (the call would block while iterating over
all results from Xapian, converting to a message list).
In this commit, we remove that initial conversion and instead provide
two alternate implementations of notmuch_messages_t (one on top of a
Xapian iterator and one on top of a message list).
With this change, I tested a "notmuch search" returning *many* results
as previously taking about 7 seconds before results started appearing,
and now taking only 2 seconds.
Previously, notmuch_query_search_threads would do all the work, so the
caller would block until all results were processed. Now, we do the
work as we go, as the caller iterates with notmuch_threads_next. This
means that once results start coming back from "notmuch search" they
just keep continually streaming.
There's still some initial blocking before the first results appear
because the notmuch_messages_t object has the same bug (for now).
This was a poor workaround around the fact that the existing
notmuch_threads_t object is implemented poorly. It's got a fine
iterartor-based interface, but the implementation does all of the
work up-front in _create rather than doing the work incrementally
while iterating.
So to start fixing this, first get rid of all the hacks we had working
around this. This drops the --first and --max-threads options from the
search command, (but hopefully nobody was using them
anyway---notmuch.el certainly wasn't).
The rudimentary aspect here is that the date ranges are specified with
UNIX timestamp values (number of seconds since 1970-01-01 UTC). One
thing that can help here is using the date program to determins
timestamps, such as:
$(date +%s -d 2009-10-01)..$(date +%s)
Long-term, we'll probably need to do our own query parsing to be able
to support directly-specified dates and also relative expressions like
"since:'2 months ago'".
Getting the count of matching threads or messages is a fairly
expensive operation. Xapian provides a very efficient mechanism that
returns an approximate value, so use that for this new command.
This returns the number of matching messages, not threads, as that is
cheap to compute.
Signed-off-by: Keith Packard <keithp@keithp.com>
I configured my database.path with a trailing /, and after running notmuch
new every notmuch search would fail with error messages like this:
Error opening /inbox/cur/1258565257.000211.mbox:2,S: No such file or directory
The actual bug was in the filename normalization for storage in the
database. The database.path was removed from the full filename, but if
the database.path from the config file contained a trailing /, the
relative file name would retain an extra leading /... which made it look
like an absolute path after it was read out from the DB.
Signed-off-by: Bart Trojanowski <bart@jukie.net>
Use the facilities of GNU make to create a magic function that will
on the first invocation print a description of how to enable verbose
compile lines and then print the quiet rule.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Carl Worth <cworth@cworth.org>
Cc: Mikhail Gusarov <dottedmag@dottedmag.net>
[ickle: Rebased, and duplicate command string eliminated.]
[ickle: Fixed verbose bug pointed out by Mikhail]
Since Xapian has a limit on the maximum length of a term, we have
to check for that before trying to add the message ID as a term.
This fixes the bug reported by Mike Hommey here:
<20091120132625.GA19246@glandium.org>
I've also constructed 20 files with a range of message ID lengths
centered around the Xapian term-length limit which I'll use to seed a
new test suite soon.
If an earlier exception occurred, then it's not unexpected for the
flush to fail as well. So in that case, we'll silently catch the
exception. Otherwise, make some noise about things going wrong at the
time of flush.
We only rarely need to actually open the database for writing, but we
always create a Xapian::WritableDatabase. This has the effect of
preventing searches and like whilst updating the index.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Acked-by: Carl Worth <cworth@cworth.org>
In my script containing a series of queries to be run on new mail for
setting up tags, it's nice to see which query I typed wrong.
Signed-off-by: Eric Anholt <eric@anholt.net>
More fallout from _get_header now returning "" for missing headers.
The bug here is that we would no longer detect that a file is not an
email message and give up on it like we should.
And this time, I actually audited all callers to
notmuch_message_get_header, so hopefully we're done fixing this
bug over and over.
There's been a fair amount of fallout from when we changed
message_file_get_header from returning NULL to returning "" for
missing headers. This is yet more fallout from that, (where we were
accepting an empty message-ID rather than generating one like we want
to).
This eliminates a crash when a message (either corrupted or a non-mail
file that wasn't properly detected as not being mail) has no In-Reply-To
header, (and so few terms that trying to skip to the prefix of the
In-Reply-To terms actually brings us to the end of the termlist).
As suggested by Keith in FLAG_PURE_NOT allows for expressions like:
notmuch search NOT tag:inbox
Note that this way a search like:
notmuch search foobar NOT tag:inbox
should not be written instead:
notmuch search foobar AND NOT tag:inbox
In my opinion, the latter feels more natural and is somewhat more explicit.
It gives a better clue of what the search is about instead of assuming that
an implicit AND operator is there.
This was recently introduced in commit:
64c03ae97f
which was adding extra checks to avoid adding a self-referencing
message.
How many times am I going to fix a dumb regression like this and say
"we really need a test suite" before I actually sit down and write the
test suite?
This is what most people want for a _search_ command. It's often
different for actually reading mail in an inbox, (where it makes more
sense to have results displayed in chronological order), but in such a
case, ther user is likely using an interface that can simply pass the
--sort=oldest-first option to "notmuch search".
Here we're also change the sort enum from NOTMUCH_SORT_DATE and
NOTMUCH_SORT_DATE_REVERSE to NOTMUCH_SORT_OLDEST_FIRST and
NOTMUCH_SORT_NEWEST_FIRST. Similarly we replace the --reverse option
to "notmuch search" with two options: --sort=oldest-first and
--sort=newest-first.
Finally, these changes are all tracked in the emacs interface, (which
has no change in its behavior).
We had exposed this to the internal implementation for a short time,
(only while we had the silly code fetching In-Reply-To values from
message files instead of from the database). Make this private again
as it should be.
Maybe ths lack of this documentation is why I forgot we were actually
storing this and wrote the ugly code to fetch In-Reply-To from message
files rather than from the database.
Which is more consistent with the XREFERENCE prefix used in the terms
in the database. Also remove some stale documentation describing the
removal of resolved references from the database (we no longer do
this).
Calling continue here worked only because we set a flag before the
continue, and, check the flag at the beginning of the loop, and *then*
break. It's much more clear to just break in the first place.
The message file header parsing code parses only enough of the file to
find the desired header fields, then it leaves the file open until the
next header parsing call or when the message is no longer in use. If a
large number of messages end up being active, this will quickly run
out of file descriptors.
Here, we add support to explicitly close the message file within a
message, (_notmuch_message_close) and call that from thread
construction code.
Signed-off-by: Keith Packard <keithp@keithp.com>
Edited-by: Carl Worth <cworth@cworth.org>:
Many portions of Keith's original patch have since been solved other
ways, (such as the code that changed the handling of the In-Reply-To
header). So the final version is clean enough that I think even Keith
would be happy to have his name on it.
In our scheme it's illegal for any message to refer to itself, (nor
would it be useful for anything anyway). Cut these self-references off
at the source, before they trip up any internal errors.
This case was happening when a message had its own message ID in its
In-Reply-To header. The thread-resolution code would find the
partially constructed message, (with no thread ID yet), get garbage
from this function, and then march right along with that garbage.
With this commit, a self-cyclic message like this will now trigger an
internal error rather than marching along silienty. (And a subsequent
commit will remove the call to this function in this case.)
This function has only one caller, and that one caller was passing the
same value for both talloc_owner and the notmuch database. Dropping
the redundant argument simplifies the documentation of this function
considerably.
