State up front that these functions may add a filename to an existing
message or remove only a filename (and not the message), respectively.
Previously, this key information was buried in return value
documentation or in "notes", which made it seem secondary to these
functions' semantics.
This addresses atomicity of tag synchronization, the last atomicity
problems in notmuch new. Each message add or remove is wrapped in its
own atomic section, so interrupting notmuch new doesn't lose progress.
Because flag synchronization is stateless, it can be performed at any
time as long as it's guaranteed to be performed after any change to a
message's filename list. Take advantage of this to synchronize tags
immediately after a filename is added or removed.
This does not yet make adding or removing a message atomic, but it is
a big step toward atomicity because it reduces the window where the
database tags are inconsistent from nearly the entire notmuch-new to
just around when the message is added or removed.
Previously, file name removal was implemented identically in two
places. Now it's captured in one function.
This is important because file name removal is about to get slightly
more complicated with eager tag synchronization and correct removal
atomicity.
Previously, pointers to these variables were passed around
individually. This was okay when only one function needed them, but
we're about to need them in a few more places.
Adding a message may involve changes to multiple database documents,
and thus needs to be done in a transaction. This makes add_message
(and, I think, the whole library) atomicity-safe: library callers only
needs to use atomic sections if they needs atomicity across multiple
library calls.
notmuch_database_find_message_by_filename is mostly stolen from
notmuch_database_remove_message, so this patch also vastly simplfies
the latter using the former.
This API is also useful in its own right and will be used in a later
patch for eager maildir flag synchronization.
According to the common Ruby function naming convention, potentially
dangerous functions or functions which operate on the object itself are
suffixed with an exclamation mark. Both of these are true for object
destroying functions.
The following modules are affected:
- Notmuch::Directory
- Notmuch::FileNames
- Notmuch::Query
- Notmuch::Threads
- Notmuch::Thread
- Notmuch::Messages
- Notmuch::Message
- Notmuch::Tags
Previously, notmuch_database_remove_message would remove the message
file name, sync the change to the message document, re-find the
message document, and then delete it if there were no more file names.
An interruption after sync'ing would result in a file-name-less,
permanently un-removable zombie message that would produce errors and
odd results in searches. We could wrap this in an atomic section, but
it's much simpler to eliminate the round-about approach and just
delete the message document instead of sync'ing it if we removed the
last filename.
notmuch_database_t now keeps a nesting count and we only start a
transaction or commit for the outermost atomic section.
Introduces a new error, NOTMUCH_STATUS_UNBALANCED_ATOMIC.
Previously, if notmuch new were interrupted between updating the
directory mtime and handling removals from that directory, a
subsequent notmuch new would not handle those removals until something
else changed in that directory. This defers recording the updated
mtime until after removals are handled to eliminate this problem.
If we use unicode objects, libnotmuch would not cope with null bytes in
the byte array, so we need to make sure they are nicely formatted as
utf-8.
Introduce a helper function _str which does this throughout the code.
Patch slightly modified by Sebastian Spaeth.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Previously, message removals were always performed, even after a
SIGINT. As a result, when a message was moved from one folder to
another, a SIGINT between processing the directory the message was
removed from and processing the directory it was added to would result
in notmuch removing that message from the database.
This tests notmuch new's ability to recover from arbitrary stopping
failures. It interrupts notmuch new after every database commit and,
on every resulting database snapshot, re-runs notmuch new to
completion and checks that the final database state is invariant.
Previously, this function would synchronize the folder list even if
removing the file name failed. Now it returns immediately if removing
the file name fails.
This means that test_subtest_known_broken needs to be called before
every known broken subtest, which is no different than what is
documented for the test_begin_subtest case.
The assumption is that every test ends up calling either skipping,
calling test_ok_ or test_failure_ and and the latter in turn delegate
to the known_broken versions in the case where
test_subtest_known_broken_ is set.
The Glib docs state "Prior to any use of the type system, g_type_init() has to
be called".[1] To not do so can lead to segfaults. The g_type system is
currently used by various "filters" that operate on uuencoded text, message
headers, etc.
[1] http://developer.gnome.org/gobject/2.28/gobject-Type-Information.html#g-type-init
Human-friendly scenario:
* open a thread where a message which ends with an HTML part is
followed by another message
* make the first message visible
* goto the beginning of the second message (first line, first colon)
* hit "RET"
Result: nothing happens except for "No URL at point" message
Expected result: the second message is shown/hidden
The root cause is that the HTML part has `keymap' text property set.
In particular, "RET" is bound to visit a URL at point. The problem is
that `keymap' property affects the next character following the region
it is set to (see elisp manual [1]). Hence, the first character of
the next message has a keymap inherited from the HTML part.
[1] http://www.gnu.org/software/emacs/elisp/html_node/Special-Properties.html
There is existing support for broken tests. But it is not convenient
to use. The primary issue is that we have to maintain a set of
test_expect_*_failure functions which are equivalent to the normal
test_expect_* counterparts except for what functions are called for
result reporting. The patch adds test_subtest_known_broken function
which marks a subset as broken, making the normal test_expect_*
functions behave as test_expect_*_failure. All test_expect_*_failure
functions are removed. Test_known_broken_failure_ is changed to
format details the same way as test_failure_ does.
Another benefit of this change is that the diff when a broken test is
fixed would be small and nice.
Documentation is updated accordingly.
Update test_emacs documentation in test/README according to the latest
changes in emacs tests. Move the note regarding setting variables
from test/emacs to test/README.
This was a minor oversite in checking of part type when outputing
content raw. This was causing gmime was to throw an exception to
stderr.
Unfortunately the gmime exception was not being caught by notmuch, or
the test suite. I'm not sure if notmuch should have done anything in
this case, but certainly the test suite should be capable of detecting
that something unexpected was output to stderr.
we now have three files to keep in sync. That seems wrong, but I guess
we will live with it for now.
The main problem is that the python code is distributed separately, so
it can't get the version from 'version'.
The choice ~rcX is for convenience with debian versioning.
The insert-part-message/rfc822 function is overhauled to properly
processes the new formatting of message/rfc822 parts. The json output
for message parts now includes "headers" and "body" fields, which are
now parsed and output appropriately.
This new function takes a GMimeMessage as input, and outputs the
formatted headers. This allows for message/rfc822 parts to be
formatted on output in a similar way to full messages (see previous
patch that overhauls the multipart test for more info).
The main goal of this overhaul is to define how message/rfc822 parts
should be handled. message/rfc822 parts should be output in a similar
fashion to the outer message, including some subset of the rfc822
headers. The following decisions about formatting of message/rfc822
parts were made:
The format and content of message/rfc822 parts shall be as similar as
possible to that of full messages. In particular, for formatted
outputs, the "content" of rfc822 part output should include "headers"
and "body" fields).
The "body" field shall include the body of the message.
The "headers" field shall include the following headers, since these
are the ones available from the GMimeMessage:
"From"
"To"
"Cc"
"Subject"
"Date"
However, for the case of --format=raw the raw rfc822 should be output,
including all headers.
A subset of relevant headers shall be output in reply.
The test embedded rfc822 message is also modified to be itself
multipart, so we can more fully test how all sub parts of the message
part are output.
Note added by Committer:
Currently, expect one test (--format=raw --part=3, rfc822 part) to fail.
Decode and Encode from/to unicode objects as required to be able to take
unicode path names. Previously we would error out when an unicode object
were handed it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The test message date, "Tue, 05 Jan 2001 15:43:57 -0000", is not
actually a real date. 05 Jan 2001 was in fact a Friday, not a
Tuesday. Date parsers (such as "date" in coreutils) will return "Fri"
as the day for this string, even if "Tue" is specified.
Also, the time zone "-0000" is actually always returned as "+0000", so
we change that here was well.
This will be relevant for later patches when we begin parsing rfc822
part headers, where gmime returns a parsed date string.
If we do want to test date parsing, we should do that in a separate
test.
There were two "--format=text --part=0" tests. One of them was
supposed to be a test for "--format=text --part=1".
There were also two errant "test_expect_equal_file OUTPUT EXPECTED"
lines, that are removed here.
Thanks to Sebastian Spaeth breaking out version.py, this can be done
without loading notmuch.py, or using sed. version.py is not (yet)
autogenerated because it seems more important to minimize the
differences between the tagged version and the tarball.
