Previously we just crashed with an internal error. With this change,
the caller can handle it better. Update notmuch-new so that it doesn't
crash with "unknown error code" because of this change.
Since the bug was first reported in [1], notmuch has gained the
ability to have the database located outside the mail root, hence this
this change differs slightly from Jani's proposed solution [2] in not
using notmuch_database_get_path, but rather the already retrieved
mail_root.
[1]: id:87mwhifu9a.fsf@trouble.defaultvalue.org
[2]: id:87ios5v59p.fsf@nikula.org
This reduces the amount of global state. Furthermore, index options
can be set (in principle) in several ways, not just in the one
function for processing indexing command line options.
It turns out that now that we pass an open database into the
subcommands, it is easy to check any requested uuid against the
database at the same time as we process the other shared
arguments. This results in overall less boilerplate code, as well as
making a CLI scope function and variable file scope in notmuch.c.
The stat is essentially replaced by the mkdir for error detection
purposes. This changes the default location for backups to make
things tidier, even in non-split configurations. Hopefully there is
not too many user scripts relying on the previous location.
Because the default location may not exist, replace the use of stat
for error detection with a call to mkdir.
This adds new state variable for the mail root, and uses it most
places db_path was used. The notable exception is dumps from
backups. The latter will be dealt with properly in a future commit.
This fixes a bug reported in [1]. In principle it could be possible
avoid one of these reopens, but it complicates the logic in main with
respect to creating new databases.
[1]: id:9C1993DF-84BD-4199-A9C8-BADA98498812@bubblegen.co.uk
This is the result of running
$ uncrustify --replace --config devel/uncrustify.cfg *.c *.h
in the top level source directory
Line breaks were then adjusted manually to keep argc and argv
together.
This enables support for hooks outside the database directory.
It relies strongly on configuration information being usable between
closing the database and destroying it.
In addition to the same type of changes as converting other
subcommands, add the possibility of creating a database at the top
level. It would probably make sense to use this for insert as well.
This will allow transitioning individual subcommands to the new
configuration framework. Eventually when they are all converted we can
remove the notmuch_config_t * argument.
For now, live with the parameter shadowing in some some subcommands;
it will go away when they are converted.
Boolean return values have no out-of-band-values to signal errors. The
change here is that a (somewhat unlikely) fatal error after indexing
will now be caught.
This means dropping GMimeCryptoContext and notmuch_config arguments.
All the argument changes are to internal functions, so this is not an
API or ABI break.
We also get to drop the #define for g_mime_3_unused.
signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
By default notmuch-new uses directory mtimes to optimize the scanning of
directories for new mail. This option allows turning that optimization
off e.g. for testing or debugging.
Add support for using /<regex>/ style regular expressions in
new.ignore, mixed with the old style verbatim file and directory
basenames. The regex is matched against the relative path from the
database path.
Enable override of the index.try_decrypt setting during "notmuch new"
on a per-invocation basis.
We update the documentation and tab completion, and also add a test.
C99 stdbool turned 18 this year. There really is no reason to use our
own, except in the library interface for backward
compatibility. Convert the cli and test binaries to stdbool.
Several changes at once, just to not have to change the same lines
several times over:
- Use designated initializers to initialize opt desc arrays.
- Only initialize the needed fields.
- Remove arg_id (short options) as unused.
- Replace opt_type and output_var with several type safe output
variables, where the output variable being non-NULL determines the
type. Introduce checks to ensure only one is set. The downside is
some waste of const space per argument; this could be saved by
retaining opt_type and using a union, but that's still pretty
verbose.
- Fix some variables due to the type safety. Mostly a good thing, but
leads to some enums being changed to ints. This is pedantically
correct, but somewhat annoying. We could also cast, but that defeats
the purpose a bit.
- Terminate the opt desc arrays using {}.
The output variable type safety and the ability to add new fields for
just some output types or arguments are the big wins. For example, if
we wanted to add a variable to set when the argument is present, we
could do so for just the arguments that need it.
Beauty is in the eye of the beholder, but I think this looks nice when
defining the arguments, and reduces some of the verbosity we have
there.
We reorder reading maildir flags to avoid overwriting 'new.tags'. The
inverted status of 'unread' means the maildir flag needs to be checked
a second time.
I backpedalled here on the idea of supporting 'new.tags' without
'unread' in the presence of maildir syncing. For files in 'new/', it
seems quite natural to tag them as 'unread'.
We need a way to pass parameters to the indexing functionality on the
first index, not just on reindexing. The obvious place is in
notmuch_database_add_message. But since modifying the argument list
would break both API and ABI, we needed a new name.
I considered notmuch_database_add_message_with_params(), but the
functionality we're talking about doesn't always add a message. It
tries to index a specific file, possibly adding a message, but
possibly doing other things, like adding terms to an existing message,
or failing to deal with message objects entirely (e.g. because the
file didn't contain a message).
So i chose the function name notmuch_database_index_file.
I confess i'm a little concerned about confusing future notmuch
developers with the new name, since we already have a private
_notmuch_message_index_file function, and the two do rather different
things. But i think the added clarity for people linking against the
future libnotmuch and the capacity for using index parameters makes
this a worthwhile tradeoff. (that said, if anyone has another name
that they strongly prefer, i'd be happy to go with it)
This changeset also adjusts the tests so that we test whether the new,
preferred function returns bad values (since the deprecated function
just calls the new one).
We can keep the deprecated n_d_add_message function around as long as
we like, but at the next place where we're forced to break API or ABI
we can probably choose to drop the name relatively safely.
NOTE: there is probably more cleanup to do in the ruby and go bindings
to complete the deprecation directly. I don't know those languages
well enough to attempt a fix; i don't know how to test them; and i
don't know the culture around those languages about API additions or
deprecations.
Recently a user reported a crash in notmuch new, but because of
missing error reporting, all they could say was "A Xapian exception
occured". This commit adds the extra information available about
the error message in the exception.
Split file ignores in count_files to fixed and user configured in
order to not have to call _entry_in_ignore_list twice when debugging
is enabled. Minor detail.
If some software other than notmuch new renames or removes files
during the notmuch new scan (specifically after scandir but before
indexing the file), keep going instead of bailing out. Failing to
index the file is just a race condition between notmuch and the other
software; the rename could happen after the notmuch new scan
anyway. It's not fatal, and we'll catch the renamed files on the next
scan.
Add a new exit code for when files vanished, so the caller has a
chance to detect the race and re-run notmuch new to recover.
Reported by Paul Wise <pabs@debian.org> at
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=843127
Many of the external links found in the notmuch source can be resolved
using https instead of http. This changeset addresses as many as i
could find, without touching the e-mail corpus or expected outputs
found in tests.
The code in add_file seems to assume that NOTMUCH_STATUS_FILE_ERROR is
never returned from add_message. This turns out to be false (although it
seems to only happen in certain fairly rare race conditions).
There was a problem with the directory documents being left behind when
the filesystem directory was removed. This was worked around in [1].
However, that ignored the fact that the directory documents are also
still listed by notmuch_directory_get_child_directories() leading to
confusing results when running notmuch new. The directory documents are
found and queued for removal over and over again.
Fix the problem for real by removing the directory documents. This fixes
the tests flagged as broken in [2].
The (non-deterministic) hack test from [3] also still passes with this
change.
[1] commit acd66cdec0
[2] commit ed9ceda623
[3] id:1441445731-4362-1-git-send-email-jani@nikula.org
The side effect is that all of add_files_state will be initialized to
zero, removing any lingering doubt that some of it might not be
initialized. It's not a small struct, and the initialization is
scattered around a bit, so this makes the code more readable.
The library does not have a function to remove a directory document
for a path. Usually this doesn't matter except for a slight waste of
space. However, if the same directory gets added to the filesystem
again, the old directory document is found with the old mtime. Reset
the directory mtime on removal to avoid problems.
The corner case that can hit this problem is renaming directories back
and forth. Renaming does not change the mtime of the directory in the
filesystem, and thus the old db directory document mtime may match the
fs mtime of the directory.
The long term fix might be to add a library function to remove a
directory document, however this is a much simpler and faster fix for
the time being.
The function notmuch_exit_if_unmatched_db_uuid is split from
notmuch_process_shared_options because it needs an open notmuch
database.
There are two exceptional cases in uuid handling.
1) notmuch config and notmuch setup don't currently open the database,
so it doesn't make sense to check the UUID.
2) notmuch compact opens the database inside the library, so we either
need to open the database just to check uuid, or change the API.
Try to narrow down what part of the code adds files and directories to
the queue(s) to be deleted.
Update one test. The output is slightly confusing, but I believe it is
correct, resulting from a directory being discovered but containing only ignored files.
Unfortunately it seems trickier to support --config globally
The non-trivial changes are in notmuch.c; most of the other changes
consists of blindly inserting two lines into every subcommand.
The compatibility wrapper ensures that clients calling
notmuch_database_open will receive consistent output for now.
The changes to notmuch-{new,search} and test/symbol-test are just to
make the test suite pass.
The use of IGNORE_RESULT is justified by two things. 1) I don't know
what else to do. 2) asprintf guarantees the output string is NULL if
an error occurs, so at least we are not passing garbage back.
