Add NOTMUCH_EXCLUDE_FLAG to notmuch_exclude_t so that it can
cover all four values of search --exclude in the cli.
Previously the way to avoid any message being marked excluded was to
pass in an empty list of excluded tags: since we now have an explicit
option we might as well honour it.
The enum is in a slightly strange order as the existing FALSE/TRUE
options correspond to the new
NOTMUCH_EXCLUDE_FLAG/NOTMUCH_EXCLUDE_TRUE options so this means we do
not need to bump the version number.
Indeed, an example of this is that the cli count and show still use
FALSE/TRUE and still work.
This allows specifying config file as a top level argument to notmuch,
and generally makes it possible to override config file options in
main(), without having to touch the subcommands.
If the config file does not exist, one will be created for the notmuch
main command and setup and help subcommands. Help is special in this
regard; the config is created just to avoid errors about missing
config, but it will not be saved.
This also makes notmuch config the talloc context for subcommands.
We now have a notmuch_config_is_new() function to query whether a
config was created or not. Change the notmuch_config_open() is_new
parameter into boolean create_new to determine whether the function
should create a new config if one doesn't exist. This reduces the
complexity of the API.
Add new format text0, which is otherwise the same as text, but use the
null character as separator instead of the newline character. This is
similar to find(1) -print0 option, and works together with the
xargs(1) -0 option.
This patch switches from the current ad-hoc printer to the structured
formatters in sprinter.h, sprinter-text.c and sprinter-json.c.
The JSON tests are changed slightly in order to make them PASS for the
new structured output formatter.
The text tests pass without adaptation.
It has been a long-standing issue that notmuch_database_open doesn't
return any indication of why it failed. This patch changes its
prototype to return a notmuch_status_t and set an out-argument to the
database itself, like other functions that return both a status and an
object.
In the interest of atomicity, this also updates every use in the CLI
so that notmuch still compiles. Since this patch does not update the
bindings, the Python bindings test fails.
This commit replaces the --no-exclude option with a
--exclude=(true|false|flag) option. The default is to omit the
excluded messages.
The flag option only makes sense if output=summary (as otherwise there
is nowhere to print the flag). In summary output exclude=false and
exclude=flag give almost identical output:
they differ in that with the exclude=flag option the match count
(i.e., the x in [x/n] in the output) is the number of matching
non-excluded messages rather than the number of matching messages.
Note this changes the default for output=summary when no --exclude=
option is given: it used to default to flag and now defaults to true
(i.e. omit excluded messages). This is neccesary to keep the cli
output uncluttered and for speed reasons.
In all cases of notmuch count/search/show where the results returned
cannot reflect the exclude flag return just the matched not-excluded
results. If the caller wishes to have all the matched results (i.e.,
including the excluded ones) they should call with the
--no-exclude option.
The relevant cases are
count: both threads and messages
search: all cases except the summary view
show: mbox format
All other config-related functions and args include the section
title in their name, so for the sake of consistency, mirror that.
Also, the "auto"matic part is a given, so that was dropped.
This adds a "search" section to the config file and an
"auto_tag_exclusions" setting in that section. The search and count
commands pass tag tags from the configuration to the library.
The switch on format_sel is slightly clunky, but it doesn't seem worth
special casing argument processing for function pointers, when I think
the function pointer approach will be modified/abandoned.
Add options --offset=[-]N and --limit=M to notmuch search to determine the
first result and maximum number of results to display.
Option --limit=M limits the maximum number of results to display to M.
Option --offset=[-]N skips the first N results; with the leading '-' skip
until the Nth result from the end.
Note that --offset with a negative N for thread or summary output requires
counting the number of matching threads in advance.
Signed-off-by: Jani Nikula <jani@nikula.org>
previously we deleted the subcommand name from argv before passing to
the subcommand. In this version, the deletion is done in the actual
subcommands. Although this causes some duplication of code, it allows
us to be more flexible about how we parse command line arguments in
the subcommand, including possibly using off-the-shelf routines like
getopt_long that expect the name of the command in argv[0].
Messages in the database can have multiple files associated with a
single message-id, but until now only one filename for each message
has been reported by "notmuch search --output=files"
Signed-off-by: Mark Anderson <ma.skies@gmail.com>
When a Subject field contained encoded CRLF sequences, these sequences
would appear unfiltered in the output of notmuch search. This confused
the notmuch emacs interface leading to "Unexpected Output"
messages. This is now fixed by replacing all characters with ASCII
code less than 32 with a question mark.
Signed-off-by: Jameson Graef Rollins <jrollins@finestructure.net>
A previous commit to fix json formatting for null results
(0b1ddc5f66) accidentally introduced a
regression that removed trailing newlines for non-json output. (There
wasn't a good test for this previously, but there is now). The
problem is due to the fundamental differences in formatting between
the json and non-json outputs. The only way to fix this was to add a
new formatting field that represents the string to output at the end
of a null result.
All output formatting tests should pass now, (in particular, the 4
recent test failures introduced to show this bug).
In the original json code, search matching nothing would return a
valid, empty json array (that is, "[]"). I broke this in commit
6dcb7592e3 when adding support for
--output=threads|messages|tags. This time, while fixing the bug also
add a test to the test suite to help avoid future regressions.
With talloc, we were already freeing all memory by the time we exited
the loop, but that didn't help with excess use of memory inside the
loop, (which was mostly from tallocing some objects with the incorrect
parent).
Thanks to Andrew Tridgell for sitting next to me and teaching me to
use talloc_report_full to find these leaks.
As mentioned in the recent test commits, this also fixes the missing
'[' and ']' characters in the --format=json variant of "notmuch search
--output=tags" as well.
This change affects both text and json formats. Now, text format
behaves as before commit 6dcb7592, but json format is changed.
Earlier, the empty search returned '[]', now it returns ''. The emacs
interface seems not to be affected by this change.
Now that notmuch_query_search_threads can return NULL, (for example,
due to a Xapian exception), we need to handle that case (rather than
just segfault). It's simple enough to just return a non-zero exit
code.
We rename 'has_more' to 'valid' so that it can function whether
iterating in a forward or reverse direction. We also rename
'advance' to 'move_to_next' to setup parallel naming with
the proposed functions 'move_to_first', 'move_to_last', and
'move_to_previous'.
The text output uses thread:<foo>, (which is a syntax directly supported
by "notmuch show"), so make the json output be "thread", "<foo>" rather
than "id", "<foo>". This should help avoid confusion of thread IDs with
message IDs, (which do use the "id" prefix in searches).
In the case of notmuch-show, "--format=json" also implies
"--entire-thread" as the thread structure is implicit in the emitted
document tree.
As a coincidence to the implementation, multipart message ID numbers are
now incremented with each part printed. This changes the previous
semantics, which were unclear and not necessary related to the actual
ordering of the message parts.
This was a poor workaround around the fact that the existing
notmuch_threads_t object is implemented poorly. It's got a fine
iterartor-based interface, but the implementation does all of the
work up-front in _create rather than doing the work incrementally
while iterating.
So to start fixing this, first get rid of all the hacks we had working
around this. This drops the --first and --max-threads options from the
search command, (but hopefully nobody was using them
anyway---notmuch.el certainly wasn't).
This way, the user gets a steady (but bursty) stream of reults. We
double the chunk size each time since each successive chunk has to
redo work from all previous chunks.
Of course, the overall time is thereby slower, as the price we pay for
increased responsiveness. With a search returning about 17000 thread
results I measured a total time of 48.8 seconds before this change and
58.4 seconds afterwards.
We only rarely need to actually open the database for writing, but we
always create a Xapian::WritableDatabase. This has the effect of
preventing searches and like whilst updating the index.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Acked-by: Carl Worth <cworth@cworth.org>
That is, give a nice error message and exit if no search terms are
provided. Thanks to Priit Laes <plaes@plaes.org> for reporting the
error and providing an early version of the fix.
This is what most people want for a _search_ command. It's often
different for actually reading mail in an inbox, (where it makes more
sense to have results displayed in chronological order), but in such a
case, ther user is likely using an interface that can simply pass the
--sort=oldest-first option to "notmuch search".
Here we're also change the sort enum from NOTMUCH_SORT_DATE and
NOTMUCH_SORT_DATE_REVERSE to NOTMUCH_SORT_OLDEST_FIRST and
NOTMUCH_SORT_NEWEST_FIRST. Similarly we replace the --reverse option
to "notmuch search" with two options: --sort=oldest-first and
--sort=newest-first.
Finally, these changes are all tracked in the emacs interface, (which
has no change in its behavior).
This is one of those cases where total time is not the metric of
interest. We increase the total time of the search, (by doing some
redundant work for the initial threads). But more significantly, we
give the user *some* results nearly instantaneously, (so that the user
might see the result of interest without ever even waiting for the
complete results to come in).
Note that the difference between thread results in date order and
thread results in reverse-date order is not simply a matter of
reversing the final results. When sorting in date order, the threads
are sorted by the oldest message in the thread. When sorting in
reverse-date order, the threads are sorted by the newest message in
the thread.
This difference means that we might want an explicit option in the
interface to reverse the order, (even though the default will be to
display the inbox in date order and global searches in reverse-date
order).
Note that we don't print the number of *unread* messages, but instead
the number of messages that matched the search terms. This is in
keeping with our philosophy that the inbox is nothing more than a
search view. If we search for messages with an inbox tag, then that's
what we'll get a count of. (And if somebody does want to see unread
counts, then they can search for the "unread" tag.)
Getting the number of matched messages is really nice when doing
historical searches. For example in a search like:
notmuch search tag:sent
(where the "sent" tag has been applied to all messages originating
from the user's email address)---here it's really nice to be able to
see a thread where the user just mentioned one point [1/13] vs. really
getting involved in the discussion [10/29].
This time, things are actually tested. The current results aren't
exactly the same as previous results since the incremental search
doesn't necessarily see all the new messages that pertain to the
thread. This means that some author names are missing.
I plan to fix this by doing an additional database search for all
messages in each thread. Of course, this will also be different than
before since now the result will display *all* authors in the thread
(rather than only those that matched the search) but that's probably
what we really want to display anyway.
The library interface now allows the caller to do incremental searches,
(such as one page of results at a time). Next we'll just need to hook
this up to "notmuch search" and the emacs interface.
It's important to have the names present for determining whether a
thread is worth reading or not. We may want to think about
abbreviating the list somehow if it is excessively long (or redundant
as in bugzilla-daemon, bugzilla-daemon, bugzilla-daemon, etc.).
All of the following commands:
notmuch dump
notmuch reply
notmuch restore
notmuch search
notmuch show
notmuch tag
were calling notmuch_database_open with an argument of NULL. This was
a legitimate call until the recent addition of configuration, after
which it is expected that all commands will lookup the correct path in
the configuration file. So fix all these commands to do that.
Also, while touching all of these commands, we fix them to use the
talloc context that is passed in rather than creating a local talloc
context. We also switch from using goto for return values, to doing
direct returns as soon as an error is detected, (which can be leak
free thanks to talloc).
Now that the client sources are alone here in their own directory,
(with all the library sources down inside the lib directory), we can
break the client up into multiple files without mixing the files up.
The hope is that these smaller files will be easier to manage and
maintain.
