Many functions that implement the search command need to access command
line options. Instead of passing each option in a separate variable, put
them in a structure and pass only this structure.
This will become handy in the following commits.
Sanitize tabs and newlines to spaces rather than question marks in
--output=summary --format=text output.
This will also hide any difference in unfolding a header that has been
folded with a tab. Our own header parser replaces tabs with spaces,
while gmime would retain the tab.
Apart from the status codes for format mismatches, the non-zero exit
status codes have been arbitrary. Make the cli consistently return
either EXIT_SUCCESS or EXIT_FAILURE.
These queries will match exactly the set of messages currently in the
thread, even if more messages later arrive. Two queries are provided:
one for matched messages and one for unmatched messages.
This can be used to fix race conditions with tagging threads from
search results. While tagging based on a thread: query can affect
messages that arrived after the search, tagging based on stable
queries affects only the messages the user was shown in the search UI.
Since we want clients to be able to depend on the presence of these
queries, this ushers in schema version 2.
Effective with --output=files, output the Nth filename associated with
each message matching the query (N is 1-based). If N is greater than
the number of files associated with the message, don't print anything.
Add NOTMUCH_EXCLUDE_FLAG to notmuch_exclude_t so that it can
cover all four values of search --exclude in the cli.
Previously the way to avoid any message being marked excluded was to
pass in an empty list of excluded tags: since we now have an explicit
option we might as well honour it.
The enum is in a slightly strange order as the existing FALSE/TRUE
options correspond to the new
NOTMUCH_EXCLUDE_FLAG/NOTMUCH_EXCLUDE_TRUE options so this means we do
not need to bump the version number.
Indeed, an example of this is that the cli count and show still use
FALSE/TRUE and still work.
This allows specifying config file as a top level argument to notmuch,
and generally makes it possible to override config file options in
main(), without having to touch the subcommands.
If the config file does not exist, one will be created for the notmuch
main command and setup and help subcommands. Help is special in this
regard; the config is created just to avoid errors about missing
config, but it will not be saved.
This also makes notmuch config the talloc context for subcommands.
We now have a notmuch_config_is_new() function to query whether a
config was created or not. Change the notmuch_config_open() is_new
parameter into boolean create_new to determine whether the function
should create a new config if one doesn't exist. This reduces the
complexity of the API.
Add new format text0, which is otherwise the same as text, but use the
null character as separator instead of the newline character. This is
similar to find(1) -print0 option, and works together with the
xargs(1) -0 option.
This patch switches from the current ad-hoc printer to the structured
formatters in sprinter.h, sprinter-text.c and sprinter-json.c.
The JSON tests are changed slightly in order to make them PASS for the
new structured output formatter.
The text tests pass without adaptation.
It has been a long-standing issue that notmuch_database_open doesn't
return any indication of why it failed. This patch changes its
prototype to return a notmuch_status_t and set an out-argument to the
database itself, like other functions that return both a status and an
object.
In the interest of atomicity, this also updates every use in the CLI
so that notmuch still compiles. Since this patch does not update the
bindings, the Python bindings test fails.
This commit replaces the --no-exclude option with a
--exclude=(true|false|flag) option. The default is to omit the
excluded messages.
The flag option only makes sense if output=summary (as otherwise there
is nowhere to print the flag). In summary output exclude=false and
exclude=flag give almost identical output:
they differ in that with the exclude=flag option the match count
(i.e., the x in [x/n] in the output) is the number of matching
non-excluded messages rather than the number of matching messages.
Note this changes the default for output=summary when no --exclude=
option is given: it used to default to flag and now defaults to true
(i.e. omit excluded messages). This is neccesary to keep the cli
output uncluttered and for speed reasons.
In all cases of notmuch count/search/show where the results returned
cannot reflect the exclude flag return just the matched not-excluded
results. If the caller wishes to have all the matched results (i.e.,
including the excluded ones) they should call with the
--no-exclude option.
The relevant cases are
count: both threads and messages
search: all cases except the summary view
show: mbox format
All other config-related functions and args include the section
title in their name, so for the sake of consistency, mirror that.
Also, the "auto"matic part is a given, so that was dropped.
This adds a "search" section to the config file and an
"auto_tag_exclusions" setting in that section. The search and count
commands pass tag tags from the configuration to the library.
The switch on format_sel is slightly clunky, but it doesn't seem worth
special casing argument processing for function pointers, when I think
the function pointer approach will be modified/abandoned.
Add options --offset=[-]N and --limit=M to notmuch search to determine the
first result and maximum number of results to display.
Option --limit=M limits the maximum number of results to display to M.
Option --offset=[-]N skips the first N results; with the leading '-' skip
until the Nth result from the end.
Note that --offset with a negative N for thread or summary output requires
counting the number of matching threads in advance.
Signed-off-by: Jani Nikula <jani@nikula.org>
previously we deleted the subcommand name from argv before passing to
the subcommand. In this version, the deletion is done in the actual
subcommands. Although this causes some duplication of code, it allows
us to be more flexible about how we parse command line arguments in
the subcommand, including possibly using off-the-shelf routines like
getopt_long that expect the name of the command in argv[0].
Messages in the database can have multiple files associated with a
single message-id, but until now only one filename for each message
has been reported by "notmuch search --output=files"
Signed-off-by: Mark Anderson <ma.skies@gmail.com>
When a Subject field contained encoded CRLF sequences, these sequences
would appear unfiltered in the output of notmuch search. This confused
the notmuch emacs interface leading to "Unexpected Output"
messages. This is now fixed by replacing all characters with ASCII
code less than 32 with a question mark.
Signed-off-by: Jameson Graef Rollins <jrollins@finestructure.net>
A previous commit to fix json formatting for null results
(0b1ddc5f66) accidentally introduced a
regression that removed trailing newlines for non-json output. (There
wasn't a good test for this previously, but there is now). The
problem is due to the fundamental differences in formatting between
the json and non-json outputs. The only way to fix this was to add a
new formatting field that represents the string to output at the end
of a null result.
All output formatting tests should pass now, (in particular, the 4
recent test failures introduced to show this bug).
In the original json code, search matching nothing would return a
valid, empty json array (that is, "[]"). I broke this in commit
6dcb7592e3 when adding support for
--output=threads|messages|tags. This time, while fixing the bug also
add a test to the test suite to help avoid future regressions.
With talloc, we were already freeing all memory by the time we exited
the loop, but that didn't help with excess use of memory inside the
loop, (which was mostly from tallocing some objects with the incorrect
parent).
Thanks to Andrew Tridgell for sitting next to me and teaching me to
use talloc_report_full to find these leaks.
As mentioned in the recent test commits, this also fixes the missing
'[' and ']' characters in the --format=json variant of "notmuch search
--output=tags" as well.
This change affects both text and json formats. Now, text format
behaves as before commit 6dcb7592, but json format is changed.
Earlier, the empty search returned '[]', now it returns ''. The emacs
interface seems not to be affected by this change.
Now that notmuch_query_search_threads can return NULL, (for example,
due to a Xapian exception), we need to handle that case (rather than
just segfault). It's simple enough to just return a non-zero exit
code.
We rename 'has_more' to 'valid' so that it can function whether
iterating in a forward or reverse direction. We also rename
'advance' to 'move_to_next' to setup parallel naming with
the proposed functions 'move_to_first', 'move_to_last', and
'move_to_previous'.
The text output uses thread:<foo>, (which is a syntax directly supported
by "notmuch show"), so make the json output be "thread", "<foo>" rather
than "id", "<foo>". This should help avoid confusion of thread IDs with
message IDs, (which do use the "id" prefix in searches).
In the case of notmuch-show, "--format=json" also implies
"--entire-thread" as the thread structure is implicit in the emitted
document tree.
As a coincidence to the implementation, multipart message ID numbers are
now incremented with each part printed. This changes the previous
semantics, which were unclear and not necessary related to the actual
ordering of the message parts.
This was a poor workaround around the fact that the existing
notmuch_threads_t object is implemented poorly. It's got a fine
iterartor-based interface, but the implementation does all of the
work up-front in _create rather than doing the work incrementally
while iterating.
So to start fixing this, first get rid of all the hacks we had working
around this. This drops the --first and --max-threads options from the
search command, (but hopefully nobody was using them
anyway---notmuch.el certainly wasn't).
This way, the user gets a steady (but bursty) stream of reults. We
double the chunk size each time since each successive chunk has to
redo work from all previous chunks.
Of course, the overall time is thereby slower, as the price we pay for
increased responsiveness. With a search returning about 17000 thread
results I measured a total time of 48.8 seconds before this change and
58.4 seconds afterwards.
We only rarely need to actually open the database for writing, but we
always create a Xapian::WritableDatabase. This has the effect of
preventing searches and like whilst updating the index.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Acked-by: Carl Worth <cworth@cworth.org>
That is, give a nice error message and exit if no search terms are
provided. Thanks to Priit Laes <plaes@plaes.org> for reporting the
error and providing an early version of the fix.
This is what most people want for a _search_ command. It's often
different for actually reading mail in an inbox, (where it makes more
sense to have results displayed in chronological order), but in such a
case, ther user is likely using an interface that can simply pass the
--sort=oldest-first option to "notmuch search".
Here we're also change the sort enum from NOTMUCH_SORT_DATE and
NOTMUCH_SORT_DATE_REVERSE to NOTMUCH_SORT_OLDEST_FIRST and
NOTMUCH_SORT_NEWEST_FIRST. Similarly we replace the --reverse option
to "notmuch search" with two options: --sort=oldest-first and
--sort=newest-first.
Finally, these changes are all tracked in the emacs interface, (which
has no change in its behavior).
