When testing error handling, it is sometimes difficult to cover a
particular error path deterministically. Introduce a test function to
allow calling lower level functions directly.
This addresses a bug report / feature request of Uwe Kleine-König. The
assumption is that we always load a config file in the CLI (i.e. we
never pass "" as the config file argument to
notmuch_database_open_with_config).
[1]: id:8baa58c3-7ab9-ec03-1bbd-28aa5be838f2@kleine-koenig.org
Setting this according to --config was requested by Uwe
Kleine-König. There are some other ways that the configuration file
might be found in current notmuch, so check those as well.
As a bonus, fix a bug in the hook tests that left NOTMUCH_CONFIG set
even though a config file was provided via NOTMUCH_PROFILE.
[1]: id:8baa58c3-7ab9-ec03-1bbd-28aa5be838f2@kleine-koenig.org
Commits or checkouts that modify a large fraction of the messages in
the database should be relatively rare (and in some automated process,
probably non-existent). For initial setup, where such operations are
expected, the user can pass --force.
This is probably more convenient than always passing a command line
argument.
Use notmuch-config for consistency with other notmuch CLI tools.
Now that there is something relevant in the config files, test the
--config option.
The previous defaults were not suitable for personal (i.e. not
bugtracking for notmuch development) use.
Provide two ways for the user to select nmbug compatible defaults;
command line argument and checking the name of the script.
If the private index file matches a previously known revision of the
database, we can update the index incrementally using the recorded
lastmod counter. This is typically much faster than a full update,
although it could be slower in the case of large changes to the
database.
The "git-read-tree HEAD" is also a bottleneck, but unfortunately
sometimes is needed. Cache the index checksum and hash to reduce the
number of times the operation is run. The overall design is a
simplified version of the PrivateIndex class.
Unlike the (current) infix query parser provided by Xapian, the
notmuch specific sexp query parser supports prefixed wildcard queries,
so use those. In addition to being somewhat faster, this avoids
needing to escape all of the user's tags to pass via the shell.
Although the code required to support both new and old environment
variables is small, it complicates the semantics of configuration, and
make the documentation harder to follow.
Having notmuch-show-next-thread return non-nil on success and nil on
failure makes it easier for users to interact with notmuch via elisp.
This commit changes notmuch-search-show-thread too since the return
value of notmuch-show-next-thread depends on notmuch-search-show-thread.
Amended by db: fix whitespace in T450-emacs-show
This is a bit more involved than replacing the use of
notmuch_database_open_verbose, as we have to effectively inline the
definition of notmuch_database_open.
The actual content type of `application/octet-stream` is up to content
type detection of the reader, and thus may not be stable across
implementations or versions. This showed up when
fd46fc19 ("emacs: document/defcustom notmuch-multipart/alternative-discouraged", 2022-05-14)
introduced a test for omitting a part of type `text/html` because it
expected a part of type `application/octet-stream` to remain in place,
i.e. a part of "unstable type". In particular, tests with `fd46fc19`
would succeed on RHEL/EPEL but fail on all current Fedoras with
```
FAIL multipart/alternative hides html by default
--- T450-emacs-show.16.notmuch-show-multipart-alternative 2022-05-26 15:34:42.100557244 +0000
+++ T450-emacs-show.16.OUTPUT 2022-05-26 15:34:42.102557207 +0000
@@ -24,7 +24,7 @@
uses 64 as the
buffer size.
[ text/html (hidden) ]
-[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: application/octet-stream (as text/x-diff) ]
+[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: application/octet-stream (as text/x-patch) ]
From e3bc4bbd7b9d0d086816ab5f8f2d6ffea1dd3ea4 Mon Sep 17 00:00:00 2001
From: Alexander Botero-Lowry <alex.boterolowry@gmail.com>
Date: Tue, 17 Nov 2009 11:30:39 -0800
```
due to the different type detected.
Fix this by giving that message a specicific type of `text/x-diff` in
the test corpus, and adjust all affected test outputs.
Signed-off-by: Michael J Gruber <git@grubix.eu>
Amended-by: db, fix some trailing whitespace
This variable is important for people who want to change the default
behaviour when displaying multipart/alternative messages. Previously
it was undocumented. Add a defcustom to help users and copy some
documentation from the wiki. The usual machinery of re-using
docstrings is a bit tricky to use here because it mangles the example
lisp code, and the link to the info node should not be in e.g. the
html page.
Add a simple test to make sure the switch from defvar to defcustom did
not break something obvious.
The extra machinery to check for the actual output format is justified
by the possibility that distros may patch this newer output format
into older versions of gmime.
Amended-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Michael J Gruber <git@grubix.eu>
Amended-again-by: db
Since Xapian does not preserve quotes when passing the subquery to a
field processor, we have to make a guess as to what the user
intended. Here the added assumption is that a string surrounded by
parens is not intended to be a phrase.
test_require_external prereq has to move to test-lib-common.sh, and
the new shell functions print_emacs_header and time_emacs are provided.
The somewhat indirect way of printing the output is to avoid the extra
"" present on string values from emacsclient.
T310-emacs is one of the largest and longest running sets of
tests. Splitting out the tagging operations will help maintainability
as well as potentially improve the parallel running time of the test
suite. Some slowdown in running the tests sequentially may result
since there is repeated setup.
This allows sharing more variable settings between the (correctness)
tests and the performance-tests. Unfortunately it seems a bit tricky
to move settings to test-lib-common.sh, as that is sourced late in
test-lib.sh, and moving it earlier breaks things.
In most cases used printf %s ... instead.
echo -n > file lines to create empty / truncate files were
changed to : > file lines, like done in in test-lib-emacs.sh
And one echo -n " " replaced with use of sed "s/^/ /" in next line.
The idea is to do as little parsing and modification of the delivered
message as possible. Luckily the position of the "envelope header"
lets us escape it by replacing the first 5 characters of the stream
with a regular header name (with ':').
This particular message is not recognized by notmuch as mail, but is
fine according to e.g. mutt. The trigger for this bad behaviour seems
to be a second "From " ocurring at the beginning of the line but
inside an attachment.
As of notmuch 0.34.2 [1], the python-cffi bindings make available the
configuration from both a config file and the database when accessing
Database.config like a dictionary. It is therefore confusing that the
iterator operations only work on the configuration information stored
in the database.
[1]: d7f9572413
This changes makes the sexp query parser consistent with the infix one
in ignoring trailing '/'. Here we do a bit better and ignore any
number of trailing '/'.
This is mainly to make sure we get trailing / removal correct. Also
add regression test for path: in the infix parser matching the
existing one for folder:.
This resolves an old bug reported by David Edmondson in 2014. The fix
is only needed for the "boolean" case, as probabilistic / phrase
searching already ignores punctuation.
This fix is only for the infix (xapian provided) query parser.
[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
This duplicates the bug reported in [1], as well as adding some simple
regression tests for 'path' and 'folder' searches which were
previously missing for sexp syntax.
[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
In [1], David Edmondson observed that the trailing / added by many
completion mechanisms causes path searches to silently fail. This
test reproduces that bug for both `path:' and `folder:' searches.
[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
It is fragile to encode the generated names into tests, as it makes
tests break when e.g. new tests are added. There is a possibility
that this will hide certain failures; in that case meaningful filenames
should be chosen for the generated messages.
These are loosely modelled on the tests just above for date ranges,
since the error conditions are similar. Some ideas also borrowed from
T570-revision-tracking.
The default argument processing overlaps somewhat with what is already
done in _notmuch_date_strings_to_query, but we can give more specific
error messages for the s-expression context.
The extra generality of _sexp_parse_range will be useful when we
implement additional range prefixes (at least 'lastmod' is needed).
This fixes the bug reported by dkg in [1]. The movement of the call to
n-m-setup-message-for-saving is so the cleanup of Fcc headers happens
in the encoded version (otherwise Fcc headers may be saved to disk).
[1]: id:87k1zm225v.fsf@fifthhorseman.net
Based on the method outlined by Daniel Kahn Gillmor in
id:87k1zm225v.fsf@fifthhorseman.net. With a delay of 0.2 seconds the
test becomes flaky on my machine. With a 1 second delay it fails
consistently for more than 1600 iterations.
This matches the heuristic used by "notmuch config set" to decide if
something is a list.
This change fixes the bug reported at [1].
[1]: id:6O3LTUhoXlrnkPWCtPJCP4cagU7mFVEdyTpcC_37BoSzStlARXDBa7oczy6hB0jyjGjBQvgj_jFV58cw0aNx-jUg1h1O-FQ7820k68C0X4M=@protonmail.com
In [1] Ian observed that notmuch setup was inconsistent with notmuch
config set when adding single items, namely adding an unneeded
semi-colon at the end. This test replicates that bug.
[1]: id:6O3LTUhoXlrnkPWCtPJCP4cagU7mFVEdyTpcC_37BoSzStlARXDBa7oczy6hB0jyjGjBQvgj_jFV58cw0aNx-jUg1h1O-FQ7820k68C0X4M=@protonmail.com
Initially only use in notmuch-hello-insert-alltags. This is a more
narrow resolution of [1], which (unlike [2]) does not disable exclude
processing for regular saved searches.
[1]: id:87wox1vovj.fsf@len.workgroup
[2]: id:20220105010606.2034601-2-david@tethera.net
Revert commit 8370e3cfe2, and remark the
corresponding test as broken. Also update the expected output of the
broken test to show excludes active in the user defined saved searches.
Commit [0] left the stemmer object accessible, but did not add
de-allocation code to notmuch_database_destroy. This commit corrects
that oversight.
Leak originally reported by Austin Ray [1].
[0]: 3202e0d1fe
[1]: id:20220105224538.m36lnjn7rf3ieonc@athena
Gregor Zattler observed that tags could be unintentionally hidden in
the "All tags" view, and Tomi Ollia worked out [2] that the issue was tags
that only occured on excluded messages. This test reproduces that bug.
[1]: id:87wox1vovj.fsf@len.workgroup
[2]: id:m28t9faaim.fsf@guru.guru-group.fi
Although it makes sense for the extra headers to be added to the copy
of the message headers included in the sexp/json, it is a bit
surprising for them to show in the new message constructed for the
reply, especially when, as here, they are always missing/empty.
This is based on a patch from Johan Parin [1], which is in turn
responding to a bug report / feature requiest from Jan Malkhovski.
The update to the structured output documented in schemata is intended
to be upward compatible, so the format version stays the same
[1]: id:20191116162723.18343-1-johan.parin@gmail.com
[2]: id:87h8sdemnr.fsf@oxij.org
Since the bug was first reported in [1], notmuch has gained the
ability to have the database located outside the mail root, hence this
this change differs slightly from Jani's proposed solution [2] in not
using notmuch_database_get_path, but rather the already retrieved
mail_root.
[1]: id:87mwhifu9a.fsf@trouble.defaultvalue.org
[2]: id:87ios5v59p.fsf@nikula.org
In [1] Rob observed that notmuch new ignored directories called
.notmuch everywhere in the tree, where they should only (and now, with
split configs, at most) be ignored at the top level. Add a test to
demonstrate the problem.
[1]: id:87mwhifu9a.fsf@trouble.defaultvalue.org
If we know the configuration is split, but there is no mail root
defined, this indicates a (lack of) configuration error. Currently
this can only arise in XDG configurations.
If notmuch_database_open_with_config finds a database, but that
database is not in a legacy, non-split configuration, then it
currently incorrectly deduces the mail root and returns SUCCESS. Add
to two tests to demonstrate this bug.
As stressed by the gpg documentation, the non-'with-colons' output
format is subject to change, and indeed it did in 2.3.x (x<=3). This
should make the the test suite more robust against such changes.
If we return regular Message objects, python will try to destroy them,
and the underlying notmuch object, causing e.g. the crash [1].
[1]: id:87sfu6utxg.fsf@tethera.net
This reproduces the bug reported at [1]. The second test hints at the
solution, making reply return OwnedMessage objects.
[1]: id:87sfu6utxg.fsf@tethera.net
It makes perfect sense for users to want to pre-create .notmuch,
e.g. to install hooks, so we should handle the case of a .notmuch
directory without an actual xapian database more gracefully.
`notmuch new' should go ahead and create the xapian database if it is
missing, even in the case where the parent .notmuch (or equivalent)
directory exists.
In [1] Mark Walters reported a problem with messages being removed
from the database when the parent directory was renamed. Jani Nikula
proposed [2] these tests but observed
This test is not suitable for merging since it's not deterministic.
After applying Jani's patch [3], the tests now pass deterministically,
and could usefully act as regression tests.
[1]: id:87siray6th.fsf@qmul.ac.uk
[2]: id:1393191650-28333-1-git-send-email-jani@nikula.org
[3]: id:1441445731-4362-2-git-send-email-jani@nikula.org
The remaining problem in this test is fixed upstream in Emacs
28. While most people are using earlier versions of emacs, the test
still provides some documentation of a known bug.
There is a certain amount of boilerplate to pass the call on the
original function, so abstract it out as a C preprocessor macro, plus
some extra includes in notmuch-test.h
Unlike the previous g_key_file_get_value, this version processes
escape codes for whitespace and \. The remaining two broken tests from
the last commit are because "notmuch config get" treats every value as
a list, and thus the previously introduces stripping of leading
whitespace applies.
glib generates the following escape characters with their usual
meanings: \n, \t, \r, and \\, along with \s for _leading_
spaces. Currently notmuch fails to unescape these on reading the
config files. These tests demonstrate this bug; the one new test that
passes is because apparently glib only escapes tabs at the beginning
of a key.
In [1] Ciprian observed that it was easy for users to mistakenly
introduce leading and trailing space to new.tags when editing a
notmuch config file. This commit strips spaces on either side of the
';' delimiter when splitting.
In principle it would be possible to support tags (or other config
values) with leading or trailing spaces by processing '\s' escapes in
the input string. Currently such processing is not done.
[1]: id:CA+Tk8fzjPLaEd3vL1f9ebk_bF_RV8PDTLzDupraTkCLCpJAmCg@mail.gmail.com
Since release 0.32, libnotmuch provides searching for database and
configuration paths. This commit changes the python module notmuch2 to
use those facilities.
This fixes the bug reported in [1], along with a couple of the
deprecation warnings in the python bindings.
Database.default_path is deprecated, since it no longer faithfully
reflects what libnotmuch is doing, and it is also no longer used in
the bindings themselves.
This commit choose the default of config=CONFIG.EMPTY (equivalent to
passing "" to notmuch_database_open_with_config). This makes the
change upward compatible API-wise (at least as far as the test suite
verifies), but changing the default to CONFIG.SEARCH would probably be
more convenient for bindings users.
[1]: id:87h7d4wp6b.fsf@tethera.net
If the user passed a path, and we opened it, then we consider that
definitive definition of "database.path". This makes libnotmuch
respond more gracefully to certain erroneous combinations of
NOTMUCH_CONFIG settings and config file contents.
This is arguably user error: having configuration file with bad
settings in it (and/or having a bad NOTMUCH_CONFIG environment
variable). On the other hand returning a different path than was
actually opened is definitely a bug.
