Commit graph

1559 commits

Author SHA1 Message Date
David Bremner
2c1d1107f5 lib: strip trailing '/' from pathnames (sexp queries).
This changes makes the sexp query parser consistent with the infix one
in ignoring trailing '/'. Here we do a bit better and ignore any
number of trailing '/'.
2022-01-27 07:48:27 -04:00
David Bremner
8358841c66 test: add multiple path, folder sexp query tests
This is mainly to make sure we get trailing / removal correct. Also
add regression test for path: in the infix parser matching the
existing one for folder:.
2022-01-27 07:48:27 -04:00
David Bremner
c62c22c9fb lib: drop trailing slash for path and folder searches (infix)
This resolves an old bug reported by David Edmondson in 2014. The fix
is only needed for the "boolean" case, as probabilistic / phrase
searching already ignores punctuation.

This fix is only for the infix (xapian provided) query parser.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
c73e273aaf test/sexp: tests for path, folder, including trailing '/' (sexp)
This duplicates the bug reported in [1], as well as adding some simple
regression tests for 'path' and 'folder' searches which were
previously missing for sexp syntax.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
053a390370 test: known broken tests for trailing / in path search (infix)
In [1], David Edmondson observed that the trailing / added by many
completion mechanisms causes path searches to silently fail.  This
test reproduces that bug for both `path:' and `folder:' searches.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
0756105832 test: sanitize generated message files names
It is fragile to encode the generated names into tests, as it makes
tests break when e.g. new tests are added.  There is a possibility
that this will hide certain failures; in that case meaningful filenames
should be chosen for the generated messages.
2022-01-27 07:48:27 -04:00
David Bremner
0a32741fce lib/parse-sexp: handle lastmod queries.
This particular choice of converting strings to integers requires C++11.
2022-01-26 07:41:02 -04:00
David Bremner
341016c8ec test/sexpr-search: add known broken tests for lastmod queries
These are loosely modelled on the tests just above for date ranges,
since the error conditions are similar. Some ideas also borrowed from
T570-revision-tracking.
2022-01-26 07:41:02 -04:00
David Bremner
77ab961a1d lib/parse-sexp: support actual date queries.
The default argument processing overlaps somewhat with what is already
done in _notmuch_date_strings_to_query, but we can give more specific
error messages for the s-expression context.

The extra generality of _sexp_parse_range will be useful when we
implement additional range prefixes (at least 'lastmod' is needed).
2022-01-26 07:41:02 -04:00
David Bremner
303f207a54 lib/parse-sexp: support zero argument date queries
These are not too practical, although they may simplify some user
query generation code. Mainly this adds a new prefix keyword to the
parser.
2022-01-26 07:41:02 -04:00
David Bremner
e02bf15a4f test/search: add known broken tests for 'date' prefix
This is documented, but apparently not implemented. Add tests for
planned supported syntax and error messages.
2022-01-26 07:41:02 -04:00
David Bremner
da302e1cba emacs: use cached encoded copy for fcc
This fixes the bug reported by dkg in [1]. The movement of the call to
n-m-setup-message-for-saving is so the cleanup of Fcc headers happens
in the encoded version (otherwise Fcc headers may be saved to disk).

[1]: id:87k1zm225v.fsf@fifthhorseman.net
2022-01-26 07:22:09 -04:00
David Bremner
8179c3d114 test/emacs: known broken test for matching fcc and sent message
Based on the method outlined by Daniel Kahn Gillmor in
id:87k1zm225v.fsf@fifthhorseman.net.  With a delay of 0.2 seconds the
test becomes flaky on my machine. With a 1 second delay it fails
consistently for more than 1600 iterations.
2022-01-26 07:22:09 -04:00
David Bremner
b559240bb1 test: define test_expect_equal_message_body
This is a relatively simple sed invocation, but rather than write a
comment everywhere, give it a descriptive name.
2022-01-26 07:22:09 -04:00
David Bremner
c8d292e25b test/emacs: match mml settings in emacs_{fcc,deliver}_message
This will allow the sending of signed messages via smtp in the test
suite.
2022-01-26 07:22:09 -04:00
David Bremner
88633bc7a7 CLI/setup: special case single item lists
This matches the heuristic used by "notmuch config set" to decide if
something is a list.

This change fixes the bug reported at [1].

[1]: id:6O3LTUhoXlrnkPWCtPJCP4cagU7mFVEdyTpcC_37BoSzStlARXDBa7oczy6hB0jyjGjBQvgj_jFV58cw0aNx-jUg1h1O-FQ7820k68C0X4M=@protonmail.com
2022-01-24 09:32:39 -04:00
David Bremner
82e1279790 test/setup: add known broken test for single items
In [1] Ian observed that notmuch setup was inconsistent with notmuch
config set when adding single items, namely adding an unneeded
semi-colon at the end. This test replicates that bug.

[1]: id:6O3LTUhoXlrnkPWCtPJCP4cagU7mFVEdyTpcC_37BoSzStlARXDBa7oczy6hB0jyjGjBQvgj_jFV58cw0aNx-jUg1h1O-FQ7820k68C0X4M=@protonmail.com
2022-01-24 09:31:00 -04:00
David Bremner
21cce961b9 emacs: define, use option :disable-excludes for n-h-query-counts
Initially only use in notmuch-hello-insert-alltags. This is a more
narrow resolution of [1], which (unlike [2]) does not disable exclude
processing for regular saved searches.

[1]: id:87wox1vovj.fsf@len.workgroup
[2]: id:20220105010606.2034601-2-david@tethera.net
2022-01-23 14:27:06 -04:00
David Bremner
68640fd967 Revert "emacs: use --exclude=false when checking for empty searches"
Revert commit 8370e3cfe2, and remark the
corresponding test as broken. Also update the expected output of the
broken test to show excludes active in the user defined saved searches.
2022-01-23 14:21:46 -04:00
David Bremner
2786aa4d54 lib/database: delete stemmer on destroy
Commit [0] left the stemmer object accessible, but did not add
de-allocation code to notmuch_database_destroy. This commit corrects
that oversight.

Leak originally reported by Austin Ray [1].

[0]: 3202e0d1fe
[1]: id:20220105224538.m36lnjn7rf3ieonc@athena
2022-01-22 21:14:29 -04:00
David Bremner
8dab460a08 test: add known broken test for memory leaks in open
This duplicates the memory leaks reported in [1]

[1]: id:20220105224538.m36lnjn7rf3ieonc@athena
2022-01-22 21:14:29 -04:00
David Bremner
cc180507b0 emacs: use --exclude=false when counting total messages
Even if a user excludes a large fraction of their messages, they still
nonetheless exist and are searchable.
2022-01-20 15:32:21 -04:00
David Bremner
704aa76f5d test/emacs: add known broken test for hidden tags
Gregor Zattler observed that tags could be unintentionally hidden in
the "All tags" view, and Tomi Ollia worked out [2] that the issue was tags
that only occured on excluded messages. This test reproduces that bug.

[1]: id:87wox1vovj.fsf@len.workgroup
[2]: id:m28t9faaim.fsf@guru.guru-group.fi
2022-01-20 15:31:29 -04:00
David Bremner
87d5a5a8aa CLI: print extra headers only for non-replies
If in the future we decide to output extra headers for replies, this
should be controlled by a separate configuration option.
2022-01-18 21:26:45 -04:00
David Bremner
2d036dbc3b test/emacs: known broken test for reply with extra headers set.
Although it makes sense for the extra headers to be added to the copy
of the message headers included in the sexp/json, it is a bit
surprising for them to show in the new message constructed for the
reply, especially when, as here, they are always missing/empty.
2022-01-18 21:09:56 -04:00
David Bremner
c5cf92aa35 CLI: print extra headers in structured output
This is based on a patch from Johan Parin [1], which is in turn
responding to a bug report / feature requiest from Jan Malkhovski.

The update to the structured output documented in schemata is intended
to be upward compatible, so the format version stays the same

[1]: id:20191116162723.18343-1-johan.parin@gmail.com
[2]: id:87h8sdemnr.fsf@oxij.org
2022-01-18 08:11:06 -04:00
David Bremner
79936ac93e lib/config: add known config key "show.extra_headers"
Used in a following commit to enable including extra headers beyond
the default in structured output.
2022-01-18 08:09:14 -04:00
David Bremner
78e6cf12c0 test: fix deprecation warning in symbol-test
Reduce the amount of noise in the build log.
2022-01-16 15:37:14 -04:00
David Bremner
6472dbf4b7 cli/new: only ignore .notmuch at top level
Since the bug was first reported in [1], notmuch has gained the
ability to have the database located outside the mail root, hence this
this change differs slightly from Jani's proposed solution [2] in not
using notmuch_database_get_path, but rather the already retrieved
mail_root.

[1]: id:87mwhifu9a.fsf@trouble.defaultvalue.org
[2]: id:87ios5v59p.fsf@nikula.org
2022-01-16 10:52:13 -04:00
David Bremner
fd6d50b38f test: add known broken test for ignoring non-toplevel .notmuch
In [1] Rob observed that notmuch new ignored directories called
.notmuch everywhere in the tree, where they should only (and now, with
split configs, at most) be ignored at the top level. Add a test to
demonstrate the problem.

[1]: id:87mwhifu9a.fsf@trouble.defaultvalue.org
2022-01-16 10:47:16 -04:00
David Bremner
fad2e7540b lib/open: no default mail root in split configurations
If we know the configuration is split, but there is no mail root
defined, this indicates a (lack of) configuration error. Currently
this can only arise in XDG configurations.
2022-01-15 15:59:39 -04:00
David Bremner
237f803fdb test/libconfig: add two tests for the config = "" case
If notmuch_database_open_with_config finds a database, but that
database is not in a legacy, non-split configuration, then it
currently incorrectly deduces the mail root and returns SUCCESS. Add
to two tests to demonstrate this bug.
2022-01-15 15:49:14 -04:00
David Bremner
c62ebcfea9 test/libconfig: save and restore config file
Currently the config file is unusable for further tests requiring a
valid database path.
2022-01-15 15:49:00 -04:00
David Bremner
f9d8f9c6ba Merge remote-tracking branch 'origin/release' 2022-01-12 13:25:29 -04:00
David Bremner
114b985ba7 test/gpgsm: use --with-colons when calculating fingerprint.
As stressed by the gpg documentation, the non-'with-colons' output
format is subject to change, and indeed it did in 2.3.x (x<=3). This
should make the the test suite more robust against such changes.
2022-01-12 13:22:51 -04:00
David Bremner
22e04ed01a Merge branch 'release' 2022-01-09 09:20:56 -04:00
David Bremner
9e7ea628e6 python-cffi: returned OwnedMessage objects from Message.replies
If we return regular Message objects, python will try to destroy them,
and the underlying notmuch object, causing e.g. the crash [1].

[1]: id:87sfu6utxg.fsf@tethera.net
2022-01-08 16:29:41 -04:00
David Bremner
d9a2b900b6 test: add known broken tests for recursive traversal of replies.
This reproduces the bug reported at [1]. The second test hints at the
solution, making reply return OwnedMessage objects.

[1]: id:87sfu6utxg.fsf@tethera.net
2022-01-08 16:29:41 -04:00
David Bremner
3eb25c94bd Merge branch 'release' 2021-12-29 14:20:49 -04:00
David Bremner
25e0f5e592 lib/open: do not consider .notmuch alone as an existing database.
It makes perfect sense for users to want to pre-create .notmuch,
e.g. to install hooks, so we should handle the case of a .notmuch
directory without an actual xapian database more gracefully.
2021-12-29 14:11:21 -04:00
David Bremner
e43bad4883 test/new: add known broken test for missing xapian directory.
`notmuch new' should go ahead and create the xapian database if it is
missing, even in the case where the parent .notmuch (or equivalent)
directory exists.
2021-12-29 14:11:10 -04:00
David Bremner
063f5e9862 test: test folder renames
In [1] Mark Walters reported a problem with messages being removed
from the database when the parent directory was renamed. Jani Nikula
proposed [2] these tests but observed

   This test is not suitable for merging since it's not deterministic.

After applying Jani's patch [3], the tests now pass deterministically,
and could usefully act as regression tests.

[1]: id:87siray6th.fsf@qmul.ac.uk
[2]: id:1393191650-28333-1-git-send-email-jani@nikula.org
[3]: id:1441445731-4362-2-git-send-email-jani@nikula.org
2021-12-26 12:30:25 -04:00
David Bremner
cea1604a08 test: remove directory names from paths in exceptions
These cause failures when building out of tree.
2021-12-23 08:47:40 -04:00
David Bremner
57f29f4cb1 test/emacs: mark one test as fixed in newer emacs.
The remaining problem in this test is fixed upstream in Emacs
28. While most people are using earlier versions of emacs, the test
still provides some documentation of a known bug.
2021-12-06 17:28:51 -04:00
David Bremner
efa7f35d4a test/new: replace use of gdb in vanishing file test
Unlike the similar change in T060-new, no symlink creation is needed
here.
2021-12-04 12:36:08 -04:00
David Bremner
9397e7e8eb test/count: replace use of gdb with a LD_PRELOAD shim
There is a certain amount of boilerplate to pass the call on the
original function, so abstract it out as a C preprocessor macro, plus
some extra includes in notmuch-test.h
2021-12-04 12:36:08 -04:00
David Bremner
1643c0459a test: move system includes to notmuch-test.h
This removes some redudant includes, and will also make it easier to
introduce "#define _GNU_SOURCE", which must come before all system
includes.
2021-12-04 12:36:08 -04:00
David Bremner
18cdd21b8b lib/config: use g_key_file_get_string to read config values
Unlike the previous g_key_file_get_value, this version processes
escape codes for whitespace and \. The remaining two broken tests from
the last commit are because "notmuch config get" treats every value as
a list, and thus the previously introduces stripping of leading
whitespace applies.
2021-12-04 12:17:09 -04:00
David Bremner
482bd3a46d test: known broken tests for escape characters in config files.
glib generates the following escape characters with their usual
meanings: \n, \t, \r, and \\, along with \s for _leading_
spaces. Currently notmuch fails to unescape these on reading the
config files. These tests demonstrate this bug; the one new test that
passes is because apparently glib only escapes tabs at the beginning
of a key.
2021-12-04 12:16:47 -04:00
David Bremner
bab633d3ac config: ignore leading/trailing spaces in ';'-delimited lists
In [1] Ciprian observed that it was easy for users to mistakenly
introduce leading and trailing space to new.tags when editing a
notmuch config file. This commit strips spaces on either side of the
';' delimiter when splitting.

In principle it would be possible to support tags (or other config
values) with leading or trailing spaces by processing '\s' escapes in
the input string. Currently such processing is not done.

[1]: id:CA+Tk8fzjPLaEd3vL1f9ebk_bF_RV8PDTLzDupraTkCLCpJAmCg@mail.gmail.com
2021-12-04 12:16:12 -04:00