Commit graph

1517 commits

Author SHA1 Message Date
David Bremner
cea1604a08 test: remove directory names from paths in exceptions
These cause failures when building out of tree.
2021-12-23 08:47:40 -04:00
David Bremner
57f29f4cb1 test/emacs: mark one test as fixed in newer emacs.
The remaining problem in this test is fixed upstream in Emacs
28. While most people are using earlier versions of emacs, the test
still provides some documentation of a known bug.
2021-12-06 17:28:51 -04:00
David Bremner
efa7f35d4a test/new: replace use of gdb in vanishing file test
Unlike the similar change in T060-new, no symlink creation is needed
here.
2021-12-04 12:36:08 -04:00
David Bremner
9397e7e8eb test/count: replace use of gdb with a LD_PRELOAD shim
There is a certain amount of boilerplate to pass the call on the
original function, so abstract it out as a C preprocessor macro, plus
some extra includes in notmuch-test.h
2021-12-04 12:36:08 -04:00
David Bremner
1643c0459a test: move system includes to notmuch-test.h
This removes some redudant includes, and will also make it easier to
introduce "#define _GNU_SOURCE", which must come before all system
includes.
2021-12-04 12:36:08 -04:00
David Bremner
18cdd21b8b lib/config: use g_key_file_get_string to read config values
Unlike the previous g_key_file_get_value, this version processes
escape codes for whitespace and \. The remaining two broken tests from
the last commit are because "notmuch config get" treats every value as
a list, and thus the previously introduces stripping of leading
whitespace applies.
2021-12-04 12:17:09 -04:00
David Bremner
482bd3a46d test: known broken tests for escape characters in config files.
glib generates the following escape characters with their usual
meanings: \n, \t, \r, and \\, along with \s for _leading_
spaces. Currently notmuch fails to unescape these on reading the
config files. These tests demonstrate this bug; the one new test that
passes is because apparently glib only escapes tabs at the beginning
of a key.
2021-12-04 12:16:47 -04:00
David Bremner
bab633d3ac config: ignore leading/trailing spaces in ';'-delimited lists
In [1] Ciprian observed that it was easy for users to mistakenly
introduce leading and trailing space to new.tags when editing a
notmuch config file. This commit strips spaces on either side of the
';' delimiter when splitting.

In principle it would be possible to support tags (or other config
values) with leading or trailing spaces by processing '\s' escapes in
the input string. Currently such processing is not done.

[1]: id:CA+Tk8fzjPLaEd3vL1f9ebk_bF_RV8PDTLzDupraTkCLCpJAmCg@mail.gmail.com
2021-12-04 12:16:12 -04:00
David Bremner
e22bbb124e test: known broken tests for leading/trailing ws in config
These tests duplicate the bug/misfeature reported in

      id:CA+Tk8fzjPLaEd3vL1f9ebk_bF_RV8PDTLzDupraTkCLCpJAmCg@mail.gmail.com
2021-12-04 12:15:49 -04:00
David Bremner
1e7d33961e Merge branch 'release' 2021-12-04 09:27:30 -04:00
David Bremner
d7f9572413 python-cffi: switch to notmuch_database_{open,create}_with_config
Since release 0.32, libnotmuch provides searching for database and
configuration paths. This commit changes the python module notmuch2 to
use those facilities.

This fixes the bug reported in [1], along with a couple of the
deprecation warnings in the python bindings.

Database.default_path is deprecated, since it no longer faithfully
reflects what libnotmuch is doing, and it is also no longer used in
the bindings themselves.

This commit choose the default of config=CONFIG.EMPTY (equivalent to
passing "" to notmuch_database_open_with_config).  This makes the
change upward compatible API-wise (at least as far as the test suite
verifies), but changing the default to CONFIG.SEARCH would probably be
more convenient for bindings users.

[1]: id:87h7d4wp6b.fsf@tethera.net
2021-12-04 08:42:31 -04:00
David Bremner
caafab01a4 test: add known broken tests for python bindings in split configs
This reproduces the bug(s) reported in id:87h7d4wp6b.fsf@tethera.net
2021-12-04 08:36:41 -04:00
David Bremner
5f077bef37 test: add python-cffi bindings to path for test_python
This will allow testing the new python bindings using test_python, in
addition to the current invocation of pytest.
2021-12-04 08:36:23 -04:00
David Bremner
59aac9cef3 lib/config: don't overwrite database.path if the caller passed it
If the user passed a path, and we opened it, then we consider that
definitive definition of "database.path". This makes libnotmuch
respond more gracefully to certain erroneous combinations of
NOTMUCH_CONFIG settings and config file contents.
2021-12-03 20:52:11 -04:00
David Bremner
378415a557 test: add known broken test for conflict with database parameter
This is arguably user error: having configuration file with bad
settings in it (and/or having a bad NOTMUCH_CONFIG environment
variable).  On the other hand returning a different path than was
actually opened is definitely a bug.
2021-12-03 20:49:36 -04:00
David Bremner
59a778ae4b test/libconfig: use 'export' for remaining sets of NOTMUCH_CONFIG
It makes the tests easier to understand if we always use export for
environment variables.
2021-12-03 20:47:05 -04:00
David Bremner
b264a49be3 rename built_with.sexpr_query to built_with.sexp_queries
It is confusing to use two different names (sexp vs sexpr) when
compared with the command line option --query=sexp and (furthermore)
singular vs plural when compared with the man page title.
2021-12-03 20:06:06 -04:00
David Bremner
fe51c275fc lib/load_config: deallocate / NULL database on fatal error
This fixes a potential memory leak, and makes the behaviour of
notmuch_database_load_config (somewhat) consistent with
n_d_{open,create} with config.
2021-10-30 13:51:50 -03:00
David Bremner
8f0b84789d lib/load_config: document expectations for db on error, add tests
This is a bit different than n_d_{open,create}_with_config, since
there are several non-zero status codes where we do want to return a
non-NULL database structure.
2021-10-30 13:49:37 -03:00
David Bremner
2ba50b5230 lib/create: fix memory leak, ensure *database=NULL on error
This code previously relied on _finish_open to free the notmuch struct
on errors (except for the case of database == NULL, which was a
potential double free). When we removed those frees from _finish_open,
we introduced a (small) memory leak.

In this commit, fix the memory leak, and harmonize the on-error
behaviour with n_d_open_with_config.
2021-10-30 13:47:47 -03:00
David Bremner
f3fcdd2dda lib/create: document expectations for db on error, add tests
It seems sensible to harmonize the behaviour with
n_d_open_with_config. In this commit we just assert the desired
behaviour.
2021-10-30 13:45:40 -03:00
David Bremner
74c4ce6d88 lib/open: fix potential double-free, ensure *database=NULL on error
During refactoring for 0.32, the code that set notmuch=NULL on various
errors was moved into _finish_open. This meant that the the code which
relied on that to set *database to NULL on error was no longer
correct. It also introduced a potential double free, since the notmuch
struct was deallocated inside _finish_open (via n_d_destroy).

In this commit we revert to "allocator frees", and leave any cleanup
to the caller of _finish_open. This allows us to get back the
behaviour of setting *database to NULL with a small change. Other
callers of _finish_open will need free notmuch on errors.
2021-10-30 13:43:58 -03:00
David Bremner
a942cb8ee3 test: add two known broken tests for missing config files
The documentation claims that the database will be set to NULL in this
case, but it is currently not happening. Based on a reproducer [1]
from Austin Ray.

[1]: id:20211021190401.imirxau2ewke6e2m@athena
2021-10-30 13:42:16 -03:00
David Bremner
e2b842afb1 test/libconfig: use 'export' when restoring NOTMUCH_CONFIG
This is required when NOTMUCH_CONFIG has been unset, and is harmless
otherwise.
2021-10-27 22:05:45 -03:00
David Bremner
3e2e724d53 notmuch 0.33.2 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmFVoPUACgkQA0U5G1Wq
 FSGSzg//WY202qUibO6pgFMPvgpYCB5g9Bq2lSzEcJzc2ZqZIH/3WvFSUeHMrInb
 GmHzJ4Q1w00K/wyB8MLYILKJ5/Tikg+O63/YkZMjXx6CDwu9zyNdk3L7TuZjkiOB
 rRiCPlEO7vvzUKedyecpY07DpgCYku1R9eOk032aSZEdOVheXIUUwDcR/cSFhxjO
 9amPls9vjZXKOQjOH8dgbxyZX8JbcTECgwYoFCT1k88NrVXdcGc3ds/ZuJowf2S7
 rIOLCZVuJD0yPGtMUBMHr5bRJ4O6e4NsHLNOgELFid3WXmQ3TzrRBDuJX5rDaaOs
 +0aqFWS57lujZLF+dZbZAYqK9CnQxZct+Au8GJlvvHZUBnS2skTTK8zu4NuGmgh0
 wckeesSXImC/NxHP/HaRNvcmrOUPTWYr3SaNKAbAl6zSjkhgzYxMnTnAFev+7HAc
 TNa50iLfMYaM0lyzD5gzhZZKvk68wwi1bxxhNMmfOF94HTjKOvL1AWxPoQkYUNN8
 07yvxURcZHwng+ufIfr/XsTDleiqSkYRJFtkpyP6+ZoYrIj815qtqQ03lzJTcYRn
 wMv3ipMrLuYMeP8T3Pr0y8DEl2WVPRyybYbN+07SkQLULzmjBhivxn3abSRqOFKm
 zDn6XiY3BMhTOPaslVIxZdP/8k7fA7Ak5qTVkXLZCPMZiV36k0w=
 =mwj4
 -----END PGP SIGNATURE-----

Merge tag '0.33.2'

notmuch 0.33.2 release
2021-09-30 08:58:48 -03:00
David Bremner
4760b4470c test/T355-smime: Use key as exported by gpgsm
As reported in id:87h7pxiek3.fsf@tethera.net, the previous version of
the test is flaky. There is some so-far undebugged interaction between
openssl and gpgsm that causes the keys to fail to import. As a
potential workaround, use the key as exported by gpgsm, and eliminate
openssl from this particular pipeline.
2021-09-29 20:04:31 -03:00
David Bremner
21e365f51a test/emacs: tests for notmuch-{tree,unthreaded} with bad CWD
These work thanks to the previous wrapping of process creation
primitives.
2021-09-11 11:11:29 -03:00
David Bremner
e722b4f48c emacs: wrap call-process
Provide safe working directory
2021-09-11 11:11:29 -03:00
David Bremner
9fe36d7dc7 test/emacs: test for notmuch-show with nonexistent CWD
Somewhat predictably, the other code path in
notmuch-call-notmuch--helper also needs to be fixed.
2021-09-11 10:30:25 -03:00
David Bremner
eb226437e1 emacs: wrap make-process
Provide a safe working directory.
2021-09-11 10:27:38 -03:00
David Bremner
88224bde62 test/emacs: test for notmuch-search with nonexistent CWD
(At least) notmuch-start-notmuch needs to be updated to set a safe
working directory.
2021-09-11 10:22:24 -03:00
David Bremner
5e5f2122f9 emacs: wrap call-process-region
As with notmuch--process-lines, initial purpose is to provide a safe
binding for default-directory. This is enough to make notmuch-hello
robust against non-existent or corrupt values default-directory, but
probably not other views.
2021-09-11 10:19:27 -03:00
David Bremner
06477bce03 test/emacs: run notmuch-hello with a nonexisting default dir
This replicates the problem reported in Debian bug #922536.
2021-09-11 10:13:22 -03:00
David Bremner
59d0d5a489 test/emacs: provide macro test-log-error
Because of the way emacs reports errors, a test form can crash and not
change the main buffer. To work around this, capture both signalled
errors and any other messages.
2021-09-11 10:10:16 -03:00
David Bremner
7556bb7da2 notmuch 0.33.1 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmE7RAYACgkQA0U5G1Wq
 FSHDHA//Q5SJhT9fD/yi2QOI4zIHvF438WtAGIL+TnYIeY8lDmdYZaRjU9Ub4O3J
 eLxJMhNcOyY35nDEGOOVga33aS8lefI+q/Ozx5sNzSge9PRaha+hGAmY2UrkOm+z
 TQBjFsdnGyWdTmJbkC9VQsaQJWjeKUKM0J4eT/sWFrJfW9vHuGnzrEnpOTQu8xk1
 OWshsYc5AkMnnvUBv0JuCJ4z4HQkTHeMrMKwd+B0/SC4OxPQ/af6FaMkoKanH0Ej
 q661AaZW+WNupynS1nu/O99z5U4BJ1V7nz361degdINfaUh/vmcXgHN7jGeoFtj4
 rTjV/51Jy/5WCbYV+11a0+RMZOEoQZi4ZBai6E6WVHpztkDXXqxHyDnho+A4SYK9
 1G/Rs6lO9FYRLTvdYhJbLH4OZgglPWXFgWJJxGLY+5cM7d86g8LC2pbJGWZ5IbAW
 eB3LRFStaqsBqjx9noxt9BcdxmbEu4o7lCm6y+EdAD04X37MGvv+Mlwsz57EMG9c
 2NCntrc1BUUTb/7mCp6oZn0miPCmYFVLtTj0ZfLAdP89aLJVTn6a36uTv8NqcszR
 ZjBFqGxB9AMU0idTEHnHJ5acNj4bLPwDBPBGYmEWnky/Tn7ZT3XqEDVvOXY2HVwa
 0RkVsgMsUIqUXutXjUYlLcl4YgmgpTsRbVpMZJh0/R4N9spDyWU=
 =P0Xd
 -----END PGP SIGNATURE-----

Merge tag '0.33.1'

notmuch 0.33.1 release
2021-09-10 21:02:10 -03:00
David Bremner
060ff57642 lib: use 'localhost' rather than fqdn for default mail address.
As discussed in the thread starting at [1], the fully qualified domain
name is a bit tricky to get reproducibly, might reveal information
people prefer to keep private, and somewhat unlikely to provide
reliable mail routing.

The new approach of $current_username@localhost is better for the
first two considerations, and probably at least as good as a test mail
address.

[1]: id:87sfyibqhj.fsf@tethera.net
2021-09-10 08:17:06 -03:00
David Bremner
dc8262bd33 test/emacs: test for functions in notmuch-search-result-format.
Based on the commit message in id:20210221151902.2301690-3-dme@dme.org

Add the function notmuch-test-result-flags to test-lib.el to avoid
repeating it in 3 T*.sh files.
2021-09-08 23:01:02 -03:00
David Bremner
4595a814c2 CLI/tag: enable sexp queries
We have to rewrite _optimize_tag_query here because it is generating
a query string in the infix Xapian syntax. Luckily this is easy to do
with the sexp query syntax.
2021-09-04 17:07:19 -07:00
David Bremner
2944d59133 CLI/{count, dump, reindex, reply, show}: enable sexp queries
The change in each case is to call notmuch_query_create_with_syntax,
relying on the already inherited shared options.  As a bonus we get
improved error handling from the new query creation API.

The remaining subcommand is 'tag', which is a bit trickier.
2021-09-04 17:07:19 -07:00
David Bremner
551254eb76 lib/parse-sexp: apply macros
Macros implement lazy evaluation and lexical scope.  The former is
needed to make certain natural constructs work sensibly (e.g. (tag
,param)) but the latter is mainly future-proofing in case the DSL is
is extended to allow local bindings.

For technical background, see chapters 6 and 17 of [1] (or some other
intermediate programming languages textbook).

[1] http://cs.brown.edu/courses/cs173/2012/book/
2021-09-04 17:07:19 -07:00
David Bremner
9b9eb1d8bd lib/parse-sexp: support saved s-expression queries
It turns out there is not really much code in query-fp.cc useful for
supporting the new syntax. The code we could potentially factor out
amounts to calling notmuch_database_get_config; both the key
construction and the parsing of the results are specific to the query
syntax involved.
2021-09-04 17:07:19 -07:00
David Bremner
81b9dbd110 CLI/config support saving s-expression queries
This commit does not enable using saved s-expression queries, only
saving and retrieving them from the config file or the database. Use
in queries will be enabled in a following commit.
2021-09-04 17:07:19 -07:00
David Bremner
6ab2d9b1a2 lib/parse-sexp: handle saved queries
This provides functionality analogous to query: in the Xapian
QueryParser based parser. Perhaps counterintuitively, the saved
queries currently have to be in the original query syntax (i.e. not
s-expressions).
2021-09-04 17:07:19 -07:00
David Bremner
a07ef8abf5 lib/parse-sexp: parse user headers
One subtle aspect is the replacement of _find_prefix with
_notmuch_database_prefix, which understands user headers. Otherwise
the code mainly consists of creating a fake prefix record (since the
user prefixes are not in the prefix table) and error handling.
2021-09-04 17:07:19 -07:00
David Bremner
cc5992a304 lib/parse-sexp: support infix subqueries
This is necessary so that programs can take infix syntax queries from
a user and use the sexp query syntax to construct e.g. a refinement of
that query.
2021-09-04 17:07:19 -07:00
David Bremner
afe85e6578 lib/parse-sexp: expand queries
The code here is just gluing together _notmuch_query_expand with the
existing sexp parser infrastructure.
2021-09-04 17:07:19 -07:00
David Bremner
1870b3ae4b lib/parse-sexp: support regular expressions
At least to the degree that the Xapian QueryParser based parser
also supports them. Support short alias 'rx' as it seems to make more
complex queries nicer to read.
2021-09-04 17:07:19 -07:00
David Bremner
0b98ad5e4e lib/query: generalize exclude handling to s-expression queries
In fact most of the code path is in common, only the caching of terms
in the query needs to be added for s-expression queries.
2021-09-04 17:07:19 -07:00
David Bremner
bafc307190 lib/parse-sexp: handle unprefixed terms.
This is equivalent to adding the same field name "" for multiple
prefixes in the Xapian query parser, but we have to explicitely
construct the resulting query.
2021-09-04 17:07:19 -07:00
David Bremner
0ca4ad2670 lib/parse-sexp: add '*' as syntactic sugar for '(starts-with "")'
Users that insist on using a literal '*' as a tag, can continue to do
so by quoting it when searching.
2021-09-04 17:07:19 -07:00