Commit graph

1645 commits

Author SHA1 Message Date
David Bremner
a942cb8ee3 test: add two known broken tests for missing config files
The documentation claims that the database will be set to NULL in this
case, but it is currently not happening. Based on a reproducer [1]
from Austin Ray.

[1]: id:20211021190401.imirxau2ewke6e2m@athena
2021-10-30 13:42:16 -03:00
David Bremner
e2b842afb1 test/libconfig: use 'export' when restoring NOTMUCH_CONFIG
This is required when NOTMUCH_CONFIG has been unset, and is harmless
otherwise.
2021-10-27 22:05:45 -03:00
David Bremner
3e2e724d53 notmuch 0.33.2 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmFVoPUACgkQA0U5G1Wq
 FSGSzg//WY202qUibO6pgFMPvgpYCB5g9Bq2lSzEcJzc2ZqZIH/3WvFSUeHMrInb
 GmHzJ4Q1w00K/wyB8MLYILKJ5/Tikg+O63/YkZMjXx6CDwu9zyNdk3L7TuZjkiOB
 rRiCPlEO7vvzUKedyecpY07DpgCYku1R9eOk032aSZEdOVheXIUUwDcR/cSFhxjO
 9amPls9vjZXKOQjOH8dgbxyZX8JbcTECgwYoFCT1k88NrVXdcGc3ds/ZuJowf2S7
 rIOLCZVuJD0yPGtMUBMHr5bRJ4O6e4NsHLNOgELFid3WXmQ3TzrRBDuJX5rDaaOs
 +0aqFWS57lujZLF+dZbZAYqK9CnQxZct+Au8GJlvvHZUBnS2skTTK8zu4NuGmgh0
 wckeesSXImC/NxHP/HaRNvcmrOUPTWYr3SaNKAbAl6zSjkhgzYxMnTnAFev+7HAc
 TNa50iLfMYaM0lyzD5gzhZZKvk68wwi1bxxhNMmfOF94HTjKOvL1AWxPoQkYUNN8
 07yvxURcZHwng+ufIfr/XsTDleiqSkYRJFtkpyP6+ZoYrIj815qtqQ03lzJTcYRn
 wMv3ipMrLuYMeP8T3Pr0y8DEl2WVPRyybYbN+07SkQLULzmjBhivxn3abSRqOFKm
 zDn6XiY3BMhTOPaslVIxZdP/8k7fA7Ak5qTVkXLZCPMZiV36k0w=
 =mwj4
 -----END PGP SIGNATURE-----

Merge tag '0.33.2'

notmuch 0.33.2 release
2021-09-30 08:58:48 -03:00
David Bremner
4760b4470c test/T355-smime: Use key as exported by gpgsm
As reported in id:87h7pxiek3.fsf@tethera.net, the previous version of
the test is flaky. There is some so-far undebugged interaction between
openssl and gpgsm that causes the keys to fail to import. As a
potential workaround, use the key as exported by gpgsm, and eliminate
openssl from this particular pipeline.
2021-09-29 20:04:31 -03:00
David Bremner
21e365f51a test/emacs: tests for notmuch-{tree,unthreaded} with bad CWD
These work thanks to the previous wrapping of process creation
primitives.
2021-09-11 11:11:29 -03:00
David Bremner
e722b4f48c emacs: wrap call-process
Provide safe working directory
2021-09-11 11:11:29 -03:00
David Bremner
9fe36d7dc7 test/emacs: test for notmuch-show with nonexistent CWD
Somewhat predictably, the other code path in
notmuch-call-notmuch--helper also needs to be fixed.
2021-09-11 10:30:25 -03:00
David Bremner
eb226437e1 emacs: wrap make-process
Provide a safe working directory.
2021-09-11 10:27:38 -03:00
David Bremner
88224bde62 test/emacs: test for notmuch-search with nonexistent CWD
(At least) notmuch-start-notmuch needs to be updated to set a safe
working directory.
2021-09-11 10:22:24 -03:00
David Bremner
5e5f2122f9 emacs: wrap call-process-region
As with notmuch--process-lines, initial purpose is to provide a safe
binding for default-directory. This is enough to make notmuch-hello
robust against non-existent or corrupt values default-directory, but
probably not other views.
2021-09-11 10:19:27 -03:00
David Bremner
06477bce03 test/emacs: run notmuch-hello with a nonexisting default dir
This replicates the problem reported in Debian bug #922536.
2021-09-11 10:13:22 -03:00
David Bremner
59d0d5a489 test/emacs: provide macro test-log-error
Because of the way emacs reports errors, a test form can crash and not
change the main buffer. To work around this, capture both signalled
errors and any other messages.
2021-09-11 10:10:16 -03:00
David Bremner
7556bb7da2 notmuch 0.33.1 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmE7RAYACgkQA0U5G1Wq
 FSHDHA//Q5SJhT9fD/yi2QOI4zIHvF438WtAGIL+TnYIeY8lDmdYZaRjU9Ub4O3J
 eLxJMhNcOyY35nDEGOOVga33aS8lefI+q/Ozx5sNzSge9PRaha+hGAmY2UrkOm+z
 TQBjFsdnGyWdTmJbkC9VQsaQJWjeKUKM0J4eT/sWFrJfW9vHuGnzrEnpOTQu8xk1
 OWshsYc5AkMnnvUBv0JuCJ4z4HQkTHeMrMKwd+B0/SC4OxPQ/af6FaMkoKanH0Ej
 q661AaZW+WNupynS1nu/O99z5U4BJ1V7nz361degdINfaUh/vmcXgHN7jGeoFtj4
 rTjV/51Jy/5WCbYV+11a0+RMZOEoQZi4ZBai6E6WVHpztkDXXqxHyDnho+A4SYK9
 1G/Rs6lO9FYRLTvdYhJbLH4OZgglPWXFgWJJxGLY+5cM7d86g8LC2pbJGWZ5IbAW
 eB3LRFStaqsBqjx9noxt9BcdxmbEu4o7lCm6y+EdAD04X37MGvv+Mlwsz57EMG9c
 2NCntrc1BUUTb/7mCp6oZn0miPCmYFVLtTj0ZfLAdP89aLJVTn6a36uTv8NqcszR
 ZjBFqGxB9AMU0idTEHnHJ5acNj4bLPwDBPBGYmEWnky/Tn7ZT3XqEDVvOXY2HVwa
 0RkVsgMsUIqUXutXjUYlLcl4YgmgpTsRbVpMZJh0/R4N9spDyWU=
 =P0Xd
 -----END PGP SIGNATURE-----

Merge tag '0.33.1'

notmuch 0.33.1 release
2021-09-10 21:02:10 -03:00
David Bremner
060ff57642 lib: use 'localhost' rather than fqdn for default mail address.
As discussed in the thread starting at [1], the fully qualified domain
name is a bit tricky to get reproducibly, might reveal information
people prefer to keep private, and somewhat unlikely to provide
reliable mail routing.

The new approach of $current_username@localhost is better for the
first two considerations, and probably at least as good as a test mail
address.

[1]: id:87sfyibqhj.fsf@tethera.net
2021-09-10 08:17:06 -03:00
David Bremner
dc8262bd33 test/emacs: test for functions in notmuch-search-result-format.
Based on the commit message in id:20210221151902.2301690-3-dme@dme.org

Add the function notmuch-test-result-flags to test-lib.el to avoid
repeating it in 3 T*.sh files.
2021-09-08 23:01:02 -03:00
David Bremner
4595a814c2 CLI/tag: enable sexp queries
We have to rewrite _optimize_tag_query here because it is generating
a query string in the infix Xapian syntax. Luckily this is easy to do
with the sexp query syntax.
2021-09-04 17:07:19 -07:00
David Bremner
2944d59133 CLI/{count, dump, reindex, reply, show}: enable sexp queries
The change in each case is to call notmuch_query_create_with_syntax,
relying on the already inherited shared options.  As a bonus we get
improved error handling from the new query creation API.

The remaining subcommand is 'tag', which is a bit trickier.
2021-09-04 17:07:19 -07:00
David Bremner
551254eb76 lib/parse-sexp: apply macros
Macros implement lazy evaluation and lexical scope.  The former is
needed to make certain natural constructs work sensibly (e.g. (tag
,param)) but the latter is mainly future-proofing in case the DSL is
is extended to allow local bindings.

For technical background, see chapters 6 and 17 of [1] (or some other
intermediate programming languages textbook).

[1] http://cs.brown.edu/courses/cs173/2012/book/
2021-09-04 17:07:19 -07:00
David Bremner
9b9eb1d8bd lib/parse-sexp: support saved s-expression queries
It turns out there is not really much code in query-fp.cc useful for
supporting the new syntax. The code we could potentially factor out
amounts to calling notmuch_database_get_config; both the key
construction and the parsing of the results are specific to the query
syntax involved.
2021-09-04 17:07:19 -07:00
David Bremner
81b9dbd110 CLI/config support saving s-expression queries
This commit does not enable using saved s-expression queries, only
saving and retrieving them from the config file or the database. Use
in queries will be enabled in a following commit.
2021-09-04 17:07:19 -07:00
David Bremner
6ab2d9b1a2 lib/parse-sexp: handle saved queries
This provides functionality analogous to query: in the Xapian
QueryParser based parser. Perhaps counterintuitively, the saved
queries currently have to be in the original query syntax (i.e. not
s-expressions).
2021-09-04 17:07:19 -07:00
David Bremner
a07ef8abf5 lib/parse-sexp: parse user headers
One subtle aspect is the replacement of _find_prefix with
_notmuch_database_prefix, which understands user headers. Otherwise
the code mainly consists of creating a fake prefix record (since the
user prefixes are not in the prefix table) and error handling.
2021-09-04 17:07:19 -07:00
David Bremner
cc5992a304 lib/parse-sexp: support infix subqueries
This is necessary so that programs can take infix syntax queries from
a user and use the sexp query syntax to construct e.g. a refinement of
that query.
2021-09-04 17:07:19 -07:00
David Bremner
afe85e6578 lib/parse-sexp: expand queries
The code here is just gluing together _notmuch_query_expand with the
existing sexp parser infrastructure.
2021-09-04 17:07:19 -07:00
David Bremner
1870b3ae4b lib/parse-sexp: support regular expressions
At least to the degree that the Xapian QueryParser based parser
also supports them. Support short alias 'rx' as it seems to make more
complex queries nicer to read.
2021-09-04 17:07:19 -07:00
David Bremner
0b98ad5e4e lib/query: generalize exclude handling to s-expression queries
In fact most of the code path is in common, only the caching of terms
in the query needs to be added for s-expression queries.
2021-09-04 17:07:19 -07:00
David Bremner
bafc307190 lib/parse-sexp: handle unprefixed terms.
This is equivalent to adding the same field name "" for multiple
prefixes in the Xapian query parser, but we have to explicitely
construct the resulting query.
2021-09-04 17:07:19 -07:00
David Bremner
0ca4ad2670 lib/parse-sexp: add '*' as syntactic sugar for '(starts-with "")'
Users that insist on using a literal '*' as a tag, can continue to do
so by quoting it when searching.
2021-09-04 17:07:19 -07:00
David Bremner
011d06f4d6 lib/parse-sexp: 'starts-with' wildcard searches
The many tests potentially overkill, but they could catch typos in the
prefixes table. As a simplifying assumption, for now we assume a
single argument to the wildcard operator, as this matches the Xapian
semantics. The name 'starts-with' is chosen to emphasize the supported
case of wildcards in currrent (1.4.x) Xapian.
2021-09-04 17:07:19 -07:00
David Bremner
8322f536f5 lib/parse-sexp: add term prefix backed fields
We use "boolean" to describe fields that should generate terms
literally without stemming or phrase splitting.  This terminology
might not be ideal but it is already enshrined in
notmuch-search-terms(7).
2021-09-04 17:07:19 -07:00
David Bremner
90d9c2ad5c lib/parse-sexp: support phrase queries.
Anything that is quoted or not purely word characters is considered a
phrase.  Phrases are not stemmed, because the stems do not have
positional information in the database. It is less efficient to scan
the term twice, but it avoids a second pass to add prefixes, so maybe
it balances out. In any case, it seems unlikely query parsing is very
often a bottleneck.
2021-09-04 17:07:19 -07:00
David Bremner
200e164dc7 lib/parse-sexp: support subject field
The broken tests are because we do not yet handle phrase searches.
2021-09-04 17:07:19 -07:00
David Bremner
f83cd2a05a lib/parse-sexp: support and, not, and or.
All operations and (Xapian) fields will eventually have an entry in
the prefixes table. The flags field is just a placeholder for now, but
will eventually distinguish between various kinds of prefixes.
2021-09-04 17:07:19 -07:00
David Bremner
a2785c3919 lib/parse-sexp: stem unquoted atoms
This is somewhat less DWIM than the Xapian query parser, but it has
the advantage of simplicity.
2021-09-04 17:07:19 -07:00
David Bremner
be7e83de96 lib/parse-sexp: parse single terms and the empty list.
There is not much of a parser here yet, but it already does some
useful error reporting. Most functionality sketched in the
documentation is not implemented yet; detailed documentation will
follow with the implementation.
2021-09-04 17:07:19 -07:00
David Bremner
e3b1a0a6a5 CLI/search+address: support sexpr queries
Initially support selection of query syntax in two subcommands to
enable testing.
2021-09-04 17:07:19 -07:00
David Bremner
a83ad52da4 configure: optional library sfsexp
The configure part is essentially the same as the other checks using
pkg-config. Since the optional inclusion of this feature changes what
options are available to the user, include it in the "built_with"
pseudo-configuration keys.
2021-09-04 17:07:19 -07:00
David Bremner
d447b694b4 CLI: make variable n_requested_db_uuid file scope.
It turns out that now that we pass an open database into the
subcommands, it is easy to check any requested uuid against the
database at the same time as we process the other shared
arguments. This results in overall less boilerplate code, as well as
making a CLI scope function and variable file scope in notmuch.c.
2021-09-04 17:07:19 -07:00
David Bremner
06b536dae0 test: known broken test for emacs display of message/rfc822 parts
There are at least 3 bugs present.

1) notmuch-show-insert-part-message/rfc822 assumes that message/rfc822
parts will have a ":content" property, but that turns out not to be
the case.

2) something deep in gnus wants gnus-newsgroup-charset, but that is
defined in gnus-sum, which is not loaded by default.

3) If gnus-sum is loaded, then the display of the message/rfc822 part
succeeds, but the buffer gets put into gnus-article-mode, which means
that, inter alia, notmuch text properties and keybindings get wiped.
2021-08-30 16:29:04 -07:00
David Bremner
e1c56f49de test/crypto: test message with rfc822 attachment.
This is intended to help track down a display problem in the emacs
front end
2021-08-30 16:24:59 -07:00
Hannu Hartikainen
717e3dcdc3 lib: consider all instances of Delivered-To header
When using notmuch-reply and guessing the From: address from
Delivered-To headers, I had the wrong address chosen today. This was
because the messages from the notmuch list contain these headers in this
order:

Delivered-To: hannu.hartikainen@gmail.com
...
Delivered-To: hannu@hrtk.in

In my .notmuch-config I have the following configuration:

primary_email=hannu@hrtk.in
other_email=hannu.hartikainen@gmail.com;...

Before this change, notmuch-reply would guess From: @gmail.com because
that is the first Delivered-To header present. After the change, the
primary address is chosen as I would expect.
2021-08-29 18:10:08 -07:00
Hannu Hartikainen
8c0bebedc5 reply: add test for multiple Delivered-To headers
Add a known broken subtest for guessing From: correctly when there are
multiple Delivered-To: headers. The address configured as primary_email
should get picked.
2021-08-29 18:07:59 -07:00
David Bremner
731697d671 CLI: define and use format version 5
This is a bit of a cheat, since the format does not actually
change. On the other hand it is fairly common to do something like
this to shared libary SONAMEs when the ABI changes in some subtle way.
It does rely on the format-version argument being early enough on the
command line to generate a sensible error message.
2021-08-22 07:05:13 -07:00
David Bremner
4f84c01b3a test/path-config: use test_expect_equal_file_nonempty
This is more robust against crashes when the expected output is also
generated by notmuch. In the case where the expected output is
explicit, it seems like overkill.
2021-08-21 14:56:13 -07:00
David Bremner
474a7f8e65 test: add test_expect_equal_file_nonempty
A common bug in tests is that the code used to generate the EXPECTED
file fails, generating no output. When the code generating the OUTPUT
file fails in the same way, the test passes, even though there is a
failure being hidden. Add a new test function that guards against
this.
2021-08-21 14:55:41 -07:00
David Bremner
78045301e7 test: factor out test_diff_file_
A following commit will use this new function in a public test
function.
2021-08-21 14:55:27 -07:00
David Bremner
f2e6f311c7 test/expect_equal_file: whitespace cleanup
No functional change, just reindent to Emacs defaults.
2021-08-21 14:51:04 -07:00
David Bremner
3df2281746 notmuch release 0.32.3-1 for unstable (sid) [dgit]
[dgit distro=debian no-split --quilt=linear]
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmEclwwACgkQA0U5G1Wq
 FSEdBw//cF+LAJy9qyWX8axQcSwo1/ixZkyxSCc0JVaDhhAal5GdVd3E+vDKZp3Q
 KWExBKncTAciB1CooKNurnvFr30s9MX+SrDK0rPCqXtObMRYS96AEDCvul9Z9mBT
 eE6thb66Y2Mrr2dyHiyG/GHrp9wces+sSnB6xM6bqO16c9UsMuqjTIIqlb0A7oP8
 eN/LLPAcVoXIMry3hVQWzv6bJWfqQK9Ho66p1fH+QswgeeKIzh5HR3ZnGdsBhnyN
 n3l9QPYuhN7/85cjWH4AE0/2EQLIF/Ewu86qYHzvOufc7oiNXRGpzQtMNqqur/da
 JIOizaylhLkFP39i/7CHvldCTx0FOCVhbHtFwb1Hm7rPqr8DhOvXyZbDHK/LkWtg
 jBKVFld4nNv/zm1uz6S3E5rDFsxuvpMVuesWKj6Yb051QvfGqVTnwRHP3EFxMyk8
 CVle5a3mMg8bnlN9o/WchB3z3ybijK4HLjGB+FWDZMUvhKXkfZL6GUeo/EQ8byaX
 mDEx5u56euqgNQS1PnIF/nTaOCbu75IqYAUSqwVXBcICi7LfYKDwGezBiEhoRiKt
 RQoulN9FNS+HuL+85LN4TdZyJOvD1Rc9xdT9TwuLr47nIc/KdAWUxyYSEo4PMiZ+
 NNiCatI5kPJrp0q1A5xDRkpFavLu5Mtu9ore+Cf1nA18iEFNQwc=
 =cECE
 -----END PGP SIGNATURE-----

Merge tag 'debian/0.32.3-1'

notmuch release 0.32.3-1 for unstable (sid) [dgit]

[dgit distro=debian no-split --quilt=linear]
2021-08-18 21:46:42 -07:00
David Bremner
686230712f CLI/config: restore "notmuch config get built_with.*"
We need to special case the config section "built_with" because it is
not (currently) handled by the library. This seems consist with the
other sub-sub-commands 'list' and 'set'.
2021-08-17 17:09:56 -07:00
David Bremner
202dde04e1 test/config: add tests for built_with
The "get" test is known broken because this functionality was dropped
during the 0.32 config rewrite.
2021-08-17 17:09:38 -07:00
David Bremner
d930011690 lib/open: look in MAILDIR for database, as documented.
This fixes the bug id:87bl9lx864.fsf@kisara.moe
2021-08-17 17:09:21 -07:00
David Bremner
8376e81441 test: add known broken tests for finding database via MAILDIR
This highlights a bug reported by several users, including
Mohsin Kaleem [1].

The inconsistent use of test_begin_subtest_known_broken is because
some of these tests pass even though the database cannot be
located. This problem is left for a future commit.

[1]: id:87bl9lx864.fsf@kisara.moe
2021-08-17 17:07:53 -07:00
David Bremner
3fc2e5f23b test: revert to mkdir -p to make results directory.
As suggested by id:m21r7al3mt.fsf@guru.guru-group.fi
2021-08-03 20:35:13 -03:00
David Bremner
c966fa6439 test: use --minversion to detect GNU Parallel.
Based on a suggestion of Ole Tange [1].

[1]: id:CA+4vN7x6jp4HCiEybZ=5g+2X6Pa7etBFX3Bbd=UYty37gR6wEQ@mail.gmail.com
2021-08-03 20:29:47 -03:00
jao
13d49b8e16 CLI/show: tests for the new --sort option
New unit tests for notmuch show --sort, covering the basic use cases.
2021-07-03 20:27:39 -03:00
Michael J Gruber
1768b0c416 test: deduplicate T590
Test numbers are a concise way to communicate about tests and to remeber
them. Currently, there is one pait of duplicates:

T590-libconfig.sh
T590-thread-breakage.sh

Renumber the latter one to 592 since this keeps the alphabetic order and
leaves room in between.

Signed-off-by: Michael J Gruber <git@grubix.eu>
2021-06-29 13:43:24 -03:00
David Bremner
37f84d6d21 emacs/tree use notmuch-show-single-message
This is more efficient than notmuch-show-only-matching-messages, since
we do not parse the potentially large thread structure to find a
single message.

This is only a partial fix for notmuch-tree view, because displaying
the thread structure in the tree-mode window still crashes on long
threads. It is however enough to make unthreaded view handle long
threads.
2021-06-27 14:41:23 -03:00
David Bremner
14326e6cb7 test: start test file for emacs unthreaded view.
Initial focus is on behaviour with large threads.

The second test replicates a bug reported by Alan Schmitt in
id:87lf7sojbq.fsf@m4x.org.
2021-06-27 14:29:08 -03:00
Felipe Contreras
814abafc3e ruby: add keyword arguments to db.query
That way we don't need pass them to the query object ourselves.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-06-27 14:13:03 -03:00
David Bremner
e2a3e5fa51 lib: autocommit after some number of completed transactions
This change addresses two known issues with large sets of changes to
the database.  The first is that as reported by Steven Allen [1],
notmuch commits are not "flushed" when they complete, which means that
if there is an open transaction when the database closes (or e.g. the
program crashes) then all changes since the last commit will be
discarded (nothing is irrecoverably lost for "notmuch new", as the
indexing process just restarts next time it is run).  This does not
really "fix" the issue reported in [1]; that seems rather difficult
given how transactions work in Xapian. On the other hand, with the
default settings, this should mean one only loses less than a minutes
worth of work.  The second issue is the occasionally reported "storm"
of disk writes when notmuch finishes. I don't yet have a test for
this, but I think committing as we go should reduce the amount of work
when finalizing the database.

[1]: id:20151025210215.GA3754@stebalien.com
2021-06-27 14:03:00 -03:00
David Bremner
8aabddb043 test: add known broken test for closing with open transaction
The expected output may need adjusting, but what is clear is that
saving none of the changes is not desirable.
2021-06-27 14:01:26 -03:00
David Bremner
2f608d2a94 lib/config: add NOTMUCH_CONFIG_AUTOCOMMIT
This will be used to control how often atomic transactions are
committed.
2021-06-27 13:59:42 -03:00
Daniel Kahn Gillmor
8c29a5da09 cli/show: produce "email" element in sigstatus
When the certificate that signs a message is known to be valid, GMime
is capable of reporting on the e-mail address embedded in the
certificate.

We pass this information along to the caller of "notmuch show", as
often only the e-mail address of the certificate has actually been
checked/verified.

Furthermore, signature verification should probably at some point
compare the e-mail address of the caller against the sender address of
the message itself.  Having to parse what gmime thinks is a "userid"
to extract an e-mail address seems clunky and unnecessary if gmime
already thinks it knows what the e-mail address is.

See id:878s41ax6t.fsf@fifthhorseman.net for more motivation and discussion.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2021-06-26 13:07:47 -03:00
David Bremner
7406abf5e0 test/libconfig: make indexing more consistent
The other tests have converted to two digit decimal, the commit does
the same to the remaining test that was printing out indices in hex.
2021-06-21 06:57:17 -03:00
David Bremner
8dbd5deb8d Merge branch 'release' 2021-06-20 17:15:20 -03:00
David Bremner
a19d2b0d2b test: check openssl prequisite for add_gpgsm_home
This is a fix for the test failures reported by Dan Čermák [1].

It is more robust to check for the prerequisite inside the function
that uses it, rather than in every test file that calls the function.

[1]: id:87k0n4fqgm.fsf@tethera.net
2021-06-08 17:15:46 -03:00
Tomi Ollila
6f0f83660e test: aggregate-results updates
notmuch-test will now call aggregate-results.sh with file list
that it compiles based on the test ran, and aggregate-results
will report failure is any of the test files are missing.

With this notmuch-test no longer has to exit in non-parallel
run if some test fail to write its report file -- so it works
as parallel tests in this sense.

Changed test_done() in test-lib.sh write report file in one write(2),
so there is (even) less chance it being partially written. Also,
now it writes 'total' last and aggregate-results.sh expects this
line to exist in all report files for reporting to be successful.

Added 'set -eu' to notmuch-test and modified code to work with
these settings. That makes it harder to get mistakes slipped
into committed code.
2021-06-07 20:16:33 -03:00
Jani Nikula
87234a8ef3 emacs: update default mailing list archives for stashing links
Gmane web interface is long gone, remove it. Make MARC the new
default. Update LKML to Lore, where it already redirects anyway. Also
add Notmuch web archive.
2021-06-05 15:39:53 -03:00
David Bremner
2c85cc29bd test: add known broken test for indexing RFC822 group names
Austin Clements diagnosed this indexing problem in [1].

[1]: id:20130711215207.GR2214@mit.edu
2021-06-05 15:31:03 -03:00
Tomi Ollila
572af27950 test: source $NOTMUCH_SRCDIR/test/test-lib-emacs.sh
Sourcing test-lib.sh will cd to TMP_DIRECTORY, so
relative path in $0 will not work in previous version
 . $(dirname "$0")/test-lib-emacs.sh

Now individual test scripts -- e.g. ./test/T310-emacs.sh
will work.
2021-06-03 09:29:27 -03:00
Felipe Contreras
85ae2bcf56 ruby: use notmuch_exclude_t enum
It exists since 2013, let's allow it to be used in Ruby.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-23 09:05:33 -03:00
David Bremner
f44c83c083 test: add regression test for large number of deletions
Prior to 9ad19e4454 there was an unhandled Xapian exception when
reindexing after a large number of deletes.  This test was used for
bisection, and will subsequently serve as a regression test.
2021-05-23 08:31:36 -03:00
Tomi Ollila
f2533b9e73 test: say_color() in one write(2)
say_color() used to call (builtin) printf (and tput(1) to stdout)
several times, which caused attempts to write messages with color
to have partial content (e.g. escape sequences) often intermixed
with other tests when parallel tests were run.

Now, with all output collected, then written out using one
printf, all strings with color print out correctly
((at least short) write(2)'s appear to write out "atomically").

While at it, used only one tput(1) execution to determine whether
color output works, and made bold/colors/sgr0 to tput(1) their
values once per test.
2021-05-23 08:05:15 -03:00
David Bremner
69c2c930ec test: quiet some extra debugging output
This output does not cause test failures, but may make it harder to
interpret the output.
2021-05-23 08:04:07 -03:00
David Bremner
119a41b9ed test: ignore debugging messages
Previously building with "-DDEBUG" broke the test suite in several places.
2021-05-23 08:03:02 -03:00
Tomi Ollila
8675be1d30 test: replace notmuch_passwd_sanitize() with _libconfig_sanitize()
notmuch_passwd_sanitize() in test-lib.sh is too generic, it cannot
work in many cases...

The more specific version _libconfig_sanitize() replaces it in
T590-libconfig.sh and the code that uses it is modified to output
the keys (ascending numbers printed in hex) so the sanitizer knows
what to sanitize in which lines...

"@" + fqdn -> "@FQDN" replacement is used as fqdn could
-- in theory -- be substring of 'USERNAME'.

'user -> 'USER_FULL_NAME replacement to work in cases where user
is empty -- as only first ' is replaced that works as expected.

In addition to ".(none)" now also ".localdomain" is filtered from
USERNAME@FQDN.

/dev/fd/{n} is not defined in posix, but it is portable enough
(if it weren't it is easy to fix -- now code looks clearer).
2021-05-22 11:51:29 -03:00
David Bremner
702635d5f6 Merge branch 'release' 2021-05-22 09:34:55 -03:00
David Bremner
3f4de98e7c lib/n_d_index_file: re-use thread-id of existing message
This prevents the message document getting multiple thread-id terms
when there are multiple files with the same message-id.

This change shifts some thread ids, requiring adjustments to other tests.
2021-05-22 09:08:02 -03:00
David Bremner
349fc29803 test: add known broken test for duplicate thread-id terms
According to my bijection, this bug has been present since commit
411675a6ce in 2017. It is not completely clear what harm it causes in
regulary use, but it (at least) makes notmuch crash when compiled with
-DDEBUG_DATABASE_SANITY.
2021-05-22 09:02:58 -03:00
Felipe Contreras
92454bc093 test: split emacs functionality to its own file
This way it's easier to identify the tests that do require emacs stuff.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:29:04 -03:00
Felipe Contreras
bba4788aec test: emacs: fix a couple of shellcheck complaints
In test-lib-emacs.sh line 20:
    test_require_external_prereq ${TEST_EMACS} || ret=1
                                 ^-----------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean:
    test_require_external_prereq "${TEST_EMACS}" || ret=1

In test-lib-emacs.sh line 21:
    test_require_external_prereq ${TEST_EMACSCLIENT} || ret=1
                                 ^-----------------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean:
    test_require_external_prereq "${TEST_EMACSCLIENT}" || ret=1

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:28:44 -03:00
Felipe Contreras
29ba523275 test: emacs: check for configured emacs
Commit d59d9c81 (test: Make the emacsclient binary user-configurable,
2012-11-27) modified the prereq check for the configured emacsclient,
but we probably want to do the same for emacs itself.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:28:33 -03:00
Felipe Contreras
cd9ec20a49 test: emacs: simplify missing dependencies check
No functional changes.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:28:23 -03:00
Felipe Contreras
4561f694e9 test: more style fixes
In order to fit the git coding style.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:28:01 -03:00
Felipe Contreras
645bea13cb test: trivial style cleanups
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:27:52 -03:00
David Bremner
c84ccb70f3 Merge branch 'release' 2021-05-15 09:10:58 -03:00
David Bremner
b3258244c8 lib/open: restore default database path of $HOME/mail
Although this default worked for "notmuch config get", it didn't work
most other places. Restore the previous functionality, with the
wrinkle that XDG locations will shadow $HOME/mail if they exist.

This fixes a bug reported by Jack Kamm in id:87eeefdc8b.fsf@gmail.com
2021-05-15 08:40:05 -03:00
Kyle Meyer
319dcfb50e emacs: restore tag-changes and query bindings for tag hooks
notmuch-before-tag-hook and notmuch-after-tag-hook are supposed to
have access to two dynamic variables, tag-changes and query, but these
were lost with the switch to lexical binding in fc4cda07 (emacs: use
lexical-bindings in all libraries, 2021-01-13).

Add a variant of Emacs's dlet (not available until Emacs 28) and use
it in notmuch-tag to expose tag-changes and query to the hooks.
2021-05-15 08:34:28 -03:00
David Bremner
a663783343 test: add known broken tests for notuch-{before,after}-tag-hook
These tests illustrate the bug reported in id:87v97ytd2s.fsf@fastmail.fm
2021-05-15 08:31:00 -03:00
David Bremner
25da4a1340 Merge branch 'release' 2021-05-12 08:42:28 -03:00
David Bremner
393c92b042 lib/notmuch_database_reopen: reload some database metadata
In some uses of reopen, new documents and threads maybe have been
added, and e.g. compaction may have changed the uuid.
2021-05-12 08:40:04 -03:00
Michael J Gruber
0729c4e81f test: change database from within pre-new hook
Due to the change in the config system, notmuch keeps a notmuch database
open when it would not do so before. Consequently, it can miss changes
to the database which are done from a hook (while notmuch holds the
databse in read only mode). When notmuch itself writes to the database
after that it uses wrong assumptions about the last used doc id etc.

Demonstrate this by triggering an assertion. (This new test succeeds
with notmuch 0.31.4.)

Signed-off-by: Michael J Gruber <git@grubix.eu>
Amended-by: db. Check for both messages
2021-05-12 08:34:34 -03:00
David Bremner
b4a4ed0df9 Merge branch 'release' 2021-05-10 11:36:56 -03:00
David Bremner
8bf3311084 test: test explicit configuration of backup directory
Including the relative path that was broken until a recent commit.
2021-05-10 11:13:07 -03:00
David Bremner
891b950219 test: test relative paths for database.hook_dir 2021-05-10 11:13:07 -03:00
David Bremner
1040e7aa07 lib/config: expand relative paths when reading from database
This makes the treatment of relative paths consistent between the
database and config files.
2021-05-10 11:12:58 -03:00
David Bremner
322a492c77 test: add known broken test for relative setting of mail_root
The behaviour should not change depending on where the configuration
is stored.
2021-05-10 11:11:42 -03:00
David Bremner
31098c4ae4 lib/config: canonicalize paths relative to $HOME.
Prior to 0.32, notmuch had the (undocumented) behaviour that it
expanded a relative value of database.path with respect to $HOME. In
0.32 this was special cased for database.path but broken for
database.mail_root, which causes problems for at least notmuch-new
when database.path is set to a relative path.

The change in T030-config.sh reflects a user visible, but hopefully
harmless behaviour change; the expanded form of the paths will now be
printed by notmuch config.
2021-05-10 11:08:18 -03:00
David Bremner
ef0b126619 test: add known broken test for relative database path in new
This test highlights a bug introduced in 0.32. The new split between
path and mail_root does not properly canonicalize relative paths in
the latter.
2021-05-10 11:06:40 -03:00
Felipe Contreras
ac64de450a test: ruby: simplify output comparison
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-05 17:00:37 -03:00
Felipe Contreras
faf5511c3e test: ruby: don't use instance variables
Local variables are perfectly fine.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-05 17:00:16 -03:00
Felipe Contreras
c4b23cf29a test: ruby: use much more standard Ruby idioms
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-05 16:40:15 -03:00
Felipe Contreras
63413a5563 test: ruby: use much more standard puts
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-02 08:12:15 -03:00
Felipe Contreras
2dbc5fdf5a test: ruby: simplify test_ruby()
We always do test_expect_equal_file, so do it in test_ruby() directly.

The only subtest where we don't (get non-existent file) can be easily
modified.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-02 08:12:05 -03:00
Felipe Contreras
9f6bc01824 test: ruby: simplify MAIL_DIR initialization
There's no need to complicate the script passing the MAIL_DIR
environment variable.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-02 08:11:56 -03:00
Felipe Contreras
c1b99d6f94 test: ruby: simplify MAIL_DIR check
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-02 08:11:42 -03:00
Felipe Contreras
a75a9a5aed test: ruby: refactor test_ruby()
There's no point in repeating the same initialization in all the tests.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-02 08:11:17 -03:00
Felipe Contreras
8cbd8e71d0 test: move test_ruby() inside the only client
Not much point in polluting the main library, and also will be useful to
modify it in tandem with the tests.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-02 08:11:04 -03:00
Felipe Contreras
fe9616aef1 test: add external prereqs to many emacs tests
The tests fail otherwise.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-01 21:15:27 -03:00
Felipe Contreras
049a652da7 test: add prereqs check in test_emacs_expect_t
test_emacs may update the external prereqs, in which case we want to
skip the test rather than fail.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-01 21:15:10 -03:00
Felipe Contreras
7579f3ac6b test: fix wrong SKIP messages
When the external prereqs are updated inside the body of the command
(e.g. test_emacs) the message in test_report_skip_ is wrong: it outputs
the body of the command instead of the subtest name.

We need to pass the same argument we pass to test_skip.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-01 21:14:55 -03:00
Felipe Contreras
0b48e65526 test: use correct fqdn in passwd_sanitize()
My fqdn is 'natae.localdomain', however, socket.getfqdn() returns
'localhost'.

To fetch the true fqdn we need socket.getaddrinfo().

For more information see: https://stackoverflow.com/a/11580042/10474

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-01 21:14:37 -03:00
Felipe Contreras
2add9ce722 test: unset NAME environment variable
Otherwise the output from the tests would be different.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-01 21:13:41 -03:00
Felipe Contreras
ff5f141bf2 test: fix passwd_sanitize()
If any of the variables is empty the output is completely messed up,
because replace("", "FOO") puts "FOO" before every single character.

I don't have my full name configured, and this is what I get:

  USER_FULL_NAME=USER_FULL_NAME=USER_FULL_NAME USER_FULL_NAMEsUSER_FULL_NAMEtUSER_FULL_NAMEdUSER_FULL_NAMEoUSER_FULL_NAMEuUSER_FULL_NAMEtUSER_FULL_NAME USER_FULL_NAME=USER_FULL_NAME=USER_FULL_NAME

Let's check for empty strings before doing any replace.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-01 21:13:23 -03:00
David Bremner
00487fa901 test: hide message from moreutils parallel.
The argument --version confuses moreutils parallel, but this is OK,
because its confusing does not include printing "GNU"
2021-04-26 09:00:11 -03:00
Tobias Backer Dirks
f2b5ad28e2 test: check for GNU parallel with --version options
The lastest versions of GNU parallel no longer make mention of GNU
within their help output. This causes the test script to mistakenly use
the moreutils parallel execution. In order to fix this, while
maintaining compatibility with previous versions of GNU parallel,
--version should be used.

Signed-off-by: Tobias Backer Dirks <omgitsaheadcrab@gmail.com>
2021-04-24 20:06:41 -03:00
David Bremner
27a2c2c80d test: replace notmuch_passwd_sanitize with python version
Apparently the -f option to hostname is not portable, and in fact it
does not seem to always behave reasonably in e.g. a chroot.

Python code originally due to Tomi [1], modified by yours truly.

[1]: id:m2lf9fbkug.fsf@guru.guru-group.fi
2021-04-24 12:11:22 -03:00
Đoàn Trần Công Danh
828a004921 test-lib: unset XDG_CONFIG_HOME
lib/open.cc:_load_key_file will only open xdg-config files in
$XDG_CONFIG_HOME if it's defined, $HOME/.config will be considered if
and only if XDG_CONFIG_HOME not defined.

Let's unset said variable before running the test.
2021-04-18 10:00:10 -03:00
David Bremner
5248f55d5f test: put shim at end of LD_PRELOAD path
Certain tools like the address-sanitizer fail if they are not the
first LD_PRELOADed library. It does not seem to matter for our shims,
as long as they are loaded before libnotmuch.
2021-04-16 09:06:39 -03:00
David Bremner
ec4b7efbe4 CLI/config: remove calls to notmuch_config_open from top level
This will allow simplifying the subcommand interface.

Change the internal API to notmuch_config_open to not tie it to the
implementation of subcommands in notmuch.c.

It also fixes a previously broken test, since notmuch_config_open does
not understand the notion of the empty string as a config file name.
2021-04-06 21:32:36 -03:00
David Bremner
77d4b26d3e CLI/config: support set/get with split configuration
There are two small code changes. The main one is to retrieve the
possibly updated config file name found during the database opening
call. The second change is to allow empty config file names, as
a (currently broken) way of specifying that configuration should only
be taken from the database.
2021-03-27 09:26:14 -03:00
David Bremner
217f819608 CLI+lib: detect missing database in split configurations.
Eventually we want to do all opening of databases in the top
level (main function). This means that detection of missing databases
needs to move out of subcommands. It also requires updating the
library to use the new NO_DATABASE status code.
2021-03-27 09:26:14 -03:00
David Bremner
dd9112e7d8 CLI/config: default to storing all config in external files
Previously the fact that some configuration options were only stored
in the database (and thus editing the config file had no effect) was a
source of user confusion. This was fixed with the series ending at
d9af0af164.

On the other hand, the underlying partition of config options into
those stored by default in the database and those stored in the config
file remained. This is also confusing, since now some invocations of
"notmuch config set" modify the config file, and others silently
modify the database instead.

With this commit, it is up to the user to decide which configuration
to modify. A new "--database" option is provided for notmuch config to
enable modifying the configuration information in the database;
otherwise the default is to update an external config file.
2021-03-27 09:26:14 -03:00
David Bremner
f055e94562 CLI/config: use notmuch_database_reopen
This allows `notmuch config` to use the common configuration
information carried by the notmuch_database_t opened at the top level.
2021-03-27 09:26:14 -03:00
David Bremner
b76da87d29 CLI/config: migrate notmuch_config_open to new config
notmuch_config_open will be preserved in the medium term for use by
the commands that are manipulating the config file directly (config
and setup)
2021-03-27 09:26:14 -03:00
David Bremner
3787fe6c84 CLI/config: switch "notmuch config list" to merged config
Use the database opened at the top level rather than opening another
notmuch_database_t.

Test output changes because keys are now listed in alphabetical order,
and because a missing database is no longer an error.
2021-03-27 09:26:14 -03:00
David Bremner
ee0e1cfb17 test/setup: check file output instead of notmuch config list
This allows verification of comments added to new / updated config
file.
2021-03-27 09:26:14 -03:00
David Bremner
ecb91d056c CLI/config: use merged config for "config get"
This commit starts the conversion of notmuch-config.c
functionality (as opposed to just interface) to the new config
framework.

The change to T030-config is because of the move of the
canonicalization database paths from the notmuch_config_t accessor to
the internal function _choose_database_path.
2021-03-27 09:26:14 -03:00
David Bremner
2e9ee47072 lib/config: set default for primary user email
This is mainly copying code from the CLI into the lib. The CLI copy
will be deleted in a later commit.
2021-03-27 09:26:14 -03:00
David Bremner
e81dc2072f lib/config: set defaults for user full name
This just copies code from from the CLI into the library. New test
infrastructure is needed because apparently we have never tested this
code path.
2021-03-27 09:26:14 -03:00
David Bremner
863b243185 lib/config: add config_pairs iterators
The layer of shims here seems a bit wasteful compared to just calling
the corresponding string map functions directly, but it allows control
over the API (calling with notmuch_database_t *) and flexibility for
future changes.
2021-03-27 09:26:14 -03:00
David Bremner
0c6db22930 lib/config: add notmuch_config_get_values_string
This is to support the less common (at least in the notmuch codebase)
case of accessing a ;-delimited list config value with an arbitrary
string key.
2021-03-27 09:26:14 -03:00
David Bremner
2fc40e24de lib: provide notmuch_database_load_config
This is mainly targetted at notmuch-config.c, or similar applications
that don't necessarily need both a configuration file and a database
to exist.
2021-03-27 09:26:14 -03:00
David Bremner
79a4b2e9e7 test: convert random-corpus to use n_d_open_with_config
Remove one more usage of notmuch_config_get_database_path
2021-03-27 09:26:14 -03:00
David Bremner
4e209ca99a CLI/new: use configuration variable for backup directory
The stat is essentially replaced by the mkdir for error detection
purposes.  This changes the default location for backups to make
things tidier, even in non-split configurations. Hopefully there is
not too many user scripts relying on the previous location.

Because the default location may not exist, replace the use of stat
for error detection with a call to mkdir.
2021-03-20 07:53:02 -03:00
David Bremner
a7873df331 lib/config: add configuration variable for backup directory
Like the hook directory, we primarily need a way to communicate this
directory between various components, but we may as well let the user
configure it.

Most of the diff is generalizing choose_hook_dir to work for both
backup and hook directories.
2021-03-20 07:51:36 -03:00
David Bremner
e3a4abc513 lib/open: fix hook directory calculation in split configuration
Choose sibling directory of xapian database, as .notmuch may not
exist.

libgen.h is already used in debugger.c, so it is not a new dependency
/ potential portability problem.
2021-03-20 07:50:13 -03:00
David Bremner
b30a59157d lib/compact: enable split config
This promotes _choose_xapian_path from static to extern linkage in
order to share between open.cc and database.cc.
2021-03-20 07:50:06 -03:00
David Bremner
74c3cc03c2 CLI/insert: support split database and mail root
The new test is in T055-path-config because it uses the helper
function split_config, and because it seems easier to put the
database path related tests in one place.
2021-03-20 07:44:08 -03:00
David Bremner
c82554193d lib/open: support XDG_DATA_HOME as a fallback database location.
This changes some error reporting, either intentionally by reporting
the highest level missing directory, or by side effect from looking in
XDG locations when given null database location.
2021-03-20 07:43:09 -03:00
David Bremner
2c879667b3 CLI/new: support split database and mail location
This adds new state variable for the mail root, and uses it most
places db_path was used. The notable exception is dumps from
backups. The latter will be dealt with properly in a future commit.
2021-03-20 07:42:06 -03:00
David Bremner
5ec6fd4dcf lib/open: check for split configuration when creating database.
The main functionality will be tested when notmuch-new is converted to
support split configuration. Here only the somewhat odd case of split
mail root which is actually symlinked to the database path is tested.
2021-03-20 07:41:04 -03:00
David Bremner
e823d05ae6 lib: support splitting mail from database location.
Introduce a new configuration value for the mail root, and use it to
locate mail messages in preference to the database.path (which
previously implied the mail messages were also in this location.

Initially only a subset of the CLI is tested in a split
configuration. Further changes will be needed for the remainder of the
CLI to work in split configurations.
2021-03-20 07:39:12 -03:00
David Bremner
986056bdbc lib/open: Use check for existing database by trial opening
This is a bit heavyweight for now, but it will make more sense when we
check multiple locations for the Xapian database.
2021-03-20 07:38:15 -03:00
David Bremner
cba540d6f5 lib/open: reuse directory checks from n_d_c_with_config
Make checks more uniform between creating new databases and opening
existing ones.
2021-03-20 07:24:59 -03:00
David Bremner
793d4305d3 lib/open: support NOTMUCH_DATABASE environment variable
The additional code is trivial, but making sure we get the priority of
various options correct takes a few tests.
2021-03-19 22:06:44 -03:00
David Bremner
6e6c319c26 CLI/show: complete conversion to new configuration framework.
In order to open the database in main() for this command, we may need
to re-open it in the (possibly less common) case where crypto options
require write access.
2021-03-19 22:06:19 -03:00
David Bremner
a9f74aeeb9 CLI/new: drop the write lock to run the pre-new hook.
This fixes a bug reported in [1]. In principle it could be possible
avoid one of these reopens, but it complicates the logic in main with
respect to creating new databases.

[1]: id:9C1993DF-84BD-4199-A9C8-BADA98498812@bubblegen.co.uk
2021-03-18 22:58:02 -03:00
David Bremner
7d286f0272 test: Add tests for write access to database from hooks.
Recent changes to configuration handling meant the pre-new hook was
run while the database was open read only, limiting what could be done
in the hook. Add some known broken tests for this problem, as well as
a regression test for write access from the post-new hook.
2021-03-18 22:54:13 -03:00
David Bremner
13efbd0e1c lib: support reopening databases for write access.
In the future Xapian will apparently support this more conveniently
for the cases other than READ_ONLY => READ_ONLY

Conceptually this function seems to fit better in lib/open.cc;
database.cc is still large enough that moving the function makes
sense.
2021-03-18 08:04:06 -03:00
David Bremner
9a5406cc94 test/setup: add check for config exists, but no database.
This code path is not currently tested, and will need updating if the
location of the xapian database changes.
2021-03-18 08:03:19 -03:00
David Bremner
dcd4df2cc0 test: add known broken test for long directory bug
In [1] Gregor Zattler explained the results of his hard work
tracking down a bug in notmuch with long directories. This test
duplicates the bug.

[1]: id:20210317194728.GB5561@no.workgroup
2021-03-18 07:57:38 -03:00
uncrustify
df4c66f85d test: run uncrustify
This is the result of running:

    $ uncrustify --replace --config ../devel/uncrustify.cfg *.cc *.c *.h

in the test directory.
2021-03-13 08:45:34 -04:00
David Bremner
97fadd0645 test: clean up some extra whitespace.
The extra space is mainly just untidy.
2021-03-12 07:19:14 -04:00
Tomi Ollila
d12d91f828 test: T020-compact.sh: fix work directory location
Fix use of $TEST_DIRECTORY ($NOTMUCH_BUILDDIR/test/) with use of
$TMP_DIRECTORY ($NOTMUCH_BUILDDIR/test/tmp.T020-compact/ in case
of T020-compact.sh) as root directory where to write test files
and directories.
2021-03-12 07:16:01 -04:00
David Bremner
4c79a2dabe notmuch 0.31.4 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmAuVjQACgkQA0U5G1Wq
 FSG6RA//bDdAtsG7QlywGONVX1FOSHxAgppDVRvSDXuluPmGgWvc2T80awbonfT5
 AHy9co41L9484QbOzd29d9Ttu5O39JdQbo+DCAJ6y2vm1M1dLCZw3HXaZGfxoUZo
 9L9Agxg/rYIEso7dAehOs3rGsia46aj2MDCVCUuA4DkVFhkbQ2QidIH+l3VPdYqm
 +1XmuJwyftO7hMifFd9W9ifO9wrBcK9WtJ23feufMhqfBsS0ItLysOhZQx+QdxSj
 7GuA6qX6V7XlIWpdohIOMKmT9tGHMDUo6Qk5m8aSc2XmKAybAXRc+qr0Kg8EpvBF
 1d9SqNptcXQL6rORJxluXR/aCuCb7m8YDgxFVSrFcp/M/twbpDC3WzjmAV+RZpDP
 GBKDH06IGaOMOj9GaYYWCe9loGROzOIT4y04Ckukit0AWpmdbaKnkAkPdrxrMWEu
 a5v5KkkMYT42q0PFU5bDjwvq+8Afmzt7oMO72zXz0mfuFNtFzZCUN/AG9LtHoRfk
 Q2ks/xBppmtCxWRAHVWc7f+Gk1OzI2PLVWnXIe9gxGimSpgT2QPUvZcxLHT/XWD5
 XRrrxtkPjKB1v6tWOFEMPI6WxE1iBzoC9AdG2h48ZP5drlLObcAxhAg5AvHJGJSr
 8D4iSa9cKjHD6qxDSXChzF6BF4aDNhF2Flq9Cs9tZb8nyt+Ix18=
 =mAE/
 -----END PGP SIGNATURE-----

Merge tag '0.31.4'

notmuch 0.31.4 release
2021-02-18 08:47:53 -04:00
David Bremner
a37d5f5976 test: Fix race condition in T568-lib-thread.sh
The assignment of thread-ids is (apparently) non-deterministic in a
way that mostly seems to show up on multicore machines. In my tests
the number is different from that previously assumed by this test
about 15% of the time on a 50 thread (25 core) Xeon.

Since message id's are fixed, use a message known to be in the thread
of interest to pick out the correct thread-id.
2021-02-18 07:15:47 -04:00
Daniel Kahn Gillmor
7061e41cd0 python: convert shebangs to python3
This is the last bit of "python" left in the notmuch codebase.

https://www.python.org/dev/peps/pep-0394/#recommendation encourages
"third-party distributors" to use more-specific shebang lines.  I'm
not certain that the notmuch project itself is a "third-party
contributor" but I think this is a safe way to encourage people to use
python3 when they're developing notmuch.

We already have python3 explicitly elsewhere in the codebase for
developers (in nmbug).

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2021-02-15 16:32:31 -04:00
David Bremner
f61d88c6f4 CLI: use configured hook directory
This enables support for hooks outside the database directory.
It relies strongly on configuration information being usable between
closing the database and destroying it.
2021-02-06 19:57:55 -04:00
David Bremner
0345bc57a0 lib/open: set HOOK_DIR on open
This is a simple two step path search.  Most error checking is
deferred until running the hooks.
2021-02-06 19:56:13 -04:00
David Bremner
4922416ccc lib/config: add HOOK_DIR
The hook directory configuration needs to be kept in synch with the
other configuration information, so add scaffolding to support this at
database opening time.
2021-02-06 19:54:09 -04:00
David Bremner
0d3bef312d cli/new: convert to new config framework
In addition to the same type of changes as converting other
subcommands, add the possibility of creating a database at the top
level. It would probably make sense to use this for insert as well.
2021-02-06 19:52:11 -04:00
David Bremner
ac67cd84ee lib: introduce notmuch_database_create_with_config
This takes a config path parameter, and can use that to decide the
new database location.
2021-02-06 19:48:34 -04:00
David Bremner
b860be6a76 lib/config: add NOTMUCH_CONFIG_NEW_IGNORE
This will be needed by (at least) the conversion of notmuch-new.c to
the new config framework
2021-02-06 19:39:33 -04:00
David Bremner
f118ef3c3d cli/compact: convert to new configuration framework
Switch to the newly created API function notmuch_database_compact_db,
which takes the database opened in main().
2021-02-06 19:36:06 -04:00
David Bremner
c56dcea7e2 cli/tag: convert to new config framework.
In addition to changing configuration access, change talloc context
for allocation.
2021-02-06 19:30:20 -04:00
David Bremner
6c28039ebb CLI/show: mostly switch show to new config framework
This will need some cleanup when the transition completes, and we stop
passing notmuch_config_t structs to the subcommands.

Unlike the general case, we open the database in the subcommand, since
we don't know whether it should be opened read/write until we parse
the command line arguments.

Add a test to make sure passing config file on the command line is not
broken by these or future config related changes.
2021-02-06 19:27:54 -04:00
David Bremner
d613d10ddd CLI/{search,address}: convert to new configuration framework
Since we are already passing a search context around as a kind of
parameter block, add a new talloc context to that to replace relying
on 'config'.

Convert notmuch-search and notmuch-address at the same time, because
they share some code.

Add a test to make sure we don't break passing configuration as a
command line argument.
2021-02-06 19:23:28 -04:00
David Bremner
8588719eb1 CLI/reply: convert to new config framework
This is messier than some of the other conversions because the
extensive use of 'config' as a talloc context.
2021-02-06 19:21:23 -04:00
David Bremner
0ab0b48be2 cli/reindex: convert new config framework
The only non-trivial part is switching the talloc context for
query_string_from args from 'config' to 'notmuch'.
2021-02-06 19:19:18 -04:00
David Bremner
f994f0e7df CLI/insert: convert to new config framework.
The new talloc context is needed to run the hook at the very end of
the function. That in turn is needed so that this process gives up the
write lock on the database.
2021-02-06 19:17:34 -04:00
David Bremner
66adcd4f53 CLI/restore: convert to new config framework
Switch one configuration check to new n_c_get_bool function, and
switch use of config as talloc context to notmuch.
2021-02-06 19:15:53 -04:00
David Bremner
53f27aaf73 cli/dump: convert to new config framework
This conversion is trivial because the only configuration information
accessed by dump is that stored in the database (in order to dump
it). We do need to be careful to keep the write lock on the database
to ensure dump consistency.
2021-02-06 19:12:34 -04:00
David Bremner
acc6331baa CLI/count: switch to new configuration framework
The main effort is changing from the old argv style config list
iterators to the new more opaque ones provided by the library (and
backed by the database+file config cache).
2021-02-06 19:11:06 -04:00
David Bremner
d071828bd5 lib/config: make values iterators restartable
This is relatively cheap, and makes it easier to transform existing
code which uses arrays of pointers to store configuration lists.
2021-02-06 19:09:39 -04:00
David Bremner
fd6f8e6c30 lib/config: add config values iterator
This is intended to avoid duplicating the string splitting and
traversal code for all clients of the config API.
2021-02-06 19:08:12 -04:00
David Bremner
06a64cf0aa lib/open: load default values for known configuration keys.
This emulates the behaviour of notmuch_config_open defined in the CLI,
in that it fills in default values if they are not otherwise defined.
2021-02-06 19:03:36 -04:00
David Bremner
d6bd87a712 lib/config: add notmuch_config_key_{get,set}
By using an enum we can have better error detection than copy pasting
key strings around.

The question of what layer this belongs in is a bit
tricky. Historically most of the keys are defined by the CLI. On the
other hand features like excludes are supported in the
library/bindings, and it makes sense to configure them from the
library as well.

The somewhat long prefix for notmuch_config_t is to avoid collisions
with the existing usage in notmuch-client.h.
2021-02-06 19:02:00 -04:00
David Bremner
39580e2d7f lib/open: add support for config profiles and default locations
Fill in the remainder of the documented functionality for
n_d_open_with_config with respect to config file location. Similar
searching default locations of the database file still needs to be
added.
2021-02-06 18:59:00 -04:00
David Bremner
e5f3c3ed50 lib: add stub for notmuch_database_open_with_config
Initially document the intended API and copy the code from
notmuch_database_open_verbose. Most of the documented functionality is
not there yet.
2021-02-06 18:57:35 -04:00
David Bremner
4743e87c2c lib: cache configuration information from database
The main goal is to allow configuration information to be temporarily
overridden by a separate config file. That will require further
changes not in this commit.

The performance impact is unclear, and will depend on the balance
between number of queries and number of distinct metadata items read
on the first call to n_d_get_config.
2021-02-06 18:56:05 -04:00
David Bremner
6a7b61b1d5 test: add (back) upgrade tests
In ee897cab8b the upgrade tests from pre v3 databases were
removed. The reasons for that are still valid, but we should still
test the code paths that do the upgrade, and it is relatively
straightforward to do that for v3 to v3 upgrades.
2021-01-17 13:24:04 -04:00
David Bremner
15d8067c0a test/T391-python-cffi
Make bindings test verbose. This helps in debugging.
2021-01-15 07:43:23 -04:00
David Bremner
003fdba7a4 test/T750-gzip: don't compress the xapian database
This causes mysterious failures in trying to detect if a database
exists.
2021-01-15 07:43:07 -04:00
Jonas Bernoulli
2ca941163d emacs: make headings outline-minor-mode compatible
`outline-minor-mode' treats comments that begin with three or more
semicolons as headings.  That makes it very convenient to navigate
code and to show/hide parts of a file.

Elips libraries typically have four top-level sections, e.g.:

;;; notmuch.el --- run notmuch within emacs...
;;; Commentary:...
;;; Code:...
;;; notmuch.el ends here

In this package many libraries lack a "Commentary:" section, which is
not optimal but okay for most libraries, except major entry points.

Depending on how one chooses to look at it, the "... ends here" line
is not really a heading that begins a section, because it should never
have a "section" body (after all it marks eof).

If the file is rather short, then I left "Code:" as the only section
that contains code.  Otherwise I split the file into multiple sibling
sections.  The "Code:" section continues to contain `require' and
`declare-function' forms and other such "front matter".

If and only if I have split the code into multiple sections anyway,
then I also added an additional section named just "_" before the
`provide' form and shortly before the "...end here" line.  This
section could also be called "Back matter", but I feel it would be
distracting to be that explicit about it.  (The IMO unnecessary but
unfortunately still obligatory "... ends here" line is already
distracting enough as far as I am concerned.)

Before this commit some libraries already uses section headings, some
of them consistently.  When a library already had some headings, then
this commit often sticks to that style, even at the cost inconsistent
styling across all libraries.

A very limited number of variable and function definitions have to be
moved around because they would otherwise end up in sections they do
not belong into.

Sections, including but not limited to their heading, can and should
be further improved in the future.
2021-01-13 07:10:27 -04:00
David Bremner
1331888374 Merge branch 'release' 2020-12-25 12:56:57 -04:00
David Bremner
6d5d28c593 test/T360-symbol-hiding.sh: trim extra output from readelf
readelf on (at least) ppc64le sometimes generates some extension to
the Ndx name inside '[]'. Remove this output to allow our simple
column based parsing to work.
2020-12-25 11:36:33 -04:00
David Bremner
76871fcf5e test: add regression test for searching with alternate config
Make sure upcoming changes to config handling do not break command
line specification.
2020-12-23 09:23:42 -04:00
David Bremner
59488ee929 test: use keys with group 'test' in T590-libconfig
In a future commit we want to interoperate better with glib KeyFiles,
which need groups for all keys.
2020-12-20 07:46:18 -04:00
David Bremner
82faa059b8 test/T360-symbol-hiding: use readelf in place of nm
It turns out that using nm -P isn't as portable as hoped. In
particular with some ELF ABIs (e.g. ppc64 ELFv1), the desired symbols
end up in the data section instead of text.

The test is currently only functional on ELF based architectures, so I
think it's legit to depend on readelf instead of nm.

The switch to readelf has the advantage that we can explicitely ask
for all of the symbols with global visibility, rather than grepping
for notmuch. That seems a more robust approach since it will catch any
strangely named global symbols.
2020-12-13 08:23:42 -04:00
David Bremner
85b5803869 test: fix syntax errors in erroring calls to notmuch insert
notmuch insert does not currently support passing a filename for the
input, so all of these tests have an extra error in addition to the
one being tested for.

Currently this does not make a difference because the error being
tested for is caught before the error of an extra command line
argument. In the future it might make a difference, and in any case it
is confusing.
2020-09-04 21:04:33 -03:00
David Bremner
fe449f779d test: fix uninitialized variable use in T562-lib-database
Fix a copy paste error of using the boolean ret as a notmuch_status_t,
and uninitialized.
2020-08-16 14:04:36 -03:00
David Bremner
25f9a42287 test: update README to reflect dropping upgrade tests
These test databases have been unneeded since ee897cab8.
2020-08-16 10:41:44 -03:00
Teemu Likonen
adb90b9bb6 Emacs: Indent first header line only when indentation is turned on
Previously in message-show mode message's first header line (From
header) was always indented, even if user had turned thread
indentation off with "<" (notmuch-show-toggle-thread-indentation)
command.

This change modifies notmuch-show-insert-headerline function so that
it doesn't indent the first header line if notmuch-show-indent-content
variable is nil.

This change also modifies tests so that they expect this new output
format:
test/emacs-show.expected-output/notmuch-show-indent-thread-content-off
2020-08-15 09:16:34 -03:00
Jonas Bernoulli
3665914f71 emacs: Do not abuse advice to monkey patch while testing
Use `cl-letf*' instead.
2020-08-09 21:17:06 -03:00
Jonas Bernoulli
96baa22318 emacs: Drop old advices that were only need for Emacs 23 2020-08-09 21:16:12 -03:00
Jonas Bernoulli
bb15524c12 test: Fix indentation
Fix it to consistently match the style we have configured in
".dir-locals.el".
2020-08-09 21:14:36 -03:00
Jonas Bernoulli
6c84dee531 Fix typos 2020-08-09 21:14:36 -03:00
Jonas Bernoulli
73b8f0b8d7 emacs: Various cosmetic changes 2020-08-09 21:14:36 -03:00
Jonas Bernoulli
c454135376 emacs: Use makefile-gmake-mode in Makefile*s
Use `makefile-gmake-mode' instead of `makefile-mode' because the
former also highlights ifdef et al. while the latter does not.

"./Makefile.global" and one "Makefile.local" failed to specify any
major mode at all but doing so is necessary because Emacs does not
automatically figure out that these are Makefiles (of any flavor).
2020-08-09 21:14:36 -03:00