Commit graph

1601 commits

Author SHA1 Message Date
David Bremner
f48d2e2ff8 lib/message: catch exceptions in _n_m_add_term
Some code movement is needed to make sure the cache is only
invalidated when the Xapian operation succeeds.
2022-06-25 12:55:02 -03:00
David Bremner
2707c06a0f test: define test_private_C
When testing error handling, it is sometimes difficult to cover a
particular error path deterministically. Introduce a test function to
allow calling lower level functions directly.
2022-06-25 12:55:02 -03:00
David Bremner
bfcf9a6c10 CLI: set NOTMUCH_CONFIG in hooks.
This addresses a bug report / feature request of Uwe Kleine-König. The
assumption is that we always load a config file in the CLI (i.e. we
never pass "" as the config file argument to
notmuch_database_open_with_config).

[1]: id:8baa58c3-7ab9-ec03-1bbd-28aa5be838f2@kleine-koenig.org
2022-06-18 08:24:46 -03:00
David Bremner
e167825bca test: add known broken tests for setting NOTMUCH_CONFIG in hooks
Setting this according to --config was requested by Uwe
Kleine-König. There are some other ways that the configuration file
might be found in current notmuch, so check those as well.

As a bonus, fix a bug in the hook tests that left NOTMUCH_CONFIG set
even though a config file was provided via NOTMUCH_PROFILE.

[1]: id:8baa58c3-7ab9-ec03-1bbd-28aa5be838f2@kleine-koenig.org
2022-06-18 08:23:31 -03:00
David Bremner
7d1e5a5348 CLI/git: add safety checks for checkout and commit
Commits or checkouts that modify a large fraction of the messages in
the database should be relatively rare (and in some automated process,
probably non-existent). For initial setup, where such operations are
expected, the user can pass --force.
2022-06-17 08:40:19 -03:00
David Bremner
99e85823c8 CLI/git: support configuration for repo location / prefix
This is probably more convenient than always passing a command line
argument.

Use notmuch-config for consistency with other notmuch CLI tools.

Now that there is something relevant in the config files, test the
--config option.
2022-06-17 08:40:19 -03:00
David Bremner
8ed68c1bbe CLI/git: change defaults for repo and prefix
The previous defaults were not suitable for personal (i.e. not
bugtracking for notmuch development) use.

Provide two ways for the user to select nmbug compatible defaults;
command line argument and checking the name of the script.
2022-06-17 08:40:19 -03:00
David Bremner
66ccf420c2 CLI/git: cache git indices
If the private index file matches a previously known revision of the
database, we can update the index incrementally using the recorded
lastmod counter. This is typically much faster than a full update,
although it could be slower in the case of large changes to the
database.

The "git-read-tree HEAD" is also a bottleneck, but unfortunately
sometimes is needed. Cache the index checksum and hash to reduce the
number of times the operation is run. The overall design is a
simplified version of the PrivateIndex class.
2022-06-17 08:40:19 -03:00
David Bremner
f67d7c9e7a CLI/git: replace enumeration of tags with sexp query.
Unlike the (current) infix query parser provided by Xapian, the
notmuch specific sexp query parser supports prefixed wildcard queries,
so use those. In addition to being somewhat faster, this avoids
needing to escape all of the user's tags to pass via the shell.
2022-06-17 08:40:19 -03:00
David Bremner
b355973abc test/git: add known broken test for tag with quotes.
There is current insufficient sanitization and/or escaping of tag names
internally in notmuch-git.
2022-06-17 08:40:19 -03:00
David Bremner
b010584163 CLI/git: rename environment variables.
Although the code required to support both new and old environment
variables is small, it complicates the semantics of configuration, and
make the documentation harder to follow.
2022-06-17 08:40:19 -03:00
David Bremner
a85c82f99a test: initial tests for notmuch-git
Exercise the main functionality of notmuch-git.  add_git_repos() will
hopefully be simplifed when an init subcommand is added.
2022-06-17 08:40:19 -03:00
jao
9695e4c38d emacs: use message-dont-reply-to-names when composing replies
notmuch-mua functions for replies now use the built-in customizable
variable message-dont-reply-to-names with the same semantics as
message-mode.
2022-06-16 08:51:19 -03:00
Leo Okawa Ericson
fb4a0967ca emacs: Make notmuch-show-next-thread return nil on failure
Having notmuch-show-next-thread return non-nil on success and nil on
failure makes it easier for users to interact with notmuch via elisp.

This commit changes notmuch-search-show-thread too since the return
value of notmuch-show-next-thread depends on notmuch-search-show-thread.

Amended by db: fix whitespace in T450-emacs-show
2022-06-13 08:06:51 -03:00
David Bremner
5cbddf7e7a CLI: pass --config to external commands via NOTMUCH_CONFIG.
This makes `notmuch --config foo external-subcommand` work
consistently with the built in subcommands.
2022-06-13 07:28:05 -03:00
David Bremner
eaa5199f5b test: initial tests for external commands
The main goal is to test the setting of NOTMUCH_CONFIG, but also
include a basic sanity test for execing scripts.
2022-06-13 07:27:06 -03:00
David Bremner
eab665c573 test: replace deprecated use of notmuch_database_create
Like notmuch_database_open, we need to conditionally print the error
message.
2022-05-29 07:36:59 -03:00
David Bremner
32f299fe13 test: replace deprecated use of notmuch_database_open
This is a bit more involved than replacing the use of
notmuch_database_open_verbose, as we have to effectively inline the
definition of notmuch_database_open.
2022-05-29 07:36:44 -03:00
David Bremner
f6e7a9dde5 test: replace deprecated use of notmuch_database_open_verbose
We need to do it some day, and it is a bit annoying to read
deprecation messages in broken tests.
2022-05-29 07:36:32 -03:00
Michael J Gruber
aec72e5806 test: make T450 independent of application/octet-stream interpretation
The actual content type of `application/octet-stream` is up to content
type detection of the reader, and thus may not be stable across
implementations or versions. This showed up when

fd46fc19 ("emacs:  document/defcustom notmuch-multipart/alternative-discouraged", 2022-05-14)

introduced a test for omitting a part of type `text/html` because it
expected a part of type `application/octet-stream` to remain in place,
i.e. a part of "unstable type". In particular, tests with `fd46fc19`
would succeed on RHEL/EPEL but fail on all current Fedoras with

```
 FAIL   multipart/alternative hides html by default
	--- T450-emacs-show.16.notmuch-show-multipart-alternative	2022-05-26 15:34:42.100557244 +0000
	+++ T450-emacs-show.16.OUTPUT	2022-05-26 15:34:42.102557207 +0000
	@@ -24,7 +24,7 @@
	 uses 64 as the
	 buffer size.
	 [ text/html (hidden) ]
	-[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: application/octet-stream (as text/x-diff) ]
	+[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: application/octet-stream (as text/x-patch) ]
	 From e3bc4bbd7b9d0d086816ab5f8f2d6ffea1dd3ea4 Mon Sep 17 00:00:00 2001
	 From: Alexander Botero-Lowry <alex.boterolowry@gmail.com>
	 Date: Tue, 17 Nov 2009 11:30:39 -0800
```

due to the different type detected.

Fix this by giving that message a specicific type of `text/x-diff` in
the test corpus, and adjust all affected test outputs.

Signed-off-by: Michael J Gruber <git@grubix.eu>
Amended-by: db, fix some trailing whitespace
2022-05-29 07:23:32 -03:00
David Bremner
fd46fc1977 emacs: document/defcustom notmuch-multipart/alternative-discouraged
This variable is important for people who want to change the default
behaviour when displaying multipart/alternative messages.  Previously
it was undocumented.  Add a defcustom to help users and copy some
documentation from the wiki. The usual machinery of re-using
docstrings is a bit tricky to use here because it mangles the example
lisp code, and the link to the info node should not be in e.g. the
html page.

Add a simple test to make sure the switch from defvar to defcustom did
not break something obvious.
2022-05-20 22:40:32 -03:00
David Bremner
90a7c1af36 emacs/reply: restrict what mime types are inlined by default
Apply the previously factored-out function used by notmuch-show.
2022-05-16 07:12:47 -03:00
David Bremner
e663231681 test: add known broken test for inlining in emacs/reply
The same mitigation already applied for show should also be applied
for reply
2022-05-16 07:11:18 -03:00
David Bremner
21b2d7dd4b test/emacs: add regression test for display of tar attachments
These should continue to display as buttons by default, even as we
update the handling for reply.
2022-05-16 07:11:04 -03:00
David Bremner
b884d7e2f5 test: start corpus for attachments
Initially these are to test the emacs frontend.
2022-05-16 07:10:12 -03:00
David Bremner
a1d139de4d lib: add sexp: prefix to Xapian (infix) query parser.
This is analogous to the "infix" prefix provided by the s-expression
based query parser.
2022-04-15 08:25:46 -03:00
David Bremner
fc3bb11808 test/sexp: add test for and of stemmed terms.
Previously only singled stemmed terms were tested.
2022-04-15 08:25:19 -03:00
David Bremner
8723e707c1 test/smime: fix signature verification test with newer gmime.
The extra machinery to check for the actual output format is justified
by the possibility that distros may patch this newer output format
into older versions of gmime.

Amended-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Michael J Gruber <git@grubix.eu>
Amended-again-by: db
2022-04-13 07:55:22 -03:00
David Bremner
04b43dc455 test: unset XDG_DATA_HOME and MAILDIR for tests.
The some of the tests (and the library functions they exercise) that
rely on XDG_CONFIG_HOME also check XDG_DATA_HOME and MAILDIR.
2022-03-25 13:08:59 -06:00
David Bremner
8ed6a172b3 lib: do not phrase parse prefixed bracketed subexpressions
Since Xapian does not preserve quotes when passing the subquery to a
field processor, we have to make a guess as to what the user
intended. Here the added assumption is that a string surrounded by
parens is not intended to be a phrase.
2022-03-19 07:27:29 -03:00
David Bremner
c9ed87f39f test: known broken tests for bracketed terms in subject
The heuristics in the field processor currently incorrectly trigger
phrase parsing.
2022-03-19 07:27:00 -03:00
David Bremner
19cf3910a5 emacs: redirect undo to notmuch-tag-undo
The double remap is a bit ugly, but it seems better than adding
another layer of keymaps for those modes where notmuch-tag-undo makes
sense.
2022-02-26 08:00:11 -04:00
David Bremner
cc2c3a383d emacs: add notmuch-tag-undo
Keybindings are deferred to a future commit.
2022-02-26 07:58:14 -04:00
David Bremner
b21fa0e433 perf-test: allow running test_emacs from performance test suite.
test_require_external prereq has to move to test-lib-common.sh, and
the new shell functions print_emacs_header and time_emacs are provided.

The somewhat indirect way of printing the output is to avoid the extra
"" present on string values from emacsclient.
2022-02-25 08:38:03 -04:00
David Bremner
c4c6045058 test/emacs: split out tagging related tests
T310-emacs is one of the largest and longest running sets of
tests. Splitting out the tagging operations will help maintainability
as well as potentially improve the parallel running time of the test
suite. Some slowdown in running the tests sequentially may result
since there is repeated setup.
2022-02-25 08:37:50 -04:00
David Bremner
371fa28ced test: split variable settings to their own file
This allows sharing more variable settings between the (correctness)
tests and the performance-tests. Unfortunately it seems a bit tricky
to move settings to test-lib-common.sh, as that is sourced late in
test-lib.sh, and moving it earlier breaks things.
2022-02-25 08:37:14 -04:00
Tomi Ollila
08da7f25e5 removed use of 'echo -n' (and echo -n -e ...)
In most cases used printf %s ... instead.

echo -n > file  lines to create empty / truncate files were
changed to : > file lines, like done in in test-lib-emacs.sh

And one echo -n "  " replaced with use of sed "s/^/  /" in next line.
2022-02-21 09:36:39 -04:00
David Bremner
b320d3fb59 CLI/insert: escape envelope from
The idea is to do as little parsing and modification of the delivered
message as possible. Luckily the position of the "envelope header"
lets us escape it by replacing the first 5 characters of the stream
with a regular header name (with ':').
2022-02-19 22:11:35 -04:00
David Bremner
4a380371e7 test: add known broken test for insert with mbox as input
It seems reasonable that notmuch should try to avoid delivering
messages in formats it cannot index.
2022-02-19 20:15:10 -04:00
David Bremner
190d8a7711 test: start new corpus of test messages for indexing code
This particular message is not recognized by notmuch as mail, but is
fine according to e.g. mutt. The trigger for this bad behaviour seems
to be a second "From " ocurring at the beginning of the line but
inside an attachment.
2022-02-19 20:14:18 -04:00
David Bremner
9ddd13f758 python-cffi: use config_pairs API in ConfigIterator
This returns all of the config keys with non-empty values, not just
those that happen to be stored in the database.
2022-02-16 21:57:22 -04:00
David Bremner
e221a4531f test: known broken test for list(db.config) in python-cffi bindings
As of notmuch 0.34.2 [1], the python-cffi bindings make available the
configuration from both a config file and the database when accessing
Database.config like a dictionary.  It is therefore confusing that the
iterator operations only work on the configuration information stored
in the database.

[1]: d7f9572413
2022-02-16 21:55:45 -04:00
David Bremner
2c1d1107f5 lib: strip trailing '/' from pathnames (sexp queries).
This changes makes the sexp query parser consistent with the infix one
in ignoring trailing '/'. Here we do a bit better and ignore any
number of trailing '/'.
2022-01-27 07:48:27 -04:00
David Bremner
8358841c66 test: add multiple path, folder sexp query tests
This is mainly to make sure we get trailing / removal correct. Also
add regression test for path: in the infix parser matching the
existing one for folder:.
2022-01-27 07:48:27 -04:00
David Bremner
c62c22c9fb lib: drop trailing slash for path and folder searches (infix)
This resolves an old bug reported by David Edmondson in 2014. The fix
is only needed for the "boolean" case, as probabilistic / phrase
searching already ignores punctuation.

This fix is only for the infix (xapian provided) query parser.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
c73e273aaf test/sexp: tests for path, folder, including trailing '/' (sexp)
This duplicates the bug reported in [1], as well as adding some simple
regression tests for 'path' and 'folder' searches which were
previously missing for sexp syntax.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
053a390370 test: known broken tests for trailing / in path search (infix)
In [1], David Edmondson observed that the trailing / added by many
completion mechanisms causes path searches to silently fail.  This
test reproduces that bug for both `path:' and `folder:' searches.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
0756105832 test: sanitize generated message files names
It is fragile to encode the generated names into tests, as it makes
tests break when e.g. new tests are added.  There is a possibility
that this will hide certain failures; in that case meaningful filenames
should be chosen for the generated messages.
2022-01-27 07:48:27 -04:00
David Bremner
0a32741fce lib/parse-sexp: handle lastmod queries.
This particular choice of converting strings to integers requires C++11.
2022-01-26 07:41:02 -04:00
David Bremner
341016c8ec test/sexpr-search: add known broken tests for lastmod queries
These are loosely modelled on the tests just above for date ranges,
since the error conditions are similar. Some ideas also borrowed from
T570-revision-tracking.
2022-01-26 07:41:02 -04:00