Commit graph

1594 commits

Author SHA1 Message Date
David Bremner
66ccf420c2 CLI/git: cache git indices
If the private index file matches a previously known revision of the
database, we can update the index incrementally using the recorded
lastmod counter. This is typically much faster than a full update,
although it could be slower in the case of large changes to the
database.

The "git-read-tree HEAD" is also a bottleneck, but unfortunately
sometimes is needed. Cache the index checksum and hash to reduce the
number of times the operation is run. The overall design is a
simplified version of the PrivateIndex class.
2022-06-17 08:40:19 -03:00
David Bremner
f67d7c9e7a CLI/git: replace enumeration of tags with sexp query.
Unlike the (current) infix query parser provided by Xapian, the
notmuch specific sexp query parser supports prefixed wildcard queries,
so use those. In addition to being somewhat faster, this avoids
needing to escape all of the user's tags to pass via the shell.
2022-06-17 08:40:19 -03:00
David Bremner
b355973abc test/git: add known broken test for tag with quotes.
There is current insufficient sanitization and/or escaping of tag names
internally in notmuch-git.
2022-06-17 08:40:19 -03:00
David Bremner
b010584163 CLI/git: rename environment variables.
Although the code required to support both new and old environment
variables is small, it complicates the semantics of configuration, and
make the documentation harder to follow.
2022-06-17 08:40:19 -03:00
David Bremner
a85c82f99a test: initial tests for notmuch-git
Exercise the main functionality of notmuch-git.  add_git_repos() will
hopefully be simplifed when an init subcommand is added.
2022-06-17 08:40:19 -03:00
jao
9695e4c38d emacs: use message-dont-reply-to-names when composing replies
notmuch-mua functions for replies now use the built-in customizable
variable message-dont-reply-to-names with the same semantics as
message-mode.
2022-06-16 08:51:19 -03:00
Leo Okawa Ericson
fb4a0967ca emacs: Make notmuch-show-next-thread return nil on failure
Having notmuch-show-next-thread return non-nil on success and nil on
failure makes it easier for users to interact with notmuch via elisp.

This commit changes notmuch-search-show-thread too since the return
value of notmuch-show-next-thread depends on notmuch-search-show-thread.

Amended by db: fix whitespace in T450-emacs-show
2022-06-13 08:06:51 -03:00
David Bremner
5cbddf7e7a CLI: pass --config to external commands via NOTMUCH_CONFIG.
This makes `notmuch --config foo external-subcommand` work
consistently with the built in subcommands.
2022-06-13 07:28:05 -03:00
David Bremner
eaa5199f5b test: initial tests for external commands
The main goal is to test the setting of NOTMUCH_CONFIG, but also
include a basic sanity test for execing scripts.
2022-06-13 07:27:06 -03:00
David Bremner
eab665c573 test: replace deprecated use of notmuch_database_create
Like notmuch_database_open, we need to conditionally print the error
message.
2022-05-29 07:36:59 -03:00
David Bremner
32f299fe13 test: replace deprecated use of notmuch_database_open
This is a bit more involved than replacing the use of
notmuch_database_open_verbose, as we have to effectively inline the
definition of notmuch_database_open.
2022-05-29 07:36:44 -03:00
David Bremner
f6e7a9dde5 test: replace deprecated use of notmuch_database_open_verbose
We need to do it some day, and it is a bit annoying to read
deprecation messages in broken tests.
2022-05-29 07:36:32 -03:00
Michael J Gruber
aec72e5806 test: make T450 independent of application/octet-stream interpretation
The actual content type of `application/octet-stream` is up to content
type detection of the reader, and thus may not be stable across
implementations or versions. This showed up when

fd46fc19 ("emacs:  document/defcustom notmuch-multipart/alternative-discouraged", 2022-05-14)

introduced a test for omitting a part of type `text/html` because it
expected a part of type `application/octet-stream` to remain in place,
i.e. a part of "unstable type". In particular, tests with `fd46fc19`
would succeed on RHEL/EPEL but fail on all current Fedoras with

```
 FAIL   multipart/alternative hides html by default
	--- T450-emacs-show.16.notmuch-show-multipart-alternative	2022-05-26 15:34:42.100557244 +0000
	+++ T450-emacs-show.16.OUTPUT	2022-05-26 15:34:42.102557207 +0000
	@@ -24,7 +24,7 @@
	 uses 64 as the
	 buffer size.
	 [ text/html (hidden) ]
	-[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: application/octet-stream (as text/x-diff) ]
	+[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: application/octet-stream (as text/x-patch) ]
	 From e3bc4bbd7b9d0d086816ab5f8f2d6ffea1dd3ea4 Mon Sep 17 00:00:00 2001
	 From: Alexander Botero-Lowry <alex.boterolowry@gmail.com>
	 Date: Tue, 17 Nov 2009 11:30:39 -0800
```

due to the different type detected.

Fix this by giving that message a specicific type of `text/x-diff` in
the test corpus, and adjust all affected test outputs.

Signed-off-by: Michael J Gruber <git@grubix.eu>
Amended-by: db, fix some trailing whitespace
2022-05-29 07:23:32 -03:00
David Bremner
fd46fc1977 emacs: document/defcustom notmuch-multipart/alternative-discouraged
This variable is important for people who want to change the default
behaviour when displaying multipart/alternative messages.  Previously
it was undocumented.  Add a defcustom to help users and copy some
documentation from the wiki. The usual machinery of re-using
docstrings is a bit tricky to use here because it mangles the example
lisp code, and the link to the info node should not be in e.g. the
html page.

Add a simple test to make sure the switch from defvar to defcustom did
not break something obvious.
2022-05-20 22:40:32 -03:00
David Bremner
90a7c1af36 emacs/reply: restrict what mime types are inlined by default
Apply the previously factored-out function used by notmuch-show.
2022-05-16 07:12:47 -03:00
David Bremner
e663231681 test: add known broken test for inlining in emacs/reply
The same mitigation already applied for show should also be applied
for reply
2022-05-16 07:11:18 -03:00
David Bremner
21b2d7dd4b test/emacs: add regression test for display of tar attachments
These should continue to display as buttons by default, even as we
update the handling for reply.
2022-05-16 07:11:04 -03:00
David Bremner
b884d7e2f5 test: start corpus for attachments
Initially these are to test the emacs frontend.
2022-05-16 07:10:12 -03:00
David Bremner
a1d139de4d lib: add sexp: prefix to Xapian (infix) query parser.
This is analogous to the "infix" prefix provided by the s-expression
based query parser.
2022-04-15 08:25:46 -03:00
David Bremner
fc3bb11808 test/sexp: add test for and of stemmed terms.
Previously only singled stemmed terms were tested.
2022-04-15 08:25:19 -03:00
David Bremner
8723e707c1 test/smime: fix signature verification test with newer gmime.
The extra machinery to check for the actual output format is justified
by the possibility that distros may patch this newer output format
into older versions of gmime.

Amended-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Michael J Gruber <git@grubix.eu>
Amended-again-by: db
2022-04-13 07:55:22 -03:00
David Bremner
04b43dc455 test: unset XDG_DATA_HOME and MAILDIR for tests.
The some of the tests (and the library functions they exercise) that
rely on XDG_CONFIG_HOME also check XDG_DATA_HOME and MAILDIR.
2022-03-25 13:08:59 -06:00
David Bremner
8ed6a172b3 lib: do not phrase parse prefixed bracketed subexpressions
Since Xapian does not preserve quotes when passing the subquery to a
field processor, we have to make a guess as to what the user
intended. Here the added assumption is that a string surrounded by
parens is not intended to be a phrase.
2022-03-19 07:27:29 -03:00
David Bremner
c9ed87f39f test: known broken tests for bracketed terms in subject
The heuristics in the field processor currently incorrectly trigger
phrase parsing.
2022-03-19 07:27:00 -03:00
David Bremner
19cf3910a5 emacs: redirect undo to notmuch-tag-undo
The double remap is a bit ugly, but it seems better than adding
another layer of keymaps for those modes where notmuch-tag-undo makes
sense.
2022-02-26 08:00:11 -04:00
David Bremner
cc2c3a383d emacs: add notmuch-tag-undo
Keybindings are deferred to a future commit.
2022-02-26 07:58:14 -04:00
David Bremner
b21fa0e433 perf-test: allow running test_emacs from performance test suite.
test_require_external prereq has to move to test-lib-common.sh, and
the new shell functions print_emacs_header and time_emacs are provided.

The somewhat indirect way of printing the output is to avoid the extra
"" present on string values from emacsclient.
2022-02-25 08:38:03 -04:00
David Bremner
c4c6045058 test/emacs: split out tagging related tests
T310-emacs is one of the largest and longest running sets of
tests. Splitting out the tagging operations will help maintainability
as well as potentially improve the parallel running time of the test
suite. Some slowdown in running the tests sequentially may result
since there is repeated setup.
2022-02-25 08:37:50 -04:00
David Bremner
371fa28ced test: split variable settings to their own file
This allows sharing more variable settings between the (correctness)
tests and the performance-tests. Unfortunately it seems a bit tricky
to move settings to test-lib-common.sh, as that is sourced late in
test-lib.sh, and moving it earlier breaks things.
2022-02-25 08:37:14 -04:00
Tomi Ollila
08da7f25e5 removed use of 'echo -n' (and echo -n -e ...)
In most cases used printf %s ... instead.

echo -n > file  lines to create empty / truncate files were
changed to : > file lines, like done in in test-lib-emacs.sh

And one echo -n "  " replaced with use of sed "s/^/  /" in next line.
2022-02-21 09:36:39 -04:00
David Bremner
b320d3fb59 CLI/insert: escape envelope from
The idea is to do as little parsing and modification of the delivered
message as possible. Luckily the position of the "envelope header"
lets us escape it by replacing the first 5 characters of the stream
with a regular header name (with ':').
2022-02-19 22:11:35 -04:00
David Bremner
4a380371e7 test: add known broken test for insert with mbox as input
It seems reasonable that notmuch should try to avoid delivering
messages in formats it cannot index.
2022-02-19 20:15:10 -04:00
David Bremner
190d8a7711 test: start new corpus of test messages for indexing code
This particular message is not recognized by notmuch as mail, but is
fine according to e.g. mutt. The trigger for this bad behaviour seems
to be a second "From " ocurring at the beginning of the line but
inside an attachment.
2022-02-19 20:14:18 -04:00
David Bremner
9ddd13f758 python-cffi: use config_pairs API in ConfigIterator
This returns all of the config keys with non-empty values, not just
those that happen to be stored in the database.
2022-02-16 21:57:22 -04:00
David Bremner
e221a4531f test: known broken test for list(db.config) in python-cffi bindings
As of notmuch 0.34.2 [1], the python-cffi bindings make available the
configuration from both a config file and the database when accessing
Database.config like a dictionary.  It is therefore confusing that the
iterator operations only work on the configuration information stored
in the database.

[1]: d7f9572413
2022-02-16 21:55:45 -04:00
David Bremner
2c1d1107f5 lib: strip trailing '/' from pathnames (sexp queries).
This changes makes the sexp query parser consistent with the infix one
in ignoring trailing '/'. Here we do a bit better and ignore any
number of trailing '/'.
2022-01-27 07:48:27 -04:00
David Bremner
8358841c66 test: add multiple path, folder sexp query tests
This is mainly to make sure we get trailing / removal correct. Also
add regression test for path: in the infix parser matching the
existing one for folder:.
2022-01-27 07:48:27 -04:00
David Bremner
c62c22c9fb lib: drop trailing slash for path and folder searches (infix)
This resolves an old bug reported by David Edmondson in 2014. The fix
is only needed for the "boolean" case, as probabilistic / phrase
searching already ignores punctuation.

This fix is only for the infix (xapian provided) query parser.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
c73e273aaf test/sexp: tests for path, folder, including trailing '/' (sexp)
This duplicates the bug reported in [1], as well as adding some simple
regression tests for 'path' and 'folder' searches which were
previously missing for sexp syntax.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
053a390370 test: known broken tests for trailing / in path search (infix)
In [1], David Edmondson observed that the trailing / added by many
completion mechanisms causes path searches to silently fail.  This
test reproduces that bug for both `path:' and `folder:' searches.

[1]: id:cunoasuolcv.fsf@gargravarr.hh.sledj.net
2022-01-27 07:48:27 -04:00
David Bremner
0756105832 test: sanitize generated message files names
It is fragile to encode the generated names into tests, as it makes
tests break when e.g. new tests are added.  There is a possibility
that this will hide certain failures; in that case meaningful filenames
should be chosen for the generated messages.
2022-01-27 07:48:27 -04:00
David Bremner
0a32741fce lib/parse-sexp: handle lastmod queries.
This particular choice of converting strings to integers requires C++11.
2022-01-26 07:41:02 -04:00
David Bremner
341016c8ec test/sexpr-search: add known broken tests for lastmod queries
These are loosely modelled on the tests just above for date ranges,
since the error conditions are similar. Some ideas also borrowed from
T570-revision-tracking.
2022-01-26 07:41:02 -04:00
David Bremner
77ab961a1d lib/parse-sexp: support actual date queries.
The default argument processing overlaps somewhat with what is already
done in _notmuch_date_strings_to_query, but we can give more specific
error messages for the s-expression context.

The extra generality of _sexp_parse_range will be useful when we
implement additional range prefixes (at least 'lastmod' is needed).
2022-01-26 07:41:02 -04:00
David Bremner
303f207a54 lib/parse-sexp: support zero argument date queries
These are not too practical, although they may simplify some user
query generation code. Mainly this adds a new prefix keyword to the
parser.
2022-01-26 07:41:02 -04:00
David Bremner
e02bf15a4f test/search: add known broken tests for 'date' prefix
This is documented, but apparently not implemented. Add tests for
planned supported syntax and error messages.
2022-01-26 07:41:02 -04:00
David Bremner
da302e1cba emacs: use cached encoded copy for fcc
This fixes the bug reported by dkg in [1]. The movement of the call to
n-m-setup-message-for-saving is so the cleanup of Fcc headers happens
in the encoded version (otherwise Fcc headers may be saved to disk).

[1]: id:87k1zm225v.fsf@fifthhorseman.net
2022-01-26 07:22:09 -04:00
David Bremner
8179c3d114 test/emacs: known broken test for matching fcc and sent message
Based on the method outlined by Daniel Kahn Gillmor in
id:87k1zm225v.fsf@fifthhorseman.net.  With a delay of 0.2 seconds the
test becomes flaky on my machine. With a 1 second delay it fails
consistently for more than 1600 iterations.
2022-01-26 07:22:09 -04:00
David Bremner
b559240bb1 test: define test_expect_equal_message_body
This is a relatively simple sed invocation, but rather than write a
comment everywhere, give it a descriptive name.
2022-01-26 07:22:09 -04:00
David Bremner
c8d292e25b test/emacs: match mml settings in emacs_{fcc,deliver}_message
This will allow the sending of signed messages via smtp in the test
suite.
2022-01-26 07:22:09 -04:00