Commit graph

1645 commits

Author SHA1 Message Date
Justus Winter
bf8aa34324 test: replace aging OpenPGP key used in the test suite
This replaces the old OpenPGPv4 key that is used in the test suite
with a more modern OpenPGPv4 key.  All cryptographic artifacts in the
test suite are updated accordingly.

Having old cryptographic artifacts in the test suite presents a
problem once the old algorithms are rejected by contemporary
implementations.

For reference, this is the old key.

  sec   rsa1024 2011-02-05 [SC]
        5AEAB11F5E33DCE875DDB75B6D92612D94E46381
  uid           [ unknown] Notmuch Test Suite <test_suite@notmuchmail.org> (INSECURE!)
  ssb   rsa1024 2011-02-05 [E]

And this is the new key.  Note that is has the same shape, but uses
Ed25519 and Cv25519 instead of 1024-bit RSA.

  sec   ed25519 2022-09-07 [SC]
        9A3AFE6C60065A148FD4B58A7E6ABE924645CC60
  uid           [ultimate] Notmuch Test Suite (INSECURE!) <test_suite@notmuchmail.org>
  ssb   cv25519 2022-09-07 [E]
2022-09-23 20:16:00 -03:00
Justus Winter
acb31939bb test: compute expected keyid from fingerprint 2022-09-19 22:19:35 -03:00
David Bremner
8eabd6388e test: add known broken test for indexing text/* attachments
The general problem of indexing attachments requires some help to turn
things into text, but (most?) text/* should be doable internally,
possibly with optimizations as for the text/html case.
2022-09-03 09:06:08 -03:00
David Bremner
a832f940e1 test: rename indexing corpus
The corpus is not really suitable for general indexing test since the
sole message is ignored (and will most likely continue to be ignored)
by notmuch-new.
2022-09-03 09:05:44 -03:00
David Bremner
2e5ef69fbf lib: add field processor for lastmod: prefix
By sharing the existing logic used by the sexp query parser, this
allows negative lastmod revisions to be interpreted as relative to the
most recent revision.
2022-09-03 08:43:33 -03:00
David Bremner
606d9b02e4 lib/sexp: provide relative lastmod queries
Test the relatively trivial logic changes for the sexp query parser
first before refactoring that logic to share with the infix query
parser.
2022-09-03 08:36:53 -03:00
David Bremner
84e4e130e2 lib/open: create database path in some cases
There is some duplication of code here, but not all of the locations
valid to find a database make sense to create. Furthermore we nead two
passes, so the control flow in _choose_database_path would get a bit
convoluted.
2022-09-03 08:24:43 -03:00
David Bremner
8ba3057d01 lib/open: return non-SUCCESS on missing database path
This simplifies the logic of creating the directory path when it doesn't
exist.
2022-09-03 08:24:43 -03:00
David Bremner
44c9338061 test/path-config: set database.mail_root but not database.path
The failing "create database" test replicates a bug reported by Sean
Whitton [1]. The other two failures also look related to the database
being (re)created in the wrong place.

[1]: id:87y1wqkw13.fsf@athena.silentflame.com.
2022-09-03 08:24:43 -03:00
David Bremner
687866111a test: add test for creating database in various configurations.
The existing database creation (via add_email_corpus) was always done
in the traditional configuration. The use of xapian-metadata is just
to portably ensure that there is a database created where we expect
there to be.
2022-09-03 08:24:43 -03:00
David Bremner
d273263d16 emacs/show: restrict inlined mimetypes on refresh.
This fixes the bug reported by Al [1]. Essentially apply the same fix
as [2] in a different place.

[1]: id:877d41nmr1.fsf@gmail.com
[2]: 90a7c1af36
2022-08-11 07:17:17 -03:00
David Bremner
4de3eb4c0e test/emacs: add known broken test for inlining on refresh.
This duplicates the bug reported by Al [1]

[1]: id:877d41nmr1.fsf@gmail.com
2022-08-11 07:17:17 -03:00
David Bremner
6d6d2a5fe7 emacs: add duplicate support to notmuch-show-view-raw-message
This fixes one of the bugs reported by Gregor in [1]

[1]: id:87edxw8jp4.fsf@no.workgroup
2022-08-07 07:05:49 -03:00
David Bremner
67f535332c test/emacs: add known broken test for viewing raw duplicate
This duplicates one of the bugs reported by Gregor in [1].

[1]: id:87edxw8jp4.fsf@no.workgroup
2022-08-07 07:05:49 -03:00
David Bremner
1849c68d0f emacs: update notmuch-show-get-filename to handle duplicates
This fixes one of the bugs reported by Gregor in [1].

[1]: id:87edxw8jp4.fsf@no.workgroup
2022-08-07 07:05:49 -03:00
David Bremner
2c7438fe51 test/emacs: add known broken test for stashing duplicate filename
This duplicates (no pun intended) one of the bugs reported by Gregor
in id:87edxw8jp4.fsf@no.workgroup
2022-08-07 07:05:49 -03:00
David Bremner
7b27119d8e tests: remove dead code from T055-path-config.sh
This case statement does nothing.
2022-08-06 09:16:13 -03:00
David Bremner
1ef7c75111 emacs/reply: reply to correct duplicate
Essentially we just need to arrange to pass the right --duplicate
argument to notmuch reply.

As a side-effect, correct the previously unused value of EXPECTED in
T453-emacs-reply.sh.
2022-07-30 08:46:22 -03:00
David Bremner
24decfc1dd CLI/reply: support --duplicate argument
We want the reply used to match that shown e.g. in the emacs
interface. As a first step provide that functionality on the command
line.

Schema does not need updating as the duplicate key was already
present (with a constant value of 1).
2022-07-30 08:45:15 -03:00
David Bremner
5ea5a5557d emacs/show: display count of duplicates in headerline
There is no real cost here, except screen real estate. Some people
might prefer hiding the duplicate count, but we leave that for a
future commit.
2022-07-30 08:44:14 -03:00
David Bremner
b3d466bf39 emacs/show: provide notmuch-show-choose-duplicate
This new command allows the user to interactively choose a different
duplicate (file) to display for a given message in
notmuch-show-mode. Since both tree and unthreaded view use
notmuch-show-mode, this provides the same facility there.
2022-07-30 08:42:12 -03:00
David Bremner
4612f3eb3d CLI/show: support --duplicate for structured output
This introduces a new mandatory key for message structures, namely
"duplicate". Per convention in devel/schemata this does _not_ increase
the format version. This means that clients are responsible for
checking that it exists, and not crashing if it does not.

The main functional change is teaching mime_node_open to understand a
'duplicate' argument.

Support for --duplicate in notmuch-reply would make sense, but we
defer it to a later commit.
2022-07-30 08:41:50 -03:00
David Bremner
cef5eaaef6 CLI/show: initial support for --duplicate for (raw output only)
Add command line argument --duplicate, analogous with that already
supported for notmuch-search.

Use of a seperate function for _get_filename is mainly a form of
documentation at this point.

md5sum is of course a weak hash, but it is good enough for
this (non-adversarial) test suite use.
2022-07-30 08:39:13 -03:00
David Bremner
bde8ea5d1d emacs/show: introduce notmuch-show-height-limit
Analogous to notmuch-show-depth-limit, it requires a bit more
calculation, but that cost seems dominated by the rendering cost.
2022-07-14 11:51:01 -04:00
David Bremner
e580ce0058 emacs/show: obey notmuch-show-max-text-part-size for part 0
This parameter was originally introduced to hide large attachements
that happened to be text/plain. From a performance point of view,
there is no reason not to also hide large message bodies.
2022-07-14 11:48:30 -04:00
David Bremner
1d254bd35e emacs/show: optionally insert bodies of "deep messages" lazily
This leverages the machinery already there to insert buttons for
attachments.

A potential use-case is browsing the top layers of the tree to decide
which of the lower subtrees to read.
2022-07-14 11:47:49 -04:00
David Bremner
bf5eacbe7e CLI/git: add --format-version argument to init subcommand
This is primarily intended to support testing upward compatibility
with legacy repos.
2022-07-07 07:05:55 -03:00
David Bremner
6219e7380a CL/git: add format version 1
The original nmbug format (now called version 0) creates 1
subdirectory of 'tags/' per message. This causes problems for more
than (roughly) 100k messages.

Version 1 introduces 2 layers of hashed directories. This scheme was
chose to balance the number of subdirectories with the number of extra
directories (and git objects) created via hashing.

This should be upward compatible in the sense that old repositories
will continue to work with the updated notmuch-git.
2022-07-07 06:56:05 -03:00
David Bremner
15b940d26a test: known broken test for top level .notmuch in split configs
In split configurations there is no special significance to a top
level directory called .notmuch in the mail root. Users should
therefore be able to have mail stored underneath it.
2022-07-05 08:09:57 -03:00
David Bremner
bf9e206925 test: add new corpus of duplicate messages
This corpus will be used to test a new --duplicate option for notmuch-show
2022-07-05 07:05:49 -03:00
David Bremner
0a3bb81d4f test: define and use notmuch_sexp_*_sanitize functions
These are based on the equivalent functions for json. Like those,
these are pretty simple-minded, and don't really understand the
syntax.
2022-07-05 07:03:37 -03:00
David Bremner
5f6645bd2a test: use notmuch_json_show_sanitize more places
This makes the tests more robust against changing output formats, by
allowing us to centralize fixes in the sanitization function. It is
not appropriate for all cases, in particular it is unneeded when using
test_json_nodes, and unhelpful when testing filenames.
2022-07-05 07:01:39 -03:00
David Bremner
6a9ae99099 lib/sexp: add parameter expansion for regex and wildcard
Fix the bug reported at [1].

The parameter expansion for regex and wildcard modifiers has to be
done a bit differently, because their arguments are not s-expressions
defining complete Xapian queries.

[1]: id:87o7yxqxy6.fsf@code.pm
2022-07-01 08:37:00 -03:00
David Bremner
4464a5d073 test/sexp: add known broken tests for macro param inside rx/wildcard
These tests replicate the problem reported by Eric Colson [1] (for the
regex case).

[1]: id:87o7yxqxy6.fsf@code.pm
2022-07-01 08:34:36 -03:00
David Bremner
e7ffb74041 lib/sexp: allow * as alias for "" in range searches.
It can be tedious to use "" inside of a string, e.g. in a shell script.
2022-06-25 19:49:55 -03:00
David Bremner
7863234586 lib/sexp: special case "" as an argument in lastmod ranges.
Support this syntax for constincy with (data from to) ranges.
2022-06-25 19:49:55 -03:00
David Bremner
254912821f test/sexp: one sided range searches.
The date range parsing machinery already knows how to do something
appropriate with an empty string, but the lastmod parsing blindly
tries to parse each atom as a number.
2022-06-25 19:49:55 -03:00
David Bremner
6f749dd24a lib: check for writable db in n_m_tags_maildir_flags
The database needs to be writable because the list of stored file
names will change in general.
2022-06-25 16:06:34 -03:00
David Bremner
3f27cce71f lib: add NOTMUCH_STATUS_CLOSED_DATABASE, use in _n_d_ensure_writable
In order for a database to actually be writeable, it must be the case that it
is open, not just the correct type of Xapian object. By explicitely
checking, we are able to provide better error reporting, in particular
for the previously broken test in T566-lib-message.
2022-06-25 16:06:18 -03:00
David Bremner
e70df92085 lib/tag: handle NULL argument to notmuch_tags_valid
Make the behaviour when passed NULL consistent with
notmuch_filenames_valid. The library already passes the result of
notmuch_message_get_tags without checking for NULL, so it should be
handled.
2022-06-25 16:05:45 -03:00
David Bremner
00ec2b271d test: add known broken test for notmuch_tags_valid (NULL)
This should return false, but currently segfaults.

Start a new file for tags library API related tests. This is maybe
overkill, but new C boilerplate which doesn't corrupt the database is
needed anyway.
2022-06-25 16:05:32 -03:00
David Bremner
8ade833ec2 test: error handling for n_m_tags_to_maildir_flags
The closed database case should fail gracefully, but currently it
segfaults.
2022-06-25 16:05:16 -03:00
David Bremner
9e9d0e5ffc test: error handling in _n_message_{add,remove}_filename
Use a closed database to force throwing a Xapian exception. Leave the
check vague to accomodate a potential explicit check for open database.
2022-06-25 12:55:02 -03:00
David Bremner
a581755fbb test: _notmuch_message_remove_term catches exceptions.
Unfortunately we can't differentiate between the two distinct error
conditions for Document::remove_term.
2022-06-25 12:55:02 -03:00
David Bremner
f48d2e2ff8 lib/message: catch exceptions in _n_m_add_term
Some code movement is needed to make sure the cache is only
invalidated when the Xapian operation succeeds.
2022-06-25 12:55:02 -03:00
David Bremner
2707c06a0f test: define test_private_C
When testing error handling, it is sometimes difficult to cover a
particular error path deterministically. Introduce a test function to
allow calling lower level functions directly.
2022-06-25 12:55:02 -03:00
David Bremner
bfcf9a6c10 CLI: set NOTMUCH_CONFIG in hooks.
This addresses a bug report / feature request of Uwe Kleine-König. The
assumption is that we always load a config file in the CLI (i.e. we
never pass "" as the config file argument to
notmuch_database_open_with_config).

[1]: id:8baa58c3-7ab9-ec03-1bbd-28aa5be838f2@kleine-koenig.org
2022-06-18 08:24:46 -03:00
David Bremner
e167825bca test: add known broken tests for setting NOTMUCH_CONFIG in hooks
Setting this according to --config was requested by Uwe
Kleine-König. There are some other ways that the configuration file
might be found in current notmuch, so check those as well.

As a bonus, fix a bug in the hook tests that left NOTMUCH_CONFIG set
even though a config file was provided via NOTMUCH_PROFILE.

[1]: id:8baa58c3-7ab9-ec03-1bbd-28aa5be838f2@kleine-koenig.org
2022-06-18 08:23:31 -03:00
David Bremner
7d1e5a5348 CLI/git: add safety checks for checkout and commit
Commits or checkouts that modify a large fraction of the messages in
the database should be relatively rare (and in some automated process,
probably non-existent). For initial setup, where such operations are
expected, the user can pass --force.
2022-06-17 08:40:19 -03:00
David Bremner
99e85823c8 CLI/git: support configuration for repo location / prefix
This is probably more convenient than always passing a command line
argument.

Use notmuch-config for consistency with other notmuch CLI tools.

Now that there is something relevant in the config files, test the
--config option.
2022-06-17 08:40:19 -03:00