Commit graph

1388 commits

Author SHA1 Message Date
Daniel Kahn Gillmor
7061e41cd0 python: convert shebangs to python3
This is the last bit of "python" left in the notmuch codebase.

https://www.python.org/dev/peps/pep-0394/#recommendation encourages
"third-party distributors" to use more-specific shebang lines.  I'm
not certain that the notmuch project itself is a "third-party
contributor" but I think this is a safe way to encourage people to use
python3 when they're developing notmuch.

We already have python3 explicitly elsewhere in the codebase for
developers (in nmbug).

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2021-02-15 16:32:31 -04:00
David Bremner
f61d88c6f4 CLI: use configured hook directory
This enables support for hooks outside the database directory.
It relies strongly on configuration information being usable between
closing the database and destroying it.
2021-02-06 19:57:55 -04:00
David Bremner
0345bc57a0 lib/open: set HOOK_DIR on open
This is a simple two step path search.  Most error checking is
deferred until running the hooks.
2021-02-06 19:56:13 -04:00
David Bremner
4922416ccc lib/config: add HOOK_DIR
The hook directory configuration needs to be kept in synch with the
other configuration information, so add scaffolding to support this at
database opening time.
2021-02-06 19:54:09 -04:00
David Bremner
0d3bef312d cli/new: convert to new config framework
In addition to the same type of changes as converting other
subcommands, add the possibility of creating a database at the top
level. It would probably make sense to use this for insert as well.
2021-02-06 19:52:11 -04:00
David Bremner
ac67cd84ee lib: introduce notmuch_database_create_with_config
This takes a config path parameter, and can use that to decide the
new database location.
2021-02-06 19:48:34 -04:00
David Bremner
b860be6a76 lib/config: add NOTMUCH_CONFIG_NEW_IGNORE
This will be needed by (at least) the conversion of notmuch-new.c to
the new config framework
2021-02-06 19:39:33 -04:00
David Bremner
f118ef3c3d cli/compact: convert to new configuration framework
Switch to the newly created API function notmuch_database_compact_db,
which takes the database opened in main().
2021-02-06 19:36:06 -04:00
David Bremner
c56dcea7e2 cli/tag: convert to new config framework.
In addition to changing configuration access, change talloc context
for allocation.
2021-02-06 19:30:20 -04:00
David Bremner
6c28039ebb CLI/show: mostly switch show to new config framework
This will need some cleanup when the transition completes, and we stop
passing notmuch_config_t structs to the subcommands.

Unlike the general case, we open the database in the subcommand, since
we don't know whether it should be opened read/write until we parse
the command line arguments.

Add a test to make sure passing config file on the command line is not
broken by these or future config related changes.
2021-02-06 19:27:54 -04:00
David Bremner
d613d10ddd CLI/{search,address}: convert to new configuration framework
Since we are already passing a search context around as a kind of
parameter block, add a new talloc context to that to replace relying
on 'config'.

Convert notmuch-search and notmuch-address at the same time, because
they share some code.

Add a test to make sure we don't break passing configuration as a
command line argument.
2021-02-06 19:23:28 -04:00
David Bremner
8588719eb1 CLI/reply: convert to new config framework
This is messier than some of the other conversions because the
extensive use of 'config' as a talloc context.
2021-02-06 19:21:23 -04:00
David Bremner
0ab0b48be2 cli/reindex: convert new config framework
The only non-trivial part is switching the talloc context for
query_string_from args from 'config' to 'notmuch'.
2021-02-06 19:19:18 -04:00
David Bremner
f994f0e7df CLI/insert: convert to new config framework.
The new talloc context is needed to run the hook at the very end of
the function. That in turn is needed so that this process gives up the
write lock on the database.
2021-02-06 19:17:34 -04:00
David Bremner
66adcd4f53 CLI/restore: convert to new config framework
Switch one configuration check to new n_c_get_bool function, and
switch use of config as talloc context to notmuch.
2021-02-06 19:15:53 -04:00
David Bremner
53f27aaf73 cli/dump: convert to new config framework
This conversion is trivial because the only configuration information
accessed by dump is that stored in the database (in order to dump
it). We do need to be careful to keep the write lock on the database
to ensure dump consistency.
2021-02-06 19:12:34 -04:00
David Bremner
acc6331baa CLI/count: switch to new configuration framework
The main effort is changing from the old argv style config list
iterators to the new more opaque ones provided by the library (and
backed by the database+file config cache).
2021-02-06 19:11:06 -04:00
David Bremner
d071828bd5 lib/config: make values iterators restartable
This is relatively cheap, and makes it easier to transform existing
code which uses arrays of pointers to store configuration lists.
2021-02-06 19:09:39 -04:00
David Bremner
fd6f8e6c30 lib/config: add config values iterator
This is intended to avoid duplicating the string splitting and
traversal code for all clients of the config API.
2021-02-06 19:08:12 -04:00
David Bremner
06a64cf0aa lib/open: load default values for known configuration keys.
This emulates the behaviour of notmuch_config_open defined in the CLI,
in that it fills in default values if they are not otherwise defined.
2021-02-06 19:03:36 -04:00
David Bremner
d6bd87a712 lib/config: add notmuch_config_key_{get,set}
By using an enum we can have better error detection than copy pasting
key strings around.

The question of what layer this belongs in is a bit
tricky. Historically most of the keys are defined by the CLI. On the
other hand features like excludes are supported in the
library/bindings, and it makes sense to configure them from the
library as well.

The somewhat long prefix for notmuch_config_t is to avoid collisions
with the existing usage in notmuch-client.h.
2021-02-06 19:02:00 -04:00
David Bremner
39580e2d7f lib/open: add support for config profiles and default locations
Fill in the remainder of the documented functionality for
n_d_open_with_config with respect to config file location. Similar
searching default locations of the database file still needs to be
added.
2021-02-06 18:59:00 -04:00
David Bremner
e5f3c3ed50 lib: add stub for notmuch_database_open_with_config
Initially document the intended API and copy the code from
notmuch_database_open_verbose. Most of the documented functionality is
not there yet.
2021-02-06 18:57:35 -04:00
David Bremner
4743e87c2c lib: cache configuration information from database
The main goal is to allow configuration information to be temporarily
overridden by a separate config file. That will require further
changes not in this commit.

The performance impact is unclear, and will depend on the balance
between number of queries and number of distinct metadata items read
on the first call to n_d_get_config.
2021-02-06 18:56:05 -04:00
David Bremner
6a7b61b1d5 test: add (back) upgrade tests
In ee897cab8b the upgrade tests from pre v3 databases were
removed. The reasons for that are still valid, but we should still
test the code paths that do the upgrade, and it is relatively
straightforward to do that for v3 to v3 upgrades.
2021-01-17 13:24:04 -04:00
David Bremner
15d8067c0a test/T391-python-cffi
Make bindings test verbose. This helps in debugging.
2021-01-15 07:43:23 -04:00
David Bremner
003fdba7a4 test/T750-gzip: don't compress the xapian database
This causes mysterious failures in trying to detect if a database
exists.
2021-01-15 07:43:07 -04:00
Jonas Bernoulli
2ca941163d emacs: make headings outline-minor-mode compatible
`outline-minor-mode' treats comments that begin with three or more
semicolons as headings.  That makes it very convenient to navigate
code and to show/hide parts of a file.

Elips libraries typically have four top-level sections, e.g.:

;;; notmuch.el --- run notmuch within emacs...
;;; Commentary:...
;;; Code:...
;;; notmuch.el ends here

In this package many libraries lack a "Commentary:" section, which is
not optimal but okay for most libraries, except major entry points.

Depending on how one chooses to look at it, the "... ends here" line
is not really a heading that begins a section, because it should never
have a "section" body (after all it marks eof).

If the file is rather short, then I left "Code:" as the only section
that contains code.  Otherwise I split the file into multiple sibling
sections.  The "Code:" section continues to contain `require' and
`declare-function' forms and other such "front matter".

If and only if I have split the code into multiple sections anyway,
then I also added an additional section named just "_" before the
`provide' form and shortly before the "...end here" line.  This
section could also be called "Back matter", but I feel it would be
distracting to be that explicit about it.  (The IMO unnecessary but
unfortunately still obligatory "... ends here" line is already
distracting enough as far as I am concerned.)

Before this commit some libraries already uses section headings, some
of them consistently.  When a library already had some headings, then
this commit often sticks to that style, even at the cost inconsistent
styling across all libraries.

A very limited number of variable and function definitions have to be
moved around because they would otherwise end up in sections they do
not belong into.

Sections, including but not limited to their heading, can and should
be further improved in the future.
2021-01-13 07:10:27 -04:00
David Bremner
1331888374 Merge branch 'release' 2020-12-25 12:56:57 -04:00
David Bremner
6d5d28c593 test/T360-symbol-hiding.sh: trim extra output from readelf
readelf on (at least) ppc64le sometimes generates some extension to
the Ndx name inside '[]'. Remove this output to allow our simple
column based parsing to work.
2020-12-25 11:36:33 -04:00
David Bremner
76871fcf5e test: add regression test for searching with alternate config
Make sure upcoming changes to config handling do not break command
line specification.
2020-12-23 09:23:42 -04:00
David Bremner
59488ee929 test: use keys with group 'test' in T590-libconfig
In a future commit we want to interoperate better with glib KeyFiles,
which need groups for all keys.
2020-12-20 07:46:18 -04:00
David Bremner
82faa059b8 test/T360-symbol-hiding: use readelf in place of nm
It turns out that using nm -P isn't as portable as hoped. In
particular with some ELF ABIs (e.g. ppc64 ELFv1), the desired symbols
end up in the data section instead of text.

The test is currently only functional on ELF based architectures, so I
think it's legit to depend on readelf instead of nm.

The switch to readelf has the advantage that we can explicitely ask
for all of the symbols with global visibility, rather than grepping
for notmuch. That seems a more robust approach since it will catch any
strangely named global symbols.
2020-12-13 08:23:42 -04:00
David Bremner
85b5803869 test: fix syntax errors in erroring calls to notmuch insert
notmuch insert does not currently support passing a filename for the
input, so all of these tests have an extra error in addition to the
one being tested for.

Currently this does not make a difference because the error being
tested for is caught before the error of an extra command line
argument. In the future it might make a difference, and in any case it
is confusing.
2020-09-04 21:04:33 -03:00
David Bremner
fe449f779d test: fix uninitialized variable use in T562-lib-database
Fix a copy paste error of using the boolean ret as a notmuch_status_t,
and uninitialized.
2020-08-16 14:04:36 -03:00
David Bremner
25f9a42287 test: update README to reflect dropping upgrade tests
These test databases have been unneeded since ee897cab8.
2020-08-16 10:41:44 -03:00
Teemu Likonen
adb90b9bb6 Emacs: Indent first header line only when indentation is turned on
Previously in message-show mode message's first header line (From
header) was always indented, even if user had turned thread
indentation off with "<" (notmuch-show-toggle-thread-indentation)
command.

This change modifies notmuch-show-insert-headerline function so that
it doesn't indent the first header line if notmuch-show-indent-content
variable is nil.

This change also modifies tests so that they expect this new output
format:
test/emacs-show.expected-output/notmuch-show-indent-thread-content-off
2020-08-15 09:16:34 -03:00
Jonas Bernoulli
3665914f71 emacs: Do not abuse advice to monkey patch while testing
Use `cl-letf*' instead.
2020-08-09 21:17:06 -03:00
Jonas Bernoulli
96baa22318 emacs: Drop old advices that were only need for Emacs 23 2020-08-09 21:16:12 -03:00
Jonas Bernoulli
bb15524c12 test: Fix indentation
Fix it to consistently match the style we have configured in
".dir-locals.el".
2020-08-09 21:14:36 -03:00
Jonas Bernoulli
6c84dee531 Fix typos 2020-08-09 21:14:36 -03:00
Jonas Bernoulli
73b8f0b8d7 emacs: Various cosmetic changes 2020-08-09 21:14:36 -03:00
Jonas Bernoulli
c454135376 emacs: Use makefile-gmake-mode in Makefile*s
Use `makefile-gmake-mode' instead of `makefile-mode' because the
former also highlights ifdef et al. while the latter does not.

"./Makefile.global" and one "Makefile.local" failed to specify any
major mode at all but doing so is necessary because Emacs does not
automatically figure out that these are Makefiles (of any flavor).
2020-08-09 21:14:36 -03:00
Jonas Bernoulli
e1a700067a emacs: Use 'when' instead of 'if' when there is no ELSE part 2020-08-09 20:52:34 -03:00
Jonas Bernoulli
09f6533c37 emacs: Use 'unless' instead of 'when' and 'not'
Also use 'unless' in a few cases where previously 'if' was used with
'not' but without an ELSE part.
2020-08-09 20:51:26 -03:00
Jonas Bernoulli
caaa108760 emacs: Fix indentation 2020-08-09 20:48:09 -03:00
Đoàn Trần Công Danh
f5ae8040b3 T355: specify hash algorithm explicitly
On some systems (notably, the one shipped with LibreSSL),
default fingerprint digest algorithm is SHA256.

On other systems, users can change default digest algorithm by changing
default_md in /etc/ssl/default_md.

Let's ask openssl to provide us specific algorithm to make the test
more deterministic.

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
2020-08-08 16:04:27 -03:00
David Bremner
0e4695abaa test: regression tests for n_indexopts_{get,set}_decrypt_policy
The main criteria is that they don't crash. Working with a closed
database is a bonus.
2020-08-03 21:05:29 -03:00
David Bremner
e3f88436b7 test: regression test for traversing config list with closed db
Also mention error return in API docs
2020-08-03 21:03:43 -03:00
David Bremner
a0b2a54aa7 test: regression test for n_d_get_config_list on closed db.
Exception is caught.
2020-08-03 21:01:53 -03:00
David Bremner
e8ffbf84ff test: regression test for n_directory_{get,set}_mtime
The mtime is cached, so closing the db is not a problem. Writing the
mtime throws an exception, which is caught.
2020-08-03 21:00:25 -03:00
David Bremner
f4f5db0f1a lib: fix return value for n_directory_delete
Falling out of the catch meant the error return was lost
2020-08-03 20:59:06 -03:00
David Bremner
b03cc6cf90 test: known broken test for n_directory_delete with closed db.
There is a return value bug in notmuch_directory_delete that is hiding
the exception.
2020-08-03 20:57:43 -03:00
David Bremner
337665e26a lib: catch exceptions in n_directory_get_child_files
Also clarify API in error case.
2020-08-03 20:56:09 -03:00
David Bremner
3890d98966 test: add known broken test for n_directory_get_child_files
This is a clone of the one for get_child_directories
2020-08-03 20:54:26 -03:00
David Bremner
9668583a49 lib: catch exceptions in n_directory_get_child_directories
Also clarify API in error case.
2020-08-03 20:53:04 -03:00
David Bremner
919b89156c test: add known broken test for n_directory_get_child_directories
Start a new test file (for the notmuch_directory_* API group) to hold
this test.
2020-08-03 20:51:40 -03:00
David Bremner
c151506773 test: regression test for n_m_get_filenames
Closing the database after the iterator is created is not a problem.
2020-08-03 20:50:15 -03:00
David Bremner
2b6e73d895 test: split header for lib-message tests.
This allows finer control over when to close the database.
2020-08-03 20:48:51 -03:00
David Bremner
e5e17926db test: add regression test for n_messages_collect_tags
Also test n_messages_destroy.
2020-08-03 20:47:18 -03:00
David Bremner
31dab9b1cd lib: return NULL from n_d_get_default_indexopts on error
This is a rare and probably serious programming error, so better not
to silently return a default value.
2020-08-03 20:45:58 -03:00
David Bremner
99a7aac8f2 test: drop use of db_ending
This will allow the dropping of the test for the default ending from
configure.
2020-08-01 20:27:30 -03:00
David Bremner
29a58ecf4a test: drop NOTMUCH_DEFAULT_XAPIAN_BACKEND from T360-symbol-test
Inspired by the suggestion of

         id:20200727154108.16269-3-congdanhqx@gmail.com

to drop the configuration test for the default backend. This version
is hopefully robust against backend changes.
2020-07-31 21:30:06 -03:00
David Bremner
00f48f397a test: destroy thread from closed database
Check for (non)-crash.
2020-07-31 07:41:00 -03:00
David Bremner
864f422f14 test: regression test for n_thread_get_tags
Code is taken from the API docs, with the twist that the underlying
database is closed. Not crashing is the main point.
2020-07-31 07:41:00 -03:00
David Bremner
43406e7d4c test: add regression tests for oldest and newest dates
These are strictly to watch for crashes, so don't be too fussy about
the actual timestamps.
2020-07-31 07:41:00 -03:00
David Bremner
c5017c76d9 test: add regression tests for n_t_get_{authors, subject}
This is returning explicitely cached data, so no database access is needed.
2020-07-31 07:41:00 -03:00
David Bremner
70b30066f6 test: add regression test for n_t_get_messages
This is similar to the case of toplevel messages. Currently everything
is cached, so no database access is necessary. This might change in
the future, but it should not crash in either case.
2020-07-31 07:41:00 -03:00
David Bremner
6ccc4338a4 test: add regression tests for n_thread_get_toplevel_messages
Include a test for the previously omitted non-trivial code path for
notmuch_thread_get_replies.
2020-07-31 07:41:00 -03:00
David Bremner
02ab473115 test: add regression test for n_thread_get_total_{messages,files}
This is returning cached info, so does not need to access the (closed)
database.
2020-07-31 07:41:00 -03:00
David Bremner
0baa581e6e test: regression test for n_t_get_thread_id
This is just cached data, so as long as we don't prematurely free
anything, it should be fine.
2020-07-31 07:41:00 -03:00
David Bremner
46468baa5a test: regression test for notmuch_query_destroy
As with other void API entries, not crashing counts as success.
2020-07-29 12:20:41 -03:00
David Bremner
451c0ce3fc test: regression tests for n_q_count_{messages, threads}
At least these exceptions are caught.
2020-07-29 12:20:30 -03:00
David Bremner
0a4c6cdeb1 test: regression test for n_q_search_messages
Exception handling matches notmuch_query_search_threads, at least for
this case.
2020-07-29 12:20:22 -03:00
David Bremner
25fc8662ed test: regression test for n_q_search_threads
At least this exception is caught.
2020-07-29 12:20:14 -03:00
David Bremner
4130b4e514 test: add regression test for n_q_add_tag_exclude
This relies on the change to not tear down the auxilary Xapian
objects, in particular the query parser, when the database is closed.
2020-07-29 12:20:06 -03:00
David Bremner
d5c84c8289 test: regression test for n_q_{set, get}_sort
We just want to make sure accessing the query owned by a closed db
does not cause a crash.
2020-07-29 12:19:57 -03:00
David Bremner
a292461e65 test: regression test for set_omit_excluded 2020-07-29 12:19:48 -03:00
David Bremner
768efe88d0 test: regression test for notmuch_query_get_database
Current functionality is too trivial to really justify a test, but
it's simpler just to test the complete API.
2020-07-29 12:19:39 -03:00
David Bremner
cbf9f890aa test: add regression test for n_q_{create,_get_query_string}
Start a new file of tests, to keep to the (emerging) scheme of one
notmuch_foo group per file
2020-07-29 12:19:09 -03:00
David Bremner
85da95e266 test: move notmuch_message_* tests to their own file
This is for consistency with the recently added tests for
notmuch_database_*.
2020-07-29 12:18:11 -03:00
Jonas Bernoulli
8b162b40f8 test: Explicitly state that we want to sign with sender
Since Emacs 27 'mml-secure-epg-sign' errors out if we don't opt-in to
signing as the sender using 'mml-secure-openpgp-sign-with-sender'.
2020-07-28 08:40:30 -03:00
Jonas Bernoulli
7b752e9eb4 test: Remove misguided emacs testing utilities
The goal of this abstraction was to save space.  But that failed as
the result actually was that four trivial lines got replace with 15
fairly complicated lines.  The opposite of what it was supposed to
do.

Also it made it harder to come up with the fix in the previous commit;
simply grepping for the relevant symbols did not work because they get
constructed at run-time instead of appearing in the source file.
2020-07-28 08:39:58 -03:00
Jonas Bernoulli
d5d8846c32 test: Deal with Emacs 27 switching to lexical scope by default
Starting with Emacs 27 undeclared variables in evaluated interactive
code uses lexical scope.  This includes code passed with '--eval' as
we do in the Emacs tests, which also happen to assume dynamic scope.

- This can affect variables defined by libraries that we use.  We
  let-bind such variables to change the behavior of functions which we
  then call with these bindings in effect.  If these libraries are not
  loaded beforehand, then the bindings are lexical and fail to have
  the effect we intended.

  At this time only 'smtpmail' has to be loaded explicitly (for the
  variables let-bound in emacs_deliver_message and emacs_fcc_message).

  'message' doesn't have to be loaded explicitly, because loading
  'notmuch' (in 'run_emacs') already takes care of that, indirectly.

- Our own testing-only variables also have to be declared explicitly.
  We should have done that anyway, but because of how and where these
  variables are used it was very easy to overlook that (i.e. it isn't
  something the byte-compiler ever looks at).  Not so in Emacs 27
  anymore; here this oversight caused four tests to fail.

  The numeric values of these variables get incremented by functions
  that we add to hooks that are run by many tests, not just the tests
  where we actually inspect the value and therefore take care to let-
  bind the values to 0 before we begin.  The global values therefore
  have to be numeric values as well.  I have chosen -100 instead of 0
  as the default in case someone writes a test that inspects the value
  but forgets to let-bind the value.  I hope that the unusual negative
  value that one is going to see in such a case will help debugging
  the issue.
2020-07-28 08:39:44 -03:00
David Bremner
24cf4381b8 test: add known broken test for n_d_get_default_indexopts
Xapian exceptions are swallowed and turned into default return value.
2020-07-22 19:52:55 -03:00
David Bremner
b268c8c071 lib: fix error return bug with n_d_set_config.
The catch block either needs to return, or the function needs to
return "status". Choose the latter for consistency with
n_d_get_config.
2020-07-22 19:52:55 -03:00
David Bremner
f6d74be840 test: add known broken test for n_d_set_config
Error status is currently lost.
2020-07-22 19:52:55 -03:00
David Bremner
a5e13d5218 test: add regression test for n_d_get_config
Xapian exceptions seem to handled OK, at least for this case.
2020-07-22 19:52:55 -03:00
David Bremner
dfab190a40 lib: add regresion test for n_d_get_all_tags
Existing error handling seems adequate, if not ideal.
2020-07-22 19:52:55 -03:00
David Bremner
64aa896151 lib: add regression test for n_d_find_message_by_filename
At least this Xapian exception is caught. Make sure it stays that way.
2020-07-22 19:52:55 -03:00
David Bremner
ce360ce4c9 lib: rename _n_d_create to _n_d_find_or_create
The error message and name were confusing when called in some "read
only" context.
2020-07-22 19:52:55 -03:00
David Bremner
97c887282a test: add known broken test for notmuch_database_remove_message
The current error message is a bit confusing; fix in next commit.
2020-07-22 19:52:55 -03:00
David Bremner
c477d7ce31 lib: convert relative filenames to absolute in n_d_index_file
The API docs promise to handle relative filenames, but the code did
not do it.

Also check for files outside the mail root, as implied by the API
description.

This fixes the bug reported at

     id:87sgdqo0rz.fsf@tethera.net
2020-07-22 19:52:55 -03:00
David Bremner
be3f4aec3f test: add known broken test for indexing relative path
The current code seems to look for the path relative to the current
working directory, rather than the mail store root.
2020-07-22 19:52:55 -03:00
David Bremner
9ab3e2e77d test: regression test for n_d_index_file closed db
Current exception handling seems OK, at least for this case.
2020-07-22 19:52:55 -03:00
David Bremner
1ca805a962 test: add regression test for n_d_get_directory
At least this exception is already handled correctly.
2020-07-22 19:52:55 -03:00
David Bremner
959cb4b7a7 test: regression test for n_d_get_revision
This function only accesses data cached by notmuch, so being closed is
not a problem.
2020-07-22 19:52:55 -03:00
David Bremner
bcb02dd8fc test: add regression test for n_d_{begin,end}_atomic
Xapian currently succeeds to begin/end a transaction on a closed database,
or at least does not throw an exception. Make the test robust against
this changing.
2020-07-22 19:52:55 -03:00
David Bremner
3f121d636e test: regression test for n_d_upgrade
The logic is that if it's acceptable to return SUCCESS for read only
database, it's acceptable for a closed one.
2020-07-22 19:52:55 -03:00
David Bremner
f25fc8e211 lib/n_d_needs_upgrade: handle error return from n_d_get_version
Also clarify documentation of error return from n_d_needs_upgrade.
2020-07-22 19:52:55 -03:00