Commit graph

7401 commits

Author SHA1 Message Date
David Bremner
f44c83c083 test: add regression test for large number of deletions
Prior to 9ad19e4454 there was an unhandled Xapian exception when
reindexing after a large number of deletes.  This test was used for
bisection, and will subsequently serve as a regression test.
2021-05-23 08:31:36 -03:00
David Bremner
8cb30b08d8 perf-test: add test for reindexing after removal/renaming msgs
At this point it is a bit tricky to measure the performance increase
from the new message deletion code, since the same commit (9ad19e4)
that improved the performance also seems to have fixed a bug with an
uncaught Xapian exception triggered by this test.
2021-05-23 08:22:31 -03:00
Tomi Ollila
f2533b9e73 test: say_color() in one write(2)
say_color() used to call (builtin) printf (and tput(1) to stdout)
several times, which caused attempts to write messages with color
to have partial content (e.g. escape sequences) often intermixed
with other tests when parallel tests were run.

Now, with all output collected, then written out using one
printf, all strings with color print out correctly
((at least short) write(2)'s appear to write out "atomically").

While at it, used only one tput(1) execution to determine whether
color output works, and made bold/colors/sgr0 to tput(1) their
values once per test.
2021-05-23 08:05:15 -03:00
David Bremner
69c2c930ec test: quiet some extra debugging output
This output does not cause test failures, but may make it harder to
interpret the output.
2021-05-23 08:04:07 -03:00
David Bremner
119a41b9ed test: ignore debugging messages
Previously building with "-DDEBUG" broke the test suite in several places.
2021-05-23 08:03:02 -03:00
David Bremner
748352693c lib/thread: add common prefix to debug messages, join lines
This will simplify filtering these message, e.g. in the test suite.
2021-05-23 08:01:38 -03:00
Jani Nikula
1222cba7e4 doc: example command-line option reference
Example reference to a command-line option using the option role
reference. This creates a hyperlink in html, and the usual boldface
style in man page. This could be used throughout the man pages.
2021-05-22 16:47:32 -03:00
Jani Nikula
f2e2f2aa96 doc: use program and option directives to document options
Use the program and option directives to document the subcommand
options. This unifies a lot of option documentation throughout.

This also makes it possible to reference options with :option:`--foo`
(within .. program::) or :option:`subcommand --foo` (globally). This
is left for later work.

See https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#directive-program

Note: There is a lot of indentation change, but intentionally there is
no reflow. Using 'git diff -w' or 'git show -w' to ignore white space
changes makes this a very easy change to review.
2021-05-22 16:43:24 -03:00
Jani Nikula
574b2436ee doc: use envvar directive and role for environment variables
Make man1/notmuch.rst the single point of truth for describing notmuch
environment variables. Use the envvar directive for that, and
reference them with the envvar role.

Drive-by cleanup configuration file and hook directory search order
documentation.
2021-05-22 16:41:20 -03:00
Jani Nikula
ff4e81ac57 doc: cross-reference notmuch man pages with actual links
Add internal hyperlink targets for man pages and cross-reference them
using the any role reference. There are a number of alternatives to
accomplish this, but this seems like the combination that retains the
man page section number and the same boldface style in the man pages.

As a bonus, we get sanity checking on the links; for example
notmuch-search-terms.rst had a reference to notmuch-properties(1)
i.e. the wrong section.

The obvious semantic follow-up change would be to only have meaningful
"see also" references instead of having them all everywhere.
2021-05-22 16:38:56 -03:00
Tomi Ollila
8675be1d30 test: replace notmuch_passwd_sanitize() with _libconfig_sanitize()
notmuch_passwd_sanitize() in test-lib.sh is too generic, it cannot
work in many cases...

The more specific version _libconfig_sanitize() replaces it in
T590-libconfig.sh and the code that uses it is modified to output
the keys (ascending numbers printed in hex) so the sanitizer knows
what to sanitize in which lines...

"@" + fqdn -> "@FQDN" replacement is used as fqdn could
-- in theory -- be substring of 'USERNAME'.

'user -> 'USER_FULL_NAME replacement to work in cases where user
is empty -- as only first ' is replaced that works as expected.

In addition to ".(none)" now also ".localdomain" is filtered from
USERNAME@FQDN.

/dev/fd/{n} is not defined in posix, but it is portable enough
(if it weren't it is easy to fix -- now code looks clearer).
2021-05-22 11:51:29 -03:00
Jani Nikula
3baa61e0e5 doc: use manpage role references to external man pages
Using manpage role references generates helpful links in html
documentation, while retaining the same boldface style in the man
pages.

The external man page site is configurable. The Debian manpage site
seems like a good fit for Notmuch.
2021-05-22 09:56:52 -03:00
David Bremner
702635d5f6 Merge branch 'release' 2021-05-22 09:34:55 -03:00
Felipe Contreras
3dae253c4f ruby: improve compilation with CFLAGS
The ruby MakeMakefile generates a makefile that is suboptimal, which has
CFLAGS like this:

  CFLAGS   = $(CCDLFLAGS) -march=x86-64 -mtune=generic \
    -O2 -pipe -fno-plt -fPIC $(ARCH_FLAG)

This works as long as the user doesn't modify the Makefile.

Certain flags (namely -fPIC) need to be present regardless of what
CFLAGS are specified.

The Makefile should have done this instead:

  CFLAGS = -march=x86-64 -mtune=generic -O2
  override CFLAGS += $(CCDLFLAGS) -pipe -fno-plt -fPIC $(ARCH_FLAG)

Unfortunately they didn't, so we need to workaround their lack of
foresight.

We can simply add the necessary flags in the parent Makefile so everyone
is happy.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-22 09:17:21 -03:00
David Bremner
3f4de98e7c lib/n_d_index_file: re-use thread-id of existing message
This prevents the message document getting multiple thread-id terms
when there are multiple files with the same message-id.

This change shifts some thread ids, requiring adjustments to other tests.
2021-05-22 09:08:02 -03:00
David Bremner
349fc29803 test: add known broken test for duplicate thread-id terms
According to my bijection, this bug has been present since commit
411675a6ce in 2017. It is not completely clear what harm it causes in
regulary use, but it (at least) makes notmuch crash when compiled with
-DDEBUG_DATABASE_SANITY.
2021-05-22 09:02:58 -03:00
David Bremner
ff07183a02 CLI/new: check status of notmuch_message_maildir_flags_to_tags
This improves error reporting since it prints the specifics of the exception.
2021-05-19 09:00:08 -03:00
David Bremner
564a83077a perf-test/T00-new: fix backup creation.
"xargs tar cf backup.tar < $manifest" recreates the tar file with each
"batch" execed by xargs. In general this results in only a fraction of
the desired files being backed up.
2021-05-19 08:54:44 -03:00
Luis Henriques
0ab28068c1 doc: fix variable name in documentation
Variable 'notmuch-saved-searches-sort-function' does not exist;
'notmuch-saved-search-sort-function' is the correct name.

Signed-off-by: Luis Henriques <henrix@camandro.org>
2021-05-19 08:53:53 -03:00
Jani Nikula
59c953656d doc: fix man page build for Sphinx 4.x
Sphinx 4.0 changed the default value of man_make_section_directory
from False to True. We create the section directories and move the
files manually, so fix the immediate man build failure by disabling
the feature.

The Sphinx documentation on this [1] is confusing, and has the change
backwards. Git history says the default changed from False to True.

[1] https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-man_make_section_directory
2021-05-19 08:52:48 -03:00
David Bremner
892f4e65ae perf-test/T00-new: fix backup creation.
"xargs tar cf backup.tar < $manifest" recreates the tar file with each
"batch" execed by xargs. In general this results in only a fraction of
the desired files being backed up.
2021-05-19 08:39:58 -03:00
Luis Henriques
e715ec9371 doc: fix variable name in documentation
Variable 'notmuch-saved-searches-sort-function' does not exist;
'notmuch-saved-search-sort-function' is the correct name.

Signed-off-by: Luis Henriques <henrix@camandro.org>
2021-05-19 08:35:25 -03:00
Jani Nikula
5197d3e11f doc: fix man page build for Sphinx 4.x
Sphinx 4.0 changed the default value of man_make_section_directory
from False to True. We create the section directories and move the
files manually, so fix the immediate man build failure by disabling
the feature.

The Sphinx documentation on this [1] is confusing, and has the change
backwards. Git history says the default changed from False to True.

[1] https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-man_make_section_directory
2021-05-19 08:26:22 -03:00
Felipe Contreras
92454bc093 test: split emacs functionality to its own file
This way it's easier to identify the tests that do require emacs stuff.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:29:04 -03:00
Felipe Contreras
bba4788aec test: emacs: fix a couple of shellcheck complaints
In test-lib-emacs.sh line 20:
    test_require_external_prereq ${TEST_EMACS} || ret=1
                                 ^-----------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean:
    test_require_external_prereq "${TEST_EMACS}" || ret=1

In test-lib-emacs.sh line 21:
    test_require_external_prereq ${TEST_EMACSCLIENT} || ret=1
                                 ^-----------------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean:
    test_require_external_prereq "${TEST_EMACSCLIENT}" || ret=1

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:28:44 -03:00
Felipe Contreras
29ba523275 test: emacs: check for configured emacs
Commit d59d9c81 (test: Make the emacsclient binary user-configurable,
2012-11-27) modified the prereq check for the configured emacsclient,
but we probably want to do the same for emacs itself.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:28:33 -03:00
Felipe Contreras
cd9ec20a49 test: emacs: simplify missing dependencies check
No functional changes.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:28:23 -03:00
Felipe Contreras
4561f694e9 test: more style fixes
In order to fit the git coding style.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:28:01 -03:00
Felipe Contreras
645bea13cb test: trivial style cleanups
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:27:52 -03:00
Felipe Contreras
5f49e3421f ruby: new notmuch_rb_object_destroy() helper
The struct used to store the types (rb_data_type_t) contains a "data"
field where we can store whatever we want. I use that field to store a
pointer to the corresponding destroy function. For example
notmuch_rb_database_type contains a pointer to notmuch_database_destroy.

I cast that pointer as a notmuch_status_t (func*)(void *) and call
that function passing the internal object (e.g. notmuch_database_t).

Using the rb_data_type_t data we can call the correct notmuch destroy
function.

Therefore this:

  ret = ((notmuch_status_t (*)(void *)) type->data) (nm_object);

Is effectively the same as this:

  ret = notmuch_database_destroy (database);

The advantage of doing it this way is that much less code is necesary
since each rb_data_type_t has the corresponding destroy function stored
in it.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:25:14 -03:00
Felipe Contreras
9574fb6099 ruby: add all data types
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:25:14 -03:00
Felipe Contreras
fba9774a81 ruby: move towards more modern RTypedData
Virtually the whole ruby core moved from RData to RTypeData, let's do so
ourselves too.

Basically the information typically passed through Data_Wrap_Struct is
now stored in a struct rb_data_type_t (mark and free functions). This
has the advantage that more information can be easily added, like the
name of the type, a custom data ponter, and more.

Data_Wrap_Struct is replaced with TypedData_Wrap_Struct, and the
information is stored in a struct rb_data_type_t, rather than passed
as arguments.

Check_Type is replaced with Check_TypedStruct, which is a wrapper for
rb_check_typeddata (with casts).

        #define Check_TypedStruct(v, t)      \
            rb_check_typeddata(RBIMPL_CAST((VALUE)(v)), (t))

We can use rb_check_typeddata directly, just like we use rb_data_object_get
directly.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:25:14 -03:00
Felipe Contreras
12c36a5e3f ruby: create Data_Wrap_Notmuch_Object helper
This makes the code more maintainable and will help in further patches.

No functional changes.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:25:14 -03:00
Felipe Contreras
682479592b ruby: add unlikely hint
The error path is very unlikely.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:25:14 -03:00
Felipe Contreras
79bb82c217 ruby: fetch class name in case of error
There is not much point in complicating the code for error messages that
can be easily constructed.

Before:

  database closed (RuntimeError)

After:

  Notmuch::Database object destroyed (RuntimeError)

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:25:14 -03:00
Felipe Contreras
78c059a24c ruby: simplify data get helper
Data_Get_Struct is nothing but a macro that calls
rb_data_object_get with a cast (unnecessary in C).

        #define Data_Get_Struct(obj, type, sval) \
            ((sval) = RBIMPL_CAST((type*)rb_data_object_get(obj)))

We can use rb_data_object_get directly, and this way we don't need to
pass the type, which is unnecessary information.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2021-05-17 07:25:14 -03:00
David Bremner
c84ccb70f3 Merge branch 'release' 2021-05-15 09:10:58 -03:00
David Bremner
b580009e2b NEWS: finalize release date 2021-05-15 09:04:24 -03:00
David Bremner
57c62c3aeb debian: changelog for 0.32.1-1 2021-05-15 09:03:19 -03:00
David Bremner
6d5531da0c version: bump to 0.32.1 2021-05-15 08:59:01 -03:00
David Bremner
44881fa53e NEWS: add news for $HOME/mail fix 2021-05-15 08:56:46 -03:00
David Bremner
b3258244c8 lib/open: restore default database path of $HOME/mail
Although this default worked for "notmuch config get", it didn't work
most other places. Restore the previous functionality, with the
wrinkle that XDG locations will shadow $HOME/mail if they exist.

This fixes a bug reported by Jack Kamm in id:87eeefdc8b.fsf@gmail.com
2021-05-15 08:40:05 -03:00
David Bremner
0e8795b28c NEWS: add NEWS for notmuch-{before,after}-tag-hook fixes 2021-05-15 08:38:23 -03:00
Kyle Meyer
319dcfb50e emacs: restore tag-changes and query bindings for tag hooks
notmuch-before-tag-hook and notmuch-after-tag-hook are supposed to
have access to two dynamic variables, tag-changes and query, but these
were lost with the switch to lexical binding in fc4cda07 (emacs: use
lexical-bindings in all libraries, 2021-01-13).

Add a variant of Emacs's dlet (not available until Emacs 28) and use
it in notmuch-tag to expose tag-changes and query to the hooks.
2021-05-15 08:34:28 -03:00
David Bremner
a663783343 test: add known broken tests for notuch-{before,after}-tag-hook
These tests illustrate the bug reported in id:87v97ytd2s.fsf@fastmail.fm
2021-05-15 08:31:00 -03:00
David Bremner
ccd2ef3871 CLI: add missing file notmuch-client-init
This file should have been created in commit 5a8d174f, but wasn't.
2021-05-15 08:14:37 -03:00
David Bremner
121f9ddad3 CLI/notmuch: make immutable tables const
Let the compiler enforce the immutability.
2021-05-14 06:44:00 -03:00
David Bremner
0c6569d97c CLI: make static message strings const
This is both a bit clearer and avoids the possibility of modification.
2021-05-14 06:42:55 -03:00
David Bremner
19983fe554 CLI/config: make immutable tables const
Let the compiler help us catch bugs.
2021-05-14 06:41:51 -03:00
David Bremner
5a8d174f67 CLI: centralize initialization in notmuch_client_init
Initially make sure gmime_filter_reply initialization is
thread-safe (assuming notmuch_client_init is only called once).
For tidyness, also put talloc initialization in the new function.
2021-05-14 06:40:37 -03:00