On ppc64el, races are detected by TSan:
WARNING: ThreadSanitizer: data race (pid=4520)
Read of size 8 at 0x7ffff20016c0 by thread T1:
#0 strlen ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:386 (libtsan.so.2+0x77c0c)
#1 strlen ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:378 (libtsan.so.2+0x77c0c)
#2 g_strdup ../../../glib/gstrfuncs.c:362 (libglib-2.0.so.0+0xa4ac4)
Previous write of size 8 at 0x7ffff20016c0 by thread T2:
#0 malloc ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:647 (libtsan.so.2+0x471f0)
#1 g_malloc ../../../glib/gmem.c:130 (libglib-2.0.so.0+0x7bb68)
Location is heap block of size 20 at 0x7ffff20016c0 allocated by thread T2:
#0 malloc ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:647 (libtsan.so.2+0x471f0)
#1 g_malloc ../../../glib/gmem.c:130 (libglib-2.0.so.0+0x7bb68)
This appears to be a false positive in GLib, as explained at
https://gitlab.gnome.org/GNOME/glib/-/issues/1672#note_1831968
In short, a call to fstat fails under TSan and GLib's g_sterror will
intern the error message, which will be reused by other threads.
Since upstream appears to be aware that GLib doesn't play nicely with
TSan, suppress everything coming from the library instead of
maintaining a fine grained list.
Reported at
https://buildd.debian.org/status/fetch.php?pkg=notmuch&arch=ppc64el&ver=0.38%7Erc0-1&stamp=1692959868&raw=0
Depending on compiler (gcc, g++, clang) and standard options (c99, c11),
string.h may or may not include strings.h, leading to possibly missing
or conflicting declarations of strcasestr.
Include both so that both detection and compilation phases use the same
(possibly optimised) implementations.
Suggested-by: Thomas Schneider <qsx@chaotikum.eu>
Suggested-by: Florian Weimer <fweimer@redhat.com>
Suggested-by: Tomi Ollila <tomi.ollila@iki.fi>
Native compilation is kindof useless in the test suite because we
throw away the cache after every subtest. The test suite could in
principle share an eln cache within a given test file; for now try to
minimize the amount of native-compilation. There is an intermittent
bug where emacs loses track of its default-directory; I suspect (but
have no proof) that bug is related to native compilation and/or race
conditions. This patch seems to prevent that bug (or at least reduce
its frequency).
According to discussion on
https://gitlab.gnome.org/GNOME/glib/-/issues/3078
it looks like upstream will stop supporting top of file comments.
It is questionable whether we really need this feature, but for now
update notmuch-config to simulate it.
As of Emacs 29.1, In-Reply-To is in the default value for
message-hidden-headers. We actually want to see that in the test
suite, so remove it again. To future proof the tests, fix a default
value for message-hidden-headers specifically for the test suite.
It is wasteful to remove a filename term when the whole message
document is about to be removed from the database. Profiling with perf
shows this takes a significant portion of the time when cleaning up
removed files in the database.
The logic of n_d_remove_message becomes a bit more convoluted here in
order to make the change minimal.
It is possible that this function can be further optimized, since the
expansion of filename terms into filenames is probably not needed
here.
It isn't really clear how this worked before. Traversing the terms of
a document after deleting it from the database seems likely to be
undefined behaviour at best
We put some effort into testing the built copy rather than some
installed copy. On the other hand for people like packagers, testing
the installed copy is also of interest.
When NOTMUCH_TEST_INSTALLED is set to a nonempty value, tests do not
require a built notmuch tree or running configure.
Some of the tests marked as broken when running against installed
notmuch are probably fixable.
When running the test suite without building first, it is desirable to
have the tests consider these variables being undefined as equivalent
to the feature not being present, and in particular for the tests not
to generate errors.
In the decade (!) since this corpus was last updated, the keyserver
network is essentially dead, and I have migrated gpg keys. Bump the
version number as a clean way of switching signatures. Also update the
instructions to suggest using "--locate-external-key" to download the
public key. By default this uses WKD, which is now supported for my
UID.
According to my reading of RFC5322, there is an obsolete syntax for
Message-Id which permits folding whitespace (i.e. to be removed /
ignored by parsers). In [1] Paul Wise observed that notmuch removed
whitespace on indexing, but does not do any corresponding
normalization of queries. Mark the latter as a bug by adding a failing
test.
[1]: id:20230409044143.4041560-1-pabs3@bonedaddy.net
Py 3.12 finally pulled the plug on the `SafeConfigParser` class which
has been deprecated since py 3.2.
We use it in the legacy bindings only, so take the easy route of
importing `ConfigParser` as `SafeConfigParser` and monkey-patching so
that the class has the expected interface.
This prevents data loss when users configure the search cache Maildir to be a
real Maildir containing their real mail data, since the search cache Maildir
is expected to contain only symlinks to the real mail data.
Prevents: <ZCsQBNmbzwkvbpHA@localhost.localdomain>
This should be be slightly faster since it avoids forking a shell
and is less code in and less dependencies for the script.
Since String::ShellQuote isn't used elsewhere, drop mention of it.
Spaces need to be stripped when querying the Message-Id,
because notmuch stores them in Xapian with spaces stripped.
All double-quote characters need to be doubled to escape them,
otherwise they will be added as extra query terms outside the id.
We pre-parse into a list of compiled regular expressions to avoid
calling regexc on the hot (indexing) path. As explained in the code
comment, this cannot be done lazily with reasonable error reporting,
at least not without touching a lot of the code in index.cc.
