Commit graph

708 commits

Author SHA1 Message Date
David Bremner
dab32dc70c lib/n_d_index_file: re-use thread-id of existing message
This prevents the message document getting multiple thread-id terms
when there are multiple files with the same message-id.

This change shifts some thread ids, requiring adjustments to other tests.

(cherry picked from commit 3f4de98e7c)
2021-06-26 22:39:07 -03:00
David Bremner
6db8b422d9 fix build failure with glib 2.67
Based on a patch from Michael J Gruber [1].  As of glib 2.67 (more
specifically [2]), including "gmime-extra.h" inside an extern "C"
block causes build failures, because glib is using C++ features.

Observing that "gmime-extra.h" is no longer needed in
notmuch-private.h, which can simply delete that include, but
we have to correspondingly move the includes which might include
it (in particular crypto.h) out of the extern "C" block also.

This seems less fragile than only moving gmime-extra, and relying on
preprocessor sentinels to keep the deeper includes from happening.

Move to the include to the outside of the extern block.

[1]: id:aee618a3d41f7889a7449aa16893e992325a909a.1613055071.git.git@grubix.eu
[2]: https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1715
2021-02-15 16:43:50 -04:00
David Bremner
2702050340 lib/config: delay setting talloc destructor
If Xapian has thrown an exception, it is not safe to invoke the
destructor when freeing the list struct.

(cherry picked from commit 43ba5ed7ec)
2020-12-23 19:48:43 -04:00
David Bremner
582e919e27 lib/config: don't set destructor until iterator is initialized.
As diagnosed by Olivier Taïbi in
id:20201027100916.emry3k2wujod4xnl@galois.lan, if an exception is
thrown while the initialization is happening (e.g. if the function is
called on a closed database), then the destructor is (sometimes)
invoked on an uninitialized Xapian object.

Solve the problem by moving the setting of the destructor until after
the placement new successfully completes. It is conceivable this might
cause a memory leak, but that seems preferable to crashing, and in any
case, there seems to be nothing better to be done if the
initialization is failing things are in an undefined state by
definition.
2020-10-29 21:13:01 -03:00
Jonas Bernoulli
6c84dee531 Fix typos 2020-08-09 21:14:36 -03:00
Jonas Bernoulli
c454135376 emacs: Use makefile-gmake-mode in Makefile*s
Use `makefile-gmake-mode' instead of `makefile-mode' because the
former also highlights ifdef et al. while the latter does not.

"./Makefile.global" and one "Makefile.local" failed to specify any
major mode at all but doing so is necessary because Emacs does not
automatically figure out that these are Makefiles (of any flavor).
2020-08-09 21:14:36 -03:00
David Bremner
e3f88436b7 test: regression test for traversing config list with closed db
Also mention error return in API docs
2020-08-03 21:03:43 -03:00
David Bremner
f4f5db0f1a lib: fix return value for n_directory_delete
Falling out of the catch meant the error return was lost
2020-08-03 20:59:06 -03:00
David Bremner
337665e26a lib: catch exceptions in n_directory_get_child_files
Also clarify API in error case.
2020-08-03 20:56:09 -03:00
David Bremner
9668583a49 lib: catch exceptions in n_directory_get_child_directories
Also clarify API in error case.
2020-08-03 20:53:04 -03:00
David Bremner
31dab9b1cd lib: return NULL from n_d_get_default_indexopts on error
This is a rare and probably serious programming error, so better not
to silently return a default value.
2020-08-03 20:45:58 -03:00
David Bremner
a09293793f lib: replace use of static_cast for writable databases
static_cast is a bit tricky to understand and error prone, so add a
second pointer to (potentially the same) Xapian database object that
we know has the right subclass.
2020-07-28 08:47:58 -03:00
David Bremner
d7d4c729ab lib: encapsulate the use of notmuch_database_t field 'mode'
The plan is to change the underlying representation.
2020-07-28 08:47:58 -03:00
David Bremner
c73d510f96 lib: drop two gratuitous assignments to database mode
I'm not sure what the point of modifying that right before destroying
the object is. In a future commit I want to remove that element of the
object, so simplify that task.
2020-07-28 08:47:58 -03:00
David Bremner
b268c8c071 lib: fix error return bug with n_d_set_config.
The catch block either needs to return, or the function needs to
return "status". Choose the latter for consistency with
n_d_get_config.
2020-07-22 19:52:55 -03:00
David Bremner
ce360ce4c9 lib: rename _n_d_create to _n_d_find_or_create
The error message and name were confusing when called in some "read
only" context.
2020-07-22 19:52:55 -03:00
David Bremner
c477d7ce31 lib: convert relative filenames to absolute in n_d_index_file
The API docs promise to handle relative filenames, but the code did
not do it.

Also check for files outside the mail root, as implied by the API
description.

This fixes the bug reported at

     id:87sgdqo0rz.fsf@tethera.net
2020-07-22 19:52:55 -03:00
David Bremner
f25fc8e211 lib/n_d_needs_upgrade: handle error return from n_d_get_version
Also clarify documentation of error return from n_d_needs_upgrade.
2020-07-22 19:52:55 -03:00
David Bremner
ab45654192 lib/n_d_get_version: catch exceptions and clarify the API
notmuch_database_get_version previously returned 0 on some errors, but
did not document this. Luckily 0 is not a valid database version.
2020-07-22 19:52:55 -03:00
David Bremner
095d3d7134 lib: move deallocation of memory from n_d_close to n_d_destroy
In order to mimic the "best effort" API of Xapian to provide
information from a closed database when possible, do not
destroy the Xapian database object too early.

Because the pointer to a Xapian database is no longer nulled on close,
introduce a flag to track whether the notmuch database is open or not.
2020-07-22 19:52:55 -03:00
David Bremner
e9867b818b lib: fix exception messages for n_m_message_*
The original generic handler had an extra '%s' in the format
string. Update tests that failed to catch this because the template to
print status strings checked 'stat', which was not set.
2020-07-22 19:52:55 -03:00
David Bremner
765ca7bc08 lib: fix return value for n_m_reindex
Also update the documentation for the behaviour of n_m_get_thread_id
that this fix relies on.
2020-07-20 08:54:42 -03:00
David Bremner
a2b90dc084 lib: handle xapian exception in n_m_remove_all_tags
At least the exception we already catch should be reported properly.
2020-07-20 08:54:42 -03:00
David Bremner
b7572ceb14 lib: add notmuch_message_has_maildir_flag_st
Initially the new function is mainly tested indirectly via the
wrapper.
2020-07-20 08:54:42 -03:00
David Bremner
b21f0fcb6a test: add regression test for notmuch_message_has_maildir_flag
This passes the NULL return inside _ensure_maildir_flags does not
break anything. Probably this should be handled more explicitely.
2020-07-20 08:45:15 -03:00
David Bremner
c8539d5e92 lib/thread: replace use of deprecated notmuch_message_get_flag
This adds one more reason why _notmuch_thread_create might return
NULL, but those were not previously enumerated, so no promises are
broken.
2020-07-18 11:03:29 -03:00
David Bremner
a4776faa9b lib/add-message: drop use of deprecated notmuch_message_get_flag.
As a side effect, we revert the switch from notmuch_bool_t to bool
here. This is because those two types are not actually compatible when
passing by reference.
2020-07-18 11:03:28 -03:00
David Bremner
2d04ed2631 lib: catch exceptions in n_m_get_flag, provide n_m_get_flag_st
It's not very nice to return FALSE for an error, so provide
notmuch_message_get_flag_st as a migration path.

Bump LIBNOTMUCH_MINOR_VERSION because the API is extended.
2020-07-18 09:52:27 -03:00
David Bremner
78e9b3467d lib: use COERCE_STATUS in n_m_{add,remove}_tag
Currently I don't know of a good way of testing this, but at least in
principle a Xapian exception in _notmuch_message_{add,remove}_term
would cause an abort in the library.
2020-07-14 07:31:45 -03:00
David Bremner
aa8e3f4487 lib: catch Xapian exceptions in n_m_remove_tag
The churn here is again mainly re-indentation.
2020-07-14 07:31:45 -03:00
David Bremner
33dd5fdc69 lib: catch Xapian exceptions in n_m_add_tag
This is mostly just (horizontal) code movement due to wrapping
everything in a try / catch.
2020-07-14 07:31:45 -03:00
David Bremner
96befd0dd0 lib: catch Xapian exceptions in n_m_count_files
This will require some care for the caller to check the sign, and not
just add error returns into a running total.
2020-07-14 07:31:37 -03:00
David Bremner
00f1abfdf4 lib: catch Xapian exceptions in n_m_get_tags
This allows the function to return an error value rather than
crashing.
2020-07-14 07:12:52 -03:00
David Bremner
e404d8a51d lib: use LOG_XAPIAN_EXCEPTION in n_m_get_date
This should not change functionality, but does slightly reduce code
duplication. Perhaps more importantly it allows consistent changes to
all of the similar exception handling in message.cc.
2020-07-14 07:12:52 -03:00
David Bremner
6eaadb43ad lib: add regression test for n_m_get_date; clarify API
This function catches Xapian exceptions. The test is intended to make
sure it stays that way.
2020-07-14 07:12:52 -03:00
David Bremner
286161b703 lib: catch exceptions in n_m_get_filenames
This is essentially copied from the change to notmuch_message_get_filename
2020-07-13 07:19:22 -03:00
David Bremner
a606cba32b lib/n_m_g_filename: catch Xapian exceptions, document NULL return
This is the same machinery as applied for

     notmuch_message_get_{thread,message}_id
2020-07-13 07:19:22 -03:00
David Bremner
a962bd2bf8 lib/n_m_get_replies: doc return, initial regression test
We need to to set a query and retrieve the threads to meaningfully
test this function.
2020-07-13 07:19:22 -03:00
David Bremner
9201c50204 lib/message: use LOG_XAPIAN_EXCEPTION in n_m_get_header
This is just for consistency, and a small reduction in the amount of
boilerplate.
2020-07-13 07:19:22 -03:00
David Bremner
b90d852a2f lib: migrate from Xapian ValueRangeProcessor to RangeProcessor
This will be mandatory as of Xapian 1.5.  The API is also more
consistent with the FieldProcessor API, which helps code re-use a bit.

Note that this switches to using the built-in Xapian support for
prefixes on ranges (i.e. deleted code at beginning of
ParseTimeRangeProcessor::operator(), added prefix to constructor).

Another side effect of the migration is that we are generating smaller
queries, using one OP_VALUE_RANGE instead of an AND of two OP_VALUE_*
queries.
2020-07-11 17:20:09 -03:00
David Bremner
a1b7cc834b lib: migrate to post Xapian 1.3.4 compact support
The old API was deprecated in Xapian 1.3.4 and (will be) removed in 1.5.0
2020-07-11 17:20:09 -03:00
David Bremner
dbdb860bb9 lib/message: catch exception in n_m_get_thread_id
This allows us to return an error value from the library.
2020-07-03 21:04:43 -03:00
David Bremner
87d462a204 lib: catch error from closed db in n_m_get_message_id
By catching it at the library top level, we can return an error value.
2020-07-03 21:03:51 -03:00
Daniel Kahn Gillmor
6cdf4b7e38 smime: Index cleartext of envelopedData when requested
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:12:00 -03:00
Daniel Kahn Gillmor
2b108728c4 crypto: Make _notmuch_crypto_decrypt take a GMimeObject
As we prepare to handle S/MIME-encrypted PKCS#7 EnvelopedData (which
is not multipart), we don't want to be limited to passing only
GMimeMultipartEncrypted MIME parts to _notmuch_crypto_decrypt.

There is no functional change here, just a matter of adjusting how we
pass arguments internally.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:11:33 -03:00
Daniel Kahn Gillmor
ad60e5d4e8 smime: Identify encrypted S/MIME parts during indexing
We don't handle them correctly yet, but we can at least mark them as
being encrypted.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:10:55 -03:00
Daniel Kahn Gillmor
38bd0df922 lib: index PKCS7 SignedData parts
When we are indexing, we should treat SignedData parts the same way
that we treat a multipart object, indexing the wrapped part as a
distinct MIME object.

Unfortunately, this means doing some sort of cryptographic
verification whose results we throw away, because GMime doesn't offer
us any way to unwrap without doing signature verification.

I've opened https://github.com/jstedfast/gmime/issues/67 to request
the capability from GMime but for now, we'll just accept the
additional performance hit.

As we do this indexing, we also apply the "signed" tag, by analogy
with how we handle multipart/signed messages.  These days, that kind
of change should probably be done with a property instead, but that's
a different set of changes.  This one is just for consistency.

Note that we are currently *only* handling signedData parts, which are
basically clearsigned messages.  PKCS#7 parts can also be
envelopedData and authEnvelopedData (which are effectively encryption
layers), and compressedData (which afaict isn't implemented anywhere,
i've never encountered it).  We're laying the groundwork for indexing
these other S/MIME types here, but we're only dealing with signedData
for now.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:10:46 -03:00
David Bremner
45cfeb2e55 lib: replace STRNCMP_LITERAL in __message_remove_indexed_terms
strncmp looks for a prefix that matches, which is very much not what
we want here. This fixes the bug reported by Franz Fellner in
id:1588595993-ner-8.651@TPL520
2020-05-04 10:55:43 -03:00
Tomi Ollila
00cdfe1071 build: drop support for xapian versions less than 1.4
Xapian 1.4 is over 3 years old now (1.4.0 released 2016-06-24),
and 1.2 has been deprecated in Notmuch version 0.27 (2018-06-13).

Xapian 1.4 supports compaction, field processors and retry locking;
conditionals checking compaction and field processors were removed
but user may want to disable retry locking at configure time so it
is kept.
2020-04-23 21:28:45 -03:00
Daniel Kahn Gillmor
f2a85904f1 Correct doxygen framing for libnotmuch.h
Apparently doxygen needs its comments formatted in a specific way to
notice that the group is closed.

Without this fix, with doxygen 1.8.16-2 we see:

```
doxygen ./doc/doxygen.cfg
…/notmuch/lib/notmuch.h:2322: warning: end of file while inside a group
```

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-03-19 21:56:05 -03:00