Commit graph

695 commits

Author SHA1 Message Date
David Bremner
c73d510f96 lib: drop two gratuitous assignments to database mode
I'm not sure what the point of modifying that right before destroying
the object is. In a future commit I want to remove that element of the
object, so simplify that task.
2020-07-28 08:47:58 -03:00
David Bremner
b268c8c071 lib: fix error return bug with n_d_set_config.
The catch block either needs to return, or the function needs to
return "status". Choose the latter for consistency with
n_d_get_config.
2020-07-22 19:52:55 -03:00
David Bremner
ce360ce4c9 lib: rename _n_d_create to _n_d_find_or_create
The error message and name were confusing when called in some "read
only" context.
2020-07-22 19:52:55 -03:00
David Bremner
c477d7ce31 lib: convert relative filenames to absolute in n_d_index_file
The API docs promise to handle relative filenames, but the code did
not do it.

Also check for files outside the mail root, as implied by the API
description.

This fixes the bug reported at

     id:87sgdqo0rz.fsf@tethera.net
2020-07-22 19:52:55 -03:00
David Bremner
f25fc8e211 lib/n_d_needs_upgrade: handle error return from n_d_get_version
Also clarify documentation of error return from n_d_needs_upgrade.
2020-07-22 19:52:55 -03:00
David Bremner
ab45654192 lib/n_d_get_version: catch exceptions and clarify the API
notmuch_database_get_version previously returned 0 on some errors, but
did not document this. Luckily 0 is not a valid database version.
2020-07-22 19:52:55 -03:00
David Bremner
095d3d7134 lib: move deallocation of memory from n_d_close to n_d_destroy
In order to mimic the "best effort" API of Xapian to provide
information from a closed database when possible, do not
destroy the Xapian database object too early.

Because the pointer to a Xapian database is no longer nulled on close,
introduce a flag to track whether the notmuch database is open or not.
2020-07-22 19:52:55 -03:00
David Bremner
e9867b818b lib: fix exception messages for n_m_message_*
The original generic handler had an extra '%s' in the format
string. Update tests that failed to catch this because the template to
print status strings checked 'stat', which was not set.
2020-07-22 19:52:55 -03:00
David Bremner
765ca7bc08 lib: fix return value for n_m_reindex
Also update the documentation for the behaviour of n_m_get_thread_id
that this fix relies on.
2020-07-20 08:54:42 -03:00
David Bremner
a2b90dc084 lib: handle xapian exception in n_m_remove_all_tags
At least the exception we already catch should be reported properly.
2020-07-20 08:54:42 -03:00
David Bremner
b7572ceb14 lib: add notmuch_message_has_maildir_flag_st
Initially the new function is mainly tested indirectly via the
wrapper.
2020-07-20 08:54:42 -03:00
David Bremner
b21f0fcb6a test: add regression test for notmuch_message_has_maildir_flag
This passes the NULL return inside _ensure_maildir_flags does not
break anything. Probably this should be handled more explicitely.
2020-07-20 08:45:15 -03:00
David Bremner
c8539d5e92 lib/thread: replace use of deprecated notmuch_message_get_flag
This adds one more reason why _notmuch_thread_create might return
NULL, but those were not previously enumerated, so no promises are
broken.
2020-07-18 11:03:29 -03:00
David Bremner
a4776faa9b lib/add-message: drop use of deprecated notmuch_message_get_flag.
As a side effect, we revert the switch from notmuch_bool_t to bool
here. This is because those two types are not actually compatible when
passing by reference.
2020-07-18 11:03:28 -03:00
David Bremner
2d04ed2631 lib: catch exceptions in n_m_get_flag, provide n_m_get_flag_st
It's not very nice to return FALSE for an error, so provide
notmuch_message_get_flag_st as a migration path.

Bump LIBNOTMUCH_MINOR_VERSION because the API is extended.
2020-07-18 09:52:27 -03:00
David Bremner
78e9b3467d lib: use COERCE_STATUS in n_m_{add,remove}_tag
Currently I don't know of a good way of testing this, but at least in
principle a Xapian exception in _notmuch_message_{add,remove}_term
would cause an abort in the library.
2020-07-14 07:31:45 -03:00
David Bremner
aa8e3f4487 lib: catch Xapian exceptions in n_m_remove_tag
The churn here is again mainly re-indentation.
2020-07-14 07:31:45 -03:00
David Bremner
33dd5fdc69 lib: catch Xapian exceptions in n_m_add_tag
This is mostly just (horizontal) code movement due to wrapping
everything in a try / catch.
2020-07-14 07:31:45 -03:00
David Bremner
96befd0dd0 lib: catch Xapian exceptions in n_m_count_files
This will require some care for the caller to check the sign, and not
just add error returns into a running total.
2020-07-14 07:31:37 -03:00
David Bremner
00f1abfdf4 lib: catch Xapian exceptions in n_m_get_tags
This allows the function to return an error value rather than
crashing.
2020-07-14 07:12:52 -03:00
David Bremner
e404d8a51d lib: use LOG_XAPIAN_EXCEPTION in n_m_get_date
This should not change functionality, but does slightly reduce code
duplication. Perhaps more importantly it allows consistent changes to
all of the similar exception handling in message.cc.
2020-07-14 07:12:52 -03:00
David Bremner
6eaadb43ad lib: add regression test for n_m_get_date; clarify API
This function catches Xapian exceptions. The test is intended to make
sure it stays that way.
2020-07-14 07:12:52 -03:00
David Bremner
286161b703 lib: catch exceptions in n_m_get_filenames
This is essentially copied from the change to notmuch_message_get_filename
2020-07-13 07:19:22 -03:00
David Bremner
a606cba32b lib/n_m_g_filename: catch Xapian exceptions, document NULL return
This is the same machinery as applied for

     notmuch_message_get_{thread,message}_id
2020-07-13 07:19:22 -03:00
David Bremner
a962bd2bf8 lib/n_m_get_replies: doc return, initial regression test
We need to to set a query and retrieve the threads to meaningfully
test this function.
2020-07-13 07:19:22 -03:00
David Bremner
9201c50204 lib/message: use LOG_XAPIAN_EXCEPTION in n_m_get_header
This is just for consistency, and a small reduction in the amount of
boilerplate.
2020-07-13 07:19:22 -03:00
David Bremner
b90d852a2f lib: migrate from Xapian ValueRangeProcessor to RangeProcessor
This will be mandatory as of Xapian 1.5.  The API is also more
consistent with the FieldProcessor API, which helps code re-use a bit.

Note that this switches to using the built-in Xapian support for
prefixes on ranges (i.e. deleted code at beginning of
ParseTimeRangeProcessor::operator(), added prefix to constructor).

Another side effect of the migration is that we are generating smaller
queries, using one OP_VALUE_RANGE instead of an AND of two OP_VALUE_*
queries.
2020-07-11 17:20:09 -03:00
David Bremner
a1b7cc834b lib: migrate to post Xapian 1.3.4 compact support
The old API was deprecated in Xapian 1.3.4 and (will be) removed in 1.5.0
2020-07-11 17:20:09 -03:00
David Bremner
dbdb860bb9 lib/message: catch exception in n_m_get_thread_id
This allows us to return an error value from the library.
2020-07-03 21:04:43 -03:00
David Bremner
87d462a204 lib: catch error from closed db in n_m_get_message_id
By catching it at the library top level, we can return an error value.
2020-07-03 21:03:51 -03:00
Daniel Kahn Gillmor
6cdf4b7e38 smime: Index cleartext of envelopedData when requested
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:12:00 -03:00
Daniel Kahn Gillmor
2b108728c4 crypto: Make _notmuch_crypto_decrypt take a GMimeObject
As we prepare to handle S/MIME-encrypted PKCS#7 EnvelopedData (which
is not multipart), we don't want to be limited to passing only
GMimeMultipartEncrypted MIME parts to _notmuch_crypto_decrypt.

There is no functional change here, just a matter of adjusting how we
pass arguments internally.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:11:33 -03:00
Daniel Kahn Gillmor
ad60e5d4e8 smime: Identify encrypted S/MIME parts during indexing
We don't handle them correctly yet, but we can at least mark them as
being encrypted.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:10:55 -03:00
Daniel Kahn Gillmor
38bd0df922 lib: index PKCS7 SignedData parts
When we are indexing, we should treat SignedData parts the same way
that we treat a multipart object, indexing the wrapped part as a
distinct MIME object.

Unfortunately, this means doing some sort of cryptographic
verification whose results we throw away, because GMime doesn't offer
us any way to unwrap without doing signature verification.

I've opened https://github.com/jstedfast/gmime/issues/67 to request
the capability from GMime but for now, we'll just accept the
additional performance hit.

As we do this indexing, we also apply the "signed" tag, by analogy
with how we handle multipart/signed messages.  These days, that kind
of change should probably be done with a property instead, but that's
a different set of changes.  This one is just for consistency.

Note that we are currently *only* handling signedData parts, which are
basically clearsigned messages.  PKCS#7 parts can also be
envelopedData and authEnvelopedData (which are effectively encryption
layers), and compressedData (which afaict isn't implemented anywhere,
i've never encountered it).  We're laying the groundwork for indexing
these other S/MIME types here, but we're only dealing with signedData
for now.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:10:46 -03:00
David Bremner
45cfeb2e55 lib: replace STRNCMP_LITERAL in __message_remove_indexed_terms
strncmp looks for a prefix that matches, which is very much not what
we want here. This fixes the bug reported by Franz Fellner in
id:1588595993-ner-8.651@TPL520
2020-05-04 10:55:43 -03:00
Tomi Ollila
00cdfe1071 build: drop support for xapian versions less than 1.4
Xapian 1.4 is over 3 years old now (1.4.0 released 2016-06-24),
and 1.2 has been deprecated in Notmuch version 0.27 (2018-06-13).

Xapian 1.4 supports compaction, field processors and retry locking;
conditionals checking compaction and field processors were removed
but user may want to disable retry locking at configure time so it
is kept.
2020-04-23 21:28:45 -03:00
Daniel Kahn Gillmor
f2a85904f1 Correct doxygen framing for libnotmuch.h
Apparently doxygen needs its comments formatted in a specific way to
notice that the group is closed.

Without this fix, with doxygen 1.8.16-2 we see:

```
doxygen ./doc/doxygen.cfg
…/notmuch/lib/notmuch.h:2322: warning: end of file while inside a group
```

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-03-19 21:56:05 -03:00
David Bremner
dc2b5a031b notmuch release 0.29.3-1 for unstable (sid) [dgit]
[dgit distro=debian no-split --quilt=linear]
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAl3eb60ACgkQA0U5G1Wq
 FSE1Ug//Wxc7AYeHc9ErjBOkK/oQjagGOI6cKsovHfyHDRqVULsDfGt/at7wBuS2
 7V1sfU5zttoNmltmamkao4T+f/dH70D5UJxWXu1vM0QDnKz+1U4g5jTGV9sDbqwA
 34XtewzZ4508GjmmhhCHLkwQN1zplI1phGfhkwDCWrWc5er8Yat9EroHvgJ2GYcA
 fvMe13hODPC4/R5jPyKHKERyFrcj5oeZPOAV6pnaNKq1qeNgOpKm1usldl73rqe5
 ehrqF5ZgMr7wVjrnK/9wV4x4FTUKTgjweLzwQ3pWkYj3tzDx5KBBM47IrIKwoLh1
 /IS4PY9znohi/Nyl8IScmIeDAVj6PYITj0lqaDCL8x0vTkuZ5f8sFiJORZcmderk
 CylvJUahr/shdV1YolBsh1TfavP4eKIz77MkfO77uTyZACPIyZF0iHlV+me8ixIJ
 IDi2BJ0Sp+pky6/211iJtbf43CUnXATfr3COraLtT/xEKfENA63lvJRL4y+6ahmr
 BrGNfUZTvnY1+K9ym8dm3wfLe49TThMM+zbkY+2b25puO6u7cIFXgjBhhIwC4rdY
 bCd3RewBngge8NzvHw//k5+fd4s7b2BfA/6rLGWlEm4g6eik2IALtDU9Qm3L1twS
 RVr1yLoXICoJNLX7hZlgaF3iv+LzMrWkyKgjaLKiZ8HYnDwKXJg=
 =qUN/
 -----END PGP SIGNATURE-----

Merge tag 'debian/0.29.3-1'

notmuch release 0.29.3-1 for unstable (sid) [dgit]

[dgit distro=debian no-split --quilt=linear]
2019-11-27 08:45:43 -04:00
David Bremner
8e22514842 lib: fix memory error in notmuch_config_list_value
The documentation for notmuch_config_list_key warns that that the
returned value will be destroyed by the next call to
notmuch_config_list_key, but it neglected to mention that calling
notmuch_config_list_value would also destroy it (by calling
notmuch_config_list_key). This is surprising, and caused a use after
free bug in _setup_user_query_fields (first noticed by an OpenBSD
porter, so kudos to the OpenBSD malloc implementation).  This change
fixes that use-after-free bug.
2019-11-27 07:58:09 -04:00
Daniel Kahn Gillmor
4b1a8fd183 index: repair "Mixed Up" messages before indexing.
When encountering a message that has been mangled in the "mixed up"
way by an intermediate MTA, notmuch should instead repair it and index
the repaired form.

When it does this, it also associates the index.repaired=mixedup
property with the message.  If a problem is found with this repair
process, or an improved repair process is proposed later, this should
make it easy for people to reindex the relevant message.  The property
will also hopefully make it easier to diagnose this particular problem
in the future.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-09-15 19:07:06 -04:00
Daniel Kahn Gillmor
9829533e92 index: avoid indexing legacy-display parts
When we notice a legacy-display part during indexing, it makes more
sense to avoid indexing it as part of the message body.

Given that the protected subject will already be indexed, there is no
need to index this part at all, so we skip over it.

If this happens during indexing, we set a property on the message:
index.repaired=skip-protected-headers-legacy-display

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-09-01 08:45:30 -03:00
Daniel Kahn Gillmor
ab0ae8b1c0 util/crypto: _n_m_crypto_potential_payload returns whether part is the payload
Our _notmuch_message_crypto_potential_payload implementation could
only return a failure if bad arguments were passed to it.  It is an
internal function, so if that happens it's an entirely internal bug
for notmuch.

It will be more useful for this function to return whether or not the
part is in fact a cryptographic payload, so we dispense with the
status return.

If some future change suggests adding a status return back, there are
only a handful of call sites, and no pressure to retain a stable API,
so it could be changed easily. But for now, go with the simpler
function.

We will use this return value in future patches, to make different
decisions based on whether a part is the cryptographic payload or not.
But for now, we just leave the places where it gets invoked marked
with (void) to show that the result is ignored.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-09-01 08:38:11 -03:00
Daniel Kahn Gillmor
1b29822cf5 repair: set up codebase for repair functionality
This adds no functionality directly, but is a useful starting point
for adding new repair functionality.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-09-01 08:20:25 -03:00
uncrustify
2b62ca2e3b lib: run uncrustify
This is the result of running

     $ uncrustify --replace --config ../devel/uncrustify.cfg *.c *.h *.cc

in the lib directory
2019-06-14 07:41:27 -03:00
Daniel Kahn Gillmor
bcee870826 fix misspelling
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-06-11 07:24:31 -03:00
Daniel Kahn Gillmor
5c3a44681f indexing: record protected subject when indexing cleartext
When indexing the cleartext of an encrypted message, record any
protected subject in the database, which should make it findable and
visible in search.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-29 08:14:44 -03:00
David Bremner
adb53b0737 lib/database: index user headers.
This essentially involves calling _notmuch_message_gen_terms once for
each user defined header.
2019-05-25 07:21:13 -03:00
David Bremner
75bdce7952 lib: support user prefix names in term generation
This should not change the indexing process yet as nothing calls
_notmuch_message_gen_terms with a user prefix name. On the other hand,
it should not break anything either.

_notmuch_database_prefix does a linear walk of the list of (built-in)
prefixes, followed by a logarithmic time search of the list of user
prefixes. The latter is probably not really noticable.
2019-05-25 07:17:27 -03:00
David Bremner
b52cda90f0 lib: cache user prefixes in database object
This will be used to avoid needing a database access to resolve a db
prefix from the corresponding UI prefix (e.g. when indexing). Arguably
the setup of the separate header map does not belong here, since it is
about indexing rather than querying, but we currently don't have any
other indexing setup to do.
2019-05-25 07:08:20 -03:00
David Bremner
575493e785 lib: setup user headers in query parser
These tests will need to be updated if the Xapian
query print/debug format changes.
2019-05-25 06:56:16 -03:00