Commit graph

753 commits

Author SHA1 Message Date
Jonas Bernoulli
c454135376 emacs: Use makefile-gmake-mode in Makefile*s
Use `makefile-gmake-mode' instead of `makefile-mode' because the
former also highlights ifdef et al. while the latter does not.

"./Makefile.global" and one "Makefile.local" failed to specify any
major mode at all but doing so is necessary because Emacs does not
automatically figure out that these are Makefiles (of any flavor).
2020-08-09 21:14:36 -03:00
David Bremner
e3f88436b7 test: regression test for traversing config list with closed db
Also mention error return in API docs
2020-08-03 21:03:43 -03:00
David Bremner
f4f5db0f1a lib: fix return value for n_directory_delete
Falling out of the catch meant the error return was lost
2020-08-03 20:59:06 -03:00
David Bremner
337665e26a lib: catch exceptions in n_directory_get_child_files
Also clarify API in error case.
2020-08-03 20:56:09 -03:00
David Bremner
9668583a49 lib: catch exceptions in n_directory_get_child_directories
Also clarify API in error case.
2020-08-03 20:53:04 -03:00
David Bremner
31dab9b1cd lib: return NULL from n_d_get_default_indexopts on error
This is a rare and probably serious programming error, so better not
to silently return a default value.
2020-08-03 20:45:58 -03:00
David Bremner
a09293793f lib: replace use of static_cast for writable databases
static_cast is a bit tricky to understand and error prone, so add a
second pointer to (potentially the same) Xapian database object that
we know has the right subclass.
2020-07-28 08:47:58 -03:00
David Bremner
d7d4c729ab lib: encapsulate the use of notmuch_database_t field 'mode'
The plan is to change the underlying representation.
2020-07-28 08:47:58 -03:00
David Bremner
c73d510f96 lib: drop two gratuitous assignments to database mode
I'm not sure what the point of modifying that right before destroying
the object is. In a future commit I want to remove that element of the
object, so simplify that task.
2020-07-28 08:47:58 -03:00
David Bremner
b268c8c071 lib: fix error return bug with n_d_set_config.
The catch block either needs to return, or the function needs to
return "status". Choose the latter for consistency with
n_d_get_config.
2020-07-22 19:52:55 -03:00
David Bremner
ce360ce4c9 lib: rename _n_d_create to _n_d_find_or_create
The error message and name were confusing when called in some "read
only" context.
2020-07-22 19:52:55 -03:00
David Bremner
c477d7ce31 lib: convert relative filenames to absolute in n_d_index_file
The API docs promise to handle relative filenames, but the code did
not do it.

Also check for files outside the mail root, as implied by the API
description.

This fixes the bug reported at

     id:87sgdqo0rz.fsf@tethera.net
2020-07-22 19:52:55 -03:00
David Bremner
f25fc8e211 lib/n_d_needs_upgrade: handle error return from n_d_get_version
Also clarify documentation of error return from n_d_needs_upgrade.
2020-07-22 19:52:55 -03:00
David Bremner
ab45654192 lib/n_d_get_version: catch exceptions and clarify the API
notmuch_database_get_version previously returned 0 on some errors, but
did not document this. Luckily 0 is not a valid database version.
2020-07-22 19:52:55 -03:00
David Bremner
095d3d7134 lib: move deallocation of memory from n_d_close to n_d_destroy
In order to mimic the "best effort" API of Xapian to provide
information from a closed database when possible, do not
destroy the Xapian database object too early.

Because the pointer to a Xapian database is no longer nulled on close,
introduce a flag to track whether the notmuch database is open or not.
2020-07-22 19:52:55 -03:00
David Bremner
e9867b818b lib: fix exception messages for n_m_message_*
The original generic handler had an extra '%s' in the format
string. Update tests that failed to catch this because the template to
print status strings checked 'stat', which was not set.
2020-07-22 19:52:55 -03:00
David Bremner
765ca7bc08 lib: fix return value for n_m_reindex
Also update the documentation for the behaviour of n_m_get_thread_id
that this fix relies on.
2020-07-20 08:54:42 -03:00
David Bremner
a2b90dc084 lib: handle xapian exception in n_m_remove_all_tags
At least the exception we already catch should be reported properly.
2020-07-20 08:54:42 -03:00
David Bremner
b7572ceb14 lib: add notmuch_message_has_maildir_flag_st
Initially the new function is mainly tested indirectly via the
wrapper.
2020-07-20 08:54:42 -03:00
David Bremner
b21f0fcb6a test: add regression test for notmuch_message_has_maildir_flag
This passes the NULL return inside _ensure_maildir_flags does not
break anything. Probably this should be handled more explicitely.
2020-07-20 08:45:15 -03:00
David Bremner
c8539d5e92 lib/thread: replace use of deprecated notmuch_message_get_flag
This adds one more reason why _notmuch_thread_create might return
NULL, but those were not previously enumerated, so no promises are
broken.
2020-07-18 11:03:29 -03:00
David Bremner
a4776faa9b lib/add-message: drop use of deprecated notmuch_message_get_flag.
As a side effect, we revert the switch from notmuch_bool_t to bool
here. This is because those two types are not actually compatible when
passing by reference.
2020-07-18 11:03:28 -03:00
David Bremner
2d04ed2631 lib: catch exceptions in n_m_get_flag, provide n_m_get_flag_st
It's not very nice to return FALSE for an error, so provide
notmuch_message_get_flag_st as a migration path.

Bump LIBNOTMUCH_MINOR_VERSION because the API is extended.
2020-07-18 09:52:27 -03:00
David Bremner
78e9b3467d lib: use COERCE_STATUS in n_m_{add,remove}_tag
Currently I don't know of a good way of testing this, but at least in
principle a Xapian exception in _notmuch_message_{add,remove}_term
would cause an abort in the library.
2020-07-14 07:31:45 -03:00
David Bremner
aa8e3f4487 lib: catch Xapian exceptions in n_m_remove_tag
The churn here is again mainly re-indentation.
2020-07-14 07:31:45 -03:00
David Bremner
33dd5fdc69 lib: catch Xapian exceptions in n_m_add_tag
This is mostly just (horizontal) code movement due to wrapping
everything in a try / catch.
2020-07-14 07:31:45 -03:00
David Bremner
96befd0dd0 lib: catch Xapian exceptions in n_m_count_files
This will require some care for the caller to check the sign, and not
just add error returns into a running total.
2020-07-14 07:31:37 -03:00
David Bremner
00f1abfdf4 lib: catch Xapian exceptions in n_m_get_tags
This allows the function to return an error value rather than
crashing.
2020-07-14 07:12:52 -03:00
David Bremner
e404d8a51d lib: use LOG_XAPIAN_EXCEPTION in n_m_get_date
This should not change functionality, but does slightly reduce code
duplication. Perhaps more importantly it allows consistent changes to
all of the similar exception handling in message.cc.
2020-07-14 07:12:52 -03:00
David Bremner
6eaadb43ad lib: add regression test for n_m_get_date; clarify API
This function catches Xapian exceptions. The test is intended to make
sure it stays that way.
2020-07-14 07:12:52 -03:00
David Bremner
286161b703 lib: catch exceptions in n_m_get_filenames
This is essentially copied from the change to notmuch_message_get_filename
2020-07-13 07:19:22 -03:00
David Bremner
a606cba32b lib/n_m_g_filename: catch Xapian exceptions, document NULL return
This is the same machinery as applied for

     notmuch_message_get_{thread,message}_id
2020-07-13 07:19:22 -03:00
David Bremner
a962bd2bf8 lib/n_m_get_replies: doc return, initial regression test
We need to to set a query and retrieve the threads to meaningfully
test this function.
2020-07-13 07:19:22 -03:00
David Bremner
9201c50204 lib/message: use LOG_XAPIAN_EXCEPTION in n_m_get_header
This is just for consistency, and a small reduction in the amount of
boilerplate.
2020-07-13 07:19:22 -03:00
David Bremner
b90d852a2f lib: migrate from Xapian ValueRangeProcessor to RangeProcessor
This will be mandatory as of Xapian 1.5.  The API is also more
consistent with the FieldProcessor API, which helps code re-use a bit.

Note that this switches to using the built-in Xapian support for
prefixes on ranges (i.e. deleted code at beginning of
ParseTimeRangeProcessor::operator(), added prefix to constructor).

Another side effect of the migration is that we are generating smaller
queries, using one OP_VALUE_RANGE instead of an AND of two OP_VALUE_*
queries.
2020-07-11 17:20:09 -03:00
David Bremner
a1b7cc834b lib: migrate to post Xapian 1.3.4 compact support
The old API was deprecated in Xapian 1.3.4 and (will be) removed in 1.5.0
2020-07-11 17:20:09 -03:00
David Bremner
dbdb860bb9 lib/message: catch exception in n_m_get_thread_id
This allows us to return an error value from the library.
2020-07-03 21:04:43 -03:00
David Bremner
87d462a204 lib: catch error from closed db in n_m_get_message_id
By catching it at the library top level, we can return an error value.
2020-07-03 21:03:51 -03:00
Daniel Kahn Gillmor
6cdf4b7e38 smime: Index cleartext of envelopedData when requested
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:12:00 -03:00
Daniel Kahn Gillmor
2b108728c4 crypto: Make _notmuch_crypto_decrypt take a GMimeObject
As we prepare to handle S/MIME-encrypted PKCS#7 EnvelopedData (which
is not multipart), we don't want to be limited to passing only
GMimeMultipartEncrypted MIME parts to _notmuch_crypto_decrypt.

There is no functional change here, just a matter of adjusting how we
pass arguments internally.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:11:33 -03:00
Daniel Kahn Gillmor
ad60e5d4e8 smime: Identify encrypted S/MIME parts during indexing
We don't handle them correctly yet, but we can at least mark them as
being encrypted.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:10:55 -03:00
Daniel Kahn Gillmor
38bd0df922 lib: index PKCS7 SignedData parts
When we are indexing, we should treat SignedData parts the same way
that we treat a multipart object, indexing the wrapped part as a
distinct MIME object.

Unfortunately, this means doing some sort of cryptographic
verification whose results we throw away, because GMime doesn't offer
us any way to unwrap without doing signature verification.

I've opened https://github.com/jstedfast/gmime/issues/67 to request
the capability from GMime but for now, we'll just accept the
additional performance hit.

As we do this indexing, we also apply the "signed" tag, by analogy
with how we handle multipart/signed messages.  These days, that kind
of change should probably be done with a property instead, but that's
a different set of changes.  This one is just for consistency.

Note that we are currently *only* handling signedData parts, which are
basically clearsigned messages.  PKCS#7 parts can also be
envelopedData and authEnvelopedData (which are effectively encryption
layers), and compressedData (which afaict isn't implemented anywhere,
i've never encountered it).  We're laying the groundwork for indexing
these other S/MIME types here, but we're only dealing with signedData
for now.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-05-22 22:10:46 -03:00
David Bremner
45cfeb2e55 lib: replace STRNCMP_LITERAL in __message_remove_indexed_terms
strncmp looks for a prefix that matches, which is very much not what
we want here. This fixes the bug reported by Franz Fellner in
id:1588595993-ner-8.651@TPL520
2020-05-04 10:55:43 -03:00
Tomi Ollila
00cdfe1071 build: drop support for xapian versions less than 1.4
Xapian 1.4 is over 3 years old now (1.4.0 released 2016-06-24),
and 1.2 has been deprecated in Notmuch version 0.27 (2018-06-13).

Xapian 1.4 supports compaction, field processors and retry locking;
conditionals checking compaction and field processors were removed
but user may want to disable retry locking at configure time so it
is kept.
2020-04-23 21:28:45 -03:00
Daniel Kahn Gillmor
f2a85904f1 Correct doxygen framing for libnotmuch.h
Apparently doxygen needs its comments formatted in a specific way to
notice that the group is closed.

Without this fix, with doxygen 1.8.16-2 we see:

```
doxygen ./doc/doxygen.cfg
…/notmuch/lib/notmuch.h:2322: warning: end of file while inside a group
```

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2020-03-19 21:56:05 -03:00
David Bremner
dc2b5a031b notmuch release 0.29.3-1 for unstable (sid) [dgit]
[dgit distro=debian no-split --quilt=linear]
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAl3eb60ACgkQA0U5G1Wq
 FSE1Ug//Wxc7AYeHc9ErjBOkK/oQjagGOI6cKsovHfyHDRqVULsDfGt/at7wBuS2
 7V1sfU5zttoNmltmamkao4T+f/dH70D5UJxWXu1vM0QDnKz+1U4g5jTGV9sDbqwA
 34XtewzZ4508GjmmhhCHLkwQN1zplI1phGfhkwDCWrWc5er8Yat9EroHvgJ2GYcA
 fvMe13hODPC4/R5jPyKHKERyFrcj5oeZPOAV6pnaNKq1qeNgOpKm1usldl73rqe5
 ehrqF5ZgMr7wVjrnK/9wV4x4FTUKTgjweLzwQ3pWkYj3tzDx5KBBM47IrIKwoLh1
 /IS4PY9znohi/Nyl8IScmIeDAVj6PYITj0lqaDCL8x0vTkuZ5f8sFiJORZcmderk
 CylvJUahr/shdV1YolBsh1TfavP4eKIz77MkfO77uTyZACPIyZF0iHlV+me8ixIJ
 IDi2BJ0Sp+pky6/211iJtbf43CUnXATfr3COraLtT/xEKfENA63lvJRL4y+6ahmr
 BrGNfUZTvnY1+K9ym8dm3wfLe49TThMM+zbkY+2b25puO6u7cIFXgjBhhIwC4rdY
 bCd3RewBngge8NzvHw//k5+fd4s7b2BfA/6rLGWlEm4g6eik2IALtDU9Qm3L1twS
 RVr1yLoXICoJNLX7hZlgaF3iv+LzMrWkyKgjaLKiZ8HYnDwKXJg=
 =qUN/
 -----END PGP SIGNATURE-----

Merge tag 'debian/0.29.3-1'

notmuch release 0.29.3-1 for unstable (sid) [dgit]

[dgit distro=debian no-split --quilt=linear]
2019-11-27 08:45:43 -04:00
David Bremner
8e22514842 lib: fix memory error in notmuch_config_list_value
The documentation for notmuch_config_list_key warns that that the
returned value will be destroyed by the next call to
notmuch_config_list_key, but it neglected to mention that calling
notmuch_config_list_value would also destroy it (by calling
notmuch_config_list_key). This is surprising, and caused a use after
free bug in _setup_user_query_fields (first noticed by an OpenBSD
porter, so kudos to the OpenBSD malloc implementation).  This change
fixes that use-after-free bug.
2019-11-27 07:58:09 -04:00
Daniel Kahn Gillmor
4b1a8fd183 index: repair "Mixed Up" messages before indexing.
When encountering a message that has been mangled in the "mixed up"
way by an intermediate MTA, notmuch should instead repair it and index
the repaired form.

When it does this, it also associates the index.repaired=mixedup
property with the message.  If a problem is found with this repair
process, or an improved repair process is proposed later, this should
make it easy for people to reindex the relevant message.  The property
will also hopefully make it easier to diagnose this particular problem
in the future.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-09-15 19:07:06 -04:00
Daniel Kahn Gillmor
9829533e92 index: avoid indexing legacy-display parts
When we notice a legacy-display part during indexing, it makes more
sense to avoid indexing it as part of the message body.

Given that the protected subject will already be indexed, there is no
need to index this part at all, so we skip over it.

If this happens during indexing, we set a property on the message:
index.repaired=skip-protected-headers-legacy-display

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-09-01 08:45:30 -03:00
Daniel Kahn Gillmor
ab0ae8b1c0 util/crypto: _n_m_crypto_potential_payload returns whether part is the payload
Our _notmuch_message_crypto_potential_payload implementation could
only return a failure if bad arguments were passed to it.  It is an
internal function, so if that happens it's an entirely internal bug
for notmuch.

It will be more useful for this function to return whether or not the
part is in fact a cryptographic payload, so we dispense with the
status return.

If some future change suggests adding a status return back, there are
only a handful of call sites, and no pressure to retain a stable API,
so it could be changed easily. But for now, go with the simpler
function.

We will use this return value in future patches, to make different
decisions based on whether a part is the cryptographic payload or not.
But for now, we just leave the places where it gets invoked marked
with (void) to show that the result is ignored.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-09-01 08:38:11 -03:00
Daniel Kahn Gillmor
1b29822cf5 repair: set up codebase for repair functionality
This adds no functionality directly, but is a useful starting point
for adding new repair functionality.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-09-01 08:20:25 -03:00
uncrustify
2b62ca2e3b lib: run uncrustify
This is the result of running

     $ uncrustify --replace --config ../devel/uncrustify.cfg *.c *.h *.cc

in the lib directory
2019-06-14 07:41:27 -03:00
Daniel Kahn Gillmor
bcee870826 fix misspelling
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-06-11 07:24:31 -03:00
Daniel Kahn Gillmor
5c3a44681f indexing: record protected subject when indexing cleartext
When indexing the cleartext of an encrypted message, record any
protected subject in the database, which should make it findable and
visible in search.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-29 08:14:44 -03:00
David Bremner
adb53b0737 lib/database: index user headers.
This essentially involves calling _notmuch_message_gen_terms once for
each user defined header.
2019-05-25 07:21:13 -03:00
David Bremner
75bdce7952 lib: support user prefix names in term generation
This should not change the indexing process yet as nothing calls
_notmuch_message_gen_terms with a user prefix name. On the other hand,
it should not break anything either.

_notmuch_database_prefix does a linear walk of the list of (built-in)
prefixes, followed by a logarithmic time search of the list of user
prefixes. The latter is probably not really noticable.
2019-05-25 07:17:27 -03:00
David Bremner
b52cda90f0 lib: cache user prefixes in database object
This will be used to avoid needing a database access to resolve a db
prefix from the corresponding UI prefix (e.g. when indexing). Arguably
the setup of the separate header map does not belong here, since it is
about indexing rather than querying, but we currently don't have any
other indexing setup to do.
2019-05-25 07:08:20 -03:00
David Bremner
575493e785 lib: setup user headers in query parser
These tests will need to be updated if the Xapian
query print/debug format changes.
2019-05-25 06:56:16 -03:00
David Bremner
97939170b3 n_m_remove_indexed_terms: reduce number of Xapian API calls.
Previously this functioned scanned every term attached to a given
Xapian document. It turns out we know how to read only the terms we
need to preserve (and we might have already done so). This commit
replaces many calls to Xapian::Document::remove_term with one call to
::clear_terms, and a (typically much smaller) number of calls to
::add_term. Roughly speaking this is based on the assumption that most
messages have more text than they have tags.

According to the performance test suite, this yields a roughly 40%
speedup on "notmuch reindex '*'"
2019-05-23 08:00:56 -03:00
David Bremner
e19954fa18 lib/message-file: close stream in destructor
Without this,

$ make time-test OPTIONS=--small

leads to fatal errors from too many open files.

Thanks to st-gourichon-fid for bringing this problem to my attention in IRC.
2019-05-10 12:26:50 -03:00
David Bremner
852167479f lib/message_file: open gzipped files
Rather than storing the lower level stdio FILE object, we store a
GMime stream. This allows both transparent decompression, and passing
the stream into GMime for parsing. As a side effect, we can let GMime
close the underlying OS stream (indeed, that stream isn't visible here
anymore).

This change is enough to get notmuch-{new,search} working, but there is still
some work required for notmuch-show, to be done in a following commit.
2019-05-03 07:48:43 -03:00
Daniel Kahn Gillmor
e9b870b692 gmime-cleanup: pass NULL as default GMimeParserOptions
This is a functional change, not a straight translation, because we
are no longer directly invoking g_mime_parser_options_get_default(),
but the GMime source has indicated that the options parameter for
g_mime_parser_construct_message() is "nullable" since upstream commit
d0ebdd2ea3e6fa635a2a551c846e9bc8b6040353 (which itself precedes GMime
3.0).

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:58:00 -03:00
Daniel Kahn Gillmor
bbe3015b3e gmime-cleanup: pass NULL arguments explicitly where GMime 3.0 expects it
Several GMime 2.6 functions sprouted a change in the argument order in
GMime 3.0.  We had a compatibility layer here to be able to handle
compiling against both GMime 2.6 and 3.0.  Now that we're using 3.0
only, rip out the compatibility layer for those functions with changed
argument lists, and explicitly use the 3.0 argument lists.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:57:27 -03:00
Daniel Kahn Gillmor
582f255aeb gmime-cleanup: use GMime 3.0 function names
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:57:16 -03:00
Daniel Kahn Gillmor
58ee5d1bb5 gmime-cleanup: drop unused gmime #defines and simplify g_mime_init ()
Several of these #defines were not actually used in the notmuch
codebase any longer.  And as of GMime 3.0, g_mime_init takes no
arguments, so we can also drop the bogus RFC2047 argument that we were
passing and then #defining away.

signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:56:58 -03:00
Daniel Kahn Gillmor
b7ac4c05e1 gmime-cleanup: drop all arguments unused in GMime 3
This means dropping GMimeCryptoContext and notmuch_config arguments.

All the argument changes are to internal functions, so this is not an
API or ABI break.

We also get to drop the #define for g_mime_3_unused.

signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:56:38 -03:00
Daniel Kahn Gillmor
591a0787c2 gmime-cleanup: drop g_mime_2_6_unref
signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:56:28 -03:00
Daniel Kahn Gillmor
bb0b119358 gmime-cleanup: always support session keys
Our minimum version of GMime 3.0 always supports good session key
handling.

signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:55:32 -03:00
Daniel Kahn Gillmor
35e21bfb6f gmime-cleanup: remove GMime 2.6 variant codeblocks
signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:50:40 -03:00
Daniel Kahn Gillmor
f6da475e35 gmime-cleanup: drop unused gmime 2.6 content_type from _index_encrypted_mime_part
In _index_mime_part, we don't need to extract the content-type from
the part until just before we use it, so we also defer it lazily.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03 06:50:27 -03:00
David Bremner
319dd95ebb lib: add 'body:' field, stop indexing headers twice.
The new `body:` field (in Xapian terms) or prefix (in slightly
sloppier notmuch) terms allows matching terms that occur only in the
body.

Unprefixed query terms should continue to match anywhere (header or
body) in the message.

This follows a suggestion of Olly Betts to use the facility (since
Xapian 1.0.4) to add the same field with multiple prefixes. The double
indexing of previous versions is thus replaced with a query time
expension of unprefixed query terms to the various prefixed
equivalent.

Reindexing will be needed for 'body:' searches to work correctly;
otherwise they will also match messages where the term occur in
headers (demonstrated by the new tests in T530-upgrade.sh)
2019-04-17 08:48:16 -03:00
David Bremner
b22386f353 lib: update commentary about path/folder terms
We missed this when we changed to binary fields.
2019-03-31 12:00:30 -03:00
David Bremner
6b0cf9b21a lib: add clarification about the use of "prefix" in the docs. 2019-03-31 11:59:59 -03:00
David Bremner
2717ff96a7 lib: drop comment about only indexing one file.
Although the situation is complicated by the value fields (which are
taken from a single file), this comment is now more false than true.
2019-03-31 11:59:46 -03:00
David Bremner
d25dcc589c lib: use phrase search for anything not ending in '*'
Anything that does not look like a wildcard should be safe to
quote. This should fix the problem searching for xapian keywords.
2019-03-28 14:34:37 -03:00
Luis Ressel
9f7e851263 Prepend regerror() messages with "regexp error: "
The exact error messages returned by regerror() aren't standardized;
relying on them isn't portable. Thus, add a a prefix to make clear that
the subsequent message is a regexp parsing error, and only look for this
prefix in the test suite, ignoring the rest of the message.
2019-03-11 22:24:55 -03:00
David Bremner
71eaa19350 Merge branch 'release'
Changes from 0.28.3
2019-03-06 08:53:26 -04:00
David Bremner
e88297c072 lib/string_map: fix return type of string_cmp
I can't figure out how checking the sign of a bool ever worked. The
following program demonstrates the problem (i.e. for me it prints 1).

 #include <stdio.h>
 #include <stdbool.h>
 int main(int argc, char **argv) {
    bool x;
    x = -1;
    printf("x = %d\n", x);
 }

This seems to be mandated by the C99 standard 6.3.1.2.
2019-03-05 21:46:41 -04:00
rhn
fac155815c docs: Use correct call to notmuch_query_search_threads in usage example
Amended by db: simplify (subjectively) the example.
2019-01-25 20:51:00 -04:00
rhn
45639881b5 lib: Explicitly state when replies will be destroyed
Without an explicit guarantee, it's not clear how to use the reference.
2019-01-25 20:34:57 -04:00
Daniel Kahn Gillmor
f5411574af index: explicitly follow GObject conventions
Use explicit labels for GTypeInfo member initializers, rather than
relying on comments and ordering.  This is both easier to read, and
harder to screw up.  This also makes it clear that we're mis-casting
GObject class initializers for gcc.

Without this patch, g++ 8.2.0-7 produces this warning:

CXX  -g -O2 lib/index.o
lib/index.cc: In function ‘GMimeFilter* notmuch_filter_discard_non_term_new(GMimeContentType*)’:
lib/index.cc:252:23: warning: cast between incompatible function types from ‘void (*)(NotmuchFilterDiscardNonTermClass*)’ {aka ‘void (*)(_NotmuchFilterDiscardNonTermClass*)’} to ‘GClassInitFunc’ {aka ‘void (*)(void*, void*)’} [-Wcast-function-type]
      (GClassInitFunc) notmuch_filter_discard_non_term_class_init,
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The definition of GClassInitFunc in
/usr/include/glib-2.0/gobject/gtype.h suggests that this function will
always be called with the class_data member of the GTypeInfo.  We set
that value to NULL in both GObject definitions in notmuch. So we mark
it as explicitly unused.

There is no functional change here, just code cleanup.
2018-10-21 10:21:26 -03:00
David Bremner
87934c432c lib: change parent strategy to use In-Reply-To if it looks sane
As reported by Sean Whitton, there are mailers (in particular the
Debian Bug Tracking System) that have sensible In-Reply-To headers,
but un-useful-for-notmuch References (in particular with the BTS, the
oldest reference is last). I looked at a sample of about 200K
messages, and only about 0.5% these had something other than a single
message-id in In-Reply-To. On this basis, if we see a single
message-id in In-Reply-To, consider that as authoritative.
2018-09-06 08:07:13 -03:00
David Bremner
b31e44c678 lib: add _notmuch_message_id_parse_strict
The idea is that if a message-id parses with this function, the MUA
generating it was probably sane, and in particular it's probably safe
to use the result as a parent from In-Reply-to.
2018-09-06 08:07:13 -03:00
David Bremner
46dce33abc lib/thread: change _resolve_thread_relationships to use depths
We (finally) implement the XXX comment. It requires a bit of care not
to reparent all of the possible toplevel messages.

_notmuch_messages_has_next is not ready to be a public function yet,
since it punts on the mset case. We know in the one case it is called,
the notmuch_messages_t is just a regular list / iterator.
2018-09-06 08:07:13 -03:00
David Bremner
21803df6ae lib/thread: rewrite _parent_or_toplevel to use depths
This is part 1/2 of changing the reparenting of alleged toplevel
messages to use a "deep" reference rather than just the first one
found.
2018-09-06 08:07:13 -03:00
David Bremner
0a7181dd16 lib: calculate message depth in thread
This will be used in reparenting messages without useful in-reply-to,
but with useful references
2018-09-06 08:07:13 -03:00
David Bremner
a330858284 lib/thread: initial use of references as for fallback parenting
This is mainly to lay out the structure of the final code. The problem
isn't really solved yet, although some very simple cases are
better (hence the fixed test). We need two passes through the messages
because we need to be careful not to re-parent too many messages and
end up without any toplevel messages.
2018-09-06 08:07:13 -03:00
David Bremner
ac2146118b use EMPTY_STRING in _parent_via_in_reply_to
This is a review suggestion [1] of Tomi. I decided not to squash it
so that the code movement remains clear.

[1]: id:m2pnxxgf5q.fsf@guru.guru-group.fi
2018-09-06 08:07:13 -03:00
David Bremner
b9d4eb0412 lib/thread: refactor in_reply_to test
This is not a complete win in code-size, but it makes the code (which
is about to get more complicated) easier to follow.
2018-09-06 08:07:13 -03:00
David Bremner
dc3cc18bf0 lib: add _notmuch_message_list_empty
There is no public notmuch_message_list_t public interface, so to this
is added to the private API. We use it immediately in thread.cc;
future commits will use it further.
2018-09-06 08:07:13 -03:00
David Bremner
040fd630bf lib/thread: add macro for debug printing of threading
This is analogous to DEBUG_DATABASE_SANITY, and is intended to help
debugging and to help users submit bug reports.
2018-09-06 08:07:12 -03:00
David Bremner
d0b844b358 lib: read reference terms into message struct.
The plan is to use these in resolving threads.
2018-09-06 08:07:12 -03:00
David Bremner
9b568e73e1 lib/thread: sort sibling messages by date
For non-root messages, this should not should anything currently, as
the messages are already added in date order. In the future we will
add some non-root messages in a second pass out of order and the
sorting will be useful. It does fix the order of multiple
root-messages (although it is overkill for that).
2018-09-06 08:07:12 -03:00
Daniel Kahn Gillmor
6a9f26b4a0 lib: make notmuch_message_get_database() take a const notmuch_message_t*
This is technically an API change, but it is not an ABI change, and
it's merely a statement that limits what the library can do.

This is in parallel to notmuch_query_get_database(), which also takes
a const pointer.
2018-05-26 07:32:01 -07:00
Daniel Kahn Gillmor
499bb78178 properties: add notmuch_message_count_properties
The user can already do this manually, of course, but (a) it's nice to
have a convenience function, and (b) exposing this interface means
that someone more clever with a _notmuch_string_map_t than i am can
write a more efficient version if they like, and it will just
accelerate the users of the convenience function.
2018-05-26 07:31:39 -07:00
David Bremner
4a6371f1d8 lib: bump minor version
This recognizes the addition of (at least)
notmuch_message_get_database to the API.
2018-05-26 07:31:27 -07:00
Daniel Kahn Gillmor
9088db76d8 lib: expose notmuch_message_get_database()
We've had _notmuch_message_database() internally for a while, and it's
useful.  It turns out to be useful on the other side of the library
interface as well (i'll use it later in this series for "notmuch
show"), so we expose it publicly now.
2018-05-26 07:30:32 -07:00
David Bremner
1883ab6650 drop use of register keyword
The performance benefits are dubious, and it's deprecated in C++11.
2018-05-14 22:18:05 -03:00
David Bremner
f0131af6c5 lib: define specialized get_thread_id for use in thread subquery
The observation is that we are only using the messages to get there
thread_id, which is kindof a pessimal access pattern for the current
notmuch_message_get_thread_id
2018-05-07 08:42:53 -03:00
David Bremner
5ad39ebf75 lib: add thread subqueries.
This change allows queries of the form

 thread:{from:me} and thread:{from:jian} and not thread:{from:dave}

This is still somewhat brute-force, but it's a big improvement over
both the shell script solution and the previous proposal [1], because it
does not build the whole thread structure just generate a
query. A further potential optimization is to replace the calls to
notmuch with more specialized Xapian code; in particular it's not
likely that reading all of the message metadata is a win here.

[1]: id:20170820213240.20526-1-david@tethera.net
2018-05-07 08:42:53 -03:00