Commit graph

1072 commits

Author SHA1 Message Date
David Bremner
e88297c072 lib/string_map: fix return type of string_cmp
I can't figure out how checking the sign of a bool ever worked. The
following program demonstrates the problem (i.e. for me it prints 1).

 #include <stdio.h>
 #include <stdbool.h>
 int main(int argc, char **argv) {
    bool x;
    x = -1;
    printf("x = %d\n", x);
 }

This seems to be mandated by the C99 standard 6.3.1.2.
2019-03-05 21:46:41 -04:00
David Bremner
09595cf577 lib: Add known broken test for string_map binary search.
Because the string_map functions are not exported, we test via message
properties.
2019-03-05 21:46:41 -04:00
David Edmondson
e8cb6b2cd6 emacs: Call `notmuch-mua-send-hook' hooks when sending a message
Previously any hook functions attached to `notmuch-mua-send-hook' were
ignored.
2018-09-28 20:22:42 -03:00
David Edmondson
4e213fe9b4 test: Check that `notmuch-mua-send-hook' is called on sending a message 2018-09-28 20:22:33 -03:00
David Bremner
c846e15ffe test: make regexp test conditional on field processors
Normally we'd mark it broken, but perversely missing regexp support
actually makes the test pass.
2018-09-14 08:54:20 -03:00
David Bremner
c6ba5522a5 test: mark thread subqueries broken without field processors
Currently these tests just fail when notmuch is built against Xapian
1.2.x
2018-09-14 08:54:09 -03:00
David Edmondson
4cd5a0a3d5 test: Absolute and relative directory paths. 2018-09-08 20:19:42 -03:00
David Bremner
87934c432c lib: change parent strategy to use In-Reply-To if it looks sane
As reported by Sean Whitton, there are mailers (in particular the
Debian Bug Tracking System) that have sensible In-Reply-To headers,
but un-useful-for-notmuch References (in particular with the BTS, the
oldest reference is last). I looked at a sample of about 200K
messages, and only about 0.5% these had something other than a single
message-id in In-Reply-To. On this basis, if we see a single
message-id in In-Reply-To, consider that as authoritative.
2018-09-06 08:07:13 -03:00
David Bremner
b31e44c678 lib: add _notmuch_message_id_parse_strict
The idea is that if a message-id parses with this function, the MUA
generating it was probably sane, and in particular it's probably safe
to use the result as a parent from In-Reply-to.
2018-09-06 08:07:13 -03:00
David Bremner
35053c2b9a test/thread-replies: mangle In-Reply-To's
In a future commit, we will start trusting In-Reply-To's when they
look sane (i.e. a single message-id). Modify these tests so they will
keep passing (i.e. keep choosing References) when that happens.
2018-09-06 08:07:13 -03:00
David Bremner
ea08032ae4 test: add known broken test for good In-Reply-To / bad References
The current scheme of choosing the replyto (i.e. the default parent
for threading purposes) does not work well for mailers that put
the oldest Reference last.
2018-09-06 08:07:13 -03:00
David Bremner
46dce33abc lib/thread: change _resolve_thread_relationships to use depths
We (finally) implement the XXX comment. It requires a bit of care not
to reparent all of the possible toplevel messages.

_notmuch_messages_has_next is not ready to be a public function yet,
since it punts on the mset case. We know in the one case it is called,
the notmuch_messages_t is just a regular list / iterator.
2018-09-06 08:07:13 -03:00
David Bremner
a330858284 lib/thread: initial use of references as for fallback parenting
This is mainly to lay out the structure of the final code. The problem
isn't really solved yet, although some very simple cases are
better (hence the fixed test). We need two passes through the messages
because we need to be careful not to re-parent too many messages and
end up without any toplevel messages.
2018-09-06 08:07:13 -03:00
David Bremner
9b568e73e1 lib/thread: sort sibling messages by date
For non-root messages, this should not should anything currently, as
the messages are already added in date order. In the future we will
add some non-root messages in a second pass out of order and the
sorting will be useful. It does fix the order of multiple
root-messages (although it is overkill for that).
2018-09-06 08:07:12 -03:00
David Bremner
ccb52edb4c test: add known broken tests for "ghost roots"
This documents the bug discussed at

     id:87efgmmysi.fsf@len.workgroup

The underlying issue is that the reply to a ghost (missing) message is
falsely classified as a root message in _resolve_thread_relationships.

There are two pairs of tests; in each case the the first test is
simpler / more robust, but also easier to fool.
2018-09-06 08:07:12 -03:00
David Bremner
ebd131ac07 test: start threading test corpus
There are 3 threads here, two synthetic, and one anonymized one using
data from Gregor. They test various aspects of thread
ordering/construction in the presence of replies to ghost messages.
2018-09-06 08:07:12 -03:00
David Bremner
dfda1745bf test/tag: add test for updating during batch tag operations
This clarifies that the breakage seen with Xapian 1.4.6 does not have
to do with "funny" tags.

This test is "known broken", but only with xapian 1.4.6, and there's
curently no convenient way to mark that.
2018-08-29 06:37:48 -03:00
Sebastian Poeplau
0f08bf7166 lib: detect mislabeled Windows-1252 parts
Use GMime functionality to detect mislabeled messages and apply the
correct (Windows) encoding instead.
2018-08-29 06:34:39 -03:00
Sebastian Poeplau
7074bb8f80 test: add known broken test for mislabeled Windows-1252 encoding
Messages that contain Windows-1252 are frequently mislabeled as ISO
8859-1, which may result in non-printable characters when displaying
the message. The test asserts that such characters (in this case
curved quotes) are displayed correctly.
2018-08-29 06:28:13 -03:00
David Bremner
46cf1a98f3 test: fix hardcoded paths in T050-new.sh
The previous paths will fail almost everywhere.
2018-05-26 16:46:44 -07:00
Daniel Kahn Gillmor
aa605f7e8a cli/show: enable --decrypt=stash
Add fancy new feature, which makes "notmuch show" capable of actually
indexing messages that it just decrypted.

This enables a workflow where messages can come in in the background
and be indexed using "--decrypt=auto".  But when showing an encrypted
message for the first time, it gets automatically indexed.

This is something of a departure for "notmuch show" -- in particular,
because it requires read/write access to the database.  However, this
might be a common use case -- people get mail delivered and indexed in
the background, but only want access to their secret key to happen
when they're directly interacting with notmuch itself.

In such a scenario, they couldn't search newly-delivered, encrypted
messages, but they could search for them once they've read them.

Documentation of this new feature also uses a table form, similar to
that found in the description of index.decrypt in notmuch-config(1).

A notmuch UI that wants to facilitate this workflow while also
offering an interactive search interface might instead make use of
these additional commands while the user is at the console:

Count received encrypted messages (if > 0, there are some things we
haven't yet tried to index, and therefore can't yet search):

     notmuch count tag:encrypted and \
         not property:index.decryption=success and \
         not property:index.decryption=failure

Reindex those messages:

     notmuch reindex --try-decrypt=true tag:encrypted and \
         not property:index.decryption=success and \
         not property:index.decryption=failure
2018-05-26 07:43:30 -07:00
Daniel Kahn Gillmor
9d114a8552 test-lib: add notmuch_show_part for "notmuch show --format=text"
Thanks to David Bremner for this improved readability!
2018-05-26 07:42:28 -07:00
David Bremner
84b2963726 test: use --full-scan in T050-new.sh
Wherever the test relies on directories being scanned, this option
should be used to avoid skipping them due to mtimes on directories
matching the database.
2018-05-22 09:29:59 -07:00
David Bremner
d1acff6153 test: add tests for notmuch new --full-scan
Most of these just check that adding the flag does not break existing
functionality. The one test that does check the full-scan
functionality had to be rewritten to output debugging info.
2018-05-22 09:29:48 -07:00
Thomas Schneider
b2e4778ea4 build: Allow user to specify ruby executable
This way, one can build for a different Ruby than $PATH/ruby
(e. g. different versions, or Ruby in other paths).

Signed-off-by: Thomas Schneider <qsx@chaotikum.eu>
2018-05-10 21:01:06 -03:00
David Bremner
5ad39ebf75 lib: add thread subqueries.
This change allows queries of the form

 thread:{from:me} and thread:{from:jian} and not thread:{from:dave}

This is still somewhat brute-force, but it's a big improvement over
both the shell script solution and the previous proposal [1], because it
does not build the whole thread structure just generate a
query. A further potential optimization is to replace the calls to
notmuch with more specialized Xapian code; in particular it's not
likely that reading all of the message metadata is a win here.

[1]: id:20170820213240.20526-1-david@tethera.net
2018-05-07 08:42:53 -03:00
Florian Klink
140e8fb118 T460-emacs-tree: sync $EXPECTED with test name
This syncs the $EXPECTED directory name with the tests filename.

Signed-off-by: Florian Klink <flokli@flokli.de>
2018-05-03 21:03:33 -03:00
Daniel Kahn Gillmor
c20a5eb805 move more http -> https
Correct URLs that have crept into the notmuch codebase with http://
when https:// is possible.

As part of this conversion, this changeset also indicates the current
preferred upstream URLs for both gmime and sup.  the new URLs are
https-enabled, the old ones are not.

This also fixes T310-emacs.sh, thanks to Bremner for catching it.
2018-05-03 20:59:20 -03:00
David Bremner
4ac23c5978 test: add known broken test for regexp search of second subject
We expect this to give the same answer as the non-regexp subject
search. It does not because the regexp search relies on the value
slot, which currently contains only one subject.
2018-05-03 07:44:49 -03:00
David Bremner
f3f4297c7f test: tests for python bindings get_property / get_properties
These roughly replicate the equivalent C tests, although they rely on
the database state created by the former tests, since the python
bindings currently provide read-only access to properties.
2018-05-02 20:30:51 -03:00
David Bremner
963ccabe93 Merge branch 'release'
reference loop fixes to be included in 0.26.2
2018-04-25 17:36:32 -03:00
David Bremner
e8ac065a50 test: re-enable disabled test in T700-reindex.sh
The extra test_done looks like a typo
2018-04-24 22:59:21 -03:00
David Bremner
491b1f4b40 lib: choose oldest message when breaking reference loops
This preserves a sensible thread order
2018-04-23 23:00:20 -03:00
David Bremner
4e085b6d92 test: add known broken test for thread ordering from a loop
The previous loop handling code chooses the last message in the
message list, which turns out to be the last in date order.
See the comment in _notmuch_thread_create.
2018-04-23 23:00:20 -03:00
David Bremner
9293d6da27 lib: break reference loop by choosing arbitrary top level msg
Other parts of notmuch (e.g. notmuch show) expect each thread to
contain at least one top level message, and crash if this expectation
is not met.
2018-04-20 11:23:31 -03:00
David Bremner
ab55ca8e0a test: add known broken test for indexing an In-Reply-To loop.
This documents the bug discussed in

     id:87d10042pu.fsf@curie.anarc.at
2018-04-20 11:23:31 -03:00
David Bremner
044cbd920c test: two new messages for the 'broken' corpus
These have an 'In-Reply-To' loop, which currently confuses "notmuch
new".
2018-04-20 11:23:31 -03:00
Georg Faerber
c117306f2d Fix typos as found by codespell
Signed-off-by: Georg Faerber <georg@riseup.net>
2018-03-24 20:09:54 -03:00
Daniel Kahn Gillmor
b6e3efde05 cli/insert: add --world-readable flag
In some cases (e.g. when building a publicly-visible e-mail archive)
it doesn't make any sense to restrict visibility of the message to the
current user account.

This adds a --world-readable boolean option for "notmuch insert", so
that those who want to archive their mail publicly can feed their
archiver with:

    notmuch insert --world-readable

Other local delivery agents (postfix's local, and dovecot's lda) all
default to delivery in mode 0600 rather than relying on the user's
umask, so this fix doesn't change the default.

Also, this does not override the user's umask.  if the umask is
already set tight, it will not become looser as the result of passing
--world-readable.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2018-03-24 20:08:11 -03:00
Daniel Kahn Gillmor
54982e520c fix typos 2018-01-04 20:35:58 -04:00
Daniel Kahn Gillmor
af8255fb71 cli/reply: make --decrypt take a keyword
This brings the --decrypt argument to "notmuch reply" into line with
the other --decrypt arguments (in "show", "new", "insert", and
"reindex").  This patch is really just about bringing consistency to
the user interface.

We also use the recommended form in the emacs MUA when replying, and
update test T350 to match.
2017-12-29 16:45:55 -04:00
Daniel Kahn Gillmor
8ea4a99d74 cli/show: make --decrypt take a keyword.
We also expand tab completion for it, update the emacs bindings, and
update T350, T357, and T450 to match.

Make use of the bool-to-keyword backward-compatibility feature.
2017-12-29 16:45:46 -04:00
Daniel Kahn Gillmor
0ada2a05c9 cli: some keyword options can be supplied with no argument
We might change some notmuch command line tools that used to be
booleans into keyword arguments.

In that case, there are some legacy tools that will expect to be able
to do "notmuch foo --bar" instead of "notmuch foo --bar=baz".

This patch makes it possible to support that older API, while
providing a warning and an encouragement to upgrade.
2017-12-29 16:45:35 -04:00
Daniel Kahn Gillmor
6aec7a76b9 python: add decrypt_policy argument to Database.index_file()
We adopt a pythonic idiom here with an optional argument, rather than
exposing the user to the C indexopts object directly.

This now includes a simple test to ensure that the decrypt_policy
argument works as expected.
2017-12-24 09:47:35 -04:00
Daniel Kahn Gillmor
b1de8e885d test: final named query test works regardless of Xapian FieldProcessor
This test passes with older versions of Xapian as well, because
neither query returns any results.

This should resolve the travis build failure at
https://travis-ci.org/notmuch/notmuch/builds/318571658

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2017-12-21 09:35:20 -04:00
David Bremner
7cfa1c6961 lib: return "" rather than NULL from notmuch_thread_get_authors
The current behaviour is at best under-documented. The modified test in
T470-missing-headers.sh previously relied on printf doing the right
thing with NULL, which seems icky.

The use of talloc_strdup here is probably overkill, but it avoids
having to enforce that thread->authors is never mutated outside
_resolve_thread_authors_string.
2017-12-21 09:22:30 -04:00
Daniel Kahn Gillmor
8e1c3fa5bb test: retrying lock tests are broken when built against older Xapian
If we're building against a version of Xapian that doesn't offer
retrying the lock, we should be honest and describe the tests as
broken, rather than marking them as missing a test prerequisite.

missing test prerequisites should be for specific components of the
test harness that are missing, not for the backend library notmuch
uses.
2017-12-19 07:14:13 -04:00
Daniel Kahn Gillmor
bace15c275 test: named query tests are broken when missing Xapian FieldProcessor
Named queries don't work without Xapian FieldProcessor.  Rather than
silently skipping them, we should explictly mark them as broken when
building against an older version of Xapian.
2017-12-19 07:14:07 -04:00
Daniel Kahn Gillmor
776a2982ae test: searching by absolute date is broken without Xapian FieldProcessor
Previously, the test suite had simply silently skipped the absolute
date test if we're using an archaic version of Xapian.  For
correctness, we should instead mark the test as broken.

This also changes from string to numeric comparison when checking
NOMTUCH_HAVE_XAPIAN_FIELD_PROCESSOR for consistency with other tests.
2017-12-19 07:14:03 -04:00
l-m-h@web.de
660f1a5a33 test: Add test to unset config items with the python bindings 2017-12-19 06:42:50 -04:00