Commit graph

936 commits

Author SHA1 Message Date
Daniel Kahn Gillmor
e5beec39d6 add "notmuch reindex" subcommand
This new subcommand takes a set of search terms, and re-indexes the
list of matching messages.
2017-08-01 21:17:47 -04:00
David Bremner
ed4a9082c0 cli/search: print total number of files matched in summary output.
The structured output formats already have all of the filenames. This
is an easy bit of UI change to make the multiple files visible.
2017-08-01 21:17:47 -04:00
David Bremner
411675a6ce lib: index message files with duplicate message-ids
The corresponding xapian document just gets more terms added to it,
but this doesn't seem to break anything. Values on the other hand get
overwritten, which is a bit annoying, but arguably it is not worse to
take the values (from, subject, date) from the last file indexed
rather than the first.
2017-08-01 21:17:47 -04:00
David Bremner
639aced9c9 test: add known broken tests for duplicate message id
There are many other problems that could be tested, but these ones we
have some hope of fixing because it doesn't require UI changes, just
indexing changes.
2017-08-01 21:17:47 -04:00
David Bremner
1d82110f83 emacs: convert remaining format-versions from 3 to 4
This is needed for consistent beheviour between notmuch built against
gmime-2.6 and gmime-3.0 w.r.t. error reporting.
2017-07-18 06:57:26 -03:00
Daniel Kahn Gillmor
87bdfbc91f Fix orthography 2017-07-18 06:50:44 -03:00
David Bremner
f1bcb6b7c2 emacs: change default for notmuch-crypto-process-mime to t
There are some cases like remote usage where this might cause
problems, but those users can easily customize the variable. The
inconvenience seems to be outweighed by the security benefit for most
users.
2017-07-15 09:11:32 -03:00
Daniel Kahn Gillmor
e142de643d config: deprecate/drop crypto.gpg_path under gmime 2.6/3.0
gmime 3.0 no longer offers a means to set the path for gpg.

Users can set $PATH anyway if they want to pick a
differently-installed gpg (e.g. /usr/local/bin/gpg), so this isn't
much of a reduction in functionality.

The one main difference is for people who have tried to use "gpg2" to
make use of gpg 2.1, but that isn't usefully co-installable anyway.
2017-07-14 21:23:52 -03:00
Daniel Kahn Gillmor
8492298a61 Add additional munged reply-to tests
The reply-to munging code might behave differently whether there's an
exact match on the strings or not, or whether the string is a raw
addr-spec instead of an name-addr.  These tests cover those variations
(i also had to tweak json output further below when this new test was
added).
2017-07-14 21:23:52 -03:00
Daniel Kahn Gillmor
23a4d09773 Ensure that "notmuch reply" succeeds during testing.
In some (bad!) cases, "notmuch reply" might fail, or might even
segfault.  If this happens, it indicates a bug, and the test suite
should notice it.
2017-07-14 21:23:52 -03:00
David Bremner
a58c75fdf6 test/crypto: mark extra space in userid as a bug in gmime-2.6
I can't see the space in the output of gpg -K --with-colons
2017-07-14 17:58:09 -03:00
David Bremner
a32dd81a55 cli: add space separator for keyword, string, and int arguments
Defer the complication of optional boolean arguments for later (never?).
2017-07-12 17:46:29 -03:00
David Bremner
de9baa29dc cli: change api of parse_option
The idea is to allow it (in a future commit) advance to the next argv
element to get a value
2017-07-12 17:44:15 -03:00
David Bremner
87ca29b7a7 test/crypto: add test for corrupted signatures
Have an empty list of signatures is not the nicest mechanism to signal
complete failure to verify, but it's the one we currently have.
2017-07-05 07:36:20 -03:00
David Bremner
4f48ae4a5f test: add test for modified pgp/mime signed message
This is a good thing to test in any case, but particularly as we hit
the dark corners of the gmime crypto API.
2017-07-05 07:36:20 -03:00
David Bremner
71f04b9a67 test: mark test as broken in gmime 3.0
Currently I'm not sure what the intent of this test is, so it's not
clear if the new answer is better or worse than the old one.
2017-07-05 07:13:02 -03:00
David Bremner
f7dd69d4c6 test: mark inclusion of headers as broken in gmime-2.x
The output of headers here reflects an underlying bug / quirk of
gmime-2.6.
2017-07-05 07:12:20 -03:00
David Bremner
3805898e3e test/multipart: reorganize creation of multipart message
We want to have the bodies of the multipart available in a file on
their own for planned modifications to tests.
2017-07-05 07:12:08 -03:00
David Bremner
e1b7d32db0 cli: implement structured output version 4
Since the error field is unused by the emacs front end, no changes are
needed other than bumping the format version number.

As it is, this is a bit overengineered, but it will reduce duplication
when we support gmime 3.0
2017-07-04 08:32:44 -03:00
David Bremner
b6f87c3085 emacs: convert to use format-version 3
Keeping the emacs interface running old format-versions makes it
impossible to test new format-versions or deprecate old ones.
2017-07-04 08:32:28 -03:00
David Bremner
6dd00d6486 lib/index: add simple html filter
The filter just drops all (HTML) tags. As an enabling change, pass the
content type to the filter constructor so we can decide which scanner
to user.
2017-07-01 12:32:27 -03:00
David Bremner
1ec634461e Merge branch 'release' 2017-06-01 07:54:38 -03:00
David Bremner
11fda0bf2f cli/dump: don't include tags when not asked for
Add in the analogous test for tags that is given for properties a few
lines below.
2017-06-01 07:07:32 -03:00
David Bremner
e68022416d test: add known broken test --include=properties
Reported in [1], --include=properties currently implies
--include=tags, but it should not.

[1] id:87y3u8vjbo.fsf@tethera.net
2017-06-01 07:03:06 -03:00
David Bremner
6ac3d8c27f test: define GMime version dependant breakage
We have some tests where the gmime 3 behaviour seems like a bug fix,
others where it's less clear, so we allow both possibilities.
2017-05-31 21:54:08 -03:00
David Bremner
51cf6dbab2 test/thread-naming: remove excess escaping from sender address.
This is another case where the behaviour of gmime-2.6 and gmime-3.0
seems to differ. It may be that we prefer the more lax parsing of the
previous version, but that should be tested separately.
2017-05-30 09:02:51 -03:00
Tomi Ollila
5f71a4d3e1 exit lingering gpg agents at the end of relevant tests
Since gnupg 2.1.20, gpg-agent no longer shut itself down when
$GNUPGHOME directory is removed.

Add exit hooks to the test modules which execute `gpgconf --kill all`

Add exit hooks to execute `gpgconf --kill all` in the modules that
create $GNUPGHOME for gpg to work with.
2017-05-26 07:26:06 -03:00
Tomi Ollila
eb157f8841 test-lib.sh: add "atexit" functionality
New function at_exit_function registers given function to be called
at script termination.

Functions so registered are called in the reverse order of their
registration; no arguments are passed.

Function is called only once; re-adding with function name already
registered will remove previous registration.

New function rm_exit_function can be used to remove registration.

Modules (and possibly test-lib.sh functions) in future commits will
register such functions.
2017-05-26 07:25:55 -03:00
Jani Nikula
bc11759dd1 build: switch to hiding libnotmuch symbols by default
The dynamic generation of the linker version script for libnotmuch
exports has grown rather complicated.

Reverse the visibility control by hiding symbols by default using
-fvisibility=hidden, and explicitly exporting symbols in notmuch.h
using #pragma GCC visibility. (We could also use __attribute__
((visibility ("default"))) for each exported function, but the pragma
is more convenient.)

The above is not quite enough alone, as it would "leak" a number of
weak symbols from Xapian and C++ standard library. Combine it with a
small static version script that filters out everything except the
notmuch_* symbols that we explicitly exposed, and the C++ RTTI
typeinfo symbols for exception handling.

Finally, as the symbol hiding test can no longer look at the generated
symbol table, switch the test to parse the functions from notmuch.h.
2017-05-12 07:17:18 -03:00
David Bremner
11d47950c1 lib: Add regexp expansion for for tags and paths
From a UI perspective this looks similar to what was already provided
for from, subject, and mid, but the implementation is quite
different. It uses the database's list of terms to construct a term
based query equivalent to the passed regular expression.
2017-05-09 07:44:29 -03:00
David Bremner
eab365c742 lib: Add regexp searching for mid: prefix
The bulk of the change is passing in the field options to the regexp
field processor, so that we can properly handle the
fallback (non-regexp case).
2017-05-09 07:44:15 -03:00
David Bremner
77c9ec1fdd test: add known broken test for indexing html
'quite' on IRC reported that notmuch new was grinding to a halt during
initial indexing, and we eventually narrowed the problem down to some
html parts with large embedded images. These cause the number of terms
added to the Xapian database to explode (the first 400 messages
generated 4.6M unique terms), and of course the resulting terms are
not much use for searching.

The second test is sanity check for any "improved" indexing of HTML.
2017-04-20 06:59:40 -03:00
David Bremner
e08f5f76e4 test: add 'lkml' corpus
These 210 messages are in several long threads, which is good for
testing our threading code, and may be useful just as a larger test
corpus in the future.
2017-04-13 21:55:43 -03:00
David Bremner
ce8c88824a test: remove debugging "output" 2017-04-06 14:37:11 -03:00
David Bremner
ec3937b5cd test: standardize argument order to test_expect_equal_file
It is annoying to debug failing tests when the interpretation of the
diffs is reversed for some tests.
2017-04-06 14:37:02 -03:00
David Bremner
33e5802195 Merge branch 'release'
Final 0.24.1 release
2017-04-01 09:36:34 -03:00
David Bremner
d47e184118 cli/dump: bump dump format version to 3
No changes to the format of the body, but the header format was
fixed, and version 2 headers probably shouldn't be relied on.
2017-04-01 09:10:15 -03:00
David Bremner
1f3c7916f8 cli/dump: fix bug in dump header
Fix copy paste error.
2017-04-01 09:09:33 -03:00
David Bremner
704bd3d8a2 test: add known broken test for dump header
Apparently nobody uses the list of "what was included in the dump"
2017-04-01 09:09:20 -03:00
David Bremner
c0a3bd956a test: add known broken test for dump header
Apparently nobody uses the list of "what was included in the dump"
2017-03-31 21:18:54 -03:00
David Bremner
e1c1d33f37 Merge branch 'release'
Another regexp search fix.
2017-03-29 20:58:34 -03:00
David Bremner
cb84f84878 lib: handle empty string in regexp field processors
The non-field processor behaviour is is convert the corresponding
queries into a search for the unprefixed terms. This yields pretty
surprising results so I decided to generate a query that would match
the terms (i.e. none with that prefix) generated for an empty header.
2017-03-29 20:44:32 -03:00
David Bremner
bc0c051e04 test: add known broken test for null from: and subject: query
These queries currently fail with field processors enabled because the
code expects a non-empty string.
2017-03-29 20:44:09 -03:00
David Bremner
d877240f4e Merge branch 'release'
wildcard search fixes, plus release busywork
2017-03-25 11:51:03 -03:00
Mikhail
9b7dbed58e tests: add compatibility layer
Make test-lib-common.sh load test-lib-<$PLATFORM>.sh to create
additional shim for platform specifics.

Use test-lib-FREEBSD.sh to call GNU utilities instead of native ones.

- amended by db following Tomi's suggestions
2017-03-25 07:56:52 -03:00
David Bremner
38a56b98f9 lib: only trigger phrase processing for regexp fields when needed
The argument is that if the string passed to the field processor has
no spaces, then the added quotes won't have any benefit except for
disabling wildcards. But disabling wildcards doesn't seem very useful
in the normal Xapian query parser, since they're stripped before
generating terms anyway. It does mean that the query 'from:"foo*"' will
not be precisely equivalent to 'from:foo' as it is for the non
field-processor version.
2017-03-24 09:24:13 -03:00
David Bremner
497b83780e test: add known broken tests wildcard search in from and subject
This was broken by the addition of regexp searching. The detection of
wildcards is not currently done in the recursive call to parse_query,
because of quoting issues.
2017-03-24 09:24:13 -03:00
David Bremner
5ce8e0b11b lib: replace deprecated n_q_count_messages with status returning version
This function was deprecated in notmuch 0.21.  We re-use the name for
a status returning version, and deprecate the _st name. One or two
remaining uses of the (removed) non-status returning version fixed at
the same time
2017-03-22 08:35:07 -03:00
David Bremner
86cbd215eb lib: replace deprecated n_q_search_messages with status returning version
This function was deprecated in notmuch 0.21.  We re-use the name for
a status returning version, and deprecate the _st name.
2017-03-22 08:35:07 -03:00
David Bremner
c39f6361d0 rename libutil.a to libnotmuch_util.a
Apparently some systems (MacOS?) have a system library called libutil
and the name conflict causes problems. Since this library is quite
notmuch specific, rename it to something less generic.
2017-03-18 21:37:43 -03:00