Commit graph

916 commits

Author SHA1 Message Date
David Bremner
6dd00d6486 lib/index: add simple html filter
The filter just drops all (HTML) tags. As an enabling change, pass the
content type to the filter constructor so we can decide which scanner
to user.
2017-07-01 12:32:27 -03:00
David Bremner
1ec634461e Merge branch 'release' 2017-06-01 07:54:38 -03:00
David Bremner
11fda0bf2f cli/dump: don't include tags when not asked for
Add in the analogous test for tags that is given for properties a few
lines below.
2017-06-01 07:07:32 -03:00
David Bremner
e68022416d test: add known broken test --include=properties
Reported in [1], --include=properties currently implies
--include=tags, but it should not.

[1] id:87y3u8vjbo.fsf@tethera.net
2017-06-01 07:03:06 -03:00
David Bremner
6ac3d8c27f test: define GMime version dependant breakage
We have some tests where the gmime 3 behaviour seems like a bug fix,
others where it's less clear, so we allow both possibilities.
2017-05-31 21:54:08 -03:00
David Bremner
51cf6dbab2 test/thread-naming: remove excess escaping from sender address.
This is another case where the behaviour of gmime-2.6 and gmime-3.0
seems to differ. It may be that we prefer the more lax parsing of the
previous version, but that should be tested separately.
2017-05-30 09:02:51 -03:00
Tomi Ollila
5f71a4d3e1 exit lingering gpg agents at the end of relevant tests
Since gnupg 2.1.20, gpg-agent no longer shut itself down when
$GNUPGHOME directory is removed.

Add exit hooks to the test modules which execute `gpgconf --kill all`

Add exit hooks to execute `gpgconf --kill all` in the modules that
create $GNUPGHOME for gpg to work with.
2017-05-26 07:26:06 -03:00
Tomi Ollila
eb157f8841 test-lib.sh: add "atexit" functionality
New function at_exit_function registers given function to be called
at script termination.

Functions so registered are called in the reverse order of their
registration; no arguments are passed.

Function is called only once; re-adding with function name already
registered will remove previous registration.

New function rm_exit_function can be used to remove registration.

Modules (and possibly test-lib.sh functions) in future commits will
register such functions.
2017-05-26 07:25:55 -03:00
Jani Nikula
bc11759dd1 build: switch to hiding libnotmuch symbols by default
The dynamic generation of the linker version script for libnotmuch
exports has grown rather complicated.

Reverse the visibility control by hiding symbols by default using
-fvisibility=hidden, and explicitly exporting symbols in notmuch.h
using #pragma GCC visibility. (We could also use __attribute__
((visibility ("default"))) for each exported function, but the pragma
is more convenient.)

The above is not quite enough alone, as it would "leak" a number of
weak symbols from Xapian and C++ standard library. Combine it with a
small static version script that filters out everything except the
notmuch_* symbols that we explicitly exposed, and the C++ RTTI
typeinfo symbols for exception handling.

Finally, as the symbol hiding test can no longer look at the generated
symbol table, switch the test to parse the functions from notmuch.h.
2017-05-12 07:17:18 -03:00
David Bremner
11d47950c1 lib: Add regexp expansion for for tags and paths
From a UI perspective this looks similar to what was already provided
for from, subject, and mid, but the implementation is quite
different. It uses the database's list of terms to construct a term
based query equivalent to the passed regular expression.
2017-05-09 07:44:29 -03:00
David Bremner
eab365c742 lib: Add regexp searching for mid: prefix
The bulk of the change is passing in the field options to the regexp
field processor, so that we can properly handle the
fallback (non-regexp case).
2017-05-09 07:44:15 -03:00
David Bremner
77c9ec1fdd test: add known broken test for indexing html
'quite' on IRC reported that notmuch new was grinding to a halt during
initial indexing, and we eventually narrowed the problem down to some
html parts with large embedded images. These cause the number of terms
added to the Xapian database to explode (the first 400 messages
generated 4.6M unique terms), and of course the resulting terms are
not much use for searching.

The second test is sanity check for any "improved" indexing of HTML.
2017-04-20 06:59:40 -03:00
David Bremner
e08f5f76e4 test: add 'lkml' corpus
These 210 messages are in several long threads, which is good for
testing our threading code, and may be useful just as a larger test
corpus in the future.
2017-04-13 21:55:43 -03:00
David Bremner
ce8c88824a test: remove debugging "output" 2017-04-06 14:37:11 -03:00
David Bremner
ec3937b5cd test: standardize argument order to test_expect_equal_file
It is annoying to debug failing tests when the interpretation of the
diffs is reversed for some tests.
2017-04-06 14:37:02 -03:00
David Bremner
33e5802195 Merge branch 'release'
Final 0.24.1 release
2017-04-01 09:36:34 -03:00
David Bremner
d47e184118 cli/dump: bump dump format version to 3
No changes to the format of the body, but the header format was
fixed, and version 2 headers probably shouldn't be relied on.
2017-04-01 09:10:15 -03:00
David Bremner
1f3c7916f8 cli/dump: fix bug in dump header
Fix copy paste error.
2017-04-01 09:09:33 -03:00
David Bremner
704bd3d8a2 test: add known broken test for dump header
Apparently nobody uses the list of "what was included in the dump"
2017-04-01 09:09:20 -03:00
David Bremner
c0a3bd956a test: add known broken test for dump header
Apparently nobody uses the list of "what was included in the dump"
2017-03-31 21:18:54 -03:00
David Bremner
e1c1d33f37 Merge branch 'release'
Another regexp search fix.
2017-03-29 20:58:34 -03:00
David Bremner
cb84f84878 lib: handle empty string in regexp field processors
The non-field processor behaviour is is convert the corresponding
queries into a search for the unprefixed terms. This yields pretty
surprising results so I decided to generate a query that would match
the terms (i.e. none with that prefix) generated for an empty header.
2017-03-29 20:44:32 -03:00
David Bremner
bc0c051e04 test: add known broken test for null from: and subject: query
These queries currently fail with field processors enabled because the
code expects a non-empty string.
2017-03-29 20:44:09 -03:00
David Bremner
d877240f4e Merge branch 'release'
wildcard search fixes, plus release busywork
2017-03-25 11:51:03 -03:00
Mikhail
9b7dbed58e tests: add compatibility layer
Make test-lib-common.sh load test-lib-<$PLATFORM>.sh to create
additional shim for platform specifics.

Use test-lib-FREEBSD.sh to call GNU utilities instead of native ones.

- amended by db following Tomi's suggestions
2017-03-25 07:56:52 -03:00
David Bremner
38a56b98f9 lib: only trigger phrase processing for regexp fields when needed
The argument is that if the string passed to the field processor has
no spaces, then the added quotes won't have any benefit except for
disabling wildcards. But disabling wildcards doesn't seem very useful
in the normal Xapian query parser, since they're stripped before
generating terms anyway. It does mean that the query 'from:"foo*"' will
not be precisely equivalent to 'from:foo' as it is for the non
field-processor version.
2017-03-24 09:24:13 -03:00
David Bremner
497b83780e test: add known broken tests wildcard search in from and subject
This was broken by the addition of regexp searching. The detection of
wildcards is not currently done in the recursive call to parse_query,
because of quoting issues.
2017-03-24 09:24:13 -03:00
David Bremner
5ce8e0b11b lib: replace deprecated n_q_count_messages with status returning version
This function was deprecated in notmuch 0.21.  We re-use the name for
a status returning version, and deprecate the _st name. One or two
remaining uses of the (removed) non-status returning version fixed at
the same time
2017-03-22 08:35:07 -03:00
David Bremner
86cbd215eb lib: replace deprecated n_q_search_messages with status returning version
This function was deprecated in notmuch 0.21.  We re-use the name for
a status returning version, and deprecate the _st name.
2017-03-22 08:35:07 -03:00
David Bremner
c39f6361d0 rename libutil.a to libnotmuch_util.a
Apparently some systems (MacOS?) have a system library called libutil
and the name conflict causes problems. Since this library is quite
notmuch specific, rename it to something less generic.
2017-03-18 21:37:43 -03:00
Tomi Ollila
9c3a27dee9 test-databases: use wget or curl to download test databases
Often Linux systems are shipped with wget(1) by default (and no curl).

Many BSDs, macOS, and e.g. some Linux minimal/container images
comes with curl(1) (and no wget).

Attempting to download with curl if wget is not available increases
the likelihood for this to succeed.
2017-03-15 22:06:12 -03:00
David Bremner
62822a4e2d lib: clamp return value of g_mime_utils_header_decode_date to >=0
For reasons not completely understood at this time, gmime (as of
2.6.22) is returning a date before 1900 on bad date input. Since this
confuses some other software, we clamp such dates to 0,
i.e. 1970-01-01.
2017-03-15 21:58:25 -03:00
David Bremner
8e7812d824 lib: add known broken test for parsing bad dates.
This reproduces the symptoms of bug report
id:20150422065630.6330.90536@hoothoot
2017-03-15 21:52:34 -03:00
Tomi Ollila
936c35da86 fix out of tree tests
Use $NOTMUCH_SRCDIR/ instead of $TEST_DIRECTORY/../ (in those 2 places)
where reference to source directory instead of build directory is
required.
2017-03-12 09:19:23 -03:00
Tomi Ollila
d6a735423b test: atomicity.py: improve exit probability on failure
Some gdb python exceptions on some os environments (e.g. macOS
Sierra, non-codesigned gdb) do not make gdb exit (but to drop down
to nonexistent command line?).
Mitigate this chance by explict SystemExit on all exceptions.
The contents of output file 'gdb.out' is unchanged.
2017-03-10 08:21:28 -04:00
Gaute Hope
1b5c6b2dea bindings/python: test python Database.get_revision ()
Here is a test for Database.get_revision (), based on
the first part of T570-revision-tracking.sh.
2017-03-09 10:21:02 -04:00
Jani Nikula
0497d695ca test: require test_begin_subtest before test_expect_code
Unify the subtests by requiring test_begin_subtest before
test_expect_code. (Similar change for test_expect_success has already
been done.)

This increases clarity in the test scripts by having a separate line
for the start of the subtest with the heading, and makes it possible
to simplify the test infrastructure by making all subtests similar.
2017-03-09 09:03:40 -04:00
Jani Nikula
d0cd253b37 test: require test_begin_subtest before test_expect_success
Unify the subtests by requiring test_begin_subtest before
test_expect_success. (Similar change for test_expect_code will
follow.)

This increases clarity in the test scripts by having a separate line
for the start of the subtest with the heading, and makes it possible
to simplify the test infrastructure by making all subtests similar.
2017-03-09 09:01:21 -04:00
Jani Nikula
b8f12bd371 test: drop the implicit prereq check mechanism from test_expect_*
The only place where we use the implicit prereq check is T000-basic.sh
where we check that it works. It's an added complication that we don't
use. Remove it.

The test_have_prereq function can still be used for the same effect in
subtests that use test_begin_subtest. For now, this will make it
impossible to have prereqs in one-line subtests that don't require
test_begin_subtest. This will be fixed in follow-up work.
2017-03-09 09:00:16 -04:00
Jani Nikula
e563b8cafd test: ensure test_begin_subtest has been called before test_expect_*
This is the expectation, increase robustness of the test suite by
requiring it.
2017-03-09 08:58:28 -04:00
Jani Nikula
13a59c4a8f test: only accept short and long options, not silly in-betweens
It's not notmuch style to accept sloppy parameter names.
2017-03-09 08:55:38 -04:00
Jani Nikula
de66230b46 test: remove unused and no-op --long-tests parameter
It's been unused since its introduction in commit 0083854b12 ("Copy
test framework from Git").
2017-03-09 08:55:17 -04:00
Jani Nikula
8af6fc5e6e test: remove unused test_external and test_external_without_stderr
They've been unused since their introduction in commit 0083854b12
("Copy test framework from Git"), only causing maintenance burden.
2017-03-09 08:55:06 -04:00
Jani Nikula
cfc0909511 test: remove unused filter functions
They've been unused since their introduction in commit 0083854b12
("Copy test framework from Git").
2017-03-09 08:54:53 -04:00
Jani Nikula
1db3244bf1 test: remove unused regexp convenience variables
They've been unused since their introduction in commit 0083854b12
("Copy test framework from Git").
2017-03-09 08:53:12 -04:00
Jani Nikula
14c0862bac test: suppress diff for broken test without V=1
Known broken tests are, well, known broken. Do not print the result
diff for them unless V=1 is specified. Now that the test description
is printed also when known broken tests fail, the user can also skip
to running the individual failing tests.
2017-03-09 08:48:08 -04:00
Jani Nikula
740f3296b8 test: print test description also for failing known broken tests
With the test description, the user can see the test script name, and
debug with that alone.
2017-03-09 08:47:57 -04:00
Jani Nikula
85df8b3915 test: shrink T590-thread-breakage test decription to one line
The test description is used for log output, I think the intention is
to keep it as a one-liner. Leave the rest of the long description as a
comment.
2017-03-04 21:21:11 -04:00
Mark Walters
a8f91b74e7 Test: emacs: test for folding long headers.
Headers of more than 998 characters should be folded when sending.
However, until recently, emacs did not do this.

This adds a (known broken) test for this when sending messages in
emacs. We will backport the fix to notmuch-emacs in the next
changeset.
2017-03-04 21:15:55 -04:00
David Bremner
dfacfe14f3 lib: query make exclude handling non-destructive
We filter added exclude at add time, rather than modifying the query by
count search. As noted in the comments, there are several ignored
conditions here.
2017-03-04 20:47:25 -04:00