Commit graph

841 commits

Author SHA1 Message Date
David Bremner
87f1d68440 cli/dump: bump dump format version to 3
No changes to the format of the body, but the header format was
fixed, and version 2 headers probably shouldn't be relied on.
2017-04-01 21:08:32 -03:00
David Bremner
1044775fc0 test: move GNUPGHOME to TEST_TMPDIR
We already use this directory for dtach sockets, so it makes sense to
put gnupg sockets there as well. There doesn't seem to be a clean way
to put a fully functional socket in a different location than
GNUPGHOME.
2017-02-27 20:01:42 -04:00
David Bremner
5a42bb96c1 Revert "test: use gpgconf --create-socketdir if available"
This reverts commit e7b88e8b0a.

It turns out that this does not work well in environments without a
running systemd (or some other provider of /run/user)
2017-02-27 18:26:47 -04:00
David Bremner
e7b88e8b0a test: use gpgconf --create-socketdir if available
This enables the shortened socket pathes in /run or equivalent. The
explicit call to gpgconf is needed for nonstandard GNUPGHOME settings.

(amended according to id:m2fujatr4k.fsf@guru.guru-group.fi)
2017-02-21 07:45:40 -04:00
David Bremner
d74c534570 cli/insert: return EX_TEMPFAIL for some errors
Attempt to distinguish between errors indicating misconfiguration or
programmer error, which we consider "permanent", in the sense that
automatic retries are unlikely to be useful, and those indicating
transient error conditions. We consider XAPIAN_EXCEPTION transient
because it covers the important special case of locking failure.
2016-12-07 07:00:40 -04:00
Tomi Ollila
27e293f653 test: gdb insert: redirect input inside gdb script
Running `gdb command < input` is not as reliable way to give input
to the command (some installations of gdb consume it). Use "set args"
gdb command to have input redirected at gdb 'run' time.
2016-12-07 07:00:39 -04:00
Daniel Kahn Gillmor
d31161c212 tests: account for varying-size cryptographic signatures
GnuPG 2.1.16 is now injecting the full issuer fingerprint in its
signatures, which makes them about 32 octets larger when
ascii-armored.

This change in size means that the size of the MIME parts will vary
depending on the version of gpg that the user has installed.  at any
rate, the signature part should be non-zero (this is true for
basically any MIME part), so we just test for that instead of an exact
size.
2016-11-24 20:22:12 -04:00
Tomi Ollila
c008dd1352 test: use vt100 as "smart" terminal for known dumb/unknown terminals
Otherwise use whatever user environment has set for TERM so
that there is more chance to test on users' actual environments.
2016-10-22 12:18:43 -03:00
Tomi Ollila
a3ee732c52 test-lib.sh: rename $DTACH_TERM to $SMART_TERM 2016-10-22 12:13:55 -03:00
Tomi Ollila
b57426a435 test: test-lib.sh: use $BASH instead of ${SHELL-sh} to relaunch
In case of the test script is to be relaunced under valgrind, or --tee
is requested, use the $BASH shell variable to locate the command
interpreter. The $SHELL variable is re-set by non-interactive shells
so in case the shell uses some other shell (e.g. zsh) for interactive
use these bash scripts continue to work.
2016-09-24 10:10:40 -03:00
David Bremner
5292e9b1ae test/emacs: add broken test for message replying to msg to self
This is a strange corner case where the removing of the user's address
from the To: header does the wrong thing. If we think it is
worth (eventually) fixing, this test can serve as a reminder.
2016-09-24 10:05:04 -03:00
David Bremner
1d4812fd61 test: fix printf format
notmuch_status_t is an integer type, printing it as a string is a very
bad idea.
2016-09-22 08:11:42 -03:00
Daniel Kahn Gillmor
693ca8d8a8 add property: query prefix to search for specific properties
We want to be able to query the properties directly, like:

   notmuch count property:foo=bar

which should return a count of messages where the property with key
"foo" has value equal to "bar".
2016-09-21 18:14:25 -03:00
David Bremner
651da30c09 cli: optionally restore message properties from dump file
This somewhat mimics the config line parsing, except there can be
arbitrarily many key value pairs, so one more level of looping is
required.
2016-09-21 18:14:25 -03:00
David Bremner
b7345d277e CLI: add properties to dump output
Part of providing extensibility via properties is to make sure that user
data is not lost. Thus we need to be able to dump and restore
properties.
2016-09-21 18:14:25 -03:00
David Bremner
58fe8fce1d lib: iterator API for message properties
This is a thin wrapper around the string map iterator API just introduced.
2016-09-21 18:14:25 -03:00
David Bremner
b8bb6d7964 lib: basic message-property API
Initially, support get, set and removal of single key/value pair, as
well as removing all properties.
2016-09-21 18:14:24 -03:00
Jani Nikula
a63e674b34 test: silence the output of notmuch new mid-test
Fix this during test run:

T470-missing-headers: Testing messages with missing headers
Added 2 new messages to the database.
2016-09-19 20:42:09 -03:00
Mark Walters
a448879e49 emacs: reply: remove wrong sig/enc status buttons
This stops the (usually incorrect) sigstatus and encstatus buttons
appearing when replying in emacs, and updates the test suite to match.

Overriding the status button functions is a little unusual but much
less intrusive than passing an argument all the way down the call
chain. It also makes it clear exactly what it does.

We also hide the application/pgp-encrypted part as it can only contain
"Version: 1". We do this in notmuch show, which means it also happens
when replying.
2016-09-18 10:57:27 -03:00
David Bremner
653c5ee221 test/crypto: test reply to encrypted message in emacs
This test considers the extra output about encryption/signature status
as a bug, to be fixed in the next commit
2016-09-18 10:56:54 -03:00
Jani Nikula
536b1f9df9 cli/reply: return internet address list from get header funcs
Pass in GMimeMessage to simplify To/Cc/Bcc headers. We'll eventually
remove the notmuch message passing altogether, but keep both for now
to not make too big changes at once.

Getting the headers from GMimeMessage using GMime functions fixes the
error on duplicate Cc headers reported by Daniel Kahn Gillmor
<dkg@fifthhorseman.net> in id:87d1ngv95p.fsf@alice.fifthhorseman.net.

Get rid of an intermediate function.

The small annoyance is the ownership differences in the address lists.
2016-09-17 09:11:07 -03:00
Jani Nikula
36416c74e0 test: add known broken test for reply to message with multiple Cc headers
As Daniel Kahn Gillmor <dkg@fifthhorseman.net> reports in
id:87d1ngv95p.fsf@alice.fifthhorseman.net, notmuch show combines
multiple Cc: fields into one, while notmuch reply does not. While such
messages are in violation of RFC 5322, it would be reasonable to
expect notmuch to be consistent. Add a known broken test to document
this expectation.

This also starts a new "broken" corpus for messages which are broken.

Details:

The original message is formatted using the message printing in
notmuch-show.c. For Cc:, it uses g_mime_message_get_recipients(),
which apparently combines all Cc: fields into one internally.

The addresses in the reply headers, OTOH, are based on headers queried
through libnotmuch. It boils down to g_mime_object_get_header() in
lib/message-file.c, which returns only the first occurence of header.
2016-09-17 08:41:29 -03:00
Jani Nikula
971cdc72cd test: make it possible to have multiple corpora
We largely use the corpus under test/corpus for
testing. Unfortunately, many of our tests have grown to depend on
having exactly this set of messages, making it hard to add new message
files for testing specific cases.

We do use a lot of add_message from within the tests, but it's not
possible to use that for adding broken messages, and adding several
messages at once can get unwieldy.

Move the basic corpus under tests/corpora/default, and make it
possible to add new, independent corpora along its side. This means
tons of renames with a few tweaks to add_email_corpus function in
test-lib.sh to let tests specify which corpus to use.
2016-09-17 08:39:34 -03:00
David Bremner
07dff49630 Merge branch 'release'
hand fixed conflicts with NEWS
2016-09-08 22:18:37 -03:00
David Bremner
7edded2740 test/crypto: hard code fingerprint
Originally the intent was to make the test more robust against changing
test keys. It turns out that (unscientifically) gpg --with-colons output
changes more often than our test key. Rather than making the script more
complex, just hard code the fingerprint.

This fixes Debian bug #847013; I expect similar test failures as other
distros adopt gnupg 2.1.15
2016-09-08 09:08:09 -03:00
Mark Walters
fce8146a8b emacs: maildir: add the actual insert code
With all the preparation it is now simple to add the actual insert
code. Since insert can fail for many reasons we let the user decide
interactively deal with it.

We modify test-lib.el to set file fcc, so that all the old tests and
emacs_fcc_message from test-lib.sh still work
2016-09-04 08:23:14 -03:00
David Bremner
cf8aabdd37 test: make gdb even quieter
gdb sometimes writes warnings to stdout, which we don't need/want, and
for some reason --batch-silent isn't enough to hide. So in this commit
we write them to a log file, which is probably better for debugging
anyway. To see an illustrative test failure before this change, run

% make
% touch notmuch-count.c
% cd test && ./T060-count.sh

(cherry picked from commit f45fa5bdd3)
2016-08-14 13:27:57 +09:00
Daniel Kahn Gillmor
1ba73d1437 Omit User-Agent: header by default
The User-Agent: header can be fun and interesting, but it also leaks
quite a bit of information about the user and their software stack.

This represents a potential security risk (attackers can target the
particular stack) and also an anonymity risk (a user trying to
preserve their anonymity by sending mail from a non-associated account
might reveal quite a lot of information if their choice of mail user
agent is exposed).

This change also avoids hiding the User-Agent header by default, so
that people who decide they want to send it will at least see it (and
can edit it if they want to) before sending.

It makes sense to have safer defaults.
2016-08-11 20:16:00 +09:00
Mark Walters
999d473299 emacs: wash: word-wrap bugfix
Previously notmuch-wash made the width of the text (approximately) the
window-width minus the depth in thread. This is correct for the
default indentation of 1 per message depth, but is incorrect for any
other setting of notmuch-show-indent-messages-width.

As notmuch-show-indent-messages-width is customisable, and notmuch-tree
sets it to zero to avoid indenting messages in the message pane, this
bug can show up in real use.

Two of the tests had to be updated: when
notmuch-show-indent-messages-width is 0, then the new (correct) word
wrapping happens later, when notmuch-show-indent-messages-width is 4,
then the new word wrapping happens sooner.
2016-08-09 09:34:25 +09:00
David Bremner
f45fa5bdd3 test: make gdb even quieter
gdb sometimes writes warnings to stdout, which we don't need/want, and
for some reason --batch-silent isn't enough to hide. So in this commit
we write them to a log file, which is probably better for debugging
anyway. To see an illustrative test failure before this change, run

% make
% touch notmuch-count.c
% cd test && ./T060-count.sh
2016-08-09 09:33:59 +09:00
David Bremner
48d33532bb test: don't use dump and restore in a pipeline
This has been wrong since bbbdf0478e, but the race condition was not
previously been (often?) triggered in the tests. With the DB_RETRY_LOCK
patches, it manifests itself as a deadlock.
2016-06-30 17:47:36 +02:00
David Bremner
3a45d29ed4 lib: add built_with handling for XAPIAN_DB_RETRY_LOCK
This support will be present only if the appropriate version of xapian
is available _and_ the user did not disable the feature when
building. So there really needs to be some way for the user to check.
2016-06-29 09:05:49 +02:00
David Bremner
6450813e8f test: initial tests for locking retry
Currently there's not much to test, so we simulate contention, and check
that the modifications to the database are serialized.
2016-06-29 09:05:28 +02:00
David Bremner
496dccf44f test: don't use dump and restore in a pipeline
This has been wrong since bbbdf0478e, but the race condition was not
previously been (often?) triggered in the tests. With the DB_RETRY_LOCK
patches, it manifests itself as a deadlock.
2016-06-29 09:03:14 +02:00
Tomi Ollila
be55acf4f0 test/notmuch-test: simple streamlining
A few simple things that applies well to test/notmuch-test.sh

- Shell does pathname expansion also without doing `echo ...` in subshell.

- Redirections >/dev/null 2>/dev/null hide (improbable but) potential
  serious errors; adding -f to rm instead.

- Inter-file capitalization consistency in comments.

- Unnecesary space removal.
2016-06-28 09:23:39 +02:00
David Bremner
cc1a6d2a94 test: factor out some boilerplate from C tests
The trick of having a common header file doesn't work to share between
test scripts, so make an include file in the test directory.

The use of #include <notmuch-test.h> looks slightly pretentious, but
the include file is not actually in the current (temporary) directory.
2016-06-13 21:52:33 -03:00
Tomi Ollila
2333a44ab7 test: test_python: set PYTHONPATH to the python execution environment
Place PYTHONPATH to the environment when python is executed in a way
that current shell environment is not affected. This also allows adding
the old value of PYTHONPATH to the end of the new value (otherwise it
would have been appended again and again when test_python is called).

At the same time, use -B option to avoid writing .pyc files to
bindings/python/* (which are not cleared out by distclean).

Drop the (unused) prefix code which preserved the original stdout of the
python program and opened sys.stdout to OUTPUT. In place of that there
is now note how (debug) information can be printed to original stdout.
2016-06-11 13:12:08 -03:00
Tomi Ollila
f0e5317bcd test: set LD_LIBRARY_PATH early and keep its old contents
Previously LD_LIBRARY_PATH was exported (and environment changed)
in the middle of test case execution, when a function setting it
was called.

Previously the old contents of LD_LIBRARY_PATH was lost (if any)
when it was re-set and exported. In some systems the old contents of
LD_LIBRARY_PATH was needed to e.g. locate suitable gmime library.
2016-06-11 13:11:49 -03:00
Tomi Ollila
36492f20a7 test: fix die() in test-lib-common.sh
In scripts that include test-lib-common.sh but not test-lib.sh
the die() implementation needs to be a bit different due to
fd redirection differences. test-lib-common.sh implements die()
only if it was not implemented already.
2016-06-11 13:03:44 -03:00
Tomi Ollila
d53b417d1f test: add function die () and have use of it in add_email_corpus ()
Added die() function to test-lib.sh with the following first use of it:

If notmuch new fails during email corpus addition the database is
most probably inexistent or broken and the added corpus would be
unusable while running single tests, giving misleading failures
("only" full 'make test' cleans out old corpus).
2016-06-11 13:03:09 -03:00
Tomi Ollila
1ca752fa36 test-lib.sh: renamed die...()s to trap...()s and exit...()
Now the function names more accurately describes what the functions do.
2016-06-11 13:03:00 -03:00
Daniel Kahn Gillmor
6a833a6e83 Use https instead of http where possible
Many of the external links found in the notmuch source can be resolved
using https instead of http.  This changeset addresses as many as i
could find, without touching the e-mail corpus or expected outputs
found in tests.
2016-06-05 08:32:17 -03:00
David Bremner
b9bf3f44ea lib: add support for named queries
This relies on the optional presense of xapian field processors, and the
library config API.
2016-05-25 07:40:44 -03:00
David Bremner
2d2a13966c CLI: add notmuch-config support for named queries
Most of the infrastructure here is general, only the validation/dispatch
is hardcoded to a particular prefix.

A notable change in behaviour is that notmuch-config now opens the
database e.g. on every call to list, which fails with an error message
if the database doesn't exit yet.
2016-05-25 07:40:44 -03:00
David Bremner
c6fcc555dd CLI: optionally restore config data.
The default to restore config data seems safest, especially since
currently we have no config data to mess up.
2016-05-25 07:40:44 -03:00
David Bremner
e042a25a3f CLI: add optional config data to dump output.
Note that it changes the default dump output format, but doesn't break
existing notmuch-restore. It might break user scripts though.
2016-05-25 07:07:56 -03:00
David Bremner
92e59568fa lib: config list iterators
Since xapian provides the ability to restrict the iterator to a given
prefix, we expose this ability to the user. Otherwise we mimic the other
iterator interfances in notmuch (e.g. tags.c).
2016-05-25 06:51:16 -03:00
David Bremner
3458e3c89c lib: provide config API
This is a thin wrapper around the Xapian metadata API. The job of this
layer is to keep the config key value pairs from colliding with other
metadata by transparently prefixing the keys, along with the usual glue
to provide a C interface.

The split of _get_config into two functions is to allow returning of the
return value with different memory ownership semantics.
2016-05-24 08:53:03 -03:00
Ludovic LANGE
7e6e23c36e ruby: add bindings for notmuch_database_get_all_tags
The Ruby bindings were missing a way to get all the tags of the
database. Now you should be able to access this with the public
instance method `all_tags` of your database object.

Example of use:
    notmuchdb = Notmuch::Database.new path, { :create => false,
    	:mode => Notmuch::MODE_READ_ONLY }

    my_tags = notmuchdb.all_tags

    my_tags.each { |tag|
      print tag
    }

    my_tags.destroy!

Amended by db: improve error reporting, add test
2016-05-19 08:02:43 -03:00
Tomi Ollila
b183f2635e test: copyright information updates
Files in test directories had only copyright of a single individual,
of which code was adapted here as a base of the test system.
Since then many Notmuch Developers have contributed to the test
system, which is now acknowledged with a constant string in some
of the test files.

The README file in test directory instructed new files contain a
copyright notice, but that has never been done (and it is also not
needed). To simplify things a bit (and lessen confusion) this
instruction is now removed.

As a side enchangement, all of the 3 entries in the whole source
tree cd'ing to `dirname` of "$0" now uses syntax cd "$(dirname "$0")".
This makes these particular lines work when current working directory
is e.g. /c/Program Files/notmuch/test/.
(Probably it would fail elsewhere, though.)
2016-05-19 07:56:58 -03:00