Commit graph

867 commits

Author SHA1 Message Date
David Bremner
dfacfe14f3 lib: query make exclude handling non-destructive
We filter added exclude at add time, rather than modifying the query by
count search. As noted in the comments, there are several ignored
conditions here.
2017-03-04 20:47:25 -04:00
David Bremner
e209b71873 lib: centralize query parsing, store results.
The main goal is to prepare the way for non-destructive (or at least
less destructive) exclude tag handling. It does this by having a
pre-parsed query available for further processing. This also allows us
to provide slightly more precise error messages.
2017-03-04 20:47:25 -04:00
David Bremner
6cb1c617a7 lib: add mid: as a synonym for id:
mid: is the url scheme suggested by URL 2392. We also plan to
introduce more flexible searches for mid: than are possible with
id: (in order not to break assumptions about the special behaviour of
id:, e.g. identifying at most one message).
2017-03-03 17:46:48 -04:00
David Bremner
55524bb063 lib: regexp matching in 'subject' and 'from'
the idea is that you can run

% notmuch search subject:/<your-favourite-regexp>/
% notmuch search from:/<your-favourite-regexp>/

or

% notmuch search subject:"your usual phrase search"
% notmuch search from:"usual phrase search"

This feature is only available with recent Xapian, specifically
support for field processors is needed.

It should work with bindings, since it extends the query parser.

This is easy to extend for other value slots, but currently the only
value slots are date, message_id, from, subject, and last_mod. Date is
already searchable;  message_id is left for a followup commit.

This was originally written by Austin Clements, and ported to Xapian
field processors (from Austin's custom query parser) by yours truly.
2017-03-03 17:46:48 -04:00
David Bremner
508b5c20fa Merge branch 'release'
Second gnugpg test suite fix
2017-02-28 21:21:07 -04:00
Jani Nikula
ea20a932f1 cli/show: add content-disposition to structured output message parts
Help the clients decide how to display parts.

Test updates by Mark Walters <markwalters1009@gmail.com>.
One more test fix by db
2017-02-28 08:03:00 -04:00
David Bremner
1044775fc0 test: move GNUPGHOME to TEST_TMPDIR
We already use this directory for dtach sockets, so it makes sense to
put gnupg sockets there as well. There doesn't seem to be a clean way
to put a fully functional socket in a different location than
GNUPGHOME.
2017-02-27 20:01:42 -04:00
David Bremner
5a42bb96c1 Revert "test: use gpgconf --create-socketdir if available"
This reverts commit e7b88e8b0a.

It turns out that this does not work well in environments without a
running systemd (or some other provider of /run/user)
2017-02-27 18:26:47 -04:00
David Bremner
bd29098e47 test: make T640 compatible with old C compilers
Apparently our test system does not use the same flags for compiling
tests as it does for compiling notmuch. Make the test compatible with
C89. Also remove one unused loop index.
2017-02-26 08:16:45 -04:00
Jani Nikula
75bb23f748 test: test format version difference between v2 and v3
Schemata v3 changed message filename field to a list. Test both
versions 2 and 3.
2017-02-26 07:42:02 -04:00
Jani Nikula
14c60cf168 cli/show: list all filenames of a message in the formatted output
Instead of just having the first filename for the message, list all
duplicate filenames of the message as a list in the formatted
outputs. This bumps the format version to 3.
2017-02-26 07:41:33 -04:00
David Bremner
e0b22c139c lib: handle DatabaseModifiedError in _n_message_ensure_metadata
The retries are hardcoded to a small number, and error handling aborts
than propagating errors from notmuch_database_reopen. These are both
somewhat justified by the assumption that most things that can go
wrong in Xapian::Database::reopen are rare and fatal. Here's the brief
discussion with Xapian upstream:

   24-02-2017 08:12:57 < bremner> any intuition about how likely
      Xapian::Database::reopen is to fail? I'm catching a
      DatabaseModifiedError somewhere where handling any further errors is
      tricky, and wondering about treating a failed reopen as as "the
      impossible happened, stopping"

   24-02-2017 16:22:34 < olly> bremner: there should not be much scope for
    failure - stuff like out of memory or disk errors, which are probably a
    good enough excuse to stop
2017-02-25 21:13:50 -04:00
David Bremner
ae731c183a test: add known broken test for uncaught DatabaseModifiedError
There are several of these to track down, but one that is in quite a
few code paths is _notmuch_message_ensure_metadata.
2017-02-25 21:07:18 -04:00
Tomi Ollila
2c5eed16b3 test: replaced use of python with $NOTMUCH_PYTHON (twice)
$NOTMUCH_PYTHON is sourced from sh.config, configured by
./configure and stated to be used as:

"Name of python command to use in configure and the test suite."
2017-02-23 08:55:24 -04:00
David Bremner
9951598d11 Merge branch 'release'
Merge changes to use gpgconf --create-socketdir
2017-02-21 08:16:07 -04:00
David Bremner
e7b88e8b0a test: use gpgconf --create-socketdir if available
This enables the shortened socket pathes in /run or equivalent. The
explicit call to gpgconf is needed for nonstandard GNUPGHOME settings.

(amended according to id:m2fujatr4k.fsf@guru.guru-group.fi)
2017-02-21 07:45:40 -04:00
Tomi Ollila
f8d5c9b80e test: wrap 'wc -l' results in arithmetic evaluation to strip whitespace
Some new unwrapped 'wc -l's have been added since Jani's 60e79e3a9f
2017-01-27 22:02:36 -04:00
Tomi Ollila
95efe2d484 test: allow user to choose which gdb to run tests with
The variable used for selecting gdb is TEST_GDB, consistent with
TEST_CC and TEST_EMACS{,CLIENT}.
2017-01-08 10:50:28 -04:00
David Bremner
71f1228a51 Merge branch 'release'
Merge changes for notmuch-insert error handling, debian bugfix release
2016-12-07 07:30:12 -04:00
David Bremner
d74c534570 cli/insert: return EX_TEMPFAIL for some errors
Attempt to distinguish between errors indicating misconfiguration or
programmer error, which we consider "permanent", in the sense that
automatic retries are unlikely to be useful, and those indicating
transient error conditions. We consider XAPIAN_EXCEPTION transient
because it covers the important special case of locking failure.
2016-12-07 07:00:40 -04:00
Tomi Ollila
27e293f653 test: gdb insert: redirect input inside gdb script
Running `gdb command < input` is not as reliable way to give input
to the command (some installations of gdb consume it). Use "set args"
gdb command to have input redirected at gdb 'run' time.
2016-12-07 07:00:39 -04:00
Jani Nikula
e682c8f8f8 test: check the handling of files vanishing between scandir and indexing
Add a file for scandir to find, but use gdb to remove it before it
gets indexed.
2016-12-03 07:18:49 -04:00
David Bremner
f781b7e7d2 test: clean up corpora.email
When we renamed corpus to corpora, we missed a spot.
2016-12-03 07:16:12 -04:00
David Bremner
d35c2c15f6 Merge branch 'release'
Initial set of changes for 0.23.3
2016-11-24 21:21:16 -04:00
Daniel Kahn Gillmor
d31161c212 tests: account for varying-size cryptographic signatures
GnuPG 2.1.16 is now injecting the full issuer fingerprint in its
signatures, which makes them about 32 octets larger when
ascii-armored.

This change in size means that the size of the MIME parts will vary
depending on the version of gpg that the user has installed.  at any
rate, the signature part should be non-zero (this is true for
basically any MIME part), so we just test for that instead of an exact
size.
2016-11-24 20:22:12 -04:00
Mark Walters
70c6ac377b Test: fix draft test for emacs23
emacs24 and emacs23 have different secure tag defaults: in particular,
mml-secure-message-sign only signs the part on emacs23 but the whole
message on emacs24. This difference makes one of the draft tests fail
(which causes a cascade of later failures) on emacs23. It seems that
travis uses emacs23 so it is useful to fix this.

We do this by forcing the whole message to be signed in either case --
the code snippet is extracted from mml-secure-message-sign on emacs24.
2016-11-14 18:01:36 -04:00
David Bremner
2dd96d6bd6 emacs: resume messages
Provide functionality to resume editing a message previously saved with
notmuch-draft-save, including decoding the X-Notmuch-Emacs-Secure
header.

Resume gets the raw file from notmuch and using the emacs function
mime-to-mml reconstructs the message (including attachments).

'e' is bound to resume a draft from show or tree mode.
2016-11-13 13:15:31 -04:00
David Bremner
90248f862b emacs: check drafts for encryption tags before saving
In general the user may not want to save plaintext copies of messages
that they are sending encrypted, so give them a chance to abort.
2016-11-13 13:15:31 -04:00
Mark Walters
d820b97477 emacs: postpone a message
This provides initial support for postponing in the emacs frontend;
resuming will follow in a later commit. On saving/postponing it uses
notmuch insert to put the message in the notmuch database

Current bindings are C-x C-s to save a draft, C-c C-p to postpone a
draft (save and exit compose buffer).

Previous drafts get tagged deleted on subsequent saves, or on the
message being sent.

Each draft gets its own message-id, and we use the namespace
draft-.... for draft message ids (so, at least for most people, drafts
are easily distinguisable).
2016-11-13 12:55:24 -04:00
Tomi Ollila
8a82ad716f test/test-lib.sh: execute basename(1)s lazier in test_expect_equal_file ()
Moved the 2 basename(1) executions to the test failure branch in
test_expect_equal_file ().

The output of basename(1) executions in function test_expect_equal_file ()
are only used when tests fails -- when all tests pass these 2 basename(1)
executions are no longer done at all.
2016-10-25 18:13:29 -03:00
David Bremner
d93e9cee13 Merge in test-suite, docstring fixes, release prep 2016-10-22 12:44:39 -03:00
Tomi Ollila
c008dd1352 test: use vt100 as "smart" terminal for known dumb/unknown terminals
Otherwise use whatever user environment has set for TERM so
that there is more chance to test on users' actual environments.
2016-10-22 12:18:43 -03:00
Tomi Ollila
a3ee732c52 test-lib.sh: rename $DTACH_TERM to $SMART_TERM 2016-10-22 12:13:55 -03:00
David Bremner
169e29d431 test: add known broken test for nondestructiveness of count
Thanks to Lucas (id:147263183913.27784.12274024193186585889@mbp) for the
bug report and the test case.

I decided to use the python version because the python bindings could
use more exercise.
2016-10-15 10:22:40 -03:00
Tomi Ollila
6a3ce94aaf test/T000-basic: replaced use of which(1) with shell builtin command -v
Some minimal chroot/container environments don't have which(1) installed.
2016-10-11 22:43:17 -03:00
Tomi Ollila
b57426a435 test: test-lib.sh: use $BASH instead of ${SHELL-sh} to relaunch
In case of the test script is to be relaunced under valgrind, or --tee
is requested, use the $BASH shell variable to locate the command
interpreter. The $SHELL variable is re-set by non-interactive shells
so in case the shell uses some other shell (e.g. zsh) for interactive
use these bash scripts continue to work.
2016-09-24 10:10:40 -03:00
David Bremner
5292e9b1ae test/emacs: add broken test for message replying to msg to self
This is a strange corner case where the removing of the user's address
from the To: header does the wrong thing. If we think it is
worth (eventually) fixing, this test can serve as a reminder.
2016-09-24 10:05:04 -03:00
David Bremner
1d4812fd61 test: fix printf format
notmuch_status_t is an integer type, printing it as a string is a very
bad idea.
2016-09-22 08:11:42 -03:00
Daniel Kahn Gillmor
693ca8d8a8 add property: query prefix to search for specific properties
We want to be able to query the properties directly, like:

   notmuch count property:foo=bar

which should return a count of messages where the property with key
"foo" has value equal to "bar".
2016-09-21 18:14:25 -03:00
David Bremner
651da30c09 cli: optionally restore message properties from dump file
This somewhat mimics the config line parsing, except there can be
arbitrarily many key value pairs, so one more level of looping is
required.
2016-09-21 18:14:25 -03:00
David Bremner
b7345d277e CLI: add properties to dump output
Part of providing extensibility via properties is to make sure that user
data is not lost. Thus we need to be able to dump and restore
properties.
2016-09-21 18:14:25 -03:00
David Bremner
58fe8fce1d lib: iterator API for message properties
This is a thin wrapper around the string map iterator API just introduced.
2016-09-21 18:14:25 -03:00
David Bremner
b8bb6d7964 lib: basic message-property API
Initially, support get, set and removal of single key/value pair, as
well as removing all properties.
2016-09-21 18:14:24 -03:00
Jani Nikula
a63e674b34 test: silence the output of notmuch new mid-test
Fix this during test run:

T470-missing-headers: Testing messages with missing headers
Added 2 new messages to the database.
2016-09-19 20:42:09 -03:00
Mark Walters
a448879e49 emacs: reply: remove wrong sig/enc status buttons
This stops the (usually incorrect) sigstatus and encstatus buttons
appearing when replying in emacs, and updates the test suite to match.

Overriding the status button functions is a little unusual but much
less intrusive than passing an argument all the way down the call
chain. It also makes it clear exactly what it does.

We also hide the application/pgp-encrypted part as it can only contain
"Version: 1". We do this in notmuch show, which means it also happens
when replying.
2016-09-18 10:57:27 -03:00
David Bremner
653c5ee221 test/crypto: test reply to encrypted message in emacs
This test considers the extra output about encryption/signature status
as a bug, to be fixed in the next commit
2016-09-18 10:56:54 -03:00
Jani Nikula
536b1f9df9 cli/reply: return internet address list from get header funcs
Pass in GMimeMessage to simplify To/Cc/Bcc headers. We'll eventually
remove the notmuch message passing altogether, but keep both for now
to not make too big changes at once.

Getting the headers from GMimeMessage using GMime functions fixes the
error on duplicate Cc headers reported by Daniel Kahn Gillmor
<dkg@fifthhorseman.net> in id:87d1ngv95p.fsf@alice.fifthhorseman.net.

Get rid of an intermediate function.

The small annoyance is the ownership differences in the address lists.
2016-09-17 09:11:07 -03:00
Jani Nikula
36416c74e0 test: add known broken test for reply to message with multiple Cc headers
As Daniel Kahn Gillmor <dkg@fifthhorseman.net> reports in
id:87d1ngv95p.fsf@alice.fifthhorseman.net, notmuch show combines
multiple Cc: fields into one, while notmuch reply does not. While such
messages are in violation of RFC 5322, it would be reasonable to
expect notmuch to be consistent. Add a known broken test to document
this expectation.

This also starts a new "broken" corpus for messages which are broken.

Details:

The original message is formatted using the message printing in
notmuch-show.c. For Cc:, it uses g_mime_message_get_recipients(),
which apparently combines all Cc: fields into one internally.

The addresses in the reply headers, OTOH, are based on headers queried
through libnotmuch. It boils down to g_mime_object_get_header() in
lib/message-file.c, which returns only the first occurence of header.
2016-09-17 08:41:29 -03:00
Jani Nikula
971cdc72cd test: make it possible to have multiple corpora
We largely use the corpus under test/corpus for
testing. Unfortunately, many of our tests have grown to depend on
having exactly this set of messages, making it hard to add new message
files for testing specific cases.

We do use a lot of add_message from within the tests, but it's not
possible to use that for adding broken messages, and adding several
messages at once can get unwieldy.

Move the basic corpus under tests/corpora/default, and make it
possible to add new, independent corpora along its side. This means
tons of renames with a few tweaks to add_email_corpus function in
test-lib.sh to let tests specify which corpus to use.
2016-09-17 08:39:34 -03:00
David Bremner
07dff49630 Merge branch 'release'
hand fixed conflicts with NEWS
2016-09-08 22:18:37 -03:00