Commit graph

3545 commits

Author SHA1 Message Date
Peter Wang
b96ba6326a show: indicate length, encoding of omitted body content
If a leaf part's body content is omitted, return the encoded length and
transfer encoding in --format=json output.  This information may be used
by the consumer, e.g. to decide whether to download a large attachment
over a slow link.

Returning the _encoded_ content length is more efficient than returning
the _decoded_ content length.  Returning the transfer encoding allows
the consumer to estimate the decoded content length.
2012-12-17 09:10:35 -04:00
Peter Wang
35860e00d1 show: indicate charset for all omitted parts
Write a "charset" field for all omitted parts for which it is applicable,
not only text/html parts. Factor out the code to a separate function.
It will be extended with more fields next.
2012-12-17 09:10:00 -04:00
Peter Wang
ee425ae2ad test: normalize only message filenames in show json
notmuch_json_show_sanitize replaced "filename" field values even in part
structures, where the value is predictable.  Make it only normalize the
filename value if it is an absolute path (begins with slash), which is
true of the Maildir filenames that were intended to be normalized away.
2012-12-17 09:08:04 -04:00
Austin Clements
2cdb3f54f7 emacs: Use --format-version for search, show, and reply 2012-12-16 17:22:26 -04:00
Austin Clements
0df6dcfe76 emacs: Special handling for version mismatch errors
Since Emacs has more semantic information, we suppress the generic
format version error from the CLI and give a more informative error.
2012-12-16 17:22:14 -04:00
Austin Clements
e723e21f75 test: Sanity tests for the --format-version argument 2012-12-16 17:21:49 -04:00
Austin Clements
f6adf6615c reply: Support --format-version 2012-12-16 17:21:32 -04:00
Austin Clements
aecff2c9fd show: Support --format-version 2012-12-16 17:21:00 -04:00
Austin Clements
3152cf4b12 search: Support --format-version 2012-12-16 17:20:43 -04:00
Austin Clements
1c6195b9e3 cli: Framework for structured output versioning
Currently there is a period of pain whenever we make
backward-incompatible changes to the structured output format, which
discourages not only backward-incompatible improvements to the format,
but also backwards-compatible additions that may not be "perfect".  In
the end, these problems limit experimentation and innovation.

This series of patches introduces a way for CLI callers to request a
specific format version on the command line and to determine if the
CLI does not supported the requested version (and perhaps present a
useful diagnostic to the user).  Since the caller requests a format
version, it's also possible for the CLI to support multiple
incompatible versions simultaneously, unlike the alternate approach of
including version information in the output.

This patch lays the groundwork by introducing a versioning convention,
standard exit codes, and a utility function to check the requested
version and produce standardized diagnostic messages and exit
statuses.
2012-12-16 17:20:33 -04:00
Austin Clements
1e12b91b3c test: Test search's handling of subprocess errors 2012-12-16 17:17:58 -04:00
Austin Clements
19e5b2d912 emacs: Use unified error handling in search
This slightly changes the output of an existing test since we now
report non-zero exits with a pop-up buffer instead of at the end of
the search results.
2012-12-16 17:17:41 -04:00
Austin Clements
b3dc31f78d test: Test show's handling of subprocess errors 2012-12-16 17:04:24 -04:00
Austin Clements
e1d5e88156 emacs: Improve error handling for notmuch-call-notmuch-json
This checks for non-zero exit status from JSON CLI calls and pops up
an error buffer with stderr and stdout.  A consequence of this is that
show and reply now handle errors, rather than ignoring them.
2012-12-16 17:04:08 -04:00
Austin Clements
66c935cff3 emacs: Factor out synchronous notmuch JSON invocations
Previously this code was duplicated between show and reply.  This
factors out synchronously invoking notmuch and parsing the output as
JSON.
2012-12-16 17:00:22 -04:00
Austin Clements
0844af35eb emacs: Use unified error handling in notmuch-call-notmuch-process
This makes notmuch-call-notmuch-process use the unified CLI error
handling, which basically refines the error handling this function
already did.
2012-12-16 17:00:00 -04:00
Austin Clements
693cf03cee emacs: Centralize notmuch command error handling
This provides library functions for unified handling of errors from
the notmuch CLI.  Follow-up patches will convert some scattered error
handling to use this and add error handling where we currently ignore
errors.
2012-12-16 16:58:14 -04:00
Mark Walters
780a98d14e contrib: pick: archive message updated
Update pick's archive message to respect notmuch-archive-tags. Also
split archive message into an archiving part and a separate
"then-next" part, to move more inline with show. Update the keybinding
so default behaviour is unchanged.
2012-12-15 13:48:02 -04:00
Austin Clements
d0ebd6cb53 emacs: Fix bug in resynchronizing after a JSON parse error
Previously, if the input stream consisted only of an error message,
notmuch-json-begin-compound would signal a (wrong-type-argument
number-or-marker-p nil) error when reaching the end of the error
message.  This happened because notmuch-json-scan-to-value would think
that it reached a value and put the parser into the 'value state.
Even after notmuch-json-begin-compound signaled the syntax error, the
parser would remain in this state and when the resynchronization logic
reached the end of the buffer, the parser would fail because the
'value state indicates that characters are available.

This fixes this problem by restoring the parser's previous state if it
encounters a syntax error.
2012-12-15 13:47:49 -04:00
David Bremner
327f30a8f3 perf-test: use nmbug tags in dump-restore tests
This makes the tag set a bit less trivial.

Note that if you use the small corpus, this is not so interesting (and
is also a bit noisy) since the messages will not be found. In the
future this could be checked for.

Conflicts:
	performance-test/01-dump-restore
2012-12-15 08:30:13 -04:00
David Bremner
51b0f8ff33 perf-test: split basic into 00-new, 01-dump-restore, and 02-tag
We use the new "time_start" function to restore the database from cache
if possible.
2012-12-15 08:29:08 -04:00
David Bremner
5c35791d7c perf-test: unpack tags.
There is only one set of tags, independant of the size of message
corpus chosen.
2012-12-15 08:17:58 -04:00
David Bremner
5f8e376912 perf-test: bump corpus version to 0.3
The new version ships with some tags, and an updated archive of the
notmuch mailing list.
2012-12-15 08:17:58 -04:00
David Bremner
d7ba84dfad perf-test: update README
Describe new argument parsing and mention cache handling routines.
2012-12-15 08:17:58 -04:00
David Bremner
ceaf5ca6c0 perf-test: add caching of xapian database
The caching and uncaching seem to be necessarily manual, as timing the
initial notmuch new is one of our goals with this suite.
2012-12-15 08:17:58 -04:00
David Bremner
74a883562b perf-test: cache unpacked corpus
Unpacking is not really the expensive step (compared to the initial
notmuch new), but this is a pre-requisite to caching the database.
2012-12-15 08:17:57 -04:00
David Bremner
925ebd1fde perf-test: optionally print description for each group of tests
Output from tests is indented slightly in the same style as the
correctness tests.
2012-12-15 08:17:57 -04:00
David Bremner
a3137c61e1 perf-test: add corpus size to output, compact I/O stats
Austin suggested a while ago that the corpus size be printed in the
header. In the end it seems the corpus will be fixed per test script,
so this suggestion indeed makes sense.

The tabbing was wrapping on my usual 80 column terminal, so I joined
the input and output columns together.
2012-12-14 22:23:55 -04:00
David Bremner
e7c661d31a perf-test: add argument parsing for performance tests
This patch just sets (non-exported) variables. The variable $debug is
already used, and $corpus_size will be used in following commits.
2012-12-14 22:23:55 -04:00
David Bremner
e52597016f perf-test: propagate non-zero returns from /usr/bin/time
Unlike in the correctness tests, the most common cause of non-zero
return seems to be the user interrupting, so killing the run seems
like the friendly thing to do.
2012-12-14 22:23:54 -04:00
Pieter Praet
e7bd40aa4c test: emacs: new tests "notmuch-show: {add,remove} multiple tags {to,from} single message"
* test/emacs:

  - Rename subtests "{Add,Remove} tag from notmuch-show view" to
    "notmuch-show: {add,remove} single tag {to,from} single message"
    to be consistent with the following tests.

  - New subtest "notmuch-show: add multiple tags to single message":
    `notmuch-show-add-tag' ("+") can add multiple tags to a message.

  - New subtest "notmuch-show: remove multiple tags from single message":
    `notmuch-show-remove-tag' ("-") can remove multiple tags from a message.
2012-12-11 10:01:40 -04:00
Mark Walters
8f1a8c7b09 contrib: pick: bugfix for pick splitting the window excessively
Previously if you carried on past the last message in a pick view pick
would get confused and `forget' about the split pane and would try and
re-split when moving up again. This was due to faulty logic in
notmuch-pick-show-message: something that should have been in the (when message)
clause was not.

Thanks to jrollins for the bug report.
2012-12-11 10:01:18 -04:00
Tomi Ollila
0f9c630a8a NEWS: under-the-hood Emacs interface fixes
Added the following Emacs Interface NEWS entries:

Catch errors bodypart insertions may throw,
Improved text/calendar content handling and
Disabled coding conversions when reading in
`with-current-notmuch-show-message`.

Thanks to Austin and David for content improvements.
2012-12-10 08:19:49 -04:00
Jani Nikula
11a220a3c1 emacs: add support for stashing the thread id in show view
Add a prefix argument to notmuch-show-stash-message-id to stash thread
id instead of message id.
2012-12-09 19:09:46 -04:00
David Bremner
5c7990f251 tag-util: optimization of tag application
The idea is not to bother with restore operations if they don't change
the set of tags. This is actually a relatively common case.

In order to avoid fancy datastructures, this method is quadratic in
the number of tags; at least on my mail database this doesn't seem to
be a big problem.
2012-12-09 13:33:34 -04:00
David Bremner
60cd3b9a06 notmuch-{dump,restore}.1: document new format options
More or less arbitrarily, notmuch-dump.1 gets the more detailed
description of the format.
2012-12-09 13:33:34 -04:00
David Bremner
0f066ece0f test/dump-restore: add test for warning/error messages
We want to test both that error/warning messages are generated when
they should be, and not generated when they should not be. This varies
between restore and batch tagging.
2012-12-09 13:33:34 -04:00
David Bremner
f9878f9173 test: second set of dump/restore --format=batch-tag tests
These one need the completed functionality in notmuch-restore. Fairly
exotic tags are tested, but no weird message id's.

We test each possible input to autodetection, both explicit (with
--format=auto) and implicit (without --format).
2012-12-09 13:33:34 -04:00
David Bremner
452f8748c4 test: update dump-restore roundtripping test for batch-tag format
Now we can actually round trip these crazy tags and and message ids.
hex-xcode is no longer needed as it's built in.
2012-12-09 13:33:34 -04:00
David Bremner
ea85725450 notmuch-restore: normalize case of error messages.
In English, (unlike German) one does not capitalize the first word
after a colon.
2012-12-09 13:33:34 -04:00
David Bremner
33434d76f3 notmuch-restore: add support for input format 'batch-tag'
This can be enabled with the new --format=batch-tag command line
option to "notmuch restore". The input must consist of lines of the
format:

    +<tag>|-<tag> [...] [--] id:<msg-id>

Each line is interpreted similarly to "notmuch tag" command line
arguments. The delimiter is one or more spaces ' '. Any characters in
<tag> and <search-terms> MAY be hex encoded with %NN where NN is the
hexadecimal value of the character. Any ' ' and '%' characters in
<tag> and <msg-id> MUST be hex encoded (using %20 and %25,
respectively). Any characters that are not part of <tag> or
<search-terms> MUST NOT be hex encoded.

Leading and trailing space ' ' is ignored. Empty lines and lines
beginning with '#' are ignored.

Commit message mainly stolen from Jani's batch tagging commit, to
follow.
2012-12-09 13:33:34 -04:00
David Bremner
fb50cc41fe tag-util.[ch]: New files for common tagging routines
These are meant to be shared between notmuch-tag and notmuch-restore.

The bulk of the routines implement a "tag operation list" abstract
data type act as a structured representation of a set of tag
operations (typically coming from a single tag command or line of
input).
2012-12-09 13:33:34 -04:00
David Bremner
9ff72a83bd util: add string-util.[ch]
This is to give a home to strtok_len. It's a bit silly to add a header
for one routine, but it needs to be shared between several compilation
units (or at least that's the most natural design).
2012-12-08 10:56:11 -04:00
David Bremner
10085656d5 test: add sanity check for dump --format=batch-tag.
It's important this does not rely on restore, since it hasn't been
written yet.
2012-12-08 10:40:55 -04:00
David Bremner
bfe66da432 notmuch-dump: add --format=(batch-tag|sup)
sup is the old format, and remains the default, at least until
restore is converted to parse this format.

Each line of the batch-tag format is modelled on the syntax of notmuch tag:
- "notmuch tag" is omitted from the front of the line
- The dump format only uses query strings of a single message-id.
- Each space seperated tag/message-id is 'hex-encoded' to remove
  trouble-making characters.
- It is permitted (and will be useful) for there to be no tags before
  the query.

In particular this format won't have the same problem with e.g. spaces
in message-ids or tags; they will be round-trip-able.
2012-12-08 10:40:54 -04:00
Peter Feigl
9a989772cb Updating man pages for new S-Expression output format.
Add sections about the new S-Expression output format (--format=sexp) to
the notmuch-search, notmuch-reply and notmuch-show man pages.
2012-12-08 09:30:57 -04:00
Peter Feigl
76271e098c Adding tests for --format=sexp.
Add basic tests, the same as for json, for the S-Expression output
format.
2012-12-08 09:30:46 -04:00
Peter Feigl
102f58d7a4 Use the S-Expression structured printer in notmuch-show, notmuch-reply and notmuch-search.
This patch uses the new S-Expression printer in the notmuch CLI (show,
search and reply). You can now use --format=sexp for any of them.
2012-12-08 09:30:19 -04:00
Peter Feigl
1bf3720b6e Rename the -json printer functions in notmuch-reply and notmuch-show to generic -sprinter functions.
All the structured output functions in notmuch-reply and notmuch-show
are renamed to a generic name (as they do not contain any json-specific
code anyway). This patch is a preparation to actually using the new
S-Expression sprinter in notmuch-reply and notmuch-show.
2012-12-08 09:28:19 -04:00
Peter Feigl
fd3ffe35a2 Adding an S-expression structured output printer.
This commit adds a structured output printer for Lisp
S-Expressions. Later commits will use this printer in notmuch search,
show and reply.

The structure is the same as json, but:
- arrays are written as lists: ("foo" "bar" "baaz" 1 2 3)
- maps are written as p-lists: (:key "value" :other-key "other-value")
- true is written as t
- false is written as nil
- null is written as nil

[ whitespace changes by db ]
2012-12-08 09:26:29 -04:00