Commit graph

3325 commits

Author SHA1 Message Date
David Bremner
51b0f8ff33 perf-test: split basic into 00-new, 01-dump-restore, and 02-tag
We use the new "time_start" function to restore the database from cache
if possible.
2012-12-15 08:29:08 -04:00
David Bremner
5c35791d7c perf-test: unpack tags.
There is only one set of tags, independant of the size of message
corpus chosen.
2012-12-15 08:17:58 -04:00
David Bremner
5f8e376912 perf-test: bump corpus version to 0.3
The new version ships with some tags, and an updated archive of the
notmuch mailing list.
2012-12-15 08:17:58 -04:00
David Bremner
d7ba84dfad perf-test: update README
Describe new argument parsing and mention cache handling routines.
2012-12-15 08:17:58 -04:00
David Bremner
ceaf5ca6c0 perf-test: add caching of xapian database
The caching and uncaching seem to be necessarily manual, as timing the
initial notmuch new is one of our goals with this suite.
2012-12-15 08:17:58 -04:00
David Bremner
74a883562b perf-test: cache unpacked corpus
Unpacking is not really the expensive step (compared to the initial
notmuch new), but this is a pre-requisite to caching the database.
2012-12-15 08:17:57 -04:00
David Bremner
925ebd1fde perf-test: optionally print description for each group of tests
Output from tests is indented slightly in the same style as the
correctness tests.
2012-12-15 08:17:57 -04:00
David Bremner
a3137c61e1 perf-test: add corpus size to output, compact I/O stats
Austin suggested a while ago that the corpus size be printed in the
header. In the end it seems the corpus will be fixed per test script,
so this suggestion indeed makes sense.

The tabbing was wrapping on my usual 80 column terminal, so I joined
the input and output columns together.
2012-12-14 22:23:55 -04:00
David Bremner
e7c661d31a perf-test: add argument parsing for performance tests
This patch just sets (non-exported) variables. The variable $debug is
already used, and $corpus_size will be used in following commits.
2012-12-14 22:23:55 -04:00
David Bremner
e52597016f perf-test: propagate non-zero returns from /usr/bin/time
Unlike in the correctness tests, the most common cause of non-zero
return seems to be the user interrupting, so killing the run seems
like the friendly thing to do.
2012-12-14 22:23:54 -04:00
Pieter Praet
e7bd40aa4c test: emacs: new tests "notmuch-show: {add,remove} multiple tags {to,from} single message"
* test/emacs:

  - Rename subtests "{Add,Remove} tag from notmuch-show view" to
    "notmuch-show: {add,remove} single tag {to,from} single message"
    to be consistent with the following tests.

  - New subtest "notmuch-show: add multiple tags to single message":
    `notmuch-show-add-tag' ("+") can add multiple tags to a message.

  - New subtest "notmuch-show: remove multiple tags from single message":
    `notmuch-show-remove-tag' ("-") can remove multiple tags from a message.
2012-12-11 10:01:40 -04:00
Mark Walters
8f1a8c7b09 contrib: pick: bugfix for pick splitting the window excessively
Previously if you carried on past the last message in a pick view pick
would get confused and `forget' about the split pane and would try and
re-split when moving up again. This was due to faulty logic in
notmuch-pick-show-message: something that should have been in the (when message)
clause was not.

Thanks to jrollins for the bug report.
2012-12-11 10:01:18 -04:00
Tomi Ollila
0f9c630a8a NEWS: under-the-hood Emacs interface fixes
Added the following Emacs Interface NEWS entries:

Catch errors bodypart insertions may throw,
Improved text/calendar content handling and
Disabled coding conversions when reading in
`with-current-notmuch-show-message`.

Thanks to Austin and David for content improvements.
2012-12-10 08:19:49 -04:00
Jani Nikula
11a220a3c1 emacs: add support for stashing the thread id in show view
Add a prefix argument to notmuch-show-stash-message-id to stash thread
id instead of message id.
2012-12-09 19:09:46 -04:00
David Bremner
5c7990f251 tag-util: optimization of tag application
The idea is not to bother with restore operations if they don't change
the set of tags. This is actually a relatively common case.

In order to avoid fancy datastructures, this method is quadratic in
the number of tags; at least on my mail database this doesn't seem to
be a big problem.
2012-12-09 13:33:34 -04:00
David Bremner
60cd3b9a06 notmuch-{dump,restore}.1: document new format options
More or less arbitrarily, notmuch-dump.1 gets the more detailed
description of the format.
2012-12-09 13:33:34 -04:00
David Bremner
0f066ece0f test/dump-restore: add test for warning/error messages
We want to test both that error/warning messages are generated when
they should be, and not generated when they should not be. This varies
between restore and batch tagging.
2012-12-09 13:33:34 -04:00
David Bremner
f9878f9173 test: second set of dump/restore --format=batch-tag tests
These one need the completed functionality in notmuch-restore. Fairly
exotic tags are tested, but no weird message id's.

We test each possible input to autodetection, both explicit (with
--format=auto) and implicit (without --format).
2012-12-09 13:33:34 -04:00
David Bremner
452f8748c4 test: update dump-restore roundtripping test for batch-tag format
Now we can actually round trip these crazy tags and and message ids.
hex-xcode is no longer needed as it's built in.
2012-12-09 13:33:34 -04:00
David Bremner
ea85725450 notmuch-restore: normalize case of error messages.
In English, (unlike German) one does not capitalize the first word
after a colon.
2012-12-09 13:33:34 -04:00
David Bremner
33434d76f3 notmuch-restore: add support for input format 'batch-tag'
This can be enabled with the new --format=batch-tag command line
option to "notmuch restore". The input must consist of lines of the
format:

    +<tag>|-<tag> [...] [--] id:<msg-id>

Each line is interpreted similarly to "notmuch tag" command line
arguments. The delimiter is one or more spaces ' '. Any characters in
<tag> and <search-terms> MAY be hex encoded with %NN where NN is the
hexadecimal value of the character. Any ' ' and '%' characters in
<tag> and <msg-id> MUST be hex encoded (using %20 and %25,
respectively). Any characters that are not part of <tag> or
<search-terms> MUST NOT be hex encoded.

Leading and trailing space ' ' is ignored. Empty lines and lines
beginning with '#' are ignored.

Commit message mainly stolen from Jani's batch tagging commit, to
follow.
2012-12-09 13:33:34 -04:00
David Bremner
fb50cc41fe tag-util.[ch]: New files for common tagging routines
These are meant to be shared between notmuch-tag and notmuch-restore.

The bulk of the routines implement a "tag operation list" abstract
data type act as a structured representation of a set of tag
operations (typically coming from a single tag command or line of
input).
2012-12-09 13:33:34 -04:00
David Bremner
9ff72a83bd util: add string-util.[ch]
This is to give a home to strtok_len. It's a bit silly to add a header
for one routine, but it needs to be shared between several compilation
units (or at least that's the most natural design).
2012-12-08 10:56:11 -04:00
David Bremner
10085656d5 test: add sanity check for dump --format=batch-tag.
It's important this does not rely on restore, since it hasn't been
written yet.
2012-12-08 10:40:55 -04:00
David Bremner
bfe66da432 notmuch-dump: add --format=(batch-tag|sup)
sup is the old format, and remains the default, at least until
restore is converted to parse this format.

Each line of the batch-tag format is modelled on the syntax of notmuch tag:
- "notmuch tag" is omitted from the front of the line
- The dump format only uses query strings of a single message-id.
- Each space seperated tag/message-id is 'hex-encoded' to remove
  trouble-making characters.
- It is permitted (and will be useful) for there to be no tags before
  the query.

In particular this format won't have the same problem with e.g. spaces
in message-ids or tags; they will be round-trip-able.
2012-12-08 10:40:54 -04:00
Peter Feigl
9a989772cb Updating man pages for new S-Expression output format.
Add sections about the new S-Expression output format (--format=sexp) to
the notmuch-search, notmuch-reply and notmuch-show man pages.
2012-12-08 09:30:57 -04:00
Peter Feigl
76271e098c Adding tests for --format=sexp.
Add basic tests, the same as for json, for the S-Expression output
format.
2012-12-08 09:30:46 -04:00
Peter Feigl
102f58d7a4 Use the S-Expression structured printer in notmuch-show, notmuch-reply and notmuch-search.
This patch uses the new S-Expression printer in the notmuch CLI (show,
search and reply). You can now use --format=sexp for any of them.
2012-12-08 09:30:19 -04:00
Peter Feigl
1bf3720b6e Rename the -json printer functions in notmuch-reply and notmuch-show to generic -sprinter functions.
All the structured output functions in notmuch-reply and notmuch-show
are renamed to a generic name (as they do not contain any json-specific
code anyway). This patch is a preparation to actually using the new
S-Expression sprinter in notmuch-reply and notmuch-show.
2012-12-08 09:28:19 -04:00
Peter Feigl
fd3ffe35a2 Adding an S-expression structured output printer.
This commit adds a structured output printer for Lisp
S-Expressions. Later commits will use this printer in notmuch search,
show and reply.

The structure is the same as json, but:
- arrays are written as lists: ("foo" "bar" "baaz" 1 2 3)
- maps are written as p-lists: (:key "value" :other-key "other-value")
- true is written as t
- false is written as nil
- null is written as nil

[ whitespace changes by db ]
2012-12-08 09:26:29 -04:00
Jani Nikula
540a34d096 test: use perl instead of sed -r for portability
Our OS X users report -r is not a supported option for sed. Use perl
instead.
2012-12-08 09:19:34 -04:00
Jani Nikula
60e79e3a9f test: wrap 'wc -l' results in arithmetic evaluation to strip whitespace
This is for portability, as 'wc -l' emits whitespace on some BSD
variants. Suggested by Tomi Ollila <tomi.ollila@iki.fi>.
2012-12-08 09:19:34 -04:00
Jani Nikula
423e4fbfeb test: fix count test
The quoting for ${SEARCH} is broken when it's supposed to be '*', and
it seems tricky to get it right. Just drop the variable and use '*'
directly. Before this, none of the messages ever matched, and the test
was comparing zeros.
2012-12-08 09:19:34 -04:00
Austin Clements
21326a1e6b test: Fix UTF-8 JSON tests in Python 3
test_expect_equal_json uses json.tool from the system Python.  While
Python 2 wasn't picky about the encoding of stdin, Python 3 decodes
stdin strictly according to the environment.  Since we set LC_ALL=C
for the tests, Python 3's json.tool was assuming stdin would be in
ASCII and aborting when it couldn't decode the UTF-8 characters from
some of the JSON tests.  This patch sets the PYTHONIOENCODING
environment variable to utf-8 when invoking json.tool to override
Python's default encoding choice.
2012-12-08 09:19:34 -04:00
Peter Feigl
2bd922ff06 Changing build tool for test/random-corpus to CXX instead of CC.
Without this change, GCC complains as follows:
gcc  test/random-corpus.o test/database-test.o notmuch-config.o command-line-arguments.o lib/libnotmuch.a util/libutil.a parse-time-string/libparse-time-string.a -o test/random-corpus -lgmime-2.6 -lgio-2.0 -lgobject-2.0 -lglib-2.0   -Wl,-rpath,/usr/lib -ltalloc   -lxapian
/usr/bin/ld: lib/libnotmuch.a(database.o): undefined reference to symbol '_ZNSs4_Rep10_M_destroyERKSaIcE@@GLIBCXX_3.4'
/usr/bin/ld: note: '_ZNSs4_Rep10_M_destroyERKSaIcE@@GLIBCXX_3.4' is defined in DSO /usr/lib/libstdc++.so.6 so try adding it to the linker command line
/usr/lib/libstdc++.so.6: could not read symbols: Invalid operation
collect2: error: ld returned 1 exit status
make: *** [test/random-corpus] Error 1
2012-12-06 17:14:36 -04:00
Peter Feigl
0f123ab3aa Adding parse-time to test/.gitignore
test/parse-time is a binary that is generated when running make test. It should be ignored by git.
2012-12-06 17:14:24 -04:00
Jani Nikula
bcf1665cfc NEWS: removal of notmuch-folders 2012-12-06 17:14:10 -04:00
Mark Walters
776e21d953 emacs: show: refresh buffer did not remove overlays
Previously refreshing the notmuch show buffer did not remove overlays
which meant that if the user refreshed a message with images the
images would remain and then the new text was added after.

One might have guessed that erase-buffer would have removed them but
it seems not.  Thus force the removal of overlays with remove-overlays.
2012-12-06 17:13:23 -04:00
Mark Walters
5382eebcdc emacs: stash bugfix
Currently an attempt to stash a non-existent field (eg cc when not
present) throws an error. Catch this case and give the user a warning
message.
2012-12-06 17:12:47 -04:00
Jani Nikula
11365abb90 cli: convert "notmuch new" to the argument parser
Use the notmuch argument parser to handle arguments in "notmuch
new". As a side effect, this fixes broken STRNCMP_LITERAL usage that
accepts, for example, --verbosefoo for --verbose.
2012-12-04 09:10:33 -04:00
Jani Nikula
8ce50f300f cli: fix notmuch top level argument parsing
Use strcmp instead of STRNCMP_LITERAL, which matches the prefix
instead of the whole argument.
2012-12-04 09:10:19 -04:00
Jani Nikula
fb6edb0800 emacs: drop support for deprecated notmuch-folders
Remove notmuch-folders which has been deprecated since

commit a466921760
Author: Carl Worth <cworth@cworth.org>
Date:   Mon Apr 26 22:42:07 2010 -0700

    emacs: Rip out all of the notmuch-folder code.

This lets us simplify the notmuch-saved-searches code slightly.
2012-12-04 09:08:02 -04:00
Jani Nikula
7f54db1f04 test: fix an evident copy-paste error in argument parsing test 2012-12-04 09:07:32 -04:00
Mark Walters
bd918e35d4 contrib: pick: use notmuch-clean-address
Now notmuch-clean-address is split out in show pick can use that (with a small
wrapper).
2012-12-03 07:34:31 -04:00
David Bremner
953c3fa853 test: add broken roundtrip test
We demonstrate the current notmuch restore parser being confused by
message-id's and tags containing non alpha numeric characters
(particularly space and parentheses are problematic because they are
not escaped by notmuch dump).

We save the files as hex escaped on disk so that terminal emulators
will not get confused if the test fails (as we mostly expect it to do).
2012-12-02 19:09:01 -04:00
David Bremner
76fa93e2a2 test: add generator for random "stub" messages
Initial use case is testing dump and restore, so we only have
message-ids and tags.

The message ID's are nothing like RFC compliant, but it doesn't seem
any harder to roundtrip random UTF-8 strings than RFC-compliant ones.

Tags are UTF-8, even though notmuch is in principle more generous than
that.

updated for id:m2wr04ocro.fsf@guru.guru-group.fi

- talk about Unicode value rather some specific encoding
- call talloc_realloc less times
2012-12-02 15:51:32 -04:00
David Bremner
990e3988ce test: add database routines for testing
Initially, provide a way to create "stub" messages in the notmuch
database without corresponding files.  This is essentially cut and
paste from lib/database.cc. This is a seperate file since we don't
want to export these symbols from libnotmuch or bloat the library with
non-exported code.
2012-12-02 15:51:32 -04:00
David Bremner
a09115ae1d test/hex-escaping: new test for hex escaping routines
These are more like unit tests, to (try to) make sure the library
functionality is working before building more complicated things on
top of it.
2012-12-02 15:51:32 -04:00
David Bremner
4216e830e9 test/hex-xcode: new test binary
This program is used both as a test-bed/unit-tester for
../util/hex-escape.c, and also as a utility in future tests of dump
and restore.
2012-12-02 09:14:59 -04:00
David Bremner
902f2e19bd hex-escape: (en|de)code strings to/from restricted character set
The character set is chosen to be suitable for pathnames, and the same
as that used by contrib/nmbug

[With additions by Jani Nikula]
2012-12-02 09:14:59 -04:00