This new JSON format for replies includes headers generated for a
reply message as well as the headers of the original message. Using
this data, a client can intelligently create a reply. For example, the
emacs client will be able to create replies with quoted HTML parts by
parsing the HTML parts.
This adds three tests for --output=messages searches. One test is for
the case when one exclude tag does not occur in the Xapian
database. This triggers a Xapian bug in some cases and causes the
whole exclusion to fail. The next commit avoids this bug.
The tests for the exclude code in search and count use the line
notmuch config set search.exclude_tags = deleted
which actually sets the exclude tags to be "=" and "deleted". Remove
the "=" from this line.
Before the change, messages generated by generate_message() used "Test
message #N" for default subject where N is the generated messages
counter. Since message subject is commonly present in expected
results, there is a chance of breaking other tests when a new
generate_message() call is added. The patch changes default subject
value for generated messages to subtest name if it is available. If
subtest name is not available (i.e. message is generated during test
initialization), the old default value is used (in this case it is
fine to have the counter in the subject).
Another benefit of this change is a sane default value for subject in
generated messages, which would allow to simplify code like:
test_begin_subtest "test for a cool feature"
add_message [subject]="message for test for a cool feature"
Before the change, the first subtest in raw format tests just
generated messages and checked that they are added successfully. This
is not really a raw format test, it is creating of environment
required for other subtests to run. The patch removes the first
subtest from raw and replaces it with bare add_message calls, similar
to how it is done in other tests.
TODO: we should check that test environment was created successfully.
Currently, many tests do add_message(), notmuch new and other calls
without checking the results. We should come up with a general
solution for this, i.e. if any command during test initialization
fails, all tests should be skipped with appropriate error message.
This is fully compatible for root and leaf parts, but now has proper
support for interior parts. This requires some design decisions that
were guided by what I would want if I were to save a part.
Specifically:
- Leaf parts are printed without headers and with transfer decoding.
This is what makes sense for saving attachments. (Furthermore, the
transfer decoding is necessary since, without the headers, the
caller would not be able to interpret non-transfer-decoded output.)
- Message parts are printed with their message headers, but without
enclosing part headers. This is what makes sense for saving a
message as a whole (which is a message part) and for saving attached
messages. This is symmetric for whole messages and for attached
messages, though we special-case the whole message for performance
reasons (and corner-case correctness reasons: given malformed input,
GMime may not be able to reproduce it from the parsed
representation).
- Multipart parts are printed with their headers and all child parts.
It's not clear what the best thing to do for multipart is, but this
was the most natural to implement and can be justified because such
parts can't be interpreted without their headers.
As an added benefit, we can move the special-case code for part 0 into
the raw formatter.
Previously, there was only one CRLF between the terminating boundary
of the embedded multipart/alternative and the boundary of the
containing multipart. However, according the RFC 1341, 7.2.1:
The boundary must be followed immediately either by another CRLF and
the header fields for the next part, or by two CRLFs, in which case
there are no header fields for the next part
and
The CRLF preceding the encapsulation line is considered part of the
boundary so that it is possible to have a part that does not end
with a CRLF (line break).
Thus, there must be *two* CRLFs between these boundaries: one that
ends the terminating boundary and one that begins the enclosing
boundary.
While GMime accepted the message we had before, it could not produce
such a message.
gmime-2.6 had a bug [1] which made it impossible to tell why a signature
verification failed when the signer key was unavailable (empty "sigstatus" field
in the JSON output). Since 00b5623d the corresponding test is marked as broken
when using gmime-2.6 (2.4 is fine).
This bug has been fixed in gmime 2.6.5, which is now the minimal gmime-2.6
version required for building notmuch (gmime-2.4 is still available). As a
consequence the version check in test/crypto can be removed.
[Added by db]
Although less unambigously a bug, Gmime 2.6 prior to 2.6.7 also was
more strict about parsing, and rejected messages with initial "From "
headers. This restriction is relaxed in [2]. For reasons explained in [3],
we want to keep this more relaxed parsing for now.
[1] https://bugzilla.gnome.org/show_bug.cgi?id=668085
[2] http://git.gnome.org/browse/gmime/commit/?id=d311f576baf750476e06e9a1367a2dc1793ea7eb
[3] id:"1331385931-1610-1-git-send-email-david@tethera.net"
notmuch show outputs the exclude flag so many tests using notmuch
show failed. This commit adds "excluded:0" or "excluded: false" to
the expected outputs. After this commit there should be no failing
tests.
notmuch-search.c now returns all matching threads even if it the
match is a search.tag_excluded message (but with a mark indicating
this). Update the test to reflect this.
This has three ramifications:
- Blank To and Cc headers are no longer output for messages.
- Dates are now canonicalized for messages, which means they always
have a day of the week and GMT is printed +0000 (never -0000)
- Invalid From message headers are handled slightly differently, since
they get parsed by GMime now instead of notmuch.
Previously, top-level message headers were printed as Subject, From,
To, Date, while embedded message headers were printed From, To,
Subject, Date. This makes both cases use the former order and updates
the tests accordingly.
Emails that are encoded differently than as ASCII or UTF-8 are not
indexed properly by notmuch. It is not possible to search for non-ASCII
words within those messages.
Consensus seems to be that people prefer that refreshing show buffers
retains state by default, rather than resetting it by default. This
turns out to be the case in the code, as well. In fact, there's even
a test for this that's been marked broken for several months, which
this patch finally gets to mark as fixed.
* emacs/notmuch-show.el
(notmuch-show-stash-mlarchive-link-alist):
New defcustom of type `alist' (key = name, value = URI),
containing Mailing List Archive URI's for searching by Message-Id.
(notmuch-show-stash-mlarchive-link-default):
New defcustom, default MLA to use when `notmuch-show-stash-mlarchive-link'
received no user input whatsoever. Available choices are generated using
the contents of `notmuch-show-stash-mlarchive-link-alist'.
(notmuch-show-stash-map):
Added keybinds "l" and "L" for `notmuch-show-stash-mlarchive-link'
respectively `notmuch-show-stash-mlarchive-link-and-go'.
(notmuch-show-stash-mlarchive-link):
New function, stashes a URI pointing to the current message at one
of the MLAs configured in `notmuch-show-stash-mlarchive-link-alist'.
Prompts user with `completing-read' if not provided with an MLA key.
(notmuch-show-stash-mlarchive-link-and-go):
New function, uses `notmuch-show-stash-mlarchive-link' to
stash a URI, and then visits it using the browser configured
in `browse-url-browser-function'.
Based on original work [1] by David Edmondson <dme@dme.org>.
[1] id:"1327397873-20596-1-git-send-email-dme@dme.org"
`notmuch-show-stash-mlarchive-link' stashes a URI pointing to the current message
at one of the MLAs configured in `notmuch-show-stash-mlarchive-link-alist'.
Marked as "broken": fixed in next commit.
When tests are skipped due to missing prereqs, those prereqs are only
displayed when running with the `--verbose' option. This is essential
information when troubleshooting, so always send to stdout.
A new configuration key 'new.ignore' is used to determine which
files and directories user wants not to be scanned as new mails.
Mark the corresponding test as no longer broken.
This work merges my previous attempts and Andreas Amann's work
in id:"ylp7hi23mw8.fsf@tyndall.ie"
Files and directories which are specified in 'new.ignore' in the
config file shouldn't be indexed nor reported by `notmuch new'.
This is basically Pieter's work with Austin's comments addressed.
This makes the text formatter take advantage of the new code
structure. The previously duplicated header logic is now unified,
several things that we used to compute repeatedly across different
callbacks are now computed once, and the code is simpler overall and
32% shorter.
Unifying the header logic causes this to format some dates slightly
differently, so the two affected test cases are updated.
After the recent tagging operations changes, functions bound to "+"
and "-" in notmuch-search and notmuch-show views always read input
from the minibuffer. Use kbd macros instead of calling them directly.
`Notmuch-wash-region-to-button' is the function that creates hidden
regions with buttons for signatures, citations and original messages.
Before the change, it did not work correctly if the to-be-hidden
region started at the beginning of a message: the visibility toggle
button was hidden as well. The patch fixes this. There are two parts
in the fix:
* Use `insert-before-markers' instead of `insert' for creating the
button, so that it does not get added to the hidden overlay.
* Stop using PREFIX argument for adding a newline before the button.
The newline should not be added before a button at the beginning of
buffer.
The corresponding test is fixed now.
The test is currently broken and will be fixed by a subsequent patch.
The patch adds a new file for tests of Emacs notmuch-show view.
Based on patch by David Edmondson [1].
[1] id:"1327562380-12894-4-git-send-email-dme@dme.org"
Emacs message-mode uses certain text strings to indicate how to attach
files to outgoing mail. If these are present in the text of an email,
and a user is tricked into replying to the message, the user’s files
could be exposed.
Edited-by: Pieter Praet <pieter@praet.org>: Rebased to release branch.
The test is broken at this time; the next commit will introduce a fix.
Edited-by: Pieter Praet <pieter@praet.org>:
Rebased to release branch, moved expected output into the actual test,
and fixed "Fcc:" line.
`mail-header-parse-address' expects un-decoded mailbox parts, which is
not what we have at this point. Replace it with simple string
deconstruction.
Mark the corresponding test as no longer broken.
Minor whitespace cleanup.
Before the change, "s" in notmuch-hello buffer would jump to the
search box. The patch changes the binding to `notmuch-search' which
is consistent with all other notmuch buffers.
Add a new test function to allow simpler testing of emacs
functionality.
`test_emacs_expect_t' takes one argument - a lisp expression to
evaluate. The test passes if the expression returns `t', otherwise it
fails and the output is reported to the tester.
When checking for a running emacs, test_emacs evaluates the empty list
'()'. This returns 'nil' when emacs is running, which is then
prepended to the actual test result. Given that it is not part of the
actual test output the test harness can incorrectly report test
failure (or success).
Currently, the 'search.exclude_tags' option is automatically
set to "deleted;spam;" if it's missing from the config file.
This violates the Principle of Least Surprise, so *only* set
'search.exclude_tags' to "deleted;spam;" if we didn't find a
configuration file at all.
This patch is actually Austin Clements' work:
id:"20120117203211.GQ16740@mit.edu"
Currently, the 'search.exclude_tags' option is automatically set to
"deleted;spam;" if it's missing from the config file.
This violates the Principle of Least Surprise, so update the tests to
*only* expect the exclusion of messages which are tagged "deleted" if the
'search.exclude_tags' option is explicitly set *and* contains that tag.
Previously, top-level message headers were printed as Subject, From,
To, Date, while embedded message headers were printed From, To,
Subject, Date. This makes both cases use the former order and updates
the tests accordingly.
Strangely, the raw format also uses this function, so this also fixes
the two raw format tests affected by this change.
emacsclient --eval '(kill-emacs)' makes emacs versions 23.1
and 23.2 ask user input from running emacs. Redefining
yes-or-no-p function when kill-emacs is executed for these
emacs versions in test-lib.el avoids this test problem.
Used emacs (whitespace-cleanup) function to "cleanup blank problems"
in test files where that could be done without breaking tests;
test/emacs was partially, and test/multipart was fully reverted.
There are lots of API changes in gmime 2.6 crypto handling. By adding
preprocessor directives, it is however possible to add gmime 2.6 compatibility
while preserving compatibility with gmime 2.4 too.
This is mostly based on id:"8762i8hrb9.fsf@bookbinder.fernseed.info".
This was tested against both gmime 2.6.4 and 2.4.31. With gmime 2.4.31, the
crypto tests all work fine (as expected). With gmime 2.6.4, one crypto test is
currently broken (signature verification with signer key unavailable), most
likely because of a bug in gmime which will hopefully be fixed in a future
version.
This makes `show-trailing-whitespace' happy, i.e. it does not mark the
whole search box line as trailing spaces.
Since the dot is invisible, this change makes no visible difference
for `notmuch-hello'.
Edited-by: Pieter Praet <pieter@praet.org> to fix the tests.
Add an explicit note to the README explaining what programs are
necessary and the perhaps-surprising behavior of skipping tests if
they aren't present.
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
This adds a "search" section to the config file and an
"auto_tag_exclusions" setting in that section. The search and count
commands pass tag tags from the configuration to the library.
If a message was received to the user's address that was in a named
group list, notmuch reply does not use that address for picking the
from address.
Groups lists are of the form: foo:bar@example.com,baz@example.com;
Signed-off-by: Jani Nikula <jani@nikula.org>
When running the Emacs tests in verbose mode, only the first missing
prereq is reported because the `run_emacs' function is short-circuited
early:
#+begin_example
emacs: Testing emacs interface
missing prerequisites: [0] emacs(1)
skipping test: [0] Basic notmuch-hello view in emacs
SKIP [0] Basic notmuch-hello view in emacs
#+end_example
This can lead to situations reminiscent of "dependency hell", so instead
of returning based on each individual `test_require_external_prereq's exit
status, we now do so only after checking all the prereqs:
#+begin_example
emacs: Testing emacs interface
missing prerequisites: [0] dtach(1) emacs(1) emacsclient(1)
skipping test: [0] Basic notmuch-hello view in emacs
SKIP [0] Basic notmuch-hello view in emacs
#+end_example
Also added missing prereq for dtach(1).
As reported in
id:"CAEbOPGyuHnz4BPtDutnTPUHcP3eYcRCRkXhYoJR43RUMw671+g@mail.gmail.com"
sometimes gmime tries to access a NULL pointer, e.g. g_mime_iconv_open()
tries to access iconv_cache that is NULL if g_mime_init() is not called.
This causes notmuch to segfault when calling gmime functions.
Calling g_mime_init() initializes iconv_cache and others variables needed
by gmime, making sure they are initialized when notmuch calls gmime
functions.
Test marked fix by db.
One is quoted printable, the other users 8 bit encoding. The latter
triggers a bug in the python bindings due to missing call to
g_mime_init. The corresponding test is marked broken in this commit.
Tester may have set LD_LIBRARY_PATH to find libraries required
by notmuch. Therefore add $TEST_DIRECTORY/../lib to the beginning
of current list of library paths in $LD_LIBRARY_PATH before running
symbol-test.
It makes no sense to run test-lib.sh, so it makes no sense to give it
an interpreter. This is particularly annoying for Emacs users who
have executable-insert set, since the presence of the #! line will
cause Emacs to mark test-lib.sh executable when saving it, which will
in turn case the 'basic' test to fail.
January 5, 2001 was a Tuesday, not a Friday. Jameson fixed this exact
problem for the multipart test in ec2b0a98cc, but not for
generate_message itself.
As Jameson pointed out in ec2b0a98cc, if we want to test date parsing,
we should do it separately.
As we start to pay more attention to emacs24, it helps to be able to
select a different version of emacs to run the tests with to verify
version specific bugs.
A separate variable TEST_EMACS is needed to avoid being overwritten by the
make variable EMACS in Makefile.config
For what it's worth, the value of emacs is chosen at the time
tmp.emacs/run_emacs is created, so is fixed for all subtests.
Test that `notmuch-hello-refresh-hook' is called once when
`notmuch-hello' is called and twice when calling
`notmuch-hello-update' after that.
The tests are very similar to tests for `notmuch-hello-mode-hook'.
`notmuch-hello' should call `notmuch-hello-mode' function only when
run for the first time. But before the change, `notmuch-hello' used
`kill-all-local-variables' to remove editable widgets fields. This
caused the major mode to be reset, and `notmuch-hello-mode' to be
called every time.
The patch manually deletes all editable widget fields and removes
`kill-all-local-variables' call.
Add `notmuch-hello-mode-hook-counter' hook to count how many times
`notmuch-hello-mode-hook' was called. The counter function increments
`notmuch-hello-mode-hook-counter' variable value if it is bount,
otherwise it does nothing.
The idea is that $test_count could be used in tests to label
intermediate files. The output enabled by this patch (and --debug)
helps figure out which OUTPUT.nn file belongs to which test in case
several subtests write to OUTPUT.$test_count
binutils-2.22 changes the behaviour of ld by defaulting to
--no-copy-dt-needed-entries, which means that required objects/libs are not
"indirectly" linked through intermediate objects/libs anymore. As a consequence,
when using binutils-2.22, building symbol-test fails with the following error:
/usr/bin/ld: test/symbol-test.o: undefined reference to symbol
'std::basic_string<char, std::char_traits<char>, std::allocator<char>
>::~basic_string()@@GLIBCXX_3.4'
/usr/bin/ld: note: 'std::basic_string<char, std::char_traits<char>,
std::allocator<char> >::~basic_string()@@GLIBCXX_3.4' is defined in DSO
/usr/lib/libstdc++.so.6 so try adding it to the linker command line
/usr/lib/libstdc++.so.6: could not read symbols: Invalid operation
An easy fix is to link using CXX instead of CC.
This reverts commit c6a3a768fe.
This test is unreliable, showing BROKEN in some environments and FIXED
in others. The confusion seems to outweigh the benefits, for now.
Conflicts:
test/emacs
Some distros (Arch Linux) ship Python as python2 and Python 3 as python.
Checking for python2 is necessary for the Python tests to work on these
platforms.
The new test_python() function makes writing Python tests a little easier:
- it sets the environment variables as needed
- it redirects stdout to the OUTPUT file (like test_emacs()).
This commit also declares python as an external prereq.
The stdout redirection is required to avoid trouble when running commands like
"python 'script' | sort > OUTPUT": in such a case, any error due to a missing
external prereq would be "swallowed" by sort, resulting to a failed test instead
of a skipped one.
expect the date_relative field for thread entries
in notmuch search's json output
note from Commiter: we don't have to worry about the date changing
because the date in question is more than 180 days old.
Before the change, there was a workaround to avoid notmuch show calls
for parts with application/* Content-Type. But non-inlinable parts
are not limited to this Content-Type (e.g. mp3 files have audio/mpeg
Content-Type and are not inlinable). For such parts
`notmuch-show-insert-part-*/*' handler is called which unconditionally
fetches contents for all parts.
The patch moves content fetching from `notmuch-show-insert-part-*/*'
to `notmuch-show-mm-display-part-inline' function after MIME inlinable
checks are done to avoid useless notmuch show calls. The
application/* hack is no longer needed and removed.
The patch adds two new test cases:
* Do not call notmuch for non-inlinable application/mpeg parts
* Do not call notmuch for non-inlinable audio/mpeg parts
The application/mpeg test passes thanks to a workaround for
application/* Content-Types. The audio/mpeg is currently broken.
The patch adds two auxiliary functions and a variable:
notmuch_counter_reset
$notmuch_counter_command
notmuch_counter_value
They allow to count how many times notmuch binary is called.
notmuch_counter_reset() function generates a script that counts how
many times it is called and resets the counter to zero. The function
sets $notmuch_counter_command variable to the path to the generated
script that should be called instead of notmuch to do the counting.
The notmuch_counter_value() function returns the current counter
value.
We start modestly, with a (slightly modified) test case from Kazuo
Teramoto. Originally it just made sure the bindings didn't crash; here
we check that by comparing the output with that of notmuch search.
The fake missing binary functions check if the binary has already be
added to the diagnostic message to avoid duplicates. Unfortunately,
this check was buggy because the message string does not have the
trailing space.
test_missing_external_prereq_${binary}_ variable indicates that the
binary is missing. It must be set in test_declare_external_prereq()
outside of the fake $binary() function.
If emacs is not available, test_expect_equal would be called with only
one argument. The patch fixes this by quoting the (possibly empty)
$(cat OUTPUT) argument.
Some tests (e.g. crypto) do a common initialization required for all
subtests. The patch adds a check for missing external dependencies
during this initialization. If any prerequisites are missing, all
subtests are skipped.
The check is run on the first call of test_reset_state_ function, so
no changes for the tests are needed.
There is existing support for general prerequisites in the test suite.
But it is not very convenient to use: every test case has to keep
track for it's dependencies and they have to be explicitly listed.
The patch aims to add better support for a particular type of external
dependencies: external executables. The main idea is to replace
missing external binaries with shell functions that have the same
name. These functions always fail and keep track of missing
dependencies for a subtest. The result reporting functions later can
check that an external binaries are missing and correctly report SKIP
result instead of FAIL. The primary benefit is that the test cases do
not need to declare their dependencies or be changed in any way.
If symbol-test is built in symbol-hiding with hardcoded g++ invokation,
it's not so easy to pass $(srcdir) which is required to find notmuch.h
when srcdir and builddir are separate directories.
Basic test 'Ensure that all available tests will be run by
notmuch-test' compares all tests that are run with listing of test/
directory. There is a growing list of exceptions for files and
directories which located in the test/ directory but are not tests.
Moreover some (probably buggy) tests do create files in the the test/
directory which may be left behind in case of failure. This makes the
basic test fail.
The patch changes the test to look only for regular executable files.
This makes the exception list much smaller. And since no tests should
create executables in the test/ directory (if there are, they should
be fixed), the basic test should not be affected by failed or
interrupted tests.
The error is easy to miss, because the test passes and stderr is not
printed. But if you run basic tests in verbose mode (./basic
--verbose), you get:
sed: can't read notmuch-test: No such file or directory
The issue is that sed command is given two files: notmuch-test and
$TEST_DIRECTORY/notmuch-test. And there is no notmuch-test file in
the current directory (test/tmp.basic/). The patch just removes the
non-existing file from the sed command.
Previous behaviour was to indent messages in a thread according
to depth by one space per level. This commit tests if setting
notmuch-indent-messages-width to `4' provides a message thread
with four spaces of indentation thread depth.
Previous behaviour was to indent messages in a thread according
to depth by one space per level. In
id:1311028119-50637-1-git-send-email-fgeller@gmail.com Felix
Geller proposed a patch in order to turn indentation off. This
commit tests if instead setting notmuch-indent-messages-width to
`0' does turn off indentation.
Previous behaviour was to indent messages in a thread according
to depth by one space per level. This is still the case with
notmuch-indent-messages-width default value `1'. This test
succeeds if output with default value is same as in "Basic
notmuch-show view in emacs".
Some tests don't break when HUP signal is sent tho those (by
pressing ctrl-c on the terminal). Therefore, the top-level
test script catches the HUP and sends TERM signal to the
started test script.
If mail sending from emacs fails before it has chance to connect
to the smtp-dummy mail server, the opportunistic QUIT message
sending makes smtp-dummy to exit.
Due to 108-character limit in unix domain socket path this change
is required; it is more probable that length of ${TMPDIR:-/tmp} is
shorter than length of path to the current directory of notmuch test
source directory. One can expect to create reasonable-length unix
domain sockets wherever $TMPDIR points to.
The TEST_TMPDIR if first needed to hold dtach's socket (due
to 108-character limit in socket file names). Later it can be
used to hold other temporary files; directory deleted at exit.
Do not redirect test_emacs stderr to /dev/null. Test_emacs uses
emacsclient(1) now and it does not print unwanted messages (like
those from `message') to stderr. But it does print useful
errors, e.g. when emacs server connection fails, given expression
is not valid or undefined function is called.
It is very convenient when C-e (bound to `widget-end-of-line') ignores
trailing spaces inside the search widget. But it only does so if a
widget is not followed by a newline (that is why it works in the saved
search widgets). The patch just adds an invisible space after the
search widget to get the desirable behavior of `widget-end-of-line'.
The extra space is also added to expected results of emacs tests.
Use `previous-single-char-property-change' instead of going
through each character by hand and testing it's visibility. This
fixes `notmuch-show-advance-and-archive' to work for the last
message in thread with hidden signature.
Set SCREENRC and SYSSCREENRC environment variables to "/dev/null"
as suggested by Jim Paris to avoid potential problems with
screen(1) configuration files.
Before the change, emacs run in daemon mode without any visible
buffers. Turns out that this affects emacs behavior in some
cases. In particular, `window-end' function returns `point-max'
instead of the last visible position. That makes it hard or
impossible to implement some tests. The patch runs emacs in a
detached screen(1) session. So that it works exactly as if it
has a visible window.
Note: screen terminates when emacs exits. So the patch does not
introduce new "running processes left behind" issues.
