When encountering a message that has been mangled in the "mixed up"
way by an intermediate MTA, notmuch should instead repair it and index
the repaired form.
When it does this, it also associates the index.repaired=mixedup
property with the message. If a problem is found with this repair
process, or an improved repair process is proposed later, this should
make it easy for people to reindex the relevant message. The property
will also hopefully make it easier to diagnose this particular problem
in the future.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
When we notice a legacy-display part during indexing, it makes more
sense to avoid indexing it as part of the message body.
Given that the protected subject will already be indexed, there is no
need to index this part at all, so we skip over it.
If this happens during indexing, we set a property on the message:
index.repaired=skip-protected-headers-legacy-display
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
This adds no functionality directly, but is a useful starting point
for adding new repair functionality.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
The extra flexibility of having both HAVE_EMACS (for yes, there is an
emacs we can use) and WITH_EMACS (the user wants emacs support) lead
to confusion and bugs. We now just force WITH_EMACS to 0 if no
suitable emacs is detected.
In 40b025 we stopped building the notmuch-emacs documentation if
HAVE_EMACS=0 (i.e. no emacs was detected by configure). Unfortunately
we continued to try to install the (non-existent) documentation, which
causes build/install failures.
As a bonus, we also avoid installing the documentation if the user
configures --without-emacs.
Thanks to Ralph Seichter for reporting the problem, and testing
previous versions of this fix.
Since the docstrings are not built in the case of --without-emacs,
even if emacs is detected, don't let sphinx build the emacs docs. This
avoids a large number of error messages due to missing includes. It's
actually a bit surprising sphinx doesn't generate an error for the
missing include files.
It seems our previous attempt with order-only targets was not
sufficient to avoid problems with sphinx-builds doctree cache [0].
Looking around at other people's approaches [1], using separate
doctrees was suggested. I guess there might be a slight loss of
efficiency, but it seems more robust.
[0]: build failures were first noticed in Debian experimental, but I was able to duplicate it in
my usual build environment about 1 in 8 builds.
[1]: in particular
9e3fc1657d
The new `body:` field (in Xapian terms) or prefix (in slightly
sloppier notmuch) terms allows matching terms that occur only in the
body.
Unprefixed query terms should continue to match anywhere (header or
body) in the message.
This follows a suggestion of Olly Betts to use the facility (since
Xapian 1.0.4) to add the same field with multiple prefixes. The double
indexing of previous versions is thus replaced with a query time
expension of unprefixed query terms to the various prefixed
equivalent.
Reindexing will be needed for 'body:' searches to work correctly;
otherwise they will also match messages where the term occur in
headers (demonstrated by the new tests in T530-upgrade.sh)
Without this change, we see this during the build:
sphinx-build -b html -d doc/_build/doctrees -q ./doc doc/_build/html
…/doc/notmuch-emacs.rst:67: WARNING: Unexpected indentation.
…/doc/notmuch-emacs.rst:165: WARNING: Unexpected indentation.
…/doc/notmuch-emacs.rst:306: WARNING: Unexpected indentation.
This source change doesn't seem to have any effect on the generated
HTML, at least.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
This should silence some warnings about the jobserver, but also make
it easier to build the docs where GNU make is called something other
than make.
Based on a patch from aidecoe.
In certain conditions the parallel calls to sphinx-build could
collide, yielding a crash like
Exception occurred:
File "/usr/lib/python3/dist-packages/sphinx/environment.py", line 1261, in get_doctree
doctree = pickle.load(f)
EOFError: Ran out of input
Many of the manpages didn't treat literal text as literal text. I've
tried to normalize some of the restructured text to make it a bit more
regular.
several of the synopsis lines are still untouched by this cleanup, but
i'm not sure what the right way to represent those is in .rst,
actually.
In particular find that if i rebuild the manpages, sometimes i end up
with some of the synopsis lines showing – (U+2013 EN DASH) where they
should have -- (2 × U+002D HYPHEN-MINUS) in the generated nroff
output, though i have not tracked down the source of this error yet.
This is mainly to improve discoverability. It seems that doing
variable cross-references is not easy without using some sphinx
extension/customization.
Add fancy new feature, which makes "notmuch show" capable of actually
indexing messages that it just decrypted.
This enables a workflow where messages can come in in the background
and be indexed using "--decrypt=auto". But when showing an encrypted
message for the first time, it gets automatically indexed.
This is something of a departure for "notmuch show" -- in particular,
because it requires read/write access to the database. However, this
might be a common use case -- people get mail delivered and indexed in
the background, but only want access to their secret key to happen
when they're directly interacting with notmuch itself.
In such a scenario, they couldn't search newly-delivered, encrypted
messages, but they could search for them once they've read them.
Documentation of this new feature also uses a table form, similar to
that found in the description of index.decrypt in notmuch-config(1).
A notmuch UI that wants to facilitate this workflow while also
offering an interactive search interface might instead make use of
these additional commands while the user is at the console:
Count received encrypted messages (if > 0, there are some things we
haven't yet tried to index, and therefore can't yet search):
notmuch count tag:encrypted and \
not property:index.decryption=success and \
not property:index.decryption=failure
Reindex those messages:
notmuch reindex --try-decrypt=true tag:encrypted and \
not property:index.decryption=success and \
not property:index.decryption=failure
I think we've diverged enough from the Xapian query parser
that we can't rely on that syntax description [1]. As far as I can
tell, [1] also only discusses quotes in the context of phrases.
[1]: https://xapian.org/docs/queryparser.html
Currently, notmuch has the levers needed to set coherent crypto policy
around how cleartext is indexed, which also has an impact on how
messages are rendered. But we don't have a lot of documentation about
how to do sensible things. This is an initial attempt to address
that.
The first example shows a way to selectively index specific messages.
The next two examples are about aligning the existing database with
crypto indexing policy
The default crypto policy is to not index cleartext, and to only
decrypt messages on display when explicitly requested.
The other sensible crypto policy is to index cleartext while stashing
session keys. messages indexed in this way will be searchable, and
will be decrypted on display automatically unless the user explicitly
asks for it to *not* be decrypted.
The policy for indexing *new* messages is stored in the database as
the config variable index.decrypt.
But setting policy for new messages doesn't retroactively affect
already indexed messages.
This patch attempts to document ways that someone can efficiently
align their pre-existing database with their new policy.
I'm not sure this is the right place to document these examples, but i
do want them to be user-facing and relatively easy to find. I'm happy
to entertain suggestions for where else we should put them.
In some cases (e.g. when building a publicly-visible e-mail archive)
it doesn't make any sense to restrict visibility of the message to the
current user account.
This adds a --world-readable boolean option for "notmuch insert", so
that those who want to archive their mail publicly can feed their
archiver with:
notmuch insert --world-readable
Other local delivery agents (postfix's local, and dovecot's lda) all
default to delivery in mode 0600 rather than relying on the user's
umask, so this fix doesn't change the default.
Also, this does not override the user's umask. if the umask is
already set tight, it will not become looser as the result of passing
--world-readable.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Need to be clearer about specifying time ranges using timestamps.
Legacy syntax which predates the date prefix is still supported, but
timestamps used in conjunction with the date prefix require additional
syntax.
Make all parameter descriptions etc. use reStructuredText definition
lists with uniform style and indentation. Remove redundant indentation
from around the lists. Remove blank lines between term lines and
definition blocks. Use four spaces for indentation.
This is almost completely whitespace and paragraph reflow changes.
