Originally contributed by Brian Sniffen [1]. Quite a few unpublished
fixes from Daniel Kahn Gilmour. We've been running it a few years now
as the "official" archive of the notmuch mailing list. There also a
few fixes from myself and Austin Ray. Finally I converted it to the
new python bindings.
This commit squashes the several years of development history and
moves it under devel, in recognition of the now established role it
plays in the project infrastructure.
[1]: id:87tvyvp4f2.fsf@istari.evenmere.org
In most cases used printf %s ... instead.
echo -n > file lines to create empty / truncate files were
changed to : > file lines, like done in in test-lib-emacs.sh
And one echo -n " " replaced with use of sed "s/^/ /" in next line.
This is based on a patch from Johan Parin [1], which is in turn
responding to a bug report / feature requiest from Jan Malkhovski.
The update to the structured output documented in schemata is intended
to be upward compatible, so the format version stays the same
[1]: id:20191116162723.18343-1-johan.parin@gmail.com
[2]: id:87h8sdemnr.fsf@oxij.org
'check-notmuch-commit' is an updated version of a script I have been
using (although not always as consistently as I should) before sending
patches to the list.
Although it requires a bit more tooling, encouraging people to use
check-notmuch-commit might reduce the number of round trips to the
list for style nitpicks.
This is a bit of a cheat, since the format does not actually
change. On the other hand it is fairly common to do something like
this to shared libary SONAMEs when the ABI changes in some subtle way.
It does rely on the format-version argument being early enough on the
command line to generate a sensible error message.
This new command for notmuch-tree-mode is analogous to
notmuch-search-filter-by-tag, bound to "t" in notmuch-search-mode; it
gets therefore the same "t" keybinding in notmuch-tree-mode (replacing
the current assignment to notmuch-search-by-tag).
BSD xargs does not have the -d option. Here we use tr to convert
newlines to NUL characters, then pass -0 to xargs (which BSD does
support).
I looked at passing -z to 'git ls-files', but I did not find a BSD
grep option to turn on NUL deliminted line processing.
New --sort CLI option documented in notmuch-show's man page, and
notmuch-search-toggle-order mentioned in doc/notmuch-emacs.rst and
devel/emacs-keybindings.org (in the latter, there's also some
whitespace changes in a table introduced by org-mode).
When the certificate that signs a message is known to be valid, GMime
is capable of reporting on the e-mail address embedded in the
certificate.
We pass this information along to the caller of "notmuch show", as
often only the e-mail address of the certificate has actually been
checked/verified.
Furthermore, signature verification should probably at some point
compare the e-mail address of the caller against the sender address of
the message itself. Having to parse what gmime thinks is a "userid"
to extract an e-mail address seems clunky and unnecessary if gmime
already thinks it knows what the e-mail address is.
See id:878s41ax6t.fsf@fifthhorseman.net for more motivation and discussion.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Whenever setting background color, set also corresponding
foreground color. Don't expect default foreground color to
be #000 (or something close); user may have changed it.
This is quite fragile, but it works for now, unlike the python
version.
In general it seems conf.py is not intended to be evaluated outside of
sphinx, as it assumes certain global names (in particular "tags") are
defined.
- Since Emacs 25 comes with `load-prefer-newer' we can remove the
complicated variant of the advice, which implemented a poorman's
version of that.
- Since Emacs 25 comes with the new advice mechanism, we can use
that now for the simple variant of the advice, which just informs
about the library that is being required.
These are less crucial since we stopped generating new database
versions and relied primarily on features. They also rely on a
pre-generated v1 database which happens to be chert format. This
backend is not supported by Xapian 1.5.
Also drop the tool gen-testdb.sh, which is currently broken, due to
changes in the testing infrastructure.
As an initial heuristic, report anyone with at least 15 lines of code
in the current source tree. Test corpora are excluded, although
probabably this doesn't change much about the list of authors
produced.
This is like notmuch-search-filter-by-tag, but creates a new search
rather than filtering the current search. We add this to
notmuch-common-keymap since this can be used by many contexts. We bind
to the key "t", which is the same key used by
notmuch-search-filter-by-tag in notmuch-search-mode-map. This is done
intentionally since the keybinding for notmuch-search-mode-map can be
seen as a specialization of creating a new search.
This change was motivated for use in "notmuch-hello". It is a more
convenient way to search a tag than expanding the list of all tags. I
also noticed many saved searches people use are simply tags.
nmbug and notmuch-report are developer tools. It's 2018, and all
developers should have python3 available.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
mailscripts 0.11 now ships a derivative of devel/printmimestructure
called email-print-mime-structure. Maintenance for that utility will
happen in mailscripts from now on, so we should not track an
independent copy of it in notmuch's source tree.
See https://bugs.debian.org/939993 for more details about the
adoption.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
The header-mask member of the per-message crypto object allows a
clever UI frontend to mark whether a header was protected (or not).
And if it was protected, it contains enough information to show useful
detail to an interested user. For example, an MUA could offer a "show
what this message's Subject looked like on the wire" feature in expert
mode.
As before, we only handle Subject for now, but we might be able to
handle other headers in the future.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Amended by db: tweaked schemata notation.
This allows MUAs that don't want to think about per-mime-part
cryptographic status to have a simple high-level overview of the
message's cryptographic state.
Sensibly structured encrypted and/or signed messages will work fine
with this. The only requirement for the simplest encryption + signing
is that the message have all of its encryption and signing protection
(the "cryptographic envelope") in a contiguous set of MIME layers at
the very outside of the message itself.
This is because messages with some subparts signed or encrypted, but
with other subparts with no cryptographic protection is very difficult
to reason about, and even harder for the user to make sense of or work
with.
For further characterization of the Cryptographic Envelope and some of
the usability tradeoffs, see here:
https://dkg.fifthhorseman.net/blog/e-mail-cryptography.html#cryptographic-envelope
Correct URLs that have crept into the notmuch codebase with http://
when https:// is possible.
As part of this conversion, this changeset also indicates the current
preferred upstream URLs for both gmime and sup. the new URLs are
https-enabled, the old ones are not.
This also fixes T310-emacs.sh, thanks to Bremner for catching it.
Changes since 0.2:
* Accept failures to unset core.worktree in clone (0a155847,
2017-10-10, unreleased).
* Use --no-renames in log (f9189a06, 2016-09-26, v0.24).
* Auto-checkout in clone if it wouldn't clobber (7ef3b653, 2017-10-10,
unreleased).
* Add a 'help' command for folks who don't like --help
(9d25c97d, 2014-10-03, v0.20).
* Setup a 'config' branch on clone to track origin/config (244f8739,
2015-03-22, v0.20). This branch may be consumed by
notmuch-report(1).
* Only error for invalid diff lines in tags/ (57225988, 2017-10-16,
unreleased).
* Ignore # comments in 'notmuch dump ...' output (9bbc54bd,
2016-03-27, v0.22).
* Respect 'expect' in _spawn(..., wait=True) (e263c5b1, 2017-10-10,
unreleased).
* Update URLs in documentation (554b90b5 and 6a833a6e8, 2016-06-02,
v0.23).
Avoid:
Traceback (most recent call last):
File "/home/nmbug/bin/nmbug", line 834, in <module>
args.func(**kwargs)
File "/home/nmbug/bin/nmbug", line 385, in checkout
status = get_status()
File "/home/nmbug/bin/nmbug", line 580, in get_status
maybe_deleted = _diff_index(index=index, filter='D')
File "/home/nmbug/bin/nmbug", line 658, in _diff_index
for id, tag in _unpack_diff_lines(stream=p.stdout):
File "/home/nmbug/bin/nmbug", line 678, in _unpack_diff_lines
'Invalid line in diff: {!r}'.format(line.strip()))
ValueError: Invalid line in diff: u'.mailmap'
With this commit, folks can commit READMEs, .mailmap, etc. to their
nmbug repositories, and 'nmbug diff' and 'status' won't choke on them.
If you want to check for this sort of thing, you can set --log-level
to info or greater. nmbug will still error if the unrecognized path
is under tags/, since that's more likely to be a user error.
We currently auto-checkout after pull and merge to make those more
convenient. They're guarded against data-loss with a leading
_insist_committed(). This commit adds the same convenience to clone,
since in most cases users will have no NMBPREFIX-prefixed tags in
their database when they clone. Users that *do* have
NMBPREFIX-prefixed tags will get a warning (and I've bumped the
default log level to warning so folks who don't set --log-level will
see it) like:
$ nmbug clone http://nmbug.notmuchmail.org/git/nmbug-tags.git
Cloning into '/tmp/nmbug-clone.g9dvd0tv'...
Checking connectivity: 16674, done.
Branch config set up to track remote branch config from origin.
Not checking out to avoid clobbering existing tags: notmuch::0.25, ...
Since 6311cfaf (init: do not set unnecessary core.worktree,
2016-09-25, 2.11.0 [1]), Git has no longer set core.worktree when
--separate-git-dir is used. This broke clone with:
$ nmbug clone http://nmbug.notmuchmail.org/git/nmbug-tags.git
Cloning into '/tmp/nmbug-clone.33gg442e'...
Checking connectivity: 16674, done.
['git', '--git-dir', '/home/wking/.nmbug', 'config', '--unset', 'core.worktree'] exited with 5
$ echo $?
1
The initial discussion that lead to the Git change is in [2], and
there is some more discussion around this specific change in [3].
There is some useful background on working trees in this 2009 message
[4]. There is also a git-worktree(1) since df0b6cfb (worktree: new
place for "git prune --worktrees", 2015-06-29, 2.5.0 [5]) which grew
the ability to add new worktrees in 799767cc (Merge branch
'es/worktree-add', 2015-07-13, 2.5.0 [6]). Folks relying on
core.worktree in the --separate-git-dir case fall into the "former
case" in [4], and as Junio pointed out in that message, Git
operations like 'add' don't really work there.
In nmbug we don't want core.worktree, because our effective working
tree is the notmuch database. By accepting failed core.worktree
unsets, clone will work with Gits older and younger than 2.11.0.
[1]: 6311cfaf93
[2]: https://public-inbox.org/git/CALqjkKZO_y0DNcRJjooyZ7Eso7yBMGhvZ6fE92oO4Su7JeCeng@mail.gmail.com/
[3]: https://public-inbox.org/git/87h94d8cwi.fsf@kyleam.com/
[4]: https://public-inbox.org/git/7viqbsw2vn.fsf@alter.siamese.dyndns.org/
[5]: df0b6cfbda
[6]: 799767cc98
Reported-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Fixing a bug from 7f2cb3be (nmbug: Translate to Python, 2014-10-03).
The bug had no direct impact though, because none of the wait=True
callers were setting expect.
Also add expected codes to the debug messages, to help log readers
understand why nonzero exits are occasionally accepted.
Renamed from out-of-tree-build-check.sh to be consistent with
other files in this directory.
Fixed quoting in "$srcdir" usage for additional robustness, other
quoting changes for consistency.
A leading / in paths in a .gitignore file matches the beginning of the
path, meaning that for patterns without slashes, git will match files
only in the current directory as opposed to in any subdirectory.
Prefix relevant paths with / in .gitignore files, to prevent
accidentally ignoring files in subdirectories and possibly slightly
improve the performance of "git status".
Replace numeric errors with human readable flags. Not all sig_error
keys will necessarily be generated with a given version of gmime.
Drop status "none" as it's currrently unused and I don't know what
it's for.
Instead of just having the first filename for the message, list all
duplicate filenames of the message as a list in the formatted
outputs. This bumps the format version to 3.
This adds a file under devel listing all the keybindings sorted by key
in the main three modes (search, show and tree).
To reduce clutter it only lists the single key "unmodified"
keybindings: I think all our uses of prefixes and modifiers are
natural variants of the unmodified binding (eg M-n compared to n in
show mode)
This should make easier to see what keybindings are available when
adding new features.
Git has supported this since b68ea12e (diff.c: respect diff.renames
config option, 2006-07-07, v1.4.2). All of our information is in the
paths (the files are empty), so we don't want rename detection. By
using --no-renames, we get entries like:
$ nmbug log -- e473b453a2
commit e473b453a25c072b5df67d834d822121373321f5
Author: David Bremner <david@tethera.net>
Date: Sun Sep 25 07:54:11 2016 -0300
D tags/1474196252-31700-1-git-send-email-markwalters1009@gmail.com/0.23
A tags/1474196252-31700-1-git-send-email-markwalters1009@gmail.com/pushed
...
Instead of the old:
$ nmbug log -- e473b453a2
commit e473b453a25c072b5df67d834d822121373321f5
Author: David Bremner <david@tethera.net>
Date: Sun Sep 25 07:54:11 2016 -0300
R100 tags/1474196252-31700-1-git-send-email-markwalters1009@gmail.com/0.23 tags/1474196252-31700-1-git-send-email-markwalters1009@gmail.com/pushed
Setting locale environment variables (LC_* and LANG) to e.g.
en_US.utf8 works fine on Linux, and that is what locale -a
returns (in Linux). However this does not work e.g. in some *BSD
systems.
In these systems, en_US.UTF-8 works. This also works in Linux
systems (which may look like a surprising thing on the first sight(*)).
But that *UTF-8 format seems to be widely used in the Linux system:
Grep it through the files in /etc/, for example.
Easy way to test: Run the following command lines. First should
complain about setting locale failed, and second should not.
$ LC_ALL=en_US.UTF-1 perl -e ''
$ LC_ALL=en_US.UTF-8 perl -e ''
(*) and who knows what the "standard" is...
Many of the external links found in the notmuch source can be resolved
using https instead of http. This changeset addresses as many as i
could find, without touching the e-mail corpus or expected outputs
found in tests.
Lines starting with # have always (for a long time, anyway) been ignored
by notmuch-restore, but have not been generated by notmuch-dump
previously. In order to make nmbug robust against such output, ignore
comment lines.
To describe the script and config file format, so folks don't have to
dig through NEWS or the script's source to get that information.
The Makefile and conf.py are excerpted from the main doc/ directory
with minor simplifications and adjustments. The devel/nmbug/ scripts
are largely independent of notmuch, and separating the docs here
allows packagers to easily build the docs and install the scripts in a
separate package, without complicating notmuch's core build/install
process.
status-config.json wasn't obviously associated with the old
nmubg-status, now notmuch-report. The new name is
${CONFIGURED_SCRIPT}.json, so the association should be clear.
This script generates reports based on notmuch queries, and doesn't
really have anything to do with nmbug, except for sharing the NMBGIT
environment variable.
