Commit graph

165 commits

Author SHA1 Message Date
Daniel Kahn Gillmor
6a833a6e83 Use https instead of http where possible
Many of the external links found in the notmuch source can be resolved
using https instead of http.  This changeset addresses as many as i
could find, without touching the e-mail corpus or expected outputs
found in tests.
2016-06-05 08:32:17 -03:00
Daniel Kahn Gillmor
554b90b58e nmbug: updated to modern URL for PEP 343 2016-06-05 08:27:13 -03:00
David Bremner
9bbc54bd40 nmbug: ignore # comments
Lines starting with # have always (for a long time, anyway) been ignored
by notmuch-restore, but have not been generated by notmuch-dump
previously.  In order to make nmbug robust against such output, ignore
comment lines.
2016-03-28 09:29:42 -03:00
W. Trevor King
d6cbb24fcf notmuch-report: Add notmuch-report(1) and notmuch-report.json(5) man pages
To describe the script and config file format, so folks don't have to
dig through NEWS or the script's source to get that information.

The Makefile and conf.py are excerpted from the main doc/ directory
with minor simplifications and adjustments.  The devel/nmbug/ scripts
are largely independent of notmuch, and separating the docs here
allows packagers to easily build the docs and install the scripts in a
separate package, without complicating notmuch's core build/install
process.
2016-03-24 07:41:24 -03:00
W. Trevor King
399c857eba notmuch-report.json: Rename from status-config.json
status-config.json wasn't obviously associated with the old
nmubg-status, now notmuch-report.  The new name is
${CONFIGURED_SCRIPT}.json, so the association should be clear.
2016-03-24 07:41:06 -03:00
W. Trevor King
074f45e305 notmuch-report: Rename from nmbug-status
This script generates reports based on notmuch queries, and doesn't
really have anything to do with nmbug, except for sharing the NMBGIT
environment variable.
2016-03-24 07:40:39 -03:00
W. Trevor King
88171f34ed status-config.json: Remove parens from query entry
These are now added by nmbug-status.
2016-03-24 07:39:57 -03:00
W. Trevor King
6c0b6c8e1c nmbug-status: Wrap query phrases in parentheses when and-ing together
For example:

  "query": ["tag:a", "tag:b or tag:c"]

is now converted to:

  ( tag:a ) and ( tag:b or tag:c )

instead of the old:

  tag:a and tag:b or tag:c

This helps us avoid confusion due to Xapian's higher-precedence AND
[1], where the old query would be interpreted as:

  ( tag:a and tag:b ) or tag:c

[1]: http://xapian.org/docs/queryparser.html
2016-03-24 07:39:42 -03:00
W. Trevor King
031ca3f1bd nmbug-status: Add meta.message-url config setting
So you can link to archives other than Gmane.  For example, I'm doing
this in [1].

[1]: https://github.com/wking/nmbug-oci
2016-03-24 07:39:25 -03:00
David Bremner
9f8b2091c8 STYLE: suggest long names
Hopefully the exceptions are common sense.
2016-03-04 20:45:06 -04:00
David Bremner
99a0a90f65 STYLE: document some rules about variable declarations
No-one seemed opposed to C99 style loop variable declarations. The
requirement to declare variables at the top of blocks is maybe a little
more contested, but I believe it reflects the status quo.
2016-03-04 20:44:57 -04:00
Tomi Ollila
3959d98535 devel/emacs: add devel/try-emacs-mua
devel/try-emacs-mua provides an easy way to try and experiment
with the notmuch emacs client distributed in emacs subdirectory of
the notmuch source tree.

try-emacs-mua starts a new emacs process and if initial checks pass
*scratch* buffer is filled with information of how to begin.

Normal emacs command line arguments can be used, like -q or -Q.
These arguments are appended verbatim to the starting emacs process.

If the emacs version in use is smaller than 24.4, special care is taken
to ensure that notmuch*.elc files older than corresponding .el files
are not loaded. Since emacs 24.4, setting `load-prefer-newer' variable
takes care of this.
2016-01-13 10:36:19 -04:00
W. Trevor King
d6204e8362 nmbug-status: Fix unbalanced <p> tags in default header/footer
These were broken by b70386a4 (Move the generated date from the top of
the page to the footer, 2014-05-31), which moved 'Generated ...' to
the footer with the opening tag, but didn't replace the blurb opening
tag or add a closing tag after 'Generated ...'.
2016-01-13 10:29:14 -04:00
W. Trevor King
8fc32dd42e nmbug-status: Style headers with smaller fonts
We only use h1 through h3, and David prefers smaller headers [1], so
shift over to the font sizes usually used for h2 through h4 [2,3,4].
I haven't bothered with the W3C's default margins, since a bit of
extra whitespace doesn't seem like a big deal.

[1]: id:87k2nl8r0k.fsf@zancas.localnet
     http://article.gmane.org/gmane.mail.notmuch.general/21595
[2]: http://www.w3.org/TR/html-markup/h2.html
[3]: http://www.w3.org/TR/html-markup/h3.html
[4]: http://www.w3.org/TR/html-markup/h4.html
2016-01-13 10:28:49 -04:00
W. Trevor King
e3d586d038 nmbug-status: Adjust headers to start with h1
We've been leading off with h2s since 3e5fb88f (contrib/nmbug: add
nmbug-status script, 2012-07-07), but the semantically-correct headers
are:

  <h1>{title}</h1>
  ...
  <h2>Views</h2>
  ...
  <h3>View 1</h3>
  ...
  <h3>View 2</h3>
  ...

We can always add additional CSS if the default h1 formatting is too
intense.
2016-01-13 10:28:38 -04:00
W. Trevor King
a07cafae00 nmbug-status: Avoid hard-coded filename in error message
We already have a 'filename' variable with the name, so stay DRY and
use that variable here.

Also fix a missing-whitespace error from bed8b674 (nmbug-status:
Clarify errors for illegible configs, 2014-05-10), wrapping on the
sentence to match similar error-generation earlier in this function.
2016-01-07 09:15:35 -04:00
Tomi Ollila
36ad271704 devel/release-checks: added checking of copyright year in documentation
Check that copyright year will be current year in generated documentation.

Checking is done my matching that copyright line contains current year
as a substring which is good enough "approximation" in this context.
2015-11-19 07:34:29 -04:00
Jani Nikula
018e69f558 nmbug-status: print config errors to stderr
Particularly scripted usage with stdout redirection can be confusing
if errors are printed to stdout instead of stderr.
2015-10-04 07:22:03 -03:00
Jani Nikula
8b35b8f71a nmbug-status: add support for specifying sort order for each view
Let each view have a "sort" key, typically used with values
"oldest-first" or "newest-first" (although all values in Query.SORT
are accepted), and sort the results accordingly. Oldest first remains
the default.

The dynamic approach of mapping sort values is as suggested by
W. Trevor King <wking@tremily.us>.
2015-09-29 08:25:25 -03:00
David Bremner
765556c1f1 build: extract library versions from notmuch.h
- Make lib/notmuch.h the canonical location for the library versioning
information.

- Since the release-check should never fail now, remove it to reduce
complexity.

- Make the version numbers in notmuch.h consistent with the (now
  deleted) ones in lib/Makefile.local
2015-08-10 13:53:55 +02:00
Tomi Ollila
02a2eeb427 test: make script exit (1) if it "fails" to source (.) a file
The files (test) scripts source (with builtin command `.`) provides
information which the scripts depend, and without the `source` to
succeed allowing script to continue may lead to dangerous situations
(e.g. rm -rf "${undefined_variable}"/*).

At the end of all source (.) lines construct ' || exit 1' was added;
In our case the script script will exit if it cannot find (or read) the
file to be sourced. Additionally script would also exits if the last
command of the sourced file exited nonzero.
2015-08-07 21:56:39 +02:00
Tomi Ollila
6d5c7b3ca5 devel/release-checks.sh: made python version check work with python 3
This trivial change consists of just putting print() argument in
parentheses.
2015-08-04 16:27:34 +02:00
David Bremner
7e2d0ef105 lib: define NOTMUCH_DEPRECATED macro, document its use.
This has been tested with gcc and clang.
2015-08-04 09:11:17 +02:00
Tomi Ollila
a5a21bbe78 release-checks: check that git working directory is clean
Before release check that there are no uncommitted changes and
that there are no files in working directory that possibly should
have been added to the repository.

Amended by db: remove --ignored, since that seems like too much
trouble.
2015-06-09 19:33:22 +02:00
W. Trevor King
244f873954 nmbug-status: Use 'show-ref --heads' for loading configs
When loading configs from Git, the bare branch name (without a
refs/heads/ prefix or similar) matches all branches of that name
(including remote-tracking branches):

  .nmbug $ git show-ref config
  48f3bbf1d1492e5f3d2f01de6ea79a30d3840f20 refs/heads/config
  48f3bbf1d1492e5f3d2f01de6ea79a30d3840f20 refs/remotes/origin/config
  4b6dbd9ffd152e7476f5101eff26747f34497cee refs/remotes/wking/config

Instead of relying on the ordering of the matching references, use
--heads to ensure we only match local branches.
2015-04-01 06:16:06 +02:00
W. Trevor King
bed8b6743d nmbug-status: Clarify errors for illegible configs
Carl Worth pointed out that errors like:

  $ ./nmbug-status
  fatal: Not a git repository: '/home/cworth/.nmbug'
  fatal: Not a git repository: '/home/cworth/.nmbug'
  Traceback (most recent call last):
    File "./nmbug-status", line 254, in <module>
      config = read_config(path=args.config)
    File "./nmbug-status", line 73, in read_config
      return json.load(fp)
    File "/usr/lib/python2.7/json/__init__.py", line 290, in load
      **kw)
    File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
      return _default_decoder.decode(s)
    File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
      obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
      raise ValueError("No JSON object could be decoded")
  ValueError: No JSON object could be decoded

are not particularly clear.  With this commit, we'll get output like:

  $ ./nmbug-status
  fatal: Not a git repository: '/home/wking/.nmbug'
  No local branch 'config' in /home/wking/.nmbug.  Checkout a local
  config branch or explicitly set --config.

which is much more accessible.  I've also added user-friendly messages
for a number of other config-parsing errors.
2015-03-15 09:28:15 +01:00
Jani Nikula
6d44e5ac47 lib: make notmuch_query_count_messages explicitely exact
The default is actually exact if no checkatleast parameter is
specified. This change makes that explicit, mainly for documentation,
but also to be safe in the unlikely event of a change of default.

[ commit message rewritten by db based on id:87lho0nlkk.fsf@nikula.org
]
2015-03-13 07:58:55 +01:00
W. Trevor King
9d25c97d8b nmbug: Add a 'help' command for folks who don't like --help
The 'if args.func == help' block at the end avoids:

    AttributeError: 'functools.partial' object has no attribute '__code__'
2015-01-18 11:00:51 +01:00
David Bremner
2bb906a6dd build: eliminate use of python execfile command
As discussed in
id:8cc9dd580ad672527e12f43706f9803b2c8e99d8.1405220724.git.wking@tremily.us,
execfile is unavailable in python3.

The approach of this commit avoids modifying the python module path,
which is arguably preferable since it avoids potentially accidentally
importing a module from the wrong place.
2015-01-03 15:18:54 +01:00
Tomi Ollila
34d2c62b82 devel: man-to-mdwn.pl: add GPLv3+ notice to manpages.mdwn
Make the generated head manual page (in the notmuchmail wiki)
display the licence information the whole notmuch software is licenced
under.
2014-11-02 19:46:17 +01:00
Tomi Ollila
f7be8f4ebf devel: make man-to-mdwn.pl to work with generated manual pages
The new manual pages converted from rst using sphinx or rst2man
has somewhat different syntax. man-to-mdwn.pl is now adjusted
to produce even better output from this syntax. The changes also
include using utf-8 locale (e.g. for tables and generated hypens)
and and quite a few bugs fixes.
This tool still produces better results than just using the
html pages generated using sphinx / rst2html. For example those
tools don't create inter-page hyperlinks -- and the preformatted
pages written by man-to-mdwn.pl just works well with manual page
content.
2014-11-02 19:46:06 +01:00
W. Trevor King
7f2cb3be4e nmbug: Translate to Python
This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages.  It should be compatible with Python 2.7 and later
(including the 3.x series).

Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:

* 'nmbug help' has been split between the general 'nmbug --help' and
  the command-specific 'nmbug COMMAND --help'.

* Commands are no longer split into "most common", "other useful", and
  "less common" sets.  If we need something like this, I'd prefer
  workflow examples highlighting common commands in the module
  docstring (available with 'nmbug --help').

* 'nmbug commit' now only uses a single argument for the optional
  commit-message text.  I wanted to expose more of the underlying 'git
  commit' UI, since I personally like to write my commit messages in
  an editor with the notes added by 'git commit -v' to jog my memory.
  Unfortunately, we're using 'git commit-tree' instead of 'git
  commit', and commit-tree is too low-level for editor-launching.  I'd
  be interested in rewriting commit() to use 'git commit', but that
  seemed like it was outside the scope of this rewrite.  So I'm not
  supporting all of Git's commit syntax in this patch, but I can at
  least match 'git commit -m MESSAGE' in requiring command-line commit
  messages to be a single argument.

* The default repository for 'nmbug push' and 'nmbug fetch' is now the
  current branch's upstream (branch.<name>.remote) instead of
  'origin'.  When we have to, we extract this remote by hand, but
  where possible we just call the Git command without a repository
  argument, and leave it to Git to figure out the default.

* 'nmbug push' accepts multiple refspecs if you want to explicitly
  specify what to push.  Otherwise, the refspec(s) pushed depend on
  push.default.  The Perl version hardcoded 'master' as the pushed
  refspec.

* 'nmbug pull' defaults to the current branch's upstream
  (branch.<name>.remote and branch.<name>.merge) instead of hardcoding
  'origin' and 'master'.  It also supports multiple refspecs if for
  some crazy reason you need an octopus merge (but mostly to avoid
  breaking consistency with 'git pull').

* 'nmbug log' now execs 'git log', as there's no need to keep the
  Python process around once we've launched Git there.

* 'nmbug status' now catches stderr, and doesn't print errors like:

    No upstream configured for branch 'master'

  The Perl implementation had just learned to avoid crashing on that
  case, but wasn't yet catching the dying subprocess's stderr.

* 'nmbug archive' now accepts positional arguments for the tree-ish
  and additional 'git archive' options.  For example, you can run:

    $ nmbug archive HEAD -- --format tar.gz

  I wish I could have preserved the argument order from 'git archive'
  (with the tree-ish at the end), but I'm not sure how to make
  argparse accept arbitrary possitional arguments (some of which take
  arguments).  Flipping the order to put the tree-ish first seemed
  easiest.

* 'nmbug merge' and 'pull' no longer checkout HEAD before running
  their command, because blindly clobbering the index seems overly
  risky.

* In order to avoid creating a dirty index, 'nmbug commit' now uses
  the default index (instead of nmbug.index) for composing the commit.
  That way the index matches the committed tree.  To avoid leaving a
  broken index after a failed commit, I've wrapped the whole thing in
  a try/except block that resets the index to match the pre-commit
  treeish on errors.  That means that 'nmbug commit' will ignore
  anything you've cached in the index via direct Git calls, and you'll
  either end up with an index matching your notmuch tags and the new
  HEAD (after a successful commit) or an index matching the original
  HEAD (after a failed commit).
2014-10-05 07:16:29 +02:00
W. Trevor King
567aab405b nmbug: Handle missing @upstream in is_unmerged
If we don't have an upstream, there is nothing to merge, so nothing is
unmerged.  This avoids errors like:

  $ nmbug status
  error: No upstream configured for branch 'master'
  error: No upstream configured for branch 'master'
  fatal: ambiguous argument '@{upstream}': unknown revision or path not in the working tree.
  Use '--' to separate paths from revisions, like this:
  'git <command> [<revision>...] -- [<file>...]'
  'git rev-parse @{upstream}' exited with nonzero value

You might not have an upstream if you're only using nmbug locally to
version-control your tags.
2014-07-16 19:31:28 -03:00
W. Trevor King
4697e86a52 nmbug: Add a git_with_status helper function
Sometimes we want to catch Git errors and handle them, instead of
dying with an error message.  This lower-level version of git() allows
us to get the error status when we want it.
2014-07-16 06:31:23 -03:00
W. Trevor King
7f2bbe93a5 nmbug-status: Optionally load the header and footer templates from the config
For folks that don't like the default templates for whatever reason.
2014-07-15 20:15:23 -03:00
W. Trevor King
f10024f30d nmbug-status: Add the time to the footer's build-date
Our repository [1] has a post-update hook that rebuilds the status
page after each push.  Since that may happen several times a day, we
might as well show the build time (as well as the date) in the footer.

The trailing 'Z' is the ISO 8601 designator for UTC.  Now that we're
showing times, it's nice to be explicit about the timezone we're
using.

The rename from date -> datetime gives us backward-compatibility for
folks that *do* only want the date.  We keep the old date formatting
to support those folks.

[1]: http://nmbug.tethera.net/git/nmbug-tags.git
2014-07-15 20:08:39 -03:00
W. Trevor King
7832258468 nmbug-status: Factor out header/footer context into a shared dict
Rather than splitting this context into header-only and footer-only
groups, just dump it all in a shared dict.  This will make it easier
to eventually split the header/footer templates out of this script
(e.g. if we want to load them from the config file).
2014-07-15 20:08:24 -03:00
Carl Worth
b70386a4cb Move the generated date from the top of the page to the footer.
It's useful reference information, but anyone who wants it will look
for and find it.  We don't need this front-and-center.  Follow the
pattern set by our header template with a triple-quoted string.

The gray <hr> styling is less agressive.  IE uses 'color' for drawing
the rule, while Gecko and Opera use the border or 'background-color'
[1].

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=239386
2014-07-15 20:07:18 -03:00
Carl Worth
b5bdf5f4a8 Add a docstring describing the program
Prefer a docstring to a header comment so we can use it as the
ArgumentParser description (formatted with 'nmbug-status --help').
Script readers still have it near the top of the file.  Since it's a
docstring, use PEP 257's summary-line-and-body format [1].

[1]: http://legacy.python.org/dev/peps/pep-0257/#multi-line-docstrings
2014-07-15 20:07:06 -03:00
Carl Worth
d1d84cab1d Add explicit license information
It's nice to have for situations where this script is found outside of
the notmuch source repository (e.g. after being installed).
2014-07-15 20:06:59 -03:00
Tomi Ollila
24b24be151 devel: news2wiki to recognize yyyy-mm-dd or UNRELEASED as release date
The yyyy-mm-dd (actually \d\d\d\d-\d\d-\d\d) for a bit more restrictive
(and self-documentative) than the \w\w\w\w-... that used to be there and
UNRELEASED so that developers can test the latest NEWS converted to mdwn
format before submitting NEWS patches.
2014-07-13 12:35:37 -03:00
David Bremner
dd24fdd33a nmbug: mark repository as bare on clone
If a git repository is non-bare, and core.worktree is not set, git
tries to deduce the worktree. This deduction is not always helpful, e.g.

% git --git-dir=$HOME/.nmbug clean -f

would likely delete most of the files in the current directory
2014-04-15 17:20:19 -03:00
W. Trevor King
c200167426 nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}
With two branches getting fetched (master and config), the branch
referenced by FETCH_HEAD is ambiguous.  For example, I have:

  $ cat FETCH_HEAD
  41d7bfa7184cc93c9dac139d1674e9530799e3b0 \
    not-for-merge   branch 'config' of http://nmbug.tethera.net/git/nmbug-tags
  acd379ccb973c45713eee9db177efc530f921954 \
    not-for-merge   branch 'master' of http://nmbug.tethera.net/git/nmbug-tags

(where I wrapped the line by hand).  This means that FETCH_HEAD
references the config branch:

  $ git rev-parse FETCH_HEAD
  41d7bfa7184cc93c9dac139d1674e9530799e3b0

which breaks all of the FETCH_HEAD logic in nmbug (where FETCH_HEAD is
assumed to point to the master branch).

Instead of relying on FETCH_HEAD, use @{upstream} as the
remote-tracking branch that should be merged/diffed/integrated into
HEAD.  @{upstream} was added in Git v1.7.0 (2010-02-12) [1], so
relying on it should be fairly safe.  One tricky bit is that bare
repositories don't set upstream tracking branches by default:

  $ git clone --bare http://nmbug.tethera.net/git/nmbug-tags.git nmbug-bare
  $ cd nmbug-bare
  $ git remote show origin
  * remote origin
    Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
    Push  URL: http://nmbug.tethera.net/git/nmbug-tags.git
    HEAD branch: master
    Local refs configured for 'git push':
      config pushes to config (up to date)
      master pushes to master (up to date)

While in a non-bare clone:

  $ git clone http://nmbug.tethera.net/git/nmbug-tags.git
  $ cd nmbug-tags
  $ git remote show origin
  * remote origin
    Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
    Push  URL: http://nmbug.tethera.net/git/nmbug-tags.git
    HEAD branch: master
    Remote branches:
      config tracked
      master tracked
    Local branch configured for 'git pull':
      master merges with remote master
    Local ref configured for 'git push':
      master pushes to master (up to date)

From the clone docs [2]:

  --bare::
        Make a 'bare' Git repository…
        Also the branch heads at the remote are copied directly
        to corresponding local branch heads, without mapping
        them to `refs/remotes/origin/`.  When this option is
        used, neither remote-tracking branches nor the related
        configuration variables are created.

To use @{upstream}, we need to the local vs. remote-tracking
distinction, so this commit adds 'nmbug clone', replacing the
previously suggested --bare clone with a non-bare --no-checkout
--separate-git-dir clone into a temporary work directory.  After
which:

  $ git rev-parse @{upstream}
  acd379ccb973c45713eee9db177efc530f921954

gives us the master-branch commit.  Existing nmbug users will have to
run the configuration tweaks and re-fetch by hand.  If you don't have
any local commits, you could also blow away your NMBGIT repository and
re-clone from scratch:

  $ nmbug clone http://nmbug.tethera.net/git/nmbug-tags.git

Besides removing the ambiguity of FETCH_HEAD, this commit allows users
to configure which upstream branch they want nmbug to track via 'git
config', in case they want to change their upstream repository.

[1]: http://git.kernel.org/cgit/git/git.git/tree/Documentation/RelNotes/1.7.0.txt
[2]: http://git.kernel.org/cgit/git/git.git/tree/Documentation/git-clone.txt
2014-04-08 07:39:28 -03:00
Tomi Ollila
5608e39a6b release-checks: removed manual page version check
Manual pages are now generated and during the generation the version
string is read from `version` file, so this (currently failing) test
checking manual page versions can be removed.

While at it, changed the case pattern *[^0-9.]*
to its portable alternative *[!0-9.]*
2014-04-08 07:32:34 -03:00
David Bremner
b6751cf875 doc: move doxgen config from devel/ to doc/
a first step towards actually instally the API docs
2014-04-08 07:30:42 -03:00
Jani Nikula
87c2cd78fd nmbug-status: make output title and blurb configurable
Make nmbug-status more generally usable outside of nmbug by not
hardcoding notmuch related things.

This lets anyone publish html search views to mailing list messages
with a custom config file, independent of nmbug.
2014-03-23 08:33:34 -03:00
Jani Nikula
81a1aae2dc nmbug-status: parameterize title and blurb in the page header
Prepare for more general use.
2014-03-23 08:32:40 -03:00
Jani Nikula
6af2b05612 devel: add script to generate test databases
Add script to generate notmuch test databases using specified versions
of notmuch. This is useful for generating material for database
upgrade tests.

This reuses the test infrastructure to have a sandbox environment for
notmuch new etc.
2014-03-11 19:51:22 -03:00
Tomi Ollila
18d71908b2 nmbug-status: replace __values__() with values() in OrderedDict stub
Python dict() object does not have __values__() function which
OrderedDict().values() (the stub provided in nmbug-status) could call
to provide ordered list of values. By renaming this thinko to
values() will make our stub work as expected -- dict items listed out
in order those were added to the dictionary.
2014-02-21 21:12:45 -04:00
W. Trevor King
320d4a856e nmbug-status: Hardcode UTF-8 instead of using the user's locale
David [1] and Tomi [2] both feel that the user's choice of LANG is not
explicit enough to have such a strong effect on nmbug-status.  For
example, cron jobs usually default to LANG=C, and that is going to
give you ASCII output:

  $ LANG=C python -c 'import locale; print(locale.getpreferredencoding())'
  ANSI_X3.4-1968

Trying to print Unicode author names (and other strings) in that
encoding would crash nmbug-status with a UnicodeEncodeError.  To avoid
that, this patch hardcodes UTF-8, which can handle generic Unicode,
and is the preferred encoding (regardless of LANG settings) for
everyone who has chimed in on the list so far.  I'd prefer trusting
LANG, but in the absence of any users that prefer non-UTF-8 encodings
I'm fine with this approach.

While we could achieve the same effect on the output content by
dropping the previous patch (nmbug-status: Encode output using the
user's locale), Tomi also wanted UTF-8 hardcoded as the config-file
encoding [2].  Keeping the output encoding patch and then adding this
to hardcode both the config-file and output encodings at once seems
the easiest route, now that fd29d3f (nmbug-status: Decode Popen output
using the user's locale, 2014-02-10) has landed in master.

[1]: id="877g8z4v4x.fsf@zancas.localnet"
     http://article.gmane.org/gmane.mail.notmuch.general/17202
[2]: id="m2vbwj79lu.fsf@guru.guru-group.fi"
     http://article.gmane.org/gmane.mail.notmuch.general/17209
2014-02-14 21:45:07 -04:00
W. Trevor King
ffed8f2866 nmbug-status: Encode output using the user's locale
Instead of always writing UTF-8, allow the user to configure the
output encoding using their locale.  This is useful for previewing
output in the terminal, for poor souls that don't use UTF-8 locales
;).
2014-02-14 21:44:11 -04:00
W. Trevor King
0d563dae1d nmbug-status: Add inter-message padding
We already had the tbody with a blank row separating threads (which is
not colored); this commit adds a bit of spacing to separate messages
within a thread.  It will also add a bit of colored padding above the
first message and below the final message, but the main goal is to add
padding *between* two-row message blocks.

                                               <--- new padding
  thread-1, message-1, row-1  (class="message-first")
  thread-1, message-1, row-2  (class="message-last")
                                               <--- new padding
    spacer tbody with a blank row
                                               <--- new padding
  thread-2, message-1, row-1  (class="message-first")
  thread-2, message-1, row-2  (class="message-last")
                                               <--- new padding
                                               <--- new padding
  thread-2, message-2, row-1  (class="message-first")
  thread-2, message-2, row-2  (class="message-last")
                                               <--- new padding
2014-02-14 21:42:36 -04:00
W. Trevor King
1aa56c2ac5 nmbug-status: Color threads in HTML output
Add tbody sections so we don't have to color every row.  Multiple
tbody sections are allowed [1].  Use CSS 3's nth-child to handle
even/odd coloring (skipping the spacer rows) [2], which is supported
on the major browsers [3].

border-spacing is from CCS 2.1 [4,5].  I'm using it to avoid
whitespace between td cells.

border-radius is from CCS 3 [6,7].  I'm using it to make the colored
sections a bit less harsh.  I tried adding rounded borders to the
tbody itself doesn't work, but I couldn't get that to work without
setting the tbody's display to 'block'.  That rounded the corners, but
collapsed the cell spacing (e.g. columns were no longer aligned).
This commit's by-corner-td approach is not particularly elegant, but
it works.  The td padding entries just ensure that the cell body is
suitably far from the edges that it doesn't fall outside of the
rounded corners.

The doubled-braces are escapes from Python's str.format.

[1]: http://www.w3.org/TR/html5/tabular-data.html#the-table-element
[2]: http://www.w3.org/TR/css3-selectors/#nth-child-pseudo
[3]: https://developer.mozilla.org/en-US/docs/Web/CSS/:nth-child#Browser_compatibility
[4]: http://www.w3.org/TR/CSS2/tables.html#propdef-border-spacing
[5]: https://developer.mozilla.org/en-US/docs/Web/CSS/border-spacing#Browser_compatibility
[6]: http://www.w3.org/TR/css3-background/#the-border-radius
[7]: https://developer.mozilla.org/en-US/docs/Web/CSS/border-radius#Browser_compatibility
2014-02-14 21:40:20 -04:00
W. Trevor King
aaa7f0d92e nmbug-status: Escape &, <, and > in HTML display data
'message-id' and 'from' now have sensitive characters escaped using
xml.sax.saxutils.escape [1].  The 'subject' data was already being
converted to a link into Gmane; I've escape()d that too, so it doesn't
need to be handled ain the same block as 'message-id' and 'from'.

This prevents broken HTML by if subjects etc. contain characters that
would otherwise be interpreted as HTML markup.

[1]: http://docs.python.org/3/library/xml.sax.utils.html#xml.sax.saxutils.escape
2014-02-14 08:29:33 -04:00
W. Trevor King
aa32d2579b nmbug-status: Use <code> and <p> markup where appropriate
* Wrap free text in <p> tags.
* Convert <blockquote> to <p><code> for query strings.
* Wrap message-id-term (id:"...") in <code>.

The <code> tags get nicer default markup (smaller monospace font) for
notmuch query terms [1].  The <p> tags don't have much effect without
attached CSS, but bare text (phrasing content [2]) in <body> (which
expects flow content [3,4]) feels wrong.

[1]: http://www.w3.org/TR/html5/text-level-semantics.html#the-code-element
[2]: http://www.w3.org/TR/html5/dom.html#phrasing-content-1
[3]: http://www.w3.org/TR/html5/dom.html#flow-content-1
[4]: http://www.w3.org/TR/html5/sections.html#the-body-element
2014-02-13 21:47:37 -04:00
W. Trevor King
711d04c821 nmbug-status: Slug the title when using it as an id
Also allow manual id overrides from the JSON config.  Sluggin avoids
errors like:

   Bad value '#Possible bugs' for attribute href on element a:
   Whitespace in fragment component. Use %20 in place of spaces.

from http://validator.w3.org.

I tried just quoting the titles (e.g. 'Possible%20bugs'), but that
didn't work (at least with Firefox 24.2.0).  Slugging avoids any
ambiguity over when the quotes are expanded in the client.  The specs
are unclear about quoting, saying only [1]:

  Value: Any string, with the following restrictions:

    must be at least one character long
    must not contain any space characters

[1]: http://dev.w3.org/html5/markup/global-attributes.html#common.attrs.id
2014-02-13 21:45:40 -04:00
W. Trevor King
e485b5bd1d nmbug-status: Anchor with h3 ids instead of a names
HTML 5 allows id attributes on all HTML elements [1], but restricts
names to particular cases [2].  Attaching the id attribute to the h3
element allows us to drop the anchor a element altogether.

[1]: http://www.w3.org/TR/html5/dom.html#the-id-attribute
[2]: http://www.w3.org/TR/html5/index.html#attributes-1
2014-02-13 21:44:13 -04:00
W. Trevor King
9b98c5f8e9 nmbug-status: Convert from XHTML 1.0 to HTML 5
HTML 5 for the win :).  I also de-namespaced the language; the HTML 5
spec allows a vestigial xml:lang attribute, but it's a no-op [1], so I
stripped it.

This shouldn't break anything at tethera, which already serves the
status as text/html:

  $ wget -S http://nmbug.tethera.net/status/
  --2014-02-02 21:20:39--  http://nmbug.tethera.net/status/
  Resolving nmbug.tethera.net... 87.98.215.224
  Connecting to nmbug.tethera.net|87.98.215.224|:80... connected.
  HTTP request sent, awaiting response...
    HTTP/1.1 200 OK
    Vary: Accept-Encoding
    Content-Type: text/html
  ...

This also matches the Content-Type in the generated HTML's http-equiv
meta.

[1]: http://www.w3.org/TR/html5/dom.html#the-lang-and-xml:lang-attributes
2014-02-10 22:55:04 -04:00
W. Trevor King
a3a3725e43 nmbug-status: Normalize table HTML indentation
I don't think I've ever seen '</td><td>{value}\n' before :p.  The new
formatting avoids mixing tag levels and content across lines.
2014-02-10 22:53:28 -04:00
W. Trevor King
c48a12365f nmbug-status: Add an OrderedDict stub for Python 2.6
Tomi Ollila and David Bremner (and presumably others) are running
Python 2.6 on their nmbug-status boxes, so it makes sense to keep
support for that version.  This commit adds a really minimal
OrderedDict stub (e.g. it doesn't handle key removal), but it gets the
job done for Page._get_threads.  Once we reach a point where Python
2.6 is no longer important (it's already out of it's security-fix
window [1]), we can pull this stub back out.

[1]: http://www.python.org/download/releases/2.6.9/
2014-02-10 22:51:40 -04:00
W. Trevor King
98cb4779c0 nmbug-status: Add Page and HtmlPage for modular rendering
I was having trouble understanding the logic of the longish print_view
function, so I refactored the output generation into modular bits.
The basic text rendering is handled by Page, which has enough hooks
that HtmlPage can borrow the logic and slot-in HTML generators.

By modularizing the logic it should also be easier to build other
renderers if folks want to customize the layout for other projects.

Timezones
=========

This commit has not effect on the output, except that some dates have
been converted from the sender's timezone to UTC due to:

  -            val = m.get_header(header)
  -            ...
  -            if header == 'date':
  -                val = str.join(' ', val.split(None)[1:4])
  -                val = str(datetime.datetime.strptime(val, '%d %b %Y').date())
  ...
  +                value = str(datetime.datetime.utcfromtimestamp(
  +                    message.get_date()).date())

I also tweaked the HTML header date to be utcnow instead of the local
now() to make all times independent of the generator's local time.
This matches Gmane, which converts all Date headers to UTC (although
they use a 'GMT' suffix).  Notmuch uses
g_mime_utils_header_decode_date to calculate the UTC timestamps, but
uses a NULL tz_offset which drops the information we'd need to get
back to the sender's local time [1].  With the generator's local time
arbitrarily different from the sender's and viewer's local time,
sticking with UTC seems the best bet.

[1]: https://developer.gnome.org/gmime/stable/gmime-gmime-utils.html#g-mime-utils-header-decode-date
2014-02-10 22:51:03 -04:00
W. Trevor King
7b7a83cc32 nmbug-status: Add a Python-3-compatible urllib.parse.quote import
Python 2's urllib.quote [1] has moved to urllib.parse.quote in Python
3 [2].

[1]: http://docs.python.org/2/library/urllib.html#urllib.quote
[2]: http://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote
2014-02-10 22:49:26 -04:00
W. Trevor King
a6ff03df3f nmbug-status: Consolidate HTML header printing
Make this all one big string, using '...{date}...'.format(date=...) to
inject the date [1].  This syntax was added in Python 2.6, and is
preferred to %-formatting in Python 3 [1].

[1]: http://docs.python.org/2/library/stdtypes.html#str.format
2014-02-10 22:47:52 -04:00
W. Trevor King
a7e4d9a18f nmbug-status: Don't require write access
The database in only used for notmuch.Query, so there's no need for
write access.  This allows nmbug-status to run while the database is
being updated, without raising:

  A Xapian exception occurred opening database: Unable to get write lock on …: already locked
  Traceback (most recent call last):
    File "./nmbug-status", line 182, in <module>
      db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
    File "/…/notmuch/database.py", line 154, in __init__
      self.open(path, mode)
    File "/…/notmuch/database.py", line 214, in open
      raise NotmuchError(status)
  notmuch.errors.XapianError
2014-02-10 22:46:37 -04:00
W. Trevor King
e4d79bfddb nmbug-status: Consolidate functions and main code
The definitions of Thread, output_with_separator, and print_view were
between the main argparse and view-printing code.  Group them together
with our existing read_config at the top of the module, which makes
for easier reading in the main section.

I also:

* Made 'headers' a print_view argument instead of a module-level
  global.  The list -> tuple conversion avoids having a mutable
  default argument, which makes some people jumpy ;).

* Made 'db' a print_view argument instead of relying on the global
  namespace to access it from print_view.
2014-02-10 22:45:28 -04:00
W. Trevor King
a2b64211b2 nmbug-status: Add metavars for --config and --get-query
Now the suggested usage (listed by 'nmbug-status --help') is:

  usage: nmbug-status [-h] [--text] [--config PATH] [--list-views]
                      [--get-query VIEW]

instead of the less obvious:

  usage: nmbug-status [-h] [--text] [--config CONFIG] [--list-views]
                      [--get-query GET_QUERY]
2014-02-10 21:13:37 -04:00
W. Trevor King
b7e6d2cc30 nmbug-status: Factor config-loading out into read_config
By isolating this peripheral handling, we make the core logic of
nmbug-status easier to read.
2014-02-10 21:09:16 -04:00
W. Trevor King
fd29d3f4fb nmbug-status: Decode Popen output using the user's locale
Avoid:

  $ ./nmbug-status --list-views
  Traceback (most recent call last):
    File "./nmbug-status", line 47, in <module>
      'cat-file', 'blob', sha1+':status-config.json'],
  TypeError: can't concat bytes to str

by explicitly converting the byte-stream read from Popen into a
Unicode string.  On Python 2, this conversion is str -> unicode; on
Python 3 it is bytes -> str.

_ENCODING is derived from the user's locale (or system default) in an
attempt to match Git's output encoding.  It may be more robust to skip
the encoding/decoding by using a Python wrapper like pygit2 [1] for
Git access.  That's a fairly heavy dependency though, and using the
locale will probably work.

[1]: http://www.pygit2.org/
2014-02-10 21:04:55 -04:00
W. Trevor King
ee6b5c372f nmbug-status: Use email.utils instead of rfc822
rfc822 has been deprecated since Python 2.3, and it's gone in
Python 3 [1].

[1]: http://docs.python.org/2/library/rfc822.html
2014-02-10 21:03:12 -04:00
W. Trevor King
a8b81adc8e nmbug-status: Convert to Python-3-compatible print functions
We shouldn't require folks to install Python 2 to run nmbug-status.
2014-02-10 20:52:23 -04:00
Jani Nikula
f230fd167a devel: add doxygen configuration file
This is a pretty basic config to get started, generated using 'doxygen
-s -g' and mildly tweaked.

To generate the library man page man/man3/notmuch.3 from lib/notmuch.h
use:

$ doxygen devel/doxygen.cfg
2014-01-05 08:59:52 -04:00
Tomi Ollila
0c097d9f6e devel/release-checks.sh: adjust to LIBNOTMUCH version checks
NOTMUCH_VERSION_* macros in lib/notmuch.h are replaced with
LIBNOTMUCH_VERSION_* macros. Check that the values of those
match the LIBNOTMUCH_*_VERSION values in lib/Makefile.local.
2013-12-30 20:39:25 -04:00
Tomi Ollila
001d932816 devel/release-checks.sh: check NOTMUCH_(MAJOR|MINOR|MICRO)_VERSION
New defines NOTMUCH_MAJOR_VERSION, NOTMUCH_MINOR_VERSION and
NOTMUCH_MICRO_VERSION were added to lib/notmuch.h.
Check that these match the current value defined in ./version.
2013-11-11 07:44:44 -04:00
Austin Clements
96c0ce28f8 emacs: Fix search tagging races
This fixes races in thread-local and global tagging in notmuch-search
(e.g., "+", "-", "a", "*", etc.).  Previously, these would modify tags
of new messages that arrived after the search.  Now they only operate
on the messages that were in the threads when the search was
performed.  This prevents surprises like archiving messages that
arrived in a thread after the search results were shown.

This eliminates `notmuch-search-find-thread-id-region(-search)'
because these functions strongly encouraged racy usage.

This fixes the two broken tests added by the previous patch.
2013-11-08 20:52:00 -04:00
Austin Clements
d622cb942a Add TODO about more efficient stable thread queries 2013-11-08 20:46:26 -04:00
Austin Clements
abeac48522 search: Add stable queries to thread search results
These queries will match exactly the set of messages currently in the
thread, even if more messages later arrive.  Two queries are provided:
one for matched messages and one for unmatched messages.

This can be used to fix race conditions with tagging threads from
search results.  While tagging based on a thread: query can affect
messages that arrived after the search, tagging based on stable
queries affects only the messages the user was shown in the search UI.

Since we want clients to be able to depend on the presence of these
queries, this ushers in schema version 2.
2013-11-08 20:43:29 -04:00
Austin Clements
ec630c6635 schemata: Disambiguate non-terminal names
Previously, the show schema and the search schema used different
"thread" non-terminals.  While these schemata don't interact, this is
still confusing, so rename search's "thread" to "thread_summary".  To
further limit confusion, prefix all top-level search non-terminals now
begin with "search_".
2013-11-08 20:13:47 -04:00
Jani Nikula
195aaa6232 TODO: keybindings for next/previous thread done 2013-05-31 21:51:54 -03:00
David Bremner
71e1522da4 devel: add dkg's printmimestructure script to notmuch devel scripts
I find this script pretty useful when figuring out who to blame for
MIME rendering problems.

The notmuch repo will be the new primary home for this script, unless
and until a better home turns up.
2013-05-23 08:01:37 -03:00
Jani Nikula
566e8f1eba TODO: remove some completed todo items from the list
Bash completion has been rewritten. Configuration file saves follow
symlinks. There is --config=FILE top level option to specify
configuration file.
2013-04-14 19:46:17 -03:00
Jani Nikula
02cb7cf152 nmbug-status: only import notmuch when needed
Make it possible to use the script to query search views without
notmuch python bindings installed.
2013-04-06 08:36:34 -03:00
Jani Nikula
73fcfa861c nmbug-status: add support for querying the search views
Make it easy for scripts to read the views and corresponding searches.
2013-04-06 08:36:26 -03:00
Tomi Ollila
a629b2e1cb devel: add post-release tools news2wiki.pl and man-to-mdwn.pl
After new notmuch release has been published the NEWS and manual
pages have been updated using these 2 programs.

Adding the tools to notmuch repository eases their use, adds more
transparency to the "process" and gives more people chance to
do the updates is one is unavailable to do it at the time being.
2013-03-29 09:23:19 -04:00
Tomi Ollila
5872cba1eb devel/STYLE: information how to enable standard pre-commit hook
It is easier to enable git standard pre-commit hook, when the
operation to do so is presented.
2013-03-29 09:21:44 -04:00
David Bremner
16bf7b4b89 nmbug: allow empty prefix
Current code does not distinguish between an empty string in the
NMBPREFIX environment variable and the variable being undefined. This
makes it impossible to define an empty prefix, if, e.g. somebody wants
to dump all of their tags with nmbug.
2013-03-02 10:42:45 -04:00
David Bremner
f7a7f78aeb nmbug: replace hard-coded magic hash with git-hash-object
This is at least easier to understand than the magic hash. It may also
be a bit more robust, although it is hard to imagine these numbers
changing without many other changes in git.
2013-03-02 10:42:27 -04:00
David Bremner
17b468ac9f nmbug: use 'notmuch tag --batch'
This should be more robust with respect to tags with whitespace and
and other special characters. It also (hopefully) fixes a remaining
bug handling message-ids with whitespace.  It should also be
noticeably faster for large sets of changes since it does one exec per
change set as opposed to one exec per tag changed.
2013-03-02 10:42:16 -04:00
David Bremner
2ec4ea610c nmbug: use dump --format=batch-tag
This should make nmbug tolerate tags with whitespace and other special
characters it.  At the moment this relies on _not_ passing calls to
notmuch tag through the shell, which is a documented feature of perl's
system function.
2013-03-02 10:42:06 -04:00
Tomi Ollila
2bedb03005 devel/release-checks.sh: added check that 1st NEWS header is tidy
Check that the underlining '===...' for first (header) line in NEWS
file is of the same length as the header text and it is all '=':s.

-- extra execs removed by db.
2013-02-18 20:28:49 -04:00
David Bremner
41a29a8472 nmbug: move from contrib to devel
There seems to be consensus to use presence in contrib as
documentation of limited support by the notmuch developers; in fact
nmbug is pretty integrated into our current development process, so
devel seems more appropriate.
2013-02-16 07:54:33 -04:00
Jani Nikula
aa48b23f38 TODO: update entry on date range queries 2013-01-24 07:32:16 -04:00
Jani Nikula
b12b81bbb5 TODO: cleanup stuff that has been done
M-RET notmuch-show-open-or-close-all opens all closed messages.

The archiving change is mentioned twice, remove dupe.

"notmuch search" supports --format=text0 to work with xargs -0
2013-01-24 07:32:16 -04:00
David Bremner
a76682a2d2 CLI: remove alias machinery, and "part", "search-tags" commands
The commands are long deprecated, so removal is probably overdue. The
real motivation is to simplify argument handling for notmuch so that
we can migrate to the common argument parsing framework.
2013-01-22 21:19:08 -04:00
Tomi Ollila
bc89dd7688 devel/release-checks.sh: version string problem does not halt execution
Version string has strict format requirements in release-check.sh:
only numbers and periods (in sane order) are accepted.
Mismatch there used to halt further execution.
In this case, checking versions like '*~rc1' for (more) problems
was not possible.
This 'fatal error' is now changed buffered error message like in
following tests, and is displayed at the end of execution.
2013-01-19 14:29:53 -04:00
Peter Wang
b96ba6326a show: indicate length, encoding of omitted body content
If a leaf part's body content is omitted, return the encoded length and
transfer encoding in --format=json output.  This information may be used
by the consumer, e.g. to decide whether to download a large attachment
over a slow link.

Returning the _encoded_ content length is more efficient than returning
the _decoded_ content length.  Returning the transfer encoding allows
the consumer to estimate the decoded content length.
2012-12-17 09:10:35 -04:00
Austin Clements
1c6195b9e3 cli: Framework for structured output versioning
Currently there is a period of pain whenever we make
backward-incompatible changes to the structured output format, which
discourages not only backward-incompatible improvements to the format,
but also backwards-compatible additions that may not be "perfect".  In
the end, these problems limit experimentation and innovation.

This series of patches introduces a way for CLI callers to request a
specific format version on the command line and to determine if the
CLI does not supported the requested version (and perhaps present a
useful diagnostic to the user).  Since the caller requests a format
version, it's also possible for the CLI to support multiple
incompatible versions simultaneously, unlike the alternate approach of
including version information in the output.

This patch lays the groundwork by introducing a versioning convention,
standard exit codes, and a utility function to check the requested
version and produce standardized diagnostic messages and exit
statuses.
2012-12-16 17:20:33 -04:00
Peter Feigl
102f58d7a4 Use the S-Expression structured printer in notmuch-show, notmuch-reply and notmuch-search.
This patch uses the new S-Expression printer in the notmuch CLI (show,
search and reply). You can now use --format=sexp for any of them.
2012-12-08 09:30:19 -04:00
Peter Feigl
1bf3720b6e Rename the -json printer functions in notmuch-reply and notmuch-show to generic -sprinter functions.
All the structured output functions in notmuch-reply and notmuch-show
are renamed to a generic name (as they do not contain any json-specific
code anyway). This patch is a preparation to actually using the new
S-Expression sprinter in notmuch-reply and notmuch-show.
2012-12-08 09:28:19 -04:00
Austin Clements
ae459a3f4c devel: Document schemata in STYLE 2012-11-08 09:49:54 -04:00
Austin Clements
70dacc1fd9 devel: Add Reply-to to the schemata
The code got out of sync with the documentation in 7d3c06dc.
2012-11-08 09:49:22 -04:00