Commit graph

140 commits

Author SHA1 Message Date
W. Trevor King
bed8b6743d nmbug-status: Clarify errors for illegible configs
Carl Worth pointed out that errors like:

  $ ./nmbug-status
  fatal: Not a git repository: '/home/cworth/.nmbug'
  fatal: Not a git repository: '/home/cworth/.nmbug'
  Traceback (most recent call last):
    File "./nmbug-status", line 254, in <module>
      config = read_config(path=args.config)
    File "./nmbug-status", line 73, in read_config
      return json.load(fp)
    File "/usr/lib/python2.7/json/__init__.py", line 290, in load
      **kw)
    File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
      return _default_decoder.decode(s)
    File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
      obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
      raise ValueError("No JSON object could be decoded")
  ValueError: No JSON object could be decoded

are not particularly clear.  With this commit, we'll get output like:

  $ ./nmbug-status
  fatal: Not a git repository: '/home/wking/.nmbug'
  No local branch 'config' in /home/wking/.nmbug.  Checkout a local
  config branch or explicitly set --config.

which is much more accessible.  I've also added user-friendly messages
for a number of other config-parsing errors.
2015-03-15 09:28:15 +01:00
Jani Nikula
6d44e5ac47 lib: make notmuch_query_count_messages explicitely exact
The default is actually exact if no checkatleast parameter is
specified. This change makes that explicit, mainly for documentation,
but also to be safe in the unlikely event of a change of default.

[ commit message rewritten by db based on id:87lho0nlkk.fsf@nikula.org
]
2015-03-13 07:58:55 +01:00
W. Trevor King
9d25c97d8b nmbug: Add a 'help' command for folks who don't like --help
The 'if args.func == help' block at the end avoids:

    AttributeError: 'functools.partial' object has no attribute '__code__'
2015-01-18 11:00:51 +01:00
David Bremner
2bb906a6dd build: eliminate use of python execfile command
As discussed in
id:8cc9dd580ad672527e12f43706f9803b2c8e99d8.1405220724.git.wking@tremily.us,
execfile is unavailable in python3.

The approach of this commit avoids modifying the python module path,
which is arguably preferable since it avoids potentially accidentally
importing a module from the wrong place.
2015-01-03 15:18:54 +01:00
Tomi Ollila
34d2c62b82 devel: man-to-mdwn.pl: add GPLv3+ notice to manpages.mdwn
Make the generated head manual page (in the notmuchmail wiki)
display the licence information the whole notmuch software is licenced
under.
2014-11-02 19:46:17 +01:00
Tomi Ollila
f7be8f4ebf devel: make man-to-mdwn.pl to work with generated manual pages
The new manual pages converted from rst using sphinx or rst2man
has somewhat different syntax. man-to-mdwn.pl is now adjusted
to produce even better output from this syntax. The changes also
include using utf-8 locale (e.g. for tables and generated hypens)
and and quite a few bugs fixes.
This tool still produces better results than just using the
html pages generated using sphinx / rst2html. For example those
tools don't create inter-page hyperlinks -- and the preformatted
pages written by man-to-mdwn.pl just works well with manual page
content.
2014-11-02 19:46:06 +01:00
W. Trevor King
7f2cb3be4e nmbug: Translate to Python
This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages.  It should be compatible with Python 2.7 and later
(including the 3.x series).

Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:

* 'nmbug help' has been split between the general 'nmbug --help' and
  the command-specific 'nmbug COMMAND --help'.

* Commands are no longer split into "most common", "other useful", and
  "less common" sets.  If we need something like this, I'd prefer
  workflow examples highlighting common commands in the module
  docstring (available with 'nmbug --help').

* 'nmbug commit' now only uses a single argument for the optional
  commit-message text.  I wanted to expose more of the underlying 'git
  commit' UI, since I personally like to write my commit messages in
  an editor with the notes added by 'git commit -v' to jog my memory.
  Unfortunately, we're using 'git commit-tree' instead of 'git
  commit', and commit-tree is too low-level for editor-launching.  I'd
  be interested in rewriting commit() to use 'git commit', but that
  seemed like it was outside the scope of this rewrite.  So I'm not
  supporting all of Git's commit syntax in this patch, but I can at
  least match 'git commit -m MESSAGE' in requiring command-line commit
  messages to be a single argument.

* The default repository for 'nmbug push' and 'nmbug fetch' is now the
  current branch's upstream (branch.<name>.remote) instead of
  'origin'.  When we have to, we extract this remote by hand, but
  where possible we just call the Git command without a repository
  argument, and leave it to Git to figure out the default.

* 'nmbug push' accepts multiple refspecs if you want to explicitly
  specify what to push.  Otherwise, the refspec(s) pushed depend on
  push.default.  The Perl version hardcoded 'master' as the pushed
  refspec.

* 'nmbug pull' defaults to the current branch's upstream
  (branch.<name>.remote and branch.<name>.merge) instead of hardcoding
  'origin' and 'master'.  It also supports multiple refspecs if for
  some crazy reason you need an octopus merge (but mostly to avoid
  breaking consistency with 'git pull').

* 'nmbug log' now execs 'git log', as there's no need to keep the
  Python process around once we've launched Git there.

* 'nmbug status' now catches stderr, and doesn't print errors like:

    No upstream configured for branch 'master'

  The Perl implementation had just learned to avoid crashing on that
  case, but wasn't yet catching the dying subprocess's stderr.

* 'nmbug archive' now accepts positional arguments for the tree-ish
  and additional 'git archive' options.  For example, you can run:

    $ nmbug archive HEAD -- --format tar.gz

  I wish I could have preserved the argument order from 'git archive'
  (with the tree-ish at the end), but I'm not sure how to make
  argparse accept arbitrary possitional arguments (some of which take
  arguments).  Flipping the order to put the tree-ish first seemed
  easiest.

* 'nmbug merge' and 'pull' no longer checkout HEAD before running
  their command, because blindly clobbering the index seems overly
  risky.

* In order to avoid creating a dirty index, 'nmbug commit' now uses
  the default index (instead of nmbug.index) for composing the commit.
  That way the index matches the committed tree.  To avoid leaving a
  broken index after a failed commit, I've wrapped the whole thing in
  a try/except block that resets the index to match the pre-commit
  treeish on errors.  That means that 'nmbug commit' will ignore
  anything you've cached in the index via direct Git calls, and you'll
  either end up with an index matching your notmuch tags and the new
  HEAD (after a successful commit) or an index matching the original
  HEAD (after a failed commit).
2014-10-05 07:16:29 +02:00
W. Trevor King
567aab405b nmbug: Handle missing @upstream in is_unmerged
If we don't have an upstream, there is nothing to merge, so nothing is
unmerged.  This avoids errors like:

  $ nmbug status
  error: No upstream configured for branch 'master'
  error: No upstream configured for branch 'master'
  fatal: ambiguous argument '@{upstream}': unknown revision or path not in the working tree.
  Use '--' to separate paths from revisions, like this:
  'git <command> [<revision>...] -- [<file>...]'
  'git rev-parse @{upstream}' exited with nonzero value

You might not have an upstream if you're only using nmbug locally to
version-control your tags.
2014-07-16 19:31:28 -03:00
W. Trevor King
4697e86a52 nmbug: Add a git_with_status helper function
Sometimes we want to catch Git errors and handle them, instead of
dying with an error message.  This lower-level version of git() allows
us to get the error status when we want it.
2014-07-16 06:31:23 -03:00
W. Trevor King
7f2bbe93a5 nmbug-status: Optionally load the header and footer templates from the config
For folks that don't like the default templates for whatever reason.
2014-07-15 20:15:23 -03:00
W. Trevor King
f10024f30d nmbug-status: Add the time to the footer's build-date
Our repository [1] has a post-update hook that rebuilds the status
page after each push.  Since that may happen several times a day, we
might as well show the build time (as well as the date) in the footer.

The trailing 'Z' is the ISO 8601 designator for UTC.  Now that we're
showing times, it's nice to be explicit about the timezone we're
using.

The rename from date -> datetime gives us backward-compatibility for
folks that *do* only want the date.  We keep the old date formatting
to support those folks.

[1]: http://nmbug.tethera.net/git/nmbug-tags.git
2014-07-15 20:08:39 -03:00
W. Trevor King
7832258468 nmbug-status: Factor out header/footer context into a shared dict
Rather than splitting this context into header-only and footer-only
groups, just dump it all in a shared dict.  This will make it easier
to eventually split the header/footer templates out of this script
(e.g. if we want to load them from the config file).
2014-07-15 20:08:24 -03:00
Carl Worth
b70386a4cb Move the generated date from the top of the page to the footer.
It's useful reference information, but anyone who wants it will look
for and find it.  We don't need this front-and-center.  Follow the
pattern set by our header template with a triple-quoted string.

The gray <hr> styling is less agressive.  IE uses 'color' for drawing
the rule, while Gecko and Opera use the border or 'background-color'
[1].

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=239386
2014-07-15 20:07:18 -03:00
Carl Worth
b5bdf5f4a8 Add a docstring describing the program
Prefer a docstring to a header comment so we can use it as the
ArgumentParser description (formatted with 'nmbug-status --help').
Script readers still have it near the top of the file.  Since it's a
docstring, use PEP 257's summary-line-and-body format [1].

[1]: http://legacy.python.org/dev/peps/pep-0257/#multi-line-docstrings
2014-07-15 20:07:06 -03:00
Carl Worth
d1d84cab1d Add explicit license information
It's nice to have for situations where this script is found outside of
the notmuch source repository (e.g. after being installed).
2014-07-15 20:06:59 -03:00
Tomi Ollila
24b24be151 devel: news2wiki to recognize yyyy-mm-dd or UNRELEASED as release date
The yyyy-mm-dd (actually \d\d\d\d-\d\d-\d\d) for a bit more restrictive
(and self-documentative) than the \w\w\w\w-... that used to be there and
UNRELEASED so that developers can test the latest NEWS converted to mdwn
format before submitting NEWS patches.
2014-07-13 12:35:37 -03:00
David Bremner
dd24fdd33a nmbug: mark repository as bare on clone
If a git repository is non-bare, and core.worktree is not set, git
tries to deduce the worktree. This deduction is not always helpful, e.g.

% git --git-dir=$HOME/.nmbug clean -f

would likely delete most of the files in the current directory
2014-04-15 17:20:19 -03:00
W. Trevor King
c200167426 nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}
With two branches getting fetched (master and config), the branch
referenced by FETCH_HEAD is ambiguous.  For example, I have:

  $ cat FETCH_HEAD
  41d7bfa7184cc93c9dac139d1674e9530799e3b0 \
    not-for-merge   branch 'config' of http://nmbug.tethera.net/git/nmbug-tags
  acd379ccb973c45713eee9db177efc530f921954 \
    not-for-merge   branch 'master' of http://nmbug.tethera.net/git/nmbug-tags

(where I wrapped the line by hand).  This means that FETCH_HEAD
references the config branch:

  $ git rev-parse FETCH_HEAD
  41d7bfa7184cc93c9dac139d1674e9530799e3b0

which breaks all of the FETCH_HEAD logic in nmbug (where FETCH_HEAD is
assumed to point to the master branch).

Instead of relying on FETCH_HEAD, use @{upstream} as the
remote-tracking branch that should be merged/diffed/integrated into
HEAD.  @{upstream} was added in Git v1.7.0 (2010-02-12) [1], so
relying on it should be fairly safe.  One tricky bit is that bare
repositories don't set upstream tracking branches by default:

  $ git clone --bare http://nmbug.tethera.net/git/nmbug-tags.git nmbug-bare
  $ cd nmbug-bare
  $ git remote show origin
  * remote origin
    Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
    Push  URL: http://nmbug.tethera.net/git/nmbug-tags.git
    HEAD branch: master
    Local refs configured for 'git push':
      config pushes to config (up to date)
      master pushes to master (up to date)

While in a non-bare clone:

  $ git clone http://nmbug.tethera.net/git/nmbug-tags.git
  $ cd nmbug-tags
  $ git remote show origin
  * remote origin
    Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
    Push  URL: http://nmbug.tethera.net/git/nmbug-tags.git
    HEAD branch: master
    Remote branches:
      config tracked
      master tracked
    Local branch configured for 'git pull':
      master merges with remote master
    Local ref configured for 'git push':
      master pushes to master (up to date)

From the clone docs [2]:

  --bare::
        Make a 'bare' Git repository…
        Also the branch heads at the remote are copied directly
        to corresponding local branch heads, without mapping
        them to `refs/remotes/origin/`.  When this option is
        used, neither remote-tracking branches nor the related
        configuration variables are created.

To use @{upstream}, we need to the local vs. remote-tracking
distinction, so this commit adds 'nmbug clone', replacing the
previously suggested --bare clone with a non-bare --no-checkout
--separate-git-dir clone into a temporary work directory.  After
which:

  $ git rev-parse @{upstream}
  acd379ccb973c45713eee9db177efc530f921954

gives us the master-branch commit.  Existing nmbug users will have to
run the configuration tweaks and re-fetch by hand.  If you don't have
any local commits, you could also blow away your NMBGIT repository and
re-clone from scratch:

  $ nmbug clone http://nmbug.tethera.net/git/nmbug-tags.git

Besides removing the ambiguity of FETCH_HEAD, this commit allows users
to configure which upstream branch they want nmbug to track via 'git
config', in case they want to change their upstream repository.

[1]: http://git.kernel.org/cgit/git/git.git/tree/Documentation/RelNotes/1.7.0.txt
[2]: http://git.kernel.org/cgit/git/git.git/tree/Documentation/git-clone.txt
2014-04-08 07:39:28 -03:00
Tomi Ollila
5608e39a6b release-checks: removed manual page version check
Manual pages are now generated and during the generation the version
string is read from `version` file, so this (currently failing) test
checking manual page versions can be removed.

While at it, changed the case pattern *[^0-9.]*
to its portable alternative *[!0-9.]*
2014-04-08 07:32:34 -03:00
David Bremner
b6751cf875 doc: move doxgen config from devel/ to doc/
a first step towards actually instally the API docs
2014-04-08 07:30:42 -03:00
Jani Nikula
87c2cd78fd nmbug-status: make output title and blurb configurable
Make nmbug-status more generally usable outside of nmbug by not
hardcoding notmuch related things.

This lets anyone publish html search views to mailing list messages
with a custom config file, independent of nmbug.
2014-03-23 08:33:34 -03:00
Jani Nikula
81a1aae2dc nmbug-status: parameterize title and blurb in the page header
Prepare for more general use.
2014-03-23 08:32:40 -03:00
Jani Nikula
6af2b05612 devel: add script to generate test databases
Add script to generate notmuch test databases using specified versions
of notmuch. This is useful for generating material for database
upgrade tests.

This reuses the test infrastructure to have a sandbox environment for
notmuch new etc.
2014-03-11 19:51:22 -03:00
Tomi Ollila
18d71908b2 nmbug-status: replace __values__() with values() in OrderedDict stub
Python dict() object does not have __values__() function which
OrderedDict().values() (the stub provided in nmbug-status) could call
to provide ordered list of values. By renaming this thinko to
values() will make our stub work as expected -- dict items listed out
in order those were added to the dictionary.
2014-02-21 21:12:45 -04:00
W. Trevor King
320d4a856e nmbug-status: Hardcode UTF-8 instead of using the user's locale
David [1] and Tomi [2] both feel that the user's choice of LANG is not
explicit enough to have such a strong effect on nmbug-status.  For
example, cron jobs usually default to LANG=C, and that is going to
give you ASCII output:

  $ LANG=C python -c 'import locale; print(locale.getpreferredencoding())'
  ANSI_X3.4-1968

Trying to print Unicode author names (and other strings) in that
encoding would crash nmbug-status with a UnicodeEncodeError.  To avoid
that, this patch hardcodes UTF-8, which can handle generic Unicode,
and is the preferred encoding (regardless of LANG settings) for
everyone who has chimed in on the list so far.  I'd prefer trusting
LANG, but in the absence of any users that prefer non-UTF-8 encodings
I'm fine with this approach.

While we could achieve the same effect on the output content by
dropping the previous patch (nmbug-status: Encode output using the
user's locale), Tomi also wanted UTF-8 hardcoded as the config-file
encoding [2].  Keeping the output encoding patch and then adding this
to hardcode both the config-file and output encodings at once seems
the easiest route, now that fd29d3f (nmbug-status: Decode Popen output
using the user's locale, 2014-02-10) has landed in master.

[1]: id="877g8z4v4x.fsf@zancas.localnet"
     http://article.gmane.org/gmane.mail.notmuch.general/17202
[2]: id="m2vbwj79lu.fsf@guru.guru-group.fi"
     http://article.gmane.org/gmane.mail.notmuch.general/17209
2014-02-14 21:45:07 -04:00
W. Trevor King
ffed8f2866 nmbug-status: Encode output using the user's locale
Instead of always writing UTF-8, allow the user to configure the
output encoding using their locale.  This is useful for previewing
output in the terminal, for poor souls that don't use UTF-8 locales
;).
2014-02-14 21:44:11 -04:00
W. Trevor King
0d563dae1d nmbug-status: Add inter-message padding
We already had the tbody with a blank row separating threads (which is
not colored); this commit adds a bit of spacing to separate messages
within a thread.  It will also add a bit of colored padding above the
first message and below the final message, but the main goal is to add
padding *between* two-row message blocks.

                                               <--- new padding
  thread-1, message-1, row-1  (class="message-first")
  thread-1, message-1, row-2  (class="message-last")
                                               <--- new padding
    spacer tbody with a blank row
                                               <--- new padding
  thread-2, message-1, row-1  (class="message-first")
  thread-2, message-1, row-2  (class="message-last")
                                               <--- new padding
                                               <--- new padding
  thread-2, message-2, row-1  (class="message-first")
  thread-2, message-2, row-2  (class="message-last")
                                               <--- new padding
2014-02-14 21:42:36 -04:00
W. Trevor King
1aa56c2ac5 nmbug-status: Color threads in HTML output
Add tbody sections so we don't have to color every row.  Multiple
tbody sections are allowed [1].  Use CSS 3's nth-child to handle
even/odd coloring (skipping the spacer rows) [2], which is supported
on the major browsers [3].

border-spacing is from CCS 2.1 [4,5].  I'm using it to avoid
whitespace between td cells.

border-radius is from CCS 3 [6,7].  I'm using it to make the colored
sections a bit less harsh.  I tried adding rounded borders to the
tbody itself doesn't work, but I couldn't get that to work without
setting the tbody's display to 'block'.  That rounded the corners, but
collapsed the cell spacing (e.g. columns were no longer aligned).
This commit's by-corner-td approach is not particularly elegant, but
it works.  The td padding entries just ensure that the cell body is
suitably far from the edges that it doesn't fall outside of the
rounded corners.

The doubled-braces are escapes from Python's str.format.

[1]: http://www.w3.org/TR/html5/tabular-data.html#the-table-element
[2]: http://www.w3.org/TR/css3-selectors/#nth-child-pseudo
[3]: https://developer.mozilla.org/en-US/docs/Web/CSS/:nth-child#Browser_compatibility
[4]: http://www.w3.org/TR/CSS2/tables.html#propdef-border-spacing
[5]: https://developer.mozilla.org/en-US/docs/Web/CSS/border-spacing#Browser_compatibility
[6]: http://www.w3.org/TR/css3-background/#the-border-radius
[7]: https://developer.mozilla.org/en-US/docs/Web/CSS/border-radius#Browser_compatibility
2014-02-14 21:40:20 -04:00
W. Trevor King
aaa7f0d92e nmbug-status: Escape &, <, and > in HTML display data
'message-id' and 'from' now have sensitive characters escaped using
xml.sax.saxutils.escape [1].  The 'subject' data was already being
converted to a link into Gmane; I've escape()d that too, so it doesn't
need to be handled ain the same block as 'message-id' and 'from'.

This prevents broken HTML by if subjects etc. contain characters that
would otherwise be interpreted as HTML markup.

[1]: http://docs.python.org/3/library/xml.sax.utils.html#xml.sax.saxutils.escape
2014-02-14 08:29:33 -04:00
W. Trevor King
aa32d2579b nmbug-status: Use <code> and <p> markup where appropriate
* Wrap free text in <p> tags.
* Convert <blockquote> to <p><code> for query strings.
* Wrap message-id-term (id:"...") in <code>.

The <code> tags get nicer default markup (smaller monospace font) for
notmuch query terms [1].  The <p> tags don't have much effect without
attached CSS, but bare text (phrasing content [2]) in <body> (which
expects flow content [3,4]) feels wrong.

[1]: http://www.w3.org/TR/html5/text-level-semantics.html#the-code-element
[2]: http://www.w3.org/TR/html5/dom.html#phrasing-content-1
[3]: http://www.w3.org/TR/html5/dom.html#flow-content-1
[4]: http://www.w3.org/TR/html5/sections.html#the-body-element
2014-02-13 21:47:37 -04:00
W. Trevor King
711d04c821 nmbug-status: Slug the title when using it as an id
Also allow manual id overrides from the JSON config.  Sluggin avoids
errors like:

   Bad value '#Possible bugs' for attribute href on element a:
   Whitespace in fragment component. Use %20 in place of spaces.

from http://validator.w3.org.

I tried just quoting the titles (e.g. 'Possible%20bugs'), but that
didn't work (at least with Firefox 24.2.0).  Slugging avoids any
ambiguity over when the quotes are expanded in the client.  The specs
are unclear about quoting, saying only [1]:

  Value: Any string, with the following restrictions:

    must be at least one character long
    must not contain any space characters

[1]: http://dev.w3.org/html5/markup/global-attributes.html#common.attrs.id
2014-02-13 21:45:40 -04:00
W. Trevor King
e485b5bd1d nmbug-status: Anchor with h3 ids instead of a names
HTML 5 allows id attributes on all HTML elements [1], but restricts
names to particular cases [2].  Attaching the id attribute to the h3
element allows us to drop the anchor a element altogether.

[1]: http://www.w3.org/TR/html5/dom.html#the-id-attribute
[2]: http://www.w3.org/TR/html5/index.html#attributes-1
2014-02-13 21:44:13 -04:00
W. Trevor King
9b98c5f8e9 nmbug-status: Convert from XHTML 1.0 to HTML 5
HTML 5 for the win :).  I also de-namespaced the language; the HTML 5
spec allows a vestigial xml:lang attribute, but it's a no-op [1], so I
stripped it.

This shouldn't break anything at tethera, which already serves the
status as text/html:

  $ wget -S http://nmbug.tethera.net/status/
  --2014-02-02 21:20:39--  http://nmbug.tethera.net/status/
  Resolving nmbug.tethera.net... 87.98.215.224
  Connecting to nmbug.tethera.net|87.98.215.224|:80... connected.
  HTTP request sent, awaiting response...
    HTTP/1.1 200 OK
    Vary: Accept-Encoding
    Content-Type: text/html
  ...

This also matches the Content-Type in the generated HTML's http-equiv
meta.

[1]: http://www.w3.org/TR/html5/dom.html#the-lang-and-xml:lang-attributes
2014-02-10 22:55:04 -04:00
W. Trevor King
a3a3725e43 nmbug-status: Normalize table HTML indentation
I don't think I've ever seen '</td><td>{value}\n' before :p.  The new
formatting avoids mixing tag levels and content across lines.
2014-02-10 22:53:28 -04:00
W. Trevor King
c48a12365f nmbug-status: Add an OrderedDict stub for Python 2.6
Tomi Ollila and David Bremner (and presumably others) are running
Python 2.6 on their nmbug-status boxes, so it makes sense to keep
support for that version.  This commit adds a really minimal
OrderedDict stub (e.g. it doesn't handle key removal), but it gets the
job done for Page._get_threads.  Once we reach a point where Python
2.6 is no longer important (it's already out of it's security-fix
window [1]), we can pull this stub back out.

[1]: http://www.python.org/download/releases/2.6.9/
2014-02-10 22:51:40 -04:00
W. Trevor King
98cb4779c0 nmbug-status: Add Page and HtmlPage for modular rendering
I was having trouble understanding the logic of the longish print_view
function, so I refactored the output generation into modular bits.
The basic text rendering is handled by Page, which has enough hooks
that HtmlPage can borrow the logic and slot-in HTML generators.

By modularizing the logic it should also be easier to build other
renderers if folks want to customize the layout for other projects.

Timezones
=========

This commit has not effect on the output, except that some dates have
been converted from the sender's timezone to UTC due to:

  -            val = m.get_header(header)
  -            ...
  -            if header == 'date':
  -                val = str.join(' ', val.split(None)[1:4])
  -                val = str(datetime.datetime.strptime(val, '%d %b %Y').date())
  ...
  +                value = str(datetime.datetime.utcfromtimestamp(
  +                    message.get_date()).date())

I also tweaked the HTML header date to be utcnow instead of the local
now() to make all times independent of the generator's local time.
This matches Gmane, which converts all Date headers to UTC (although
they use a 'GMT' suffix).  Notmuch uses
g_mime_utils_header_decode_date to calculate the UTC timestamps, but
uses a NULL tz_offset which drops the information we'd need to get
back to the sender's local time [1].  With the generator's local time
arbitrarily different from the sender's and viewer's local time,
sticking with UTC seems the best bet.

[1]: https://developer.gnome.org/gmime/stable/gmime-gmime-utils.html#g-mime-utils-header-decode-date
2014-02-10 22:51:03 -04:00
W. Trevor King
7b7a83cc32 nmbug-status: Add a Python-3-compatible urllib.parse.quote import
Python 2's urllib.quote [1] has moved to urllib.parse.quote in Python
3 [2].

[1]: http://docs.python.org/2/library/urllib.html#urllib.quote
[2]: http://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote
2014-02-10 22:49:26 -04:00
W. Trevor King
a6ff03df3f nmbug-status: Consolidate HTML header printing
Make this all one big string, using '...{date}...'.format(date=...) to
inject the date [1].  This syntax was added in Python 2.6, and is
preferred to %-formatting in Python 3 [1].

[1]: http://docs.python.org/2/library/stdtypes.html#str.format
2014-02-10 22:47:52 -04:00
W. Trevor King
a7e4d9a18f nmbug-status: Don't require write access
The database in only used for notmuch.Query, so there's no need for
write access.  This allows nmbug-status to run while the database is
being updated, without raising:

  A Xapian exception occurred opening database: Unable to get write lock on …: already locked
  Traceback (most recent call last):
    File "./nmbug-status", line 182, in <module>
      db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
    File "/…/notmuch/database.py", line 154, in __init__
      self.open(path, mode)
    File "/…/notmuch/database.py", line 214, in open
      raise NotmuchError(status)
  notmuch.errors.XapianError
2014-02-10 22:46:37 -04:00
W. Trevor King
e4d79bfddb nmbug-status: Consolidate functions and main code
The definitions of Thread, output_with_separator, and print_view were
between the main argparse and view-printing code.  Group them together
with our existing read_config at the top of the module, which makes
for easier reading in the main section.

I also:

* Made 'headers' a print_view argument instead of a module-level
  global.  The list -> tuple conversion avoids having a mutable
  default argument, which makes some people jumpy ;).

* Made 'db' a print_view argument instead of relying on the global
  namespace to access it from print_view.
2014-02-10 22:45:28 -04:00
W. Trevor King
a2b64211b2 nmbug-status: Add metavars for --config and --get-query
Now the suggested usage (listed by 'nmbug-status --help') is:

  usage: nmbug-status [-h] [--text] [--config PATH] [--list-views]
                      [--get-query VIEW]

instead of the less obvious:

  usage: nmbug-status [-h] [--text] [--config CONFIG] [--list-views]
                      [--get-query GET_QUERY]
2014-02-10 21:13:37 -04:00
W. Trevor King
b7e6d2cc30 nmbug-status: Factor config-loading out into read_config
By isolating this peripheral handling, we make the core logic of
nmbug-status easier to read.
2014-02-10 21:09:16 -04:00
W. Trevor King
fd29d3f4fb nmbug-status: Decode Popen output using the user's locale
Avoid:

  $ ./nmbug-status --list-views
  Traceback (most recent call last):
    File "./nmbug-status", line 47, in <module>
      'cat-file', 'blob', sha1+':status-config.json'],
  TypeError: can't concat bytes to str

by explicitly converting the byte-stream read from Popen into a
Unicode string.  On Python 2, this conversion is str -> unicode; on
Python 3 it is bytes -> str.

_ENCODING is derived from the user's locale (or system default) in an
attempt to match Git's output encoding.  It may be more robust to skip
the encoding/decoding by using a Python wrapper like pygit2 [1] for
Git access.  That's a fairly heavy dependency though, and using the
locale will probably work.

[1]: http://www.pygit2.org/
2014-02-10 21:04:55 -04:00
W. Trevor King
ee6b5c372f nmbug-status: Use email.utils instead of rfc822
rfc822 has been deprecated since Python 2.3, and it's gone in
Python 3 [1].

[1]: http://docs.python.org/2/library/rfc822.html
2014-02-10 21:03:12 -04:00
W. Trevor King
a8b81adc8e nmbug-status: Convert to Python-3-compatible print functions
We shouldn't require folks to install Python 2 to run nmbug-status.
2014-02-10 20:52:23 -04:00
Jani Nikula
f230fd167a devel: add doxygen configuration file
This is a pretty basic config to get started, generated using 'doxygen
-s -g' and mildly tweaked.

To generate the library man page man/man3/notmuch.3 from lib/notmuch.h
use:

$ doxygen devel/doxygen.cfg
2014-01-05 08:59:52 -04:00
Tomi Ollila
0c097d9f6e devel/release-checks.sh: adjust to LIBNOTMUCH version checks
NOTMUCH_VERSION_* macros in lib/notmuch.h are replaced with
LIBNOTMUCH_VERSION_* macros. Check that the values of those
match the LIBNOTMUCH_*_VERSION values in lib/Makefile.local.
2013-12-30 20:39:25 -04:00
Tomi Ollila
001d932816 devel/release-checks.sh: check NOTMUCH_(MAJOR|MINOR|MICRO)_VERSION
New defines NOTMUCH_MAJOR_VERSION, NOTMUCH_MINOR_VERSION and
NOTMUCH_MICRO_VERSION were added to lib/notmuch.h.
Check that these match the current value defined in ./version.
2013-11-11 07:44:44 -04:00
Austin Clements
96c0ce28f8 emacs: Fix search tagging races
This fixes races in thread-local and global tagging in notmuch-search
(e.g., "+", "-", "a", "*", etc.).  Previously, these would modify tags
of new messages that arrived after the search.  Now they only operate
on the messages that were in the threads when the search was
performed.  This prevents surprises like archiving messages that
arrived in a thread after the search results were shown.

This eliminates `notmuch-search-find-thread-id-region(-search)'
because these functions strongly encouraged racy usage.

This fixes the two broken tests added by the previous patch.
2013-11-08 20:52:00 -04:00
Austin Clements
d622cb942a Add TODO about more efficient stable thread queries 2013-11-08 20:46:26 -04:00
Austin Clements
abeac48522 search: Add stable queries to thread search results
These queries will match exactly the set of messages currently in the
thread, even if more messages later arrive.  Two queries are provided:
one for matched messages and one for unmatched messages.

This can be used to fix race conditions with tagging threads from
search results.  While tagging based on a thread: query can affect
messages that arrived after the search, tagging based on stable
queries affects only the messages the user was shown in the search UI.

Since we want clients to be able to depend on the presence of these
queries, this ushers in schema version 2.
2013-11-08 20:43:29 -04:00
Austin Clements
ec630c6635 schemata: Disambiguate non-terminal names
Previously, the show schema and the search schema used different
"thread" non-terminals.  While these schemata don't interact, this is
still confusing, so rename search's "thread" to "thread_summary".  To
further limit confusion, prefix all top-level search non-terminals now
begin with "search_".
2013-11-08 20:13:47 -04:00
Jani Nikula
195aaa6232 TODO: keybindings for next/previous thread done 2013-05-31 21:51:54 -03:00
David Bremner
71e1522da4 devel: add dkg's printmimestructure script to notmuch devel scripts
I find this script pretty useful when figuring out who to blame for
MIME rendering problems.

The notmuch repo will be the new primary home for this script, unless
and until a better home turns up.
2013-05-23 08:01:37 -03:00
Jani Nikula
566e8f1eba TODO: remove some completed todo items from the list
Bash completion has been rewritten. Configuration file saves follow
symlinks. There is --config=FILE top level option to specify
configuration file.
2013-04-14 19:46:17 -03:00
Jani Nikula
02cb7cf152 nmbug-status: only import notmuch when needed
Make it possible to use the script to query search views without
notmuch python bindings installed.
2013-04-06 08:36:34 -03:00
Jani Nikula
73fcfa861c nmbug-status: add support for querying the search views
Make it easy for scripts to read the views and corresponding searches.
2013-04-06 08:36:26 -03:00
Tomi Ollila
a629b2e1cb devel: add post-release tools news2wiki.pl and man-to-mdwn.pl
After new notmuch release has been published the NEWS and manual
pages have been updated using these 2 programs.

Adding the tools to notmuch repository eases their use, adds more
transparency to the "process" and gives more people chance to
do the updates is one is unavailable to do it at the time being.
2013-03-29 09:23:19 -04:00
Tomi Ollila
5872cba1eb devel/STYLE: information how to enable standard pre-commit hook
It is easier to enable git standard pre-commit hook, when the
operation to do so is presented.
2013-03-29 09:21:44 -04:00
David Bremner
16bf7b4b89 nmbug: allow empty prefix
Current code does not distinguish between an empty string in the
NMBPREFIX environment variable and the variable being undefined. This
makes it impossible to define an empty prefix, if, e.g. somebody wants
to dump all of their tags with nmbug.
2013-03-02 10:42:45 -04:00
David Bremner
f7a7f78aeb nmbug: replace hard-coded magic hash with git-hash-object
This is at least easier to understand than the magic hash. It may also
be a bit more robust, although it is hard to imagine these numbers
changing without many other changes in git.
2013-03-02 10:42:27 -04:00
David Bremner
17b468ac9f nmbug: use 'notmuch tag --batch'
This should be more robust with respect to tags with whitespace and
and other special characters. It also (hopefully) fixes a remaining
bug handling message-ids with whitespace.  It should also be
noticeably faster for large sets of changes since it does one exec per
change set as opposed to one exec per tag changed.
2013-03-02 10:42:16 -04:00
David Bremner
2ec4ea610c nmbug: use dump --format=batch-tag
This should make nmbug tolerate tags with whitespace and other special
characters it.  At the moment this relies on _not_ passing calls to
notmuch tag through the shell, which is a documented feature of perl's
system function.
2013-03-02 10:42:06 -04:00
Tomi Ollila
2bedb03005 devel/release-checks.sh: added check that 1st NEWS header is tidy
Check that the underlining '===...' for first (header) line in NEWS
file is of the same length as the header text and it is all '=':s.

-- extra execs removed by db.
2013-02-18 20:28:49 -04:00
David Bremner
41a29a8472 nmbug: move from contrib to devel
There seems to be consensus to use presence in contrib as
documentation of limited support by the notmuch developers; in fact
nmbug is pretty integrated into our current development process, so
devel seems more appropriate.
2013-02-16 07:54:33 -04:00
Jani Nikula
aa48b23f38 TODO: update entry on date range queries 2013-01-24 07:32:16 -04:00
Jani Nikula
b12b81bbb5 TODO: cleanup stuff that has been done
M-RET notmuch-show-open-or-close-all opens all closed messages.

The archiving change is mentioned twice, remove dupe.

"notmuch search" supports --format=text0 to work with xargs -0
2013-01-24 07:32:16 -04:00
David Bremner
a76682a2d2 CLI: remove alias machinery, and "part", "search-tags" commands
The commands are long deprecated, so removal is probably overdue. The
real motivation is to simplify argument handling for notmuch so that
we can migrate to the common argument parsing framework.
2013-01-22 21:19:08 -04:00
Tomi Ollila
bc89dd7688 devel/release-checks.sh: version string problem does not halt execution
Version string has strict format requirements in release-check.sh:
only numbers and periods (in sane order) are accepted.
Mismatch there used to halt further execution.
In this case, checking versions like '*~rc1' for (more) problems
was not possible.
This 'fatal error' is now changed buffered error message like in
following tests, and is displayed at the end of execution.
2013-01-19 14:29:53 -04:00
Peter Wang
b96ba6326a show: indicate length, encoding of omitted body content
If a leaf part's body content is omitted, return the encoded length and
transfer encoding in --format=json output.  This information may be used
by the consumer, e.g. to decide whether to download a large attachment
over a slow link.

Returning the _encoded_ content length is more efficient than returning
the _decoded_ content length.  Returning the transfer encoding allows
the consumer to estimate the decoded content length.
2012-12-17 09:10:35 -04:00
Austin Clements
1c6195b9e3 cli: Framework for structured output versioning
Currently there is a period of pain whenever we make
backward-incompatible changes to the structured output format, which
discourages not only backward-incompatible improvements to the format,
but also backwards-compatible additions that may not be "perfect".  In
the end, these problems limit experimentation and innovation.

This series of patches introduces a way for CLI callers to request a
specific format version on the command line and to determine if the
CLI does not supported the requested version (and perhaps present a
useful diagnostic to the user).  Since the caller requests a format
version, it's also possible for the CLI to support multiple
incompatible versions simultaneously, unlike the alternate approach of
including version information in the output.

This patch lays the groundwork by introducing a versioning convention,
standard exit codes, and a utility function to check the requested
version and produce standardized diagnostic messages and exit
statuses.
2012-12-16 17:20:33 -04:00
Peter Feigl
102f58d7a4 Use the S-Expression structured printer in notmuch-show, notmuch-reply and notmuch-search.
This patch uses the new S-Expression printer in the notmuch CLI (show,
search and reply). You can now use --format=sexp for any of them.
2012-12-08 09:30:19 -04:00
Peter Feigl
1bf3720b6e Rename the -json printer functions in notmuch-reply and notmuch-show to generic -sprinter functions.
All the structured output functions in notmuch-reply and notmuch-show
are renamed to a generic name (as they do not contain any json-specific
code anyway). This patch is a preparation to actually using the new
S-Expression sprinter in notmuch-reply and notmuch-show.
2012-12-08 09:28:19 -04:00
Austin Clements
ae459a3f4c devel: Document schemata in STYLE 2012-11-08 09:49:54 -04:00
Austin Clements
70dacc1fd9 devel: Add Reply-to to the schemata
The code got out of sync with the documentation in 7d3c06dc.
2012-11-08 09:49:22 -04:00
Tomi Ollila
439fe710eb uncrustify.cfg: added 3 new types for uncrustify to know
Added FILE, notmuch_show_params_t and sprinter_t to be
types when uncrustifying sources. This affect spacing
when uncrustify is deciding for type declaration instead
of binary multiplication operation.
2012-11-07 08:00:03 -04:00
Tomi Ollila
a0ce84c3af devel: add release-checks.sh
Currently Makefile.local contains some machine executable release
checking functionality. This is unnecessarily complex way to do it:

Multiline script functionality is hard to embed -- from Makefile point
of view there is just one line split using backslashes and every line
ends with ';'. It is hard to maintain such "script" when it gets longer.

The embedded script does not fail as robust as separate script; set -eu
could be added to get same level of robustness -- but the provided
Bourne Again Shell (bash) script exceeds this with 'set -o pipefail',
making the script to fail when any of the commands in pipeline fails
(and not just the last one).

Checking for release is done very seldom compared to all other use;
The whole Makefile.local gets simpler and easier to grasp when most
release checking targets are removed.

When release checking is done, the steps are executed sequentially;
nothing is allowed to be skipped due to some satisfied dependency.
2012-09-05 08:33:45 -03:00
Mark Walters
ed93d79199 schemata: update for --body=true|false option
Previously body: was a compulsory field in a message. The new
--body=false option causes notmuch show to omit this field so update
schemata to reflect this.
2012-07-24 15:49:33 -03:00
Austin Clements
e7f5302114 Add missing "tags" field to search schema
This field is output by search, but it didn't make it into the
documentation.
2012-07-09 15:45:37 -06:00
Mark Walters
51a7cd3ddb Minor correction to devel/schemata
In id:"87sjdm12d1.fsf@awakening.csail.mit.edu" Austin pointed out that
devel/schemata needs a slight correction with the new
--entire-thread=false option. This is that correction.
2012-06-30 08:21:36 -03:00
Mark Walters
fdce20d365 Update devel/schemata for --entire-thread=false
Also remove the Json --entire-thread item from devel/TODO.
2012-06-29 22:33:13 -03:00
Tomi Ollila
00a8581e4d uncrustify.cfg: comments and more types
Changes to devel/uncrustify.cfg:

* Updated header comment to state this is config file for *notmuch*.
* Added comment about the reason of 'type' keyword used.
* Added some more custom types woth 'type' keyword.
* Have (every) multiline comment lines start with '*'.
2012-06-03 13:39:57 -03:00
Austin Clements
98cad5a207 Sync schemata with current code structure
The schema itself hasn't changed, but many of the references to
functions in notmuch-show.c were out of date.
2012-04-15 09:42:14 -03:00
Adam Wolfe Gordon
5abc9c1097 schemata: Add documentation for JSON reply format. 2012-03-19 21:56:25 -03:00
Adam Wolfe Gordon
06a34f1407 TODO: Add replying to multiple messages 2012-03-19 21:53:30 -03:00
Austin Clements
661c357123 Document the JSON schemata used by show and search 2012-02-27 22:32:49 -04:00
David Bremner
022a281080 STYLE: Initial draft of coding style document
This was edited by (at least) Austin, Tomi, and myself.

Amended with Austin's proposed wording for indentation.
2012-02-12 11:58:19 -05:00
Tomi Ollila
9e701465eb uncrustify.cfg: label indent, some known types, not, # and ##
Adjusted some uncrustify variables to get closer to prevailing style:

* Label indent (for goto) relative to current indentation.
* Registered GMimeObject and mime_node_t being as types.
* Space after ! (not) operator.
* No space after 'stringify' (#) preprosessor token.
* No spacing change around ## (option not versatile enough).

There are at least 3 cases where attention needs to be paid:

* If there is newline between function name and open paren in function
  call, the paren (and args) are indented too far right.
* #define HOUR (60 *MINUTE) -- i.e. no space after star (*).
* void (*foo)(args) -- i.e no space between (name) and (args).
2012-01-25 07:40:50 -04:00
David Bremner
871fc32837 uncrustify.cfg: initial support for notmuch coding style
Uncrustify is a free (as in GPL2+) tool that indents and beautifies
C/C++ code. It is similar to GNU indent in functionality although
probably more configurable (in fairness, indent has better
documentation).  Uncrustify does not have the indent mis-feature of
needing to have every typedef'ed type defined in the
configuration (even standard types like size_t).

This configuration starts with the linux-kernel style from the
uncrustify config, disables aggressive re-indenting of structs,
and fine tunes the handling 'else' and braces.

In an ideal situation, running uncrustify on notmuch code would be
NOP; currently this is not true for all files because 1) the
configuration is not perfect 2) the coding style of notmuch is not
completely consistent; in particular the treatment of braces after
e.g. for (_) is not consistent.

Some fine tuning by Tomi Olilla.
2012-01-21 15:11:25 -04:00
David Bremner
d51b784214 Start devel directory for developer tools and documentation.
We had a lot of back and forth about the name of this directory, but
nothing very conclusive. In the end, I just chose "devel" just to move
on.
2012-01-17 22:53:31 -04:00