Commit graph

4739 commits

Author SHA1 Message Date
David Bremner
a7eaa4d84a notmuch-new: backup tags before database upgrade
All we do here is calculate the backup filename, and call the existing
dump routine.

Also take the opportunity to add a message about being safe to
interrupt.
2014-04-12 07:59:44 -03:00
David Bremner
4c62d9366a restore: transparently support gzipped input
We rely completely on zlib to do the right thing in detecting gzipped
input. Since our dump format is chosen to be 7 bit ascii, this should
be fine.
2014-04-12 07:59:44 -03:00
David Bremner
de71c4d734 test: restore with missing final newline
Recent proposed patches for gzipped input had a bug with handling
missing newlines that was not caught by the current test suite
2014-04-12 07:59:44 -03:00
David Bremner
85d9219a62 util: add gz_readline
The idea is to provide a more or less drop in replacement for readline
to read from zlib/gzip streams.  Take the opportunity to replace
malloc with talloc.
2014-04-12 07:59:44 -03:00
David Bremner
3c13bc0321 dump: support gzipped and atomic output
The main goal is to support gzipped output for future internal
calls (e.g. from notmuch-new) to notmuch_database_dump.

The additional dependency is not very heavy since xapian already pulls
in zlib.

We want the dump to be "atomic", in the sense that after running the
dump file is either present and complete, or not present.  This avoids
certain classes of mishaps involving overwriting a good backup with a
bad or partial one.
2014-04-12 07:59:44 -03:00
Mark Walters
69867c33fa News for emacs saved-searches change.
The important point is that the changed search variable is not forward
compatible (it *is* backwards compatible): that is previous version
of notmuch-emacs will be unusable with a new style
notmuch-saved-search variable.
2014-04-11 10:38:53 -03:00
Mark Walters
f225ee59eb emacs: Add a sort-order option to saved-searches
This adds a sort-order option to saved-searches, stores it in the
saved-search buttons (widgets), and uses the stored value when the
button is pressed.

Storing the sort-order in the widget was suggested by Jani in
id:4c3876274126985683e888641b29cf18142a5eb8.1391771337.git.jani@nikula.org.
2014-04-11 10:27:26 -03:00
Mark Walters
355d24fb94 emacs: hello: switch notmuch-hello-insert-buttons to plists
Switching notmuch-hello-insert-buttons to plists means we can easily
pass extra options through to the buttons.
2014-04-11 10:26:51 -03:00
Mark Walters
79c262be66 emacs: hello: add a customize for saved-searches
Make the defcustom for notmuch-saved-searches use the new plist
format. It should still work with oldstyle saved-searches but will
write the newstyle form.
2014-04-11 10:26:41 -03:00
Mark Walters
5e26d14536 emacs: hello: use the saved-search helper functions
This uses the helper functions: the saved searches format has not
changed yet but backwards compatibility means everything still works.
2014-04-11 10:26:30 -03:00
Mark Walters
4767e89bc7 emacs: hello: add helper functions for saved-searches
Add helper functions to for saved searches to ease the transition to
the new plist form while maintaining backwards compatibility. They
will be used in the next patch.
2014-04-11 10:26:19 -03:00
Tomi Ollila
8977b1a259 emacs: defun notmuch-hello-versions and bind 'v' in hello mode to it
If notmuch cli & notmuch emacs MUA versions differ, print also the
emacs MUA version string (along with the cli version) to the
minibuffer.
2014-04-10 23:28:06 -03:00
Tomi Ollila
cab1415a94 emacs: add notmuch-version.el.tmpl and create notmuch-version.el from it
The notmuch cli program and emacs lisp versions may differ (especially
in remote usage). It helps to resolve problems if we can determine
the versions of notmuch cli and notmuch emacs mua separately.

The build process now creates notmuch-version.el from template file
by filling the version info to notmuch-emacs-version variable.
2014-04-10 23:25:36 -03:00
Tomi Ollila
feb3ce9572 build: write version.stamp file containing $(VERSION) string
This version file will be as prerequisite to the target files
that use the version info for some purpose, like printing
it for the user to examine. The contents of the version.stamp
file is seldom read by the build system itself as the $(VERSION)
variable has the same information.

Thanks to Trevor, David and Mark for their contributions.
2014-04-10 23:24:10 -03:00
W. Trevor King
c200167426 nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}
With two branches getting fetched (master and config), the branch
referenced by FETCH_HEAD is ambiguous.  For example, I have:

  $ cat FETCH_HEAD
  41d7bfa7184cc93c9dac139d1674e9530799e3b0 \
    not-for-merge   branch 'config' of http://nmbug.tethera.net/git/nmbug-tags
  acd379ccb973c45713eee9db177efc530f921954 \
    not-for-merge   branch 'master' of http://nmbug.tethera.net/git/nmbug-tags

(where I wrapped the line by hand).  This means that FETCH_HEAD
references the config branch:

  $ git rev-parse FETCH_HEAD
  41d7bfa7184cc93c9dac139d1674e9530799e3b0

which breaks all of the FETCH_HEAD logic in nmbug (where FETCH_HEAD is
assumed to point to the master branch).

Instead of relying on FETCH_HEAD, use @{upstream} as the
remote-tracking branch that should be merged/diffed/integrated into
HEAD.  @{upstream} was added in Git v1.7.0 (2010-02-12) [1], so
relying on it should be fairly safe.  One tricky bit is that bare
repositories don't set upstream tracking branches by default:

  $ git clone --bare http://nmbug.tethera.net/git/nmbug-tags.git nmbug-bare
  $ cd nmbug-bare
  $ git remote show origin
  * remote origin
    Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
    Push  URL: http://nmbug.tethera.net/git/nmbug-tags.git
    HEAD branch: master
    Local refs configured for 'git push':
      config pushes to config (up to date)
      master pushes to master (up to date)

While in a non-bare clone:

  $ git clone http://nmbug.tethera.net/git/nmbug-tags.git
  $ cd nmbug-tags
  $ git remote show origin
  * remote origin
    Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
    Push  URL: http://nmbug.tethera.net/git/nmbug-tags.git
    HEAD branch: master
    Remote branches:
      config tracked
      master tracked
    Local branch configured for 'git pull':
      master merges with remote master
    Local ref configured for 'git push':
      master pushes to master (up to date)

From the clone docs [2]:

  --bare::
        Make a 'bare' Git repository…
        Also the branch heads at the remote are copied directly
        to corresponding local branch heads, without mapping
        them to `refs/remotes/origin/`.  When this option is
        used, neither remote-tracking branches nor the related
        configuration variables are created.

To use @{upstream}, we need to the local vs. remote-tracking
distinction, so this commit adds 'nmbug clone', replacing the
previously suggested --bare clone with a non-bare --no-checkout
--separate-git-dir clone into a temporary work directory.  After
which:

  $ git rev-parse @{upstream}
  acd379ccb973c45713eee9db177efc530f921954

gives us the master-branch commit.  Existing nmbug users will have to
run the configuration tweaks and re-fetch by hand.  If you don't have
any local commits, you could also blow away your NMBGIT repository and
re-clone from scratch:

  $ nmbug clone http://nmbug.tethera.net/git/nmbug-tags.git

Besides removing the ambiguity of FETCH_HEAD, this commit allows users
to configure which upstream branch they want nmbug to track via 'git
config', in case they want to change their upstream repository.

[1]: http://git.kernel.org/cgit/git/git.git/tree/Documentation/RelNotes/1.7.0.txt
[2]: http://git.kernel.org/cgit/git/git.git/tree/Documentation/git-clone.txt
2014-04-08 07:39:28 -03:00
Tomi Ollila
5608e39a6b release-checks: removed manual page version check
Manual pages are now generated and during the generation the version
string is read from `version` file, so this (currently failing) test
checking manual page versions can be removed.

While at it, changed the case pattern *[^0-9.]*
to its portable alternative *[!0-9.]*
2014-04-08 07:32:34 -03:00
Tomi Ollila
703dec7754 doc: added 'Init File' section to notmuch-emacs info source
While adding that fixed (also other) typos noticed by aspell(1) run,
and capitalized Emacs and (most) Notmuch terms to match how emacs
Info documentation seems to look in general.
2014-04-08 07:31:32 -03:00
David Bremner
b6751cf875 doc: move doxgen config from devel/ to doc/
a first step towards actually instally the API docs
2014-04-08 07:30:42 -03:00
David Bremner
af5c3afa91 compat: add canonicalize_file_name
the POSIX 2008 behaviour of realpath is not available everywhere so we
provide a simple wrapper function.  We use (and provide) the gnu
extension canonicalize_file_name to make it cleaner to test for the
feature we need; otherwise we have to rely on realpath segfaulting if
the second argument is null.
2014-04-08 07:27:14 -03:00
David Bremner
134dbadaf5 debian: add build conflicts against ruby1.8
Debian stable has ruby 1.9, so this should be OK. The issue is that
the ruby bindings don't build if "ruby" points to ruby1.8
2014-04-08 07:26:53 -03:00
David Bremner
bc10f63f9c debian: really delete madduck from uploaders
I'm not sure how it got into debian/changelog without actually
happening, but actually delete martin from debian/control per
request in Debian bug #719100
2014-04-06 09:56:19 -03:00
David Bremner
f516b7caa2 debian: ignore performance corpus when making source package
Currently "make debian-snapshot" will include the performance corpus
tarball in the source package, which slows things down and wastes disk
space.  tar-ignore is needed twice to keep the default exclude rules
(e.g. to exclude .git)
2014-04-05 12:53:40 -03:00
Jani Nikula
473930bb6f lib: replace the header parser with gmime
The notmuch library includes a full blown message header parser. Yet
the same message headers are parsed by gmime during indexing. Switch
to gmime parsing completely.

These are the main changes:

* Gmime stops header parsing at the first invalid header, and presumes
  the message body starts from there. The current parser is quite
  liberal in accepting broken headers. The change means we will be
  much pickier about accepting invalid messages.

* The current parser converts tabs used in header folding to
  spaces. Gmime preserve the tabs. Due to a broken python library used
  in mailman, there are plenty of mailing lists that produce headers
  with tabs in header folding, and we'll see plenty of tabs. (This
  change has been mitigated in preparatory patches.)

* For pure header parsing, the current parser is likely faster than
  gmime, which parses the whole message rather than just the
  headers. Since we parse the message and its headers using gmime for
  indexing anyway, this avoids and extra header parsing round when
  adding new messages. In case of duplicate messages, we'll end up
  parsing the full message although just headers would be
  sufficient. All in all this should still speed up 'notmuch new'.

* Calls to notmuch_message_get_header() may be slightly slower than
  previously for headers that are not indexed in the database, due to
  parsing of the whole message. Within the notmuch code base, notmuch
  reply is the only such user.
2014-04-05 12:53:04 -03:00
Jani Nikula
6812136bf5 lib: drop support for single-message mbox files
We've supported mbox files containing a single message for historical
reasons, but the support has been deprecated, with a warning message
while indexing, since Notmuch 0.15. Finally drop the support, and
consider all mbox files non-email.
2014-04-05 12:52:42 -03:00
Jani Nikula
3863755f6d cli: abstract dump file open from the dump command
Also expose the dump function to the rest of notmuch. No functional
changes, except for slight improvement in error handling.
2014-03-30 19:23:16 -03:00
Jani Nikula
2a79d81211 cli: abstract database dumping from the dump command
No functional changes, except for slight improvement in error
handling.
2014-03-30 19:22:21 -03:00
Jani Nikula
d0c4cd7e58 emacs: push mark before signature on reply
We push mark on reply so user can cut the quote. Push the mark before
signature, if any, instead of end of buffer so the signature is
preserved.

This is consistent with message-kill-to-signature.
2014-03-30 19:22:05 -03:00
Tomi Ollila
31fc76b782 emacs: add defcustom notmuch-init-file and load it if exists
So that users can easily organize their notmuch-specific configurations
to separate file and they don't have to have notmuch configurations in
*every* emacs installation they launch, especially if those need to
'(require notmuch) to make the configurations possible.
2014-03-30 11:02:28 -03:00
Tomi Ollila
7341b78aba emacs: instruct user to autoload notmuch instead of require'ing it
When (require 'notmuch) is added to ~/.emacs notmuch is loaded to every
instance of emacs although it may not be used in majority of
those instances.

When (autoload 'notmuch "notmuch" ...) is added to ~/.emacs notmuch
is loaded (only) when user invokes the notmuch function.

User may want to add other entrypoints to notmuch by adding more
autoloads -- the autoload instruction given should offer them clue how
to do so.
2014-03-30 11:02:08 -03:00
Jani Nikula
b3e7e7925e cli: fix notmuch help additional topics
The help for hooks was missing.
2014-03-27 22:18:43 -03:00
Jani Nikula
d7d728a622 test: conditionally test help system depending on configured support
If neither sphinx nor rst2man is available, the notmuch man pages will
not be available. Take this into account in the help system test.
2014-03-26 07:43:36 -03:00
Austin Clements
03680d1006 cli: Flush stdout before fork()ing to run hooks
Without this flush, if stdout is block buffered (which will happen if
it's a pipe or a file, for example) and the hook also writes to
stdout, then notmuch new's output will appear *after* the hook output.

This situation may be a little esoteric, but it's good practice to
flush before you fork anyway.
2014-03-25 21:22:17 -03:00
Jani Nikula
31a6333aa4 cli: sanitize the received header before scanning for replies
This makes the from guessing agnostic to header folding by spaces or
tabs.
2014-03-25 21:22:03 -03:00
Jani Nikula
b6238f7e51 completion: complete directory parameters to directories only
i.e. don't complete to files if only directories are acceptable.
2014-03-25 20:00:24 -03:00
Jani Nikula
d5822224da completion: add proper completion of folder: and path:
Complete folder: to maildir folders and path: to directories in mail
store.
2014-03-25 20:00:13 -03:00
Austin Clements
92c3fd938e emacs: Use whitelist instead of blacklist for term escaping
Previously, the term escaper used a blacklist of characters that
needed escaping.  This blacklist turned out to be somewhat incomplete;
for example, it did not contain non-whitespace ASCII control
characters or Unicode "fancy quotes", both of which do require the
term to be escaped.

Switch to a whitelist of characters that are definitely safe to leave
unquoted.  This fixes the broken test introduced by the previous
patch.
2014-03-25 19:57:06 -03:00
Austin Clements
716af7deb8 test: Add broken test for Emacs boolean term escaping
The current term escaper gets most of these right, but fails to escape
things containing Unicode "fancy quotes" or things containing
non-whitespace control characters.
2014-03-25 19:50:47 -03:00
Jani Nikula
d647a19173 test: conditionally test compact depending on configured support
I still have one machine with old enough Xapian to not have compaction
support. Make the tests check for unsupported compact operation when
compact is not available.
2014-03-25 18:51:06 -03:00
David Bremner
57b4ef6f30 doc: fix out-of-tree build
The subtle part is adding .rst and .py files to vpath so they can be
used as dependencies without prefixing with $(srcdir)

We also change the interface to mkbuildeps.py: rather than getting the
containing directory from the conf file path, we go the other way.
2014-03-25 08:32:10 -03:00
David Bremner
68c2c5d31c test: use $(srcdir) instead of . as include path
This is needed for out of tree builds. The functional change is the
modification of extra_cflags; the other changes are cosmetic.
2014-03-25 08:32:10 -03:00
David Bremner
26556f9b11 build: move canonical list of subdirectories to configure script
The configure script needs this list for out of tree builds. Grabbing
it from the Makefile via sed was fragile and broken.
2014-03-25 08:32:10 -03:00
Mark Walters
274355776b emacs: tree: use orig-tags in search
This uses the recent functionality to show the tag changes in the tree
buffer. Currently this is only used to show changes the tree buffer
makes itself: i.e., it does not make display any changes reflecting
tagging done by other notmuch-buffers.
2014-03-24 19:48:55 -03:00
Mark Walters
af8c8c2b46 emacs: search: use orig-tags in search
This uses the recent functionality to show the tag changes in the
search buffer. Currently this is only used to show changes the search
buffer makes itself: i.e., it does not make display any changes
reflecting tagging done by other notmuch-buffers.
2014-03-24 19:48:40 -03:00
Mark Walters
a673d618d1 emacs: show: use orig-tags for tag display
This uses the previous patch to show the tag changes that have occured
in the show buffer since it was last loaded/refreshed.
2014-03-24 19:48:22 -03:00
Mark Walters
941e172724 emacs: show: mark tags changed since buffer loaded
This allows (and requires) the original-tags to be passed along with
the current-tags to be passed to notmuch-tag-format-tags. This allows
the tag formatting to show added and deleted tags.By default a removed
tag is displayed with strike-through in red (if strike-through is not
available, eg on a terminal, inverse video is used instead) and an
added tag is displayed underlined in green.

If the caller does not wish to use the new feature it can pass
current-tags for both arguments and, at this point, we do exactly that
in the three callers of this function.

Note, we cannot tidily allow original-tags to be optional because we would
need to distinguish nil meaning "we are not specifying original-tags"
from nil meaning there were no original-tags (an empty list).

We use this in subsequent patches to make it clear when a message was
unread when you first loaded a show buffer (previously the unread tag
could be removed before a user realised that it had been unread).

The code adds into the existing tag formatting code. The user can
specify exactly how a tag should be displayed normally, when deleted,
or when added.

Since the formatting code matches regexps a user can match all deleted
tags with a ".*" in notmuch-tag-deleted-formats.  For example setting
notmuch-tag-deleted-formats to '((".*" nil)) tells notmuch not to show
deleted tags at all.

All the variables are customizable; however, more complicated cases
like changing the face depending on the type of display will require
custom lisp.

Currently this overrides notmuch-tag-deleted-formats for the tests
setting it to '((".*" nil)) so that they get removed from the display
and, thus, all tests still pass.
2014-03-24 19:48:04 -03:00
Mark Walters
d5acfdda5f emacs: tag: add customize for deleted/added tag formats
Add customize options for deleted/added tag formats.  These are not
used yet but will be later in the series.

We switch to using `notmuch-apply-face' rather than `propertize' in
the defcustom for faces so that the faces for deleted/added tags add
to the default face attributes for the tag.

We special case deleting the unread tag as that tag is a strong visual
cue and we don't need that cue when we are just saying it used to be
unread. Thus, we revert to the normal tag face with strikethough for
deleted unread tags.
2014-03-24 19:47:49 -03:00
Mark Walters
b9a777e0a3 emacs: tag split customise option for format-tags into a widget
We will re-use the customize option for format-tags for formattting
deleted tags to added tags in the next patch so split it into a
widget. There should be no functional change.
2014-03-24 19:47:35 -03:00
Austin Clements
7023466ece Make keys of notmuch-tag-formats regexps and use caching
This modifies `notmuch-tag-format-tag' to treat the keys of
`notmuch-tag-formats' as (anchored) regexps, rather than literal
strings.  This is clearly more flexible, as it allows for prefix
matching, defining a fallback format, etc.  This may cause compatibility
problems if people have customized `notmuch-tag-formats' to match tags
that contain regexp specials, but this seems unlikely.

Regular expression matching has quite a performance hit over string
lookup, so this also introduces a simple cache from exact tags to
formatted strings.  The number of unique tags is likely to be quite
small, so this cache should have a high hit rate.  In addition to
eliminating the regexp lookup in the common case, this cache stores
fully formatted tags, eliminating the repeated evaluation of potentially
expensive, user-specified formatting code.  This makes regexp lookup at
least as fast as assoc for unformatted tags (e.g., inbox) and *faster*
than the current code for formatted tags (e.g., unread):

                    inbox (usec)   unread (usec)
    assoc:              0.4            2.8
    regexp:             3.2            7.2
    regexp+caching:     0.4            0.4

(Though even at 7.2 usec, tag formatting is not our top bottleneck.)

This cache must be explicitly cleared to keep it coherent, so this adds
the appropriate clearing calls.
2014-03-24 19:43:00 -03:00
Austin Clements
dfab8e5e49 emacs: Combine notmuch-combine-face-text-property{, -string}
This combines our two face combining functions into one, easy to use
function with a much shorter name: `notmuch-apply-face'.  This
function takes the full set of arguments that
`notmuch-combine-face-text-property' took, but takes them in a more
convenient order and provides smarter defaults that make the function
easy to use on both strings and buffers.
2014-03-24 19:42:33 -03:00
Jani Nikula
87c2cd78fd nmbug-status: make output title and blurb configurable
Make nmbug-status more generally usable outside of nmbug by not
hardcoding notmuch related things.

This lets anyone publish html search views to mailing list messages
with a custom config file, independent of nmbug.
2014-03-23 08:33:34 -03:00