Commit graph

270 commits

Author SHA1 Message Date
David Bremner
551254eb76 lib/parse-sexp: apply macros
Macros implement lazy evaluation and lexical scope.  The former is
needed to make certain natural constructs work sensibly (e.g. (tag
,param)) but the latter is mainly future-proofing in case the DSL is
is extended to allow local bindings.

For technical background, see chapters 6 and 17 of [1] (or some other
intermediate programming languages textbook).

[1] http://cs.brown.edu/courses/cs173/2012/book/
2021-09-04 17:07:19 -07:00
David Bremner
81b9dbd110 CLI/config support saving s-expression queries
This commit does not enable using saved s-expression queries, only
saving and retrieving them from the config file or the database. Use
in queries will be enabled in a following commit.
2021-09-04 17:07:19 -07:00
David Bremner
6ab2d9b1a2 lib/parse-sexp: handle saved queries
This provides functionality analogous to query: in the Xapian
QueryParser based parser. Perhaps counterintuitively, the saved
queries currently have to be in the original query syntax (i.e. not
s-expressions).
2021-09-04 17:07:19 -07:00
David Bremner
a07ef8abf5 lib/parse-sexp: parse user headers
One subtle aspect is the replacement of _find_prefix with
_notmuch_database_prefix, which understands user headers. Otherwise
the code mainly consists of creating a fake prefix record (since the
user prefixes are not in the prefix table) and error handling.
2021-09-04 17:07:19 -07:00
David Bremner
cc5992a304 lib/parse-sexp: support infix subqueries
This is necessary so that programs can take infix syntax queries from
a user and use the sexp query syntax to construct e.g. a refinement of
that query.
2021-09-04 17:07:19 -07:00
David Bremner
afe85e6578 lib/parse-sexp: expand queries
The code here is just gluing together _notmuch_query_expand with the
existing sexp parser infrastructure.
2021-09-04 17:07:19 -07:00
David Bremner
1870b3ae4b lib/parse-sexp: support regular expressions
At least to the degree that the Xapian QueryParser based parser
also supports them. Support short alias 'rx' as it seems to make more
complex queries nicer to read.
2021-09-04 17:07:19 -07:00
David Bremner
0ca4ad2670 lib/parse-sexp: add '*' as syntactic sugar for '(starts-with "")'
Users that insist on using a literal '*' as a tag, can continue to do
so by quoting it when searching.
2021-09-04 17:07:19 -07:00
David Bremner
011d06f4d6 lib/parse-sexp: 'starts-with' wildcard searches
The many tests potentially overkill, but they could catch typos in the
prefixes table. As a simplifying assumption, for now we assume a
single argument to the wildcard operator, as this matches the Xapian
semantics. The name 'starts-with' is chosen to emphasize the supported
case of wildcards in currrent (1.4.x) Xapian.
2021-09-04 17:07:19 -07:00
David Bremner
8322f536f5 lib/parse-sexp: add term prefix backed fields
We use "boolean" to describe fields that should generate terms
literally without stemming or phrase splitting.  This terminology
might not be ideal but it is already enshrined in
notmuch-search-terms(7).
2021-09-04 17:07:19 -07:00
David Bremner
90d9c2ad5c lib/parse-sexp: support phrase queries.
Anything that is quoted or not purely word characters is considered a
phrase.  Phrases are not stemmed, because the stems do not have
positional information in the database. It is less efficient to scan
the term twice, but it avoids a second pass to add prefixes, so maybe
it balances out. In any case, it seems unlikely query parsing is very
often a bottleneck.
2021-09-04 17:07:19 -07:00
David Bremner
200e164dc7 lib/parse-sexp: support subject field
The broken tests are because we do not yet handle phrase searches.
2021-09-04 17:07:19 -07:00
David Bremner
f83cd2a05a lib/parse-sexp: support and, not, and or.
All operations and (Xapian) fields will eventually have an entry in
the prefixes table. The flags field is just a placeholder for now, but
will eventually distinguish between various kinds of prefixes.
2021-09-04 17:07:19 -07:00
David Bremner
a2785c3919 lib/parse-sexp: stem unquoted atoms
This is somewhat less DWIM than the Xapian query parser, but it has
the advantage of simplicity.
2021-09-04 17:07:19 -07:00
David Bremner
be7e83de96 lib/parse-sexp: parse single terms and the empty list.
There is not much of a parser here yet, but it already does some
useful error reporting. Most functionality sketched in the
documentation is not implemented yet; detailed documentation will
follow with the implementation.
2021-09-04 17:07:19 -07:00
David Bremner
84347ffcad doc/emacs: use :code: for some missing references
It's not obvious how to reference (non-notmuch) emacs variables and
functions in a sphinx document.
2021-08-22 07:12:37 -07:00
David Bremner
d9072d9eb3 doc: read notmuch-tree.rsti for rst_epilog
This is needed so that docstrings from notmuch-tree.el (in particular
notmuch-tree-toggle-order) can be used in the emacs documentation.
2021-08-22 07:12:22 -07:00
jao
357dd488ca emacs: new command notmuch-tree-filter-by-tag
This new command for notmuch-tree-mode is analogous to
notmuch-search-filter-by-tag, bound to "t" in notmuch-search-mode; it
gets therefore the same "t" keybinding in notmuch-tree-mode (replacing
the current assignment to notmuch-search-by-tag).
2021-08-21 19:53:42 -07:00
jao
7857457833 emacs: new command notmuch-tree-filter
This command is analogous to notmuch-filter, but is defined on tree
mode buffers.
2021-08-21 19:48:13 -07:00
David Bremner
be6edee496 doc: document database search algorithm.
Essentially a translation of the function _choose_database_path for
human consumption. As a bonus, document environment variable
NOTMUCH_DATABASE
2021-08-21 07:54:44 -07:00
David Bremner
dd91621d80 replace references to freenode with references to libera
I left the reference to freenode in the test suite data, since it is
historical.
2021-07-06 17:20:03 -03:00
jao
32f42581e3 doc: new notmuch show --sort and related emacs commands
New --sort CLI option documented in notmuch-show's man page, and
notmuch-search-toggle-order mentioned in doc/notmuch-emacs.rst and
devel/emacs-keybindings.org (in the latter, there's also some
whitespace changes in a table introduced by org-mode).
2021-07-03 20:38:34 -03:00
David Bremner
42b5cb53ef doc: document database.autocommit variable
This exposes some database internals that most users will probably not
understand.
2021-06-27 14:04:43 -03:00
David Bremner
4b0c6fb2f1 Merge branch 'release' 2021-06-25 09:34:29 -03:00
David Bremner
5be9e024d1 doc: tweak hook configuration documentation.
Add a historical note, and hint to look below for more about hooks.
Capitalized the heading for consistency, removed blank line.
2021-06-23 08:42:48 -03:00
David Bremner
d7ddfa0d0e emacs: drop setting mail-user-agent, and document how to set it
After some discussion [1], I decided it is better to make notmuch users
who rely on this behaviour customize mail-user-agent. This is
consistent with the behaviour of other emacs mail packages.

[1]: id:87k0nuhfrk.fsf@toryanderson.com
2021-06-04 20:05:31 -03:00
Jani Nikula
1222cba7e4 doc: example command-line option reference
Example reference to a command-line option using the option role
reference. This creates a hyperlink in html, and the usual boldface
style in man page. This could be used throughout the man pages.
2021-05-22 16:47:32 -03:00
Jani Nikula
f2e2f2aa96 doc: use program and option directives to document options
Use the program and option directives to document the subcommand
options. This unifies a lot of option documentation throughout.

This also makes it possible to reference options with :option:`--foo`
(within .. program::) or :option:`subcommand --foo` (globally). This
is left for later work.

See https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#directive-program

Note: There is a lot of indentation change, but intentionally there is
no reflow. Using 'git diff -w' or 'git show -w' to ignore white space
changes makes this a very easy change to review.
2021-05-22 16:43:24 -03:00
Jani Nikula
574b2436ee doc: use envvar directive and role for environment variables
Make man1/notmuch.rst the single point of truth for describing notmuch
environment variables. Use the envvar directive for that, and
reference them with the envvar role.

Drive-by cleanup configuration file and hook directory search order
documentation.
2021-05-22 16:41:20 -03:00
Jani Nikula
ff4e81ac57 doc: cross-reference notmuch man pages with actual links
Add internal hyperlink targets for man pages and cross-reference them
using the any role reference. There are a number of alternatives to
accomplish this, but this seems like the combination that retains the
man page section number and the same boldface style in the man pages.

As a bonus, we get sanity checking on the links; for example
notmuch-search-terms.rst had a reference to notmuch-properties(1)
i.e. the wrong section.

The obvious semantic follow-up change would be to only have meaningful
"see also" references instead of having them all everywhere.
2021-05-22 16:38:56 -03:00
Jani Nikula
3baa61e0e5 doc: use manpage role references to external man pages
Using manpage role references generates helpful links in html
documentation, while retaining the same boldface style in the man
pages.

The external man page site is configurable. The Debian manpage site
seems like a good fit for Notmuch.
2021-05-22 09:56:52 -03:00
Luis Henriques
0ab28068c1 doc: fix variable name in documentation
Variable 'notmuch-saved-searches-sort-function' does not exist;
'notmuch-saved-search-sort-function' is the correct name.

Signed-off-by: Luis Henriques <henrix@camandro.org>
2021-05-19 08:53:53 -03:00
Jani Nikula
59c953656d doc: fix man page build for Sphinx 4.x
Sphinx 4.0 changed the default value of man_make_section_directory
from False to True. We create the section directories and move the
files manually, so fix the immediate man build failure by disabling
the feature.

The Sphinx documentation on this [1] is confusing, and has the change
backwards. Git history says the default changed from False to True.

[1] https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-man_make_section_directory
2021-05-19 08:52:48 -03:00
Luis Henriques
e715ec9371 doc: fix variable name in documentation
Variable 'notmuch-saved-searches-sort-function' does not exist;
'notmuch-saved-search-sort-function' is the correct name.

Signed-off-by: Luis Henriques <henrix@camandro.org>
2021-05-19 08:35:25 -03:00
Jani Nikula
5197d3e11f doc: fix man page build for Sphinx 4.x
Sphinx 4.0 changed the default value of man_make_section_directory
from False to True. We create the section directories and move the
files manually, so fix the immediate man build failure by disabling
the feature.

The Sphinx documentation on this [1] is confusing, and has the change
backwards. Git history says the default changed from False to True.

[1] https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-man_make_section_directory
2021-05-19 08:26:22 -03:00
David Bremner
bfbe2e55f2 doc: document database.backup_dir
Most users will not need to change this, but documenting it helps
preserve the interface.
2021-05-10 11:14:57 -03:00
David Bremner
a7de593f72 doc: document (tersely) the intended behaviour of relative paths. 2021-05-10 11:14:57 -03:00
David Bremner
dd9112e7d8 CLI/config: default to storing all config in external files
Previously the fact that some configuration options were only stored
in the database (and thus editing the config file had no effect) was a
source of user confusion. This was fixed with the series ending at
d9af0af164.

On the other hand, the underlying partition of config options into
those stored by default in the database and those stored in the config
file remained. This is also confusing, since now some invocations of
"notmuch config set" modify the config file, and others silently
modify the database instead.

With this commit, it is up to the user to decide which configuration
to modify. A new "--database" option is provided for notmuch config to
enable modifying the configuration information in the database;
otherwise the default is to update an external config file.
2021-03-27 09:26:14 -03:00
David Bremner
e823d05ae6 lib: support splitting mail from database location.
Introduce a new configuration value for the mail root, and use it to
locate mail messages in preference to the database.path (which
previously implied the mail messages were also in this location.

Initially only a subset of the CLI is tested in a split
configuration. Further changes will be needed for the remainder of the
CLI to work in split configurations.
2021-03-20 07:39:12 -03:00
David Bremner
4c79a2dabe notmuch 0.31.4 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmAuVjQACgkQA0U5G1Wq
 FSG6RA//bDdAtsG7QlywGONVX1FOSHxAgppDVRvSDXuluPmGgWvc2T80awbonfT5
 AHy9co41L9484QbOzd29d9Ttu5O39JdQbo+DCAJ6y2vm1M1dLCZw3HXaZGfxoUZo
 9L9Agxg/rYIEso7dAehOs3rGsia46aj2MDCVCUuA4DkVFhkbQ2QidIH+l3VPdYqm
 +1XmuJwyftO7hMifFd9W9ifO9wrBcK9WtJ23feufMhqfBsS0ItLysOhZQx+QdxSj
 7GuA6qX6V7XlIWpdohIOMKmT9tGHMDUo6Qk5m8aSc2XmKAybAXRc+qr0Kg8EpvBF
 1d9SqNptcXQL6rORJxluXR/aCuCb7m8YDgxFVSrFcp/M/twbpDC3WzjmAV+RZpDP
 GBKDH06IGaOMOj9GaYYWCe9loGROzOIT4y04Ckukit0AWpmdbaKnkAkPdrxrMWEu
 a5v5KkkMYT42q0PFU5bDjwvq+8Afmzt7oMO72zXz0mfuFNtFzZCUN/AG9LtHoRfk
 Q2ks/xBppmtCxWRAHVWc7f+Gk1OzI2PLVWnXIe9gxGimSpgT2QPUvZcxLHT/XWD5
 XRrrxtkPjKB1v6tWOFEMPI6WxE1iBzoC9AdG2h48ZP5drlLObcAxhAg5AvHJGJSr
 8D4iSa9cKjHD6qxDSXChzF6BF4aDNhF2Flq9Cs9tZb8nyt+Ix18=
 =mAE/
 -----END PGP SIGNATURE-----

Merge tag '0.31.4'

notmuch 0.31.4 release
2021-02-18 08:47:53 -04:00
David Bremner
346b999ae6 doc: bump copyright year 2021-02-18 07:55:28 -04:00
David Bremner
d9af0af164 doc: describe new config framework
Remove STORED IN DATABASE discussion, describe search rules.
Currently profiles seem a bit pointless. They will make more sense
when they apply to databases as well.
2021-02-06 19:59:47 -04:00
Daniel Kahn Gillmor
c4cce82fa0 docs: drop deprecated doxygen TCL_SUBST flag
notmuch has no tcl code, and doxygen upstream is deprecating/removing
tcl support anyway:
48a7afc0ca
2021-01-09 11:25:43 -04:00
David Bremner
ed7ca948ae build/docs: move docstring prereq to file targets
Under a sufficiently high level of parallelism [1] there seems to be a
a race condition that allows sphinx-build to start running before the
docstrings are extracted. This change moves the docstring stamp from
the phony targets sphinx-html and sphinx-info to the file targets that
they depend on. I'm not sure why this makes things better, but I am
fairly confident it does not make things worse, and experimentally it
seems to eliminate the race condition.

[1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=976934
2020-12-09 21:55:38 -04:00
Ralph Seichter
981d5a0168 Rename version to version.txt
Building Notmuch on macOS is known to cause problems because the Notmuch
distribution archive contains two files named "version". These names
clash with the <version> header as defined in C++20. Therefore, the
existing naming will likely become a problem on other platforms as well,
once compilers adopt the new standard.

Signed-off-by: Ralph Seichter <github@seichter.de>
Amended-by: db s/keyword/header/ in commit message.
2020-10-30 16:14:24 -03:00
Jonas Bernoulli
c454135376 emacs: Use makefile-gmake-mode in Makefile*s
Use `makefile-gmake-mode' instead of `makefile-mode' because the
former also highlights ifdef et al. while the latter does not.

"./Makefile.global" and one "Makefile.local" failed to specify any
major mode at all but doing so is necessary because Emacs does not
automatically figure out that these are Makefiles (of any flavor).
2020-08-09 21:14:36 -03:00
David Bremner
a05da45523 doc: add new python bindings to main documentation tree.
A separate conf.py and doc directory (or will be needed if someone wants
to build the bindings docs separately from notmuch.
2020-07-15 08:37:11 -03:00
David Bremner
0e03e2d45e doc: replace use of environment variables with a generated config
It is getting unwieldy to pass configuration options on the
sphinx-build command line, and I anticipate further use of
conditionals.

As far as I could tell, execing a string is the idiomatic way to
emulate include in Python.
2020-07-15 08:32:15 -03:00
Jonas Witschel
a962842d9b doc: make gzipped man pages reproducible
gzip includes the name of the uncompressed file and its modification
timestamp into the compressed archive. The latter makes it hard to
reproduce the generated files bit for bit at a later time, so omit this
information from the archive using the "--no-name" option. This is a
reproducibility best practice, see
https://wiki.debian.org/ReproducibleBuilds/TimestampsInGzipHeaders
2020-07-11 13:57:17 -03:00
Tomi Ollila
507d2f07a6 doc: field processor support now always included, adjust manual pages
The features that require field processor support, are now just
documented w/o mentioning **Xapian Field Processors**' is needed
for those.

Replaced "compact" and "field_processor" with "retry_lock" in
build_with config option, as it is currently the only one that
is optionally excluded. The former 2 are now documented as
features always included.

Dropped one 'we' "passive" in notmuch-search-terms.rst. It was the
only one, and inconsistent with rest of the documentation in that
file.

Dropped message about conditional open-ended ranges support, as
those are now always supported.
2020-06-06 07:54:34 -03:00