Commit graph

6971 commits

Author SHA1 Message Date
David Bremner
88224bde62 test/emacs: test for notmuch-search with nonexistent CWD
(At least) notmuch-start-notmuch needs to be updated to set a safe
working directory.
2021-09-11 10:22:24 -03:00
David Bremner
5e5f2122f9 emacs: wrap call-process-region
As with notmuch--process-lines, initial purpose is to provide a safe
binding for default-directory. This is enough to make notmuch-hello
robust against non-existent or corrupt values default-directory, but
probably not other views.
2021-09-11 10:19:27 -03:00
David Bremner
a890241138 emacs: wrap process-lines
Initially just set the working directory, to avoid (the implicit)
call-process crashing when the default-directory points to a
non-existent location.

Use of a macro here is over-engineering for this change, but the same
change needs to be applied to several other process creation
primitives.
2021-09-11 10:16:47 -03:00
David Bremner
06477bce03 test/emacs: run notmuch-hello with a nonexisting default dir
This replicates the problem reported in Debian bug #922536.
2021-09-11 10:13:22 -03:00
David Bremner
59d0d5a489 test/emacs: provide macro test-log-error
Because of the way emacs reports errors, a test form can crash and not
change the main buffer. To work around this, capture both signalled
errors and any other messages.
2021-09-11 10:10:16 -03:00
David Bremner
7556bb7da2 notmuch 0.33.1 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmE7RAYACgkQA0U5G1Wq
 FSHDHA//Q5SJhT9fD/yi2QOI4zIHvF438WtAGIL+TnYIeY8lDmdYZaRjU9Ub4O3J
 eLxJMhNcOyY35nDEGOOVga33aS8lefI+q/Ozx5sNzSge9PRaha+hGAmY2UrkOm+z
 TQBjFsdnGyWdTmJbkC9VQsaQJWjeKUKM0J4eT/sWFrJfW9vHuGnzrEnpOTQu8xk1
 OWshsYc5AkMnnvUBv0JuCJ4z4HQkTHeMrMKwd+B0/SC4OxPQ/af6FaMkoKanH0Ej
 q661AaZW+WNupynS1nu/O99z5U4BJ1V7nz361degdINfaUh/vmcXgHN7jGeoFtj4
 rTjV/51Jy/5WCbYV+11a0+RMZOEoQZi4ZBai6E6WVHpztkDXXqxHyDnho+A4SYK9
 1G/Rs6lO9FYRLTvdYhJbLH4OZgglPWXFgWJJxGLY+5cM7d86g8LC2pbJGWZ5IbAW
 eB3LRFStaqsBqjx9noxt9BcdxmbEu4o7lCm6y+EdAD04X37MGvv+Mlwsz57EMG9c
 2NCntrc1BUUTb/7mCp6oZn0miPCmYFVLtTj0ZfLAdP89aLJVTn6a36uTv8NqcszR
 ZjBFqGxB9AMU0idTEHnHJ5acNj4bLPwDBPBGYmEWnky/Tn7ZT3XqEDVvOXY2HVwa
 0RkVsgMsUIqUXutXjUYlLcl4YgmgpTsRbVpMZJh0/R4N9spDyWU=
 =P0Xd
 -----END PGP SIGNATURE-----

Merge tag '0.33.1'

notmuch 0.33.1 release
2021-09-10 21:02:10 -03:00
David Bremner
8e59438025 debian: drop debian specific test exclusion
In principle these tests should pass now.
2021-09-10 08:30:24 -03:00
David Bremner
b5c6fdd480 debian: changelog for 0.33.1-1 2021-09-10 08:30:07 -03:00
David Bremner
76ec0b409f NEWS: update for 0.33.1 2021-09-10 08:28:04 -03:00
David Bremner
c6524148fc version: bump to 0.33.1 2021-09-10 08:21:28 -03:00
David Bremner
060ff57642 lib: use 'localhost' rather than fqdn for default mail address.
As discussed in the thread starting at [1], the fully qualified domain
name is a bit tricky to get reproducibly, might reveal information
people prefer to keep private, and somewhat unlikely to provide
reliable mail routing.

The new approach of $current_username@localhost is better for the
first two considerations, and probably at least as good as a test mail
address.

[1]: id:87sfyibqhj.fsf@tethera.net
2021-09-10 08:17:06 -03:00
David Bremner
dc8262bd33 test/emacs: test for functions in notmuch-search-result-format.
Based on the commit message in id:20210221151902.2301690-3-dme@dme.org

Add the function notmuch-test-result-flags to test-lib.el to avoid
repeating it in 3 T*.sh files.
2021-09-08 23:01:02 -03:00
David Edmondson
17bfc25bb3 emacs: Allow functions in notmuch-{tree,unthreaded}-result-format
If the car of an element in notmuch-tree-result-format or
notmuch-unthreaded-result-format is a function, insert the result of
calling the function into the buffer.
2021-09-08 23:00:45 -03:00
David Edmondson
4f4ec48df2 emacs: Allow functions in notmuch-search-result-format
If the car of an element in notmuch-search-result-format is a
function, insert the result of calling the function into the buffer.

This allows a user to generate custom fields in the output of a search
result. For example, with:

(defun -notmuch-result-flags (format-string result)
  (let ((tags-to-letters '(("flagged" . "!")
			   ("unread" . "u")
			   ("mine" . "m")
			   ("sent" . "s")
			   ("replied" . "r")))
	(tags (plist-get result :tags)))

    (format format-string
	    (mapconcat (lambda (t2l)
			 (if (member (car t2l) tags)
			     (cdr t2l)
			   " "))
		       tags-to-letters ""))))

(setq notmuch-search-result-format '((-notmuch-result-flags . "%s ")
				     ("date" . "%12s ")
				     ("count" . "%9s ")
				     ("authors" . "%-30s ")
				     ("subject" . "%s ")
				     ("tags" . "(%s)")))

The first few characters on each line of the search result are used to
show information about some significant tags associated with the
thread.
2021-09-08 22:59:48 -03:00
David Edmondson
071fb57a35 emacs: Use pcase in notmuch-search-insert-field
Rather than lots of string-equal calls, use the pcase macro.
2021-09-08 22:59:25 -03:00
Jonas Bernoulli
b03b0d4e41 emacs: notmuch-fcc-header-setup: fix regression
With [1: 16b2db09] we lost the (undocumented) option to use no Fcc
header only for From addresses matching a regexp.  This brings back
that feature and documents it.

1: 2021-01-15 16b2db0986
   emacs: various cosmetic improvements
2021-09-08 22:40:53 -03:00
Tomi Ollila
bd5eef1c6a NEWS: typo fix; some trailing periods
The trailing periods to smaller diff between devel/news2wiki.pl
output and what has been pushed to notmuch-wiki.
2021-09-08 22:26:23 -03:00
David Bremner
49aa44bb01 doc/sexp-queries: update synopsis and description
I chose to go with a somewhat terse synopsis to try to keep the length
of the page down.
2021-09-04 17:07:19 -07:00
David Bremner
4595a814c2 CLI/tag: enable sexp queries
We have to rewrite _optimize_tag_query here because it is generating
a query string in the infix Xapian syntax. Luckily this is easy to do
with the sexp query syntax.
2021-09-04 17:07:19 -07:00
David Bremner
2944d59133 CLI/{count, dump, reindex, reply, show}: enable sexp queries
The change in each case is to call notmuch_query_create_with_syntax,
relying on the already inherited shared options.  As a bonus we get
improved error handling from the new query creation API.

The remaining subcommand is 'tag', which is a bit trickier.
2021-09-04 17:07:19 -07:00
David Bremner
a2e7af5b69 CLI: move query syntax to shared option
This will allow easy addition of a query syntax option to other subcommands.
2021-09-04 17:07:19 -07:00
David Bremner
551254eb76 lib/parse-sexp: apply macros
Macros implement lazy evaluation and lexical scope.  The former is
needed to make certain natural constructs work sensibly (e.g. (tag
,param)) but the latter is mainly future-proofing in case the DSL is
is extended to allow local bindings.

For technical background, see chapters 6 and 17 of [1] (or some other
intermediate programming languages textbook).

[1] http://cs.brown.edu/courses/cs173/2012/book/
2021-09-04 17:07:19 -07:00
David Bremner
3eca7fcf10 lib/parse-sexp: thread environment argument through parser
No functionality change, just an extra argument carried everywhere.
2021-09-04 17:07:19 -07:00
David Bremner
9b9eb1d8bd lib/parse-sexp: support saved s-expression queries
It turns out there is not really much code in query-fp.cc useful for
supporting the new syntax. The code we could potentially factor out
amounts to calling notmuch_database_get_config; both the key
construction and the parsing of the results are specific to the query
syntax involved.
2021-09-04 17:07:19 -07:00
David Bremner
81b9dbd110 CLI/config support saving s-expression queries
This commit does not enable using saved s-expression queries, only
saving and retrieving them from the config file or the database. Use
in queries will be enabled in a following commit.
2021-09-04 17:07:19 -07:00
David Bremner
6ab2d9b1a2 lib/parse-sexp: handle saved queries
This provides functionality analogous to query: in the Xapian
QueryParser based parser. Perhaps counterintuitively, the saved
queries currently have to be in the original query syntax (i.e. not
s-expressions).
2021-09-04 17:07:19 -07:00
David Bremner
036734252d lib: factor out expansion of saved queries.
This is intended to allow use outside of the Xapian query parser.
2021-09-04 17:07:19 -07:00
David Bremner
a07ef8abf5 lib/parse-sexp: parse user headers
One subtle aspect is the replacement of _find_prefix with
_notmuch_database_prefix, which understands user headers. Otherwise
the code mainly consists of creating a fake prefix record (since the
user prefixes are not in the prefix table) and error handling.
2021-09-04 17:07:19 -07:00
David Bremner
cc5992a304 lib/parse-sexp: support infix subqueries
This is necessary so that programs can take infix syntax queries from
a user and use the sexp query syntax to construct e.g. a refinement of
that query.
2021-09-04 17:07:19 -07:00
David Bremner
afe85e6578 lib/parse-sexp: expand queries
The code here is just gluing together _notmuch_query_expand with the
existing sexp parser infrastructure.
2021-09-04 17:07:19 -07:00
David Bremner
4083fd8bec lib/thread-fp: factor out query expansion, rewrite in Xapian
It will be convenient not to have to construct a notmuch query object
when parsing subqueries, so the commit rewrites the query
expansion (currently only used for thread:{} queries) using only
Xapian. As a bonus it seems about 15% faster in initial experiments.
2021-09-04 17:07:19 -07:00
David Bremner
b3bbaf1bc2 lib/query: factor out _notmuch_query_string_to_xapian_query
When dealing with recursive queries (i.e. thread:{foo}) it turns out
to be useful just to deal with the underlying Xapian objects, and not
wrap them in notmuch objects.
2021-09-04 17:07:19 -07:00
David Bremner
c62f3f77a7 lib: generate actual Xapian query for "*" and ""
The previous code had the somewhat bizarre effect that the (notmuch
specific) query string was "*" (interpreted as MatchAll) and the
allegedly parsed xapian_query was "MatchNothing".

This commit also reduces code duplication.
2021-09-04 17:07:19 -07:00
David Bremner
1870b3ae4b lib/parse-sexp: support regular expressions
At least to the degree that the Xapian QueryParser based parser
also supports them. Support short alias 'rx' as it seems to make more
complex queries nicer to read.
2021-09-04 17:07:19 -07:00
David Bremner
5cb452c325 lib: factor out query construction from regexp
This will allow re-use of this code outside of the Xapian query parser.
2021-09-04 17:07:19 -07:00
David Bremner
0b98ad5e4e lib/query: generalize exclude handling to s-expression queries
In fact most of the code path is in common, only the caching of terms
in the query needs to be added for s-expression queries.
2021-09-04 17:07:19 -07:00
David Bremner
bafc307190 lib/parse-sexp: handle unprefixed terms.
This is equivalent to adding the same field name "" for multiple
prefixes in the Xapian query parser, but we have to explicitely
construct the resulting query.
2021-09-04 17:07:19 -07:00
David Bremner
0ca4ad2670 lib/parse-sexp: add '*' as syntactic sugar for '(starts-with "")'
Users that insist on using a literal '*' as a tag, can continue to do
so by quoting it when searching.
2021-09-04 17:07:19 -07:00
David Bremner
011d06f4d6 lib/parse-sexp: 'starts-with' wildcard searches
The many tests potentially overkill, but they could catch typos in the
prefixes table. As a simplifying assumption, for now we assume a
single argument to the wildcard operator, as this matches the Xapian
semantics. The name 'starts-with' is chosen to emphasize the supported
case of wildcards in currrent (1.4.x) Xapian.
2021-09-04 17:07:19 -07:00
David Bremner
8322f536f5 lib/parse-sexp: add term prefix backed fields
We use "boolean" to describe fields that should generate terms
literally without stemming or phrase splitting.  This terminology
might not be ideal but it is already enshrined in
notmuch-search-terms(7).
2021-09-04 17:07:19 -07:00
David Bremner
90d9c2ad5c lib/parse-sexp: support phrase queries.
Anything that is quoted or not purely word characters is considered a
phrase.  Phrases are not stemmed, because the stems do not have
positional information in the database. It is less efficient to scan
the term twice, but it avoids a second pass to add prefixes, so maybe
it balances out. In any case, it seems unlikely query parsing is very
often a bottleneck.
2021-09-04 17:07:19 -07:00
David Bremner
48ad0e1ff3 util/unicode: allow calling from C++
The omission of the 'extern "C"' machinery seems like an oversight.
2021-09-04 17:07:19 -07:00
David Bremner
200e164dc7 lib/parse-sexp: support subject field
The broken tests are because we do not yet handle phrase searches.
2021-09-04 17:07:19 -07:00
David Bremner
f83cd2a05a lib/parse-sexp: support and, not, and or.
All operations and (Xapian) fields will eventually have an entry in
the prefixes table. The flags field is just a placeholder for now, but
will eventually distinguish between various kinds of prefixes.
2021-09-04 17:07:19 -07:00
David Bremner
a2785c3919 lib/parse-sexp: stem unquoted atoms
This is somewhat less DWIM than the Xapian query parser, but it has
the advantage of simplicity.
2021-09-04 17:07:19 -07:00
David Bremner
3202e0d1fe lib: leave stemmer object accessible
This enables using the same stemmer in both query parsers.
2021-09-04 17:07:19 -07:00
David Bremner
be7e83de96 lib/parse-sexp: parse single terms and the empty list.
There is not much of a parser here yet, but it already does some
useful error reporting. Most functionality sketched in the
documentation is not implemented yet; detailed documentation will
follow with the implementation.
2021-09-04 17:07:19 -07:00
David Bremner
9ae4188610 lib: add new status code for query syntax errors.
This will help provide more meaningful error messages without special
casing on the client side.
2021-09-04 17:07:19 -07:00
David Bremner
e3b1a0a6a5 CLI/search+address: support sexpr queries
Initially support selection of query syntax in two subcommands to
enable testing.
2021-09-04 17:07:19 -07:00
David Bremner
c4f2f33a50 lib: define notmuch_query_create_with_syntax
Set the parsing syntax when the (notmuch) query object is
created. Initially the library always returns a trivial query that
matches all messages when using s-expression syntax.

It seems better to select the syntax at query creation time because
the lazy parsing is an implementation detail.
2021-09-04 17:07:19 -07:00