Commit graph

849 commits

Author SHA1 Message Date
David Bremner
3eb25c94bd Merge branch 'release' 2021-12-29 14:20:49 -04:00
David Bremner
25e0f5e592 lib/open: do not consider .notmuch alone as an existing database.
It makes perfect sense for users to want to pre-create .notmuch,
e.g. to install hooks, so we should handle the case of a .notmuch
directory without an actual xapian database more gracefully.
2021-12-29 14:11:21 -04:00
David Bremner
18cdd21b8b lib/config: use g_key_file_get_string to read config values
Unlike the previous g_key_file_get_value, this version processes
escape codes for whitespace and \. The remaining two broken tests from
the last commit are because "notmuch config get" treats every value as
a list, and thus the previously introduces stripping of leading
whitespace applies.
2021-12-04 12:17:09 -04:00
David Bremner
1e7d33961e Merge branch 'release' 2021-12-04 09:27:30 -04:00
David Bremner
59aac9cef3 lib/config: don't overwrite database.path if the caller passed it
If the user passed a path, and we opened it, then we consider that
definitive definition of "database.path". This makes libnotmuch
respond more gracefully to certain erroneous combinations of
NOTMUCH_CONFIG settings and config file contents.
2021-12-03 20:52:11 -04:00
David Bremner
d467444085 lib/open: track which parameters are passed
This will be used to fine tune the loading of configuration for
certain special configuration items (initially just "database.path").
2021-12-03 20:50:03 -04:00
David Bremner
b264a49be3 rename built_with.sexpr_query to built_with.sexp_queries
It is confusing to use two different names (sexp vs sexpr) when
compared with the command line option --query=sexp and (furthermore)
singular vs plural when compared with the man page title.
2021-12-03 20:06:06 -04:00
David Bremner
efbf5bafaf lib/open: replace call to deprecated notmuch_database_open_verbose
Essentially inline the existing shim definition of
notmuch_database_open_verbose.
2021-10-30 15:04:28 -03:00
David Bremner
c7705fb95e lib/compact: replace deprecated notmuch_database_open_verbose
It should not be necesary to have any config information here, hence
passing "" to n_d_open_with_config.
2021-10-30 15:04:00 -03:00
David Bremner
e7bb7919a2 Merge branch 'release' 2021-10-30 15:01:01 -03:00
David Bremner
fe51c275fc lib/load_config: deallocate / NULL database on fatal error
This fixes a potential memory leak, and makes the behaviour of
notmuch_database_load_config (somewhat) consistent with
n_d_{open,create} with config.
2021-10-30 13:51:50 -03:00
David Bremner
8f0b84789d lib/load_config: document expectations for db on error, add tests
This is a bit different than n_d_{open,create}_with_config, since
there are several non-zero status codes where we do want to return a
non-NULL database structure.
2021-10-30 13:49:37 -03:00
David Bremner
2ba50b5230 lib/create: fix memory leak, ensure *database=NULL on error
This code previously relied on _finish_open to free the notmuch struct
on errors (except for the case of database == NULL, which was a
potential double free). When we removed those frees from _finish_open,
we introduced a (small) memory leak.

In this commit, fix the memory leak, and harmonize the on-error
behaviour with n_d_open_with_config.
2021-10-30 13:47:47 -03:00
David Bremner
f3fcdd2dda lib/create: document expectations for db on error, add tests
It seems sensible to harmonize the behaviour with
n_d_open_with_config. In this commit we just assert the desired
behaviour.
2021-10-30 13:45:40 -03:00
David Bremner
74c4ce6d88 lib/open: fix potential double-free, ensure *database=NULL on error
During refactoring for 0.32, the code that set notmuch=NULL on various
errors was moved into _finish_open. This meant that the the code which
relied on that to set *database to NULL on error was no longer
correct. It also introduced a potential double free, since the notmuch
struct was deallocated inside _finish_open (via n_d_destroy).

In this commit we revert to "allocator frees", and leave any cleanup
to the caller of _finish_open. This allows us to get back the
behaviour of setting *database to NULL with a small change. Other
callers of _finish_open will need free notmuch on errors.
2021-10-30 13:43:58 -03:00
David Bremner
fb02817943 lib: document n_o_w_config can return NOTMUCH_STATUS_NO_CONFIG
This should be treated as fatal by callers, since we didn't succeed in
opening a Xapian database.
2021-10-30 13:41:57 -03:00
David Bremner
c128c995bc lib: make indexopts pointers opaque
There is no reason for anything outside the indexopts.c compilation
unit to have access to structure members.
2021-10-23 09:48:39 -03:00
Jani Nikula
6987286a5b lib: remove enum names from typedefs
There are some enum typedefs with the enum name:

    typedef enum _name_t { ... } name_t;

We don't need or use the enum names _name_t for anything, and not all
of the enum typedefs have them. We have the typedefs specifically to
use the typedef name.

Use the anonymous enum in the typedefs:

    typedef enum { ... } name_t;
2021-10-23 08:38:53 -03:00
Jani Nikula
e0834e376a lib: fix commented out NOTMUCH_DEPRECATED()
Remove the comment markers from the placeholder NOTMUCH_DEPRECATED(),
added in commit e5f3c3ed50 ("lib: add stub for
notmuch_database_open_with_config").
2021-10-23 08:34:06 -03:00
David Bremner
8383b6cb3a lib: bump lib minor version to 5
One new function (and an enum) was added to the API/ABI.
2021-10-15 08:34:39 -03:00
David Bremner
7556bb7da2 notmuch 0.33.1 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmE7RAYACgkQA0U5G1Wq
 FSHDHA//Q5SJhT9fD/yi2QOI4zIHvF438WtAGIL+TnYIeY8lDmdYZaRjU9Ub4O3J
 eLxJMhNcOyY35nDEGOOVga33aS8lefI+q/Ozx5sNzSge9PRaha+hGAmY2UrkOm+z
 TQBjFsdnGyWdTmJbkC9VQsaQJWjeKUKM0J4eT/sWFrJfW9vHuGnzrEnpOTQu8xk1
 OWshsYc5AkMnnvUBv0JuCJ4z4HQkTHeMrMKwd+B0/SC4OxPQ/af6FaMkoKanH0Ej
 q661AaZW+WNupynS1nu/O99z5U4BJ1V7nz361degdINfaUh/vmcXgHN7jGeoFtj4
 rTjV/51Jy/5WCbYV+11a0+RMZOEoQZi4ZBai6E6WVHpztkDXXqxHyDnho+A4SYK9
 1G/Rs6lO9FYRLTvdYhJbLH4OZgglPWXFgWJJxGLY+5cM7d86g8LC2pbJGWZ5IbAW
 eB3LRFStaqsBqjx9noxt9BcdxmbEu4o7lCm6y+EdAD04X37MGvv+Mlwsz57EMG9c
 2NCntrc1BUUTb/7mCp6oZn0miPCmYFVLtTj0ZfLAdP89aLJVTn6a36uTv8NqcszR
 ZjBFqGxB9AMU0idTEHnHJ5acNj4bLPwDBPBGYmEWnky/Tn7ZT3XqEDVvOXY2HVwa
 0RkVsgMsUIqUXutXjUYlLcl4YgmgpTsRbVpMZJh0/R4N9spDyWU=
 =P0Xd
 -----END PGP SIGNATURE-----

Merge tag '0.33.1'

notmuch 0.33.1 release
2021-09-10 21:02:10 -03:00
David Bremner
060ff57642 lib: use 'localhost' rather than fqdn for default mail address.
As discussed in the thread starting at [1], the fully qualified domain
name is a bit tricky to get reproducibly, might reveal information
people prefer to keep private, and somewhat unlikely to provide
reliable mail routing.

The new approach of $current_username@localhost is better for the
first two considerations, and probably at least as good as a test mail
address.

[1]: id:87sfyibqhj.fsf@tethera.net
2021-09-10 08:17:06 -03:00
David Bremner
551254eb76 lib/parse-sexp: apply macros
Macros implement lazy evaluation and lexical scope.  The former is
needed to make certain natural constructs work sensibly (e.g. (tag
,param)) but the latter is mainly future-proofing in case the DSL is
is extended to allow local bindings.

For technical background, see chapters 6 and 17 of [1] (or some other
intermediate programming languages textbook).

[1] http://cs.brown.edu/courses/cs173/2012/book/
2021-09-04 17:07:19 -07:00
David Bremner
3eca7fcf10 lib/parse-sexp: thread environment argument through parser
No functionality change, just an extra argument carried everywhere.
2021-09-04 17:07:19 -07:00
David Bremner
9b9eb1d8bd lib/parse-sexp: support saved s-expression queries
It turns out there is not really much code in query-fp.cc useful for
supporting the new syntax. The code we could potentially factor out
amounts to calling notmuch_database_get_config; both the key
construction and the parsing of the results are specific to the query
syntax involved.
2021-09-04 17:07:19 -07:00
David Bremner
6ab2d9b1a2 lib/parse-sexp: handle saved queries
This provides functionality analogous to query: in the Xapian
QueryParser based parser. Perhaps counterintuitively, the saved
queries currently have to be in the original query syntax (i.e. not
s-expressions).
2021-09-04 17:07:19 -07:00
David Bremner
036734252d lib: factor out expansion of saved queries.
This is intended to allow use outside of the Xapian query parser.
2021-09-04 17:07:19 -07:00
David Bremner
a07ef8abf5 lib/parse-sexp: parse user headers
One subtle aspect is the replacement of _find_prefix with
_notmuch_database_prefix, which understands user headers. Otherwise
the code mainly consists of creating a fake prefix record (since the
user prefixes are not in the prefix table) and error handling.
2021-09-04 17:07:19 -07:00
David Bremner
cc5992a304 lib/parse-sexp: support infix subqueries
This is necessary so that programs can take infix syntax queries from
a user and use the sexp query syntax to construct e.g. a refinement of
that query.
2021-09-04 17:07:19 -07:00
David Bremner
afe85e6578 lib/parse-sexp: expand queries
The code here is just gluing together _notmuch_query_expand with the
existing sexp parser infrastructure.
2021-09-04 17:07:19 -07:00
David Bremner
4083fd8bec lib/thread-fp: factor out query expansion, rewrite in Xapian
It will be convenient not to have to construct a notmuch query object
when parsing subqueries, so the commit rewrites the query
expansion (currently only used for thread:{} queries) using only
Xapian. As a bonus it seems about 15% faster in initial experiments.
2021-09-04 17:07:19 -07:00
David Bremner
b3bbaf1bc2 lib/query: factor out _notmuch_query_string_to_xapian_query
When dealing with recursive queries (i.e. thread:{foo}) it turns out
to be useful just to deal with the underlying Xapian objects, and not
wrap them in notmuch objects.
2021-09-04 17:07:19 -07:00
David Bremner
c62f3f77a7 lib: generate actual Xapian query for "*" and ""
The previous code had the somewhat bizarre effect that the (notmuch
specific) query string was "*" (interpreted as MatchAll) and the
allegedly parsed xapian_query was "MatchNothing".

This commit also reduces code duplication.
2021-09-04 17:07:19 -07:00
David Bremner
1870b3ae4b lib/parse-sexp: support regular expressions
At least to the degree that the Xapian QueryParser based parser
also supports them. Support short alias 'rx' as it seems to make more
complex queries nicer to read.
2021-09-04 17:07:19 -07:00
David Bremner
5cb452c325 lib: factor out query construction from regexp
This will allow re-use of this code outside of the Xapian query parser.
2021-09-04 17:07:19 -07:00
David Bremner
0b98ad5e4e lib/query: generalize exclude handling to s-expression queries
In fact most of the code path is in common, only the caching of terms
in the query needs to be added for s-expression queries.
2021-09-04 17:07:19 -07:00
David Bremner
bafc307190 lib/parse-sexp: handle unprefixed terms.
This is equivalent to adding the same field name "" for multiple
prefixes in the Xapian query parser, but we have to explicitely
construct the resulting query.
2021-09-04 17:07:19 -07:00
David Bremner
0ca4ad2670 lib/parse-sexp: add '*' as syntactic sugar for '(starts-with "")'
Users that insist on using a literal '*' as a tag, can continue to do
so by quoting it when searching.
2021-09-04 17:07:19 -07:00
David Bremner
011d06f4d6 lib/parse-sexp: 'starts-with' wildcard searches
The many tests potentially overkill, but they could catch typos in the
prefixes table. As a simplifying assumption, for now we assume a
single argument to the wildcard operator, as this matches the Xapian
semantics. The name 'starts-with' is chosen to emphasize the supported
case of wildcards in currrent (1.4.x) Xapian.
2021-09-04 17:07:19 -07:00
David Bremner
8322f536f5 lib/parse-sexp: add term prefix backed fields
We use "boolean" to describe fields that should generate terms
literally without stemming or phrase splitting.  This terminology
might not be ideal but it is already enshrined in
notmuch-search-terms(7).
2021-09-04 17:07:19 -07:00
David Bremner
90d9c2ad5c lib/parse-sexp: support phrase queries.
Anything that is quoted or not purely word characters is considered a
phrase.  Phrases are not stemmed, because the stems do not have
positional information in the database. It is less efficient to scan
the term twice, but it avoids a second pass to add prefixes, so maybe
it balances out. In any case, it seems unlikely query parsing is very
often a bottleneck.
2021-09-04 17:07:19 -07:00
David Bremner
200e164dc7 lib/parse-sexp: support subject field
The broken tests are because we do not yet handle phrase searches.
2021-09-04 17:07:19 -07:00
David Bremner
f83cd2a05a lib/parse-sexp: support and, not, and or.
All operations and (Xapian) fields will eventually have an entry in
the prefixes table. The flags field is just a placeholder for now, but
will eventually distinguish between various kinds of prefixes.
2021-09-04 17:07:19 -07:00
David Bremner
a2785c3919 lib/parse-sexp: stem unquoted atoms
This is somewhat less DWIM than the Xapian query parser, but it has
the advantage of simplicity.
2021-09-04 17:07:19 -07:00
David Bremner
3202e0d1fe lib: leave stemmer object accessible
This enables using the same stemmer in both query parsers.
2021-09-04 17:07:19 -07:00
David Bremner
be7e83de96 lib/parse-sexp: parse single terms and the empty list.
There is not much of a parser here yet, but it already does some
useful error reporting. Most functionality sketched in the
documentation is not implemented yet; detailed documentation will
follow with the implementation.
2021-09-04 17:07:19 -07:00
David Bremner
9ae4188610 lib: add new status code for query syntax errors.
This will help provide more meaningful error messages without special
casing on the client side.
2021-09-04 17:07:19 -07:00
David Bremner
c4f2f33a50 lib: define notmuch_query_create_with_syntax
Set the parsing syntax when the (notmuch) query object is
created. Initially the library always returns a trivial query that
matches all messages when using s-expression syntax.

It seems better to select the syntax at query creation time because
the lazy parsing is an implementation detail.
2021-09-04 17:07:19 -07:00
David Bremner
34733fa25e lib: split notmuch_query_create
Most of the function will be re-usable when creating a query from an
s-expression.
2021-09-04 17:07:19 -07:00
David Bremner
a83ad52da4 configure: optional library sfsexp
The configure part is essentially the same as the other checks using
pkg-config. Since the optional inclusion of this feature changes what
options are available to the user, include it in the "built_with"
pseudo-configuration keys.
2021-09-04 17:07:19 -07:00