Commit graph

7435 commits

Author SHA1 Message Date
David Bremner
1d254bd35e emacs/show: optionally insert bodies of "deep messages" lazily
This leverages the machinery already there to insert buttons for
attachments.

A potential use-case is browsing the top layers of the tree to decide
which of the lower subtrees to read.
2022-07-14 11:47:49 -04:00
David Bremner
ded2eb1a50 perf-test/emacs: test showing longish threads
This particular thread takes about 100 times longer to display in
emacs than on the command line.

The parameter notmuch-show-max-text-part-size exists, but is currently
ineffective for this task because it only hides attachments, not
part 0.

In future commits, the parameter notmuch-show-{depth,height}-limit
will trigger a similar kind of hiding for bodies as
n-s-m-text-part-size already does for attachments.
2022-07-14 04:50:33 -03:00
David Bremner
09fa6bcc0d debian: add git as a build-dependency, for the test suite
This is needed to run (and test) notmuch-git.
2022-07-07 07:24:58 -03:00
David Bremner
bf5eacbe7e CLI/git: add --format-version argument to init subcommand
This is primarily intended to support testing upward compatibility
with legacy repos.
2022-07-07 07:05:55 -03:00
David Bremner
6219e7380a CL/git: add format version 1
The original nmbug format (now called version 0) creates 1
subdirectory of 'tags/' per message. This causes problems for more
than (roughly) 100k messages.

Version 1 introduces 2 layers of hashed directories. This scheme was
chose to balance the number of subdirectories with the number of extra
directories (and git objects) created via hashing.

This should be upward compatible in the sense that old repositories
will continue to work with the updated notmuch-git.
2022-07-07 06:56:05 -03:00
David Bremner
b07e121923 CLI/git: replace most mentions of nmbug
Particularly in help messages, nmbug is confusing for users who may
have never heard of it.
2022-07-07 06:11:04 -03:00
David Bremner
15b940d26a test: known broken test for top level .notmuch in split configs
In split configurations there is no special significance to a top
level directory called .notmuch in the mail root. Users should
therefore be able to have mail stored underneath it.
2022-07-05 08:09:57 -03:00
David Bremner
db44af75ad bindings/python-cffi: fix docstring
the method Database.get_message does exist any more (if it ever
did). This makes the docstring unhelpful as an example.
2022-07-05 07:34:25 -03:00
David Bremner
bf9e206925 test: add new corpus of duplicate messages
This corpus will be used to test a new --duplicate option for notmuch-show
2022-07-05 07:05:49 -03:00
David Bremner
0a3bb81d4f test: define and use notmuch_sexp_*_sanitize functions
These are based on the equivalent functions for json. Like those,
these are pretty simple-minded, and don't really understand the
syntax.
2022-07-05 07:03:37 -03:00
David Bremner
5f6645bd2a test: use notmuch_json_show_sanitize more places
This makes the tests more robust against changing output formats, by
allowing us to centralize fixes in the sanitization function. It is
not appropriate for all cases, in particular it is unneeded when using
test_json_nodes, and unhelpful when testing filenames.
2022-07-05 07:01:39 -03:00
David Bremner
3cb936b7c4 doc: replace symlink with copies for nmbug, notmuch-setup
Previously only man page aliases were being added as symlinks.  The
addition to man_pages in conf.py automatically propagates to the list
of generated info pages.

Installation of the new pages is handled by existing recipes.
2022-07-03 18:01:57 -03:00
David Bremner
5500868bd9 emacs: mark notmuch-query.el as obsolete
The only functionality actually used by notmuch is the base function
notmuch-query-get-threads; the other functions in this file have
nothing to do with that (single) use.  Move that function into
notmuch-lib.el and rename to reflect use. Deprecate the other
functions in notmuch-query.el.
2022-07-03 17:08:08 -03:00
David Bremner
6a9ae99099 lib/sexp: add parameter expansion for regex and wildcard
Fix the bug reported at [1].

The parameter expansion for regex and wildcard modifiers has to be
done a bit differently, because their arguments are not s-expressions
defining complete Xapian queries.

[1]: id:87o7yxqxy6.fsf@code.pm
2022-07-01 08:37:00 -03:00
David Bremner
4464a5d073 test/sexp: add known broken tests for macro param inside rx/wildcard
These tests replicate the problem reported by Eric Colson [1] (for the
regex case).

[1]: id:87o7yxqxy6.fsf@code.pm
2022-07-01 08:34:36 -03:00
Michael J Gruber
2a5e1cf98b make git ignore new build products
nmbug and notmuch-git are new build products. Make git ignore them just
like other build products.

Signed-off-by: Michael J Gruber <git@grubix.eu>
2022-07-01 08:30:40 -03:00
David Bremner
d73ddec5b8 doc/sexp: document range queries.
Give examples for date fields, as these are commonly useful for user
queries.
2022-06-25 19:52:42 -03:00
David Bremner
e7ffb74041 lib/sexp: allow * as alias for "" in range searches.
It can be tedious to use "" inside of a string, e.g. in a shell script.
2022-06-25 19:49:55 -03:00
David Bremner
7863234586 lib/sexp: special case "" as an argument in lastmod ranges.
Support this syntax for constincy with (data from to) ranges.
2022-06-25 19:49:55 -03:00
David Bremner
254912821f test/sexp: one sided range searches.
The date range parsing machinery already knows how to do something
appropriate with an empty string, but the lastmod parsing blindly
tries to parse each atom as a number.
2022-06-25 19:49:55 -03:00
David Bremner
6f749dd24a lib: check for writable db in n_m_tags_maildir_flags
The database needs to be writable because the list of stored file
names will change in general.
2022-06-25 16:06:34 -03:00
David Bremner
3f27cce71f lib: add NOTMUCH_STATUS_CLOSED_DATABASE, use in _n_d_ensure_writable
In order for a database to actually be writeable, it must be the case that it
is open, not just the correct type of Xapian object. By explicitely
checking, we are able to provide better error reporting, in particular
for the previously broken test in T566-lib-message.
2022-06-25 16:06:18 -03:00
David Bremner
7e654e2a45 lib: Add missing private status values.
These were missed when the corresponding status codes were added.
2022-06-25 16:06:10 -03:00
David Bremner
e70df92085 lib/tag: handle NULL argument to notmuch_tags_valid
Make the behaviour when passed NULL consistent with
notmuch_filenames_valid. The library already passes the result of
notmuch_message_get_tags without checking for NULL, so it should be
handled.
2022-06-25 16:05:45 -03:00
David Bremner
00ec2b271d test: add known broken test for notmuch_tags_valid (NULL)
This should return false, but currently segfaults.

Start a new file for tags library API related tests. This is maybe
overkill, but new C boilerplate which doesn't corrupt the database is
needed anyway.
2022-06-25 16:05:32 -03:00
David Bremner
8ade833ec2 test: error handling for n_m_tags_to_maildir_flags
The closed database case should fail gracefully, but currently it
segfaults.
2022-06-25 16:05:16 -03:00
David Bremner
9e9d0e5ffc test: error handling in _n_message_{add,remove}_filename
Use a closed database to force throwing a Xapian exception. Leave the
check vague to accomodate a potential explicit check for open database.
2022-06-25 12:55:02 -03:00
David Bremner
879ec9d76a lib/message: check return status from _n_m_add_{path,folder}_terms
Mainly to propagate information about Xapian exceptions.
2022-06-25 12:55:02 -03:00
David Bremner
b102d0ad11 lib/message: check return status of _n_m_{add,remove}_term
Xapian exceptions are not something that can be ignored, in general.
2022-06-25 12:55:02 -03:00
David Bremner
7d5a9bd3ae lib: define macro NODISCARD
In either C++17 (or later) mode, or when running cppcheck, this can be
used to selectively generate warnings about discarded return values.
2022-06-25 12:55:02 -03:00
David Bremner
bc80ff829a lib/message: drop _notmuch_message_get_thread_id_only
This function has been unused since commit 4083fd8.
2022-06-25 12:55:02 -03:00
David Bremner
a581755fbb test: _notmuch_message_remove_term catches exceptions.
Unfortunately we can't differentiate between the two distinct error
conditions for Document::remove_term.
2022-06-25 12:55:02 -03:00
David Bremner
f48d2e2ff8 lib/message: catch exceptions in _n_m_add_term
Some code movement is needed to make sure the cache is only
invalidated when the Xapian operation succeeds.
2022-06-25 12:55:02 -03:00
David Bremner
2707c06a0f test: define test_private_C
When testing error handling, it is sometimes difficult to cover a
particular error path deterministically. Introduce a test function to
allow calling lower level functions directly.
2022-06-25 12:55:02 -03:00
David Bremner
bfcf9a6c10 CLI: set NOTMUCH_CONFIG in hooks.
This addresses a bug report / feature request of Uwe Kleine-König. The
assumption is that we always load a config file in the CLI (i.e. we
never pass "" as the config file argument to
notmuch_database_open_with_config).

[1]: id:8baa58c3-7ab9-ec03-1bbd-28aa5be838f2@kleine-koenig.org
2022-06-18 08:24:46 -03:00
David Bremner
e167825bca test: add known broken tests for setting NOTMUCH_CONFIG in hooks
Setting this according to --config was requested by Uwe
Kleine-König. There are some other ways that the configuration file
might be found in current notmuch, so check those as well.

As a bonus, fix a bug in the hook tests that left NOTMUCH_CONFIG set
even though a config file was provided via NOTMUCH_PROFILE.

[1]: id:8baa58c3-7ab9-ec03-1bbd-28aa5be838f2@kleine-koenig.org
2022-06-18 08:23:31 -03:00
David Bremner
8d3d98991d debian: install notmuch-git
Use a separate binary package to avoid dragging in dependencies on
python and git for those that do not want them.
2022-06-17 08:40:19 -03:00
David Bremner
7d1e5a5348 CLI/git: add safety checks for checkout and commit
Commits or checkouts that modify a large fraction of the messages in
the database should be relatively rare (and in some automated process,
probably non-existent). For initial setup, where such operations are
expected, the user can pass --force.
2022-06-17 08:40:19 -03:00
David Bremner
99e85823c8 CLI/git: support configuration for repo location / prefix
This is probably more convenient than always passing a command line
argument.

Use notmuch-config for consistency with other notmuch CLI tools.

Now that there is something relevant in the config files, test the
--config option.
2022-06-17 08:40:19 -03:00
David Bremner
8ed68c1bbe CLI/git: change defaults for repo and prefix
The previous defaults were not suitable for personal (i.e. not
bugtracking for notmuch development) use.

Provide two ways for the user to select nmbug compatible defaults;
command line argument and checking the name of the script.
2022-06-17 08:40:19 -03:00
David Bremner
e50a413906 doc/notmuch-git: initial documentation
This is mainly derived from the various help outputs from the script,
with some massaging of markup and addition of links.
2022-06-17 08:40:19 -03:00
David Bremner
66ccf420c2 CLI/git: cache git indices
If the private index file matches a previously known revision of the
database, we can update the index incrementally using the recorded
lastmod counter. This is typically much faster than a full update,
although it could be slower in the case of large changes to the
database.

The "git-read-tree HEAD" is also a bottleneck, but unfortunately
sometimes is needed. Cache the index checksum and hash to reduce the
number of times the operation is run. The overall design is a
simplified version of the PrivateIndex class.
2022-06-17 08:40:19 -03:00
David Bremner
5ef56fe812 CLI/git: add @timed decorator, time a few functions
Perf will show which binaries are using the CPU cycles, and standard
python profilers will show which python functions, but neither is
great at finding which call to an external binary is taking time, or
locating I/O hotspots.
2022-06-17 08:40:19 -03:00
David Bremner
f67d7c9e7a CLI/git: replace enumeration of tags with sexp query.
Unlike the (current) infix query parser provided by Xapian, the
notmuch specific sexp query parser supports prefixed wildcard queries,
so use those. In addition to being somewhat faster, this avoids
needing to escape all of the user's tags to pass via the shell.
2022-06-17 08:40:19 -03:00
David Bremner
b355973abc test/git: add known broken test for tag with quotes.
There is current insufficient sanitization and/or escaping of tag names
internally in notmuch-git.
2022-06-17 08:40:19 -03:00
David Bremner
d4d5737165 CLI/git: suppress warnings about initial branch name
The canonical nmbug repository still uses "master" as the main branch
name, so defer any potential switch away from that name.
2022-06-17 08:40:19 -03:00
David Bremner
b010584163 CLI/git: rename environment variables.
Although the code required to support both new and old environment
variables is small, it complicates the semantics of configuration, and
make the documentation harder to follow.
2022-06-17 08:40:19 -03:00
David Bremner
a85c82f99a test: initial tests for notmuch-git
Exercise the main functionality of notmuch-git.  add_git_repos() will
hopefully be simplifed when an init subcommand is added.
2022-06-17 08:40:19 -03:00
W. Trevor King
14ac68ae11 CLI/git: Add an 'init' command
For folks that want to start versioning a new tag-space, instead of
cloning one that someone else has already started.

The empty-blob hash-object call avoids errors like:

  $ nmbug commit
  error: invalid object 100644 e69de29bb2 for
'tags/...'
  fatal: git-write-tree: error building trees
  'git HASH(0x9ef3eb8) write-tree' exited with nonzero value

David Bremner suggested [1]:

  $ git hash-object -w /dev/null

instead of my Python version of:

  $ git hash-object -w --stdin <&-

but I expect that closing stdin is more portable than the /dev/null
path (which doesn't exist on Windows, for example).

The --bare init and use of NMBGIT as the work tree (what could go
wrong with an empty commit?) are suggestions from Michal Sojka [2].

[1]: id:87y4vu6uvf.fsf@maritornes.cs.unb.ca
     http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18720
[2]: id:87a93a5or2.fsf@resox.2x.cz
     http://thread.gmane.org/gmane.mail.notmuch.general/19495/focus=19767
2022-06-17 08:40:19 -03:00
David Bremner
a01378e48d CLI/git: make existance of config branch optional on clone
This branch is actually only used by an associated
utility (notmuch-report), and notmuch-git works fine without it.
2022-06-17 08:40:19 -03:00