Any messages retrieved from a query - either directly via
search_messages() or indirectly via thread objects - are owned by that
query. Retrieving the same message (i.e. corresponding to the same
message ID / database object) several times will always yield the same
C object.
The caller is allowed to destroy message objects owned by a query before
the query itself - which can save memory for long-lived queries.
However, that message must then never be retrieved again from that
query.
The python-notmuch2 bindings will currently destroy every message object
in Message._destroy(), which will lead to an invalid free if the same
message is then retrieved again. E.g. the following python program leads
to libtalloc abort()ing:
import notmuch2
db = notmuch2.Database(mode = notmuch2.Database.MODE.READ_ONLY)
t = next(db.threads('*'))
msgs = list(zip(t.toplevel(), t.toplevel()))
msgs = list(zip(t.toplevel(), t.toplevel()))
Fix this issue by creating a subclass of Message, which is used for
"standalone" message which have to be freed by the caller. Message class
is then used only for messages descended from a query, which do not need
to be freed by the caller.
Even though we use collections.abc.Set which implements all these
methods under their operator names, the actual named variations of
these methods are shockingly missing. So let's add them manually.
crypto.gpg_path was only used when we built against gmime versions
before 3.0. Since we now depend on gmime 3.0.3 or later, it is
meaningless.
The removal of the field from the _notmuch_config struct would be an
ABI change if that struct were externally exposed, but it is not, so
it's safe to unilaterally remove it.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Another fix to the docstrings, this time for the English part of the
docstrings, not the Python class name. No functional changes here.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
I was supposed to amend the original patch that added this function,
but somehow I botched that. The original version runs, so make an
extra commit for the tidying.
This moves away from the deprecated notmuch_database_add_message API
and instead uses the notmuch_database_index_file API. This means
instroducing a class to manage the index options and bumping the
library version requirement to 5.1.
This add the notmuch version and absolute path of the binary used
in the pytest header. This is nice when running the tests
interactively as you get confirmation you're testing the version you
thought you were testing.
This introduces CFFI-based Python3-only bindings.
The bindings aim at:
- Better performance on pypy
- Easier to use Python-C interface
- More "pythonic"
- The API should not allow invalid operations
- Use native object protocol where possible
- Memory safety; whatever you do from python, it should not coredump.
From notmuch 0.28, notmuch support relative database path in
notmuch-config(1), but python binding haven't taught this yet.
afew denied to work with a perfectly fine notmuch-config due to this.
In Python bindings, Message.get_property fails with an AttributeError when trying to fetch a property that doesn't exist.
From d712832ba982085975c27b23bb502af82e638b39 Mon Sep 17 00:00:00 2001
From: hydrargyrum <dev@indigo.re>
Date: Sun, 6 Jan 2019 16:08:55 +0100
Subject: [PATCH] python: fix get_property error when property doesn't exist
Indent the directive properly to attach it to Threads autoclass
documentation.
Fixes:
WARNING: don't know which module to import for autodocumenting
'__str__' (try placing a "module" or "currentmodule" directive in the
document, or giving an explicit module name)
The simplistic mocking in conf.py falls short on python 3.7. Just use
unittest.mock instead.
Fixes:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/sphinx/config.py", line 368, in eval_config_file
execfile_(filename, namespace)
File "/usr/lib/python3/dist-packages/sphinx/util/pycompat.py", line 150, in execfile_
exec_(code, _globals)
File "/path/to/notmuch/bindings/python/docs/source/conf.py", line 39, in <module>
from notmuch import __VERSION__,__AUTHOR__
File "/path/to/notmuch/bindings/python/notmuch/__init__.py", line 54, in <module>
from .database import Database
File "/path/to/notmuch/bindings/python/notmuch/database.py", line 25, in <module>
from .globals import (
File "/path/to/notmuch/bindings/python/notmuch/globals.py", line 48, in <module>
class NotmuchDatabaseS(Structure):
TypeError: __mro_entries__ must return a tuple
This way, one can build for a different Ruby than $PATH/ruby
(e. g. different versions, or Ruby in other paths).
Signed-off-by: Thomas Schneider <qsx@chaotikum.eu>
Correct URLs that have crept into the notmuch codebase with http://
when https:// is possible.
As part of this conversion, this changeset also indicates the current
preferred upstream URLs for both gmime and sup. the new URLs are
https-enabled, the old ones are not.
This also fixes T310-emacs.sh, thanks to Bremner for catching it.
It is unlikely this still works since it has not been updated since
2010. The python packages for debian are now built by the top level
debian/ packaging.
We adopt a pythonic idiom here with an optional argument, rather than
exposing the user to the C indexopts object directly.
This now includes a simple test to ensure that the decrypt_policy
argument works as expected.
The old name has a bit of a feeling of hungarian notation. Also many
generators in the core are named with the suffix "s" to indicate
iterables: dict.items, dict.keys for example.
It makes the function a little more intuitive to use and does not
diverge much from the original function signature.
Also an example is added to the docstring.
The C functions notmuch_database_get_config,
notmuch_database_get_config_list and notmuch_database_set_config are
part of the official C bindings. So there should also be some python
bindings for them.
Also they are the only way to access the named queries introduced in
b9bf3f44.
The interface of the python functions is designed to be close to the C
functions.
currently, notmuch's get_message_parts() opens the file in text mode and passes
the file object to email.message_from_file(fp). In case the email contains
UTF-8 characters, reading might fail inside email.parser with the following exception:
File "/usr/lib/python3.6/site-packages/notmuch/message.py", line 591, in get_message_parts
email_msg = email.message_from_binary_file(fp)
File "/usr/lib/python3.6/email/__init__.py", line 62, in message_from_binary_file
return BytesParser(*args, **kws).parse(fp)
File "/usr/lib/python3.6/email/parser.py", line 110, in parse
return self.parser.parse(fp, headersonly)
File "/usr/lib/python3.6/email/parser.py", line 54, in parse
data = fp.read(8192)
File "/usr/lib/python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 1865: invalid continuation byte
To fix this, read file in binary mode and pass to
email.message_from_binary_file(fp).
Unfortunately, Python 2 doesn't support
email.message_from_binary_file(fp), so keep using
email.message_from_file(fp) there.
Signed-off-by: Florian Klink <flokli@flokli.de>
The deprecated Database.add_message now calls the new index_file with
correct number of arguments (without an extra `self`), and returns the
tuple from index_file - as it used to do before.
This change also adds a DeprecationWarning to the function.
We need a way to pass parameters to the indexing functionality on the
first index, not just on reindexing. The obvious place is in
notmuch_database_add_message. But since modifying the argument list
would break both API and ABI, we needed a new name.
I considered notmuch_database_add_message_with_params(), but the
functionality we're talking about doesn't always add a message. It
tries to index a specific file, possibly adding a message, but
possibly doing other things, like adding terms to an existing message,
or failing to deal with message objects entirely (e.g. because the
file didn't contain a message).
So i chose the function name notmuch_database_index_file.
I confess i'm a little concerned about confusing future notmuch
developers with the new name, since we already have a private
_notmuch_message_index_file function, and the two do rather different
things. But i think the added clarity for people linking against the
future libnotmuch and the capacity for using index parameters makes
this a worthwhile tradeoff. (that said, if anyone has another name
that they strongly prefer, i'd be happy to go with it)
This changeset also adjusts the tests so that we test whether the new,
preferred function returns bad values (since the deprecated function
just calls the new one).
We can keep the deprecated n_d_add_message function around as long as
we like, but at the next place where we're forced to break API or ABI
we can probably choose to drop the name relatively safely.
NOTE: there is probably more cleanup to do in the ruby and go bindings
to complete the deprecation directly. I don't know those languages
well enough to attempt a fix; i don't know how to test them; and i
don't know the culture around those languages about API additions or
deprecations.
A leading / in paths in a .gitignore file matches the beginning of the
path, meaning that for patterns without slashes, git will match files
only in the current directory as opposed to in any subdirectory.
Prefix relevant paths with / in .gitignore files, to prevent
accidentally ignoring files in subdirectories and possibly slightly
improve the performance of "git status".
This function was deprecated in notmuch 0.21. We re-use the name for
a status returning version, and deprecate the _st name. One or two
remaining uses of the (removed) non-status returning version fixed at
the same time
This function was deprecated in notmuch 0.21. We finally remove the
deprecated API, and rename the status returning version to the simpler
name. The status returning is kept as a deprecated alias.
In addition to use ${srcdir} and deliver ${NOTMUCH_SRCDIR} where needed,
source from ruby bindings had to be copied to the out-of-tree target
directory -- if the source files in source directory were referenced
in build and there were also built object files there, those could have
been considered as target files (and then not found when attempting
to create bindings/ruby/notmuch.so).
This signals two things, an intent to be more liberal about accepting
patches, and an intent to stop distributing the bindings if maintenance
doesn't pick up.
Many of the external links found in the notmuch source can be resolved
using https instead of http. This changeset addresses as many as i
could find, without touching the e-mail corpus or expected outputs
found in tests.
Currently, http://packages.python.org/notmuch/ goes through a series
of redirections and ends up pointing to readthedocs. Since we're
using readthedocs directly anyway, just point to it directly.
readthedocs are also now sensibly using a separate domain
(readthedocs.io) for their hosted documentation as distinct from their
own domain (readthedocs.org), so use the correct tld.
The Ruby bindings were missing a way to get all the tags of the
database. Now you should be able to access this with the public
instance method `all_tags` of your database object.
Example of use:
notmuchdb = Notmuch::Database.new path, { :create => false,
:mode => Notmuch::MODE_READ_ONLY }
my_tags = notmuchdb.all_tags
my_tags.each { |tag|
print tag
}
my_tags.destroy!
Amended by db: improve error reporting, add test
The usual make message on everything being up to date is:
make: Nothing to be done for 'all'.
However, since
commit d038b93209
Author: David Bremner <david@tethera.net>
Date: Mon Jun 1 09:08:59 2015 +0200
build: integrate building ruby bindings into notmuch build process
if one doesn't have the ruby dependencies installed, the message has
been:
Missing dependency, skipping ruby bindings
Restore the usual behaviour by dropping the message. It's redundant
during build anyway, since the configure script already outputs:
Checking for ruby development files... No (skipping ruby bindings)
The notmuch python bindings document that database.remove_message
should raise an exception when the message removal fails, but they
don't actually do it.
Drop unused imports, and avoid warning about unused imports when we
import something on behalf of another module.
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
Remove the __len__ functions, as they exhaust the iterator, breaking
'list(x)'.
This is a follow-up to 8866a89e.
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
Use 'notmuch_query_search_{threads,messages}_st' instead of their
deprecated counterpart.
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
Add support for the new notmuch status codes UNSUPPORTED_OPERATION,
UPGRADE_REQUIRED, and PATH_ERROR.
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
This gives some additional access to debugging information when using
the python bindings.
Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>
This is supposed to help build on systems like MacOS with different
conventions for naming shared libraries. We have already computed the
relevant names, so doing it again in ruby seems like a bad idea.
Because ruby generates a Makefile, we have to use recursive make.
Because mkmf.rb hardcodes the name Makefile, put our Makefile{.local}
in the parent directory.
Failing to update this string in globals.py causes failures when the
SONAME changes. In order to hopefully reduce the number of such
errors, automate the process of setting the SONAME in the python
bindings.
