Commit graph

135 commits

Author SHA1 Message Date
Michal Sojka
b884ab2ef1 Makefile: Create include directory when installing headers
When I wanted to create a debian package from the current master, make
install failed because of non-existent include directory. This patch
fixes this minor issue.
2010-04-01 05:13:21 -07:00
Carl Worth
c0961e6a82 lib: Switch to a 3-part version number for the library interface.
With a carefully documented description of how to increment the
various version components.
2010-04-01 00:41:25 -07:00
Carl Worth
33d5cc415e Makefiles: Make the install rules quiet like the compilation rules.
The output from make is looking better all the time, (though the
columns still aren't lined up).
2010-03-31 23:54:21 -07:00
Carl Worth
7b52b2c318 Move installation of library from top-level to lib/Makefile.local
We had a fairly ugly violation of modularity with the top-level
Makefile.local isntalling everything, (even when the build commands
for the library were down in lib/Makefile.local).
2010-03-31 22:54:15 -07:00
Saleem Abdulrasool
07378d0d14 Fix target dependencies for multiple jobs
Signed-off-by: Ingmar Vanhassel <ingmar@exherbo.org>
2010-03-31 17:41:28 -07:00
Ben Gamari
266ab595a2 Build and link against notmuch shared library, install notmuch.h
Signed-off-by: Ingmar Vanhassel <ingmar@exherbo.org>
2010-03-31 17:38:27 -07:00
Carl Worth
b957a1b029 emacs: Fix the notmuch-search-authors-width variable.
This variable existed previously, but wasn't actually used for anything.
2010-03-31 13:32:00 -07:00
Carl Worth
e002fe8a7a Clarify documentation of notmuch_database_add_message.
For the case of adding a file that already exist, (with the same
filename). In this case, nothing will happen to the database, but
that wasn't clear before.
2010-03-31 13:31:10 -07:00
Carl Worth
86232e62ab Makefile: Fix Makefiles to depend on all child Makefile fragments.
We were previously maintaining two lists of the child Makefile
fragments---one for the includes and another for the dependencies. So,
of course, they drifted and the dependency list wasn't up to date.

We fix this by adding a single subdirs variable, and then using GNU
Makefile substitution to generate both the include and the dependency
lists.

Some side effect of this change caused the '=' assignment of the dir
variable to not work anymore. I'm not sure why that is, but using ':='
makes sense here and fixes the problem.
2010-03-10 10:59:57 -08:00
Carl Worth
e3046c688b Add is:<tag> as a synonym for tag:<tag> in search terms.
I like the readability of this, it provides compatibility with people
trained in this syntax by sup, and it even saves one character.
2010-03-09 16:03:58 -08:00
Carl Worth
c446f22dee lib: Silence a compiler warning.
The original code was harmless, but apparently some compilers aren't
able to think deep enough to catch that.
2010-03-09 12:07:26 -08:00
Fernando Carrijo
7f2629520c Fix a few documentation typos in notmuch.h
Fix a few documentation typos in notmuch.h

Signed-off-by: Fernando Carrijo <fcarrijo@yahoo.com.br>
2010-03-09 10:32:58 -08:00
Fernando Carrijo
bc69bf09cb Update documentation of notmuch_query_create
Commit cd467caf renamed notmuch_query_search to notmuch_query_search_messages.
Commit 1ba3d46f created notmuch_query_search_threads. We better keep the docs
of notmuch_query_create consistent with those changes.

Signed-off-by: Fernando Carrijo <fcarrijo@yahoo.com.br>

Edited-by: Carl Worth to explicitly list the full name of each
function being referenced.
2010-03-09 10:29:38 -08:00
Carl Worth
64646841f7 lib: Document what move_to_next does at the end of the list.
Explicitly mention that there's an invalid position after the last
item in the list.
2010-03-09 09:24:45 -08:00
Carl Worth
4e5d2f22db lib: Rename iterator functions to prepare for reverse iteration.
We rename 'has_more' to 'valid' so that it can function whether
iterating in a forward or reverse direction. We also rename
'advance' to 'move_to_next' to setup parallel naming with
the proposed functions 'move_to_first', 'move_to_last', and
'move_to_previous'.
2010-03-09 09:22:29 -08:00
Carl Worth
e0a8dee8bc Fix printf for when uint64_t != unsigned long long int
Thanks to Michal Sojka <sojkam1@fel.cvut.cz> for pointing out the
correct fix, which I verified in the freely-available WG14/N1124 draft
(from the C99 working group) which is available here:

http://www.open-std.org/JTC1/SC22/wg14/www/docs/n1124.pdf
2010-02-09 11:14:16 -08:00
Carl Worth
9439b217c3 Switch from random to sequential thread identifiers.
The sequential identifiers have the advantage of being guaranteed to
be unique (until we overflow a 64-bit unsigned integer), and also take
up half as much space in the "notmuch search" output (16 columns
rather than 32).

This change also has the side effect of fixing a bug where notmuch
could block on /dev/random at startup (waiting for some entropy to
appear). This bug was hit hard by the test suite, (which could easily
exhaust the available entropy on common systems---resulting in large
delays of the test suite).
2010-02-09 11:14:11 -08:00
Carl Worth
7a9bacac67 notmuch.h: Fix a couple of typos in the documentation.
Obviously, the spell-checker isn't able to catch every mistake
I make.
2010-02-05 17:31:40 -08:00
Carl Worth
2bc0af15aa Eliminate some useless gobject boilerplate.
If we had external users of this filter then they might expect some of
these macros to exist. But since this is just internal, that's just
unneeded noise.
2010-02-04 17:26:00 -08:00
Carl Worth
3767c6f9f9 notmuch new: Don't index uuencoded data.
With modern MIME attachments, we're already avoiding indexing the
attachments. But for old-school uuencoded data in the mail, we have
been directly indexing the encoded data as terms, (which is not useful
at all---nobody will ever ytry to search based on the seemingly random
uuencoded data).

Additionally, indexing a modestly large uuencoded file seems to make
Xapian go insane, (consuming *lots* of memory).

We fix both problems by detecting uuencoded content and not performing
any indexing of it.
2010-02-04 17:08:11 -08:00
Carl Worth
c340c1bd11 notmuch new: Print upgrade progress report as a percentage.
Previously we were printing a number of messages upgraded so far. The
original motivation for this was to accurately reflect the fact that
there are two passes, (so each message is processed twice and it's not
accurate to represent with a single count). But as it turns out, the
second pass takes zero time (relatively speaking) so we're still not
accounting for it.

If nothing else, the percentage-based reporting makes for a cleaner
API for the progress_notify function.
2010-01-09 17:38:23 -08:00
Carl Worth
ccf2e0cc42 lib: Add non-content terms with a WDF value of 0.
The WDF is the "within-document frequency" value for a particular
term. It's intended to provide an indication of how frequent a term is
within a document, (for use in computing relevance). Xapian's term
generator already computes WDF values when we use that, (which we do
for indexing all mail content).

We don't use the term generator when adding single terms for things
that don't actually appear in the mail document, (such as tags, the
filename, etc.). In this case, the WDF value for these terms doesn't
matter much.

But Xapian's flint backend can be more efficient with changes to terms
that don't affect the document "length". So there's a performance
advantage for manipulating tags (with the flint backend) if the WDF of
these terms is 0.
2010-01-09 11:18:27 -08:00
Carl Worth
45b1856782 lib: Explicitly set BoolWeight when searching.
All notmuch searches currently sort by value (either date or message
ID) so it's just wasted effort for Xapian to compute relevance values
for each result. We now explicitly tell Xapian that we're uninterested
in the relevance values.
2010-01-09 11:16:40 -08:00
Carl Worth
d12801c8b4 lib: Split the database upgrade into two phases for safer operation.
The first phase copies data from the old format to the new format
without deleting anything. This allows an old notmuch to still use the
database if the upgrade process gets interrupted. The second phase
performs the deletion (after updating the database version number). If
the second phase is interrupted, there will be some unused data in the
database, but it shouldn't cause any actual harm.
2010-01-09 11:13:12 -08:00
Carl Worth
5fe5e802ab lib: Delete stale timestamp documents during database upgrade.
Once we move the timestamp to the new directory document, we don't
need the old one anymore.
2010-01-08 09:57:09 -08:00
Carl Worth
1c86b48329 notmuch new: Fix progress notification on database upgrade.
This was firing continuously rather than just once per second as
intended.
2010-01-07 21:24:44 -08:00
Carl Worth
909f52bd8c lib: Implement versioning in the database and provide upgrade function.
The recent support for renames in the database is our first time
(since notmuch has had more than a single user) that we have a
database format change. To support smooth upgrades we now encode a
database format version number in the Xapian metadata.

Going forward notmuch will emit a warning if used to read from a
database with a newer version than it natively supports, and will
refuse to write to a database with a newer version.

The library also provides functions to query the database format
version:

	notmuch_database_get_version

to ask if notmuch wants a newer version than that:

	notmuch_database_needs_upgrade

and a function to actually perform that upgrade:

	notmuch_database_upgrade
2010-01-07 18:26:31 -08:00
Carl Worth
807aef93d3 Prefer READ_ONLY consistently over READONLY.
Previously we had NOTMUCH_DATABASE_MODE_READ_ONLY but
NOTMUCH_STATUS_READONLY_DATABASE which was ugly and confusing. Rename
the latter to NOTMUCH_STATUS_READ_ONLY_DATABASE for consistency.
2010-01-07 10:29:05 -08:00
Carl Worth
f93b7218c3 lib: Consolidate checks for read-only database.
Previously, many checks were deep in the library just before a cast
operation. These have now been replaced with internal errors and new
checks have instead been added at the beginning of all top-levelentry
points requiring a read-write database.

The new checks now also use a single function for checking and
printing the error message. This will give us a convenient location to
extend the check, (such as based on database version as well).
2010-01-07 10:19:44 -08:00
Carl Worth
6ed606c19e lib: Clarify internal documentation of _notmuch_database_filename_to_direntry
The original wording made it sound like this function was just doing
some string manipulation. But this function actually creates new
directory documents as a side effect. So make that explicit in its
documentation.
2010-01-07 09:31:58 -08:00
Carl Worth
a274848f95 notmuch_message_get_filename: Support old-style filename storage.
When a notmuch database is upgraded to the new database format, (to
support file rename and deletion), any message documents corresponding
to deleted files will not currently be upgraded. This means that a
search matching these documents will find no filenames in the expected
place.

Go ahead and return the filename as originally stored, (rather than
aborting with an internal error), in this case.
2010-01-07 09:22:34 -08:00
Carl Worth
957ae198e7 lib: Treat NULL as a valid (and empty) notmuch_filenames_t iterator.
This will be convenient to avoid some special-casing in higher-level
code.
2010-01-06 14:35:11 -08:00
Carl Worth
4b418343f6 lib: Indicate whether notmuch_database_remove_message removed anything.
Similar to the return value of notmuch_database_add_message, we now
enhance the return value of notmuch_database_remove_message to
indicate whether the message document was entirely removed (SUCCESS)
or whether only this filename was removed and the document exists
under other filenamed (DUPLICATE_MESSAGE_ID).
2010-01-06 10:32:06 -08:00
Carl Worth
777cd23d9d lib: Update documentation of notmuch_database_add_message.
Previously, adding a filename with the same message ID as an existing
message would do nothing. But we recently fixed this to instead add
the new filename to the existing message document. So update the
documentation to match now.
2010-01-06 10:32:06 -08:00
Carl Worth
6ef6ddba80 Index content from citations and signatures.
In the presentation we often omit citations and signatures, but this
is not content that should be omitted from the index, (especially
when the citation detection is wrong---see cases where a line
beginning with "From" is corrupted to ">From" by mail processing
tools).
2010-01-06 10:32:06 -08:00
Carl Worth
341d49b061 Makefiles: Use .DEFAULT to support arbitrary targets from sub directories.
Taking advantage of the .DEFAULT construct means that we won't need to
explicitly list targets such as "clean", etc. in each sub-Makefile.
2010-01-06 10:32:06 -08:00
Carl Worth
3f32fd8a1c Add missing comment for NOTMUCH_STATUS_READONLY_DATABASE.
And adjust the string representation of the same to match.
2010-01-06 10:32:06 -08:00
Carl Worth
d807e28f43 lib: Implement new notmuch_directory_t API.
This new directory ojbect provides all the infrastructure needed to
detect when files or directories are deleted or renamed. There's still
code needed on top of this (within "notmuch new") to actually do that
detection.
2010-01-06 10:32:06 -08:00
Carl Worth
ba07fe1819 Revamp the proposed directory-tracking API slightly.
This commit contains my changes to the API proposed by Keith. Nothing
is dramatically different. There are minor things like changing
notmuch_files_t to notmuch_filenames_t and then various things needed
for completeness as noticed while implementing this, (such as
notmuch_directory_destroy and notmuch_directory_set_mtime).
2010-01-06 10:32:06 -08:00
Keith Packard
95deec1b27 Prototypes for directory tracking
There's no functionality here yet---just a sketch of what the
interface could look like.
2010-01-06 10:32:06 -08:00
Carl Worth
f11aaa3678 database: Add new, public notmuch_database_remove_message
This will allow applications to support the removal of messages, (such
as when a file is deleted from the mail store). No removal support is
provided yet in commands such as "notmuch new".
2010-01-06 10:32:06 -08:00
Carl Worth
44a74912c7 database: Add new find_doc_ids_for_term interface.
The existing find_doc_ids function is convenient when the caller
doesn't want to be bothered constructing a term. But when the caller
*does* have the term already, that interface is just wasteful. So we
export a lower-level interface that maps a pre-constructed term to a
document-ID iterators.
2010-01-06 10:32:06 -08:00
Carl Worth
d7e5f5827e database: Make find_unique_doc_id enforce uniqueness (for a debug build)
Catching any violation of this unique-ness constraint is very much in
line with similar, existing INTERNAL_ERROR cases.
2010-01-06 10:32:06 -08:00
Carl Worth
498edff503 database: Abstract _filename_to_direntry from _add_message
The code to map a filename to a direntry is something that we're going
to want in a future _remove_message function, so put it in a new
function _notmuch_database_filename_to_direntry .
2010-01-06 10:32:05 -08:00
Carl Worth
1376a90db6 database: Allowing storing multiple filenames for a single message ID.
The library interface is unchanged so far, (still just
notmuch_database_add_message), but internally, the old
_set_filename function is now _add_filename instead.
2010-01-06 10:32:05 -08:00
Carl Worth
6ca6c089e9 database: Store mail filename as a new 'direntry' term, not as 'data'.
Instead of storing the complete message filename in the data portion
of a mail document we now store a 'direntry' term that contains the
document ID of a directory document and also the basename of the
message filename within that directory. This will allow us to easily
store multple filenames for a single message, and will also allow us
to find mail documents for files that previously existed in a
directory but that have since been deleted.
2010-01-06 10:32:05 -08:00
Carl Worth
84742d86ab database: Split _find_parent_id into _split_path and _find_directory_id
Some pending commits want the _split_path functionality separate from
mapping a directory to a document ID. The split_path function now
returns the basename as well as the directory name.
2010-01-06 10:32:05 -08:00
Carl Worth
4c1cca888f database: Store directory path in 'data' of directory documents.
We're planning to have mail documents refer to directory documents for
the path of the containing directory. To support this, we need the
path in the data, (since the path in the 'directory' term can be
irretrievable as it will be the SHA1 sum of the path for a very long
path).
2010-01-06 10:32:05 -08:00
Carl Worth
406ec4b15d database: Export _notmuch_database_find_parent_id for internal use.
We'll soon have mail documents referring to their parent directory's
directory documents, so we'll need access to _find_parent_id in files
such as message.cc.
2010-01-06 10:32:05 -08:00
Carl Worth
e890b0cf40 database: Store the parent ID for each directory document.
Storing the document ID of the parent of each directory document will
allow us to find all child-directory documents for a given directory
document. We will need this in order to detect directories that have
been removed from the mail store, (though we aren't yet doing this).
2010-01-06 10:32:05 -08:00