database: add n_d_index_file (deprecates n_d_add_message)

We need a way to pass parameters to the indexing functionality on the
first index, not just on reindexing.  The obvious place is in
notmuch_database_add_message.  But since modifying the argument list
would break both API and ABI, we needed a new name.

I considered notmuch_database_add_message_with_params(), but the
functionality we're talking about doesn't always add a message.  It
tries to index a specific file, possibly adding a message, but
possibly doing other things, like adding terms to an existing message,
or failing to deal with message objects entirely (e.g. because the
file didn't contain a message).

So i chose the function name notmuch_database_index_file.

I confess i'm a little concerned about confusing future notmuch
developers with the new name, since we already have a private
_notmuch_message_index_file function, and the two do rather different
things.  But i think the added clarity for people linking against the
future libnotmuch and the capacity for using index parameters makes
this a worthwhile tradeoff.  (that said, if anyone has another name
that they strongly prefer, i'd be happy to go with it)

This changeset also adjusts the tests so that we test whether the new,
preferred function returns bad values (since the deprecated function
just calls the new one).

We can keep the deprecated n_d_add_message function around as long as
we like, but at the next place where we're forced to break API or ABI
we can probably choose to drop the name relatively safely.

NOTE: there is probably more cleanup to do in the ruby and go bindings
to complete the deprecation directly.  I don't know those languages
well enough to attempt a fix; i don't know how to test them; and i
don't know the culture around those languages about API additions or
deprecations.

bindings/python/docs/source/database.rst

@@ -25,7 +25,7 @@
 
    .. automethod:: get_directory
 
-   .. automethod:: add_message
+   .. automethod:: index_file
 
    .. automethod:: remove_message
 

bindings/python/notmuch/database.py

@@ -285,7 +285,7 @@ class Database(object):
         """Does this database need to be upgraded before writing to it?
 
         If this function returns `True` then no functions that modify the
-        database (:meth:`add_message`,
+        database (:meth:`index_file`,
         :meth:`Message.add_tag`, :meth:`Directory.set_mtime`,
         etc.) will work unless :meth:`upgrade` is called successfully first.
 
@@ -399,12 +399,13 @@ class Database(object):
         # return the Directory, init it with the absolute path
         return Directory(abs_dirpath, dir_p, self)
 
-    _add_message = nmlib.notmuch_database_add_message
+    _index_file = nmlib.notmuch_database_index_file
-    _add_message.argtypes = [NotmuchDatabaseP, c_char_p,
+    _index_file.argtypes = [NotmuchDatabaseP, c_char_p,
+                             c_void_p,
                              POINTER(NotmuchMessageP)]
-    _add_message.restype = c_uint
+    _index_file.restype = c_uint
 
-    def add_message(self, filename, sync_maildir_flags=False):
+    def index_file(self, filename, sync_maildir_flags=False):
         """Adds a new message to the database
 
         :param filename: should be a path relative to the path of the
@@ -455,7 +456,7 @@ class Database(object):
         """
         self._assert_db_is_initialized()
         msg_p = NotmuchMessageP()
-        status = self._add_message(self._db, _str(filename), byref(msg_p))
+        status = self._index_file(self._db, _str(filename), c_void_p(None), byref(msg_p))
 
         if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
             raise NotmuchError(status)
@@ -467,6 +468,11 @@ class Database(object):
             msg.maildir_flags_to_tags()
         return (msg, status)
 
+    def add_message(self, filename, sync_maildir_flags=False):
+        """Deprecated alias for :meth:`index_file`
+        """
+        self.index_file(self, filename, sync_maildir_flags=sync_maildir_flags)
+
     _remove_message = nmlib.notmuch_database_remove_message
     _remove_message.argtypes = [NotmuchDatabaseP, c_char_p]
     _remove_message.restype = c_uint

bindings/python/notmuch/directory.py

@@ -93,7 +93,7 @@ class Directory(object):
 
         * Read the mtime of a directory from the filesystem
 
-        * Call :meth:`Database.add_message` for all mail files in
+        * Call :meth:`Database.index_file` for all mail files in
           the directory
 
         * Call notmuch_directory_set_mtime with the mtime read from the

bindings/python/notmuch/message.py

@@ -566,7 +566,7 @@ class Message(Python3StringMixIn):
         logical OR operator.)
 
         As a convenience, you can set the sync_maildir_flags parameter in
-        :meth:`Database.add_message` to implicitly call this.
+        :meth:`Database.index_file` to implicitly call this.
 
         :returns: a :class:`STATUS`. In short, you want to see
             notmuch.STATUS.SUCCESS here. See there for details."""

bindings/ruby/database.c

@@ -291,7 +291,7 @@ notmuch_rb_database_add_message (VALUE self, VALUE pathv)
     SafeStringValue (pathv);
     path = RSTRING_PTR (pathv);
 
-    ret = notmuch_database_add_message (db, path, &message);
+    ret = notmuch_database_index_file (db, path, NULL, &message);
     notmuch_rb_status_raise (ret);
     return rb_assoc_new (Data_Wrap_Struct (notmuch_rb_cMessage, NULL, NULL, message),
         (ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) ? Qtrue : Qfalse);

contrib/go/src/notmuch/notmuch.go

@@ -170,7 +170,7 @@ func (self *Database) GetVersion() uint {
 /* Does this database need to be upgraded before writing to it?
  *
  * If this function returns TRUE then no functions that modify the
- * database (notmuch_database_add_message, notmuch_message_add_tag,
+ * database (notmuch_database_index_file, notmuch_message_add_tag,
  * notmuch_directory_set_mtime, etc.) will work unless the function
  * notmuch_database_upgrade is called successfully first. */
 func (self *Database) NeedsUpgrade() bool {

lib/add-message.cc

@@ -458,9 +458,10 @@ _notmuch_database_link_message (notmuch_database_t *notmuch,
 }
 
 notmuch_status_t
-notmuch_database_add_message (notmuch_database_t *notmuch,
+notmuch_database_index_file (notmuch_database_t *notmuch,
-			      const char *filename,
+			     const char *filename,
-			      notmuch_message_t **message_ret)
+			     notmuch_param_t unused (*indexopts),
+			     notmuch_message_t **message_ret)
 {
     notmuch_message_file_t *message_file;
     notmuch_message_t *message = NULL;
@@ -575,3 +576,14 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
 
     return ret;
 }
+
+notmuch_status_t
+notmuch_database_add_message (notmuch_database_t *notmuch,
+			      const char *filename,
+			      notmuch_message_t **message_ret)
+{
+    return notmuch_database_index_file (notmuch, filename,
+					NULL,
+					message_ret);
+
+}

lib/notmuch.h

@@ -237,7 +237,7 @@ typedef struct _notmuch_param notmuch_param_t;
  * The database will not yet have any data in it
  * (notmuch_database_create itself is a very cheap function). Messages
  * contained within 'path' can be added to the database by calling
- * notmuch_database_add_message.
+ * notmuch_database_index_file.
  *
  * In case of any failure, this function returns an error status and
  * sets *database to NULL (after printing an error message on stderr).
@@ -562,6 +562,10 @@ notmuch_database_get_directory (notmuch_database_t *database,
  * terms from the identified file to the existing message's index, and
  * adds 'filename' to the list of filenames known for the message.
  *
+ * 'indexopts' can be NULL (meaning, use the indexing defaults from
+ * the database), or can be an explicit choice of indexing options
+ * that should govern the indexing of this specific 'filename'.
+ *
  * If 'message' is not NULL, then, on successful return
  * (NOTMUCH_STATUS_SUCCESS or NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) '*message'
  * will be initialized to a message object that can be used for things
@@ -593,8 +597,25 @@ notmuch_database_get_directory (notmuch_database_t *database,
  *
  * NOTMUCH_STATUS_UPGRADE_REQUIRED: The caller must upgrade the
  * 	database to use this function.
+ *
+ * @since libnotmuch 5.1 (notmuch 0.26)
  */
 notmuch_status_t
+notmuch_database_index_file (notmuch_database_t *database,
+			     const char *filename,
+			     notmuch_param_t *indexopts,
+			     notmuch_message_t **message);
+
+/**
+ * Deprecated alias for notmuch_database_index_file called with
+ * NULL indexopts.
+ *
+ * @deprecated Deprecated as of libnotmuch 5.1 (notmuch 0.26). Please
+ * use notmuch_database_index_file instead.
+ *
+ */
+NOTMUCH_DEPRECATED(5,1)
+notmuch_status_t
 notmuch_database_add_message (notmuch_database_t *database,
 			      const char *filename,
 			      notmuch_message_t **message);
@@ -1401,7 +1422,7 @@ notmuch_message_get_filenames (notmuch_message_t *message);
  * Re-index the e-mail corresponding to 'message' using the supplied index options
  *
  * Returns the status of the re-index operation.  (see the return
- * codes documented in notmuch_database_add_message)
+ * codes documented in notmuch_database_index_file)
  *
  * After reindexing, the user should discard the message object passed
  * in here by calling notmuch_message_destroy, since it refers to the
@@ -1584,7 +1605,7 @@ notmuch_message_remove_all_tags (notmuch_message_t *message);
  *
  * A client can ensure that notmuch database tags remain synchronized
  * with maildir flags by calling this function after each call to
- * notmuch_database_add_message. See also
+ * notmuch_database_index_file. See also
  * notmuch_message_tags_to_maildir_flags for synchronizing tag changes
  * back to maildir flags.
  */
@@ -1946,7 +1967,7 @@ notmuch_tags_destroy (notmuch_tags_t *tags);
  *
  *   o Read the mtime of a directory from the filesystem
  *
- *   o Call add_message for all mail files in the directory
+ *   o Call index_file for all mail files in the directory
  *
  *   o Call notmuch_directory_set_mtime with the mtime read from the
  *     filesystem.

notmuch-insert.c

@@ -384,7 +384,7 @@ add_file (notmuch_database_t *notmuch, const char *path, tag_op_list_t *tag_ops,
     notmuch_message_t *message;
     notmuch_status_t status;
 
-    status = notmuch_database_add_message (notmuch, path, &message);
+    status = notmuch_database_index_file (notmuch, path, NULL, &message);
     if (status == NOTMUCH_STATUS_SUCCESS) {
 	status = tag_op_list_apply (message, tag_ops, 0);
 	if (status) {

notmuch-new.c

@@ -261,7 +261,7 @@ add_file (notmuch_database_t *notmuch, const char *filename,
     if (status)
 	goto DONE;
 
-    status = notmuch_database_add_message (notmuch, filename, &message);
+    status = notmuch_database_index_file (notmuch, filename, NULL, &message);
     switch (status) {
     /* Success. */
     case NOTMUCH_STATUS_SUCCESS:

test/T070-insert.sh

@@ -200,7 +200,7 @@ cat <<EOF > index-file-$code.gdb
 set breakpoint pending on
 set logging file index-file-$code.log
 set logging on
-break notmuch_database_add_message
+break notmuch_database_index_file
 commands
 return NOTMUCH_STATUS_$code
 continue
@@ -212,13 +212,13 @@ done
 gen_insert_msg
 
 for code in  FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR; do
-    test_begin_subtest "EXIT_FAILURE when add_message returns $code"
+    test_begin_subtest "EXIT_FAILURE when index_file returns $code"
     test_expect_code 1 \
          "${TEST_GDB} --batch-silent --return-child-result \
 	     -ex 'set args insert < $gen_msg_filename' \
 	     -x index-file-$code.gdb notmuch"
 
-    test_begin_subtest "success exit with --keep when add_message returns $code"
+    test_begin_subtest "success exit with --keep when index_file returns $code"
     test_expect_code 0 \
          "${TEST_GDB} --batch-silent --return-child-result \
 	     -ex 'set args insert --keep < $gen_msg_filename' \
@@ -226,13 +226,13 @@ for code in  FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR; do
 done
 
 for code in OUT_OF_MEMORY XAPIAN_EXCEPTION ; do
-    test_begin_subtest "EX_TEMPFAIL when add_message returns $code"
+    test_begin_subtest "EX_TEMPFAIL when index_file returns $code"
     test_expect_code 75 \
          "${TEST_GDB} --batch-silent --return-child-result \
 	     -ex 'set args insert < $gen_msg_filename' \
 	     -x index-file-$code.gdb notmuch"
 
-    test_begin_subtest "success exit with --keep when add_message returns $code"
+    test_begin_subtest "success exit with --keep when index_file returns $code"
     test_expect_code 0 \
          "${TEST_GDB} --batch-silent --return-child-result \
 	     -ex 'set args insert --keep < $gen_msg_filename' \

test/T560-lib-error.sh

@@ -127,7 +127,7 @@ int main (int argc, char** argv)
    if (stat != NOTMUCH_STATUS_SUCCESS) {
      fprintf (stderr, "error opening database: %d\n", stat);
    }
-   stat = notmuch_database_add_message (db, "/dev/null", NULL);
+   stat = notmuch_database_index_file (db, "/dev/null", NULL, NULL);
    if (stat)
        fputs (notmuch_database_status_string (db), stderr);
 
@@ -152,7 +152,7 @@ int main (int argc, char** argv)
    if (stat != NOTMUCH_STATUS_SUCCESS) {
      fprintf (stderr, "error opening database: %d\n", stat);
    }
-   stat = notmuch_database_add_message (db, "./nonexistent", NULL);
+   stat = notmuch_database_index_file (db, "./nonexistent", NULL, NULL);
    if (stat) {
        char *status_string = notmuch_database_status_string (db);
        if (status_string) fputs (status_string, stderr);