This is primarily for notmuch-show, although the functionality is
added to show-message. Once signatures are processed a new
part_sigstatus formatter is emitted, and the entire multipart/signed
part is replaced with the contents of the signed part.
At the moment only a json part_sigstatus formatting function is
available. Emacs support to follow.
The original work for this patch was done by
Daniel Kahn Gillmor <dkg@fifthhorseman.net>
whose help with this functionality I greatly appreciate.
Ever since we added support for "notmuch search --output=tags" the
"notmuch search-tags" command has been redundant. The recent addition
of alias support makes it easy to drop the explicit search-tags
command in favor of a simple alias that runs "notmuch search
--output=tags *".
So there's no longer any documentation of the search-tags command, but
existing scripts will not break at all.
We recently made the --part option to "notmuch show" trigger a default
format of "raw", (since the previous default of "text" is not often
useful with a single part---especially a non-text part).
Here, we update the documentation to match.
We unifed the "notmuch part" functionality into "notmuch show" where
the implementation is both simpler and more powerful. But there's no
good reason to break users of the old interface.
Add support for aliases, which are undocumented means of getting at
functionality through deprecated names. The first such alias is
"notmuch part" as implemented here.
Outputting of single MIME parts is moved to an option of notmuch show,
instead of being handled in it's own sub-command. The recent rework
of multipart mime allowed for this change but consolidating part
handling into a single recursive function (show_message_part) that
includes formatting. This allows for far simpler handling single
output of a single part, including formatting.
Previously, notmuch show flattened all output, losing information
about the nesting of the MIME hierarchy. Now, the output is properly
nested, (both in the --format=text and --format=json output), so that
clients can analyze the original MIME structure.
Internally, this required splitting the final closing delimiter out of
the various show_part functions and putting it into a new
show_part_end function instead. Also, the show_part function now
accepts a new "first" argument that is set not only for the first MIME
part of a message, but also for each first MIME part within a series
of multipart parts. This "first" argument controls the omission of a
preceding comma when printing a part (for json).
Many thanks to David Edmondson <dme@dme.org> for originally
identifying the lack of nesting in the json output and submitting an
early implementation of this feature. Thanks as well to Jameson Graef
Rollins <jrollins@finestructure.net> for carefully shepherding David's
patches through a remarkably long review process, patiently explaining
them, and providing a cleaned up series that led to this final
implementation. Jameson also provided the new emacs code here.
With talloc, we were already freeing all memory by the time we exited
the loop, but that didn't help with excess use of memory inside the
loop, (which was mostly from tallocing some objects with the incorrect
parent).
Thanks to Andrew Tridgell for sitting next to me and teaching me to
use talloc_report_full to find these leaks.
This is part of an effort to avoid proliferation of excessive
top-level notmuch commands. Also, "raw" better captures the
functionality here, (as opposed to "cat" which is a fairly oblique
reference to a bad Unix abbreviation whose metaphor doesn't work here
since "notmuch cat" operates only on a single message and hence cannot
"con'cat'enate" anything).
This command outputs a raw message matched by search term to the
standard output. It allows MUAs to access the messages for piping,
attachment manipulation, etc. by running notmuch cat rather then
directly access the file. This will simplify the MUAs when they need
to operate on a remote database.
Edited-by: Carl Worth <cworth@cworth.org>: Remove trailing whitespace,
add missing "test_done" to new test script to avoid "Unexpected exit"
error.
We previously output "notmuch version 0.1" as response to notmuch --version.
Shorten this to "notmuch 0.1" as we know that we will receive a version
number when we explicitely ask for it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
From both the implementation and from the documentation. This is
handled generically in the library for all search-based commands,
so count doesn't need special treatment.
If no parameters are given to notmuch-count, or just '' or '*' are
given, return the total number of messages in the database.
update notmuch count help
I don't really know why we ended up having everything indented by two
tabs, (perhaps trying to make it match the man page)? But wihout any
containing context to justify that it just looks odd.
The output was far too busy otherwise. It's more useful to just
show the argument list in the case of "notmuch help <command>"
for a specific command.
(Credit due to running "git help" and seeing a much more readable
list than what was coming out of "notmuch help".)
Previously, only "notmuch help" worked while a call to "notmuch --help"
would just print a message telling the user to call "notmuch help".
Instead of the redirection, just support --help directly now.
A new 'part' subcommand allows the user to extract a single part from
a MIME message. Usage:
notmuch part --part=<n> <search terms>
The search terms must match only a single message
(e.g. id:foo@bar.com). The part number specified refers to the part
identifiers output by `notmuch show'. The content of the part is
written the stdout with no formatting or identification marks. It is
not JSON formatted.
In the case of notmuch-show, "--format=json" also implies
"--entire-thread" as the thread structure is implicit in the emitted
document tree.
As a coincidence to the implementation, multipart message ID numbers are
now incremented with each part printed. This changes the previous
semantics, which were unclear and not necessary related to the actual
ordering of the message parts.
This was really the last thing keeping the initial run of "notmuch
new" being different from all other runs. And I'm taking a fresh
look at the performance of "notmuch new" anyway, so I think we can
safely drop this optimization.
This patch changes the default behaviour of notmuch show to display only
messages that match the search expression. However, --entire-thread
option is provided to display all messages in threads that matched the
search expression.
It is deemed that will be more useful for human users on the command line.
Scripts can be modified to include the --entire-thread option so that they
can display all messages once more.
Example:
$ notmuch search subject:git AND thread:23d99d0f364f93e90e15df8b42eddb5b
thread:23d99d0f364f93e90e15df8b42eddb5b July 31 [4/12] Johan Herland; [RFCv2 00/12] Foreign VCS helper program for CVS repositories (inbox unread)
Note that in this thread 4 out of 12 messages matched. The default show
behaviour is to show only those messages that match:
$ notmuch show subject:git AND thread:23d99d0f364f93e90e15df8b42eddb5b | grep 'message{' | wc -l
4
With the --entire-thread option the output will contain all dozen
messages:
$ notmuch show --entire-thread subject:git AND thread:23d99d0f364f93e90e15df8b42eddb5b | grep 'message{' | wc -l
12
Signed-off-by: Bart Trojanowski <bart@jukie.net>
This patch adds support for search-terms to 'notmuch search-tags'. If
no search-term is provided then the command returns a list of all tags
from the database.
If the user provides one or more search-terms as arguments then the
command collects tags from matching messages only.
This could be used by functions in the Emacs mode to further limit the
list of tags offered for completion. For example, functions that remove
tags from message(s) could offer only tags present in the message(s).
Signed-off-by: Jan Janak <jan@ryngle.com>
This is a new notmuch command that can be used to search for all tags
found in the database. The resulting list is alphabetically sorted.
The primary use-case for this new command is to provide the tag
completion feature in Emacs (and other interfaces).
Signed-off-by: Jan Janak <jan@ryngle.com>
This was a poor workaround around the fact that the existing
notmuch_threads_t object is implemented poorly. It's got a fine
iterartor-based interface, but the implementation does all of the
work up-front in _create rather than doing the work incrementally
while iterating.
So to start fixing this, first get rid of all the hacks we had working
around this. This drops the --first and --max-threads options from the
search command, (but hopefully nobody was using them
anyway---notmuch.el certainly wasn't).
The rudimentary aspect here is that the date ranges are specified with
UNIX timestamp values (number of seconds since 1970-01-01 UTC). One
thing that can help here is using the date program to determins
timestamps, such as:
$(date +%s -d 2009-10-01)..$(date +%s)
Long-term, we'll probably need to do our own query parsing to be able
to support directly-specified dates and also relative expressions like
"since:'2 months ago'".
Getting the count of matching threads or messages is a fairly
expensive operation. Xapian provides a very efficient mechanism that
returns an approximate value, so use that for this new command.
This returns the number of matching messages, not threads, as that is
cheap to compute.
Signed-off-by: Keith Packard <keithp@keithp.com>
For very large mail boxes, it is desirable to know which files are being
processed e.g. when a crash occurs to know which one was the cause. Also,
it may be interesting to have a better idea of how the operation is
progressing when processing mailboxes with big messages.
This patch adds support for printing messages as they are processed by
"notmuch new":
* The "new" command now supports a "--verbose" flag.
* When running in verbose mode, the file path of the message about to be
processed is printed in the following format:
current/total: /path/to/message/file
Where "current" is the number of messages processed so far and "total" is
the total count of files to be processed.
The status line is erased using an ANSI sequence "\033[K" (erase current
line from the cursor to the end of line) each time it is refreshed. This
should not pose a problem because nearly every terminal supports it.
* The signal handler for SIGALRM and the timer are not enabled when running
in verbose mode, because we are already printing progress with each file,
periodical reports are not neccessary.
We take the recently created text from the notmuch manual page and
update the "notmuch help" command to use similar text. In particular,
we add a new "notmuch help search-terms" for documenting the search
syntax that is common to several commands.
If this is run first, it will run "notmuch setup" directly. After that
is successful, it will look for a databae and tell the user to run
"notmuch new" if the database doesn't exist yet. Finally, if the
database is present, it will provide some example "notmuch search"
commands for the user to try.
It's quite possible for someone to read the documentation and run
"notmuch setup" rather than just "notmuch". In that case, we don't
want to be any less welcoming.
This will allow for things like the database path to be specified
without any cheesy NOTMUCH_BASE environment variable. It also will
allow "notmuch reply" to recognize the user's email address when
constructing a reply in order to do the right thing, (that is, to use
the user's address to which mail was sent as From:, and not to reply
to the user's own addresses).
With this change, the "notmuch setup" command is now strictly for
changing the configuration of notmuch. It no longer creates the
database, but instead instructs the user to call "notmuch new" to do
that.
The advantage here is that we actually get the necessary folding of
long headers, (particularly the References header, but also things
like Subject). This also gives us parsed recipient addresses so that
we can easily elide the sender's address(es) from the recipient list
(just as soon as we have a configured value for the recipient's
address(es)).
Reviewed-by: Carl Worth <cworth@cworth.org>
Keith wrote all the code here against notmuch before notmuch.c was
split up into multiple files. So I've pushed the code around in
various ways to match the new code structure, but have generally tried
to avoid making any changes to the behavior of the code.
I did fix one bug---a missing call to g_mime_stream_file_set_owner in
show_part which would cause "notmuch show" to go off into the weeds
when trying to show multiple messages, (since the first stream would
fclose stdout).
Now that the client sources are alone here in their own directory,
(with all the library sources down inside the lib directory), we can
break the client up into multiple files without mixing the files up.
The hope is that these smaller files will be easier to manage and
maintain.
I recently added a print of the subject line for use as part of a
two-line summary in the emacs client. But of course, the subject was
already being printed on the next line. So I didn't really need to add
anything, I could have just stopped hiding what was already
printed. Anyway, we now avoid printing it twice in a row.
The more general command is more consistent, and more useful.
We also fix "notmuch search" to output copy-and-pasteable search terms
for the thread with "thread:" prepended already. Similarly, the
message-ID in the output of "notmuch show" is also now printed as a
valid search term, ("id:<message-id>" rather than "ID: <message-id>").
Naturally, the emacs code is also changed to track these changes.
We were inadvertently calling g_object_unref on a wild pointer leading
to the following error message:
GLib-GObject-CRITICAL **: g_object_unref: assertion
`G_IS_OBJECT (object)' failed
Now, why glib doesn't abort on critical errors, I'll never understand.
I previously had a hack that special-cased the "unread" tag and
printed it on the same line as the message ID. But now that we are
printing all tags at the end of the one-line summary we don't need
this anymore. Get rid of it, and just read "unread" from the list of
tags just like any other tag.
Also hide all markers.
From here, all we really need for legibility is the following:
* Hide away citations and signatures
* Call out the one-line summary some way, (larger font size?)
* Add nesting for replies
We were previously using things like "%message{" which were not
guaranteed to never appear in an email message. Using a control
character (^L or '\f' instead of '%') gives us better assurance that
our delimiter doesn't show up in an original email message.
This still isn't entirely safe since we're decoding encoded text in
the body of the email message so almost all bets are off really.
I had noticed several times earlier that having a talloc context
passed in would make things more convenient. I'm not exercising
that convenience yet, but the context is there now, (and there's
one fewer item on our TODO list).
We were aware of this bug when we wrote the function, (that a date
six days in the past would be treated as the "Friday" or as the
"Oct. 23" case depending on whether its time was before or after
the current time today). We thought it wouldn't be a problem, but
in practice it is. In scanning search results with this output,
the transition between formats makes it look like a day boundary,
(so it would be easy to mistakenly think "Oct. 23" is Thursday).
Fix this to avoid confusion, (still being careful to never print
"Thursday" for a date 7 days in the past when today is Thursday).
We're using a delimiter syntax that Keith is optimistic about
being able to easily parse in emacs. Note: We're not escaping
any occurrence of the delimiters in the message yet, so we'll
need to fix that.
With the recent addition of full-text indexing, printing only once per
1000 files just isn't often enough. The new timer-based approach will
be reliable regardless of the speed of adding message.
The original documentation of implicit AND is what we want, but
Xapian doesn't actually let us get that today. So be honest about
what the user can actually expect. And let's hope the Xapian
wizards give us the feature we want soon:
http://trac.xapian.org/ticket/402
Putting all of our documentation into a single help message was getting
a bit unwieldy. Now, the simple output of "notmuch help" is a reasonable
reminder and a quick reference. Then we now support a new syntax of:
"notmuch help <command>" for the more detailed help messages.
This gives us freedom to put more detailed caveats, etc. into some
sub-commands without worrying about the usage statement getting too
long.
This uses the same search functionality as "notmuch search" so
it should be quite powerful. And this global search might be
quick enough to be used for "automatic" adding of tags to new
messages.
Of course, this will all be a lot more useful when we can search
for actual text of messages and not just tags.
I'm trying to stick to a habit of fixing previously-introduced bugs
on side branches off of the commit that introduced the bug. The
idea here is to make it easy to find the commits to cherry pick
if bisecting in the future lands on one of the broken commits.
We were incorrectly only destroying messages in the case of
successful addition to the database, and not in other cases,
(such as failure due to FILE_NOT_EMAIL).
I'm still not entirely sure why this was performing abysmally, (as in
making an operation that should take a small fraction of a second take
10 seconds), nor why it was causing the database to entirely fail to
get new results.
But fortunately, this all seems to work now.
The recent addition of support for automatically adding tags to
new messages for "notmuch new" caused "notmuch setup" to segfault.
The fix is simple, (just need to move a destroy function to inside
a nearby if block).
Did I mention recently we need to add a test suite?
This means that the restore operation will now properly pick up the
removal of tags indicated by the tag just not being present in the
dump file.
We added a few new public functions in order to support this:
notmuch_message_freeze
notmuch_message_remove_all_tags
notmuch_message_thaw
Somehow this naming with an underscore crept in, (but only in the
private header, so notmuch.c was compiling with no prototype). Fix
to be the notmuch_thread_get_subject originally intended.
We want to be able to iterate over tags stored in various ways, so
the previous TermIterator-based tags object just wasn't general
enough. The new interface is nice and simple, and involves only
C datatypes.
We've now got a new notmuch_query_search_threads and a
notmuch_threads_result_t iterator. The thread object itself
doesn't do much yet, (just allows one to get the thread_id),
but that's at least enough to see that "notmuch search" is
actually doing something now, (since it has been converted
to print thread IDs instead of message IDs).
And maybe that's all we need. Getting the messages belonging
to a thread is as simple as a notmuch_query_search_messages
with a string of "thread:<thread-id>".
Though it would be convenient to add notmuch_thread_get_messages
which could use the existing notmuch_message_results_t iterator.
Now we just need an implementation of "notmuch show" and we'll
have something somewhat usable.
Along with renaming notmuch_results_t to notmuch_message_results_t.
The new type is quite a mouthful, but I don't expect it to be
used much other than the for-loop idiom in the documentation,
(which does at least fit nicely within 80 columns).
This is all in preparation for the addition of a new
notmuch_query_search_threads of course.
Having to enumerate all the enum values at every switch is annoying,
but this warning actually found a bug, (missing support for
NOTMUCH_STATUS_OUT_OF_MEMORY in notmuch_status_to_string).
