debian/notmuch
1
0
Fork 0
mirror of https://git.notmuchmail.org/git/notmuch synced 2024-11-21 18:38:08 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
notmuch/test/T650-regexp-query.sh

182 lines
7.8 KiB
Bash
Raw Normal View History

lib: regexp matching in 'subject' and 'from' the idea is that you can run % notmuch search subject:/<your-favourite-regexp>/ % notmuch search from:/<your-favourite-regexp>/ or % notmuch search subject:"your usual phrase search" % notmuch search from:"usual phrase search" This feature is only available with recent Xapian, specifically support for field processors is needed. It should work with bindings, since it extends the query parser. This is easy to extend for other value slots, but currently the only value slots are date, message_id, from, subject, and last_mod. Date is already searchable; message_id is left for a followup commit. This was originally written by Austin Clements, and ported to Xapian field processors (from Austin's custom query parser) by yours truly.
2017-02-27 03:34:20 +01:00
#!/usr/bin/env bash
test_description='regular expression searches'
test: use $(dirname "$0") for sourcing test-lib.sh Don't assume the tests are always run from within the source tree.
2017-09-25 22:38:19 +02:00
. $(dirname "$0")/test-lib.sh || exit 1
lib: regexp matching in 'subject' and 'from' the idea is that you can run % notmuch search subject:/<your-favourite-regexp>/ % notmuch search from:/<your-favourite-regexp>/ or % notmuch search subject:"your usual phrase search" % notmuch search from:"usual phrase search" This feature is only available with recent Xapian, specifically support for field processors is needed. It should work with bindings, since it extends the query parser. This is easy to extend for other value slots, but currently the only value slots are date, message_id, from, subject, and last_mod. Date is already searchable; message_id is left for a followup commit. This was originally written by Austin Clements, and ported to Xapian field processors (from Austin's custom query parser) by yours truly.
2017-02-27 03:34:20 +01:00
lib: Add regexp expansion for for tags and paths From a UI perspective this looks similar to what was already provided for from, subject, and mid, but the implementation is quite different. It uses the database's list of terms to construct a term based query equivalent to the passed regular expression.
2017-03-08 03:32:26 +01:00
add_message '[dir]=bad' '[subject]="To the bone"'
add_message '[dir]=.' '[subject]="Top level"'
add_message '[dir]=bad/news' '[subject]="Bears"'
mkdir -p "${MAIL_DIR}/duplicate/bad/news"
cp "$gen_msg_filename" "${MAIL_DIR}/duplicate/bad/news"
add_message '[dir]=things' '[subject]="These are a few"'
add_message '[dir]=things/favorite' '[subject]="Raindrops, whiskers, kettles"'
add_message '[dir]=things/bad' '[subject]="Bites, stings, sad feelings"'
test_begin_subtest "empty path:// search"
notmuch search 'path:""' > EXPECTED
notmuch search 'path:/^$/' > OUTPUT
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "empty folder:// search"
notmuch search 'folder:""' > EXPECTED
notmuch search 'folder:/^$/' > OUTPUT
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "unanchored folder:// specification"
output=$(notmuch search folder:/bad/ | notmuch_search_sanitize)
test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread)
cli/search: print total number of files matched in summary output. The structured output formats already have all of the filenames. This is an easy bit of UI change to make the multiple files visible.
2017-06-04 14:32:32 +02:00
thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread)
lib: Add regexp expansion for for tags and paths From a UI perspective this looks similar to what was already provided for from, subject, and mid, but the implementation is quite different. It uses the database's list of terms to construct a term based query equivalent to the passed regular expression.
2017-03-08 03:32:26 +01:00
thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)"
test_begin_subtest "anchored folder:// search"
output=$(notmuch search 'folder:/^bad$/' | notmuch_search_sanitize)
test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread)"
test_begin_subtest "unanchored path:// specification"
output=$(notmuch search path:/bad/ | notmuch_search_sanitize)
test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread)
cli/search: print total number of files matched in summary output. The structured output formats already have all of the filenames. This is an easy bit of UI change to make the multiple files visible.
2017-06-04 14:32:32 +02:00
thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread)
lib: Add regexp expansion for for tags and paths From a UI perspective this looks similar to what was already provided for from, subject, and mid, but the implementation is quite different. It uses the database's list of terms to construct a term based query equivalent to the passed regular expression.
2017-03-08 03:32:26 +01:00
thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)"
test_begin_subtest "anchored path:// search"
output=$(notmuch search 'path:/^bad$/' | notmuch_search_sanitize)
test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread)"
# Use "standard" corpus from here on.
add_email_corpus
lib: regexp matching in 'subject' and 'from' the idea is that you can run % notmuch search subject:/<your-favourite-regexp>/ % notmuch search from:/<your-favourite-regexp>/ or % notmuch search subject:"your usual phrase search" % notmuch search from:"usual phrase search" This feature is only available with recent Xapian, specifically support for field processors is needed. It should work with bindings, since it extends the query parser. This is easy to extend for other value slots, but currently the only value slots are date, message_id, from, subject, and last_mod. Date is already searchable; message_id is left for a followup commit. This was originally written by Austin Clements, and ported to Xapian field processors (from Austin's custom query parser) by yours truly.
2017-02-27 03:34:20 +01:00
notmuch search --output=messages from:cworth > cworth.msg-ids
test: add known broken test for null from: and subject: query These queries currently fail with field processors enabled because the code expects a non-empty string.
2017-03-24 23:52:50 +01:00
# these headers will generate no document terms
add_message '[from]="-" [subject]="empty from"'
add_message '[subject]="-"'
test_begin_subtest "null from: search"
notmuch search 'from:""' | notmuch_search_sanitize > OUTPUT
cat <<EOF > EXPECTED
thread:XXX 2001-01-05 [1/1] -; empty from (inbox unread)
EOF
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "null subject: search"
notmuch search 'subject:""' | notmuch_search_sanitize > OUTPUT
cat <<EOF > EXPECTED
thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; - (inbox unread)
EOF
test_expect_equal_file EXPECTED OUTPUT
test: add known broken tests wildcard search in from and subject This was broken by the addition of regexp searching. The detection of wildcards is not currently done in the recursive call to parse_query, because of quoting issues.
2017-03-17 13:17:48 +01:00
test_begin_subtest "xapian wildcard search for from:"
notmuch search --output=messages 'from:cwo*' > OUTPUT
test_expect_equal_file cworth.msg-ids OUTPUT
test_begin_subtest "xapian wildcard search for subject:"
test_expect_equal $(notmuch count 'subject:count*') 1
test: add two known broken tests searching for xapian keywords Thanks to plujon for pointing out this problem on IRC. The underlying issue is that the quotes are stripped before the field processors get the query string, and the heuristic for putting them back is not quite right.
2018-09-30 18:27:54 +02:00
add_message '[from]="and"' '[subject]="and-and-and"'
printf "id:$gen_msg_id\n" > EXPECTED
test_begin_subtest "quoted xapian keyword search for from:"
notmuch search --output=messages 'from:"and"' > OUTPUT
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "quoted xapian keyword search for subject:"
notmuch search --output=messages 'subject:"and-and-and"' > OUTPUT
test_expect_equal_file EXPECTED OUTPUT
lib: regexp matching in 'subject' and 'from' the idea is that you can run % notmuch search subject:/<your-favourite-regexp>/ % notmuch search from:/<your-favourite-regexp>/ or % notmuch search subject:"your usual phrase search" % notmuch search from:"usual phrase search" This feature is only available with recent Xapian, specifically support for field processors is needed. It should work with bindings, since it extends the query parser. This is easy to extend for other value slots, but currently the only value slots are date, message_id, from, subject, and last_mod. Date is already searchable; message_id is left for a followup commit. This was originally written by Austin Clements, and ported to Xapian field processors (from Austin's custom query parser) by yours truly.
2017-02-27 03:34:20 +01:00
test_begin_subtest "regexp from search, case sensitive"
notmuch search --output=messages from:/carl/ > OUTPUT
test_expect_equal_file /dev/null OUTPUT
test_begin_subtest "empty regexp or query"
notmuch search --output=messages from:/carl/ or from:/cworth/ > OUTPUT
test_expect_equal_file cworth.msg-ids OUTPUT
test_begin_subtest "non-empty regexp and query"
notmuch search from:/cworth@cworth.org/ and subject:patch | notmuch_search_sanitize > OUTPUT
cat <<EOF > EXPECTED
thread:XXX 2009-11-18 [1/2] Carl Worth| Alex Botero-Lowry; [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread)
thread:XXX 2009-11-18 [1/2] Carl Worth| Ingmar Vanhassel; [notmuch] [PATCH] Typsos (inbox unread)
thread:XXX 2009-11-18 [1/2] Carl Worth| Jan Janak; [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread)
thread:XXX 2009-11-18 [1/2] Carl Worth| Keith Packard; [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread)
thread:XXX 2009-11-18 [2/5] Carl Worth| Mikhail Gusarov, Keith Packard; [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread)
EOF
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "regexp from search, duplicate term search"
notmuch search --output=messages from:/cworth/ > OUTPUT
test_expect_equal_file cworth.msg-ids OUTPUT
test_begin_subtest "long enough regexp matches only desired senders"
notmuch search --output=messages 'from:"/C.* Wo/"' > OUTPUT
test_expect_equal_file cworth.msg-ids OUTPUT
test_begin_subtest "shorter regexp matches one more sender"
notmuch search --output=messages 'from:"/C.* W/"' > OUTPUT
{ echo id:1258544095-16616-1-git-send-email-chris@chris-wilson.co.uk; cat cworth.msg-ids; } > EXPECTED
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "regexp subject search, non-ASCII"
notmuch search --output=messages subject:/accentué/ > OUTPUT
echo id:877h1wv7mg.fsf@inf-8657.int-evry.fr > EXPECTED
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "regexp subject search, punctuation"
notmuch search subject:/\'X\'/ | notmuch_search_sanitize > OUTPUT
cat <<EOF > EXPECTED
thread:XXX 2009-11-18 [2/2] Keith Packard, Carl Worth; [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread)
EOF
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "regexp subject search, no punctuation"
notmuch search subject:/X/ | notmuch_search_sanitize > OUTPUT
cat <<EOF > EXPECTED
thread:XXX 2009-11-18 [2/2] Keith Packard, Carl Worth; [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread)
thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)
EOF
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "combine regexp from and subject"
notmuch search subject:/-C/ and from:/.an.k/ | notmuch_search_sanitize > OUTPUT
cat <<EOF > EXPECTED
thread:XXX 2009-11-17 [1/2] Jan Janak| Carl Worth; [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread)
EOF
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "regexp error reporting"
Prepend regerror() messages with "regexp error: " The exact error messages returned by regerror() aren't standardized; relying on them isn't portable. Thus, add a a prefix to make clear that the subsequent message is a regexp parsing error, and only look for this prefix in the test suite, ignoring the rest of the message.
2019-03-11 18:14:49 +01:00
notmuch search 'from:/unbalanced[/' 2>&1 | sed -e '/^A Xapian/ s/[^:]*$//' > OUTPUT
lib: regexp matching in 'subject' and 'from' the idea is that you can run % notmuch search subject:/<your-favourite-regexp>/ % notmuch search from:/<your-favourite-regexp>/ or % notmuch search subject:"your usual phrase search" % notmuch search from:"usual phrase search" This feature is only available with recent Xapian, specifically support for field processors is needed. It should work with bindings, since it extends the query parser. This is easy to extend for other value slots, but currently the only value slots are date, message_id, from, subject, and last_mod. Date is already searchable; message_id is left for a followup commit. This was originally written by Austin Clements, and ported to Xapian field processors (from Austin's custom query parser) by yours truly.
2017-02-27 03:34:20 +01:00
cat <<EOF > EXPECTED
notmuch search: A Xapian exception occurred
Prepend regerror() messages with "regexp error: " The exact error messages returned by regerror() aren't standardized; relying on them isn't portable. Thus, add a a prefix to make clear that the subsequent message is a regexp parsing error, and only look for this prefix in the test suite, ignoring the rest of the message.
2019-03-11 18:14:49 +01:00
A Xapian exception occurred parsing query: Regexp error:
lib: regexp matching in 'subject' and 'from' the idea is that you can run % notmuch search subject:/<your-favourite-regexp>/ % notmuch search from:/<your-favourite-regexp>/ or % notmuch search subject:"your usual phrase search" % notmuch search from:"usual phrase search" This feature is only available with recent Xapian, specifically support for field processors is needed. It should work with bindings, since it extends the query parser. This is easy to extend for other value slots, but currently the only value slots are date, message_id, from, subject, and last_mod. Date is already searchable; message_id is left for a followup commit. This was originally written by Austin Clements, and ported to Xapian field processors (from Austin's custom query parser) by yours truly.
2017-02-27 03:34:20 +01:00
Query string was: from:/unbalanced[/
EOF
test_expect_equal_file EXPECTED OUTPUT
lib: Add regexp searching for mid: prefix The bulk of the change is passing in the field options to the regexp field processor, so that we can properly handle the fallback (non-regexp case).
2017-02-15 13:58:28 +01:00
test_begin_subtest "empty mid search"
notmuch search --output=messages mid:yoom > OUTPUT
cp /dev/null EXPECTED
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "non-empty mid regex search"
notmuch search --output=messages mid:/yoom/ > OUTPUT
test_expect_equal_file cworth.msg-ids OUTPUT
test_begin_subtest "combine regexp mid and subject"
notmuch search subject:/-C/ and mid:/y..m/ | notmuch_search_sanitize > OUTPUT
cat <<EOF > EXPECTED
thread:XXX 2009-11-18 [1/2] Carl Worth| Jan Janak; [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread)
EOF
test_expect_equal_file EXPECTED OUTPUT
lib: Add regexp expansion for for tags and paths From a UI perspective this looks similar to what was already provided for from, subject, and mid, but the implementation is quite different. It uses the database's list of terms to construct a term based query equivalent to the passed regular expression.
2017-03-08 03:32:26 +01:00
test_begin_subtest "unanchored tag search"
notmuch search tag:signed or tag:inbox > EXPECTED
notmuch search tag:/i/ > OUTPUT
test_expect_equal_file EXPECTED OUTPUT
notmuch tag +testsi '*'
test_begin_subtest "anchored tag search"
notmuch search tag:signed > EXPECTED
notmuch search tag:/^si/ > OUTPUT
test_expect_equal_file EXPECTED OUTPUT
lib: regexp matching in 'subject' and 'from' the idea is that you can run % notmuch search subject:/<your-favourite-regexp>/ % notmuch search from:/<your-favourite-regexp>/ or % notmuch search subject:"your usual phrase search" % notmuch search from:"usual phrase search" This feature is only available with recent Xapian, specifically support for field processors is needed. It should work with bindings, since it extends the query parser. This is easy to extend for other value slots, but currently the only value slots are date, message_id, from, subject, and last_mod. Date is already searchable; message_id is left for a followup commit. This was originally written by Austin Clements, and ported to Xapian field processors (from Austin's custom query parser) by yours truly.
2017-02-27 03:34:20 +01:00
test_done
Reference in a new issue Copy permalink