Hello!

Recently I've faced with strange issue. So, I want to filter some mails which do not contain cyrillic symbols. I would not like receive email on foreign language except russian and I'm using rule below, but it does not work when text of mail has unicode symbol u2019 or ’ - right single quotation mark.

|require ["body","regex"];|

|# rule:[Regexp test]|

|if not body :text :regex ".*[аАбБвВгГдДеЕёЁжЖзЗиИйЙкКлЛмМнНоОпПрРсСтТуУфФхХцЦчЧшШщЩъЪыЫьЬэЭюЮяЯ].*"|

|{|

|    discard;|

|    stop;|

|}|

I checked this behavior on different versions of dovecot and pigeonhole, and it was the same in all cases. If I change u2019 to, for instance to apostrophe, it begins work as I expect - the mail discarded.

Below some information which was got from sieve-test utility. The text consist of only one phrase *Test’test*.

Check string *Test’test *- without u2019, using apostrophe

root@a4e4b17d33a1:/srv/mail# tail -2 1586937347.M574837P24389.vps.kveri.ru\,S\=1904\,W\=1944\:2\,S

Test'test

sieve-test output

• Script metadata (block: 0):

class = file class.version = 0 location = /srv/mail/roundcube.sieve

• Required extensions (block: 1):

  0: body (id: 18) 1: regex (id: 13)

• Main program (block: 2):

Address   Line  Code 00000000:       DEBUG BLOCK: 3 00000001:       EXTENSIONS [2]: 00000002:         body 00000004:         regex 00000006:    3: BODY 00000007:         BODY-TRANSFORM: TEXT 0000000b:         match type: regex 0000000d:         key list: STR[138] ".*[аАбБвВгГдДеЕёЁжЖзЗиИйЙкКлЛмМнНоОпПрРс... 0000009b:    3: JMPTRUE 6 [000000a2] 000000a0:    5: DISCARD 000000a1:    6: STOP 000000a2:    6: [End of code]

Performed actions:

 * discard

Implicit keep:

  (none)

In this case the rule works as I expect

The second test *Test’test* with ’ instead of apostrophe

root@a4e4b17d33a1:/srv/mail# tail -2 1586937347.M574837P24389.vps.kveri.ru\,S\=1904\,W\=1944\:2\,S

Test’test

sieve-test output

• Script metadata (block: 0):

class = file class.version = 0 location = /srv/mail/roundcube.sieve

• Required extensions (block: 1):

  0: body (id: 18) 1: regex (id: 13)

• Main program (block: 2):

Address   Line  Code 00000000:       DEBUG BLOCK: 3 00000001:       EXTENSIONS [2]: 00000002:         body 00000004:         regex 00000006:    3: BODY 00000007:         BODY-TRANSFORM: TEXT 0000000b:         match type: regex 0000000d:         key list: STR[138] ".*[аАбБвВгГдДеЕёЁжЖзЗиИйЙкКлЛмМнНоОпПрРс... 0000009b:    3: JMPTRUE 6 [000000a2] 000000a0:    5: DISCARD 000000a1:    6: STOP 000000a2:    6: [End of code]

Performed actions:

  (none)

Implicit keep:

 * store message in folder: INBOX

In this case email "was located" into INBOX, but I expected it should be discarded. As I said this behavior does not depend on dovecot and pigeonhole version - I've tried dovecot 2.3.7, 2.2.30.x, 2.3.9.3 and 2.3.10, pigeonhole 0.5.7.2, 0.5.9 and 0.5.10, fresh install and working in docker container. The dovecot-sysreport was taken from the last one. What am I doing wrong? Is it pigeonhole bug or smth like that?