I would like to create a sieve rule where I do a regex match on ALL headers, not a specific header. I've been trying to find out the proper syntax, but haven't found anything.
Is this possible, and if so, how do I write it?
Thanks, Shawn
On 8 Apr 2019, at 16:35, Shawn Heisey via dovecot dovecot@dovecot.org wrote:
I would like to create a sieve rule where I do a regex match on ALL headers, not a specific header.
This is a really bad idea. Headers can be quite long, contain data that you do not have control over, and checking all headers will be very expensive and may leave you open to various regex attacks.
https://en.wikipedia.org/wiki/ReDoS
I don't know if it is possible, but I certainly wouldn't do it.
-- what was supposed to be so special about a full moon? It was only a big circle of light. And the dark of the moon was only darkness. But half-way between the two, when the moon was between the worlds of light and dark, when even the moon lived on the edge... maybe then a witch could believe in the moon. --Witches Abroad
On 4/8/2019 4:55 PM, @lbutlr via dovecot wrote:
On 8 Apr 2019, at 16:35, Shawn Heisey via dovecot dovecot@dovecot.org wrote:
I would like to create a sieve rule where I do a regex match on ALL headers, not a specific header.
This is a really bad idea. Headers can be quite long, contain data that you do not have control over, and checking all headers will be very expensive and may leave you open to various regex attacks.
I want to catch any email where a specific IP address appears in any header. I do not know what header it might appear in - that could vary widely depending on what email account is being used to send the message.
This will appear in exactly one sieve script (the one for my mailbox), and I will be in complete control of the regex used, so the regular expression denial of service is extremely unlikely.
I'm already potentially vulnerable to that because I have a handful of external users on my mail server and they can create whatever sieve scripts they want via the managesieve service. Thankfully all of those people are pretty trustworthy folks.
Thanks, Shawn
On 8 Apr 2019, at 17:16, Shawn Heisey via dovecot dovecot@dovecot.org wrote:
On 4/8/2019 4:55 PM, @lbutlr via dovecot wrote:
On 8 Apr 2019, at 16:35, Shawn Heisey via dovecot dovecot@dovecot.org wrote:
I would like to create a sieve rule where I do a regex match on ALL headers, not a specific header. This is a really bad idea. Headers can be quite long, contain data that you do not have control over, and checking all headers will be very expensive and may leave you open to various regex attacks.
I want to catch any email where a specific IP address appears in any header. I do not know what header it might appear in - that could vary widely depending on what email account is being used to send the message.
Really? Where outside the Received headers do IPs appear in your email headers?
But sure, you can come up with a myriad of reasons why you might want to do it. It doesn't make it a good idea.
https://wiki.dovecot.org/Pigeonhole/Sieve
That lists all the supported extensions. I don't see anything that would work for all headers, so I don't think it is possible, but I'll defer to anyone else more knowledgable who comes along.
-- I find your lack of faith disturbing.
On Mon, 8 Apr 2019, @lbutlr wrote:
Really? Where outside the Received headers do IPs appear in your email headers?
Well, let's see. Running a rough grep on 270-message spam folder
# grep -E '^[-A-Za-z0-9]+:.*[^.0-9]{0,1}[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ~/mail/FN | sort -u -k1,1 -t:
Authentication-Results: spf=fail (sender IP is 52.233.28.167)
List-Help: <http://121.242.224.101/lists/admin/?p=preferences&uid=fb545e011f371409028a40346e99f6ff>
List-Subscribe: <http://121.242.224.101/lists/admin/?p=subscribe>
List-Unsubscribe: <http://121.242.224.101/lists/admin/?p=unsubscribe&uid=fb545e011f371409028a40346e99f6ff&jo=1>
Message-ID: <0.0.8.0.1D4BD9273731DDA.4A40C20@scotiabank-ses.com>
Received: from sonic308-11.consmr.mail.ne1.yahoo.com (sonic308-11.consmr.mail.ne1.yahoo.com [66.163.187.34])
Received-SPF: pass (google.com: domain of notification@facebookmail.com designates 66.220.155.142 as permitted sender) client-ip=66.220.155.142;
X-Cyberoam-smtpxy-version: 1.0.6.3
X-EN-OrigIP: 190.5.95.101
X-MDRemoteIP: 116.206.165.50
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=2396b2c2-187d-4b86-8827-064ef261b437;Ip=[52.233.28.167];Helo=[[10.0.0.5]]
X-Mailer: DM Pro6 [GB - 6.1.6.52]
X-Originating-IP: 18.213.73.4
X-PHP-Script: www.bi.cz/index.php for 77.51.76.64
X-Rambler-User: Wareafrequentv987210@rambler.ru/45.74.4.160
X-SENDER-IP:113.227.63.115
X-SES-Outgoing: 2019.01.09-54.240.4.4
X-SentFromServer: 207.8.96.25
X-Source-IP: 196.42.37.18
X-Source-Sender: ppp-196-42-37-18.coqui.net ([10.8.4.39]) [196.42.37.18]:60668
X-SourceIP: 197.211.63.193
X-Spam-Summary: 30,2,0,,d41d8cd98f00b204,petroleum@scientmed.com,:,RULES_HIT:2:10:41:355:379:541:542:560:960:962:967:969:973:982:988:989:1021:1029:1155:1189:1221:1260:1263:1308:1309:1313:1314:1345:1381:1436:1437:1516:1517:1518:1535:1575:1587:1588:1589:1592:1594:1691:1730:1776:1792:2198:2199:2525:2526:2527:2528:2553:2559:2562:2682:2685:2693:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3194:3353:3362:3740:3865:3866:3867:3868:3870:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4120:4321:4361:4379:4425:4552:5007:6117:6631:6658:6678:6684:7628:7688:7903:8603:8957:9025:9163:9388:9868:10026:10049:10128:10197:10848:10919:11656:11658:11914:12043:12050:12438:12457:12663:12895:13138:13139:13174:13229:13231:13439:14096:14659:21080:21212:21324:21325:21433:21450:21451:21499:21524:21627:21819:30018:30021:30022:30026:30054:30056:30062:30070:30090,0,RBL:125.99.156.6:@scientmed.com:.lbl8.mailshell.net-62.6.117.100 64.201.201.201,CacheIP:non
X-SpamExperts-Username: 89.42.221.17
X-TCPREMOTEIP: 115.97.184.63
X-VirtualServer: Transactional, sv016071.hosted.strongview.com, 172.18.101.71
x-originating-ip: [46.252.109.60]
x-pmwin-version: 3.1.3.0, Antivirus-Engine: 3.74.1, Antivirus-Data: 5.60That's a *small* sample of where IPs can show up.
A non-trivial IP pattern is probably more likely to be missed by a selective header match than false matched by a non-selective header search. However, it's worth double checking what you're matching against (e.g. Subject: so that you can mention this IP without mangling your subject title).
Joseph Tam jtam.home@gmail.com
On 9 Apr 2019, at 17:07, Joseph Tam via dovecot dovecot@dovecot.org wrote:
On Mon, 8 Apr 2019, @lbutlr wrote:
Really? Where outside the Received headers do IPs appear in your email headers?
Well, let's see. Running a rough grep on 270-message spam folder
# grep -E '^[-A-Za-z0-9]+:.*[^.0-9]{0,1}[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ~/mail/FN | sort -u -k1,1 -t: Authentication-Results: spf=fail (sender IP is 52.233.28.167)
a better IP regex: \b(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b
or
\b(?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b
I checked a quarter million messages and all listed a domain in Authentication-result, not an IP, though to be fair some had matches like:
Authentication-Results: hostedmail.net.au; auth=pass smtp.auth=110.232.141.226@out.zuver.net.au
List-Help: http://121.242.224.101/lists/admin/?p=preferences&uid=fb545e011f371409028a40346e99f6ff List-Subscribe: http://121.242.224.101/lists/admin/?p=subscribe List-Unsubscribe: http://121.242.224.101/lists/admin/?p=unsubscribe&uid=fb545e011f371409028a40346e99f6ff&jo=1
I am quite sure I have never seen a mailing list with IPs in those header. But I did check. None. Some false matches along the lines of
List-Unsubscribe: mailto:465465-4654-117-162.xxx.xxx.xxx-gmail@abuse.*munged*.com
Message-ID: 0.0.8.0.1D4BD9273731DDA.4A40C20@scotiabank-ses.com
Not an IP
Received: from sonic308-11.consmr.mail.ne1.yahoo.com (sonic308-11.consmr.mail.ne1.yahoo.com [66.163.187.34])
Yes, I said received.
Received-SPF: pass (google.com: domain of notification@facebookmail.com designates 66.220.155.142 as permitted sender) client-ip=66.220.155.142;
Yes, this is also a Received header, but sure.
<many X-headers>
And I wouldn't count any x-header.
Regardless, I don't think you can search all headers.
-- 'You know the worst of it?' said Rincewind. 'Oook?' 'I don't even remember walking under a mirror.' --Mort
You could send the message to an external program, which checks if the specific IP address is mentioned.
You need to manually activate the pigeonhole extension vnd.dovecot.execute: https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms and https://wiki2.dovecot.org/Pigeonhole/Sieve
This page links to an RFC describing the action/test execute. https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bos...
Sieve Test:
require [ "vnd.dovecot.execute" ]
if execute :pipe "check4ip.sh" { #actions to process, when check4ip.sh found specific IP action; }
/in/dovecot/defined/path/by/sieve_execute_bin_dir/check4ip.sh
#!/bin/bash
IP='127.0.0.1'
#take standard input and save a grep for $IP as $M
M=cat | grep "$IP"
#exit with success (exit 0) only if $M is not empty if [-z "$M" ]; then exit 1 fi exit 0
A test named pipe exists too, but it doesn't seem to be implemented in pigeonhole? http://mailutils.org/manual/html_chapter/Sieve-Language.html :
Test: pipe [:envelope] [:header] [:body] [:exit code(number)] [:signal code(number)] command(string)
Synopsis:
require "test-pipe";
if pipe command { … }
Description: The pipe test executes a shell command specified by its argument and pipes the entire message (including envelope) to its standard input. When given, tags :envelope, :header, and :body control what parts of the message to pipe to the command.
In the absence of the :exit tag, the test returns true if the command exits with code 0. If :exit is given, the test returns true if the command exits with code equal to its argument.
The :signal tag determines the result of the test in case if the program exits on signal. By default, the test returns false. If :signal is given and the number of signal which caused the program to terminate matches its argument, the test returns true.
On Tue, 2019-04-09 at 18:09 -0600, @lbutlr via dovecot wrote:
On 9 Apr 2019, at 17:07, Joseph Tam via dovecot dovecot@dovecot.org wrote:
On Mon, 8 Apr 2019, @lbutlr wrote:
Really? Where outside the Received headers do IPs appear in your email headers?
Well, let's see. Running a rough grep on 270-message spam folder
# grep -E '^[-A-Za-z0-9]+:.*[^.0-9]{0,1}[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ~/mail/FN | sort -u -k1,1 -t: Authentication-Results: spf=fail (sender IP is 52.233.28.167)
a better IP regex: \b(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b
or
\b(?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b
I checked a quarter million messages and all listed a domain in Authentication-result, not an IP, though to be fair some had matches like:
Authentication-Results: hostedmail.net.au; auth=pass smtp.auth=110.232.141.226@out.zuver.net.au
List-Help: http://121.242.224.101/lists/admin/?p=preferences&uid=fb545e011f371409028a40346e99f6ff List-Subscribe: http://121.242.224.101/lists/admin/?p=subscribe List-Unsubscribe: http://121.242.224.101/lists/admin/?p=unsubscribe&uid=fb545e011f371409028a40346e99f6ff&jo=1
I am quite sure I have never seen a mailing list with IPs in those header. But I did check. None. Some false matches along the lines of
List-Unsubscribe: mailto:465465-4654-117-162.xxx.xxx.xxx-gmail@abuse.*munged*.com
Message-ID: 0.0.8.0.1D4BD9273731DDA.4A40C20@scotiabank-ses.com
Not an IP
Received: from sonic308-11.consmr.mail.ne1.yahoo.com (sonic308-11.consmr.mail.ne1.yahoo.com [66.163.187.34])
Yes, I said received.
Received-SPF: pass (google.com: domain of notification@facebookmail.com designates 66.220.155.142 as permitted sender) client-ip=66.220.155.142;
Yes, this is also a Received header, but sure.
<many X-headers>
And I wouldn't count any x-header.
Regardless, I don't think you can search all headers.
participants (4)
-
＠lbutlr
-
Joseph Tam
-
Martin Johannes Dauser
-
Shawn Heisey