- dovecot - dovecot.org

Error: Broken file dovecot-uidlist
by Sohin Vyacheslav 16 Feb '23

16 Feb '23

Hi All, In mail.log are exists error messages "Error: Broken file /data/mail/vhosts/domain.com/user(a)domain.com/Maildir/dovecot-uidlist line XX: Invalid data: for some email accounts. For example, dovecot-uidlist line 21197: Invalid data: When I open line 21197 in editor: # vim +21197 dovecot-uidlist 1750207 :1676454069.M698819P4439.mail-b,S=963716,W=976359 and then check this file size # ls -l /data/mail/vhosts/domain.com/user@domain.com/Maildir/cur/1676454069.M698819P4439.mail-b,S=963716,W=976359:2,S -rw------- 1 vmail vmail 963716 Feb 15 10:41 I see that size is the same: 963176 bytes. So what exactly mentioned in error message "dovecot-uidlist line 21197: Invalid data:"? -- Best wishes Sohin Vyacheslav

3 6

Corrupted sizes in cache once again
by Tim Evers 15 Feb '23

15 Feb '23

Hi, I run a fairly large Dovecot Installation (around 100k mailboxes) on several servers. gzip compression is on. Every once in a while I get the dreaded "cache corruption" messages in the log: Error: Corrupted record in index cache file /[redacted]/Maildir/dovecot.index.cache: UID 3868: Broken physical size in mailbox INBOX: read(zlib(/[redacted]/Maildir/cur/1674129792.M797543P21755.node2,S=8099,W=8276:2,)) failed: Cached message size smaller than expected (2877 < 8099, box=INBOX, UID=3868) Error: Corrupted record in index cache file /[redacted]/Maildir/dovecot.index.cache: UID 3875: Broken physical size in mailbox INBOX: read(zlib(/[redacted]/Maildir/cur/1674212201.M985809P29112.node2,S=13907,W=14121:2,)) failed: Cached message size smaller than expected (5533 < 8192, box=INBOX, UID=3875) The first entry shows 2877 (size on disk) vs. 8099 (real size unzipped, also in the filename: S=8099). The second entry shows 5533 (size on disk) vs. 8192 - this is not correct in any way. Size on disk is 13907 as noted in the filename. Both mails were delivered trough LMTP and retrieved by the POP3 service. Anyone with an idea what might be happening here? I've read all available info in the doc and in the previous discussions / bug reports, but nothing seems to match my case. And where does that 8192 come from - it looks suspicious? Version is 2.3.7.2 (Ubuntu 20.04) doveconf -n excerpt: protocol lmtp { mail_plugins = " old_stats mail_log notify zlib sieve quota" } protocol imap { mail_max_userip_connections = 50 mail_plugins = " old_stats mail_log notify zlib imap_old_stats quota imap_quota imap_sieve" } protocol pop3 { mail_max_userip_connections = 50 mail_plugins = " old_stats mail_log notify zlib" } Tim

5 12

Error: Corrupted index cache file
by Sohin Vyacheslav 15 Feb '23

15 Feb '23

Hi All, In mail.log exists an error messages "Error: Corrupted index cache file /data/mail/vhosts/domain.com/user(a)domain.com/Maildir/dovecot.index.cache: invalid record size" for some accounts. I tried these steps to fix it: 1. Running commands # doveadm -D -v index -u user(a)domain.com INBOX # doveadm -v force-resync -u user(a)domain.com INBOX after this I saw that the following files are updated in ../Maildir/: dovecot.index.cache dovecot.index.log dovecot-uidlist But after some time error messages occurred again. 2. Added NFS mount option 'nordirplus' to /etc/fstab with remount /data partition, add 'mmap_disable = yes' and 'mail_fsync = always' to 10-mail.conf as advised on https://doc.dovecot.org/configuration_manual/nfs/ (because /data partition mounted via NFS); 3. Renaming 3 files with/without Dovecot service pre-stop. # mv dovecot.index old.dovecot.index # mv dovecot.index.cache old.dovecot.index.cache # mv dovecot.index.log old.dovecot.index.log Unfortunately, all done steps didn't fix issue. What is the proper way to fix corrupted index cache errors? dovecot package: 1:2.2.33.2-1ubuntu4.8 (Ubuntu-18.04). p.s. for some of email accounts also exists error messages "Error: Broken file ../Maildir/dovecot-uidlist line ###: Invalid data:" besides mentioned cache error. -- Best wishes Sohin Vyacheslav

4 7

NFS and performances
by tomate aceite 15 Feb '23

15 Feb '23

Hello, i have some question about NFS, dovecot director, and imap settings. I was reading all dovecot documentation ad mail lists, but some aspect are not clear to me. I am looking for performance / tunning my infra to work in a more efficient way because we experiences some issues some days ago. This is my infra: I got an infra with 2 dovecot-directors and 3 imap backend. I got all the emails stored in a common NFS share filer to all the imap nodes. ( Index are locally stored in each imap node.) My NFS mount options: (0)#: nfsstat -m /data/mail from myipaddress:/export/mail/maildirs > Flags: > rw,nosuid,noexec,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.205,mountvers=3,mountport=20048,mountproto=tcp,local_lock=none,addr=10.0.0.205 > *Questions*: ( https://wiki.dovecot.org/PerformanceTuning , https://doc.dovecot.org/configuration_manual/nfs/ >> i am following these steps ) 1) Is my NFS correct setup with the mount options well optimized ? Not sure if someone is using the same flags that me or got a better recomendation to used. 2) Set *mmap_disable = yes ??? * >> This must be set to yes if you store indexes to shared filesystems. In my case i got them locally in each imap node not in NFS share folder. I got setup mmap_disable = no , is this correct? I think no is the correct option here with indexes locally. because i can read here: https://wiki1.dovecot.org/NFS > > >> High performance NFS setup with indexes on local disk (see below for > benefits): > > mmap_disable = no > 3) Set* mail_fsync = always ???* Documentation: https://wiki.dovecot.org/PerformanceTuning always > > Use fsync after all disk writes. > > Recommended for NFS to make sure there aren’t any delayed write()s. > 3.a) where i can setup this option *mail_fsync = always , *because i run doveconf -n in director, and imap nodes, and they are not showing nothing. 3.b) *In which node ? *Do i need to add the setting in dovecot.conf in *director node or in imap node or in both ?* Not sure if this is the correct way: This is an attemp of setup, not sure if is correct? 0)#: doveconf -n > # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.13 (cdd19fe3) > # OS: Linux 5.10.0-20-amd64 x86_64 Debian 11.6 > > mail_debug = yes > mail_fsync = always > mail_max_userip_connections = 20 > mail_nfs_storage = yes > mail_plugins = " notify mail_log" > mail_privileged_group = mail > > > > protocol lmtp { > mail_fsync = always > mail_plugins = " notify mail_log sieve mail_lua push_notification > push_notification_lua" > plugin { > ... > } > 4) Do not set *mail_nfs_index *or *mail_nfs_storage* (i.e. keep them as no) ????? First option make sense but the second one not. https://doc.dovecot.org/settings/core/#core_setting-mail_nfs_storage mail_nfs_storage >>> >>> Default: no >>> >>> Values: Boolean >>> >>> Flush NFS caches whenever it is necessary to do so. >>> >> > This setting should only be enabled if you are using multiple servers > on NFS. > So should be possible to enable this option *mail_nfs_storage = yes ?????* 4) I got this setting in *dovecot-sql.conf ( director ) * driver = mysql > connect = host=myserver.X dbname=maildb user=dovecot > password=XXXXXXXXXXXXXXXX > default_pass_scheme = SHA > password_query = select login as user, crypt_sha1 as password, home as > userdb_home, uid AS userdb_uid, gid AS userdb_gid, > concat(maildir,':INDEX=/data/indexes/',login) as userdb_mail from mailbox > left join aliases on aliases.systemid = aliasid where login = '%u' and > inactive = 0; > user_query = select home, maildir as mail, uid, gid from mailbox left join > aliases on aliases.systemid = aliasid where login = '%u' and inactive = 0; > iterate_query = select distinct login as user from mailbox; > I would like to implement * ITERINDEX* and probably *VOLATILEDIR *but not sure if this is the correct change or if i need to change something into the database. from: password_query = select login as user, crypt_sha1 as password, home as > userdb_home, uid AS userdb_uid, gid AS userdb_gid, > concat(maildir,':INDEX=/data/indexes/',login) as userdb_mail from mailbox > left join aliases on aliases.systemid = aliasid where login = '%u' and > inactive = 0; > to: password_query = select login as user, crypt_sha1 as password, home as > userdb_home, uid AS userdb_uid, gid AS userdb_gid, > concat(maildir,':INDEX=/data/indexes/',login,':ITERINDEX',':VOLATILEDIR=/tmp/%2.256Nu/%u',login) > as userdb_mail from mailbox left join aliases on aliases.systemid = aliasid > where login = '%u' and inactive = 0; > But not sure if is working correctly, * i checked debuging* in my imap node in this way: auth_verbose = yes > auth_verbose_passwords = no > auth_debug = yes > auth_debug_passwords = yes > mail_debug = yes > verbose_ssl = yes > This is the output log: Feb 15 09:32:53 my-imap-server dovecot: auth: Debug: sql(manolo1(a)mydomain,10.X.X.109,<eWEh7bj09N4KAjxt>): cache hit: {SHA}8376922a27e83b9eadcdec3596a70bf6c4db5730#011user=manolo1@mydomain #011userdb_home=/data/mail/tmydomain/users/manolo1#011userdb_uid=700#011userdb_gid=700#011userdb_mail=/data/mail/mydomain/users/manolo1/Maildir/:INDEX=/data/indexes/manolo1@mydomain :ITERINDEX:VOLATILEDIR=/tmp/3a/manolo1@mydomain *Not sure if is correct the output or should i need to change something into the database or this change only afect locally?* 5) *mailbox_list_index = yes ???* Mailbox list indexes can be used to optimize IMAP STATUS commands. They are > also required for IMAP NOTIFY extension to be enabled. > *It is this option necesary to add? in order to work with ITERINDEX option ?* https://www.mail-archive.com/dovecot@dovecot.org/msg75035.html ( i see your answer to this post ) 6) *What is the best way to do a maintenance in a backend?* The last time i flushed all the user in a director from a backend node to do a maintenance, after the maintenance done i rebalanced the director cluster, but the next day when user started tp connect we got high peak I/O , etc with a lot of problems, because new users in this backend node were re-indexes o creating new indexes. *Are there any way to do in a safer way? *Without loosing the hashed for the directors nodes. Thanks in advance!!!!!!!

3 2

Hide local IP from non delivery notifications
by Claudio Corvino 15 Feb '23

15 Feb '23

Hi everyone, I have an external MTA configured with Postfix that delivers email to an internal IMAP/LMTP Dovecot server configured to bind an LDAP to check if users exist. If the user does not exist, my MX sends a non delivery notification, and into the e-mail there's the local IP address of the Dovecot server. I can't find a way to remove or hide that IP, here is an example of line that contains the IP (with X) that I would like to hide/remove: *"The mail system<user@domain>: host hostname.domain.com <http://hostname.domain.com>[XXX.XXX.XXX.XXX] said: 550"* Thanks in advance! Cheers -- Claudio

5 6

crash when trying to upgrade from 2.0.x to 2.3.x
by Eugene M. Zheganin 15 Feb '23

15 Feb '23

Hello, I'm trying to migrate dovecot mail database from 2.0.9 on Linux to 2.3.20 on FreeBSD. I've rsync'd the entire database, grabbed all the configs, changed the paths and permission, but when trying to access the mail from Thunderbird I have reproducible crash in the logs: ===Cut=== Feb 14 17:12:10 it-r-support dovecot[5888]: auth-worker(6378): Debug: mysql(localhost): Finished query 'SELECT maildir, 1143 AS uid, 1143 AS gid FROM mailbox WHERE username = 'zheganin-em(a)alamics.ru' AND active = '1'' in 0 msecs Feb 14 17:12:10 it-r-support dovecot[5888]: auth-worker(6378): Debug: conn unix:auth-worker (uid=143): auth-worker<16>: sql(zheganin-em(a)alamics.ru,192.168.57.177,<oE2zjKn0FTXAqDmx>): Finished userdb lookup Feb 14 17:12:10 it-r-support dovecot[5888]: auth-worker(6378): Debug: conn unix:auth-worker (uid=143): auth-worker<16>: Finished Feb 14 17:12:10 it-r-support dovecot[5888]: auth: Debug: sql(zheganin-em(a)alamics.ru,192.168.57.177,<oE2zjKn0FTXAqDmx>): Finished userdb lookup Feb 14 17:12:10 it-r-support dovecot[5888]: auth: Debug: master userdb out: USER 1282408449zheganin-em(a)alamics.ru maildir=alamics.ru/zheganin-em/ uid=1143 gid=1143 auth_mech=PLAIN auth_token=15cceab95e1251ea97a8e5ff62c70d1b7a76c96f Feb 14 17:12:10 it-r-support dovecot[5888]: imap-login: Login: user=<zheganin-em(a)alamics.ru>, method=PLAIN, rip=192.168.57.177, lip=172.16.11.13, mpid=6405, session=<oE2zjKn0FTXAqDmx> Feb 14 17:12:10 it-r-support dovecot[5888]: imap(zheganin-em(a)alamics.ru)<6405><oE2zjKn0FTXAqDmx>: Panic: file array.c: line 10 (array_idx_modifiable_i): assertion failed: (idx < array->buffer->used / array->element_size) Feb 14 17:12:10 it-r-support dovecot[5888]: imap(zheganin-em(a)alamics.ru)<6405><oE2zjKn0FTXAqDmx>: Fatal: master: service(imap): child 6405 killed with signal 6 (core dumped) ===Cut=== There's an older thread here mentioning that probably doveadm force-resync would help, but it doesn't fix this. Is there something that could be done here ? I've also reported this in the FreeBSD bugtracker, but just in case decided to ask here too. Backtrace: ===Cut=== # lldb /usr/local/libexec/dovecot/imap --core imap-6404.core (lldb) target create "/usr/local/libexec/dovecot/imap" --core "imap-6404.core" Core file '/var/cores/imap-6404.core' (x86_64) was loaded. (lldb) bt * thread #1, name = 'imap', stop reason = signal SIGABRT * frame #0: 0x000000001151133a libc.so.7`__sys_thr_kill + 10 frame #1: 0x0000000011489c74 libc.so.7`__raise + 52 frame #2: 0x000000001153b109 libc.so.7`abort + 73 frame #3: 0x000000001137a336 libdovecot.so.0`___lldb_unnamed_symbol5855 + 150 frame #4: 0x0000000011378ad1 libdovecot.so.0`___lldb_unnamed_symbol5844 + 49 frame #5: 0x0000000011379881 libdovecot.so.0`___lldb_unnamed_symbol5846 + 33 frame #6: 0x0000000011378ceb libdovecot.so.0`i_panic + 187 frame #7: 0x000000001136a514 libdovecot.so.0`array_idx_modifiable_i + 100 frame #8: 0x0000000012054a78 lib20_fts_plugin.so`fts_user_autoindex_exclude + 72 frame #9: 0x0000000012051b3b lib20_fts_plugin.so`fts_mailbox_allocated + 267 frame #10: 0x000000001113ad0b libdovecot-storage.so.0`hook_mailbox_allocated + 155 frame #11: 0x0000000011134491 libdovecot-storage.so.0`mailbox_alloc + 449 frame #12: 0x0000000001044311 imap`cmd_select_full + 353 frame #13: 0x000000000104cd92 imap`command_exec + 178 frame #14: 0x000000000104b64a imap`___lldb_unnamed_symbol1265 + 58 frame #15: 0x000000000104b7b6 imap`___lldb_unnamed_symbol1265 + 422 frame #16: 0x000000000104a151 imap`client_handle_input + 465 frame #17: 0x00000000010488a8 imap`client_input + 120 frame #18: 0x00000000113936df libdovecot.so.0`io_loop_call_io + 111 frame #19: 0x0000000011396214 libdovecot.so.0`io_loop_handler_run_internal + 388 frame #20: 0x0000000011393dea libdovecot.so.0`io_loop_handler_run + 378 frame #21: 0x0000000011393c28 libdovecot.so.0`io_loop_run + 88 frame #22: 0x00000000112f6208 libdovecot.so.0`master_service_run + 24 frame #23: 0x000000000105af67 imap`main + 1383 frame #24: 0x000000000103c11d imap`_start + 269 (lldb) ===Cut=== Same backtrace in GDB: ===Cut=== # gdb /usr/local/libexec/dovecot/imap --core imap-6404.core GNU gdb (GDB) 12.1 [GDB v12.1 for FreeBSD] Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd13.1". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/local/libexec/dovecot/imap... (No debugging symbols found in /usr/local/libexec/dovecot/imap) [New LWP 100485] warning: Could not load shared library symbols for [vdso]. Do you need "set solib-search-path" or "set sysroot"? Core was generated by `dovecot/imap'. Program terminated with signal SIGABRT, Aborted. Sent by thr_kill() from pid 6404 and user 1143. #0 0x000000001151133a in thr_kill () from /lib/libc.so.7 (gdb) bt full #0 0x000000001151133a in thr_kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000011489c74 in raise () from /lib/libc.so.7 No symbol table info available. #2 0x000000001153b109 in abort () from /lib/libc.so.7 No symbol table info available. #3 0x000000001137a336 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x0000000011378ad1 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x0000000011379881 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x0000000011378ceb in i_panic () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x000000001136a514 in array_idx_modifiable_i () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x0000000012054a78 in fts_user_autoindex_exclude () from /usr/local/lib/dovecot/lib20_fts_plugin.so No symbol table info available. #9 0x0000000012051b3b in fts_mailbox_allocated () from /usr/local/lib/dovecot/lib20_fts_plugin.so No symbol table info available. #10 0x000000001113ad0b in hook_mailbox_allocated () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x0000000011134491 in mailbox_alloc () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x0000000001044311 in cmd_select_full () No symbol table info available. #13 0x000000000104cd92 in command_exec () No symbol table info available. #14 0x000000000104b64a in ?? () No symbol table info available. #15 0x000000000104b7b6 in ?? () No symbol table info available. #16 0x000000000104a151 in client_handle_input () No symbol table info available. #17 0x00000000010488a8 in client_input () No symbol table info available. #18 0x00000000113936df in io_loop_call_io () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x0000000011396214 in io_loop_handler_run_internal () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x0000000011393dea in io_loop_handler_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #21 0x0000000011393c28 in io_loop_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. --Type <RET> for more, q to quit, c to continue without paging-- #22 0x00000000112f6208 in master_service_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #23 0x000000000105af67 in main () No symbol table info available. (gdb) ===Cut=== Thanks. Eugene.

2 3

Feature Request: login_trusted_networks to take FQDN
by Sean Gallagher 15 Feb '23

15 Feb '23

In a previous post to this list I described a problem I was having validating client certificates on inet_listener lmtp connections. Subject: "Please Help: Dovecot ssl_ca selection based on remote IP address filtering not working." The problem there was that Dovecot does not "inspect" the subject name on the client certificate on LMTP connections. As such Any valid certificate will pass. In this context "valid" means the same as OpenSSL SSL_set_verify( ,SSL_VERIFY_PEER, ). I.e. the certificate chain is well formed and can be traced back to a trusted root. It does not say anything about the peer's identity. I propose here, that the "login_trusted_networks" setting be allowed to take a domain name - possibly with wildcards. Then the name on the client certificate could be checked against login_trusted_networks in much the same way that web browsers work. If you tell your web browser that you want to connect to www.example.com, the browser will check that the server's certificate matches "www.example.com". In the present case, if you tell Dovecot (through the login_trusted_networks setting) to allow connections from "smtp.example.com", then Dovecot could check the name on the client's certificate matches "smtp.example.com". More generally, example.com could issue client certificates with names matching "*.mua.example.com". Then you could tell Dovecot to allow connections from "*.mua.example.com" through the login_trusted_networks setting. These usages could largely replace the IP host and CIDR subnet usages currently allowed in the login_trusted_networks setting but both could exist side by side. Of course, more elaborate schemes could be devised involving database lookups, but the outlined proposal would be relatively easy to implement and cover a good majority of use cases. The alternative is to force the use of application-specific certificate authorities, or just ignore it and hope that no one knows how to spoof network traffic. That's My two cents... Sean. -- This email has been checked for viruses by AVG antivirus software. www.avg.com

1 0

IMAP folder subscription on Apple/Mac Mail
by Alex 13 Feb '23

13 Feb '23

Hi, We have a user with an iPhone who can't see their maildir mailboxes on dovecot-2.3.20 on fedora37. I've created a "subscriptions" file with all maildir mailboxes with the proper format (converted from mboxes using mb2md), but the user can only see their inbox. It appears this may be a known problem? https://wiki.dovecot.org/Clients Perhaps it's a problem with my "subscriptions" file, despite being able to see all mailboxes using K-9 on my android? Ideas for troubleshooting would be greatly appreciated.

1 1

Re: Pigeonhole Sieve Vacation Reply-To peculiarity with inbound AWS-SES
by Dr. Rolf Jansen 11 Feb '23

11 Feb '23

> Am 08.02.2023 um 20:03 schrieb Michael Peddemors <michael(a)linuxmagic.com>: > > Dovecot vacation message issues.. > Tough for any system to do correctly. > >> The problem here is that inbound mails from third parties utilizing AWS-SES come in with an unpersonalized envelope address and SES takes returns to this as bounce messages and changes the body's From: to „MAILER-DAEMON(a)xx-zzzz-1.amazonses.xn--com-5o0a, which is not even our MAILER-DAEMON but the one of the receiver of our reply. So the receiver gets no chance to know from the headers the identity of whom replied - he may assume it from the context the actual message, though. > > We addressed this by NOT returning vacation messages to systems that don't use 'proper' values in the MAIL FROM.. Eg Mailing Lists, Sender Rewrite schemes, and a slurry of other rules. Who is we? Your organization or the Pigeonhole developers? Actually, the question is, whether this is addressed somewhere in Pigeonhole’s code already? > But the problem is that if you are using the header From, or Reply-To etc, it's too easy to be sending to forged email addresses. > > Vacation bombing attacks for instance.. You got a point here, and of course I want to prevent this. > Now, there are legitimate cases of the MAIL FROM and header from not aligning, so it is best to send to the MAIL FROM addresses.. IF you don't send it to certain MAIL FROM formats, usually by not responding to anything with mailing list identifiers, auto-suppress headers, and a few others, you only end up with clean MAIL FROM to respond to. From the point of the view of our industrial customers, who are operating processes with our chemicals, this consideration is irrelevant. If they inform a production issue by mail to the responsible service technician, they expect an immediate response, since a production stop is unacceptable. OoO notices play a role here, because we would inform alternative addresses and fone numbers for attending the support case. That said, with Pigeonhole, we are almost there. > But if you have an example that is particularly bothering you, and represents your problem, we can walk through that as an example. I send an email from an account of a mail server (Postfix/Dovecot - outbound relay SES) running on an AWS-EC2 instance in São Paulo (Brazil) to another mail address of mine of a mail server (Postfix/Dovecot direct MX) on an AWS-EC2 instance in Frankfurt Germany, and here the Pigeonhole’s vacation reply is activated. In the following I changed my real mail address in Brazil to rolf(a)example.br and the real one in Germany to rolf(a)example.de: The Point of view of the both involved Postfixes of said transactions are: Sender (Brazil): postfix/submission/smtpd[29165]: 97006638E8: client=201-68-62-42.dsl.telesp.net.br[201.68.62.42], sasl_method=CRAM-MD5, sasl_username=rolf(a)example.br postfix/cleanup[29182]: 97006638E8: message-id=<707A1777-8F09-4335-99BA-70C0367C129A(a)example.br> postfix/qmgr[2058]: 97006638E8: from=<rolf(a)example.br>, size=39626, nrcpt=1 (queue active) postfix/smtp[29183]: 97006638E8: to=<rolf(a)example.de>, relay=email-smtp.sa-east-1.amazonaws.com[52.67.192.29]:587, delay=0.37, delays=0.05/0.01/0.13/0.18, dsn=2.0.0, status=sent (250 Ok 010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000) Receiver (Germany): postfix/smtpd[86956]: connect from d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] postfix/smtpd[86956]: A44AB676E3: client=d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] postfix/cleanup[86957]: A44AB676E3: message-id=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000(a)sa-east-1.amazonses.com> postfix/qmgr[915]: A44AB676E3: from=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000(a)sa-east-1.amazonses.com>, size=40862, nrcpt=1 (queue active) postfix/local[86958]: A44AB676E3: passing <rolf(a)example.de> to transport=lmtp postfix/lmtp[86959]: A44AB676E3: to=<rolf(a)example.de>, relay=mail.example.de[private/dovecot-lmtp], delay=0.94, delays=0.83/0.01/0.04/0.07, dsn=2.0.0, status=sent (250 2.0.0 <rolf(a)example.de> +/goFGgl5mOwUwEAEYr/fg Saved) Sender of the OoO (Germany): postfix/qmgr[915]: 60242676F0: from=<rolf(a)example.de>, size=1177, nrcpt=1 (queue active) postfix/cleanup[86957]: 60242676F0: message-id=<dovecot-sieve-1676027240-343352-0(a)example.de> postfix/pickup[62932]: 60242676F0: uid=xxxx from=<rolf(a)example.de> postfix/smtp[86963]: 60242676F0: to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000(a)sa-east-1.amazonses.com>, relay=email-smtp.eu-central-1.amazonaws.com[3.74.180.161]:587, delay=0.31, delays=0.01/0.02/0.13/0.15, dsn=2.0.0, status=sent (250 Ok 010701863b022070-bb3e4541-8306-4438-b649-8879ec8ff666-000000) Receiver of the OoO notice (Brazil): postfix/smtpd[29184]: connect from d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] postfix/smtpd[29184]: 5F1346394E: client=d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] postfix/cleanup[29182]: 5F1346394E: message-id=<010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000000(a)sa-east-1.amazonses.com> postfix/qmgr[2058]: 5F1346394E: from=<>, size=3150, nrcpt=1 (queue active) postfix/local[29185]: 5F1346394E: passing <rolf(a)example.br> to transport=lmtp postfix/lmtp[29186]: 5F1346394E: to=<rolf(a)example.br>, relay=mail.example.br[private/dovecot-lmtp], delay=0.05, delays=0.01/0.01/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0 <rolf(a)example.br> RRZHGWwl5mMDcgAAjZNhtw Saved) The headers of the received OoO message (I removed the DKIM stuff for clarity) are: Return-Path: <> Delivered-To: rolf(a)example.br Received: from mail.example.br by example.br with LMTP id RRZHGWwl5mMDcgAAjZNhtw (envelope-from <>) for <rolf(a)example.br>; Fri, 10 Feb 2023 08:07:24 -0300 Received: from d215-244.smtp-out.sa-east-1.amazonses.com (d215-244.smtp-out.sa-east-1.amazonses.com [23.249.215.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.example.br (Postfix) with ESMTPS id 5F1346394E for <rolf(a)example.br>; Fri, 10 Feb 2023 08:07:24 -0300 (-03) DKIM-Signature: ... X-Sieve: Pigeonhole Sieve 0.5.20 (149edcf2) Message-ID: <010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000000(a)sa-east-1.amazonses.com> Date: Fri, 10 Feb 2023 11:07:23 +0000 From: MAILER-DAEMON(a)sa-east-1.amazonses.com To: rolf(a)example.br Subject: Out of Office - automatic notice In-Reply-To: <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000(a)sa-east-1.amazonses.com> References: <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000(a)sa-east-1.amazonses.com> Auto-Submitted: auto-replied (vacation) Precedence: bulk X-Auto-Response-Suppress: All MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Feedback-ID: 1.sa-east-1.E7DnIghDRHBvwjrx2QL73gyWAj+NYISxiq7bqOVD27E=:AmazonSES X-SES-Outgoing: 2023.02.10-23.249.215.244 Note how there is not a single reference of the real origin of the OoO message, rolf(a)example.de. Instead we see From: MAILER-DAEMON(a)sa-east-1.amazonses.com. The receiver of this message need to guess the real address from the outside context. The actual problem is that Pigeonhole responds to to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000(a)sa-east-1.amazonses.com> instead of to=<rolf(a)example.de>. Amazon SES of the original sender (not our relay) takes this as a bounce and changes the relevant headers. This is not under our control. Perhaps this could be mitigated by adding a Reply-To: header which informs the original sender. Then the receiver of the OoO noice would at least have a reference to what and whom this is about. Best regards Rolf > On 2023-02-08 13:21, Dr. Rolf Jansen wrote: >> Am 08.02.2023 um 12:57 schrieb Michael Peddemors <michael(a)linuxmagic.com>: >>> On 2023-02-08 07:47, Dr. Rolf Jansen wrote: >>>> Am 08.02.2023 um 12:23 schrieb Michael Peddemors <michael(a)linuxmagic.com>: >>>>> On 2023-02-07 14:57, Dr. Rolf Jansen wrote: >>>>>> Am 07.02.2023 um 19:49 schrieb Michael Peddemors <michael(a)linuxmagic.com>: >>>>>>> On 2023-02-07 13:33, jeremy ardley wrote: >>>>>>>> On 8/2/23 05:08, Dr. Rolf Jansen wrote: >>>>>>>>> Am 07.02.2023 um 17:54 schrieb jeremy ardley<jeremy(a)ardley.org>: >>>>>>>>>> On 7/2/23 22:01, Dr. Rolf Jansen wrote: >>>>>>>>>> To begin with, usage of Amazons Simple Email Service (SES) is mandatory for outgoing mails from AWS-EC2 instances. >>>>>>>>>> I run AWS-EC2 instances using postfix to send a receive mail. They can send direct assuming I set up suitable SPF, but they typically forward mail to another host under my control that is not on AWS to use as the outgoing server. >>>>>>>>> OK, that’s another use case. Many do use a full fledged Postfix/Dovecot installation. However the outgoing port 25 into the internet is blocked by AWS, and therefore we may either use a third party relay for our outgoing emails or may use SES, which is not that bad - except some unusual peculiarities. >>>>>>>>> >>>>>>>> This is off topic, but to be precise: >>>>>>>> - AWS throttles but does not block traffic to a *destination* port 25. >>>>>>>> - The *origin* port on the EC2 instance is an unprivilged port, not port 25 >>>>>>>> - If you use a relayhost you typically send from an unprivilged EC2 port to port 587 on the relay host >>>>>>>> Jeremy >>>>>>> >>>>>>> And if you DO intend to send out to port 25, remember to update the PTR on your EC2 instance. >>>>>> I respond off-list. Generally a good hint but it’s like trying to bath salts (https://en.wikipedia.org/wiki/Bath_salts) when you are taking a shower. >>>>>> https://aws.amazon.com/premiumsupport/knowledge-center/ec2-port-25-throttle/ >>>>>> Although the link text (ec2-port-25-throttle) suggests throttling, it is actually blocking - read the text. This cannot be removed from standard EC2 instances. >>>>>> This also corresponds to my experience. On the console of an AWS-EC2 instance, I entered just now: >>>>>> telnet cyclaero.com 25 >>>>>> This hangs until timeout. At the same time cyclaero.com received emails from the internet on port 25. >>>>>> Trying 3.126.110.101... >>>>>> telnet: connect to address 3.126.110.101: Operation timed out >>>>>> telnet: Unable to connect to remote host >>>>> >>>>> There are a lot of malicous or compromised EC2 instances..It is good they block port by default, and the one day delay before unblocking (you can request it) helps stop those short term spam instances. >>>>> >>>>> Now, wish they blocked port 587 by default ;) >>>>> >>>>> Or at least tracked the attacks. >>>>> >>>>> Strange though, that people still try to set up email on EC2, doesn't make sense, and it isn't the cheapest alternative.

5 8

Please Help: Dovecot ssl_ca selection based on remote IP address filtering not working.
by Sean Gallagher 11 Feb '23

11 Feb '23

Hi All, I hope this is the correct place to post this. If not, apologies. I am in the process of updating my company's email servers and am trying to put Dovecot into an Alpine Linux container, hosted on ProxMox. In my setup, local mail deliveries via LMTP can come from the MSA (for intra-company mail) and an MTA (for inbound mail). The LMTP data is sent over IPv4 and is protected by TLS. Client certificates are used to authenticate sources. I wanted to prevent Dovecot from accepting LMTP from any other source and the method I came up with to do this was to use filtering on the "remote" address. I would create a "Fake" CA that had never signed a certificate and use that to evaluate the certificate presented by incoming connections from addresses other than the two legitimate sources. This would cause all connections from "bad" addresses to be rejected. This seemed simple enough on paper but Dovecot just isn't playing along. It seems that no matter what I do, I cannot get "remote" filtering to switch the "ssl_ca" parameter. I have put together a test bed to demonstrate. I then use openssl s_client to attempt to connect to the test bed container. I've attached the instructions to build the test bed from scratch. In all cases s_client reports "Acceptable client certificate CA names" "CN = Fake CA" I.e. it want's a certificate from the CA that has never signed a certificate and never will. Why not do the obvious thing? I use LDAP to authenticate individual users and the user name will be reported over LMTP. So to use the normal authentication mechanisms to authenticate the connection source, I would need to do two LDAP lookups, one for the connecting machine and one for the mail recipient. It is not clear if this is possible from the documentation.. *Questions:* Is config file filtering broken or am I doing it wrong? Is it possible to provide a different "ssl_ca" based on the remote IP address? Is there an easier way to restrict LMTP connections to specific remote IP addresses? p.s. Filtering just doesn't work like it's supposed to. Someone should look at that (especially multi-level) .. See this: https://dovecot.org/pipermail/dovecot/2016-June/104770.html Any help would be greatly appreciated. I am at my wits' end with this. -- This email has been checked for viruses by AVG antivirus software. www.avg.com

1 1