- dovecot - dovecot.org

lmtp panic with many recipients
by Olaf Hopp 11 May '18

11 May '18

Hi, I had an email with 58 recipients in the "To" and 13 in the "CC" Delivering it from exim to dovecot lmtp panics (see below) Panic: file smtp-address.c: line 533 (smtp_address_write): assertion failed: (smtp_char_is_qpair(*p)) # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.devel (61b47828) # OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final) Regards, Olaf May 8 10:01:52 irams2 dovecot: lmtp(17557): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= May 8 10:01:52 irams2 dovecot: lmtp(17557): Connect from local May 8 10:01:52 irams2 dovecot: lmtp(ms2test(a)irams2.ira.uka.de)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: Loading modules from directory: /usr/lib64/dovecot May 8 10:01:52 irams2 dovecot: lmtp(ms2test(a)irams2.ira.uka.de)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: Module loaded: /usr/lib64/dovecot/lib90_sieve_plugin.so May 8 10:01:52 irams2 dovecot: lmtp(ms2test(a)irams2.ira.uka.de)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: auth USER input: ms2test system_groups_user=ms2test uid=10852 gid=1602 home=/home/atis/ms2test May 8 10:01:52 irams2 dovecot: lmtp(ms2test(a)irams2.ira.uka.de)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: changed username to ms2test May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: Effective uid=10852, gid=1602, home=/home/atis/ms2test May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: maildir++: root=/home/atis/ms2test/Maildir, index=, indexpvt=, control=, inbox=/home/atis/ms2test/Maildir, alt= May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Pigeonhole version 0.5.devel (61b47828) initializing May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Using active Sieve script path: /home/atis/ms2test/.dovecot.sieve May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Using script storage path: /home/atis/ms2test/sieve May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Using Sieve script path: /home/atis/ms2test/.dovecot.sieve May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file script: Opened script `ingo' from `/home/atis/ms2test/.dovecot.sieve' May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Using Sieve script path: /etc/dovecot/sieve-master May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file script: Opened script `sieve-master' from `/etc/dovecot/sieve-master' May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Executed before user's personal Sieve script(1): /etc/dovecot/sieve-master May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Using the following location for user's Sieve script: /home/atis/ms2test/.dovecot.sieve May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: Mailbox <lmtp DATA local>: Opened mail UID=1 because: header Message-ID (Cache file is unusable) May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Opening script 1 of 2 from `/etc/dovecot/sieve-master' May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Loading script /etc/dovecot/sieve-master May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Script binary /etc/dovecot/sieve-master.svbin successfully loaded May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: binary save: not saving binary /etc/dovecot/sieve-master.svbin, because it is already stored May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Executing script from `/etc/dovecot/sieve-master.svbin' May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: Mailbox <lmtp DATA local>: Opened mail UID=1 because: header List-Id (Cache file is unusable) May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Panic: file smtp-address.c: line 533 (smtp_address_write): assertion failed: (smtp_char_is_qpair(*p)) May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xca92a) [0x7fd7b6d0a92a] -> /usr/lib64/dovecot/libdovecot.so.0(i_syslog_fatal_handler+0x33) [0x7fd7b6d0b003] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3ce61) [0x7fd7b6c7ce61] -> /usr/lib64/dovecot/libdovecot.so.0(+0x403fb) [0x7fd7b6c803fb] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_address_encode+0x29) [0x7fd7b6c80429] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x46892) [0x7fd7b4146892] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_match+0xbf) [0x7fd7b41475ff] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x4d9ec) [0x7fd7b414d9ec] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0x71) [0x7fd7b413c4b1] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7fd7b413d07b] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x522fb) [0x7fd7b41522fb] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_multiscript_run+0x5a) [0x7fd7b415252a] -> /usr/lib64/dovecot/lib90_sieve_plugin.so(+0x36ad) [0x7fd7b43bb6ad] -> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0xd2) [0x7fd7b7314942] -> dovecot/lmtp [local DATA](+0x6ffe) [0x7fd7b7756ffe] -> dovecot/lmtp [local DATA](lmtp_local_data+0x156) [0x7fd7b7757466] -> dovecot/lmtp [local DATA](cmd_data_continue+0x218) [0x7fd7b7756088] -> /usr/lib64/dovecot/libdovecot.so.0(+0x4ee3d) [0x7fd7b6c8ee3d] -> /usr/lib64/dovecot/libdovecot.so.0(+0x4f242) [0x7fd7b6c8f242] -> /usr/lib64/dovecot/libdovecot.so.0(+0x4f837) [0x7fd7b6c8f837] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_server_command_next_to_reply+0x3b) [0x7fd7b6c9212b] -> /usr/lib64/dovecot/libdovecot.so.0(+0x55a30) [0x7fd7b6c95a30] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7fd7b6d21785] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x95) [0x7fd7b6d218b5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd7b6d21a98] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fd7b6c9f493] -> dovecot/lmtp [loc May 8 10:01:52 irams2 dovecot: lmtp: Fatal: master: service(lmtp): child 17557 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) -- Karlsruher Institut für Technologie (KIT) ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik Dipl.-Geophys. Olaf Hopp - Leitung IT-Dienste - Am Fasanengarten 5, Gebäude 50.34, Raum 009 76131 Karlsruhe Telefon: +49 721 608-43973 Fax: +49 721 608-46699 E-Mail: Olaf.Hopp(a)kit.edu atis.informatik.kit.edu www.kit.edu KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.

3 8

Dovecot 2.2.36-rc1 and $HasAttachment
by Alessio Cecchi 10 May '18

10 May '18

Hi, I installed dovecot 2.2.36-rc1 and enabled: mail_attachment_detection_options = add-flags-on-save but when messages are delivery via dovecot-lda into mailbox (Maildir) keywords are not added. Only when the messages is open keywords ($HasAttachment or $HasNoAttachment) are added. From the wiki "attachments are detected and marked during save", delivery via LDA is "save"? For info, doveadm rebuild attachments works fine. Since 2.2.36 will be the last v2.2.x release I hope that this feature will be fixed since is very useful. Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 1

Error compiling with --with-storages=
by Andrea Gabellini 10 May '18

10 May '18

Hello, Usually I compile Dovecot with option --with-storages=maildir Starting with 2.2.30 and up to 2.2.36 RC this option cause the compilation to fail with this error: .libs/quota-imapc.o: In function `imapc_quota_refresh': quota-imapc.c:(.text+0x82d): undefined reference to `imapc_client_get_capabilities' quota-imapc.c:(.text+0x87d): undefined reference to `imapc_simple_context_init' quota-imapc.c:(.text+0x887): undefined reference to `imapc_simple_callback' quota-imapc.c:(.text+0x89d): undefined reference to `imapc_client_cmd' quota-imapc.c:(.text+0x8ba): undefined reference to `imapc_command_sendf' quota-imapc.c:(.text+0x8c2): undefined reference to `imapc_simple_run' quota-imapc.c:(.text+0xa2a): undefined reference to `imapc_storage_client_register_untagged' quota-imapc.c:(.text+0xa4d): undefined reference to `imapc_storage_client_register_untagged' quota-imapc.c:(.text+0xa8e): undefined reference to `imapc_simple_context_init' quota-imapc.c:(.text+0xa98): undefined reference to `imapc_simple_callback' quota-imapc.c:(.text+0xaae): undefined reference to `imapc_client_cmd' quota-imapc.c:(.text+0xacb): undefined reference to `imapc_command_sendf' quota-imapc.c:(.text+0xad3): undefined reference to `imapc_simple_run' collect2: ld returned 1 exit status Compiling without this option or with --with-storages=maildir,imapc works Thanks, Andrea -- ---------------------------------------------------------------- The most likely way for the world to be destroyed, most experts agree, is by accident. That's where we come in; We're computer professionals. We cause accidents. ---------------------------------------------------------------- Ing. Andrea Gabellini Email: andrea.gabellini(a)telecomitalia.sm Skype: andreagabellini Tel: (+378) 0549 886111 Fax: (+378) 0549 886188 Telecom Italia San Marino S.p.A. Via XXVIII Luglio, 212 - Piano -2 47893 Borgo Maggiore Republic of San Marino http://www.telecomitalia.sm

2 2

mmap failed messages in syslog
by Shawn Heisey 10 May '18

10 May '18

I'm getting mmap failed messages in syslog on version 1.2.15 for dovecot.index.cache files. This old dovecot version is installed with debian packages. The OS and all software is 64-bit. I've found info on this log message, with a suggested fix of raising vsz_limit ... but this seems to be a config option for dovecot 2.x, not 1.x. The file it complains about being unable to mmap is only 90MB, and the biggest cache file I have found on the system is only 180MB. The system has 4GB of total memory with about 1GB of that allocated to OS disk cache right now. The mmap functionality provided by the OS doesn't use *actual* memory, so the amount of memory I have should be irrelevant. Checking 'ulimit -a' with several users shows that vmemory is unlimited for all of them. How do I go about allowing docevot 1.x to use more virtual memory? I do have a migration planned to Ubuntu 16, where I can run a 2.x version, but that's not going to happen for a while. Thanks, Shawn

2 3

possible to disable dh_key/ssl-parameters.dat generation when only using ECDHE ciphers.
by Erik de Waard 09 May '18

09 May '18

Hi, I want to disable dh_key/ssl-parameters.dat entirely since i'm only using ECDHE ciphers. > # 2.2.34 (874deae): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.22 (22940fb7) # OS: Linux 4.9.0-6-amd64 x86_64 Debian 9.4 # Hostname: somehost.com auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 days auth_username_chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@#" default_client_limit = 1500 default_vsz_limit = 600 M disable_plaintext_auth = no info_log_path = /var/log/mail.log.info listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_max_userip_connections = 100 mail_privileged_group = mail mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { sieve_execute_bin_dir = /etc/dovecot/sieve-executables sieve_global_extensions = +vnd.dovecot.execute sieve_plugins = sieve_extprograms } protocols = imap lmtp service anvil { unix_listener anvil-auth-penalty { mode = 0600 } } service auth { user = root } service imap-login { client_limit = 6000 process_limit = 4 process_min_avail = 4 service_count = 0 vsz_limit = 600 M } service imap { client_limit = 1 process_limit = 1024 service_count = 50 } service lmtp { inet_listener lmtp { port = 24 } } ssl_cert = </etc/dovecot/dovecot.crt ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = " sieve" plugin { sieve = ~/filters.sieve sieve_after = /etc/dovecot/sieve/after.sieve sieve_before = /etc/dovecot/sieve/before.sieve } userdb { args = /etc/dovecot/dovecot-sql-lmtp.conf driver = sql name = } }

2 1

replicator: User listing returned failure
by Alexey 09 May '18

09 May '18

Hello. I've tried to configure TCP replication to slave server. But I got some errors... mx1:~# dovecot --version 2.2.27 (c0f36b0) From dovecot.log: May 07 19:27:41 auth-worker(34348): Warning: mysql: Query failed, retrying: Unknown column 'username' in 'field list' May 07 19:27:41 auth-worker(34348): Debug: sql(*): SELECT username, domain FROM users May 07 19:27:41 auth-worker(34348): Error: sql: Iterate query failed: Unknown column 'username' in 'field list' (using built-in default iterate_query: SELECT username, domain FROM users) May 07 19:27:41 auth-worker(34348): Debug: sql(*): SELECT id AS username, domain FROM users May 07 19:27:41 replicator: Error: User listing returned failure May 07 19:27:41 replicator: Error: listing users failed, can't replicate existing data mx1:~# doveadm replicator status Queued 'sync' requests 0 Queued 'high' requests 0 Queued 'low' requests 0 Queued 'failed' requests 0 Queued 'full resync' requests 0 Waiting 'failed' requests 2 Total number of known users 2 mx1:~# doveadm replicator status '*' username priority fast sync full sync success sync failed abuse(a)example.com none 00:03:01 01:06:36 - y admin(a)exmaple.com none 00:03:41 01:31:49 - y From slave: mx2:~# cat /etc/dovecot/conf.d/90-replication.conf # use tcp:hostname as the dsync target plugin { mail_replica = tcp:mx1 # use doveadm_port } As I can see nothing happens automatically. But mx2:~# doveadm -D sync -u abuse(a)example.com -d -N -l 30 -U successfully executed and getting mails from mx1. dovecot-sql.conf.ext: driver = mysql connect = host=localhost dbname=vmail user=sqlmail password=sqL_hidden033|TGPAS default_pass_scheme = SHA512-CRYPT password_query = SELECT id AS username, password, domain FROM users WHERE id = '%n' AND domain = '%d' AND active = 'Y' user_query = SELECT id AS username, uid, gid, home, concat('*:storage=', quota, 'M' ) as quota_rule FROM users WHERE id = '%n' AND domain = '%d' iterate_query = SELECT id AS username, domain FROM users Regards, Alexey mx1:~# dovecot -n # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-6-amd64 x86_64 Debian 9.4 auth_master_user_separator = * auth_verbose = yes auth_verbose_passwords = sha1 default_vsz_limit = 512 M doveadm_password = # hidden, use -P to show it doveadm_port = 994 hostname = mx1.example.com imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags imap_idle_notify_interval = 12 mins lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes log_path = /var/log/dovecot/dovecot.log mail_access_groups = mail mail_location = mdbox:~/mdbox:UTF-8 mail_plugins = " notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=100G quota_rule2 = Trash:storage=+10G quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve_before = /srv/vmail/dovecot_global_filter.sieve sieve_dir = ~/sieve sieve_global_extensions = +vnd.dovecot.duplicate } postmaster_address = postmaster(a)example.com protocols = " imap lmtp sieve" service aggregator { fifo_listener replication-notify-fifo { group = mail mode = 0660 user = dovecot } unix_listener replication-notify { group = mail mode = 0660 user = dovecot } } service auth { unix_listener auth-client { group = mail mode = 0660 } unix_listener auth-userdb { group = mail } } service dict { unix_listener dict { group = mail mode = 0660 user = dovecot } } service doveadm { inet_listener { port = 994 } } service lmtp { executable = lmtp -L unix_listener lmtp { group = mail mode = 0660 } } service managesieve-login { inet_listener sieve { port = 4190 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = mail mode = 0660 user = dovecot } } ssl = required ssl_cert = </etc/ssl/example.com/fullchain-combo.crt ssl_cipher_list = ALL:HIGH:!SSLv2:!SSLv3:!MEDIUM:!LOW:!EXP:!RC4:!MD5:!aNULL:@STRENGTH ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { log_path = /var/log/dovecot/lmtp.log mail_plugins = " notify replication sieve" } protocol lda { log_path = /var/log/dovecot/lda.log mail_plugins = " notify replication sieve" } protocol imap { mail_plugins = " notify replication" }

2 6

[sieve][regex] Matching multiple strings in the "Received" header
by Adi Pircalabu 08 May '18

08 May '18

Hi, I'm trying to use this expression in Sieve, but for some reason the filter doesn't work: require ["fileinto","regex"]; # rule:[gmail-outlook-yahoo-aol-friends] if header :regex "received" ".from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)." { fileinto "INBOX.gmail-hotmail-yahoo-aol-friends"; stop; } However, it's working fine with egrep: egrep ".from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)." * 1525657297.M401428P1459.host01.quick.net.au,S=10073,W=10275:2,S:Received: from APC01-PU1-obe.outbound.protection.outlook.com (mail-oln040092254061.outbound.protection.outlook.com [40.92.254.61]) Am I using Sieve correctly here? Is ".from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)." expression valid for Sieve? Or do I have to split it in an array as per https://wiki2.dovecot.org/Pigeonhole/Sieve/Examples#Flagging_or_Highlightin… example? Thanks, -- Adi Pircalabu

4 6

Released Pigeonhole v0.4.24.rc1 for Dovecot v2.2.36.rc1.
by Stephan Bosch 08 May '18

08 May '18

Hello Dovecot users, Here's a first release candidate for the upcoming Pigeonhole release for Dovecot v2.2.36. It only contains bugfixes for now. There's one experimental plugin still in the pipeline. Changelog v0.4.24: - Fix assert panics triggered by empty messages that are being forwarded using redirect or vnd.dovecot.report. This does not likely normally occur, but this is seen as a result of certain benign failures in object storage. - Make the length of the subject header for the vacation response configurable and enforce the limit in UTF-8 codepoints rather than bytes. The subject header for a vacation response was statically truncated to 256 bytes, which is too limited for multi-byte UTF-8 characters. - Sieve editheader extension: Fix assertion panic occurring when it is used to manipulate a message header with a very large header field. - Properly abort execution of the sieve_discard script upon error. Before, the LDA Sieve plugin attempted to execute the sieve_discard script when an error occurs. This can lead to the message being lost. - Fix the interaction between quota and the sieve_discard script. When quota was used together with a sieve_discard script, the message delivery did not bounce when the quota was exceeded. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.24… http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.24… Refer to http://pigeonhole.dovecot.org and the Dovecot wiki for more information. Have fun testing this release candidate and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

2.3.1 Replication is throwing scary errors
by Andy Weal 06 May '18

06 May '18

Hi all, New to the mailing lists but have joined up because of above */2.3.1 Replication is throwing scary errors /*Brief system configuration MX1 - Main Freebsd 11.1-Release-p9 Hosted on a Vultr VM in Sydney AU MTA = Postfix 3.4-20180401 Dovecot = 2.3.1 File system = ufs MX2 - Backup Freebsd 11.1-Release-p9 Running on bare metal - no VM or jails MTA = Postfix 3.4-20180401 Dovecot = 2.3.1 File system = ufs ( on SSD) */ /*Brief sequence of events 1. apx 10 days back upgraded both mx1 and mx2 to dovecot 2.3.1_2 from 2.3.0 (service dovecot stop, portmaster upgrade, service dovecot start) 2. both systems ran ok with no errors for 10 days. 3. Last night I shutdown mx2 and restarted it a few hours later 4. within minutes i was getting the following types of errors on mx2 May 06 12:56:29 doveadm: Error: Couldn't lock /var/mail/vhosts/example.net/user1/.dovecot-sync.lock: fcntl(/var/mail/vhosts/example.net/user1/.dovecot-sync.lock, write-lock, F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by pid 1960) Before i venture down the rabbit hole of fault finding and excess coffee consumption I was wondering if any of you had any updates on the problems discussed below. Cheers for now, Andy Hi, [Formatting is a bit rough, replying from a trimmed digest email] >/Message: 1 />/Date: Fri, 6 Apr 2018 15:04:35 +0200 />/From: Michael Grimm <trashcan at ellael.org <https://dovecot.org/mailman/listinfo/dovecot>> />/To: Dovecot Mailing List <dovecot at dovecot.org <https://dovecot.org/mailman/listinfo/dovecot>> />/Subject: Re: 2.3.1 Replication is throwing scary errors />/Message-ID: <E7E7A927-68F8-443F-BA59-E66CED8FE878 at ellael.org <https://dovecot.org/mailman/listinfo/dovecot>> />/Content-Type: text/plain; charset=utf-8 />//>/Reuben Farrelly wrote: />>/From: Michael Grimm <trashcan at ellael.org <https://dovecot.org/mailman/listinfo/dovecot>> />//>>>/[This is Dovecot 2.3.1 at FreeBSD STABLE-11.1 running in two jails at distinct servers.] />>>/I did upgrade from 2.2.35 to 2.3.1 today, and I do become pounded by error messages at server1 (and vice versa at server2) as follows: />>>/| Apr 2 17:12:18 <mail.err> server1.lan dovecot: doveadm: Error: dsync(server2.lan): I/O has stalled, \ />>>/no activity for 600 seconds (last sent=mail_change, last recv=mail_change (EOL)) />>>/| Apr 2 17:12:18 <mail.err> server1.lan dovecot: doveadm: Error: Timeout during state=sync_mails \ />>>/(send=changes recv=mail_requests) />>>/[?] />>>/| Apr 2 18:59:03 <mail.err> server1.lan dovecot: doveadm: Error: dsync(server2.lan): I/O has stalled, \ />>>/no activity for 600 seconds (last sent=mail, last recv=mail (EOL)) />>>/| Apr 2 18:59:03 <mail.err> server1.lan dovecot: doveadm: Error: Timeout during state=sync_mails \ />>>/(send=mails recv=recv_last_common) />>>/I cannot see in my personal account any missing replications, *but* I haven't tested this thoroughly enough. I do have customers being serviced at these productive servers, *thus* I'm back to 2.2.35 until I do understand or have learned what is going on. />//>/In my reply to this statement of mine I mentioned that I have seen those timeouts quite some times during the past year. Thus, I upgraded to 2.3.1 again, and boom: after some hours I ended up in hanging processes [1] like (see Remko's mail in addition) ... />//>/doveadm-server: [IP4/6 <USER1> SOME/MAILBOX import:0/0] (doveadm-server) />//>/? at server2 paired with a file like ? />//>/-rw------- 1 vmail dovecot uarch 0 Apr 3 16:52 /home/to/USER1/.dovecot-sync.lock />//>/Corresponding logfile entries at server2 are like ? />//>/Apr 3 17:10:49 <mail.err> server2.lan dovecot: doveadm: Error: Couldn't lock /home/to/USER1/.dovecot-sync.lock: \ />/fcntl(/home/to/USER1/.dovecot-sync.lock, write-lock, F_SETLKW) locking failed: Timed out after 30 seconds \ />/(WRITE lock held by pid 51110) />//>/[1] Even stopping dovecot will not end those processes. One has to manually kill those before restarting dovecot. />//>/After one day of testing 2.3.1 with a couple of those episodes of locking/timeout, and now missing mails depending with server your MUA will connect to, I went back to 2.2.35. After two days at that version I never had such an episode again. />//>>/It's not just you. This issue hit me recently, and it was impacting />>/replication noticeably. I am following git master-2.3 . />/[...] />>/There is also a second issue of a long standing race with replication />>/occurring somewhere whereby if a mail comes in, is written to disk, is />>/replicated and then deleted in short succession, it will reappear />>/again to the MUA. I suspect the mail is being replicated back from />>/the remote. A few people have reported it over the years but it's not />>/reliable or consistent, so it has never been fixed. />>/And lastly there has been an ongoing but seemingly minor issue />>/relating to locking timing out after 30s particularly on the remote />>/host that is being replicated to. I rarely see the problem on my />>/local disk where almost all of the mail comes in, it's almost always />>/occurring on the replicate/remote system. />//>/It might be time to describe our setups in order to possibly find common grounds that might trigger this issue you describe and Rimko and myself ran into as well. />//>/Servers: Cloud Instances (both identical), around 25ms latency apart. />/Intel Core Processor (Haswell, no TSX) (3092.91-MHz K8-class CPU) />/Both servers are connected via IPsec/racoon tunnels />/OS: FreeBSD 11.1-STABLE (both servers) />/Filesystem: ZFS />/MTA: postfix 3.4-20180401 (postfix delivers via dovecot's LMTP) />/IMAP: dovecot running in FreeBSD jails (issues with 2.3.1, fine with 2.2.35) />/Replication: unsecured tcp / master-master />/MUA: mainly iOS or macOS mail.app, rarely roundcube / For me: Servers: Main: VM on a VMWare ESXi local system (light load), local SSD disks (no NFS) Redundant: Linode VM in Singapore, around 250ms away Also no NFS. Linode use SSDs for IO. There is native IPv6 connectivity between both VMs. As I am using TCPs I don't have a VPN between them - just raw IPv6 end to end. OS: Gentoo Linux x86_64 kept well up to date Filesystem: EXT4 for both MTA: Postfix 3.4-x (same as you) IMAP: Dovecot running natively on the machine (no jail/chroot) Replication: Secured TCP / master-master (but only one master is actively used for SMTP and MUA access) MUA: Thunderbird, Maildroid, MS Mail, Gmail client, Roundcube, LG Mail client on handset etc >/I believe it is worthwhile to mention here that I run a poor man's fail-over approach (round-robin DNS) as follows: />//>/DNS: mail.server.tld resolves to one IP4 and one IP6 address of each server, thus 4 IP addresses in total / For me I have a simple pair of servers that replicate mail to each other. Clients connect to the main server only, which is where mail comes in. The second one that experiences the problem with locking is only a redundant server which is a secondary MX, that only gets occasional SMTP deliveries from time to time. So the amount of replication back from the "secondary" is very minimal; the vast bulk of the replication is the primary replicating changes out. Reuben

2 1

Re: expiring mail from root's Maildirs?
by LuKreme 05 May '18

05 May '18

(Sorry about previous mail) > On May 4, 2018, at 07:45, Aki Tuomi <aki.tuomi(a)dovecot.fi> wrote: > Dovecot has hardcoded root prevention. For delivery, too. So dovecot cannot expire, reindex, clean, repair, etc root mail? The mail is crontab mail that is sent out to users via mutt. What would be the "right” way to do this? (The cron tasks run as root because they are scanning system logs on behalf of certain users and those scans cannot be run as the user.) -- My main job is trying to come up with new and innovative and effective ways to reject even more mail. I'm up to about 97% now.

6 14