- dovecot - dovecot.org

SMTP and IMAP authentication critically fails out of the blue
by Felix Plapper 14 Nov '17

14 Nov '17

Hello Community and first things First: dovecot --version2.2.9dovecot -n# 2.2.9: /etc/dovecot/dovecot.conf# OS: Linux 3.13.0-042stab125.5 x86_64 Ubuntu 14.04.5 LTSauth_mechanisms = plain logindict { sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf}listen = *,[::]log_timestamp = "%Y-%m-%d %H:%M:%S "login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %kmail_fsync = alwaysmail_home = /var/vmail/%d/%nmail_location = maildir:~/mail_nfs_index = yesmail_nfs_storage = yesmail_plugins = quota aclmanagesieve_notify_capability = mailtomanagesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihavemmap_disable = yesnamespace { list = yes location = maildir:%%h/:INDEXPVT=~/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared}namespace inbox { inbox = yes location = mailbox Archiv { special_use = \Archive } mailbox Archive { auto = subscribe special_use = \Archive } mailbox Archives { special_use = \Archive } mailbox "Deleted Messages" { special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Entwürfe { special_use = \Drafts } mailbox "Gelöschte Objekte" { special_use = \Trash } mailbox Gesendet { special_use = \Sent } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Papierkorb { special_use = \Trash } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = /}passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql}plugin { acl = vfile acl_anyone = allow acl_shared_dict = file:/var/vmail/shared-mailboxes.db quota = dict:User quota::proxy::sqlquota quota_rule2 = Trash:storage=+100%% sieve = /var/vmail/sieve/%u.sieve sieve_after = /var/vmail/sieve/global.sieve sieve_max_script_size = 1M sieve_quota_max_scripts = 0 sieve_quota_max_storage = 0}protocols = imap sieve lmtp pop3service auth { unix_listener /var/spool/postfix/private/auth_dovecot { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } unix_listener auth-userdb { mode = 0600 user = vmail } user = root}service dict { unix_listener dict { group = vmail mode = 0660 user = vmail }}service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail}service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 2 service_count = 1 vsz_limit = 128 M}service managesieve { process_limit = 256}ssl_cert = </etc/ssl/mail/mail.crtssl_cipher_list = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHAssl_dh_parameters_length = 2048ssl_key = </etc/ssl/mail/mail.keyssl_protocols = !SSLv3 !SSLv2userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql}protocol imap { mail_plugins = quota imap_quota imap_acl acl}protocol lmtp { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota sieve acl postmaster_address = postmaster(a)domain1.com}protocol sieve { managesieve_logout_format = bytes=%i/%o}remote 127.0.0.1 { disable_plaintext_auth = no}########################################### Mail.errNov 13 23:59:06 webdev dovecot: auth: Error: PLAIN(account(a)domain2.com, XXX.XXX.XXX.XXX,<y869CoDETEST4dHk>): Request 29154.1 timed out after 150 secs, state=1########################################### Mail.logNov 13 23:27:54 webdev dovecot: auth: Error: LOGIN(account(a)domain1.com,IP.IP.IP.IP,<oN4ly+TestDZ6dHk>): Request 28118.1 timed out after 150 secs, state=1Nov 13 23:27:57 webdev dovecot: auth: Error: PLAIN( account(a)domain2.com,XXX.XXX.XXX.XXX,<FAxKe+JaatES7tHk>): Request 28120.1 timed out after 150 secs, state=1Nov 13 23:28:24 webdev dovecot: imap-login: Disconnected: Inactivity during authentication (disconnected while authenticating, waited 180 secs): user=<>, method=LOGIN, rip=ClientIP, lip=ServerIP, TLS: Disconnected, TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits) [...]Nov 13 23:47:15 webdev dovecot: imap-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=84.119.151.17, lip=62.75.185.32########################################### I did not change anything on my client nor serverside setup and i suddenly could not reach the Mailserver anymore. Obviously I can reach the Server using SSH or HTTP. I hope I provided all infos you need to help me in this situation. And am grateful for every hint to solve this Note: Older mailinglist threads did not help because a link which solved a similar case is not available anymore.plus I tried to make the given pieces of information structured using styles. Maybe it gets delivered,displayed and enhances the ease of reading.

2 1

Carbon_name path hardcoded
by Giordano Fechio (Giu) 13 Nov '17

13 Nov '17

Hello, I am setting up the imap_stats for dovecot and stumb up on a problem with carbon_name setup. const char *prefix = t_strdup_printf("dovecot.%s.global”, But I need to setup the path myself so my index cam be compliant to the index I already have. Wonder if changing it to something like the following so other users have a way to configure their own path or use the default in case they don’t want to worry about that. string_t *str = t_str_new(256); - const char *prefix = t_strdup_printf("dovecot.%s.global", + if ( strstr(carbon_name , "." != NULL) { + const char *prefix = t_strdup_printf("", stats_settings->carbon_name); + } else { + const char *prefix = t_strdup_printf("dovecot.%s.global", + stats_settings->carbon_name); + } If that is not the goal or if I am not using it right , let me know. Bests, G Fechio

1 0

ACL on shared folder
by T.Robers＠smtp.tutech.de 13 Nov '17

13 Nov '17

Hello everybody, we're using shared mailboxes as an archive to which everyone has full rights, set with doveadm. The rights look like this: doveadm acl get -u bob(a)example.com shared/archive(a)example.com ID Global Rights user=bob(a)example.com admin create delete expunge insert lookup \ post read write write-deleted write-seen Looking at this i would this i would think i should be able to move an existing folder to shared/archive(a)example.com. But i get a "permission denied". A rawlog session from trying to copy looks like this: 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY ACL RIGHTS=texk] Logged in * ID ("name" "Dovecot") 2 OK ID completed (0.001 + 0.000 secs). * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1510573115] UIDs valid * OK [UIDNEXT 1] Predicted next UID 3 OK [READ-WRITE] Select completed (0.005 + 0.000 + 0.004 secs). * MYRIGHTS shared/archive(a)example.com/IT-Sachen/blablub lrwstipekxacd 4 OK Myrights completed (0.001 + 0.000 secs). + idling * OK Still here 5 OK Idle completed (4.007 + 4.006 + 4.006 secs). 6 OK NOOP completed (0.001 + 0.000 secs). + idling 7 OK Idle completed (42.014 + 42.013 + 42.013 secs). 8 OK List completed (0.015 + 0.000 + 0.014 secs). 9 OK Close completed (0.001 + 0.000 secs). 10 NO [NOPERM] Permission denied (0.005 + 0.000 + 0.004 secs). I don't understand why there's a permission denied because doveadm tells something else. The shared mailbox folder ist the inbox of the account archive(a)example.com What has to be changed that it's possible to save messages or to create folders in resp. under a shared inbox, or is this not possible? Any suggestion is welcome. Tia Thomas My configuration: # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 2.6.32-696.13.2.el6.x86_64 x86_64 CentOS release 6.9 (Final) ext4 auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 imap_max_line_length = 2 M mail_debug = yes mail_location = maildir:/export/home/imap/%Lu/Maildir mail_plugins = acl mail_log notify mailbox_idle_check_interval = 10 secs mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = file:/export/home/shared-db/shared-mailboxes mail_log_events = append delete undelete expunge copy mailbox_delete mailbox_rename flag_change mail_log_fields = uid box msgid size from flags mail_replica = tcp:mail2.example.com sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global = /var/lib/dovecot/sieve/global/ sieve_user_log = ~/.dovecot.sieve.log } protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service config { unix_listener config { user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_limit = 500 process_min_avail = 20 } service imap { executable = imap postlogin } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service pop3 { executable = pop3 postlogin } service postlogin { executable = script-login -d rawlog } service replicator { unix_listener replicator-doveadm { mode = 0666 } } ssl = required ssl_cert = </etc/pki/dovecot/certs/mail2.example.com.crt_chain ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL:!EXPORT ssl_key = # hidden, use -P to show it ssl_protocols = !SSLv3 !SSLv2 syslog_facility = local6 userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap } protocol lmtp { mail_plugins = acl mail_log notify sieve } protocol imap { mail_max_userip_connections = 100 mail_plugins = acl mail_log notify imap_acl rawlog_dir = /tmp/rawlog/%u }

1 0

rawlog segfaults (error 4 in libdovecot.so.0.0.0)
by T. Robers 13 Nov '17

13 Nov '17

Hello everybody, i tried to debug imap sessions with the rawlog feature and rawlog creates files but when i try to dump them doveadm tells me [...] Error: zlib.read((file)): unexpected EOF at [...]. I looked at syslog files and i see, that rawlog gets terminated with a segfault, e.g.: segfault at 10 ip 00007ff6da362596 sp 00007fffe725a080 error 4 in libdovecot.so.0.0.0[7ff6da2a4000+122000] Is there a way to debug why rawlog ist terminated? I haven't found anything.I would be very thankful, if somebody could give a hint. My system is: # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 2.6.32-696.3.1.el6.x86_64 x86_64 CentOS release 6.9 (Final) ext4 auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no imap_max_line_length = 2 M mail_debug = yes mail_location = maildir:/export/home/imap/%Lu/Maildir mail_plugins = acl zlib mail_log notify mailbox_idle_check_interval = 10 secs mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = file:/export/home/shared-db/shared-mailboxes mail_log_events = append delete undelete expunge copy mailbox_delete mailbox_rename flag_change mail_log_fields = uid box msgid size from flags sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global = /var/lib/dovecot/sieve/global/ sieve_user_log = ~/.dovecot.sieve.log zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 500 process_min_avail = 20 } service imap { executable = imap postlogin } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service pop3 { executable = pop3 postlogin } service postlogin { executable = script-login -d rawlog -f in } ssl = required ssl_cert = </etc/pki/dovecot/certs/mail.tutech.de.crt_chain ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL:!EXPORT ssl_key = # hidden, use -P to show it ssl_protocols = !SSLv3 !SSLv2 syslog_facility = local6 userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap } protocol lmtp { mail_plugins = acl zlib mail_log notify sieve } protocol imap { mail_max_userip_connections = 100 mail_plugins = acl zlib mail_log notify imap_zlib imap_acl rawlog_dir = /tmp/rawlog/%u } Tia Thomas

4 3

imap coredump after doveadm altmove
by Frank Koch 13 Nov '17

13 Nov '17

Hello, i tried to move mails to an alt storage with the following command: doveadm altmove -u user(a)domain.de mailbox Inbox After that, the imap process produces a core dump (files has not been moved): Maillog: dovecot: imap-login: Login: user=<user(a)domain.de>, method=PLAIN, rip=X.X.X.X, lip=X.X.X.X, mpid=5393, TLS, session=<83JiHqNdTwCsEAIf> dovecot: imap(user(a)domain.de): Fatal: master: service(imap): child 5393 killed with signal 11 (core dumped) Version: Dovecot 2.2.18.2 OS: CenotOS6 2.6.32-696.10.2.el6.x86_64 local storage (ext4) #gdb --args /usr/libexec/dovecot/imap -u user(a)domain.de A1 STATUS INBOX (MESSAGES) Program received signal SIGSEGV, Segmentation fault. 0x00007ffff74b3c9a in __strrchr_sse2 () from /lib64/libc.so.6 (gdb) bt full #0 0x00007ffff74b3c9a in __strrchr_sse2 () from /lib64/libc.so.6 No symbol table info available. #1 0x00007ffff7b148cd in sdbox_file_move () from /usr/lib64/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x00007ffff7b18284 in sdbox_sync_begin () from /usr/lib64/dovecot/libdovecot-storage.so.0 No symbol table info available. #3 0x00007ffff7b18382 in sdbox_sync () from /usr/lib64/dovecot/libdovecot-storage.so.0 No symbol table info available. #4 0x00007ffff7b18437 in sdbox_storage_sync_init () from /usr/lib64/dovecot/libdovecot-storage.so.0 No symbol table info available. #5 0x00007ffff7af64b3 in mailbox_sync_init () from /usr/lib64/dovecot/libdovecot-storage.so.0 No symbol table info available. #6 0x00007ffff7af7d27 in mailbox_sync () from /usr/lib64/dovecot/libdovecot-storage.so.0 No symbol table info available. #7 0x00007ffff7b62af2 in index_storage_get_status () from /usr/lib64/dovecot/libdovecot-storage.so.0 No symbol table info available. #8 0x00007ffff67eb32c in ?? () from /usr/lib64/dovecot/lib10_quota_plugin.so No symbol table info available. #9 0x00007ffff6c021e4 in ?? () from /usr/lib64/dovecot/lib01_acl_plugin.so No symbol table info available. #10 0x00007ffff7af8931 in mailbox_get_status () from /usr/lib64/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x000000000041e07a in imap_status_get () No symbol table info available. #12 0x0000000000413629 in cmd_status () No symbol table info available. #13 0x00000000004178ed in command_exec () No symbol table info available. #14 0x00000000004168d0 in ?? () No symbol table info available. #15 0x0000000000416966 in ?? () No symbol table info available. #16 0x0000000000416c5d in client_handle_input () No symbol table info available. #17 0x0000000000416fdf in client_input () No symbol table info available. #18 0x00007ffff7852fbb in io_loop_call_io () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #19 0x00007ffff785418f in io_loop_handler_run_internal () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #20 0x00007ffff7853049 in io_loop_handler_run () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #21 0x00007ffff78530c8 in io_loop_run () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #22 0x00007ffff77f2043 in master_service_run () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #23 0x00000000004206c1 in main () No symbol table info available. This is only on folders i have tried to move. best regards, Frank Koch --

1 0

IMAP connections with ".eml" in the username - bot attack.
by James Brown 13 Nov '17

13 Nov '17

We are seeing lots of IMAP login attempts like this: dovecot[363]: imap-login: Disconnected (auth failed, 1 attempts in 6 secs): user=<xcentrex5fxnewx5fxyorkx5fxquotex5fxisx5fxreadyx2dxx2dx426453.eml>, method=PLAIN, rip=197.255.60.118, or dovecot[363]: imap-login: Disconnected (auth failed, 1 attempts in 6 secs): user=<xmatchingx5fxyourx5fxrecentx5fxvisitx5fxonx5fxx2dxx2dx121584.eml(a)bordo.com.au>, method=PLAIN, rip=37.235.28.229, etc. We are running fail2ban, but as each login attempt is from a different IP it is not able to stop them. We are running Sophos UTM firewall but that has no IMAP Proxy and never will. Is anyone else experiencing this? How is such an attack is supposed to ever succeed? What are they trying to accomplish? Any ideas on how to mitigate it? Thanks, James.

3 3

Doveadm: Skipping module doveadm_mail_crypt_plugin loading
by vkosharskiy＠gmail.com 10 Nov '17

10 Nov '17

# dovecot --version Doveadm unable to load plugin bacause plugin doveadm_mail_crypt_plugin.so not compiled with function mail_crypt_box_get_pvt_digests, who defned in mail-crypt-key.c Fix for these problem: dovecot_doveadm_mail_crypt_fix.diff: --- src/plugins/mail-crypt/Makefile.am.old 2017-11-10 19:51:18.155088706 +0200 +++ src/plugins/mail-crypt/Makefile.am 2017-11-10 20:06:30.557814167 +0200 @@ -70,7 +70,8 @@ libfs_mail_crypt_la_LDFLAGS = -module -avoid-version libdoveadm_mail_crypt_plugin_la_SOURCES = \ - doveadm-mail-crypt.c + doveadm-mail-crypt.c \ + mail-crypt-key.c libdoveadm_mail_crypt_plugin_la_LIBADD = $(LIBDOVECOT) libdoveadm_mail_crypt_plugin_la_DEPENDENCIES = $(LIBDOVECOT_DEPS) libdoveadm_mail_crypt_plugin_la_LDFLAGS = -module -avoid-version # doveadm -D sis _*Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) *_ After compile: # doveadm -D sis

1 0

Replication oddities - different sizes between replicated nodes
by Remko Lodder 10 Nov '17

10 Nov '17

Dear Dovecot community, As discussing on IRC with Aki, I have the following thing happening: I got an alert from my mailservice last night that I was running over quota. My quota reached almost 91% (coming from 30%) in one day. I do not recall receiving so much email (we talk about multiple gb’s additional space usage), so something was acting up. When I logged into my mailservers (I’ll describe my setup later on), it mentioned that my FreeBSD mailbox was reaching 12gb of space usage. While the seeming replicated machine (named A) only had 100mb of space in use (which, with the amount of messages in the various mailboxes is consistent with what I would expect). I found out that the “Junk” mailbox was the ‘major' issue. I vaporized the replicated (12GB) mailbox and issued a ‘doveadm sync -u $user -N -U tcps:hostB` so that I would expect the hostB to reach around 100mb-ish of email. I let it go for a little and saw that the mail was done. When I issued a simple ‘df -h’ on the directory, it was already multiple 100mb’s large and growing. It grew back to multiple GB in little time. Again the Junk mailbox was the culprit. In the mean time I noticed my mailclient downloading 44k mails in that mailbox (where there were only a few in the Junk mailbox normally). I again vaporized the entire mailbox on hostB and cleaned out Junk and issued a manual synchronisation. It again grew quickly. Aki did an investigation with me and noticed that my rspamc script was not very well written. I rewrote it as suggested (As written on the site). This morning I continued with the investigation and it was again large on hostB where it was still 100mb-ish on hostA. I vaporized host B again and issued the manual sync with -Dv included per request of Aki. After letting it run for a little hostA grew to 300mb and hostB restarted to reach 1.2GB and remained static there. After half an hour or so the mailbox on hostA grew to 900mb and stayed there. I investigated the mailboxes and saw 8 times the same email. Cleaning them up with doveadm deduplicate -u $user mailbox ‘*’ made sure a lot of email was cut from both hosts. I am now in the state where: hostA: 268MB hostB: 579MB Both are for the same mailbox (I suspect that other mailboxes have similiar issues). When I look at the amount of messages counted by Apple Mail in the particular mailbox I see 1399 emails. The amount of ‘u’ files is 14722 on hostA and 23590 on hostB. I do not understand the difference from my mail client and the amount of u files. from mutt: Mutt: =mailbox [Msgs:1415 New:1 Flag:1 24M]-, which is roughly the same as Apple mail reports. I am a bit lost on where to look next, I would suspect that the synchronised mailboxes would be similar in size, but not the one 2 as large, or before that it grew and grew and grew. My personal feeling is that the sync takes place, but somehow it is not registering that a message had been synchronised and does that again, and again, and again till the message gets synchronised right and the acknowledge is received and it stays the same. That would at least support the 8times the same email that I saw at first and might also support the roughly double as big size of the other host (one sync goes wrong, the next sync went right and got acked). At the bottom I also referenced two previous emails of mine which report similar oddities. Messages that were removed on hostA got back within seconds as “new message” . My feeling there is that the hostB has the ‘failed sync copy with a different uid/guid’ and tells hostA that there is a new message and sends it over. My doveconf -n for hostA (hostB follows), I did sanitize a bit wrt. IP addresses and hostnames: 2.2.33.2 (d6601f4ec): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: FreeBSD 11.1-RELEASE amd64 auth_mechanisms = plain login disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_debug = yes mail_fsync = always mail_location = sdbox:~/sdbox mail_plugins = " quota notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . } passdb { driver = pam } plugin { imapsieve_mailbox1_before = file:/usr/local/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcps:hostB:12346 sieve = ~/.dovecot.sieve sieve_after = /usr/local/etc/dovecot/sieve/after/ sieve_before = /usr/local/etc/dovecot/sieve/global/ sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve sieve_global = /usr/local/etc/dovecot/sieve/global/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_global_path = /usr/local/etc/dovecot/sieve/global/dovecot.sieve sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_dont_check_recipient = yes } postmaster_address = postmaster@isp protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0666 } } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service doveadm { inet_listener { port = 12346 ssl = yes } } service imap-login { service_count = 1 } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { address = external ipv4 and ipv6 127.0.0.1 ::1 port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3 { process_limit = 1024 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl_ca = </usr/local/etc/dehydrated/certs/hostA/fullchain.pem ssl_cert = </usr/local/etc/dehydrated/certs/hostA/cert.pem ssl_client_ca_file = /usr/local/certificates/letsencrypt-ca.pem ssl_dh_parameters_length = 2048 ssl_key = # hidden, use -P to show it ssl_protocols = !SSLv2 !SSLv3 TLSv1 TLSv1.1 TLSv1.2 userdb { driver = passwd } verbose_proctitle = yes protocol lmtp { auth_username_format = %n mail_fsync = optimized mail_plugins = " quota notify replication sieve" postmaster_address = postmaster@isp } protocol lda { mail_plugins = " quota notify replication sieve" } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 50 mail_plugins = " quota notify replication imap_quota imap_sieve" } for hostB: # 2.2.33.2 (d6601f4ec): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: FreeBSD 11.1-RELEASE-p1 amd64 auth_mechanisms = plain login disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_debug = yes mail_fsync = always mail_location = sdbox:~/sdbox mail_plugins = " quota notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . } passdb { driver = pam } plugin { imapsieve_mailbox1_before = file:/usr/local/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcps:hostA:12346 sieve = ~/.dovecot.sieve sieve_after = /usr/local/etc/dovecot/sieve/after/ sieve_before = /usr/local/etc/dovecot/sieve/global/ sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve sieve_global = /usr/local/etc/dovecot/sieve/global/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_global_path = /usr/local/etc/dovecot/sieve/global/dovecot.sieve sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_dont_check_recipient = yes } postmaster_address = postmaster@isp protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0666 } } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service doveadm { inet_listener { port = 12346 ssl = yes } } service imap-login { service_count = 1 } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { address = ipv4 and ipv6 address 127.0.0.1 ::1 port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3 { process_limit = 1024 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl_ca = </usr/local/etc/dehydrated/certs/hostA/fullchain.pem ssl_cert = </usr/local/etc/dehydrated/certs/hostA/cert.pem ssl_client_ca_file = /usr/local/certificates/letsencrypt-ca.pem ssl_dh_parameters_length = 2048 ssl_key = # hidden, use -P to show it ssl_protocols = !SSLv2 !SSLv3 TLSv1 TLSv1.1 TLSv1.2 userdb { driver = passwd } verbose_proctitle = yes protocol lmtp { auth_username_format = %n mail_fsync = optimized mail_plugins = " quota notify replication sieve" postmaster_address = postmaster@isp } protocol lda { mail_plugins = " quota notify replication sieve" } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 50 mail_plugins = " quota notify replication imap_quota imap_sieve” } Both machines run FreeBSD, with a ZFS backend. HostB has quota’s on the zfs shares but none of them are full or almost full and have plenty of space left on their directories (so there is no quota or disk full issue there) Mail is arriving on both hosts by upstream MX’es. Most of my users use hostB as their main host. I use hostA as my primary host. Both machines are located at Hetzner in Germany. HostA is in DC6 and HostB is in DC12. Not situated next to eachother but low latency links between them. Both machines are connected to gbit uplinks and are not very highly loaded. The machines consist of 2x2TB disk (mirrored with ZFS) and 8cores, 32GB RAM. There were warnings when manually doing the sync that the mailbox changed in between and should be reissued, after which the mailboxes kept growing. Previously I reported two messages here which might be the foundation of the same issue: https://dovecot.org/list/dovecot/2016-July/104873.html and https://www.dovecot.org/list/dovecot/2017-March/107459.html

1 0

Sieve global path?
by Gao 10 Nov '17

10 Nov '17

Hello, I am building a new server on CentOS 7 and the global sieve filter can not be loaded. The debug shows: Nov 9 15:23:09 mail dovecot: lmtp(11182, gao(a)mydomain.com) Debug: sieve: Pigeonhole version 0.4.2 initializing Nov 9 15:23:09 mail dovecot: lmtp(11182, gao(a)mydomain.com) Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. The pigeonhole version: # yum list dovecot-pigeonhole dovecot-pigeonhole.x86_64 1:2.2.10-8.el7 I googled and got confused on how to make sieve point to the correct global filter (which I want auto deliver spams to Junk folder). So what is the correct setting? Thanks in advance. Gao Here is my doveconf -n output: ----------------------------------------------------------- [root@mail ~]# doveconf -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-693.5.2.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) xfs auth_mechanisms = plain login cram-md5 first_valid_gid = 5000 first_valid_uid = 5000 last_valid_gid = 5000 last_valid_uid = 5000 listen = * mail_debug = yes mail_gid = 5000 mail_location = maildir:/home/vmail/%d/%n/:INDEX=/home/vmail/%d/%n/indexes mail_plugins = " quota" mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/conf.d/dovecot-mysql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_exceeded_message = ERROR:422 - Mailbox full, sorry. quota_rule = Junk:ignore quota_rule2 = Trash:storage=+100M quota_warning = storage=90%% quota-warning 90 %u %d quota_warning2 = storage=80%% quota-warning 80 %u %d sieve = ~/sieve/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /home/vmail/sieve/globalfilter.sieve sieve_max_script_size = 1M sievedir = ~/sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/overquota.sh group = vmail unix_listener quota-warning { group = vmail user = vmail } user = vmail } ssl = required ssl_cert = </etc/letsencrypt/live/mydomain.com/fullchain.pem ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4 ssl_key = </etc/letsencrypt/live/mydomain.com/privkey.pem ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 userdb { args = /etc/dovecot/conf.d/dovecot-mysql.conf.ext driver = sql } protocol lmtp { mail_plugins = " quota sieve" postmaster_address = postmaster(a)mydomain.com } protocol imap { mail_plugins = " quota imap_quota" } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 }

5 8

Postlogin script
by j.emerlik 10 Nov '17

10 Nov '17

Hi, I would like to prepare postlogin a script that allow imap connection to roundcube for all but restrict imap access for selected users. My question is that: Is possible in condition IF use IP addresses as range or with mask (because I've more than one web servers) ? My script: #!/bin/sh if [ "$IP" = "172.11.0.28" ] ; then printf "* [ALERT] Access allowed from that IP\r\n" exec "$@" fi CHECK_USER=`PGPASSWORD="somepass" /usr/local/pg950/bin/psql -q -t -U someuser -d maildb -c "select imap_allowed from __users where name = '$USER' LIMIT 1"` if [ $CHECK_USER == "f" ] ; then exit 0 fi if [ $CHECK_USER == "t" ] ; then exec "$@" fi Regards, Jack

3 3