February 2017 - dovecot

Dovecot performance and proxy loops with IPv6
by Daniel Betz 02 Feb '17

02 Feb '17

Hello list, i run here an large mailsetup with some million mailboxes and got strange performance problems, cause i think i have overseen or forgotten an simple setting. Here are some details: 21 CentOS 7 Servers with dovecot 2.2.25 and ldap userdb/passdb via socket behind an hardware loadbalancer. The storage behind is an ISCSI Storage with 4 10Gbit/s multipath paths, splitted up to 10 TB volumes for each server with LVM and xfs filesystem. No Cluster FS Each server has about 60.000 to 75.000 mailboxes on it. mailboxes can have up to 10Gbyte space. The Log says this sometimes and complete random: Feb 1 10:42:49 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: imap-login: Error: net_connect_unix(imap) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Sure i have read the SocketUnavailabe wiki page and changed some settings, but the errors are not gone. Could you please look over my dovecot config and give me some tips or hints what to change. The next this is, when adding IPv6 via DNS to the hosts and login with IPv6 i will become an proxy loop. Settings in nameserver: server1.domain.com IN A 123.123.123.123 server1.domain.com IN AAAA 2001:123::1 The host entry comes from the ldap and says: mailHost: server1.domain.com Imap Login with IPv6 to server1.domain.com tries to proxy from server1.domain.com ( IPv6 ) to server1.domain.com ( IPv6 ) and loops then. I have removed the IPv6 AAAA entries in the dns to stop this loops. Sorry, but i have no logs for this anymore. Thanks in advise, Daniel And here system configs and dovecot configs: sysctl: fs.inotify.max_user_instances = 65535 fs.inotify.max_user_watches = 16384 systemd startup with ulimit settings: [Unit] Description=Dovecot Mailservice IMAP/POP [Service] Type=simple LimitCORE=0 LimitNPROC=5000000 LimitNOFILE=65535 LimitSTACK=81920 LimitDATA=infinity LimitMEMLOCK=infinity LimitRSS=infinity LimitAS=infinity ExecStart=/usr/local/dovecot2/sbin/dovecot -F -c /usr/local/dovecot2/etc/dovecot/dovecot.conf [Install] WantedBy=multi-user.target dovecot-ldap.conf: uris = ldapi://%2Fvar%2Frun%2Fldapi dn = cn=xxxxxxx,o=domain,c=com dnpass = xxxxxxxxxxxxx auth_bind = no ldap_version = 3 base = o=domain,c=com user_attrs = mail=user,mailMessageStore=home,\ mailQuota=quota_rule=*:storage=%$ iterate_filter= (|(mailHost=server1.domain.com)(mailHost=popserver1.domain.com)) user_filter = (&(accountstatus=active)(|(uid=%u)(mail=%u))) pass_attrs = mail=user,userPassword=password,=proxy_maybe=y,mailHost=host,=destuser=%u[%r] pass_filter = (&(accountstatus=active)(|(uid=%u)(mail=%u))) dovecot.conf: # 2.2.25 (7be1766): /usr/local/dovecot2/etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.36.3.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_cache_negative_ttl = 1 mins auth_cache_size = 64 M auth_cache_ttl = 2 hours auth_mechanisms = plain login auth_username_chars = auth_verbose = yes base_dir = /var/run/dovecot/ debug_log_path = /dev/null default_login_user = dovecot disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 first_valid_gid = 1001 first_valid_uid = 1001 info_log_path = /dev/stderr lda_mailbox_autocreate = yes lda_original_recipient_header = X-Envelope-To log_path = /dev/stderr log_timestamp = login_log_format_elements = user=[%u] method=%m rip=%r lip=%l %c mail_gid = 1001 mail_location = mdbox:~:INDEX=%h/INDEX mail_plugins = "notify replication stats" mail_uid = 1001 mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_warning = storage=85%% quota-warning 85 %u stats_refresh = 30 secs stats_track_cmds = yes } replication_max_conns = 30 sendmail_path = /usr/local/exim/bin/exim service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = popuser } unix_listener replication-notify { mode = 0666 user = popuser } } service anvil { client_limit = 60000 } service auth { client_limit = 60000 unix_listener auth-userdb { mode = 0666 user = popuser } unix_listener auth { mode = 0666 user = popuser } } service config { unix_listener config { user = popuser } } service dict { unix_listener dict { mode = 0666 user = popuser } } service doveadm { inet_listener { port = 12345 } user = popuser } service imap-login { chroot = login process_limit = 60000 process_min_avail = 16 } service imap { executable = /usr/local/dovecot2/libexec/dovecot/imap process_limit = 250000 } service ipc { client_limit = 60000 unix_listener ipc { mode = 0650 user = dovecot } unix_listener login/ipc-proxy { mode = 0650 user = dovecot } } service lmtp { unix_listener lmtp { mode = 0666 user = popuser } } service pop3-login { chroot = login process_limit = 60000 process_min_avail = 16 } service pop3 { executable = /usr/local/dovecot2/libexec/dovecot/pop3 process_limit = 250000 } service replicator { unix_listener replicator-doveadm { mode = 0600 user = popuser } } service stats { fifo_listener stats-mail { mode = 0600 user = popuser } } ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert1.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert1.key ssl_protocols = TLSv1.2 TLSv1.1 TLSv1 !SSLv3 !SSLv2 userdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap.conf driver = ldap } verbose_proctitle = yes protocol doveadm { mail_plugin_dir = /usr/local/dovecot2/lib/dovecot/ mail_plugins = quota } protocol lda { info_log_path = /var/log/dovecot-lda log_path = /var/log/dovecot-lda log_timestamp = "%b %d %H:%M:%S " mail_plugin_dir = /usr/local/dovecot2/lib/dovecot/ mail_plugins = quota postmaster_address = %u } protocol imap { imap_client_workarounds = mail_max_userip_connections = 100 mail_plugin_dir = /usr/local/dovecot2/lib/dovecot/ mail_plugins = quota imap_quota stats imap_stats } protocol pop3 { mail_plugin_dir = /usr/local/dovecot2/lib/dovecot/ mail_plugins = quota stats pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_enable_last = yes pop3_fast_size_lookups = yes pop3_reuse_xuidl = no pop3_uidl_format = %g } local xxx.xxx.xxx.54 { protocol imap { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert1.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert1.key } } local xxx.xxx.xxx.54 { protocol pop3 { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert1.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert1.key } } local xxx.xxx.xxx.55 { protocol imap { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert2.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert2.key } } local xxx.xxx.xxx.55 { protocol pop3 { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert2.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert2.key } } local xxx.xxx.xxx.56 { protocol imap { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert3.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert3.key } } local xxx.xxx.xxx.56 { protocol pop3 { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert3.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert3.key } } local xxxx:xxxx::fa { protocol imap { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert1.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert1.key } } local xxxx:xxxx::fa { protocol pop3 { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert1.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert1.key } } local xxxx:xxxx::fb { protocol imap { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert2.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert2.key } } local xxxx:xxxx::fb { protocol pop3 { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert2.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert2.key } } local xxxx:xxxx::fc { protocol imap { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert3.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert3.key } } local xxxx:xxxx::fc { protocol pop3 { ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert3.crt ssl_key = </usr/local/dovecot2/etc/dovecot/certs/cert3.key } }

1 0

SNI with mixed certs
by Ruga 02 Feb '17

02 Feb '17

Dovecot SNI is failing hard today. Server with n domains, each with a startssl certificate of its own, all certificates expired this morning. Decision: move to Letsencrypt. Firsr certificate issued and installed. Other domains in the pipeline. Dovecot server rebooted. Expected result: one domain returning the new cert, and the n-1 domains returning the expiration notification. Actual result: the domain with LE is returning startssl expired notifications. Manual check of the key and pem files is OK...

1 0

Doveadm option for a non-wildcard single-user with userdb
by Ivan Zahariev 02 Feb '17

02 Feb '17

Hello, Most "doveadm" commands accept "[-A|-u user|-F file]" for user selection, or the environment "USER". I'm testing with "doveadm quota recalc". The problem is that there is no way to ask "doveadm" to work in (1) single-user mode with (2) no wildcard support, and at the same time to (3) make a lookup in "userdb", in order to get the user's specific configuration. We have mailboxes which contain "?" and "*" symbols, and we can't work with them using "doveadm" now. * If we use "-A", this works with all users. Not our case at all. * If we use "-F" and provide just one user in the file, this works for a single user + lookup in "userdb", does not interpret wildcard, but "doveadm" works in a "users list" mode and the output is different. What's more problematic is that errors for an mbox do not end up in "doveadm" exiting with a non-zero exit code. * If we use "-u", this works for a single user + lookup in "userdb", but interprets wildcards. Does not work for mailboxes which contain "?" and "*". * If we use the USER environment, this works for a single users and does not interpret wildcards but does not do a lookup in "userdb". Should we add another user-selection argument, for example "-U", which (1) selects a single-user like "-u", does a "userdb" lookup like "-u" does, but does not interpret wildcards unlike "-u" ? Best regards. --Ivan

1 0

Restoring a deleted imap folder from backup to user Maildir
by Kepa Lyman 02 Feb '17

02 Feb '17

Hi, A user accidentally delete a folder from their Maildir (probably while using Thunderbird). The folder was at Archives/Sent/2015 (/.Archives.Sent.2015 in the Maildir format). I attempted to copy the folder from a tar backup into the Maildir folder but the new folder does not show up in Thunderbird or in the Subscribe option. I also copied the .Archives.Sent. folder as well. A "Sent" folder shows up after this in Thunderbird's Subscribe menu but not the 2015 subfolder. Is there some other additional step(s) to undertake when moving Maildir folders around so they show up in Dovecot (or Thunderbird- though I did try deleting Thunderbird's local mail tmp folder). Re: this message about SELinux on CentOS, SELinux is set to permissive: https://dovecot.org/list/dovecot/2009-January/036257.html Thanks! Kepa

2 1

what do you think about that?
by dovecot 02 Feb '17

02 Feb '17

Dear! I just wanted to show you my last article and to ask your opinion about it, please read it here http://gianluigi.bigshowonline.net/0607 dovecot

1 0