- dovecot - dovecot.org

Migrating from external server to Dovecot (doveadm sync issues)
by Sven Strickroth 12 Apr '19

12 Apr '19

Hi, I'm experiencing serious issues when using doveadm sync not working at all or do not properly sync. I wanted to use this for migrating external mailboxes to my server (in order to overcome the time when some servers/clients still use the old server and some already the my server). I tried to find a few steps in order to replicate this: For testing purposes I simulate both the remote and local server on my Dovecot instance. I created remote(a)DOMAIN.TLD and local(a)DOMAIN.TLD both use mdbox and are empty (i.e., the mail_location is still non-existent). Then I send two mails to remote(a)DOMAIL.TLD (Subjects: Test1 and Test2). Start backup remote IMAP account (remote(a)DOMAIN.TLD) to local account (local(a)DOMAIN.TLD) * doveadm -o imapc_user="remote(a)DOMAIN.TLD" -o imapc_password="PW" -o imapc_host="localhost" -o mail_prefetch_count=20 -o imapc_features="rfc822.size fetch-headers" backup -R -u local(a)DOMAIN.TLD imapc: (without this I often got errors such as "Error: Backend didn't update mailbox Trash GUID" on sync runs, which seems to be an unrecoverable error). Now the two mailboxes seem to be the same (can confirm this when logging into the two mailboxes). Now, run first sync: * doveadm -o imapc_user="remote(a)DOMAIN.TLD" -o imapc_password="PW" -o imapc_host="localhost" -o mail_prefetch_count=20 -o imapc_features="rfc822.size fetch-headers" sync -R -u local(a)DOMAIN.TLD imapc: Seems to work and does nothing at the moment. Deliver one message to local(a)DOMAIN.TLD (Subject Test3). * doveadm -o imapc_user="remote(a)DOMAIN.TLD" -o imapc_password="PW" -o imapc_host="localhost" -o mail_prefetch_count=20 -o imapc_features="rfc822.size fetch-headers" sync -R -u local(a)DOMAIN.TLD imapc: Now Test1,2,3 are in both mailboxes (further runs do change nothing). Everything OK up the here. Now deliver one messages to remote(a)DOMAIN.TLD (Test4) and delete one old message (Test3). Deliver two messages to local(a)DOMAIN.TLD (Subjects Test5 and Test6) and delete one message (Test2). In sum both mailboxes look like this: remote(a)DOMAIN.TLD has Test1,2,4 and local(a)DOMAIN.TLD has 1,3,5,6. Now sync: * doveadm -o imapc_user="remote(a)DOMAIN.TLD" -o imapc_password="PW" -o imapc_host="localhost" -o mail_prefetch_count=20 -o imapc_features="rfc822.size fetch-headers" sync -R -u local(a)DOMAIN.TLD imapc: dsync(local(a)DOMAIN.TLD): Warning: Mailbox changes caused a desync. You may want to run dsync again: 3 UIDs changed due to UID conflicts Rerun again and see that mails start duplicating and deletings are not synced: remote(a)DOMAIN.TLD: Test1,2,4,3,5,6,5,6,6,5,3,4 (12 mails) and local(a)DOMAIN.TLD: 1,6,4,3,5,6,4,3,5,6,5 (11 mails) When I re-run again I sometimes see the above error message again, mails keep duplicating in both mailboxes and the mailboxes are not synced at all. Am I using doveadm sync wrongly? Is there a way to use "sync" directly w/o backup from the beginning (i.e., how to get rid of the error "Error: Backend didn't update mailbox Trash GUID"). Context: # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.8 imap_client_workarounds = delay-newmail I can't upgrade to latest Dovecot right now. :( -- Best regards, Sven Strickroth PGP key id F5A9D4C4 @ any key-server

1 0

Mail account brute force / harassment
by Marc Roos 12 Apr '19

12 Apr '19

Say for instance you have some one trying to constantly access an account Has any of you made something creative like this: * configure that account to allow to login with any password * link that account to something like /dev/zero that generates infinite amount of messages (maybe send an archive of virusses?) * transferring TB's of data to this harassing client. I think it would be interesting to be able to do such a thing.

3 2

Emails redownloading
by azurit＠pobox.sk 12 Apr '19

12 Apr '19

Hi, recently, we had a problem on one of our mail servers and, after reboot, HDD with emails wasn't mounted into system. Until we fixed it, LOTS of users logged in (Dovecot allowed login and recreated directory structure of mailboxes) and saw empty mailboxes. Now they are redownloading all email, which isn't fun, as it's about 2 TB of data. Anyway, is there a way how to prevent this in the future? Can i set Dovecot to disallow logins in such situations? Thanks. azur

3 5

Doveadm stats
by ＠lbutlr 12 Apr '19

12 Apr '19

Are there any examples of the utility of doveadm stats? in the man page: > dump accepts following types: command, session, user, domain, ip and > global. > > Filter can be > > user=<wildcard> > Match given user. I've tried the following commands after looking at the (sparse and example-less) man page. # doveadm stats dump metric_name field count sum min max avg median stddev %95 # doveadm stats dump user Error: Extraneous arguments found: user # doveadm stats dump user user="*" Error: Extraneous arguments found: user user=* # doveadm stats dump global Error: Extraneous arguments found: global # doveadm stats dump global user="*" Error: Extraneous arguments found: global user=* # I assume there is more to the configuration for stats than merely making it not report an error. So, the first question is what sorts of stats would be most likely to be useful and what do I do in the configuration to get those? Second question is why is this required to be enabled if, by default, it has no functionality? -- Beware of geeks bearing .GIF's

2 1

Mailbox INBOX: Opening INBOX failed: Mailbox doesn't exist: INBOX. Maybe master user + namespace problem?
by Usuario prueba 12 Apr '19

12 Apr '19

2 2

Secure Client-Initiated Renegotiation
by sergio 12 Apr '19

12 Apr '19

Hello. I've just tested my system that runs dovecot 2.3.4.1 on debian buster with testssl.sh (https://testssl.sh/) and is says: Secure Renegotiation (CVE-2009-3555) not vulnerable (OK) Secure Client-Initiated Renegotiation VULNERABLE (NOT ok), potential DoS threat Is this a configuration or a compilation issue and how to solve it? -- sergio.

2 1

Re: High availability of Dovecot
by Gerald Galster 12 Apr '19

12 Apr '19

> Am 11.04.2019 um 11:48 schrieb luckydog xf <luckydogxf(a)gmail.com>: > > As your statement, nothing speical is needed to do except setting up DNS MX records, right? MX records are for incoming MAIL: yourdomain.com <http://yourdomain.com/> IN MX 100 mail1.yourdomain.com <http://server1.yourdomain.com/> yourdomain.com <http://yourdomain.com/> IN MX 100 mail2.yourdomain.com <http://mail2.yourdomain.com/> -> both priority 100 = 50/50 load balancing (globally, not when checked on a single resolver!) Then you need A Records (AAAA for ipv6) mail1.yourdomain.com IN A 192.168.10.1 mail2.yourdomain.com <http://mail2.yourdomain.com/> IN A 192.168.20.1 mail.yourdomain.com IN A 192.168.10.1 mail.yourdomain.com IN A 192.168.20.1 mail1/mail2 is for direct connection (MTAs) Your users (outlook, thunderbird, ...) connect to mail.yourdomain.com <http://mail.yourdomain.com/> which returns the two ip addresses. In this scenario MUA just connects to mail.yourdomain.com <http://mail.yourdomain.com/> and randomly uses one of the two ips. You can't control which one, but this gives you active/active loadbalancing. In case one server is down the MUA just uses the other ip. dsync replicates bi-directionally so that both servers are up-to-date. You don't need shared storage, every server is a copy of the other. If you want to use shared storage, then dsync is not for you because there is nothing to sync at that stage. I would use shared storage only if you need to have more than two servers. The above setup has no locking problems and is performant due to local filesystems. It depends on how many users you have and how much storage you need. You could buy two 2HE servers with 24 2.5" disks each (up to 96 with 4 HE), which may be sufficient for your needs. > User's mail store is running on shared storage, basically user's MUA connects to primary MX , the backup one is used once Primary is down. If you're not using Maildir beware of locking issues with concurrent access. It could crash indices. > It's a native HA of email system? I'll test those solution out. Yes, it works well with small setups. For big setups you'd typically use dovecot director, shared storage, object storage ... but you need more servers and it is way more complex and expensive. Best regards Gerald

3 3

failed: read(/var/run/dovecot/dns-client)
by Laura Smith 12 Apr '19

12 Apr '19

=== dsync(foobar(a)example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer dsync(foobar(a)example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server === Initially I thought "oh no, not another AppArmor block". But then surely the second message would not appear if the DNS lookup was not successful ? Also "dig foobar.example.com" works fine. How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? So many questions !

3 18

auth-worker unknown user
by Laura Smith 11 Apr '19

11 Apr '19

pam(foobar(a)example.com,192.0.1.1,<9zMTUUCGNfHZzMpL>): unknown user (SHA1 of given password: ff75068c2f4d700a49dae204d56477a5ffa5d23d) The password is correct, i.e. 'echo -n 'passed' | openssl dgst -sha1' matches. The user is setup correctly in /etc/dovecot/users (the /etc/dovecot/users was copied from another known-good server, so the syntax is correct and appropriate adjustments have been made for chmod and directory). doveconf -N follows: # 2.3.3 (dcead646b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.3 (f018bbab) # OS: Linux 4.12.14-lp150.12.48-default x86_64 # Hostname: foobar auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = sha1 doveadm_password = # hidden, use -P to show it first_valid_uid = 471 imapc_features = rfc822.size fetch-headers imapc_host = foobar.example.com imapc_password = # hidden, use -P to show it imapc_port = 993 imapc_ssl = imaps imapc_user = %u mail_location = maildir:~/Maildir mail_plugin_dir = /usr/lib64/dovecot/modules mail_prefetch_count = 20 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body environment mailbox date ihave enotify namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam name = } plugin { sieve = file:~/.dovecot.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service imap-login { process_min_avail = 3 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = my_virtmailuser } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieves { address = port = 5190 ssl = yes } } ssl = required ssl_ca = </etc/dovecot/ssl/certbot_ca.pem ssl_cert = </etc/dovecot/ssl/certbot.pem ssl_cipher_list = ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd name = } userdb { args = scheme=ARGON2ID username_format=%u /etc/dovecot/users auth_verbose = yes driver = passwd-file name = } protocol lmtp { mail_plugins = sieve postmaster_address = foobar(a)example.com } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address = postmaster quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r } protocol imap { mail_max_userip_connections = 20 }

2 7

LMTP, PAM session and home directory autocreation
by Ivars Strazdins 11 Apr '19

11 Apr '19

Hi, mail is delivered by Dovecot's LMTP locally and I need user's home directory to be created if it doesn't exist yet. There is a setting in Dovecot's configuration, "session=yes", in /etc/Dovecot/conf.d/auth-system.conf.ext, which should do that. passdb { driver = pam args = session=yes dovecot } But I think it does not work in my setup because I do not see any PAM log entry for Dovecot in system log when this error happens: Apr 9 13:01:55 mailhost dovecot: lmtp(2935): Connect from local Apr 9 13:01:55 mailhost dovecot: lmtp(2935, testuser): Error: User initialization failed: Namespace '': mkdir(/home/testuser/Maildir) failed: Permission denied (euid=174000327(testuser) egid=174000327(testuser ) missing +w perm: /home, dir owned by 0:0 mode=0755) Apr 9 13:01:55 mailhost dovecot: lmtp(2935): Disconnect from local: Successful quit The error above seems expected, because it is not LMTP agent's job to create user's home directory but pam_oddjob_mkhomedir.so module should do that. Right? And there are common PAM log entries for every user session: Apr 9 13:24:42 mailhost auth: pam_sss(dovecot:auth): authentication success; logname= uid=0 euid=0 tty=dovecot ruser=validuser rhost=::1 user= validuser Apr 9 13:24:42 mailhost auth: pam_unix(dovecot:session): session opened for user validuser by (uid=0) Apr 9 13:24:42 mailhost auth: pam_unix(dovecot:session): session closed for user validuser How to debug this problem and find out why Dovecot does not open PAM session or - if I am wrong and it does, then what else is going wrong? Home directory autocreation is configured with command "authconfig --enablemkhomedir --update" and it works if user logs into system via shell or webmail. I tried to enable "mail_debug" in Dovecot's settings, but it did not give me any more information on PAM session. Running on Centos 7.6, with Dovecot 2.2.36. It looks like a common mistake or issue, because I am not alone: http://tinyurl.com/y6kjhsnw Thank you very much in advance for your time. Ivars /etc/pam.d/dovecot #%PAM-1.0 auth required pam_nologin.so auth include password-auth account include password-auth session include password-auth /etc/pam.d/password-auth #%PAM-1.0 # This file is auto-generated. # User changes will be destroyed the next time authconfig is run. auth required pam_env.so auth [default=1 success=ok] pam_localuser.so auth [success=done ignore=ignore default=die] pam_unix.so nullok try_first_pass auth requisite pam_succeed_if.so uid >= 1000 quiet_success auth sufficient pam_sss.so forward_pass auth required pam_deny.so account required pam_unix.so account sufficient pam_localuser.so account sufficient pam_succeed_if.so uid < 1000 quiet account [default=bad success=ok user_unknown=ignore] pam_sss.so account required pam_permit.so password requisite pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type= password sufficient pam_unix.so sha512 shadow nullok try_first_pass use_authtok password sufficient pam_sss.so use_authtok password required pam_deny.so session optional pam_keyinit.so revoke session required pam_limits.so -session optional pam_systemd.so session optional pam_oddjob_mkhomedir.so umask=0077 session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid session required pam_unix.so session optional pam_sss.so doveconf -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 3.10.0-957.10.1.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: mailhost.example.com auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-master auth_username_format = %Ln auth_verbose = yes default_client_limit = 3500 default_process_limit = 500 disable_plaintext_auth = no first_valid_uid = 203 imap_client_workarounds = tb-lsub-flags tb-extra-mailbox-sep lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes mail_location = maildir:~/Maildir:INBOX=~/Maildir:LAYOUT=fs mail_plugins = " fts fts_lucene" mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve mbox_write_locks = fcntl namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = session=yes dovecot driver = pam } plugin { autocreate = Junk autocreate2 = Sent autocreate3 = Drafts autocreate4 = Trash autosubscribe = Junk autosubscribe2 = Sent autosubscribe3 = Drafts autosubscribe4 = Trash fts = lucene fts_lucene = whitespace_chars=@. imapsieve_mailbox1_before = file:/usr/lib64/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/usr/lib64/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve = file:~/sieve;active=~/roundcube.sieve sieve_before = /var/lib/sieve/junk.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /usr/lib64/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %v.%u protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = user mode = 0660 user = root } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = </etc/letsencrypt/live/webmail.example.com/fullchain.pem ssl_key = # hidden, use -P to show it syslog_facility = local0 userdb { driver = passwd } valid_chroot_dirs = /var/mail:/home protocol lmtp { mail_fsync = never mail_plugins = " fts fts_lucene sieve" postmaster_address = postmaster(a)example.com } protocol lda { mail_fsync = never mail_plugins = " fts fts_lucene sieve expire" } protocol imap { mail_max_userip_connections = 25 mail_plugins = " fts fts_lucene fts fts_squat expire imap_sieve" } protocol sieve { managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date } protocol pop3 { mail_max_userip_connections = 20 mail_plugins = " fts fts_lucene fts fts_squat expire" }

1 0