July 2020 - dovecot - dovecot.org

How to backup maildir
by Robert Moskowitz 04 Sep '20

04 Sep '20

I have been thinking, and reading, on how to back up my mailserver. I have not found any approach that seems ready to use. I have run years without any backup, but would really like to have something in place. I figure I can attach a USB drive and backup to that, then from there rsync to something elsewhere. Further if that USB drive is a full mailserver image, I actually have a 'hot backup' where I only have to put the backup drive into a system and boot up at the last backup. But this means properly copying all of /home/vmail and probably /home/sieve plus the /var/lib/mysql Are there good tools that nicely does this? Or do I choose a time late at night (only I am sometimes in non-US timezones) to shut down all services and just use rsync? And stopping services itself is thought provoking. What if Dovecot, amavis, mysql, or whatelse is in the middle of writing out a mail file what happens to that file and restart. Just scary stuff and, in part, why I have never tackled this in the past. thanks for all feedback

4 6

Rootless wiki page is not up to date?
by Marc Roos 20 Aug '20

20 Aug '20

If I run a docker image with the mesos containerizer and altering the dovecot config mentioned here[1] I think the root detection is incorrect because it looks like dovecot is still thinking it is root. I still get errors like: >> log(829825): Fatal: We couldn't drop root group privileges (wanted=10053(dovecot), gid=0(root), egid=0(root)) Why does it want to drop to root, if we are not even running as root? >> Error: service(ipc): chown(/var/dovecot/login/ipc-proxy, 91, 4294967295) failed: Operation not permitted Why does it want to chown, when it is not root? pop3-login: Fatal: setgid(101(dovenull)) failed with euid=10053(dovecot), gid=10053(dovecot), egid=10053(dovecot): Operation not permitted (This binary should probably be called with process group set to 101(dovenull) instead of 10053(dovecot)) [1] https://wiki.dovecot.org/HowTo/Rootless?action=edit&editor=text

2 2

Correcting Mail Home, Maildir and Sieve folder locations
by David White 12 Aug '20

12 Aug '20

I have Dovecot 2.3.7.2 on Ubuntu 20.04. I just migrated the server from CentOS 8, and only migrated to the CentOS 8 machine a few months ago off of CentOS 7. I am pasting `doveconf -n` at the bottom of this email. Everything is working. However, I'm trying to clean up warnings in the logs, and just discovered that I'm not managing home directories properly. (c/f https://wiki.dovecot.org/VirtualUsers/Home) This is in the logs: > Jun 28 06:09:50 mail.example.com dovecot: imap(email(a)example.com)<3786><fICEJCKpbfatQHg1>: Error: stat(/var/vmail/example.com/email@example.com/.dovecot.sieve/tmp) failed: Not a directory Can I please get a sanity check? To resolve, I've come up with a bash loop to create a home folder, and fix the existing structure for each mailbox on the system. I ran this for 1 of the domains on the system. > for i in */ ; do mv "$i" mail; mkdir "$i"; mv mail "$i"; mv "$i"/.dovecot* "$i"; done Then, I would simply edit /etc/dovecot/conf.d/10-mail.conf so that it looks like this: > mail_home = /var/vmail/%d/%u > mail_location = maildir:/var/vmail/%d/%u/mail To confirm, should all of the 'hidden' dovecot files be moved to the user's new home folder like I'm trying to do in the above script? Or are there other files that I will need to move? After running that loop above on a test server (don't worry, I would never do this in production without testing), one of the mailbox's home folder on the server looks like this: > total 56 > drwxr-xr-x 4 vmail vmail 4096 Jun 28 06:19 . > drwx------ 6 vmail vmail 4096 Jun 28 06:09 .. > -rw------- 1 vmail vmail 54 Jul 4 2019 .dovecot.lda-dupes > lrwxrwxrwx 1 vmail vmail 21 May 26 2016 .dovecot.sieve -> sieve/roundcube.sieve > -rw------- 1 vmail vmail 0 Oct 17 2016 .dovecot.sieve.log > -rw------- 1 vmail vmail 218 Jul 6 2019 .dovecot.svbin > drwx------ 277 vmail vmail 32768 Jun 28 06:19 mail > drwx------ 3 vmail vmail 4096 Feb 2 17:31 sieve ... and connecting to that mailbox over Thunderbird seems to be working fine. But I want to make sure I'm not missing anything. Here is my doveconf -n, with identifying information snipped out: > root@mail:# doveconf -n > # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.7.2 () > # OS: Linux 5.4.0-39-generic x86_64 Ubuntu 20.04 LTS ext4 > # Hostname: {snip} > auth_mechanisms = plain login > mail_location = maildir:/var/vmail/%d/%u > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext > namespace inbox { > inbox = yes > location = > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Spam { > auto = create > special_use = \Junk > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > prefix = > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > sieve = file:~/sieve;active=~/.dovecot.sieve > } > protocols = imap pop3 lmtp sieve > service auth-worker { > user = $default_internal_user > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-userdb { > group = postfix > mode = 0666 > user = vmail > } > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0666 > user = postfix > } > } > ssl = required > ssl_ca = </etc/ssl/certs/snip.ca-bundle > ssl_cert = </etc/ssl/certs/snip.crt > ssl_cipher_list = ALL:!LOW:!SSLv3:!EXP:!aNULL:!MD5 > ssl_key = # hidden, use -P to show it > userdb { > args = uid=vmail gid=vmail home=/var/vmail/%d/%u > driver = static > } Sent with ProtonMail Secure Email.

1 1

Can't move/delete folder "Renaming not supported across conflicting directory permissions"
by Johannes Rohr 12 Aug '20

12 Aug '20

Dear all, several users have reported that they were unable to move/delete folders using Thunderbird and they got the message "Renaming not supported across conflicting directory permissions". When I google for this message, I see hints like "your mail may not be all on the same volume", but here this is definitely not the case. All mail is in /var/mail/virtual and owned by mail:mail Curiously, when I tried the same operation in Roundcube, there was no error reported, so something about the combination of Thunderbird and Dovecot seems to be causing the error. Does anyone have advise on how I should go about tracking down the cause of the error? Thanks a lot in advance! Johannes

3 2

Client shows incorrect message counts
by Antonio Leding 11 Aug '20

11 Aug '20

Hello all… First, a disclaimer…I am aware that this is very likely not a Dovecot related issue but I am truly pulling whatever hair I have left out of my head and given the brain-power of this mailer, I thought someone might know what is going in here. The issues is this… I check the message counts for a given account using doveadm and get a given number. I then configure the same account in 3 mail clients: Apple Mail, Mailmate, and Thunderbird. Both the Mailmate and Thunderbird message counts match exactly with the doveadm numbers. Apple Mail is low in almost all cases. I have tried every idea I have found including: — Delete and recreate the mail account — Rebuild mailbox — Resync mailbox — Reindex Spotlight — Removing the "~/Library/Mail/V*/Envelope*” files; restart Mail — Remove the entire "~/Library/Mail/V*” directory; restart Mail So, at wit’s end and again, apologies for this not being a Dovecot-centric issue but I know you are all on top of this stuff and may have an idea… Thanks in advance...

2 7

Question about migration 2.2 - 2.3 with replication
by Ralf Becker 03 Aug '20

03 Aug '20

Do both replication nodes need to be updated at the same time? Or can a 2.2(.36.4) node replicate with a 2.3(.10.1)? Ralf -- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Geschäftsführer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0

1 1

Trying to use solr
by Francis Augusto Medeiros-Logeay 02 Aug '20

02 Aug '20

Hi, I am finally trying to use an fts engine with dovecot. My version is 2.3.7.2 under Ubuntu 20.04. I installed solr 7.7.3 and then 8.6.0 to see if this was a version-related error. I copied the schema from 7.7.0 as many people said this was fine. I get the following error when trying to reindex a user's mailbox: doveadm(francis(a)francisaugusto.com): Error: fts_solr: Indexing failed: 400 Bad Request doveadm(francis(a)francisaugusto.com): Error: Mailbox INBOX: Transaction commit failed: FTS transaction commit failed: backend deinit doveadm(francis(a)francisaugusto.com): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb: Disconnected: Connection closed (fd=10) On Solr I get this error: org.apache.solr.common.SolrException: Exception writing document id 210/9fd7941e8297d25d91600000c3fdd3da/francis(a)francisaugusto.com to the index; possible analysis error: cannot change field "box" from index options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index options=DOCS at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:246) Parallel to this, I got some log messages on Solr before attempting to reindex the user (sorry for the garbadged text: TIME (LOCAL) LEVEL CORE LOGGER MESSAGE 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieLongField]. Please consult documentation how to replace it accordingly. 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.SynonymFilterFactory]. Please consult documentation how to replace it accordingly. 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.WordDelimiterFilterFactory]. Please consult documentation how to replace it accordingly. 7/22/2020, 6:43:46 PM ERROR false x:dovecot RequestHandlerBase java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:46 PM ERROR false x:dovecot HttpSolrCall null:java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:49 PM ERROR false x:dovecot RequestHandlerBase java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:49 PM ERROR false x:dovecot HttpSolrCall null:java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:56 PM ERROR false x:dovecot RequestHandlerBase java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:56 PM ERROR false x:dovecot HttpSolrCall null:java.lang.IllegalSta Could anyone point me the right direction to make this work? Best,

3 5

Migrating from qmail server, invalid option "o" in dsync
by Justin Coleman 01 Aug '20

01 Aug '20

Hi folks, I have an old server running qmail with several years of mails in maildir format I want to migrate to a new server, without requiring fresh client download of everything. Old server is debian 6(?). Local IP is 192.168.0.1. New server is Debian 10, local IP is 192.168.0.25. I have dovecot 2.3.4.1 (latest from apt stable) installed and apparently working - if I manually copy the home directory for my account, I can connect (using thunderbird) and I get all the emails appearing as new, forcing redownload. Seems like the new config is at least functional. I'm trying to follow the migration guide at https://wiki2.dovecot.org/Migration/Dsync, but I am getting an error message: root@newserver:~# doveadm backup -R -u _user_ -o pop3c_user=_user_ -o pop3c_password=_password_ backup: invalid option -- 'o' doveadm backup [-u <user>|-A] [-S <socket_path>] [-fPRU] [-l <secs>] [-r <rawlog path>] [-m <mailbox>] [-g <mailbox_guid>] [-n <namespace> | -N] [-x <exclude>] [-s <state>] [-t <start date>] -d|<dest> Have tried this a couple different ways and according to everything i can find -o should be valid. Can anyone tell me what I'm doing wrong here? Thanks! # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-9-amd64 x86_64 Debian 10.4 # Hostname: newserver.domain.com disable_plaintext_auth = no mail_location = maildir:~/Maildir mail_privileged_group = mail namespace { hidden = yes list = no location = pop3c: prefix = POP3-MIGRATION-NS/ } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX } pop3_uidl_format = %f pop3c_host = 192.168.0.1 protocols = " imap pop3" ssl = no ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol doveadm { mail_plugins = " pop3_migration" }

4 6

imap doesn't like my passwd
by James 31 Jul '20

31 Jul '20

I spent days on this problem. I have dovecot setup to use SSL and IMAP. I also have a pop account that I had no trouble getting working. Both use PAM authentication on Linux. openssl s_client -connect localhost:993 a login test_user test_secure_password The response I got back was: a BAD Invalid characters in atom I changed the password to "abc" and it worked. The secure password used to work but it was a different architecture (x86_64 I think). The current architecture is armhf. It may be different versions of the software too. Dovecot v2.1.7

5 7

dovecot replication and pop3 deletion
by Nigel Metheringham 30 Jul '20

30 Jul '20

Hi, I am investigating the use of dovecot for a voicemail storage system (which is effectively a SMTP/POP3 system), which we want to use replication to give us some resilience. Tests have mostly been on dovecot 2.3.8 (9df20d2db) - as packaged on Centos 8 - but I have also used a couple of similar versions on different platforms, so this is not a specific build problem. The platform has replication working, over an SSL TCP link, for delivery. With the pop3 configuration as default as is possible (although the mail_plugins are explicitly included into the protocol sections to ensure notify and replication is called for), any pop3 delete operation does not cause a notify or replication operation. The two servers show the mailboxes in a different state up until something else prods a replication into progress. I would suggest that a pop3 delete/expunge of a message should be flagged as a mailbox change can cause a notify/replication, and that the current behaviour is a bug. There are workarounds to this, and I am considering adding a note to the wiki page https://wiki.dovecot.org/Replication to flag this - not sure what your wiki editing policy is. # Workaround 1 - Change pop3 deletion to set a flag If the setting pop3_deleted_flag = '$POP3Deleted' is added, then rather than the mail being deleted, it is flagged, and this change causes a notification and replication to the other server. However since the mail is not actually deleted, you additionally need another (cron?) task to periodically remove flagged mail, for example:- doveadm expunge -A MAILBOX inbox KEYWORD '$POP3Deleted' # Workaround 2 - Use Lazy Expunge You can use the lazy_expunge plugin - this has the same effect, and automates the removal of the messages after a period of time. https://doc.dovecot.org/configuration_manual/lazy_expunge_plugin/ For example, you can set the following changed config fragments:- mail_plugins = $mail_plugins notify replication lazy_expunge namespace inbox { separator = / prefix = inbox = yes # automatic lazy expunge mailbox mailbox .EXPUNGED { autoexpunge = 3 days autoexpunge_max_mails = 1000 } } plugin { # Move messages to an .EXPUNGED mailbox lazy_expunge = .EXPUNGED } Nigel. -- [ Nigel Metheringham --------------------------- nigel(a)dotdot.cloud ] [ Ellipsis Intangible Cloudy Technologies ]

1 0