July 2020 - dovecot - dovecot.org

Syncing emails from external server like Gmail
by Vitalii 14 Jul '20

14 Jul '20

Greeting I've managed to backup my emails from external email server via dsync and imapc: protocol like this: doveadm -Dv -o imapc_user='user(a)gmail.com' -o imapc_password="pass" -o imapc_host=imap.gmail.com backup -R -u duke(a)ejoom.com imapc: taken from https://wiki.dovecot.org/Migration/Dsync I wish to do regular syncing of 5min interval to substitute 'getmail' program. When I repeat the command It doesn't download all the attachments(!good) but It seems that at retrieve all the the GUIDs and UIDs(!bad) again that take quite long considering 30K emails. The question is how to speed it up and retrieve only new emails? Does anyone have a solution for regular synchronization? What is the best way? Logs look like this: https://gist.githubusercontent.com/vnagara/87e0da1b21b51f68fd74270b3ec653b3… It wrights that "Mailboxes are equal" but still verifies all the UIDs again. Thanks.

4 11

Fwd: Dsync mail migration from v2.2 --> v2.3
by Antonio Leding 14 Jul '20

14 Jul '20

Two items I forgot to mention in my initial post: (A) I am running the sync command on the new v2.3 server. (B) To get sync to work, I had to add the following to the old v2.2 server: auth_socket_path = /var/run/dovecot/auth-master I think this relates to what I said in my initial post about AWS linux installing all of the sockets into a non-standard location. I say this because when I was trying get this working but before I had added the above setting, I noticed the following message: Error: userdb lookup: connect(/usr/local/var/run/dovecot/auth-userdb) failed: No such file or directory This seemed to indicate that the old server was looking for the auth socket in the incorrect location. So, I decided to correct this with the setting I noted above. Please let me know if I had this correct or perhaps there is something else going on here… Thanks... > Begin forwarded message: > > From: Antonio Leding <tech(a)leding.net> > Subject: Dsync mail migration from v2.2 --> v2.3 > Date: July 13, 2020 at 8:45:18 PM PDT > To: dovecot(a)dovecot.org > > Hello all, > > I’ve been battling how to migrate my imap mail following a new server install that has Dovecot and Postfix upgrades. > > The research seems pretty clear that the preferred method is to use dsync to get the mail from Server-A to B. After several hours and several different combos, I think I might finally have this working. However, before committing to my procedure and running it against all mailboxes, I wanted to bounce this off of the community and get any feedback that might catch possible issues and\or offer some advice. > > Environment > > Old server: Dovecot 2.2.2 running on AWS linux 1 (I believe that is based on RHEL 4 or 5). > New server: Dovecot 2.3.10.1 running on CentOS 7.6. > All authentication is done by Dovecot against a mySQL (and Maria) database — nothing is in files. > All mail data and mailboxes are owned by a dedicated user “vmail” and I’m using a static userdb. > SSH is via public key file only — no PW based SSH is allowed. > SSH is on a non-standard port > > Operations > > I’ve decided to migrate one mailbox per command. I only have about 30 accounts on the server so this shoudn't be a huge deal. > Here is the command that I believe is good to go: > > sudo doveadm -Dv sync -u [ MBOX_TO_MIGRATE ] remote:vmail@[ SERVER ] > > The SERVER references a host stanza in root's SSH connection file: /root/.ssh/config > I need to use the config file because I (a) use a non-standard SSH port; (b) need to login as “vmail” ; (c) need to use the “vmail" identity file. > > Comments & notes > > The Dovecot 2.2 package for AWS Linux was apparently setup to install all of the sockets in /usr/local/var/run/dovecot. This must have caused authentication issues because to get it working properly, I then had to add "path = /var/run/dovecot/auth-master” statement to the “auth" stanza in dovecot.conf file. > Also, the fact that I do not use a “standard” SSH config was an issue early on but once I committed to allowing vmail to SSH into the old server, that part became moot. For security purposes, I will probably disable vmail SSH after the data is migrated. > > Questions > > First and foremost — any obvious issues here? > Next, any advice and\or lessons learned with this type of migration? > Finally, presuming this is the proper way to do this, will this migrate all mail content including Sent, Deleted, Drafts, etc.? > > > Thanks in advance for taking the time and providing any feedback — very much appreciated... > >

1 0

Dsync mail migration from v2.2 --> v2.3
by Antonio Leding 14 Jul '20

14 Jul '20

Hello all, I’ve been battling how to migrate my imap mail following a new server install that has Dovecot and Postfix upgrades. The research seems pretty clear that the preferred method is to use dsync to get the mail from Server-A to B. After several hours and several different combos, I think I might finally have this working. However, before committing to my procedure and running it against all mailboxes, I wanted to bounce this off of the community and get any feedback that might catch possible issues and\or offer some advice. Environment Old server: Dovecot 2.2.2 running on AWS linux 1 (I believe that is based on RHEL 4 or 5). New server: Dovecot 2.3.10.1 running on CentOS 7.6. All authentication is done by Dovecot against a mySQL (and Maria) database — nothing is in files. All mail data and mailboxes are owned by a dedicated user “vmail” and I’m using a static userdb. SSH is via public key file only — no PW based SSH is allowed. SSH is on a non-standard port Operations I’ve decided to migrate one mailbox per command. I only have about 30 accounts on the server so this shoudn't be a huge deal. Here is the command that I believe is good to go: sudo doveadm -Dv sync -u [ MBOX_TO_MIGRATE ] remote:vmail@[ SERVER ] The SERVER references a host stanza in root's SSH connection file: /root/.ssh/config I need to use the config file because I (a) use a non-standard SSH port; (b) need to login as “vmail” ; (c) need to use the “vmail" identity file. Comments & notes The Dovecot 2.2 package for AWS Linux was apparently setup to install all of the sockets in /usr/local/var/run/dovecot. This must have caused authentication issues because to get it working properly, I then had to add "path = /var/run/dovecot/auth-master” statement to the “auth" stanza in dovecot.conf file. Also, the fact that I do not use a “standard” SSH config was an issue early on but once I committed to allowing vmail to SSH into the old server, that part became moot. For security purposes, I will probably disable vmail SSH after the data is migrated. Questions First and foremost — any obvious issues here? Next, any advice and\or lessons learned with this type of migration? Finally, presuming this is the proper way to do this, will this migrate all mail content including Sent, Deleted, Drafts, etc.? Thanks in advance for taking the time and providing any feedback — very much appreciated...

1 0

Re: Dovecot permission denied errors on NFS after upgrade to 2.2.17
by Claudio Corvino 13 Jul '20

13 Jul '20

Hi Jochen, the ID has not changed, in addition the "permission denied" error is random, most of the time Dovecot works well. How can I check the mappings NFS uses? After the Debian upgrade I was using these NFS mount options: rw,user,rsize=8192,wsize=8192,timeo=30,intr Now trying to fix the problem I set "defaults" mount option, but nothing has changed. Thanks On 13/07/20 13:08, Jochen Bern wrote: > On 07/13/2020 12:57 PM, Claudio Corvino wrote: >> we just upgraded our mail server from Debian 8 to Debian 9, so our >> dovecot packages just got upgraded from version 2.2.13 to 2.2.17. > [...] >> Every directory and sub-dir under >> /mnt/mail-storage >> is assigned to vmail:vmail with >> drwx------ >> permission set. > Just a quick shot into the dark, you verified the ownerships on the NFS > *client*, not the NFS *server*, yes? Because NFS speaks UIDs and GIDs, > not their names, so if your Debian 9 installation assigned different IDs > to "vmail" ... > > (You might also want to check what mappings the NFS mount uses, > including possible *default* behaviour.) > > Good luck,

1 0

Dovecot permission denied errors on NFS after upgrade to 2.2.17
by Claudio Corvino 13 Jul '20

13 Jul '20

Hi everyone, we just upgraded our mail server from Debian 8 to Debian 9, so our dovecot packages just got upgraded from version 2.2.13 to 2.2.17. Now we have a lot of errors in our dovecot.log about permission denied (our Mailboxes are on NFS remote server since years without problems) like these: Jul 13 12:39:23 Error: imap(XXXXXXXXX): open(/mnt/mail-storage/XXXXXXXXX/Maildir/cur/1594378191.M653755P26498.ganimede,S=12716,W=12984:2,S) failed: Permission denied (euid=501(vmail) egid=501(vmail) missing +r perm: /mnt/mail-storage/XXXXXXXXX/Maildir/cur/1594378191.M653755P26498.ganimede,S=12716,W=12984:2,S stat(/mnt/mail-storage/XXXXXXXXX/Maildir/cur/1594378191.M653755P26498.ganimede,S=12716,W=12984:2,S) failed: Permission denied) For some user this is not a problem, for some other user this blocks their mail client (Thunderbird). We use vmail user for running Dovecot and I can't find a permission problem on the path indicated by the mail error, user vmail can read and write and uid and gid seems to be correct. Permissions on /mnt/mail-storage was drwx------ 94 vmail vmail 4096 ago 29 2019 mail-storage I just changed to drwxr-xr-x 94 root root 4096 ago 29 2019 mail-storage but nothing has changed. Every directory and sub-dir under /mnt/mail-storage is assigned to vmail:vmail with drwx------ permission set. If try to stat on /mnt/mail-storage/XXXXXXXXX/Maildir/cur/1594378191.M653755P26498.ganimede,S=12716,W=12984:2,S I get permission denied. We use LMTP and Postfix to deliver mail to mailbox. mail_debug=yes already enabled; mail_fsync is set to never (should I change to always?). Please could you give us help on the matter as I don't know where to look further. Thanks Claudio

1 0

Re: Please Help me
by Kishore Potnuru 13 Jul '20

13 Jul '20

Hi Vitalii, I am using "Maildir". Thanks Kishore Potnuru On Mon, Jul 13, 2020 at 10:32 AM Vitalii <vnagara(a)yandex.com> wrote: > On Sun, 12 Jul 2020 21:17:48 +0100 > Kishore Potnuru <kishore.reachme(a)gmail.com> wrote: > > > Hi All, > > > > I need to migrate the complete data & setup/environment from an old > > dovecot environment to a new dovecot environment. > > Hi Kishore, > I was wondering what mail format do you use Maildir/mdbox/sdbox? >

1 0

Please Help me
by Kishore Potnuru 13 Jul '20

13 Jul '20

Hi All, I need to migrate the complete data & setup/environment from an old dovecot environment to a new dovecot environment. Old environment details: ======================== Linux OS - Red Hat Enterprise Linux Server release 6.5 (Santiago) Dovecot version - 2.0.9 Postfix version - 2.6.6 New Environment details: ( 2 systems) ======================== Linux OS - Red Hat Enterprise Linux Server release 7.7 (Maipo) Dovecot version - 2.2.36 (1f10bfa63) Postfix version - 2.10.1 In the new environment, I have 2 servers. I have got 2 of the above servers with above configuration and need to create HA/Resilience. I know there is advanced versions in Redhat and Dovecot. As per my organisation's existing support/policy, that is what maximum supported RHEL and official version of dovecot as per RHEL 7.7. Now my questions are: 1. I will stop the dovecot/postfix services on the server in old environment. Is it possible to move/copy the complete configuration along with data (directories/sub-directories/users/groups/existing email in new or cur folders/complete email domains along with complete structure) to the other 2 servers in new env? Could you please suggest me the commands or steps follow that? 2. Now, With the above 2 servers, I would like to create the HA/Resilience. I know there is a Replication options which are there. Could you please provide me the steps/video, how it needs to be done? I have got the systems and I need to move/copy the environment to a new environment and create the HA/Resilience. I am not sure, how to proceed and what needs to be done clearly? I am stuck here. Please help me in this step by step procedure. Thanks, Kishore Potnuru

3 2

INBOX folders disappeared after upgrading
by Dave 12 Jul '20

12 Jul '20

Hi, tried to Google/etc to see what to tweak.. not finding anything helpful. So.. I've been running my private mail through a Linux server since approx year 2000. The OS has been various flavors of Fedora or CentOS, since for work, I have to spend all day supporting some sort of Red Hat flavor. So I run SquirrelMail on top of Dovecot and Postfix on CentOS 7, things have been great for like forever. I recently ran updates and all my INBOX folders disappeared. I have made no configuration changes in at least 5 years. I can't seem to resolve/figure out what's going on. I have backups to see everything before the update. I've noticed that my subscriptions file is missing all the folders, which are all in this file in the backups. If I repopulate this file with the names, the next time I access mail, they are all removed. Most of my mail is in folders, therefore I have access to almost none of it (can see new stuff coming in, but can't sort it anywhere). k, for the details.. I was running dovecot-2.2.10 (2.2.10-8.el7.x86_64) and was updated to dovecot-2.2.36 (2.2.36-3.el7_7.1.x86_64) with yum updates. # dovecot --version 2.2.36 (1f10bfa63) # dovecot -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-1062.18.1.el7.x86_64 x86_64 CentOS Linux release 7.7.1908 (Core) # Hostname: vm1.tech...com auth_mechanisms = plain login login_greeting = IMAP ready. mail_location = maildir:~/Maildir mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0600 user = postfix } } service imap-login { inet_listener imap { address = 127.0.0.1,68.139.25.150 } inet_listener imaps { address = 127.0.0.1,68.139.25.150 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = 127.0.0.1,68.139.25.150 } } ssl = required ssl_cert = </etc/pki/tls/certs/mail.tech...com.crt ssl_cipher_list = ALL:!aNULL:!eNULL:!LOW:!EXP:!RC4 ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = passwd } Regards, -Dave

2 4

2.3.10.1 on OpenBSD
by Rupert Gallagher 12 Jul '20

12 Jul '20

Apart from a few warnings (attached), and a bit of patience, the compiler terminates without error. On patience, OBSD has two processes who took almost half the CPU when compiling: reaper and zerothread. Will look into the warnings, and the testing errors (also attached). Let see how it works on this machine. OpenBSD 6.7 (GENERIC.MP) #2: Thu Jun 4 09:55:08 MDT 2020 real mem = 17125511168 (16332MB) avail mem = 16593879040 (15825MB) bios0: Supermicro A2SDi-4C-HLN4F OpenBSD clang version 8.0.1 (tags/RELEASE_801/final) (based on LLVM 8.0.1) Target: amd64-unknown-openbsd6.7 Thread model: posix

3 6

Per-user sieve_before settings?
by Gionatan Danti 11 Jul '20

11 Jul '20

Hi all, I wonder if using a per-user sieve_before setting (ie: a relative path to user home, as ~/filter.sieve) is expected to work corretly. Full disclaimer: I am migrating an old CentOS 6.10 box with dovecot 2.0.9 and an old pigenhole version. I now want to redirect emails from old account to new account (I have a csv with old->new address pairs). My first approach using a single, big filter file somewhat works, but it is not optimal. I am currently setting sieve_before = /var/vmail/autoreply.sieve, and it that file I have something as: ... if header :contains "to" "aaa(a)olddomain.com" { redirect "aaa(a)newdomain.com"; stop; } if header :contains "to" "bbb(a)olddomain.com" { redirect "bbb(a)newdomain.com"; stop; } ... Problem is that whan an email has multiple to or cc, the filter is evaluated multiple times (one per recipient) and I end with multiple email copies on the same destination address. For example, sending and email to aaa(a)olddomain.com, cc bbb(a)olddomain.com is evaluated *two* times: when delivering the email to aaa(a)olddomain.com, with a corresponding redirect to aaa(a)newdomain.com, and when delivering the email to bbb(a)olddomain.com, with *another* redirect to aaa(a)newdomain.com (ending with two email copies). Using a per-user filter file solves the problem, as the filter is evaluated in the context of the right user only (ie: an email sent to aaa(a)olddomain.com, cc bbb(a)olddomain.com is evaluated only in aaa(a)olddomain.com context, with a corresponding single email on aaa(a)newdomain.com) However, I would avoid manipulating the already present per-user file (roundcube.sieve), as I need to add my custom rules on top of that file. Moreover, users would be able to disable the redirect filter. Using a per-user before_sieve filter file seems the right solution, *if* it works using multiple files rather than a single global one. If not, do you have any other suggestion/approach to share? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it [1] email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

1 1