- dovecot - dovecot.org

Auth process sometimes stop responding after upgrade
by Simone Lazzaris 06 Sep '18

06 Sep '18

Hi all; I've upgraded a ring of dovecot directors from 2.2.15 to 2.2.36. After the upgrade I've got some instability: a few time per day per server, seemly at random, the auth process stop responding and the clients cannot authenticate any more: Sep 6 14:45:51 imap-front13 dovecot: pop3-login: Warning: Auth process not responding, delayed sending initial response (greeting): user=<>, rip=xx.xx.xx.xx, lip=xx.xx.xx.xx, TLS, session=<1HVqRTN10MNTie+S> I can't figure it out.... Any hints? This is my configuration: # 2.2.36 (1f10bfa63): /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-686-pae i686 Debian 7.11 # Hostname: imap-front13.mailfarm.interac.it auth_mechanisms = plain login digest-md5 cram-md5 apop scram-sha-1 auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 50 base_dir = /var/run/dovecot/ default_login_user = nobody director_doveadm_port = 9091 director_mail_servers = 192.168.1.142 192.168.1.143 192.168.1.144 192.168.1.145 192.168.1.216 192.168.1.217 192.168.1.218 192.168.1.219 director_servers = 212.183.164.157 212.183.164.158 212.183.164.159 212.183.164.160 listen = * passdb { args = /usr/local/etc/dovecot/sql.conf driver = sql } protocols = imap pop3 service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director service_count = 0 vsz_limit = 128 M } service pop3-login { executable = pop3-login director service_count = 0 vsz_limit = 128 M } ssl_cert = </usr/local/etc/dovecot/qcom.wildcard.pem ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM- SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA- AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128- SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128- SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256- SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS- AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256- SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:! PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_dh_parameters_length = 2048 ssl_key = # hidden, use -P to show it ssl_parameters_regenerate = 1 weeks ssl_prefer_server_ciphers = yes ssl_protocols = !SSlv2 !SSLv3 syslog_facility = local5 userdb { driver = prefetch } *Simone Lazzaris* *Qcom S.p.A.* simone.lazzaris(a)qcom.it[1] | www.qcom.it[2] * LinkedIn[3]* | *Facebook[4]* [5] -------- [1] mailto:simone.lazzaris@qcom.it [2] https://www.qcom.it [3] https://www.linkedin.com/company/qcom-spa [4] http://www.facebook.com/qcomspa [5] https://www.qcom.it/includes/email-banner.gif

1 0

Global duplicate filtering via Sieve
by Hanno Stock 06 Sep '18

06 Sep '18

Hi everybody, I have the following problem: Our mail users are receiving mails from externally defined mail forwardings and lists which sometimes leads to duplicate messages. As far as I understand, a single user can use the Sieve duplicate extension for filtering out such duplicate messages. Would it be feasible to add a script with sieve_before that does this for all users? Do scripts that run from sieve_before store persistent data also on a per-user basis, i.e. in the user's home dir? Thanks for your help! Best regards Hanno

2 1

Specifying other mail_attachment_dir for namespace
by Pedro Ribeiro 04 Sep '18

04 Sep '18

Hello, I have a Dovecot setup where I need to allow users to access a snapshot of their messages (when they complain about missing messages/folders deleted by mistake). From time to time the system does a LVM snapshot of the volume holding the mailboxes and sis storage and that messages are available via a "special" namespace. The problem is the messages that references attachments in the single attachment storage cause the file in some cases (when all the references were delete), the attachment itself is missing from the current SIS. Is there any option to reference a specific SIS folder for the namespace like I do for the messages as below? > namespace arquivo { > type = private > separator = / > prefix = "@ARQUIVO/" > subscriptions = yes > list = yes > inbox = no > hidden = no > ignore_on_failure = yes > location = mdbox:/mail-snap/domains/%d/%-2.2n/%n > } TIA. -- Best regards, =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Pedro Ribeiro Politécnico de Lisboa, Serviços da Presidência Departamento de Sistemas de Informação e Comunicações Phone: +351 210 464 700 (general) / VoIP: 80100 Helpdesk: helpdesk(a)net.ipl.pt / https://www.net.ipl.pt =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

1 0

Re: online conversion using replication?
by Daniel Miller 04 Sep '18

04 Sep '18

That works for a one-time migration, or perhaps via a cron-job, but what I want is basically a constant one-way backup and it seems replication could do it more elegantly & efficiently. -- Daniel On 9/1/2018 11:14 PM, Aki Tuomi wrote: > You don't need to setup replication for that. See > https://wiki2.dovecot.org/Migration/MailFormat > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Daniel Miller <dmiller(a)amfes.com> > Date: 02/09/2018 04:14 (GMT+02:00) > To: Dovecot Mailing List <dovecot(a)dovecot.org> > Subject: online conversion using replication? > > With a single server - and no intent to have a second server online at > this time - is it possible to use the replication service to keep a > "live" backup? Or otherwise perform a storage format conversion? > > I'm presently using sdbox - and considering going back to mdbox though > without SIS. My intent now is ONLY a 1-way backup, to be kept current, > and no clients will utilize the converted storage (unless/until I change). > > I've successfully executed: > > doveadm backup -u <username> -n inbox > mdbox:/var/mail/<domain>/<user>/mdbox > > For a few users - but reading the docs leads me to believe I can > automate this. But there's no explicit example for this - so I'm not > sure what to set. > > Initially I'm thinking of: > > dsync_remote_cmd = doveadm backup -u %u -n inbox > mdbox:/var/mail/%d/%n/mdbox > > and if that's right - which other services/listeners do I need to setup? > > -- > Daniel >

5 5

Re: dying on osx
by Mike Makuch 04 Sep '18

04 Sep '18

I found and tried this work around on the man page: https://www.unix.com/man-page/all/5/ngroups_max/ but I still get the same "Too many extra groups" error even when I start dovecot with the above program to limit the # of groups. I suspect that dovecot is adding a number of groups when it starts up. I've hacked a work around to get it working for me on my laptop: diff --git a/src/lib/restrict-access.c b/src/lib/restrict-access.c @@ -224,7 +224,12 @@ static void fix_groups_list(const struct restrict_access_settings *set, - if (setgroups(gid_count, gid_list) < 0) { + if (setgroups(gid_count > NGROUPS_MAX ? 16 : gid_count, gid_list) < 0) { if (errno == EINVAL) { i_fatal("setgroups(%s) failed: Too many extra groups", set->extra_groups == NULL ? "" : and this works. I'm not sure what the right solution is for a PR. Any suggestions? Thanks Mike On 8/10/18 11:04, Aki Tuomi wrote: > Is the user member of mail group? > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Mike Makuch <1mikemakuch(a)gmail.com> > Date: 10/08/2018 19:02 (GMT+02:00) > To: Aki Tuomi <aki.tuomi(a)dovecot.fi> > Cc: dovecot(a)dovecot.org > Subject: Re: dying on osx > > Maybe an old problem that has resurfaced??? > > https://bugzilla.samba.org/show_bug.cgi?id=8773 > > Mike > > > > On 8/10/18 10:54, Aki Tuomi wrote: >> I have to see if this is reproducible outside mac. >> >> >> >> --- >> Aki Tuomi >> Dovecot oy >> >> -------- Original message -------- >> From: Mike Makuch <1mikemakuch(a)gmail.com> >> Date: 10/08/2018 18:46 (GMT+02:00) >> To: Aki Tuomi <aki.tuomi(a)dovecot.fi> >> Subject: Re: dying on osx >> >> I did find that page and tried a few things there. My config has all of >> the settings there except 3: >> >> mail_access_groups = mail >> mbox_read_locks = fcntl >> mbox_write_locks = fcntl >> >> I add them to my config, restart and get the same abort >> >> 20180810-103849 imap(mkm)<79213><MkA5ihZzmdh/AAAB>: Fatal: >> setgroups(mail,505) failed: Too many extra groups >> >> I've tried numerous other settings as well. >> >> Thanks for any further advice >> >> Mike >> >> >> On 8/10/18 08:45, Aki Tuomi wrote: >> > Can you try this config and report back? >> > >> > >> https://superuser.com/questions/957272/dotlock-permissions-problems-with-do… >> > >> > Aki >> > >> >> On 10 August 2018 at 16:39 Mike Makuch <1mikemakuch(a)gmail.com> wrote: >> >> >> >> >> >> OSX 10.13.6 High Sierra, dovecot 2.3.2.1 >> >> >> >> dovecot starts up and runs but dies as soon as my mail client makes a >> >> request with log and config below. >> >> >> >> And advice appreciated. >> >> >> >> Thanks >> >> >> >> Mike >> >> >> >> >> >> >> >> 20180810-083730 auth: Debug: auth client connected (pid=77432) >> >> >> >> 20180810-083730 auth: Debug: client in: AUTH 1 PLAIN >> >> service=imap secured session=xo1p2BRzZNd/AAAB lip=127.0.0.1 >> >> rip=127.0.0.1 lport=143 rport=55140 >> >> 20180810-083730 auth: Debug: client passdb out: CONT 1 >> >> 20180810-083730 auth: Debug: client in: CONT 1 AG1rbQBta20xMjM= >> >> (previous base64 data may contain sensitive data) >> >> 20180810-083730 auth: Debug: static(mkm,127.0.0.1,<xo1p2BRzZNd/AAAB>): >> >> lookup >> >> 20180810-083730 auth: Debug: client passdb out: OK 1 user=mkm >> >> host=localhost nopasswd=y >> >> 20180810-083730 auth: Debug: master in: REQUEST 4201906177 77432 >> >> 1 b8126b4b71be2959fc7716888eccc566 session_pid=77433 >> >> request_auth_token >> >> 20180810-083730 auth-worker(77426): Debug: >> >> passwd(mkm,127.0.0.1,<xo1p2BRzZNd/AAAB>): lookup >> >> 20180810-083730 auth: Debug: master userdb out: USER 4201906177 >> >> mkm system_groups_user=mkm uid=503 gid=20 home=/Users/mkm >> >> auth_token=4d2bb44168df3d63e4e1bb352e59de632bc7da49 >> >> 20180810-083730 imap-login: Info: Login: user=<mkm>, method=PLAIN, >> >> rip=127.0.0.1, lip=127.0.0.1, mpid=77433, secured, >> >> session=<xo1p2BRzZNd/AAAB> >> >> 20180810-083730 imap(mkm)<77433><xo1p2BRzZNd/AAAB>: Fatal: >> >> setgroups(505) failed: Too many extra groups >> >> >> >> # doveconf -n >> >> # 2.3.2.1 (0719df592): /usr/local/etc/dovecot/dovecot.conf >> >> # OS: Darwin 17.7.0 x86_64 >> >> # Hostname: pine >> >> auth_debug = yes >> >> auth_debug_passwords = yes >> >> default_internal_user = _dovecot >> >> default_login_user = _dovenull >> >> disable_plaintext_auth = no >> >> listen = 127.0.0.1 >> >> log_path = /var/log/dovecot >> >> log_timestamp = "%Y%m%d-%H%M%S " >> >> mail_debug = yes >> >> mail_gid = staff >> >> mail_location = mbox:~/Mail:INBOX=/var/mail/%u >> >> mail_privileged_group = mail >> >> mail_uid = _dovecot >> >> passdb { >> >> args = password=*** host=localhost nopasswd=y >> >> driver = static >> >> } >> >> passdb { >> >> args = login >> >> driver = pam >> >> } >> >> protocols = imap >> >> service auth { >> >> user = root >> >> } >> >> service imap-login { >> >> inet_listener imap { >> >> address = * >> >> port = 143 >> >> } >> >> } >> >> ssl = no >> >> userdb { >> >> driver = passwd >> >> } >> >> >> >> >> >

2 1

doveadm variables
by Daniel Miller 03 Sep '18

03 Sep '18

Are variables such as %d and %n available to doveadm when executed from the command line? Either when explicitly declaring the user via -u or when using all users with -A? -- Daniel

1 0

Failed to get attribute vendor/vendor.dovecot/pvt/server/sieve/default
by Bob Smith 03 Sep '18

03 Sep '18

Any ideas what this cryptic message means ? Despite what the message says, there is no "more information" to be found in the logs on either server ! mail.debug dovecot: dsync-server(<REMOVED>): Debug: sieve: file storage: Created storage directory /my/dir/structure/.dovecot.sieve/tmp mxd mail.debug dovecot: dsync-server(<REMOVED>): Debug: sieve: file storage: Relative path to sieve storage in active link: .dovecot.sieve/ mxd mail.debug dovecot: dsync-server(<REMOVED>): Debug: sieve: file storage: sync: Synchronization active Error: Mailbox INBOX: Failed to get attribute vendor/vendor.dovecot/pvt/server/sieve/default: Internal error occurred. Refer to server log for more information. [2018-09-03 09:52:46] mail.debug dovecot: dsync-server(<REMOVED>): Debug: brain S: Import INBOX: Import attribute vendor/vendor.dovecot/pvt/server/sieve/default: failed mail.debug dovecot: dsync-server(<REMOVED>): Debug: brain S: Import INBOX: Saved UIDs: No errors are reported by "doveadm -v backup my@user tcp:mydest:myport". doveadm-backup returns exit code 75, something which also appears to be documented nowhere !

2 1

Re: Dovecot User Listing Error - getpwent() failed: Invalid Argument
by Aki Tuomi 03 Sep '18

03 Sep '18

Reuben, could you try this: service auth-worker { executable = /usr/bin/strace -o /tmp/out /usr/libexec/dovecot/auth -w } and send the out file to me? I wasn't able to reproduce your problem with glibc-2.28. Aki On 31.08.2018 21:30, Aki Tuomi wrote: > Ah. Thanks for the pointer. > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Michael Marley <michael(a)michaelmarley.com> > Date: 31/08/2018 21:17 (GMT+02:00) > To: dovecot(a)dovecot.org > Subject: Re: Dovecot User Listing Error - getpwent() failed: Invalid > Argument > > I think this might be caused by glibc 2.28. I saw a similar error > with the "doveadm purge" command after upgrading my system to that > version. > > Michael Marley > > On 2018-08-31 13:10, Aki Tuomi wrote: > >> I'll see if this is reproducible >> >> >> >> --- >> Aki Tuomi >> Dovecot oy >> >> -------- Original message -------- >> From: Reuben Farrelly <reuben-dovecot(a)reub.net> >> Date: 31/08/2018 17:41 (GMT+02:00) >> To: Aki Tuomi <aki.tuomi(a)dovecot.fi>, Dovecot Mailing List >> <dovecot(a)dovecot.org> >> Subject: Re: Dovecot User Listing Error - getpwent() failed: Invalid >> Argument >> >> Sure: >> >> https://www.reub.net/files/dovecot/lightning-dovecot.conf >> https://www.reub.net/files/dovecot/thunderstorm-dovecot.conf >> >> Updated nightly. >> >> Reuben >> >> >> On 1/09/2018 12:26 am, Aki Tuomi wrote: >> > Can you provide doveconf -n? >> > >> > >> > >> > --- >> > Aki Tuomi >> > Dovecot oy >> > >> > -------- Original message -------- >> > From: Reuben Farrelly <reuben-dovecot(a)reub.net> >> > Date: 31/08/2018 17:12 (GMT+02:00) >> > To: Aki Tuomi <aki.tuomi(a)dovecot.fi>, Dovecot Mailing List >> > <dovecot(a)dovecot.org> >> > Subject: Re: Dovecot User Listing Error - getpwent() failed: Invalid >> > Argument >> > >> > No. Neither of those are installed on either system. >> > >> > Reuben >> > >> > On 1/09/2018 12:09 am, Aki Tuomi wrote: >> > > Could apparmor or selinux be causing this! >> > > >> > > >> > > >> > > --- >> > > Aki Tuomi >> > > Dovecot oy >> > > >> > > -------- Original message -------- >> > > From: Reuben Farrelly <reuben-dovecot(a)reub.net> >> > > Date: 31/08/2018 16:50 (GMT+02:00) >> > > To: Dovecot Mailing List <dovecot(a)dovecot.org> >> > > Subject: Dovecot User Listing Error - getpwent() failed: Invalid >> Argument >> > > >> > > Hi, >> > > >> > > I'm running dovecot-2.3 git and seeing on 2 of my Dovecot >> installations, >> > > the following message logged quite frequently: >> > > >> > > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: auth-worker(7707): >> > > Error: getpwent() failed: Invalid argument >> > > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: >> > > User listing returned failure >> > > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: >> > > listing users failed, can't replicate existing data >> > > >> > > It appears that this could be having an effect on replication, >> but it's >> > > unclear because the secondary/replica doesn't see much traffic. >> > > >> > > Authentication is via PAM and the system is Gentoo Linux x86_64. >> passb >> > > driver = pam, userdb driver = static . >> > > >> > > doveadm user -u '*' shows the system users listed twice like >> this (is >> > > this a problem?) and then prints: >> > > >> > > ---- >> > > reuben >> > > liam >> > > reuben >> > > liam >> > > <some other users> >> > > Error: User listing returned failure >> > > Fatal: user listing failed >> > > ---- >> > > >> > > To me that doesn't look right. >> > > >> > > Can anyone suggest what could be causing this? I don't recall >> seeing >> > > this message some time ago so I suspect it's a recent change in >> either >> > > dovecot, or pam or glibc etc.. >> > > >> > > Thanks, >> > > Reuben > >

1 0

online conversion using replication?
by Daniel Miller 02 Sep '18

02 Sep '18

With a single server - and no intent to have a second server online at this time - is it possible to use the replication service to keep a "live" backup? Or otherwise perform a storage format conversion? I'm presently using sdbox - and considering going back to mdbox though without SIS. My intent now is ONLY a 1-way backup, to be kept current, and no clients will utilize the converted storage (unless/until I change). I've successfully executed: doveadm backup -u <username> -n inbox mdbox:/var/mail/<domain>/<user>/mdbox For a few users - but reading the docs leads me to believe I can automate this. But there's no explicit example for this - so I'm not sure what to set. Initially I'm thinking of: dsync_remote_cmd = doveadm backup -u %u -n inbox mdbox:/var/mail/%d/%n/mdbox and if that's right - which other services/listeners do I need to setup? -- Daniel

2 1

remove non-standard flags
by Daniel Miller 02 Sep '18

02 Sep '18

Having relocated my virtual folders to a new namespace - I'm having some subscription challenges. I'm able to reach them via telnet without issue - and some clients have better luck than others. Some folders have no problems at all - some seem different. In trying to identify what might be the issue - at the moment the only thing I'm seeing is the ones that are completely without issue don't have any "weird" flags. Example: an apparently "clean" virtual folder: . select virtual/Flagged * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded \*)] Flags permitted. * 12 EXISTS * 0 RECENT * OK [UNSEEN 7] First unseen. * OK [UIDVALIDITY 1535841027] UIDs valid * OK [UIDNEXT 13] Predicted next UID * OK [HIGHESTMODSEQ 3] Highest . OK [READ-WRITE] Select completed (0.316 + 0.000 + 0.315 secs). This folder is recognized by all clients. But this one: . select virtual/INBOX * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded junk $MDNSent) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded junk $MDNSent \*)] Flags permitted. * 9540 EXISTS * 0 RECENT * OK [UNSEEN 85] First unseen. * OK [UIDVALIDITY 1535841672] UIDs valid * OK [UIDNEXT 9541] Predicted next UID . OK [READ-WRITE] Select completed (44.266 + 0.000 + 44.265 secs). Is a little more troublesome. And then... . select virtual/Archives * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent unknown-0 unknown-2 unknown-3 unknown-5 unknown-9 unknown-4 $Forwarded unknown-7 unknown-8 junk $NotJunk $label5) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent unknown-0 unknown-2 unknown-3 unknown-5 unknown-9 unknown-4 $Forwarded unknown-7 unknown-8 junk $NotJunk $label5 \*)] Flags permitted. * 142605 EXISTS * 0 RECENT * OK [UNSEEN 82998] First unseen. * OK [UIDVALIDITY 1535841824] UIDs valid * OK [UIDNEXT 142606] Predicted next UID . OK [READ-WRITE] Select completed (10.217 + 0.000 + 10.217 secs). I can't get Thunderbird to recognize it all (though RoundCube web client works). There very well may be something totally separate from the flags that's causing the problem, but, is there a way to strip the flags off? I tried: doveadm flags remove -u <username> '$MDNSent unknown-0 unknown-2 unknown-3 unknown-5 unknown-9 unknown-4 unknown-7 unknown-8 junk $NotJunk $label5' mailbox 'inbox/Archive/2004' since that's one of the source folders of the virtual folder - no error, but no apparent change. Same for operating on the virtual folder directly. -- Daniel

1 0