August 2018 - dovecot

Does domain quota work and how?
by Kai Schaetzl 01 Sep '18

01 Sep '18

Hello, I'm in the process of moving users from an old dovecot setup with system users to a modern virtual user setup. I can't find good documentation on domain quota. It is mentioned in the docs as quota2 = dict:domain:%d:proxy::quota_domain but not much more. Searching the mailing list or Google doesn't reveal much more info. Searching the German mailing list at jpberlin finds mostly information where they say that it doesn't work. So, does it or does it not? And if it works, does it work like I'm used to? e.g. with system users and aquota the filesystem quota kicks in whenever either the user or the domain quota is reached. That makes sense. It prohibits that single users can "fill" the whole domain (group) quota. I found one particular example where they say the dict quota works different, e.g. it's an either/or. They give this example for the user_query (the relevant part): CONCAT('*:bytes=', \ IF(mailbox.quota = 0, domain.maxquota*1024000, mailbox.quota)) \ as quota_rule So, this works as a default domain quota that is overridden on a user by user basis. But this means that a user quota that is set cannot honor the domain quota. So, in theory all users together can reach a maximum quota that is way over the domain quota. Not desirable. Concluding from the example in 90-quota.conf I want to set up this user_query (relevant part): concat('*:bytes=', m.quota) AS quota_rule, concat('*:bytes=', (s.quota*1024*1024)) AS quota2_rule (with m and s being the two tables for user and domain quota and s holding the domain quota in MB) Does this work together with this quota plugin setup and would provide both a user and domain quota? quota = dict:user::proxy::quota quota2 = dict:domain:%d:proxy::quota_domain quota_rule = *:storage=100M # default quota2_rule = *:storage=1000M # default I would like to know if this is supposed to work (and how) before I start testing in vain. One thing that I can confirm from several statements on the mailing lists is that the recalc option of doveadm doesn't work correctly. A doveadm quota recalc -A sets the used domain quota to the user used quota that it handles last for the domain (and a recalc -u sets the whole domain used quota to the one of that user). Which sets the domain quota completely wrong. I tried iterating thru the various users but got the same result as with -A. So, this is clearly broken, at least from my point of view. I'm using dovecot 2.3 from this repo: deb https://repo.dovecot.org/ce-2.3-latest/ubuntu/xenial xenial main I installed it because of the native blowfish encryption support. So far, it seems to be running great. Except for this particular aspect. Thanks! Kai

2 5

Re: Dovecot User Listing Error - getpwent() failed: Invalid Argument
by Michael Marley 31 Aug '18

31 Aug '18

I think this might be caused by glibc 2.28. I saw a similar error with the "doveadm purge" command after upgrading my system to that version. Michael Marley On 2018-08-31 13:10, Aki Tuomi wrote: > I'll see if this is reproducible > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Reuben Farrelly <reuben-dovecot(a)reub.net> > Date: 31/08/2018 17:41 (GMT+02:00) > To: Aki Tuomi <aki.tuomi(a)dovecot.fi>, Dovecot Mailing List <dovecot(a)dovecot.org> > Subject: Re: Dovecot User Listing Error - getpwent() failed: Invalid Argument > Sure: > > https://www.reub.net/files/dovecot/lightning-dovecot.conf > https://www.reub.net/files/dovecot/thunderstorm-dovecot.conf > > Updated nightly. > > Reuben > > On 1/09/2018 12:26 am, Aki Tuomi wrote: >> Can you provide doveconf -n? >> >> >> >> --- >> Aki Tuomi >> Dovecot oy >> >> -------- Original message -------- >> From: Reuben Farrelly <reuben-dovecot(a)reub.net> >> Date: 31/08/2018 17:12 (GMT+02:00) >> To: Aki Tuomi <aki.tuomi(a)dovecot.fi>, Dovecot Mailing List >> <dovecot(a)dovecot.org> >> Subject: Re: Dovecot User Listing Error - getpwent() failed: Invalid >> Argument >> >> No. Neither of those are installed on either system. >> >> Reuben >> >> On 1/09/2018 12:09 am, Aki Tuomi wrote: >>> Could apparmor or selinux be causing this! >>> >>> >>> >>> --- >>> Aki Tuomi >>> Dovecot oy >>> >>> -------- Original message -------- >>> From: Reuben Farrelly <reuben-dovecot(a)reub.net> >>> Date: 31/08/2018 16:50 (GMT+02:00) >>> To: Dovecot Mailing List <dovecot(a)dovecot.org> >>> Subject: Dovecot User Listing Error - getpwent() failed: Invalid Argument >>> >>> Hi, >>> >>> I'm running dovecot-2.3 git and seeing on 2 of my Dovecot installations, >>> the following message logged quite frequently: >>> >>> Aug 31 16:55:53 lightning.reub.net dovecot[7698]: auth-worker(7707): >>> Error: getpwent() failed: Invalid argument >>> Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: >>> User listing returned failure >>> Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: >>> listing users failed, can't replicate existing data >>> >>> It appears that this could be having an effect on replication, but it's >>> unclear because the secondary/replica doesn't see much traffic. >>> >>> Authentication is via PAM and the system is Gentoo Linux x86_64. passb >>> driver = pam, userdb driver = static . >>> >>> doveadm user -u '*' shows the system users listed twice like this (is >>> this a problem?) and then prints: >>> >>> ---- >>> reuben >>> liam >>> reuben >>> liam >>> <some other users> >>> Error: User listing returned failure >>> Fatal: user listing failed >>> ---- >>> >>> To me that doesn't look right. >>> >>> Can anyone suggest what could be causing this? I don't recall seeing >>> this message some time ago so I suspect it's a recent change in either >>> dovecot, or pam or glibc etc.. >>> >>> Thanks, >>> Reuben

2 1

Re: Dovecot User Listing Error - getpwent() failed: Invalid Argument
by Reuben Farrelly 31 Aug '18

31 Aug '18

No. Neither of those are installed on either system. Reuben On 1/09/2018 12:09 am, Aki Tuomi wrote: > Could apparmor or selinux be causing this! > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Reuben Farrelly <reuben-dovecot(a)reub.net> > Date: 31/08/2018 16:50 (GMT+02:00) > To: Dovecot Mailing List <dovecot(a)dovecot.org> > Subject: Dovecot User Listing Error - getpwent() failed: Invalid Argument > > Hi, > > I'm running dovecot-2.3 git and seeing on 2 of my Dovecot installations, > the following message logged quite frequently: > > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: auth-worker(7707): > Error: getpwent() failed: Invalid argument > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: > User listing returned failure > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: > listing users failed, can't replicate existing data > > It appears that this could be having an effect on replication, but it's > unclear because the secondary/replica doesn't see much traffic. > > Authentication is via PAM and the system is Gentoo Linux x86_64. passb > driver = pam, userdb driver = static . > > doveadm user -u '*' shows the system users listed twice like this (is > this a problem?) and then prints: > > ---- > reuben > liam > reuben > liam > <some other users> > Error: User listing returned failure > Fatal: user listing failed > ---- > > To me that doesn't look right. > > Can anyone suggest what could be causing this? I don't recall seeing > this message some time ago so I suspect it's a recent change in either > dovecot, or pam or glibc etc.. > > Thanks, > Reuben

2 1

Dovecot User Listing Error - getpwent() failed: Invalid Argument
by Reuben Farrelly 31 Aug '18

31 Aug '18

Hi, I'm running dovecot-2.3 git and seeing on 2 of my Dovecot installations, the following message logged quite frequently: Aug 31 16:55:53 lightning.reub.net dovecot[7698]: auth-worker(7707): Error: getpwent() failed: Invalid argument Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: User listing returned failure Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: listing users failed, can't replicate existing data It appears that this could be having an effect on replication, but it's unclear because the secondary/replica doesn't see much traffic. Authentication is via PAM and the system is Gentoo Linux x86_64. passb driver = pam, userdb driver = static . doveadm user -u '*' shows the system users listed twice like this (is this a problem?) and then prints: ---- reuben liam reuben liam <some other users> Error: User listing returned failure Fatal: user listing failed ---- To me that doesn't look right. Can anyone suggest what could be causing this? I don't recall seeing this message some time ago so I suspect it's a recent change in either dovecot, or pam or glibc etc.. Thanks, Reuben

2 1

Shared mailbox name
by Kacper 31 Aug '18

31 Aug '18

Hello, Is it possible to have a custom name on a shared inbox? Currently prefix = shared/%%u/ makes the shared inbox have the name of the user whos sharing the inbox. I would like to set a custom name on the shared inbox such as "My Recipes" (spaces included). Background: I've implemented shared inboxes by using groups in Active Directory. Each group is a user with a mailbox and has it's mailbox shared using a namespace. A post login script fetches all groups a user is member of and includes them into the ACL_GROUPS environment variable. namespace { type = shared separator = / prefix = Shared/%%u/ location = maildir:/var/mail/maildir/%%u:INDEXPVT=/var/mail/maildir/shared/%u/%%u subscriptions = no list = children } I've tried using groups which contain spaces and mixing lower/upper case letters. "doveadm acl set -u "My Recipes" inbox group="My Recipes" create expunge insert lookup read write write-deleted write-seen" works and creates "/var/mail/maildir/My Recepies", but for some reason when refreshing the imap subscriptions in Thunderbird dovecot creates "/var/mail/maildir/my recepies" and "/var/mail/maildir/myrecepies". Thanks, Kacper

1 0

SNI Dovecot
by Nicolas 31 Aug '18

31 Aug '18

Hi all, I'm testing the SNI configuration from dovecot's wiki page, to have multiple domains. I'm using letsencrypt certificates. On the 10-ssl.conf, when I only use one domain, like this, it works : ssl_ca = </etc/letsencrypt/live/mail.mydomain.fr/chain.pem ssl_cert = </etc/letsencrypt/live/mail.mydomain.fr/cert.pem ssl_key = </etc/letsencrypt/live/mail.mydomain.fr/privkey.pem I got a warning of course when using my second domain, mydomain2.fr. If I do the config : local_name mail.mydomain.fr { ssl_ca = </etc/letsencrypt/live/mail.mydomain.fr/chain.pem ssl_cert = </etc/letsencrypt/live/mail.mydomain.fr/cert.pem ssl_key = </etc/letsencrypt/live/mail.mydomain.fr/privkey.pem } local_name mail.mydomain2.fr { ssl_ca = </etc/letsencrypt/live/mail.mydomain2.fr/chain.pem ssl_cert = </etc/letsencrypt/live/mail.mydomain2.fr/cert.pem ssl_key = </etc/letsencrypt/live/mail.mydomain2.fr/privkey.pem } I got this on dovecot's start : dovecot[930]: master: Error: service(imap-login): command startup failed, throttling for 8 secs dovecot[932]: imap-login: Fatal: Couldn't parse private ssl_key: error:0906D06C:PEM routines:PEM_read_bio:no start line: Expecting: ANY PRIVATE KEY It's working without local_name, so why it can be a certificate issue? Any idea? I'm using dovecot 2.2.27-3+deb9u2 from debian. Thanks, Nicolas

3 3

Userdb by directory lookup
by Miloslav Hůla 30 Aug '18

30 Aug '18

Hi, I have the Dovecot (2.2.27-3+deb9u2) with LMTP and Postfix. Static userdb: userdb { driver = static args = uid=vmail gid=vmail home=/data/vmail/user/%n allow_all_users=yes } and passdb by LDAP, only to verify IMAP user password by bind. Problem is, when someone sends email to non-existent(a)mydomain.tld, Dovecot automatically creates its home directory and Maildir. Is there any way how deliver only when /data/vmail/user/%n directory already exists, and reject otherwise? When I remove allow_all_users=yes, LMTP stops to deliver at all with 550 code. I probably understand why, but I cannot figure how to solve it. I cannot use LDAP to user lookup. Milo

2 3

Re: where are stored attachments
by Aki Tuomi 29 Aug '18

29 Aug '18

Based on this you are not using SIS, so I wonder if those are some old emails when you were using SIS? Can you provide an example corpus of such mail? You can use e.g. https://www.dovecot.org/tools/dbox-anonymize.pl https://www.dovecot.org/tools/maildir-obfuscate.pl https://www.dovecot.org/tools/mbox-anonymize.pl https://www.dovecot.org/tools/mdbox-obfuscate.pl to obfuscate the mail. Aki On 29.08.2018 08:39, Poliman - Serwis wrote: > Sure, here it is: > root@s1:~# doveconf -n > # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.13 (7b14904) > # OS: Linux 4.4.0-134-generic x86_64 Ubuntu 16.04.5 LTS > auth_mechanisms = plain login > disable_plaintext_auth = no > listen = *,[::] > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_max_userip_connections = 100 > mail_plugins = " quota" > mail_privileged_group = vmail > passdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > plugin { > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > sieve_max_redirects = 25 > } > postmaster_address = postmaster(a)s1.poliman.net > <mailto:postmaster@s1.poliman.net> > protocols = imap pop3 > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = root > } > service imap-login { > client_limit = 1000 > process_limit = 512 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > } > ssl_cert = </etc/postfix/smtpd.cert > ssl_key = </etc/postfix/smtpd.key > ssl_protocols = !SSLv3 > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > protocol imap { > mail_plugins = quota imap_quota > } > protocol pop3 { > mail_plugins = quota > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > mail_plugins = sieve quota > postmaster_address = postmaster(a)s1.poliman.net > <mailto:postmaster@s1.poliman.net> > } > protocol lmtp { > mail_plugins = quota sieve > postmaster_address = postmaster(a)s1.poliman.net > <mailto:postmaster@s1.poliman.net> > } > > > 2018-08-29 7:37 GMT+02:00 Aki Tuomi <aki.tuomi(a)dovecot.fi > <mailto:aki.tuomi@dovecot.fi>>: > > Can you provide output of doveconf -n? > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Poliman - Serwis <serwis(a)poliman.pl <mailto:serwis@poliman.pl>> > Date: 29/08/2018 08:20 (GMT+02:00) > To: Aki Tuomi <aki.tuomi(a)dovecot.fi <mailto:aki.tuomi@dovecot.fi>> > Cc: dovecot <dovecot(a)dovecot.org <mailto:dovecot@dovecot.org>> > Subject: Re: where are stored attachments > > Sorry, I have checked before I wrote first post. Based on docs - > the path should be in /etc/dovecot/dovecot.conf or > /etc/dovecot/conf.d/10-mail.conf (I have Ubuntu 16.04 LTS). I have > some familiar lines in 10-mail.conf: > #mail_attachment_dir = > #mail_attachment_min_size = 128k > #mail_attachment_fs = sis posix > #mail_attachment_hash = %{sha1} > > but all these lines are commented out. So - in theory - dovecot > does not store attachments anywhere but when I log into i.e. > Roundcube, I have there mails with attachments. In Maildir/cur I > see formatted files related with specific emails. These with > attachments have specific hash at the bottom of content. As we > could suppose they are hashed by sha1. But still no idea where are > real files, which I can download after login to mailbox. > > > 2018-08-28 18:38 GMT+02:00 Aki Tuomi <aki.tuomi(a)dovecot.fi > <mailto:aki.tuomi@dovecot.fi>>: > > Check your configuration > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Poliman - Serwis <serwis(a)poliman.pl > <mailto:serwis@poliman.pl>> > Date: 28/08/2018 16:40 (GMT+02:00) > To: dovecot <dovecot(a)dovecot.org <mailto:dovecot@dovecot.org>> > Subject: where are stored attachments > > Hi! I would like to know where are stored attachments for > mails stored in /var/vmail/domain.com/user/Maildir > <http://domain.com/user/Maildir> ? Specific email has only > hash included at the bottom of message. > > -- > /Pozdrawiam / Best Regards > / > /Piotr Bracha/ > > > > > -- > /Pozdrawiam / Best Regards > / > /Piotr Bracha/ > > > > > -- > /Pozdrawiam / Best Regards > / > /Piotr Bracha/

2 2

Re: where are stored attachments
by Poliman - Serwis 29 Aug '18

29 Aug '18

Sorry, I have checked before I wrote first post. Based on docs - the path should be in /etc/dovecot/dovecot.conf or /etc/dovecot/conf.d/10-mail.conf (I have Ubuntu 16.04 LTS). I have some familiar lines in 10-mail.conf: #mail_attachment_dir = #mail_attachment_min_size = 128k #mail_attachment_fs = sis posix #mail_attachment_hash = %{sha1} but all these lines are commented out. So - in theory - dovecot does not store attachments anywhere but when I log into i.e. Roundcube, I have there mails with attachments. In Maildir/cur I see formatted files related with specific emails. These with attachments have specific hash at the bottom of content. As we could suppose they are hashed by sha1. But still no idea where are real files, which I can download after login to mailbox. 2018-08-28 18:38 GMT+02:00 Aki Tuomi <aki.tuomi(a)dovecot.fi>: > Check your configuration > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Poliman - Serwis <serwis(a)poliman.pl> > Date: 28/08/2018 16:40 (GMT+02:00) > To: dovecot <dovecot(a)dovecot.org> > Subject: where are stored attachments > > Hi! I would like to know where are stored attachments for mails stored in > /var/vmail/domain.com/user/Maildir ? Specific email has only hash > included at the bottom of message. > > -- > > *Pozdrawiam / Best Regards* > *Piotr Bracha* > -- *Pozdrawiam / Best Regards* *Piotr Bracha*

2 1

Perl Net::Doveadm
by Felipe Gasper 28 Aug '18

28 Aug '18

Hi all, In response to a discussion last week about API access--and notwithstanding Aki’s encouragement that integrators prefer the HTTP-based API--I’ve released a Perl module that facilitates access to Dovecot via the Doveadm protocol: https://metacpan.org/pod/Net::Doveadm I hope it’s useful. -Felipe Gasper Mississauga, ON

1 0