January 2021 - dovecot

Unknown euid and egid, wrong mail directory
by Michael Agbaglo 01 Feb '21

01 Feb '21

Hi! Trying to set up dovecot for local accounts and virtual users, sending test mail fails: dovecot sees a permission problem in /home but the mail should go to /srv/mail/example.net/alice. "euid=1001(<unknown>) egid=1001(<unknown>)" 1001 doesn't not exist in /etc/passwd - Where does it come from? M. root@broccoli:~# doveconf -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-65-generic x86_64 Ubuntu 20.04.2 LTS # Hostname: broccoli.example.net mail_home = /srv/mail/%d/%n mail_location = maildir:~/Maildir mail_privileged_group = mail namespace { list = children location = maildir:%%h/Maildir:INDEXPVT=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap lmtp" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/ssl/certs/ssl-cert-snakeoil.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } userdb { args = uid=vmail gid=vmail home=/srv/mail/%d/%n driver = static } root@broccoli:~# id postfix uid=116(postfix) gid=124(postfix) groups=124(postfix) root@broccoli:~# id dovecot uid=114(dovecot) gid=122(dovecot) groups=122(dovecot) root@broccoli:~# sendmail alice(a)example.net < /etc/issue.net root@broccoli:~# tail /var/log/mail.log Jan 30 21:31:47 broccoli postfix/pickup[4067]: 22AE117AA82: uid=0 from=<root> Jan 30 21:31:47 broccoli postfix/cleanup[35091]: 22AE117AA82: message-id=<20210130203147.22AE117AA82(a)broccoli.example.net> Jan 30 21:31:47 broccoli postfix/qmgr[854]: 22AE117AA82: from=<root(a)broccoli.example.net>, size=335, nrcpt=1 (queue active) Jan 30 21:31:47 broccoli dovecot: lmtp(35094): Connect from local Jan 30 21:31:47 broccoli dovecot: lmtp(alice(a)example.net)<35094><xGMOCjPCFWAWiQAAfCPMuA>: Error: mkdir(/home/alice/Maildir) failed: Permission denied (euid=1001(<unknown>) egid=1001(<unknown>) missing +w perm: /home, dir owned by 0:0 mode=0755) Jan 30 21:31:47 broccoli dovecot: lmtp(alice(a)example.net)<35094><xGMOCjPCFWAWiQAAfCPMuA>: Error: mkdir(/home/alice/Maildir) failed: Permission denied (euid=1001(<unknown>) egid=1001(<unknown>) missing +w perm: /home, dir owned by 0:0 mode=0755) Jan 30 21:31:47 broccoli dovecot: lmtp(alice(a)example.net)<35094><xGMOCjPCFWAWiQAAfCPMuA>: Error: lmtp-server: conn unix:pid=35093,uid=116 [1]: rcpt alice(a)example.net: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2021-01-30 21:31:47] Jan 30 21:31:47 broccoli dovecot: lmtp(alice(a)example.net)<35094><xGMOCjPCFWAWiQAAfCPMuA>: msgid=<20210130203147.22AE117AA82(a)broccoli.example.net>: save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2021-01-30 21:31:47] Jan 30 21:31:47 broccoli dovecot: lmtp(35094): Disconnect from local: Client has quit the connection (state=READY) Jan 30 21:31:47 broccoli postfix/lmtp[35093]: 22AE117AA82: to=<alice(a)example.net>, relay=broccoli.example.net[private/dovecot-lmtp], delay=0.07, delays=0.04/0.01/0/0.02, dsn=4.2.0, status=deferred (host broccoli.example.net[private/dovecot-lmtp] said: 451 4.2.0 <alice(a)example.net> Internal error occurred. Refer to server log for more information. [2021-01-30 21:31:47] (in reply to end of DATA command))

3 3

Dovecot Gmail OAuth2.0 Setting Question
by 福田泰葵 01 Feb '21

01 Feb '21

Dear Sir or Madam Unable to build OAuth2.0 authentication to Gmail using dovecot as proxy. I have a question about how to use dovecot as a proxy to perform OAuth 2.0 authentication to Gmail using a mail client. 1. Is the following all I need to do to authenticate to Gmail using dovecot as a proxy? - passdb passdb { driver = oauth2 mechanisms = oauthbearer xoauth2 args = /etc/dovecot/dovecot-oauth2.token.conf.ext } passdb { driver = oauth2 mechanisms = plain login args = /etc/dovecot/dovecot-oauth2.plain.conf.ext } - create dovecot-oauth2.token.conf.ext and dovecot-oauth2.plain.conf.ext - create gmail service account api 2. grant_url in dovecot-oauth2.token.conf.ext and dovecot-oauth2.plain.conf.ext is URL for obtaining a Google access token for a web server that I have built myself? 3. I use a Gmail service account, so I don’t need a client ID and secret ID, right? 4. Do I set introspection_url to the URL of my own web server with the access token used for authentication to Google as the response? 5. The documentation says “pass_attrs = host=127.0.0.1”, but if you are authenticating to Gmail, I should use “pass_attrs = proxy=y host=%{if;%s;eq;imap;imap.gmail.com;%{if;%s;eq;pop3;smtp .gmail.com;pop.gmail.com}} port=%{if;%s;eq;imap;993;%{if;%s;eq;pop3;587;465}} proxy_mech=xoauth2 pass=%{oauth2:access_token} user=%{oauth2:email oauth2:email}”? 6. What is the difference between dovecot-oauth2.token.conf.ext and dovecot-oauth2.plain.conf.ext ? Do I need to configure both? I used https://doc.dovecot.org/configuration_manual/authentication/oauth2/#proxy as a reference. I would appreciate your reply. Yours faithfully, ------------------------------ e-mail: taiki.fukuda(a)justsystems.com TEL: 03-5324-7900 mobile: 080-6198-7328 ------------------------------

3 21

Undefined SSL object
by Frank Elsner 31 Jan '21

31 Jan '21

Hi community, when doing imapsync to my dovecot server (Host2) I get the following Host2 capability before authentication: IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN AUTH=LOGIN AUTH DEBUG: .../IO/Socket/SSL.pm:1117: global error: Undefined SSL object Host2: Socket successfuly converted to SSL Beside this message all works fine. What is it? SSL setting? How to overcome the message? Greetings, Frank

2 1

Processing incoming mail efficiently
by Ron Garret 30 Jan '21

30 Jan '21

I’ve asked a related question on this list before but I now have a much better handle on what I’m doing and I realize that I still don’t know the answer, so I’m going to ask this again in a slightly different form. I’m writing a spam filter, so obviously I need to feed incoming mail to it somehow. The “obvious” way to do this is with a sieve script using the pipe extension. There are two problems with this: 1. This will always pipe the entire file no matter how big it is. The filter will often not need to process the body of the message, only the headers, or only the first part of a multipart MIME message. Is there any way to allow my filter to open the file in which the message is stored rather than piping it a copy of the message? 2. Once the filter has processed the message and decided if it’s spam it still needs to move the message to the appropriate folder (INBOX or Junk). To do this it needs to somehow correlate the *content* of the message that was piped to it with the UID of the message that needs to be moved. One way to do this is to pull out the message-id header and then use doveadm to find the file containing the message with that message-id, but there are two problems with this. First, not all messages have message-ids. I can work around this by adding my own message-id to messages that don’t already have them, but this just feel wrong. And second, unless dovecot keeps an index of message-ids (does it?) then this will be horribly inefficient because it will have to essentially grep for the message id every time I want to move a message. So it seems like there has to be a better way, but I can’t think of what that would be. I figure this has to be a solved problem because I am obviously not the first person to write a spam filter for dovecot. What is the Right Way to do this? Thanks, rg

3 6

What is the right way to move messages between mailboxes?
by Ron Garret 30 Jan '21

30 Jan '21

I’m working on a spam filter. Incoming messages are sent to an “incoming” folder (via a sieve script) where they are processed by the filter and moved either to INBOX or Junk. My question is: what is the right way to effect this move? Can I just mv the file? With Dovecot automatically notice this and update any connected IMAP clients? Or is there some other procedure and/or utility (doveadm?) that I should be using for this? Thanks, rg

1 0

dovecot and broken uidlist
by Maciej Milaszewski 29 Jan '21

29 Jan '21

Hi I have been trying resolve my problem with dovecot for a few days and I dont have idea.... My environment is: dovecot director+5 dovecot guest dovecot-2.2.36.4 from source Linux 3.16.0-11-amd64 storage via nfs (NetApp) all works fine but when I update OS from debian 8 (kernel 3.16.x) to debian 9 (kernel 4.9.x ) sometimes I get random in logs: Broken dovecot-uidlist examle: Error: Broken file /vmail2/po/pollygraf.xxx_pg_pollygraf/Maildir/dovecot-uidlist line 88: Invalid data: (for random users - sometimes 10 error in day per node, some times more) File looks ok But if I change kernel to 3.16.x problem with "Broken file dovecot-uidlist" - not exists if turn to 4.9 or 5.x - problem exists I have storage via nfs with opions: rw,sec=sys,noexec,noatime,tcp,hard,rsize=65536,wsize=65536,intr,nordirplus,nfsvers=3,tcp,actimeo=120 I tested with "nocto" or without "nocto" - nothing changes ...... nfs options in node: mmap_disable = yes mail_fsync = always I bet the configuration is correct and I wonder why the problem occurs with other kernels 3.x.x - ok 4.x - not ok I check and user who have problem did not connect to another node in this time I dont have idea why problem exists on the kernel 4.x but not in 3.x

6 14

Long Term Archival
by Rupert Gallagher 29 Jan '21

29 Jan '21

Hello, I would like to know what is your strategy for legally compliant long term archival of e-mails (up to 10 years). Thank you

3 2

10G mails use 90G on disk
by Markus Schönhaber 29 Jan '21

29 Jan '21

Hello, on a system I administer the disk where Dovecot's mailstore (mdbox) resides got filled up to the brim although the combined quotas for the users are much smaller than the disk's capacity. For one user I see the following: doveadm quota get -u <USER>@<DOMAIN> Username Quota name Type Value Limit % <USER>@<DOMAIN> User quota STORAGE 10231248 20971520 48 <USER>@<DOMAIN> User quota MESSAGE 11176 - 0 So this user has used ~10G of his 20G quota. But looking at the space used on disk: # du -hs /var/spool/vmail/mail/<DOMAIN>/<USER>/storage/ 90G /var/spool/vmail/mail/<DOMAIN>/<USER>/storage/ What might be the reason that mail size and disk space used differ so much? And most of all: how can I prevent that from happening? Regards mks # doveconf -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 4.19.0-13-amd64 x86_64 Debian 10.7 ext4 # Hostname: … auth_mechanisms = plain login dict { quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } imap_id_log = * login_log_format_elements = user=<%u> method=%m rip=%r lip=%l lport=%a mpid=%e %c mail_home = /var/spool/vmail/home/%d/%n mail_location = mdbox:/var/spool/vmail/mail/%d/%n mail_plugins = " quota acl" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Entwürfe { special_use = \Drafts } mailbox Entw&APw-rfe { special_use = \Drafts } mailbox "Gelöschte Objekte" { special_use = \Trash } mailbox "Gel&APY-schte Objekte" { special_use = \Trash } mailbox "Gesendete Objekte" { special_use = \Sent } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:User quota::proxy::quota recipient_delimiter = + sieve = file:~/sieve;active=~/.dovecot.sieve sieve_redirect_envelope_from = orig_recipient } protocols = " imap lmtp sieve" service auth-worker { user = $default_internal_user } service auth { inet_listener { port = 12345 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service stats { client_limit = 10000 unix_listener stats-writer { group = vmail user = vmail } } ssl_cert = </etc/letsencrypt/live/<DOMAIN>/fullchain.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = " quota acl quota sieve" } protocol lda { mail_plugins = " quota acl sieve" } protocol imap { mail_plugins = " quota acl imap_quota acl" } remote 192.168.1.0/24 { protocol imap { disable_plaintext_auth = no } }

2 4

mail event
by Marc Roos 29 Jan '21

29 Jan '21

> > Using mail_log plugin is always very recommended, > > https://doc.dovecot.org/configuration_manual/plugins/mail_event_logging/ > I am not sure if I am correct, but I think this logs every deleted message not? I was testing with this a while ago. Is there also an option that only logs something like '300 messages deleted'?

1 0

SMTP tool for Email validation
by Amol Kale 29 Jan '21

29 Jan '21

Hi, We are looking for a tool for bulk SMTP testing / Email validation using telnet or similar protocol ( which doesn't send the mails ) It generally involves steps like NSlookup to check MX of destination server, opening port25, telnet to communicate with client server, then finally checking the correctness of user email account. Please connect if you have such ready tool with you or you can develop Process example- <https://blog.mailtrap.io/verify-email-address-without-sending/> https://blog.mailtrap.io/verify-email-address-without-sending/ Thanks & regards Amol Kale Founder Director Talent Trackers HR 7350002596

4 3