LDA ignores virtual mailbox settings

newer
TLS problem with iOS 9 Mail clients

Toni Mueller

27 Dec 2020 27 Dec '20

6:19 p.m.

Hi,

I have a Debian/Buster system with Postfix and Dovecot from the Debian repo, and with virtual users only (ie, no system users). These virtual users are having all of their uids, gids, homes and mail quota in an PostgreSQL database. The intent is to have postfix deliver the email via dovecot's LDA, so I can set quota on a per-user basis.

But for some odd reason, Dovecot's LDA can't find the user data from the userdb, and then complains about not being able to write to /var/mail. The following example shows how things fail when delivering a message from Postfix's queue (therefore, the passdb failure is expected):

In /etc/dovecot.conf, I have this, amongst other things:

mail_location = maildir:~/Maildir:INBOX=~/Maildir passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } userdb { driver = prefetch } userdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } protocol lda { mail_plugins = autocreate quota mail_log trash virtual notify }

16:04:16 postfix/qmgr[4970]: 8CD6CE072E: from=user@example.com, size=880, nrcpt=1 (queue active) 16:04:16 dovecot: auth: Debug: master in: USER#0111#011user@example.com#011service=lda 16:04:16 dovecot: auth: Debug: prefetch(user@example.com): passdb didn't return userdb entries, trying the next userdb 16:04:16 dovecot: auth: Debug: sql(user@example.com): SELECT '/path-to-mailboxen/' || virtual_users.home AS home, uid , gid , quota as quota_rule FROM virtual_users WHERE email = 'user@example.com' AND status = 'A' 16:04:16 dovecot: auth: Debug: userdb out: USER#0111#011user@example.com#011home=/path-to-mailboxen/example.com/user#011uid=12345#011gid=12345#011quota_rule=*:storage=0

^^^^^^^^^^^^^^^^^^^^^^

This shows that the database lookup works. The intended effect should be that the message is delivered to

/path-to-mailboxen/example.com/user/Maidir/new

16:04:16 dovecot: lda(user@example.com)<5291>: Error: setegid(privileged) failed: Operation not permitted 16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775)

^^^^^^^^^^^^^^^^^^^^^^

And this shows that dovecot-lda just ignores the result.

16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 dovecot: lda(user@example.com)<5291>: msgid=20201226224933.014608@laptop.example.com: save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 postfix/pipe[5284]: 8CD6CE072E: to=user@example.com, orig_to=userA@example2.com, relay=dovecot, delay=62083, delays=62083/0.04/0/0.04, dsn=4.3.0, status=deferred (temporary failure)

In /etc/postfix/master.cf, I have this to call it:

dovecot unix - n n - - pipe flags=DRhu user=_mailbox argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} -a ${recipient}

I've tried strace-ing dovecot-lda, but it didn't really help me to understand why it discards the result of the userdb lookup.

Can anyone please provide a cluebat, please?

Thanks, Toni

Show replies by date

Aki Tuomi

27 Dec 27 Dec

9:18 p.m.

...

On 27/12/2020 18:19 Toni Mueller support@oeko.net wrote:

Hi,

I have a Debian/Buster system with Postfix and Dovecot from the Debian repo, and with virtual users only (ie, no system users). These virtual users are having all of their uids, gids, homes and mail quota in an PostgreSQL database. The intent is to have postfix deliver the email via dovecot's LDA, so I can set quota on a per-user basis.

But for some odd reason, Dovecot's LDA can't find the user data from the userdb, and then complains about not being able to write to /var/mail. The following example shows how things fail when delivering a message from Postfix's queue (therefore, the passdb failure is expected):

In /etc/dovecot.conf, I have this, amongst other things:

mail_location = maildir:~/Maildir:INBOX=~/Maildir passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } userdb { driver = prefetch } userdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } protocol lda { mail_plugins = autocreate quota mail_log trash virtual notify }

16:04:16 postfix/qmgr[4970]: 8CD6CE072E: from=user@example.com, size=880, nrcpt=1 (queue active) 16:04:16 dovecot: auth: Debug: master in: USER#0111#011user@example.com#011service=lda 16:04:16 dovecot: auth: Debug: prefetch(user@example.com): passdb didn't return userdb entries, trying the next userdb 16:04:16 dovecot: auth: Debug: sql(user@example.com): SELECT '/path-to-mailboxen/' || virtual_users.home AS home, uid , gid , quota as quota_rule FROM virtual_users WHERE email = 'user@example.com' AND status = 'A' 16:04:16 dovecot: auth: Debug: userdb out: USER#0111#011user@example.com#011home=/path-to-mailboxen/example.com/user#011uid=12345#011gid=12345#011quota_rule=*:storage=0

^^^^^^^^^^^^^^^^^^^^^^

This shows that the database lookup works. The intended effect should be that the message is delivered to

/path-to-mailboxen/example.com/user/Maidir/new

16:04:16 dovecot: lda(user@example.com)<5291>: Error: setegid(privileged) failed: Operation not permitted 16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775)

^^^^^^^^^^^^^^^^^^^^^^

And this shows that dovecot-lda just ignores the result.

16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 dovecot: lda(user@example.com)<5291>: msgid=20201226224933.014608@laptop.example.com: save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 postfix/pipe[5284]: 8CD6CE072E: to=user@example.com, orig_to=userA@example2.com, relay=dovecot, delay=62083, delays=62083/0.04/0/0.04, dsn=4.3.0, status=deferred (temporary failure)

In /etc/postfix/master.cf, I have this to call it:

dovecot unix - n n - - pipe flags=DRhu user=_mailbox argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} -a ${recipient}

I've tried strace-ing dovecot-lda, but it didn't really help me to understand why it discards the result of the userdb lookup.

Can anyone please provide a cluebat, please?

Thanks, Toni

Try adding

mail_privileged_group = mail

to your dovecot.conf. See https://doc.dovecot.org/settings/core/#mail-privileged-group

Aki

Toni Mueller

9:54 p.m.

Hi Aki,

On Sun, Dec 27, 2020 at 09:18:25PM +0200, Aki Tuomi wrote:

...

...
On 27/12/2020 18:19 Toni Mueller support@oeko.net wrote:

16:04:16 postfix/qmgr[4970]: 8CD6CE072E: from=user@example.com, size=880, nrcpt=1 (queue active) 16:04:16 dovecot: auth: Debug: master in: USER#0111#011user@example.com#011service=lda 16:04:16 dovecot: auth: Debug: prefetch(user@example.com): passdb didn't return userdb entries, trying the next userdb 16:04:16 dovecot: auth: Debug: sql(user@example.com): SELECT '/path-to-mailboxen/' || virtual_users.home AS home, uid , gid , quota as quota_rule FROM virtual_users WHERE email = 'user@example.com' AND status = 'A' 16:04:16 dovecot: auth: Debug: userdb out: USER#0111#011user@example.com#011home=/path-to-mailboxen/example.com/user#011uid=12345#011gid=12345#011quota_rule=*:storage=0

^^^^^^^^^^^^^^^^^^^^^^

This shows that the database lookup works. The intended effect should be that the message is delivered to

/path-to-mailboxen/example.com/user/Maidir/new

16:04:16 dovecot: lda(user@example.com)<5291>: Error: setegid(privileged) failed: Operation not permitted 16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775)

^^^^^^^^^^^^^^^^^^^^^^

And this shows that dovecot-lda just ignores the result.

16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 dovecot: lda(user@example.com)<5291>: msgid=20201226224933.014608@laptop.example.com: save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 postfix/pipe[5284]: 8CD6CE072E: to=user@example.com, orig_to=userA@example2.com, relay=dovecot, delay=62083, delays=62083/0.04/0/0.04, dsn=4.3.0, status=deferred (temporary failure)

In /etc/postfix/master.cf, I have this to call it:

dovecot unix - n n - - pipe flags=DRhu user=_mailbox argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} -a ${recipient}

Try adding

mail_privileged_group = mail

I am not sure why you recommend this. I never ever want to deliver to /var/mail, and my mailbox directory has group 'mailbox' - hence I have

mail_privileged_group = mailbox

in my configuration.

My question is why this thing wants to deliver to /var/mail, despite having a different location from the userdb, and how I can force it to use the location from the userdb.

Thanks, Toni

Aki Tuomi

10 p.m.

...

On 27/12/2020 21:54 Toni Mueller support@oeko.net wrote:

Hi Aki,

On Sun, Dec 27, 2020 at 09:18:25PM +0200, Aki Tuomi wrote:

...
...
On 27/12/2020 18:19 Toni Mueller support@oeko.net wrote:

16:04:16 postfix/qmgr[4970]: 8CD6CE072E: from=user@example.com, size=880, nrcpt=1 (queue active) 16:04:16 dovecot: auth: Debug: master in: USER#0111#011user@example.com#011service=lda 16:04:16 dovecot: auth: Debug: prefetch(user@example.com): passdb didn't return userdb entries, trying the next userdb 16:04:16 dovecot: auth: Debug: sql(user@example.com): SELECT '/path-to-mailboxen/' || virtual_users.home AS home, uid , gid , quota as quota_rule FROM virtual_users WHERE email = 'user@example.com' AND status = 'A' 16:04:16 dovecot: auth: Debug: userdb out: USER#0111#011user@example.com#011home=/path-to-mailboxen/example.com/user#011uid=12345#011gid=12345#011quota_rule=*:storage=0

^^^^^^^^^^^^^^^^^^^^^^

This shows that the database lookup works. The intended effect should be that the message is delivered to

/path-to-mailboxen/example.com/user/Maidir/new

16:04:16 dovecot: lda(user@example.com)<5291>: Error: setegid(privileged) failed: Operation not permitted 16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775)

^^^^^^^^^^^^^^^^^^^^^^

And this shows that dovecot-lda just ignores the result.

16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 dovecot: lda(user@example.com)<5291>: msgid=20201226224933.014608@laptop.example.com: save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 postfix/pipe[5284]: 8CD6CE072E: to=user@example.com, orig_to=userA@example2.com, relay=dovecot, delay=62083, delays=62083/0.04/0/0.04, dsn=4.3.0, status=deferred (temporary failure)

In /etc/postfix/master.cf, I have this to call it:

dovecot unix - n n - - pipe flags=DRhu user=_mailbox argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} -a ${recipient}

Try adding

mail_privileged_group = mail

I am not sure why you recommend this. I never ever want to deliver to /var/mail, and my mailbox directory has group 'mailbox' - hence I have

mail_privileged_group = mailbox

in my configuration.

My question is why this thing wants to deliver to /var/mail, despite having a different location from the userdb, and how I can force it to use the location from the userdb.

Thanks, Toni

Sorry, I misread your problem..

can you try checking output of

doveadm user <username>

and

doveadm auth lookup <username>

Also, including doveconf -n would be useful.

Aki

Toni Mueller

10:18 p.m.

Hi Aki,

thanks for your quick response!

On Sun, Dec 27, 2020 at 10:00:40PM +0200, Aki Tuomi wrote:

...

...
On Sun, Dec 27, 2020 at 09:18:25PM +0200, Aki Tuomi wrote:

...
...
16:04:16 dovecot: auth: Debug: master in: USER#0111#011user@example.com#011service=lda 16:04:16 dovecot: auth: Debug: prefetch(user@example.com): passdb didn't return userdb entries, trying the next userdb 16:04:16 dovecot: auth: Debug: sql(user@example.com): SELECT '/path-to-mailboxen/' || virtual_users.home AS home, uid , gid , quota as quota_rule FROM virtual_users WHERE email = 'user@example.com' AND status = 'A' 16:04:16 dovecot: auth: Debug: userdb out: USER#0111#011user@example.com#011home=/path-to-mailboxen/example.com/user#011uid=12345#011gid=12345#011quota_rule=*:storage=0

^^^^^^^^^^^^^^^^^^^^^^

This shows that the database lookup works. The intended effect should be that the message is delivered to

/path-to-mailboxen/example.com/user/Maidir/new

16:04:16 dovecot: lda(user@example.com)<5291>: Error: setegid(privileged) failed: Operation not permitted 16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775)

^^^^^^^^^^^^^^^^^^^^^^

And this shows that dovecot-lda just ignores the result.

16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 dovecot: lda(user@example.com)<5291>: msgid=20201226224933.014608@laptop.example.com: save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 postfix/pipe[5284]: 8CD6CE072E: to=user@example.com, orig_to=userA@example2.com, relay=dovecot, delay=62083, delays=62083/0.04/0/0.04, dsn=4.3.0, status=deferred (temporary failure)

In /etc/postfix/master.cf, I have this to call it:

dovecot unix - n n - - pipe flags=DRhu user=_mailbox argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} -a ${recipient}

Try adding

mail_privileged_group = mail

I am not sure why you recommend this. I never ever want to deliver to /var/mail, and my mailbox directory has group 'mailbox' - hence I have

mail_privileged_group = mailbox

in my configuration.

My question is why this thing wants to deliver to /var/mail, despite having a different location from the userdb, and how I can force it to use the location from the userdb.

Thanks, Toni

Sorry, I misread your problem..

can you try checking output of

doveadm user <username>

# doveadm user user@example.com field value uid 12345 gid 12345 home /path-to-mailboxen/example.com/user mail maildir:~/Maildir:INBOX=~/Maildir quota_rule *:storage=0

...

and

doveadm auth lookup <username>

# doveadm -v auth lookup user@example.com passdb: user@example.com user : user@example.com

...

Also, including doveconf -n would be useful.

# 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-13-amd64 x86_64 Debian 10.7 # Hostname: testvm.example.com auth_debug = yes auth_debug_passwords = yes auth_failure_delay = 0 auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = plain deliver_log_format = msgid=%m: %e -> %{to_envelope} %p %w %{delivery_time} %{storage_id} %{secured} disable_plaintext_auth = no login_access_sockets = tcpwrap mail_debug = yes mail_location = maildir:~/Maildir:INBOX=~/Maildir mail_privileged_group = mailbox managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = maildir:~/Maildir:INBOX=~/Maildir mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash 30 Spam 30 quota = maildir quota_rule = *:storage=50000 quota_rule2 = Trash:ignore sieve = file:~/sieve;active=~/.dovecot.sieve sieve_dir = ~/sieve trash = /etc/dovecot/dovecot-trash.conf } protocols = pop3 imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 0 } } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl = no ssl_cert =

Thank you!

-Toni

Tamsy

28 Dec 28 Dec

4:52 a.m.

*From:* Toni Mueller [mailto:support@oeko.net] *Sent:* Monday, December 28, 2020, 3:18 AM *To:* Aki Tuomi *Cc:* dovecot@dovecot.org *Subject:* LDA ignores virtual mailbox settings

...

Hi Aki,

thanks for your quick response!

On Sun, Dec 27, 2020 at 10:00:40PM +0200, Aki Tuomi wrote:

...
...
On Sun, Dec 27, 2020 at 09:18:25PM +0200, Aki Tuomi wrote:

...
...
16:04:16 dovecot: auth: Debug: master in:USER#0111#011user@example.com#011service=lda 16:04:16 dovecot: auth: Debug: prefetch(user@example.com): passdb didn't return userdb entries, trying the next userdb 16:04:16 dovecot: auth: Debug: sql(user@example.com): SELECT '/path-to-mailboxen/' || virtual_users.home AS home, uid , gid , quota as quota_rule FROM virtual_users WHERE email = 'user@example.com' AND status = 'A' 16:04:16 dovecot: auth: Debug: userdb out:USER#0111#011user@example.com#011home=/path-to-mailboxen/example.com/user#011uid=12345#011gid=12345#011quota_rule=*:storage=0

^^^^^^^^^^^^^^^^^^^^^^

This shows that the database lookup works. The intended effect should be that the message is delivered to

/path-to-mailboxen/example.com/user/Maidir/new

16:04:16 dovecot: lda(user@example.com)<5291>: Error: setegid(privileged) failed: Operation not permitted 16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775)

^^^^^^^^^^^^^^^^^^^^^^

And this shows that dovecot-lda just ignores the result.

16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 dovecot: lda(user@example.com)<5291>: msgid=20201226224933.014608@laptop.example.com: save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 postfix/pipe[5284]: 8CD6CE072E: to=user@example.com, orig_to=userA@example2.com, relay=dovecot, delay=62083, delays=62083/0.04/0/0.04, dsn=4.3.0, status=deferred (temporary failure)

In /etc/postfix/master.cf, I have this to call it:

dovecot unix - n n - - pipe flags=DRhu user=_mailbox argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} -a ${recipient}

Try adding

mail_privileged_group = mail I am not sure why you recommend this. I never ever want to deliver to /var/mail, and my mailbox directory has group 'mailbox' - hence I have

mail_privileged_group = mailbox

in my configuration.

My question is why this thing wants to deliver to /var/mail, despite having a different location from the userdb, and how I can force it to use the location from the userdb.

Thanks, Toni Sorry, I misread your problem..

can you try checking output of

doveadm user <username> # doveadm useruser@example.com field value uid 12345 gid 12345 home /path-to-mailboxen/example.com/user mail maildir:~/Maildir:INBOX=~/Maildir quota_rule *:storage=0

...
and

doveadm auth lookup <username> # doveadm -v auth lookupuser@example.com passdb:user@example.com user :user@example.com

...
Also, including doveconf -n would be useful. # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-13-amd64 x86_64 Debian 10.7 # Hostname: testvm.example.com auth_debug = yes auth_debug_passwords = yes auth_failure_delay = 0 auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = plain deliver_log_format = msgid=%m: %e -> %{to_envelope} %p %w %{delivery_time} %{storage_id} %{secured} disable_plaintext_auth = no login_access_sockets = tcpwrap mail_debug = yes mail_location = maildir:~/Maildir:INBOX=~/Maildir mail_privileged_group = mailbox managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = maildir:~/Maildir:INBOX=~/Maildir mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash 30 Spam 30 quota = maildir quota_rule = *:storage=50000 quota_rule2 = Trash:ignore sieve =file:~/sieve;active=~/.dovecot.sieve sieve_dir = ~/sieve trash = /etc/dovecot/dovecot-trash.conf } protocols = pop3 imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 0 } } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl = no ssl_cert =
Thank you!

-Toni

...

Hi Toni,

Try this:

mail_home = /path-to-mailboxen/%d/%n mail_location = maildir:~/Maildir:LAYOUT=fs

and

namespace inbox { separator = / prefix = inbox = yes subscriptions = yes list = yes type = private hidden = no }

Toni Mueller

11:47 a.m.

Hi Tamsy,

(your message was formatted a bit strangely)

On Mon, Dec 28, 2020 at 09:52:17AM +0700, Tamsy wrote:

...

Try this:

mail_home = /path-to-mailboxen/%d/%n mail_location = maildir:~/Maildir:LAYOUT=fs

and

namespace inbox { separator = / prefix = inbox = yes subscriptions = yes list = yes type = private hidden = no }

thank you for the idea.

I'll give it a try (there was another variable needed to be set to avoid clashes with "new", but the name escapes me atm), although it doesn't cover a number of special cases that I have, where the location of the mailboxen does not follow this pattern.

In the end, I want the database to be the single source of truth.

Thanks, Toni

Toni Mueller

3:06 p.m.

Hi Tamsy,

On Mon, Dec 28, 2020 at 09:47:54AM +0000, Toni Mueller wrote:

...

On Mon, Dec 28, 2020 at 09:52:17AM +0700, Tamsy wrote:

...
Try this:

mail_home = /path-to-mailboxen/%d/%n mail_location = maildir:~/Maildir:LAYOUT=fs

and

namespace inbox { separator = / prefix = inbox = yes subscriptions = yes list = yes type = private hidden = no }

thank you for the idea.

I'll give it a try (there was another variable needed to be set to avoid clashes with "new", but the name escapes me atm), although it doesn't cover a number of special cases that I have, where the location of the mailboxen does not follow this pattern.

In the end, I want the database to be the single source of truth.

well, I tried, but with no luck. LDA still wants to deliver to /var/mail, as before.

I am pretty sure that it must work somehow to fetch all data from a database, because it did before. It would be really strange if someone would have culled that functionaility. I am just not yet sure whether it's just me not getting it right, or whether there's actually a bug.

Next stop: Read Dovecot's source code (I dread that), unless someone comes up with a better idea.

Please.

Thanks, Toni

Aki Tuomi

3:37 p.m.

...

On 28/12/2020 15:06 Toni Mueller support@oeko.net wrote:

Hi Tamsy,

On Mon, Dec 28, 2020 at 09:47:54AM +0000, Toni Mueller wrote:

...
On Mon, Dec 28, 2020 at 09:52:17AM +0700, Tamsy wrote:

...
Try this:

mail_home = /path-to-mailboxen/%d/%n mail_location = maildir:~/Maildir:LAYOUT=fs

and

namespace inbox { separator = / prefix = inbox = yes subscriptions = yes list = yes type = private hidden = no }

thank you for the idea.

I'll give it a try (there was another variable needed to be set to avoid clashes with "new", but the name escapes me atm), although it doesn't cover a number of special cases that I have, where the location of the mailboxen does not follow this pattern.

In the end, I want the database to be the single source of truth.

well, I tried, but with no luck. LDA still wants to deliver to /var/mail, as before.

I am pretty sure that it must work somehow to fetch all data from a database, because it did before. It would be really strange if someone would have culled that functionaility. I am just not yet sure whether it's just me not getting it right, or whether there's actually a bug.

Next stop: Read Dovecot's source code (I dread that), unless someone comes up with a better idea.

Please.

Thanks, Toni

What you are describing will not be solved by looking at dovecot source code. Can you describe your configuration more? Maybe you are running dovecot in chroot or something? Can you consider using LMTP instead of LDA?

Aki

Toni Mueller

4:35 p.m.

Hi Aki,

On Mon, Dec 28, 2020 at 03:37:33PM +0200, Aki Tuomi wrote:

...

What you are describing will not be solved by looking at dovecot source code.

ok.

...

Can you describe your configuration more? Maybe you are running dovecot in chroot or something? Can you consider using LMTP instead of LDA?

I want to have as much information about the users and their mailboxes as possible in an SQL database (I use Pg), so I can easily move things around, change permissions and what-not.

I thought about using LMTP, but so far shyed away from it because I suspected that it would be unable to do uid/gid switching on a per-user basis. But I'll check this idea out as well.

I have not done anything special to the Debian installation, but checking if that contains an unwanted chroot somewhere, is a good idea!

Thanks, Toni

Aki Tuomi

4:56 p.m.

...

On 28/12/2020 16:35 Toni Mueller support@oeko.net wrote:

Hi Aki,

On Mon, Dec 28, 2020 at 03:37:33PM +0200, Aki Tuomi wrote:

...
What you are describing will not be solved by looking at dovecot source code.

ok.

...
Can you describe your configuration more? Maybe you are running dovecot in chroot or something? Can you consider using LMTP instead of LDA?

I want to have as much information about the users and their mailboxes as possible in an SQL database (I use Pg), so I can easily move things around, change permissions and what-not.

I thought about using LMTP, but so far shyed away from it because I suspected that it would be unable to do uid/gid switching on a per-user basis. But I'll check this idea out as well.

I have not done anything special to the Debian installation, but checking if that contains an unwanted chroot somewhere, is a good idea!

Thanks, Toni

LMTP can do uid/gid switching per user.

Aki

Toni Mueller

27 Dec 27 Dec

10:03 p.m.

Hi,

On Sun, Dec 27, 2020 at 07:54:56PM +0000, Toni Mueller wrote:

...

My question is why this thing wants to deliver to /var/mail, despite having a different location from the userdb, and how I can force it to use the location from the userdb.

I forgot to say that this thing has no trouble writing a file to roughly the desired location, reading

/path-to-mailboxen/example.com/user/mail/.imap /path-to-mailboxen/example.com/user/mail/.imap/dovecot.list.index.log

I've tried again by removing the entire user's directory, then saying "postfix flush" to push the email, when the file above was re-created.

Thanks, Toni

Christian Schmidt

28 Dec 28 Dec

3:44 p.m.

Hi,

Toni Mueller, 27.12.20:

...

16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 dovecot: lda(user@example.com)<5291>: msgid=20201226224933.014608@laptop.example.com: save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775) 16:04:16 postfix/pipe[5284]: 8CD6CE072E: to=user@example.com, orig_to=userA@example2.com, relay=dovecot, delay=62083, delays=62083/0.04/0/0.04, dsn=4.3.0, status=deferred (temporary failure)

In /etc/postfix/master.cf, I have this to call it:

dovecot unix - n n - - pipe flags=DRhu user=_mailbox argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} -a ${recipient}

I've tried strace-ing dovecot-lda, but it didn't really help me to understand why it discards the result of the userdb lookup.

Can anyone please provide a cluebat, please?

I'd change the setup towards postfix handing over the messages to dovecot via lmtp.

You can easily achive this by setting relay_domains = btree:/etc/postfix/relay-transport in your postfix configuration and creating a file /etc/postfix/relay-transport: your_mail_domain lmtp:unix:private/lmtp-dovecot

Mit freundlichen Grüßen Christian Schmidt

-- No signature available.

Toni Mueller

1 Jan 1 Jan

5:45 p.m.

Hi,

thanks to all the people who tried to help me. I have made one more step into the right direction and wanted to share my findings.

To recap, LDA pulls the right data from userdb, then ignores it and tries to deliver to the system default maibox location at /var/mail:

On Sun, Dec 27, 2020 at 04:19:35PM +0000, Toni Mueller wrote:

...

16:04:16 dovecot: auth: Debug: userdb out: USER#0111#011user@example.com#011home=/path-to-mailboxen/example.com/user#011uid=12345#011gid=12345#011quota_rule=*:storage=0

^^^^^^^^^^^^^^^^^^^^^^

This shows that the database lookup works. The intended effect should be that the message is delivered to

/path-to-mailboxen/example.com/user/Maidir/new

16:04:16 dovecot: lda(user@example.com)<5291>: Error: Mailbox INBOX: open(/var/mail/user@example.com) failed: Permission denied (euid=12345(mailbox) egid=12345(mailbox) missing +w perm: /var/mail, we're not in group 8(mail), dir owned by 0:8 mode=0775)

^^^^^^^^^^^^^^^^^^^^^^

And this shows that dovecot-lda just ignores the result.

Directive merging:

I found that there may be a problem in merging configuration directives. In Debian, the main config file is /etc/dovecot/dovecot.conf. This includes some snippets in /etc/dovecot/conf.d, and, at last, includes /etc/dovecot/local.conf.

In /etc/dovecot/conf.d/10-mail.conf, there are these two statements:

mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail

In local.conf, I have different values for that, but only if I comment them out in 10-mail.conf, I get LDA to try to deliver to the directory where it should deliver. Therefore, I assume that later directives are not overwriting earlier directives, but "first wins" is an uncommon strategy - usually, it's "last wins". I'm not sure whether this is a bug or not, though, but a quick search did not reveal how things are supposed to be.

Wrong mailbox settings:

I want to deliver to maildir and have these settings: Globally, ie, outside of all "blocks" (<token> [<name>] { ... }), I have

mail_location = maildir:~/Maildir:INBOX=maildir:~/Maildir

In addition to that, I have this:

namespace inbox { inbox = yes location = maildir:~/Maildir }

But Dovecot wrote the mail to a file in mbox format, and the file being at ~/mail/inbox, with ~ being the home dir as taken from the SQL database.

Autocreate fails:

I now have this for LDA and IMAP:

-------- cut protocol imap { mail_plugins = autocreate quota imap_quota mail_log trash virtual notify mail_max_userip_connections = 10 }

protocol lda { mail_plugins = autocreate quota virtual mail_log trash notify ... } -------- cut

In the first instance, the home from the database was created by Dovecot, but then I moved it aside, so as to have Dovecot create a new one. But now I get this, even after lifting the plugin settings of LDA to the global scope:

Jan 1 15:07:52 dovecot: lda(user@example.com)<13951>: Fatal: Namespace '': Mail storage autodetection failed with home=/path-to-mailboxen/example.com/user

Setting 'mail_location', as was suggested numerous times on the Internet, does not seem to have the desired effect, and I only have the 'inbox' namespace, anyway.

Any ideas, please?

TIA!

Cheers, Toni

Piotr Auksztulewicz

6:59 p.m.

On Fri, Jan 01, 2021 at 03:45:14PM +0000, Toni Mueller wrote:

...

I found that there may be a problem in merging configuration directives. In Debian, the main config file is /etc/dovecot/dovecot.conf. This includes some snippets in /etc/dovecot/conf.d, and, at last, includes /etc/dovecot/local.conf.

In local.conf, I have different values for that, but only if I comment them out in 10-mail.conf, I get LDA to try to deliver to the directory where it should deliver. Therefore, I assume that later directives are

Please note that local.conf is included by include_try directive. Check local.conf permissions versus userid running deliver, configured in master.cf. If it can't read local.conf, it will simply ignore it.

-- Piotr "Malgond" Auksztulewicz firstname@lastname.net

Toni Mueller

7:44 p.m.

Hi Piotr,

On Fri, Jan 01, 2021 at 05:59:42PM +0100, Piotr Auksztulewicz wrote:

...

Please note that local.conf is included by include_try directive. Check local.conf permissions versus userid running deliver, configured in master.cf. If it can't read local.conf, it will simply ignore it.

I was somehow not aware of this problem, but this solved it nicely.

There are some problems, still, but now things "almost work".

Thanks a lot, Toni

1414

Age (days ago)

1419

Last active (days ago)

List overview

15 comments

5 participants

participants (5)

Aki Tuomi
Christian Schmidt
Piotr Auksztulewicz
Tamsy
Toni Mueller