- dovecot - dovecot.org

[BUG?] Double quota calulation when special folder is present
by Mark Moseley 07 Aug '19

07 Aug '19

Just hoping to get some dev eyes on this. I'm incredibly reluctant to throw the word 'bug' around (since 99 times out of 100, it's not -- it's almost always the config), but I can't think of any way that this could be a config issue, esp when the pre-2.2.34 version works as expected. I noticed during troubleshooting that dovecot errors out if I try to create a subfolder called 'INBOX' but it'll happily create a subfolder called INBOX.SomethingElse (i.e. a folder called INBOX.INBOX.SomethingElse - resulting in a directory called .INBOX.SomethingElse on the filesystem, and leading to the problem described below). Is that sub-subfolder creation (where the top level subfolder matches the namespace name) supposed to be allowed? It seems odd that 'INBOX' (as a subfolder of INBOX) would be blocked but INBOX.SomethingElse (as a subfolder of INBOX) would be allowed. I'd expect INBOX.SomethingElse (i.e. INBOX.INBOX.SomethingElse) would be blocked as well. On Wed, Mar 13, 2019 at 4:46 AM Bernd Wurst via dovecot <dovecot(a)dovecot.org> wrote: > Hello, > > we're operating dovecot on a small server. Some years ago, we migrated > from courier IMAP to dovecot. Therefore, we defined our default > Namespace "inbox" with prefix "INBOX." to have this compatible. I found > this in some migration docs those days. Generally, everything worked as > expected. > > Our only namespace is configured like this: > > namespace inbox { > separator = . > prefix = INBOX. > inbox = yes > } > > Regularly, there is no folder named INBOX or .INBOX in the file system, > I suppose this is correct. But I found a special corner case today when > it comes to quota calculation. > > When - for whatever reason - a folder .INBOX.foo (for arbitrary values > of foo) exists, the whole mailbox is counted twice in quota > recalculation. Just creating .INBOX does nothing but a subfolder > triggers the problem. > > This is my shell view (replaced username and file path and deleted > unnecessary debug output) > > $ cat maildirsize > 268435456S > 14697 17 > $ maildirmake .INBOX.foo > $ sudo doveadm -D quota recalc -u <user> > [...] > doveadm(<user>): Debug: Namespace inbox: type=private, prefix=INBOX., > sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes > location=maildir:/home/.../test > doveadm(<user>): Debug: maildir++: root=/home/.../test, index=, > indexpvt=, control=, inbox=/home/.../test, alt= > doveadm(<user>): Debug: Namespace : type=private, prefix=, sep=, > inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none > doveadm(<user>): Debug: none: root=, index=, indexpvt=, control=, > inbox=, alt= > doveadm(<user>): Debug: quota: quota_over_flag check: quota_over_script > unset - skipping > doveadm(<user>): Debug: Quota root User quota: Recalculated relative > rules with bytes=268435456 count=0. Now grace=26843545 > doveadm(<user>): Debug: Namespace INBOX.: Using permissions from > /home/.../test: mode=0700 gid=default > > $ cat maildirsize > 268435456S > 29394 34 > > > So the used quota has exactly been doubled by just creating an empty > subfolder. > > Do you have any pointers for fixing my configuration or is this a bug in > dovecot? > > I coincidentally resurrected a months-old thread with this same issue a few days ago. I'm seeing the exact same after upgrading from 2.2.32 to 2.2.36. The original poster (who also narrowed it down to something in 2.2.34) mentioned a workaround that does indeed work, namely setting mailbox_list_index=no: > doveadm -o 'mailbox_list_index=no' quota recalc -u myuser I've been staring at diffs of 2.2.33 and 2.2.34 without anything jumping out at me (not a C guy, sadly). Maybe src/lib-storage/index/index-storage.c or src/lib-storage/list/mailbox-list-fs-iter.c or src/lib-storage/list/mailbox-list-index-iter.c or src/lib-storage/list/mailbox-list-index.c? The latter few have some added strcmp's against "INBOX". Then again, there's a lot of new code in the diffs under src/lib-storage that references INBOX specifically.

3 5

Dovecot replication and userdb "noreplicate".
by Reio Remma 07 Aug '19

07 Aug '19

Hello! I finally took the time and spent two days to set up replication for my server and now I have a question or two. I initially set noreplicate userdb field to 1 for all but a test user, but I could still see in the logs that all mailboxes were trying to connect to the other server via SSH. Is that normal? Jun 22 16:55:22 host dovecot: dsync-local(user(a)host.ee)<>: Error: Remote command returned error 84: ssh -i /home/vmail/.ssh/vmail.pem -l vmail backup.host.ee doveadm dsync-server -D -u user(a)host.ee Then I ended up setting mail_replica in userdb for only my test user, but I could still see in the logs that it was trying to sync the others as well, despite mail_replica being 0 for the rest. Jun 22 20:52:59 host dovecot: doveadm(user(a)host.ee): Fatal: -N parameter requires syncing with remote host I also notice (and read from recent posts) that sieve script replication doesn't work at all. Dovecot v2.3.6 and Pigeonhole from the official Dovecot CentOS repo. Thanks, Reio PS: Getting SSH for Dovecot to work with SELinux on CentOS 7 was fun as usual. :)

3 6

Incorrect link on download page
by Peter 06 Aug '19

06 Aug '19

The link to the latest 2.2 release points to 2.2.36.1 but the latest 2.2 is actually 2.2.36.3. Peter

2 1

doveadm: Error: open(/proc/self/io) failed
by me＠tdiehl.org 06 Aug '19

06 Aug '19

Hi, I am running dovecot-2.2.36-3.el7.x86_64 on a Centos 7 machine. I keep seeing the following errors in the dovecot.log: Jul 22 12:52:04 vmail2 dovecot: doveadm: Error: open(/proc/self/io) failed: Permission denied Dovecot -n is listed below: # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 3.10.0-957.21.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: vmail2.kmg.mydomain.com auth_master_user_separator = * auth_mechanisms = PLAIN LOGIN deliver_log_format = from=%{from}, envelope_sender=%{from_envelope}, subject=%{subject}, msgid=%m, size=%{size}, %$ dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf } doveadm_password = # hidden, use -P to show it doveadm_port = 2525 first_valid_uid = 2000 last_valid_uid = 2000 mail_gid = 2000 mail_location = maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/ mail_plugins = quota mailbox_alias acl mail_log notify stats replication mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace { inbox = yes location = mailbox Archive { auto = no special_use = \Archive } mailbox Archives { auto = no special_use = \Archive } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox "Junk E-mail" { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } namespace { list = children location = maildir:%%Lh/Maildir/:INDEX=%%Lh/Maildir/Shared/%%Ld/%%Ln prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-master-users driver = passwd-file master = yes } plugin { acl = vfile acl_shared_dict = proxy::acl fts_autoindex = yes fts_autoindex_max_recent_msgs = 50 imapsieve_mailbox1_before = file:/usr/lib64/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/usr/lib64/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * mail_log_events = delete undelete expunge mailbox_delete mailbox_rename mail_log_fields = uid box msgid size from subject mail_replica = tcp:vmail1.kmg.mydomain.com mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_old = Sent mailbox_alias_old2 = Sent quota = dict:user::proxy::quotadict quota_grace = 10%% quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u sieve = ~/sieve/dovecot.sieve sieve_before = /var/vmail/sieve/dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /var/vmail/sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_max_redirects = 30 sieve_pipe_bin_dir = /usr/lib64/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_send_from_recipient = yes stats_refresh = 30 secs stats_track_cmds = yes } protocols = pop3 imap sieve lmtp service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service config { unix_listener config { user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 2525 } user = vmail } service imap-login { process_limit = 500 service_count = 1 } service lmtp { executable = lmtp -L inet_listener lmtp { address = 127.0.0.1 port = 24 } process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service managesieve-login { inet_listener sieve { address = 127.0.0.1 port = 4190 } } service pop3-login { service_count = 1 } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0666 } } service stats { fifo_listener stats-mail { mode = 0644 user = vmail } inet_listener { address = 127.0.0.1 port = 24242 } } ssl = required ssl_cert = </etc/pki/tls/certs/my_cert.crt ssl_cipher_list = ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5 ssl_client_ca_file = /etc/pki/tls/cert.pem ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes syslog_facility = local5 userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } protocol lda { lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_plugins = quota mailbox_alias acl mail_log notify stats replication sieve } protocol lmtp { lmtp_save_to_detail_mailbox = yes mail_plugins = quota sieve recipient_delimiter = + } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_max_userip_connections = 30 mail_plugins = quota mailbox_alias acl mail_log notify stats replication imap_quota imap_acl imap_stats imap_sieve } protocol pop3 { mail_max_userip_connections = 30 mail_plugins = quota mailbox_alias acl mail_log notify stats replication pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } Everything seems to be working properly and the dovecot wiki says that /proc/self/io is for statistics. I am thinking they are harmless but they are generating a bunch of noise in the logs and if harmless, I would like to silence them. Can someone let me know what causes this and if I should be concerned about this? My Google foo has not been helpful on this. Regards, -- Tom me(a)tdiehl.org

7 12

GSSAPI vs group check
by Németh Ákos Ferenc 05 Aug '19

05 Aug '19

Dear All, Is it possible to make any authorization (eg. checking of group membership) in case of GSSAPI authentication? Our dovecot authenticates the users against PAM and GSSAPI. In the PAM file I'm able to check if a user is a member of a selected (e.g mailreader) group. If the user is member, he can login otherwise not (see below). If the user has a valid Kerberos ticket and he tries to login via GSSAPI, I can't restrict him if he is not a member of the selected group. How can I overcome this issue? My config: passdb { driver = pam # [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=<n>] # [cache_key=<key>] [<service name>] #args = dovecot } userdb { # <doc/wiki/AuthDatabase.Passwd.txt> driver = passwd # [blocking=no] #args = # Override fields from passwd #override_fields = home=/home/virtual/%u } ...in PAM file: auth [success=1 default=ignore] pam_succeed_if.so user ingroup mailreader auth [success=ignore default=2] pam_succeed_if.so user ingroup admins auth [success=ignore default=1] pam_succeed_if.so uid >= 1000 auth [success=3 default=ignore] pam_winbind.so krb5_auth krb5_ccache_type=FILE cached_login auth [success=ignore default=1] pam_succeed_if.so uid < 1000 auth [success=1 default=ignore] pam_unix.so nullok_secure try_first_pass auth requisite pam_deny.so auth required pam_permit.so Thank you. Br, Ákos

2 3

doveadm sis deduplicate problem
by Peter Benko 03 Aug '19

03 Aug '19

Hi all, I'm running dovecot 2.3.7.1, with mdbox and single instance storage. I use offline deduplication like this: mail_attachment_fs = sis-queue /mailboxes/%d/_attachments_/queue:posix Each week I run doveadm sis deduplicate to deduplicate the mail attachments. Everything was fine until I mistakenly run doveadm sis deduplicate twice, simultaneously. I ended up having some error messages like this: Error: unlink(/mailboxes/domain.com/_attachments_/queue/067518ec61f872420813727326… failed: No such file or directory (in doveadm-sis.c:260) Also, the sis store has a suspicious temporary (leftover?) file (in the according _attachments_/06/75/ dir): -rw------- 4 vmail vmail 941197 Aug 2 15:48 temp.server-name.9633.77f5a7183d74f72b So the question is, shall I be nervous about this temp file? Does the error mean that the deduplication did not finish correctly? Is my mail store in a consistent state now? What shall I do with the temp file? I'd really appreciate some help... Regards, Peter

1 0

Sieve redirect
by B Fischer 03 Aug '19

03 Aug '19

What is the intended way to configure Sieve for the redirect command? Can I do it using in Dovecot (e.g. using the submission config) or does it have to be done in sendmail? I'm running Dovecot in a container and the MTA/MSA is in another container. Dovecot version: 2.3.2.1 (0719df592) - Björn

1 0

index failed
by ＠lbutlr 02 Aug '19

02 Aug '19

When reindexing I go the following error Mailbox test: UID=1: read(/path/to/mail,S=4248,W=4349) failed: Cached message size larger than expected (4248 > 1182, box=test, UID=1) (read reason=mail stream) In this case, the message was an unimportant test message with no content, so I simply removed it, but had it been important what would I need to do? -- Nine-tenths of the universe is the knowledge of the position and direction of everything in the other tenth. Every atom has its biography, every star its file, every chemical exchange its equivalent of the inspector with a clipboard. It is unaccounted for because it is doing the accounting for the rest of it. Nine-tenths of the universe, in fact, is the paperwork. --The Thief of Time

1 0

Question about replication
by Coy Hile 02 Aug '19

02 Aug '19

Good morning, I’m looking at dovecot replication for the first time for a particular use case: The VM running dovecot is currently in a public cloud, and I want to move it on-premises, so I want to replicate all mail data from the older instance to the new (then cutover internal DNS) (and in the process update to newer features on the internal instance such as mdbox vs Maildir). However, the new instance I’m building is internal, would have access to the internet only via NAT, so the replication would have to be one way. Is there anything in particular that I need to keep in mind when I’m trying to setup this sort of setup? The documentation that I’ve read seems to imply (or assume) synchronous, bidirectional replication. Thanks, and apologies if this is an obvious question; this is my first foray into doing something not utterly trivial with Dovecot.

2 3

Re: Panic: Module context expire_mail_user_module missing (Debian 10)
by Alexander Dalloz 02 Aug '19

02 Aug '19

Am 2019-08-02 11:05, schrieb Thomas Krause: Thomas, please keep the discussion on list. There is zero need to copy send any list discussion to my personal address. > # diff dovecot.conf dovecot.conf-ok > 5c5 > < mail_plugins = quota mailbox_alias acl zlib expire > --- >> mail_plugins = quota mailbox_alias acl zlib #expire > 170c170 > < expire = mysql:/etc/dovecot/dovecot-expire.conf > --- >> #expire = mysql:/etc/dovecot/dovecot-expire.conf > > > # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.4 () > doveconf: Warning: NOTE: You can get a new clean config file with: > doveconf -Pn > dovecot-new.conf > doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:16: > ssl_dh_parameters_length is no longer needed > doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:17: > ssl_protocols has been replaced by ssl_min_protocol > doveconf: Warning: NOTE: You can get a new clean config file with: > doveconf -Pn > dovecot-new.conf > doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:16: > ssl_dh_parameters_length is no longer needed > doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:17: > ssl_protocols has been replaced by ssl_min_protocol Fix those obvious things. > # OS: Linux 4.19.0-5-amd64 x86_64 Debian 10.0 > # Hostname: mail.example.org > auth_mechanisms = PLAIN LOGIN > dict { > acl = mysql:/etc/dovecot/dovecot-share-folder.conf > expire = mysql:/etc/dovecot/dovecot-expire.conf > quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf > } > first_valid_uid = 1001 > last_valid_uid = 1001 > login_log_format_elements = user=<%u> rip=%r mpid=%e > session=<%{session}> > mail_gid = 1001 > mail_plugins = quota mailbox_alias acl zlib expire The expire plugin enabled under mail_plugins, but no configuration in the plugins { } section. So it has zeor function as you have not defined the mailboxes to track. > mail_uid = 1001 > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate mime foreverypart > extracttext > namespace { > inbox = yes > location = > mailbox Archive { > auto = no > special_use = \Archive > } > mailbox Archives { > auto = no > special_use = \Archive > } > mailbox "Deleted Messages" { > auto = no > special_use = \Trash > } > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > special_use = \Junk > } > mailbox "Junk E-mail" { > auto = no > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Items" { > auto = no > special_use = \Sent > } > mailbox "Sent Messages" { > auto = no > special_use = \Sent > } > mailbox Spam { > auto = no > special_use = \Junk > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > prefix = > separator = / > type = private > } > passdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > plugin { > acl = vfile > acl_shared_dict = proxy::acl > auth_socket_path = /var/run/dovecot/auth-master > mailbox_alias_new = Sent Messages > mailbox_alias_new2 = Sent Items > mailbox_alias_old = Sent > mailbox_alias_old2 = Sent > quota = dict:user::proxy::quotadict > quota_grace = 10%% > quota_rule = *:storage=1G > quota_warning = storage=100%% quota-warning 100 %u > quota_warning2 = storage=95%% quota-warning 95 %u > quota_warning3 = storage=90%% quota-warning 90 %u > quota_warning4 = storage=85%% quota-warning 85 %u > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > sieve_max_redirects = 30 > zlib_save = gz > zlib_save_level = 6 > } Above not expire configuration. See https://wiki.dovecot.org/Plugins/Expire And you should consider migrating to the autoexpunge settings. > protocols = pop3 imap sieve lmtp > service anvil { > client_limit = 1203 > } > service auth { > client_limit = 1400 > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-client { > group = www-data > mode = 0660 > } > unix_listener auth-master { > group = vmail > mode = 0666 > user = vmail > } > unix_listener auth-userdb { > group = vmail > mode = 0660 > user = vmail > } > } > service dict { > process_limit = 150 > unix_listener dict { > group = vmail > mode = 0660 > user = vmail > } > } > service imap-login { > process_limit = 500 > service_count = 1 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > user = vmail > } > service pop3-login { > process_limit = 500 > service_count = 1 > } > service quota-warning { > executable = script /usr/local/bin/dovecot-quota-warning.sh > unix_listener quota-warning { > group = vmail > mode = 0660 > user = vmail > } > } > ssl = required > ssl_cert = </etc/ssl/ssl.crt/mail.example.org.pem > ssl_cipher_list = > EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA256:EECDH:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA128-SHA:AES128-SHA > ssl_key = # hidden, use -P to show it > ssl_prefer_server_ciphers = yes > userdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > protocol lda { > auth_socket_path = /var/run/dovecot/auth-master > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > mail_plugins = quota mailbox_alias acl zlib expire sieve > postmaster_address = root > } > protocol lmtp { > lmtp_save_to_detail_mailbox = yes > mail_plugins = quota mailbox_alias acl zlib expire sieve > postmaster_address = postmaster > recipient_delimiter = + > } > protocol imap { > imap_client_workarounds = tb-extra-mailbox-sep > mail_max_userip_connections = 30 > mail_plugins = quota mailbox_alias acl zlib expire imap_quota > imap_acl > } > protocol pop3 { > mail_max_userip_connections = 30 > mail_plugins = quota mailbox_alias acl zlib expire > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > } Alexander

1 0