- dovecot - dovecot.org

Failed to map transaction log path - Stale file handle
by Gonzalo Palacios Goicolea 01 Mar '21

01 Mar '21

Hi All, We recently moved from RHEL6 to RHEL7 and updated dovecot from dovecot-2.2.10-1_14.el6.x86_64 to dovecot-2.2.36-8.el7.x86_64. We use NFS for storing the mail, and 5 servers running at the same time. Before updating dovecot we stopped the service and deleted all dovecot.index* files Now we're experiencing some problems. For example, when deleting an email it is note deleted and appears again duplicated. These are the logs: Feb 28 08:50:42 server5 dovecot: imap(user@domain): Error: Index path/Maildir/dovecot.index: Lost log for seq=5 offset=228: Missing middle file seq=5 (between 5..4294967295, we have seqs 4): Log is locked - newer log can't exist (initial_mapped=1, reason=Index mapped) Feb 28 08:50:42 server5 dovecot: imap(user@domain): Warning: fscking index file path/Maildir/dovecot.index Feb 28 08:50:42 server5 dovecot: imap(user@domain): Error: Failed to map transaction log path/Maildir/dovecot.index.log at sync_offset=10328 after locking: pread() failed: Stale file handle Feb 28 08:50:42 server5 dovecot: imap(user@domain): Error: Index path/Maildir/dovecot.index: Lost log for seq=5 offset=228: Missing middle file seq=5 (between 5..4294967295, we have seqs 4): Requested newer log than exists - still after NFS flush: Log inode is unchanged (initial_mapped=1, reason=Index mapped) Feb 28 08:50:45 server5 dovecot: imap(user@domain): Error: Failed to map transaction log path/Maildir/dovecot.index.log at sync_offset=10328 after locking: pread() failed: Stale file handle Feb 28 08:50:45 server5 dovecot: imap(user@domain): Error: Failed to map view for path/Maildir/dovecot.index: Missing middle file seq=5 (between 5..5, we have seqs 4): Requested newer log than exists - still after NFS flush: Log inode is unchanged Usually restarting the email client solves the problem for a while. We've tried deleting the indexes but it happens again some time later. This is dovecot config: dovecot -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 3.10.0-1160.6.1.el7.x86_64 x86_64 Red Hat Enterprise Linux Server release 7.9 (Maipo) # Hostname: server auth_cache_negative_ttl = 5 mins auth_cache_size = 500 M auth_cache_ttl = 5 mins auth_failure_delay = 5 secs default_vsz_limit = 1 G disable_plaintext_auth = no hostname = server imap_logout_format = in(bytes read)=%i out(bytes sent)=%o listen = * lock_method = dotlock mail_fsync = always mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " quota mail_log notify" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify mbox_write_locks = fcntl mmap_disable = yes namespace { location = maildir:%h/Maildir/_backup prefix = _backup. separator = . type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_log_events = delete undelete expunge mailbox_delete mailbox_rename mailbox_create save mail_log_fields = uid box msgid from subject quota = maildir:_backup:ns= quota_exceeded_message = El espacio asignado para el almacenamiento del correo del destinatario ha superado su capacidad total. Por ello, el correo que usted intenta enviar no se puede entregar al destinatario. quota_warning = storage=70%% quota-warning 70 %u quota_warning1 = storage=90%% quota-warning 90 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=99%% quota-warning 99 %u sieve = ~/.dovecot.sieve sieve_dir = ~/Maildir/sieve sieve_extensions = +notify +imapflags } postmaster_address = postmaster@domain protocols = imap pop3 sieve service auth { unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } process_min_avail = 20 } service imap { process_limit = 2048 } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } process_min_avail = 20 } service pop3-login { inet_listener pop3 { port = 110 } process_min_avail = 20 } service pop3 { process_limit = 2048 } service quota-warning { executable = script /etc/dovecot/scripts_notification_quota/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = # hidden, use -P to show it submission_host = smtp.domain:25 syslog_facility = local1 userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lda { mail_plugins = " quota mail_log notify sieve" } protocol imap { mail_max_userip_connections = 10000 mail_plugins = " quota mail_log notify imap_quota" } protocol pop3 { mail_max_userip_connections = 10 } Can you please help me? Thanks in advance and best regards -- *Gonzalo Palacios Goicolea* U.T de Infraestructura de Equipos Centrales /Tecnologías de la Información/ Universidad Autónoma de Madrid • Campus de Cantoblanco <3D"http://www.uam.es/"> Antes de imprimir este correo piense si es necesario.Cuidemos el medioambiente.

2 5

Issue with masterauth
by Samuel Mutel 01 Mar '21

01 Mar '21

Hello, I would like to setup masterauth account to be able to connect to imap to log in as other users without password. My users are currently setup in LDAP and it is working correctly. I follow the page https://doc.dovecot.org/configuration_manual/authentication/master_users/ to setup this but I think I forgot something. When I try to login with telnet: Master user logging in as toto => it seems that i can connect with master user ldap(toto,127.0.0.1,<quR66Hm8KgB/AAAB>): invalid credentials => however ldap is request for password check, why ? Normmally the goal of master user is to not specify the password of the final user... Thanks.

2 1

Search is uninterruptable
by van der Kamp, John 01 Mar '21

01 Mar '21

Hi, When a user is doing a search, and the data is not indexed, then a search can take a very long time. It seems that search process is uninterruptable. The IMAP connection gets a status message with an ETA every so often, but an administrator on the server cannot stop it. Any SIGTERM or doveadm kick is ignored. Also, when the client times out (it seems clients are not seeing the ETA as a reason to keep the connection open) and the client disconnects is this search process continuing, even though any write to the socket would return an error. I believe the server is stuck in this loop: https://github.com/dovecot/core/blob/master/src/lib-storage/index/index-sea… Somehow the server needs to check for disconnected clients and let the session be kicked from the server too. You probably know better what to do about this then me. John Amazon Development Center (Netherlands) B.V., Johanna Westerdijkplein 1, 2521 EN The Hague, Registration No. Chamber of Commerce 56869649, VAT: NL 852339859B01

3 2

dsync and sieve again
by Edgaras Lukoševičius 01 Mar '21

01 Mar '21

Hi, I'm starting to wonder if dsync is supposed to sync sieve scripts at all? Or should it work only as a part of replicator? Because I was not able to get it working on multiple Dovecot and Pigeon versions, and I see lots of replication/dsync and sieve related questions in the mailing list. I'm trying to get it to work using latest versions on http://repo.dovecot.org/ce-2.3-latest repo. The versions I am testing at the moment are: - Dovecot 2.3.13 (89f716dc2) - Pigeonhole 0.5.13 (cdd19fe3) It's a straightforward setup, nothing fancy. So the question is - is it supposed to work at all without replicator?

2 3

sieve_after seems to not bring any effect to life in dovecot 2.3.13 dovecot-pigeonhole-0.5.13
by Bug Report 26 Feb '21

26 Feb '21

dovecot 2.3.13 (89f716dc2) dovecot-pigeonhole-0.5.13 OS: FreeBSD amd64 12.2-RELEASE After following code ------ require ["copy"]; # rule:[redirect] if true { redirect :copy "mail(a)mail.com"; keep; } ------ (both with keep and without keep) is executed sieve_after seems to not execute at all. Same code together with sieve_after was working as supposed in earlier versions.

3 7

doveadm user '*' vs sssd: enumeration works only once
by Heiko Schlittermann 25 Feb '21

25 Feb '21

Hi, I've a quite simple setup using sssd (id_provider, auth_provider: AD), and dovecot. doveadm user * returns the full user list only once. A 2nd invocation of the same command within a small timeframe returns a subset of the local users only (the users I've in /etc/passwd with uid >= first_valid_uid). If I wait until the dovecot/auth worker processes are gone (or if I kill them), I get the full list again. Calling getent passwd repeateadly always returns the full list of users. Question: - [ ] unfortunate configuration (see attachment) - [ ] misbehaviour of sssd (more specifically its nss module) - [ ] misbehaviour of dovecot/auth processes? Dovecot: 2.3.4.1 (f79e8e7e4) Operating System: Debian GNU/Linux 10 (buster) Sssd: 1.16.30 Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE -

2 8

doveadm copy not respecting quotas?
by MAREN ZUBIZARRETA 24 Feb '21

24 Feb '21

Hello: I have a problema with the quotas at my system with dovecot 2.3.9.2, y use: quota_rule5 = RECUPERADOS_VGTIC:ignore I want to be free of quota limits at that mailbox RECUPERADOS_VGTIC when copying messages from "lazy_expunge" folder To recover from backup I use doveadm copy: doveadm copy -u nppgiibi RECUPERADOS_VGTIC mailbox .BORRADOS_VGTIC/sent-mail ALL But when I do so, quota for this mailbox RECUPERADOS_VGTIC is not ignored, and it grows to 200% Is this a bug? Or there is something that I'm doing wrongly? Thanks for your help. Maren PD: My config file for quotas follows below: /etc/dovecot/conf.d/90-quota.conf # Quota mail_vsize_bg_after_count = 100 # Limits plugin { quota = count:User quota quota_rule = *:storage=10G #for backups and lazy expunge quota_rule3 = .BORRADOS:ignore quota_rule4 = Recuperados_VGTIC:ignore quota_rule5 = RECUPERADOS_VGTIC:ignore quota_rule6 = BORRADOS_VGTIC:ignore quota_rule7 = .BORRADOS_VGTIC:ignore # This is required quota_vsizes = yes } -----Mensaje original----- De: dovecot <dovecot-bounces(a)dovecot.org> En nombre de Aki Tuomi Enviado el: miércoles, 24 de febrero de 2021 8:37 Para: Mark Constable <markc(a)renta.net>; dovecot(a)dovecot.org Asunto: Re: LMDB support? > On 24/02/2021 08:15 Mark Constable <markc(a)renta.net> wrote: > > > Hi all, is there any way I could use LMDB for userdb and passdb lookups? You can use Lua userdb/passdb. See https://doc.dovecot.org/configuration_manual/authentication/lua_based_authe… Aki

1 0

LMDB support?
by Mark Constable 23 Feb '21

23 Feb '21

Hi all, is there any way I could use LMDB for userdb and passdb lookups?

2 1

Problem with fs:User quota: Postfix quota-status returns always DUNNO
by Niccolo Rigacci 23 Feb '21

23 Feb '21

I installed a mail server on a Debian GNU/Linux 10 Buster, which provide Dovecot 2.3.4.1. It works almost everything, but I'm unable to make the Postfix quota service to work as expected. First of all I verify that the Dovecot idea of quota is OK: doveadm quota get -u username(a)example.org Quota name Type Value Limit % User quota STORAGE 2752 3500 78 Then I ask the Postfix quota-status service: printf "recipient=username(a)example.org\nsize=3000000\n\n" \ | nc localhost 12340 action=DUNNO But a size of 3 Mb should trigger the "action=554 5.2.2 Quota exceeded" message. I find a workaround for this problem: just adding the following section will "fix" the problem: plugin { quota_over_flag_value = FALSE quota_over_script = quota-warning mismatch %u } How the quota_over_flag_value is related to the quota over filesystem? Why should I add both the quota_over_flag_value AND the quota_over_script options? The script is executed always for each SMTP check (regardless of the quota_over_flag_value is FALSE or TRUE), obviously I provided a "do nothing" quota-warning script. Here the relevant parts of "dovecot -n" output: plugin { quota = fs:User quota:user quota_grace = 5%% quota_over_flag_value = FALSE quota_over_script = quota-warning mismatch %u quota_status_nouser = DUNNO quota_status_overquota = 554 5.2.2 Quota exceeded quota_status_success = DUNNO quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = ~/sieve_before.d/ sieve_extensions = +vnd.dovecot.filter sieve_filter_bin_dir = /usr/local/lib/dovecot/sieve-filter sieve_filter_socket_dir = sieve-filter sieve_plugins = sieve_extprograms } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = dovecot mode = 0666 user = dovecot } user = root } -- Niccolo Rigacci - http://www.rigacci.net/ Firenze/Prato - Italy Tel. Mobile: +39-327-5619352

1 0

Bug renaming folder hierarchies
by Klaus Steinberger 23 Feb '21

23 Feb '21

Hi, we discovered a major problem with renaming folders with subfolders testszenario: folders: test1 test1/test1-1 test1/test1-2 all 3 folders contain a test message after renaming the top level folder test1 to test2 following folders exist: test2 test2/test1-1 test2/test1-2 the subfolders test1-1 and test1-2 are empty, the top level folder contains the correct message. Now renmaing test2 back to test1: Alle folders again contain their messages. So this looks like during renaming the top level folder just the message references for the subfolders got lost, and with renaming back they are there again. I can reproduce this behavior both with thunderbird and Horde Webmail as a client, also it happens with a second different dovecot setup Here is the dovecot config: root@dmz-sv-dovecot ~]# dovecot -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 3.10.0-1160.15.2.el7.x86_64 x86_64 Scientific Linux release 7.9 (Nitrogen) nfs4 # Hostname: dmz-sv-dovecot.physik.uni-muenchen.de auth_cache_negative_ttl = 5 mins auth_cache_size = 10 M auth_cache_ttl = 5 mins auth_socket_path = /var/run/dovecot/auth-userdb auth_username_format = %Ln auth_verbose = yes default_client_limit = 32768 default_process_limit = 8192 dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_uid = 8 imap_capability = +SPECIAL-USE imap_hibernate_timeout = 1 mins imapc_features = rfc822.size imapc_host = zarafa.physik.uni-muenchen.de imapc_user = %u lda_original_recipient_header = X-Original-To mail_fsync = always mail_location = mdbox:/var/vmail/%Lu:FULLDIRNAME=DbOx-mAiLs:UTF-8:VOLATILEDIR=/var/dovecot-volatile/%Lu:INDEX=/var/vmail/%Lu/indexes mail_nfs_index = yes mail_plugins = quota acl lazy_expunge fts fts_lucene mail_prefetch_count = 20 mail_server_admin = mailto:helpdesk@physik.uni-muenchen.de mail_server_comment = IMAP Server Fakultätfür Physik LMU München mailbox_list_index_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl metric imap_command { filter = event=imap_command_finished AND tagged_reply_state=OK group_by = cmd_name } metric imap_select_no { filter = event=imap_command_finished AND cmd_name=SELECT AND tagged_reply_state=NO } metric imap_select_no_notfound { filter = event=imap_command_finished AND cmd_name=SELECT AND tagged_reply="NO*Mailbox doesn't exist:*" } metric push_notifications { filter = event=push_notification_finished } metric storage_http_gets { filter = event=http_request_finished AND category=storage AND method=get } mmap_disable = yes namespace { list = children location = mdbox:/var/vmail/%%Lu:FULLDIRNAME=DbOx-mAiLs:INDEX=/var/vmail/%%Lu/indexes:INDEXPVT=/var/vmail/%%Lu/indexes.pvt:UTF-8 prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace { hidden = yes list = children location = mdbox:/var/vmail/%Lu/expunged:FULLDIRNAME=DbOx-mAiLs:UTF-8 prefix = .EXPUNGED/ separator = / subscriptions = no type = private } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/dovecot.acl acl_shared_dict = file:/var/vmail/dovecot-dicts/shared-mailboxes fts = lucene fts_lucene = whitespace_chars=@. lazy_expunge = .EXPUNGED/ lazy_expunge_only_last_instance = yes quota = dict:User quota::proxy::quota quota_grace = 10%% quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_rule3 = Spam:storage=+100M quota_rule4 = .EXPUNGED:ignore recipient_delimiter = + sieve = file:/var/vmail/%Lu/sieve;active=/var/vmail/%Lu/dovecot.sieve sieve_vacation_use_original_recipient = yes } postmaster_address = postmaster(a)physik.uni-muenchen.de protocols = imap lmtp sieve sieve quota_full_tempfail = yes service auth { unix_listener auth-userdb { group = daemon mode = 0666 user = mail } } service dict { unix_listener dict { group = daemon mode = 0600 user = vmail } } service imap-hibernate { client_limit = 8192 unix_listener imap-hibernate { user = vmail } } service imap-login { process_min_avail = 6 service_count = 0 vsz_limit = 1 G } service imap-postlogin { executable = script-login -d /etc/dovecot/acl_groups.py user = $default_internal_user } service imap { executable = imap imap-postlogin process_limit = 8192 unix_listener imap-master { user = dovecot } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 1024 } service stats { unix_listener stats-reader { group = dovecot mode = 0660 user = vmail } unix_listener stats-writer { group = dovecot mode = 0660 user = vmail } } ssl = required ssl_cert = </etc/pki/tls/certs/dmz-sv-dovecot-with-chain.crt.pem ssl_client_ca_file = /etc/ssl/certs/ca-bundle.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = quota acl lazy_expunge fts fts_lucene sieve } protocol lda { mail_plugins = quota acl lazy_expunge fts fts_lucene sieve syslog_facility = mail } protocol imap { mail_max_userip_connections = 20 mail_plugins = quota acl lazy_expunge fts fts_lucene imap_quota imap_acl lazy_expunge } protocol sieve { mail_max_userip_connections = 10 managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 64 k

1 0