- dovecot - dovecot.org

replication Error with wrong file lock
by Frank Kirschner 18 Oct '24

18 Oct '24

Hi there, I have setup a replication between two dovecot servers. It seems working: # doveadm replicator status '*' && doveadm replicator dsync-status && doveadm replicator status username priority fast sync full sync success sync failed ******************** none 00:59:22 18:50:17 18:21:25 - ********************low 01:54:26 18:57:58 - y ********************none 21:42:55 21:47:55 21:36:24 - ******************** none 21:14:57 21:51:07 21:33:13 - ********************none 21:13:52 21:51:04 21:17:57 - ********************none 00:04:23 01:29:24 - y ********************none 02:34:15 21:56:23 02:34:15 - ******************** none 00:03:56 18:55:17 - y ********************none 01:18:42 01:18:42 19:18:46 - ********************none 21:20:04 21:24:46 21:43:49 - ********************none 00:05:13 00:09:36 21:47:55 - ********************none 00:16:50 18:55:17 18:45:57 - ********************none 00:04:23 18:55:17 38:07:24 y username type status - Not connected - Not connected - Not connected ********************normal Waiting for dsync to finish - Not connected Queued 'sync' requests 0 Queued 'high' requests 0 Queued 'low' requests 1 Queued 'failed' requests 0 Queued 'full resync' requests 0 Waiting 'failed' requests 3 Total number of known users 13 But on the one server, I found this errors repeating and this users are at the list above marked as failed: Oct 18 09:41:08 dsync-local(1(a)tld.de): Error: Couldn't lock /srv/mail/mail_storage/tld.de/1/.dovecot-sync.lock: fcntl(/srv/mail/mail_storage/tld.de/1/.dovecot-sync.lock, write-lock, F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by pid 71613) The directory shows: # ls -la /srv/mail/mail_storage/celebrate.de/1/ | grep '\.dove' drwxrwxrwx 5 vpostfix vpostfix 4096 Oct 18 08:55 .dovecot -rw------- 2 vpostfix vpostfix 0 Oct 18 09:37 .dovecot-sync.lock -rwxrwxrwx 1 vpostfix vpostfix 912 Mar 20 2023 .dovecot.sieve.log -rwxrwxrwx 1 vpostfix vpostfix 275 Oct 11 00:00 .dovecot.svbin So it seems, that there are more processes want accessing same locking file. But normally this and other files should not exist there: # ls -la /srv/mail/mail_storage/celebrate.de/fk/ | grep '\.index' -rw-rw-rw- 1 vpostfix vpostfix 194008 Oct 17 12:26 dovecot.index -rwxrwxrwx 1 vpostfix vpostfix 16890796 Oct 17 14:20 dovecot.index.cache -rw-rw-rw- 1 vpostfix vpostfix 7420 Oct 17 14:04 dovecot.index.log -rwxrwxrwx 1 vpostfix vpostfix 32824 Oct 17 11:46 dovecot.index.log.2 -rwxrwxrwx 1 vpostfix vpostfix 5752 Oct 9 05:14 dovecot.list.index -rwxrwxrwx 1 vpostfix vpostfix 7556 Oct 9 09:32 dovecot.list.index.log In my config, I have set: mail_location = maildir:~:VOLATILEDIR=/dev/shm/dovecot/%2.256Nu/%u:INDEX=/opt/dovecot/%2.256Nu/%u:ITERINDEX But below /dev/shm/dovecot and /opt/dovecot, directories and files are generated too Following my config and hoping for some useful hints: # dovecot -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 5.15.149-1-pve x86_64 CentOS Linux release 7.9.2009 (Core) # Hostname: host.tld.de disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_uid = 1000 imap_id_send = lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes log_path = /var/log/dovecot.log mail_fsync = always mail_full_filesystem_access = yes mail_gid = 1000 mail_home = /srv/mail/mail_storage/%d/%n mail_location = maildir:~:VOLATILEDIR=/dev/shm/dovecot/%2.256Nu/%u:INDEX=/opt/dovecot/%2.256Nu/%u:ITERINDEX mail_plugins = " notify replication" mail_privileged_group = vpostfix mail_uid = 1000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { mail_replica = tcp:192.168.130.192:4711 recipient_delimiter = + replication_full_sync_interval = 1 hour replication_sync_timeout = 10 sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /etc/dovecot/spam-global.sieve sieve_extensions = +vacation-seconds sieve_vacation_default_period = 1d sieve_vacation_max_period = 30d sieve_vacation_min_period = 0 sieve_vacation_use_original_recipient = yes } protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vpostfix } unix_listener replication-notify { mode = 0666 user = vpostfix } } service auth { inet_listener { address = 192.168.130.191 port = 3659 } unix_listener /var/spool/postfix/private/auth { group = vpostfix mode = 0666 user = vpostfix } unix_listener auth-userdb { group = vpostfix mode = 0600 user = vpostfix } } service config { unix_listener config { user = vpostfix } } service doveadm { inet_listener { port = 4711 } user = vpostfix } service imap-login { process_min_avail = 1 service_count = 1 } service lmtp { inet_listener lmtp { address = 192.168.130.191 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 user = vpostfix } } ssl_cert = </opt/fullchain.pem ssl_cipher_list = ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!AES256-SHA:AES:CAMELLIA:!CAMELLIA128-SHA:!DHE-RSA-CAMELLIA128-SHA:!CAMELLIA256-SHA:!DHE-RSA-CAMELLIA256-SHA ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lmtp { mail_plugins = " notify replication sieve" } protocol lda { mail_plugins = " notify replication sieve" } protocol imap { mail_max_userip_connections = 10 mail_plugins = " notify replication quota imap_quota" }

2 2

Sieve script error when quota is full
by Kosti 18 Oct '24

18 Oct '24

Hello. My sieve script gives an error; "failed with unsuccessful implicit keep". Here is maillog file output; (Yes my user does has quota exceeded, but is this the issue for the sieve script and is it normal for the script give such output?) postfix dovecot: lda(me(a)mydomain.com)<50709><iEd4O9YLDGcVxgAA/dJgOw>: sieve: msgid=<x(a)jol.mysite.com>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) postfix dovecot: lda(me(a)mydomain.com)<50709><iEd4O9YLDGcVxgAA/dJgOw>: sieve: Execution of script /etc/dovecot/sieve/default.sieve failed with unsuccessful implicit keep postfix dovecot: lda(me(a)mydomain.com)<50709><iEd4O9YLDGcVxgAA/dJgOw>: msgid=<x(a)jol.mysite.com>: rejected: Quota exceeded (mailbox for user is full) And /etc/dovecot/sieve/default.sieve file contains: require ["fileinto"]; if header :contains "X-Spam" "Yes" { fileinto "Junk"; stop; } Any idea would be much appreciated. Best regards.

1 0

Piegeonhole and PATH variable
by Julien Salort 18 Oct '24

18 Oct '24

Hello, I have a dovecot-sieve rule that has been working fine for years which calls sa-learn whenever a message is moved from or to the Junk folder. Recently, but I don't know when it started exactly, this no longer works. I get the following error message in mail.log: dovecot: imap: Error: Use of uninitialized value $ENV{"PATH"} in split at /usr/lib/x86_64-linux-gnu/perl-base/File/Spec/Unix.pm line 136. Error: program exec:/usr/lib/dovecot/sieve-pipe/sa-learn-spam.sh (2526990): Forcibly terminated with signal 15 Error: sieve: failed to pipe message to program `sa-learn-spam.sh': refer to server log for more information. [2024-10-16 10:16:51] Error: sieve: Execution of script /usr/lib/dovecot/sieve-pipe/report-spam.sieve failed The content of report-spam.sieve is require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"]; if environment :matches "imap.user" "*" { set "username" "${1}"; } pipe :copy "sa-learn-spam.sh" [ "${username}" ]; So my guess is that one of the require on the first line triggers this Unix.pm, and it fails because somehow the PATH variable is not set in this context. However, I am at a complete loss to determine how to change the environment variable in the context of dovecot-sieve. And also, I am not sure why this would be required. Does somebody know how to fix this problem, or alternatively point me to the relevant part in the documentation? Thanks, Julien

2 2

log: msgid= followed by space
by A. Schulze 18 Oct '24

18 Oct '24

Hello, I noted today unexpected spaces after 'msgid=' in the logs, dovecot-2.3.21.1 produce. First time a message from outlook was under inspection but I found also other sources. there was one (unanswered) question in the list archive, that may be related: https://dovecot.org/mailman3/archives/list/dovecot@dovecot.org/thread/JU3S2… here are some samples: lmtp(user@example)<22995><RtRIK/YvEWfTWQAAf9UaAw>: Info: save: box=INBOX/ietf/folder1/, uid=14143, msgid= <172917948321.1404061.2617721779338535905@dt-datatracker-78dc5ccf94-w8wgc>, size=7904 lmtp(user2(a)example.org)<23156><0cjVE809EWd0WgAAf9UaAw>: Info: save: box=INBOX/foo, uid=2632, msgid= <AM0PR06MB4100D06FAD74B7B33BFFDAA395472(a)AM0PR06MB4100.eurprd06.prod.outlook...., size=16755 But there are also "good" logs without the space for messages to other folders but same source (IETF in this case) lmtp(user@example)<23143><bhcjK288EWdnWgAAf9UaAw>: Info: save: box=INBOX/ietf/folder2/, uid=567, msgid=<62424081-d68e-4334-b9a2-a62d0759cea3(a)gulbrandsen.priv.no>, size=11394 I've no older logs to check if that happened also before updating to 2.3.21.1 Andreas

2 2

replicator: Panic: data stack: Out of memory when allocating 275GB
by Sergej Pupykin 15 Oct '24

15 Oct '24

Hello, I found similar reports in this ML with values less than mine but was not able to find the solution. I checked size of all dovecot.* files in /var/mail and it is less than 10Gb. Whole /var/mail dir is 1.8Tb. With vsz_limit it crashes much earlier, this log with 275GB produced by setting vsz_limit = 0 Server has 192GB of RAM, can it be some bug? Oct 14 15:51:59 mailserver dovecot[826177]: replicator: Panic: data stack: Out of memory when allocating 274877906984 bytes Oct 14 15:51:59 mailserver dovecot[826177]: replicator: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x48) [0x76ac1505e188] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x25) [0x76ac1505e305] -> /usr/lib/dovecot/libdovecot.so.0(+0x106312) [0x76ac1506c312] -> /usr/lib/dovecot/libdovecot.so.0(+0x1063b5) [0x76ac1506c3b5] -> /usr/lib/dovecot/libdovecot.so.0(+0x4c571) [0x76ac14fb2571] -> /usr/lib/dovecot/libdovecot.so.0(+0x4bbc4) [0x76ac14fb1bc4] -> /usr/lib/dovecot/libdovecot.so.0(+0xfee30) [0x76ac15064e30] -> /usr/lib/dovecot/libdovecot.so.0(+0x1288c1) [0x76ac1508e8c1] -> /usr/lib/dovecot/libdovecot.so.0(+0xfa870) [0x76ac15060870] -> /usr/lib/dovecot/libdovecot.so.0(+0xfa8f1) [0x76ac150608f1] -> /usr/lib/dovecot/libdovecot.so.0(buffer_append+0x60) [0x76ac15060c00] -> dovecot/replicator(replicator_queue_push+0x163) [0x58a11fac0a53] -> dovecot/replicator(+0x4e9d) [0x58a11fabfe9d] -> dovecot/replicator(+0x4ed8) [0x58a11fabfed8] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0x15f) [0x76ac1508381f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd6) [0x76ac15085296] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x58) [0x76ac15083ae8] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x49) [0x76ac15083ce9] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x20) [0x76ac14fed6a0] -> dovecot/replicator(main+0x1a5) [0x58a11fabe865] -> /usr/lib/libc.so.6(+0x25e08) [0x76ac14d90e08] -> /usr/lib/libc.so.6(__libc_start_main+0x8c) [0x76ac14d90ecc] -> dovecot/replicator(_start+0x25) [0x58a11fabe925]

3 3

How to deactivate "!include conf.d/auth-system.conf.ext" without touching "/etc/dovecot/conf.d/10-auth.conf"?
by postfix_dovecot＠gmx.de 14 Oct '24

14 Oct '24

Hello, I want to deactivate the auth-system.conf.ext because it slows down the system and makes no sense because I utilize passwdfile.conf.ext. I have put all my configurations into the local.conf file, and all the original configuration files remain untouched. I want to stay on this path because that way I know where I made changes, and updates won't cause me any hassle. But how can I deactivate an option that was activated in the original configuration files?" Jens

3 8

Error working with IMAP . Error code: 9 Server response: Logged In
by bdn＠hettich.ru 14 Oct '24

14 Oct '24

my program sends messages via Dovecot + Postfix . Sometimes it can try to send more than 300 messages at one time. The result is a message: Error working with IMAP . Error code: 9 Server response: Logged In What could it be? a limitation? what? and I don't see any specifics in the logs

2 2

shared mailbox error
by Marc 14 Oct '24

14 Oct '24

I am getting this error. This file does not exist. Error: t_readlink(/xxxxxx/dbox-alt-root) failed: readlink() failed: Permission denied

1 1

LMTP error reading configuration
by DaLynX 12 Oct '24

12 Oct '24

Hello, I am getting the following error on my dovecot LMTP server, upon delivery from my postfix mailserver: Oct 6 03:31:09 m dovecot[36551]: lmtp: Fatal: Error reading configuration: read(/var/run/dovecot/config) failed: read(size=8192) failed: Interrupted system call - Also failed to read config by executing doveconf: /var/run/dovecot/config is a UNIX socket (path is from CONFIG_FILE environment) This results in mail being deferred by postfix (different host), and successfully delivered at the next try: Oct 6 03:31:10 mailer postfix/lmtp[44835]: 46F0BA329: to=<d(a)l.ynx.fr>, orig_to=<root>, relay=192.168.1.101[192.168.1.101]:24, delay=17, delays=0.2/0.01/16/0, dsn=4.4.2, status=deferred (lost connection with 192.168.1.101[192.168.1.101] while receiving the initial server greeting) Any idea where it comes from? I am running dovecot 2.3.21.1 (d492236fa0) on FreeBSD 14.1-RELEASE-p5. Here is my configuration: # 2.3.21.1 (d492236fa0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21.1 (49005e73) # OS: FreeBSD 14.1-RELEASE-p3 amd64 # Hostname: m.ynx.fr mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = create special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } protocols = imap lmtp sieve service auth { inet_listener { port = 144 } } service lmtp { inet_listener lmtp { port = 24 } user = vmail } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = no userdb { args = uid=vmail gid=vmail home=/home/vmail/%u driver = static } protocol lmtp { mail_plugins = " sieve" } protocol imap { mail_max_userip_connections = 100 } Kind regards, D

3 2

last_login plugin does not update IP
by Maciej Polewczyński 11 Oct '24

11 Oct '24

When using the last_login plugin I noticed that the IP address information is not updated. If there is no entry in the sql database for a given user, the first time the login date and time information and the IP address will be saved. Each subsequent login only updates the date and time. The IP address remains unchanged. Example configuration for pop3 service: map { pattern = shared/last-login/pop3/$user/$rip table = last_login value_field = last_pop3_access value_type = uint fields { userid = $user last_pop3_ip = $rip } } This is translated into the following sql command: INSERT INTO last_login (last_pop3_access,userid,last_pop3_ip) VALUES (1727691183673,'test','1.2.3.4') ON DUPLICATE KEY UPDATE last_pop3_access=1727691183673 You can see that after "ON DUPLICATE KEY" only the time is updated. Is there any way to influence the behavior of the plugin and the sql query so that the IP address is also updated? I'm using Dovecot v2.3.4 -- Best regards, Maciej Polewczyński Do or do not. There is no try.

3 3