On 2024-06-14 14:47, George Asenov via dovecot wrote:
Christian,
Could you please post your configuration + sieve script for reporting to rspamd?
I was thinking that hitting this limit is consequence of millions of hard links and not the reason!
But I will try that. Shure, one after the other:
# doveconf -n # 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21 (f6cd4b8e) # OS: Linux 5.19.1-sec-svcs.kvm x86_64 Debian 11.9 ext4 # Hostname: uschi.sec-svcs.eu auth_verbose = yes default_vsz_limit = 1 G deliver_log_format = msgid=%m, from=%f, envelope from=%{from_envelope}, envelope to=%{to_envelope}, delivery time=%{delivery_time}ms, lmtp session time=%{session_time}ms, status=%$ dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 5000 first_valid_uid = 5000 imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags last_valid_gid = 5000 last_valid_uid = 5000 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k session=<%{session}> mail_access_groups = vmail mail_gid = vmail mail_location = maildir:/srv/mail/%u/Maildir:INDEXPVT=%h/shared/%%u:LAYOUT=fs mail_plugins = fts fts_solr virtual acl quota quota_clone notify mail_server_admin = mailto:postmaster@sec-svcs.eu mail_server_comment = sec-svcs.eu mail hosting mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Archives { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } mailbox virtual/All { special_use = \All } prefix = separator = / subscriptions = yes type = private } namespace others { list = children location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u:LAYOUT=fs prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace virtual { hidden = no inbox = no list = children location = virtual:/srv/mail/%u/virtual prefix = virtual/ separator = / subscriptions = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl fts = solr fts_autoindex = yes fts_enforced = yes fts_solr = url=http://10.1.1.50:8983/solr/dovecot/ soft_commit=no imapsieve_mailbox1_before = file:/srv/sieve/imapsieve/setflag-seen.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Trash imapsieve_mailbox2_before = file:/srv/sieve/imapsieve/report-spam.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_name = Spam imapsieve_mailbox3_before = file:/srv/sieve/imapsieve/report-ham.sieve imapsieve_mailbox3_causes = COPY imapsieve_mailbox3_from = Spam imapsieve_mailbox3_name = * mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename append mail_log_fields = uid box msgid size from vsize flags quota = count:User quota quota_clone_dict = proxy::quota quota_grace = 10%% quota_rule = *:storage=20G quota_rule2 = Trash:storage=+1000M quota_rule3 = Sent:storage=+1000M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO quota_vsizes = yes quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:/srv/sieve/%u/;active=/srv/sieve/%u/.dovecot.sieve sieve_before = /srv/sieve/before sieve_extensions = +editheader sieve_global_extensions = +vnd.dovecot.pipe sieve_pipe_bin_dir = /srv/sieve/imapsieve sieve_plugins = sieve_imapsieve sieve_extprograms } postmaster_address = postmaster@sec-svcs.eu protocols = " imap lmtp sieve pop3" service anvil { unix_listener anvil { mode = 0666 } } service auth-worker { user = doveauth } service auth { inet_listener { address = 10.1.1.10 port = 12987 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = doveauth mode = 0666 user = doveauth } } service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 4 service_count = 0 } service imap { vsz_limit = 1 G } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 11134 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } service stats { unix_listener stats-reader { mode = 0666 } unix_listener stats-writer { mode = 0666 } } ssl = required ssl_alt_cert =
Content of "report-ham.sieve": require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.mailbox" "*" { set "mailbox" "${1}"; }
if string "${mailbox}" "Trash" { stop; }
if environment :matches "imap.user" "*" { set "username" "${1}"; } #pipe :copy "learn-ham.sh" [ "${email}" ]; #pipe :copy "learn-ham.sh" [ "${username}" ]; pipe :copy "learn-ham.sh";
Content of "learn-ham.sh": #!/bin/sh exec /usr/bin/rspamc -h localhost:11334 learn_ham
content of "report-spam.sieve": require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.email" "*" { set "email" "${1}"; } pipe :copy "learn-spam.sh" [ "${email}" ];
Content of "learn-spam.sh": #!/bin/sh exec /usr/bin/rspamc -h localhost:11334 learn_spam
On 14-Jun-24 11:50 AM, Christian Kivalo via dovecot wrote:
What is the size of the system memory on your dovecot server? And what is the size of the dovecot.index.cache file when you see this error? You should be able to delete the index and recreate it using doveadm.
Servers are with enough RAM the smallest one is 64GB. The memory I'm referring is "vsz_limit=256 MB" and in my opinion it is not the issue but a consequence of the large amount of links in the mail and from there large index file and the result not enough memory to work with. Try increasing the vsz_limit to 512m or even 1g. This can very well be (part of) the problem. I have set default_vsz_limit = 1g and don't run into problems with imap_sieve calling the rspamd script to learn ham/spam.
-- Christian Kivalo