January 2020 - dovecot

Getting an extprogram script to work
by Eric Abrahamsen 21 Jan '20

21 Jan '20

Hi all, I'm trying to do the same thing everyone else is: having rspamd learn_ham or learn_spam depending on where I move a message. My current problem is that Dovecot seems to be reporting success, but nothing ever happens -- the script somehow fails to run. I'm currently using a dummy script that just writes a file to /tmp, until I can get this much working, at least. My bin_dir: --------------------- me@server:/usr/lib/dovecot/sieve# ls -l total 40 drwxr-xr-x 2 root root 4096 Jan 21 03:35 ./ drwxr-xr-x 4 root root 12288 Aug 29 06:28 ../ -rw-r--r-- 1 root root 112 Jan 21 02:17 report-ham.sieve -rw-r--r-- 1 root root 250 Jan 21 02:17 report-ham.svbin -rw-r--r-- 1 root root 112 Jun 3 2019 report-spam.sieve -rw-r--r-- 1 root root 258 Jun 3 2019 report-spam.svbin -rwxr-xr-x 1 root root 105 Jan 21 03:35 rspamd_ham.sh* -rwxr-xr-x 1 root root 43 Jun 3 2019 rspamd_spam.sh* --------------------- Then the report-ham.sieve script: --------------------- require ["vnd.dovecot.pipe", "copy", "imapsieve", "fileinto"]; pipe :copy "rspamd_ham.sh"; #fileinto "INBOX"; --------------------- And the dummy rspamd_ham.sh script: --------------------- #!/bin/bash #exec /usr/bin/rspamc learn_ham tim=`date +%M-%S` echo "${1}" > "/tmp/${tim}_results.txt" --------------------- I can confirm the above script works correctly when run from the command line, both as root and the vmail user. When run as part of the sieve process, no errors are reported, but no file is created under /tmp. From the logs: --------------------- Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.4.21 (92477967) loaded Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.4.21 (92477967) loaded ... Debug: learn/ham: Mailbox opened because: SELECT Debug: imapsieve: mailbox learn/ham: APPEND event Debug: imapsieve: Matched static mailbox rule [1] Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/report-ham.sieve Debug: sieve: file script: Opened script `report-ham' from `/usr/lib/dovecot/sieve/report-ham.sieve' Debug: sieve: Opening script 1 of 1 from `/usr/lib/dovecot/sieve/report-ham.sieve' Debug: sieve: Loading script /usr/lib/dovecot/sieve/report-ham.sieve Debug: sieve: Script binary /usr/lib/dovecot/sieve/report-ham.svbin successfully loaded Debug: sieve: binary save: not saving binary /usr/lib/dovecot/sieve/report-ham.svbin, because it is already stored Debug: sieve: Executing script from `/usr/lib/dovecot/sieve/report-ham.svbin' Debug: sieve: action pipe: running program: rspamd_ham.sh Debug: Mailbox learn/ham: Opened mail UID=249 because: mail stream Debug: waiting for program `/usr/lib/dovecot/sieve/rspamd_ham.sh' to finish after 0 msecs sieve: pipe action: piped message to program `rspamd_ham.sh' Debug: learn/ham: Mailbox opened because: lib-lda delivery sieve: left message in mailbox 'learn/ham' --------------------- Near as I can tell, the first time this runs after a dovecot restart, I get this error instead: Error: write(program stdin) failed: Broken pipe Error: sieve: pipe action: failed to program `rspamd_ham.sh': refer to server log for more information. But that only happens once, then it behaves as above. The "Opened mail UID=XXX" bit changes UID (increments monotonically?) every time. I think that ought to be everything but my dovecot -n. Thanks in advance for any help. # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 4.15.0-1057-aws x86_64 Ubuntu 18.04.3 LTS ext4 auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@+ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes mail_debug = yes mail_home = /var/mail/vmail/%d/%n mail_location = maildir:/var/mail/vmail/%d/%n/mail:LAYOUT=fs mail_plugins = " quota" mail_prefetch_count = 20 mail_privileged_group = mail mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=ssha512 username_format=%u /etc/dovecot/passwd.db driver = passwd-file } plugin { imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox1_causes = APPEND imapsieve_mailbox1_name = learn/ham quota_rule = *:storage=5G quota_rule2 = Trash:ignore quota_rule3 = Junk:ignore quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO recipient_delimiter = + sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /var/mail/vmail/sieve-after sieve_before = /var/mail/vmail/sieve-before sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.debug sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } protocols = " imap lmtp sieve" quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/dovecot-auth { mode = 0666 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0666 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } ssl = required ssl_cert = </etc/letsencrypt/live/cert-deleted ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL:!ADH:!eNULL:RC4+RSA:+HIGH:+MEDIUM ssl_client_ca_dir = /etc/ssl/certs ssl_key = # hidden, use -P to show it userdb { args = username_format=%u /etc/dovecot/passwd.db default_fields = uid=vmail gid=vmail home=/var/mail/vmail/%d/%n driver = passwd-file } protocol lmtp { mail_plugins = " quota sieve" postmaster_address = postmaster(a)ericabrahamsen.net } protocol lda { deliver_log_format = msgid=%m: %$ } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = " quota imap_quota imap_sieve" }

1 0

solr fts and removing accounts
by Wojciech Puchar 20 Jan '20

20 Jan '20

i use solr fts indexing. It worls very well but it have one database per system, not per user. Lets suppose i delete one or more e-mail users in system. How to remove them in solr database to reclaim space?

3 2

Authentication per Service
by Oliver Welter 18 Jan '20

18 Jan '20

Hi All, I use dovecot (v2.3.4) with a SQL backend for user authentication, passwords are stored in CRAMMD5 format. My SMTP server uses "doveadm auth" via auth-socket to perform sender authentification. To shut down SMTP access for hacked passwords I want to have a dedicated flag in the user database to allow/disallow SMTP for a user withouth disabling his account for IMAP. So, is there any option to use a separate passdb definition or additional parameters by either setting this in socket definition of "service auth" or using "-x auth_info"? TIA Oliver -- Protect your environment - close windows and adopt a penguin!

3 3

Sieve puts Reference:-Headers into Vacation-Reply
by Christian K 17 Jan '20

17 Jan '20

Hey, I have an issue with misformatted vacation emails. Part of the initial mail ends up in the vacation response and the headers of the vacation response end up in the body. Which also leads to the wrong charset being used by (some) MUAs. (Should be utf-8 but the inital mal was ascii only). I think I narrowed the trigger of the bug down to a long Reference: header line. It only happens if the long line is not the first one. In the attached example I sent to the mangled mailbox mymailbox(a)example.com. The sieve rule triggered is this one ("äöü" are only there to demonstrate the breaking charset): if true { vacation :days 7 :addresses ["mymailbox(a)example.com"] text: This is an automated reply; I am away and will not be able to reply to you immediately.I will get back to you as soon as I return.äöüß . ; } Christian -- $ dovecot --version 2.2.13 $ dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-9-amd64 x86_64 Debian 8.11 disable_plaintext_auth = no dotlock_use_excl = no mail_debug = yes mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = acl lazy_expunge listescape mail_log notify managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mmap_disable = yes namespace expunged { hidden = yes list = no location = maildir:~/expunged prefix = EXPUNGED. separator = / } namespace inbox { inbox = yes location = maildir:~/maildir prefix = separator = / } passdb { args = username_format=%n /etc/passwd driver = passwd-file } plugin { acl = vfile lazy_expunge = EXPUNGED. mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size recipient_delimiter = + sieve = ~/sieve/dovecot.sieve sieve_dir = ~/sieve/ sieve_extensions = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify sieve_max_redirects = 20 } protocols = imap pop3 sieve lmtp service imap-login { client_limit = 40 process_limit = 50 service_count = 0 vsz_limit = 512 M } service imap { process_limit = 2000 vsz_limit = 512 M } service lmtp { process_limit = 200 vsz_limit = 1 G } service managesieve-login { inet_listener sieve { port = 2000 } vsz_limit = 512 M } service pop3-login { client_limit = 40 process_limit = 50 service_count = 0 vsz_limit = 512 M } shutdown_clients = no ssl_cert = </etc/ssl/certs/cert.crt ssl_key = </etc/ssl/private/key.key userdb { args = username_format=%n /etc/passwd driver = passwd-file } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 150 mail_plugins = acl lazy_expunge listescape mail_log notify imap_acl } protocol lmtp { lmtp_save_to_detail_mailbox = yes mail_plugins = acl lazy_expunge listescape mail_log notify sieve }

1 0

INDEX= and dovecot.index.log / dovecot.list.index.log
by Arkadiusz Miśkiewicz 16 Jan '20

16 Jan '20

Using 2.3.8 and mail_location = maildir:/var/mail/%Ln:CONTROL=/var/lib/dovecot/control/%Ln:VOLATILEDIR=/var/lib/dovecot/control/%Ln:INDEX=/var/lib/dovecot/control/%Ln + also user_query = SELECT ... CONCAT('maildir:/var/mail/', LOWER(u.login), '/:CONTROL=/var/lib/dovecot/control/%Ln:VOLATILEDIR=/var/lib/dovecot/control/%Ln:INDEX=/var/lib/dovecot/control/%Ln') AS mail, ... but I noticed that dovecot creates dovecot.index.log dovecot.list.index.log still in /var/mail/XYZ instad of /var/lib/dovecot/control/XYZ. Is that normal? I would expect /var/lib/dovecot/control/XYZ to be used also for these two files. ps. lmtp is used here -- Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

1 1

dovecot cannot drop privileges inside singularity container
by cesco 15 Jan '20

15 Jan '20

Hi all I'm facing an issue while running dovecot inside a singularity (https://sylabs.io/singularity/) container dovecot version is 2.3.4.1 (configuration below) running on debian buster, inside a container made with singularity version 3.4.2 unfortunately, when I try to start dovecot, it gives: Singularity test.sif:~> cat /var/log/mail.log Dec 30 17:23:38 testnode dovecot: master: Dovecot v2.3.4.1 (f79e8e7e4) starting up for imap, lmtp, sieve, pop3, submission (core dumps disabled) Dec 30 17:23:38 testnode dovecot: anvil: Fatal: We couldn't drop root privileges Dec 30 17:23:38 testnode dovecot: master: Error: service(anvil): command startup failed, throttling for 2 secs the same happens on singularity containers based on debian bullseye or alpine linux 3.9.2 many thanks! nzasch Singularity test.sif:~> doveconf -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-6-amd64 x86_64 Debian 10.2 # Hostname: testnode.example.net mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = " imap lmtp sieve pop3 submission" ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd }

2 2

Upgrade 2.2.27 to 2.3.9.2: master(imap): net_connect_unix(imap) failed: Resource temporarily unavailable
by Eirik Rye 15 Jan '20

15 Jan '20

Hi, After upgrading Dovecot from version 2.2.27 to 2.3.9.2, we are sporadically seeing lots of these errors in the error log on many of our servers: imap-login: Error: master(imap): net_connect_unix(imap) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable The issue is causing significant delays and/or timeouts on login. From what I can tell, this happens because the imap-login service is unable to connect to the imap service socket, however I am unable to identify the root cause of the issue. We have increased the client_limit for the stats service, as described in the upgrade document at (https://wiki2.dovecot.org/Upgrading/2.3) It is set to the same value as the process_limit for imap service. System CPU usage does not appear to be saturated. The CPU usage sits at around 50% when these errors appear. We have doubled the number of file descriptors after upgrading from 2.2.27. It was previously set to 16392 (which worked fine): ~# cat /proc/`pidof dovecot`/limits | grep "open files" Max open files 30000 30000 files We have tried increasing default_process_limit and default_client_limit from the default 1000 to 3000, but this has no effect. Current configuration (with irrelevant parts removed): ~# doveconf -n default_client_limit = 3000 # these were raised after upgrading in attempt to remedy default_process_limit = 3000 default_vsz_limit = 512 M [...] service imap-login { client_limit = 1250 inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 16 process_min_avail = 16 service_count = 0 vsz_limit = 512 M } service imap { client_limit = 1 process_limit = 20000 } service stats { client_limit = 20000 } [...] We appear to be nowhere near the 20000 process_limit set for the imap service. We are also not seeing any warnings being logged with regards to the process_limit. ~# doveadm who -1 | wc -l 8765 ~# doveadm process status | grep "^imap " | wc -l 7583 ~# ps aux | grep "dovecot/imap" | wc -l 7449 ~# doveadm process status | grep -v "^imap " name pid available_count total_count idle_start last_status_update last_kill_sent stats 19875 12384 582387 0 1578305489 0 pop3-login 19868 1000 0 0 1578061504 0 pop3-login 19867 1000 0 0 1578061503 0 pop3-login 19866 1000 0 0 1578061502 0 pop3-login 19865 1000 0 0 1578061503 0 pop3-login 19864 1000 0 0 1578061503 0 pop3-login 19853 1000 0 0 1578061504 0 pop3-login 19852 1000 0 0 1578061504 0 pop3-login 19851 1000 0 0 1578061504 0 pop3-login 19849 1000 0 0 1578061503 0 pop3-login 19847 1000 0 0 1578061503 0 pop3-login 19846 1000 0 0 1578061503 0 pop3-login 19845 1000 0 0 1578061503 0 pop3-login 19844 1000 0 0 1578061503 0 pop3-login 19843 1000 0 0 1578061503 0 pop3-login 19842 1000 0 0 1578061503 0 pop3-login 19839 1000 0 0 1578061502 0 log 19841 2973 27 0 1578061502 0 imap-login 19873 666 14942 0 1578305488 0 imap-login 19872 523 32792 0 1578305489 0 imap-login 19871 316 62876 0 1578305489 0 imap-login 19870 167 78332 0 1578305489 0 imap-login 19869 118 97989 0 1578305489 0 imap-login 19863 0 184726 0 1578305489 0 imap-login 19862 0 193094 0 1578305489 0 imap-login 19861 0 186800 0 1578305487 0 imap-login 19860 675 17806 0 1578305487 0 imap-login 19859 0 169446 0 1578305489 0 imap-login 19858 0 143517 0 1578305489 0 imap-login 19857 0 119215 0 1578305488 0 imap-login 19856 0 151958 0 1578305489 0 imap-login 19855 483 47036 0 1578305488 0 imap-login 19854 1 185240 0 1578305489 0 imap-login 19840 567 23859 0 1578305485 0 config 19874 2967 124482 0 1578305489 0 auth 19885 2967 124185 0 1578305489 0 anvil 10017 967 7567 0 1578305463 0 -- Eirik

3 13

autoupdate broke
by Steve Egbert 14 Jan '20

14 Jan '20

When attempting to perform autoupdate after my 'autogen.sh' broke, it says that there is missing a file called: dovecot/core/doc/wiki/Makefile.am But in the Github repository, there is a misnamed file named "Makefile.am.in". I think this file is misnamed and should be renamed to "Makefile.am". https://github.com/dovecot/core/tree/master/doc/wiki

3 2

Separate TLS certificate for listener
by Peter Mogensen 13 Jan '20

13 Jan '20

Hi, I've read the howto on overriding the global ssl_cert config by using filters like "local", "remote", "protocol", "local-name". However... is it possible to have a speciel certificate just for a listener where only the port is different? /Peter

2 1

Dovecot HA/Resilience
by Kishore Potnuru 13 Jan '20

13 Jan '20

Hi Dovecot team, I need help in configuring HA/resilence for my dovecot (POP3/IMAP) server. I have one RHEL Linux POP3/IMAP server with Dovecot version 2.2.10. I also installed the postfix(version: 2.6.6) in it. At present it have around 10 domains and each domain is having around 20 email boxes. It has very simple configuration. User's reading the email via some internal applications/Outlook. I would like to create resilience/HA for this environment. Could you please suggest me, what is the easiest way to achieve this? If any one can suggest step by step that will be more helpful to me. Thanks, Kishore

8 16