April 2021 - dovecot - dovecot.org

Replication between three servers
by Miro Igov 20 Apr '21

20 Apr '21

Hello, I have dovecot replication working fine between 2 servers with LDA configured to sync mail coming from Postfix. These 2 servers are used in master/slave mode - only the one server is used by IMAP clients and the second server is just a backup in case of failure. Second server is also MX backup and it happens that incoming mails come from it via Postfix and get synchronized with the first server. I would like to include third server as backup. How could be done while mail_replica accepts only one host?

2 4

What is the proper <luceneMatchVersion> value in solrconfig.xml for dovecot?
by Steve Dondley 19 Apr '21

19 Apr '21

I'm looking at config documentation for solr on dovecot: https://doc.dovecot.org/configuration_manual/fts/solr/ In the suggested solrconfig.xml file (https://raw.githubusercontent.com/dovecot/core/master/doc/solr-config-7.7.0…) it has the following line: <luceneMatchVersion>7.7.0</luceneMatchVersion> I'm running solr version 8.8.1, however. I'm wondering if I should change this line to: <luceneMatchVersion>8.8.1</luceneMatchVersion> Things seems to work fine with the 7.7.0 value but there is a comment in the config file that says:  I'm not familiar with Lucene or Solr so I'm uncertain as to what to set this to. Thanks.

1 0

Sending alert to client after quota has reached 100%
by Dan Conway 19 Apr '21

19 Apr '21

Hello all, I hope you are doing well. In regards to the quota_warning configuration, more specifically when an email is sent to the user when quota has passed a certain percentage, i.e.: quota_warning = <limit configuration> <quota-warning socket name> <parameters> #quota_warning2 = ... #quota_warning3 = ..etc.. Is it possible to communicate with the client by sending an ALERT IMAP message to stdout much like post-login scripting, rather than sending an email to the user? i.e: #!/bin/sh # Message goes to stdout printf "* OK [ALERT] This is an alert\r\n" exec "$@" Thank you.

2 1

Only INBOX is searched when using gmail with dovecot FTS and solr
by hi-dovecot＠mcarle.io 16 Apr '21

16 Apr '21

Hi, I am using https://github.com/docker-mailserver/docker-mailserver to setup my mailserver and added solr for full-text search. Nearly everything is working as expected. But I have a problem with the full-text search, as I have a lot of folders in my mailbox and I just can't find any mail in any of these folders except INBOX when I search with the Gmail android app. After analyzing the logs, I assume, that the bug is in the Gmail app, as it is only searching in the INBOX folder. I searched a bit and found some reports of other users confirming that. As this "design decision" (or bug) is quite old, I assume that Google won't fix it. But as many people are using Gmail, I assume that there must be a way to search all folders with Gmail. Sadly, I haven't found any possibility yet. As a workaround one may put a proxy in between and simply remove the "+box:123" filter from the path (e.g. with a small regex). With that, solr will return all found mails from that user, regardless of the folder. This may be a quite simple, but very dirty hack. Hence, I hope that you guys know a better way, a good way to overcome this issue and enable all Gmail users to search for email in all folders. Best regards, Marcel

3 2

mail_max_userip_connections per remote IP not working
by Alessio Cecchi 16 Apr '21

16 Apr '21

Hi, I'm tryng to set a specific mail_max_userip_connections for a remote IP (webmail IMAP software), but it seems not working: remote 1.2.3.4 { protocol imap { mail_max_userip_connections = 100 } } and also this isn't working remote 1.2.3.4 { mail_max_userip_connections = 100 } I insert it at the end of 20-imap.conf file. Is something wrong or is not supported? I'm running dovecot 2.3.14. Thanks -- Alessio Cecchi Postmaster @http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 1

altmove reverse doesn't work
by Zdeněk Zámečník 14 Apr '21

14 Apr '21

I am having troubles with moving emails from alternative storage to primary storage. If I call command like: doveadm altmove -r -u myuser(a)mydomain.yyy ALL it is doing something (it took almost a minute at first time) but there are still emails in alternative storage. There is not reported any error. Below is my config: # Pigeonhole version 0.5.14 (1b5c82b2) # OS: Linux 5.3.18-3-pve x86_64 Debian 10.9 # Hostname: mail.mydomain.yyy auth_cache_size = 2 M auth_cache_ttl = 5 mins auth_cache_verify_password_with_worker = yes auth_master_user_separator = * auth_mechanisms = plain login dict { acl = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imap_client_workarounds = delay-newmail imap_hibernate_timeout = 5 secs imap_id_log = * imap_id_retain = yes login_trusted_networks = 127.0.0.0/8 mail_gid = 2000 mail_home = /var/vmail/%d/%n mail_location = mdbox:~/mdbox:ALT=/var/vmail-archive/%d/%n/mdbox mail_max_userip_connections = 60 mail_plugins = acl zlib fts quota mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader imapsieve vnd.dovecot.imapsieve mdbox_rotate_interval = 1 days mdbox_rotate_size = 16 M namespace { inbox = yes location = mailbox "Deleted Items" { auto = no special_use = \Trash } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox "Junk E-mail" { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { special_use = \Trash } mailbox virtual/All { auto = no special_use = \All } prefix = separator = / type = private } namespace { inbox = no list = children location = mdbox:/var/vmail/%%d/%%n/mdbox:ALT=/var/vmail-archive/%%d/%%n/mdbox prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /etc/dovecot/masters.db driver = passwd-file master = yes pass = yes } plugin { acl = vfile acl_shared_dict = proxy::acl imapsieve_mailbox1_before = file:/var/vmail/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/vmail/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * imapsieve_mailbox3_before = file:/var/vmail/sieve/report-spam.sieve imapsieve_mailbox3_causes = COPY imapsieve_mailbox3_name = Junk imapsieve_mailbox4_before = file:/var/vmail/sieve/report-ham.sieve imapsieve_mailbox4_causes = COPY imapsieve_mailbox4_from = Junk imapsieve_mailbox4_name = * mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_new3 = Deleted Items mailbox_alias_old = Sent mailbox_alias_old2 = Sent mailbox_alias_old3 = Trash quota = dict:User quota::proxy::quota quota_grace = 10%% quota_rule2 = Trash:ignore quota_rule3 = Junk:ignore quota_warning = storage=80%% quota-warning 90 %u quota_warning2 = storage=85%% quota-warning 95 %u quota_warning3 = storage=95%% quota-warning 105 %u sieve = /var/vmail/%d/%n/sieve/.sieve sieve_after = /var/vmail/%d/%n/sieve/autoreply.sieve sieve_before = /var/vmail/sieve/global.sieve sieve_dir = /var/vmail/%d/%n/sieve sieve_extensions = +editheader +vacation-seconds sieve_global_dir = /var/vmail/sieve/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_max_redirects = 20 sieve_max_script_size = 2M sieve_pipe_bin_dir = /usr/lib/dovecot sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_send_from_recipient = yes } protocols = imap sieve lmtp pop3 service auth-worker { unix_listener auth-worker { user = vmail } user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = vmail } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-hibernate { unix_listener imap-hibernate { group = $default_internal_group mode = 0660 } } service imap-login { process_min_avail = 10 service_count = 0 vsz_limit = 512 M } service imap { executable = imap process_limit = 3500 unix_listener imap-master { user = $default_internal_user } vsz_limit = 2 G } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 400 } service pop3-login { process_min_avail = 3 service_count = 0 vsz_limit = 320 M } service pop3 { process_limit = 200 vsz_limit = 320 M } service quota-warning { executable = script /etc/dovecot/quota_warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_cert = </etc/ssl/private/dovecot.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it stats_writer_socket_path = userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_compile_errors = 5 } protocol imap { imap_max_line_length = 6 M mail_plugins = quota imap_quota fts mailbox_alias imap_acl acl imap_zlib imap_sieve } protocol lmtp { mail_plugins = quota sieve acl }

3 3

Auth Cache for checkpassword
by Amol Kulkarni 14 Apr '21

14 Apr '21

Hello, I use the checkpassword driver for imap/pop authentication. To improve the performance I would like to enable authentication caching for it. However just adding the auth_cache_size and auth_cache_ttl doesn't work. No entries are cached. If I add a cache directive in the passdb, then the service doesnt start. I was wondering if the authentication caching is supported for the checkpassword driver. Please point me in the right direction. Thanks in advance, Amol

1 0

Tuning pop3-login client_limit
by Martin Olsen 13 Apr '21

13 Apr '21

Good day / evening / morning good dovecot people: Overview: We are seeing the following issue on one of our servers: pop3-login: Error: master(pop3): net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable (client-pid=5418, client-id=1, rip=__REMOVED__, created 534 msecs ago, received 0/4 bytes) We tried tuning the pop3-login -> client_limit parameter. However, as soon as we increased this parameter (and after restarting dovecot), we saw an onslaught of lmtp connection errors in postfix (unable to deliver via lmtp to dovecot). This led us to set the pop3-login -> client_limit back to 1000 again and the lmtp errors disappeared. This leads me to think that we should tune some other parameters as well. Increase overall limits or such. It seems like when increasing the client_limit for pop3-logins that somehow affected the lmtp deliverability. If anyone has any input on this that would be much appreciated. Cheers, Martin -- Version and config: # dovecot --version 2.2.36 (1f10bfa63) # dovecot -n # 2.2.36 (1f10bfa63): # OS: Centos 7.7 # Hostname: auth_mechanisms = auth_verbose = yes base_dir = __REMOVED__ default_client_limit = 2500 disable_plaintext_auth = no doveadm_password = first_valid_uid = 89 last_valid_uid = 89 lmtp_rcpt_check_quota = yes login_greeting = login_trusted_networks = mail_gid = 89 mail_plugins = " notify replication quota" mail_uid = 89 mbox_write_locks = fcntl namespace inbox { inbox = yes location = prefix = } passdb { args = driver = ldap } plugin { mail_replica = tcps: quota = maildir:User quota quota_rule = } protocols = pop3 lmtp replication_full_sync_interval = 3 hours replication_max_conns = 20 service aggregator { fifo_listener replication-notify-fifo { group = postfix mode = 0640 user = postfix } unix_listener replication-notify { group = postfix mode = 0640 user = postfix } } service auth { unix_listener auth { mode = 0660 } } service doveadm { inet_listener { port = 12345 ssl = yes } } service lmtp { process_min_avail = 10 unix_listener lmtp { group = postfix mode = 0640 user = postfix } } service pop3-login { client_limit = 1000 process_limit = 2048 process_min_avail = 10 service_count = 1 } service pop3 { client_limit = 1 process_limit = 2048 process_min_avail = 0 service_count = 1 }

1 0

json_parse_number broken by compiler optimization
by Christian Ehrhardt 13 Apr '21

13 Apr '21

Hi, the recent Ubuntu (re)builds uncovered an issue with dovecot 1:2.3.13+dfsg1-1 build log: https://launchpadlibrarian.net/529849650/buildlog_ubuntu-hirsute-amd64.dove… A coworker tried 2.3.14 but got the same result. What fails is the json_parser build time test like: test-json-parser.c:161: Assert(#25) failed: null_strcmp(json_output[pos].value, value) == 0 I was looking into that a bit more and what I found is that it is dependent on the new toolchain of gcc 10.2.0-1. Not all calls to json_parse_* fail, e.g. the first one looks all good and passes I was iterating the tests using a report function defined like (gdb) define repcon >c >p pos >p json_output[pos].type >p type >p json_output[pos].value >p value >call null_strcmp(json_output[pos].value, value) >end The first one to be bad was: Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $84 = 25 $85 = JSON_TYPE_NUMBER $86 = JSON_TYPE_NUMBER $87 = 0x555555633b25 "-12.456" $88 = 0x555555693110 "" $89 = 45 Earlier and later parsing was happy, for example Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $90 = 27 $91 = JSON_TYPE_NUMBER $92 = JSON_TYPE_NUMBER $93 = 0x555555633b32 "12.456e9" $94 = 0x555555693110 "12.456e9" $95 = 0 (gdb) We have two things we compare here. 1. json_output[] which is a static define and for this value is 67 »···{ JSON_TYPE_NUMBER, "-12.456" }, 2. the return value that json_parse_next returns. 25 »···" \"sub2\":-12.456,\n" 148 »···»···»···»···ret = json_parse_next(parser, &type, &value); I tried a non negative number and got success which is suspicious Breakpoint 5, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:164 164 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $122 = 25 $123 = JSON_TYPE_NUMBER $124 = JSON_TYPE_NUMBER $125 = 0x555555633b2c "12.456" $126 = 0x555555693110 "12.456" $127 = 0 (gdb) Also the return value otherwise LGTM, it is recognized as a number: 540 } else if ((ret = json_parse_number(parser, value_r)) >= 0) { (gdb) n 541 *type_r = JSON_TYPE_NUMBER; So parsing worked one might think, but the value was not set. (gdb) p *value_r $833 = 0x555555693110 "" It turns out json_parse_number is too optimized. Too optimized for debugging further into json_parse_number. But also optimization is causing the issue. With -O0 or -O1 it works fine, even putting the lower optimization level just around json_parse_number it is enough to avoid the issue. Not a fix but "avoidance" for now: --- a/src/lib/json-parser.c +++ b/src/lib/json-parser.c @@ -386,6 +386,8 @@ static int json_parse_int(struct json_pa return 1; } +#pragma GCC push_options +#pragma GCC optimize ("-O0") static int json_parse_number(struct json_parser *parser, const char **value_r) { int ret; @@ -415,6 +417,7 @@ static int json_parse_number(struct json *value_r = str_c(parser->value); return 1; } +#pragma GCC pop_options static int json_parse_atom(struct json_parser *parser, const char *atom) { I was trying to see what happened and tracked each str_append_c that happened inside json_parse_number. A breakpoint on str_append_c and backtraces showed that we clearly do the "right" parsing. From the backtraces I've seen it does (just from positions): For example (gdb) bt #0 str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 #1 json_parse_digits (parser=<optimized out>) at json-parser.c:366 => that is a digit It does "- digit digit dot digit digit digit" This perfectly matches the input value, but still the result stays empty: (gdb) p (char*)(str->data) $738 = 0x555555693110 "" In comparison - when parsing the latter 12.456e9 which works I see two odd things: 1. the value isn't cleared despite str_truncate(parser->value, 0); at the beginning of json_parse_number 2. the value (this time) is read exactly as I assumed Breakpoint 17, json_parse_number (value_r=0x7fffffffe1a0, parser=0x555555692ee0) at json-parser.c:394 394 str_truncate(parser->value, 0); Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $741 = 0x555555693110 "sub3" ^^ not cleared, still has "sub3" even later on Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $742 = 0x555555693110 "1ub3" ^^ reading left to right one char at a time "over the old value", which didn't update the data. So it seems in json_parse_number the ordering of a) str_truncate(parser->value, 0); b) the parsing itself is messed up by optimization. We already know from the above that reducing just this function to -O1 or less avoids the issue. But what is it really - I don't know? Maybe you have a better idea what is going on, the issue should be reproducible when building dovecot with the toolchain present on the latest Ubuntu being 21.04 (Hirsute) right now. -- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd

4 8

sieve filter bevor.spam, arnt take effect
by Maurizio Caloro 13 Apr '21

13 Apr '21

Hello Please willing to filter "bevor spam" mail... filtering arnt take effect .. please i dont see the mistake. Thanks for any possible update. Debian buster -- root@nmail:/etc/dovecot# dpkg -l | grep dovecot dovecot-core 1:2.3.4.1-5+deb10u6 dovecot-imapd 1:2.3.4.1-5+deb10u6 dovecot-lmtpd 1:2.3.4.1-5+deb10u6 dovecot-managesieved 1:2.3.4.1-5+deb10u6 dovecot-mysql 1:2.3.4.1-5+deb10u6 dovecot-sieve 1:2.3.4.1-5+deb10u6 root@nmail:/etc/dovecot/sieve# cat before.sieve # Sieve rules that are run before the user's filters. # Messages flagged as spam are moved into the Junk folder. require [ "fileinto", "mailbox" ]; if header :contains "X-Spam-Flag" "YES" { fileinto :create "INBOX.Junk"; stop; } -- already executed # sieve before.sieve -- service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0600 user = postfix group = postfix } } -- root@nmail:/etc/dovecot/conf.d# <mailto:root@nmail:/etc/dovecot/conf.d#> cat 90-sieve.conf ## ## Settings for the Sieve interpreter ## plugin { # The path to the user's main active script. If ManageSieve is used, this the # location of the symbolic link controlled by ManageSieve. # sieve = ~/sieve/.dovecot.sieve sieve = file:%h/sieve;active=%h/.dovecot.sieve # Directory for :personal include scripts for the include extension. This # is also where the ManageSieve service stores the user's scripts. sieve_dir = ~/sieve # Path to a script file or a directory containing script files that need to be # executed before the user's script. If the path points to a directory, all # the Sieve scripts contained therein (with the proper .sieve extension) are # executed. The order of execution within a directory is determined by the # file names, using a normal 8bit per-character comparison. Multiple script # file or directory paths can be specified by appending an increasing number. sieve_before = /etc/dovecot/sieve/before.sieve } -- root@nmail:/etc/dovecot# doveconf -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: nmail.caloro.ch auth_mechanisms = plain login auth_verbose = yes info_log_path = /var/log/mail.log log_timestamp = %b %d %H:%M:%S Dovecot/ mail_access_groups = vmail mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/sieve/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = mca(a)caloro.ch <mailto:mca@caloro.ch> protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-client { mode = 0660 user = mail } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service stats { unix_listener stats-reader { group = vmail mode = 0666 user = vmail } unix_listener stats-writer { group = vmail mode = 0666 user = vmail } } ssl = required ssl_cert = </etc/letsencrypt/live/nmail.caloro.ch/fullchain.pem ssl_cipher_list = ECDHE-RSA-CHACHA20-POLY1305:ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = " quota sieve" } protocol imap { mail_plugins = " quota imap_quota" ssl_cert = </etc/letsencrypt/live/nmail.caloro.ch/fullchain.pem ssl_key = # hidden, use -P to show it } protocol lmtp { mail_plugins = " sieve" } Mauri

3 6