June 2022 - dovecot - dovecot.org

Home directories with colons in it
by Jonas Pasche 16 Jun '22

16 Jun '22

Howdy folks, we're using Dovecot 2.3.19 in combination with vmailmgr as the virtual user management backend (please don't judge, there are plans to move away from it for various reasons, but as of today it's there), using the wrapper from https://wiki.dovecot.org/HowTo/VMailMgr - and we're facing an interesting problem with colons in home directories. I think this problem is not specific to vmailmgr but happens in any scenario that includes ":" in directory names. tl;dr: - The checkvpw-dovecot-wrapper from the Dovecot wiki converts ":" in directory names to "::" which works fine with general IMAP use, but at least the autoexpunge plugin and ManageSieve don't handle this escaping and thus break on those users. - When patching the ":"->"::" conversion out of the checkvpw-dovecot-wrapper, Dovecot itself bails on it, interpreting it as a sub-setting. - The latter is quite surprising because a HOME/mail_home directory is supposed to be exactly one directory (AFAIK), with no sub-settings like e.g. mail_location, so I did not expect it to be "interpreted" in any way. Short story long: When you set up a user with a name containing dots, vmailmgr converts them to colons, pretty much like qmail does (qmail maps the username "xy.z" to a file named ".qmail-xy:z"; vmailmgr creates the directory "./users/xy:z" for that username). The wrapper from the dovecot wiki escapes ":" into "::" before exporting the user's directory as the HOME environment variable: /* escape possible ':' characters before finally exporting */ escape_colon(buffer, MAXLEN); setenv("HOME", buffer, 1); The result looks like that: [root@stardust ~]# doveadm auth login xy.z(a)coltest.uber.space supersecret passdb: xy.z(a)coltest.uber.space auth succeeded extra fields: user=xy.z(a)coltest.uber.space userdb extra fields: xy.z(a)coltest.uber.space home=/home/coltest/./users/xy::z <---------- notice the double colon here uid=1009 gid=1010 auth_mech=PLAIN So far, so good: Working with IMAP generally works fine, but there's a catch: Not every module seems prepared for this "::" syntax. The first notable exception is the autoexpunge plugin which simply bails on it: Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: file_create_locked(/home/coltest/./users/xy::z/dovecot.autoexpunge.lock) failed: safe_mkstemp(/home/coltest/./users/xy::z/dovecot.autoexpunge.lock) failed: No such file or directory The second notable exception is ManageSieve which takes the escaped directory name "as is", leading to a situation where the Maildir is in "xy:z" while the Sieve stuff is being put in "xy::z" where it effectively is never picked up during delivery (as the stuff is in the wrong location and the .dovecot.sieve symlink is - silently - not created): [coltest@stardust ~]$ ls -la users/* users/xy:z: total 28 drwx------. 9 coltest coltest 4096 Jun 16 11:39 . drwxr-xr-x. 4 coltest coltest 31 Jun 16 11:40 .. drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 cur -rw-------. 1 coltest coltest 376 Jun 16 11:40 dovecot.index.log -rw-------. 1 coltest coltest 2676 Jun 16 11:39 dovecot.list.index.log -rw-------. 1 coltest coltest 96 Jun 16 11:39 dovecot.mailbox.log -rw-------. 1 coltest coltest 51 Jun 16 11:39 dovecot-uidlist -rw-------. 1 coltest coltest 8 Jun 16 11:39 dovecot-uidvalidity -r--r--r--. 1 coltest coltest 0 Jun 16 11:39 dovecot-uidvalidity.62aafa70 drwx------. 5 coltest coltest 108 Jun 16 11:39 .Drafts drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 new drwx------. 5 coltest coltest 108 Jun 16 11:39 .Sent drwx------. 5 coltest coltest 108 Jun 16 11:39 .Spam -rw-------. 1 coltest coltest 28 Jun 16 11:39 subscriptions drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 tmp drwx------. 5 coltest coltest 108 Jun 16 11:39 .Trash users/xy::z: total 0 drwx------. 3 coltest coltest 19 Jun 16 11:40 . drwxr-xr-x. 4 coltest coltest 31 Jun 16 11:40 .. drwx------. 3 coltest coltest 33 Jun 16 11:40 sieve So my initial thought was: Why is the ":" even escaped to a "::" in the first place? It shouldn't be necessary according to this clean workaround that I found in a posting from 2009 with Timo advising for this type of handling: > Hmm. Although now that I looked at your code, you seem to be using only > HOME. And using ~/ in mail_location should work around any ":" issues. > So what exactly is the problem you're seeing? https://www.dovecot.org/list/dovecot/2009-June/040590.html (In the same thread, Timo _also_ introduced the "::" handling in Dovecot.) This is exactly what we're doing at delivery time: Setting mail_home to the real pathname including a _single_ colon (not an escaped double colon) and then pointing mail_location to "~/": /usr/libexec/dovecot/dovecot-lda -a '$RECIPIENT' -f '$SENDER' -o mail_home='$MAILDIR' -o mail_location='maildir:~/' This works well. So my assumption was: If the checkvpw-dovecot-wrapper returns the HOME directory of a virtual user; where's even the need to escape a single colon to a double colon? So I gave it a try and removed that one line of escaping and recompiled the wrapper. Result: [root@stardust ~]# doveadm auth login xy.z(a)coltest.uber.space supersecret passdb: xy.z(a)coltest.uber.space auth succeeded extra fields: user=xy.z(a)coltest.uber.space userdb extra fields: xy.z(a)coltest.uber.space home=/home/coltest/./users/xy:z <---------- single colon here now uid=1009 gid=1010 auth_mech=PLAIN Using a real IMAP login, though, this lead to the "interesting" situation that the login itself worked correctly, but _then_ immediately an "Internal error occurred": [root@stardust ~]# nc localhost 143 * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SPECIAL-USE LITERAL+ STARTTLS AUTH=PLAIN] Dovecot ready. 0 LOGIN xy.z(a)coltest.uber.space supersecret 0 OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE SAVEDATE SPECIAL-USE LITERAL+ NOTIFY SPECIAL-USE] Logged in * BYE Internal error occurred. Refer to server log for more information. This is the according log: Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37 imap-login: Info: Login: user=<xy.z(a)coltest.uber.space>, method=PLAIN, rip=::1, lip=::1, mpid=22889, secured Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37 imap(xy.z(a)coltest.uber.space)<22889><utbow43hgJoAAAAAAAAAAAAAAAAAAAAB>: Error: Namespace '': Unknown setting: z Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37 imap(xy.z(a)coltest.uber.space)<22889><utbow43hgJoAAAAAAAAAAAAAAAAAAAAB>: Info: Disconnected: Namespace '': Unknown setting: z in=0 out=428 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 So it looks like Dovecot tries to do some interpretation on the HOME value, splitting it at the ":" and interpret the "z" as some kind of sub-setting - which a HOME variable (in contrast to, e.g. mail_location) is not supposed to have. Here I'm a bit stuck. My idea is to work around this by eliminating the ":" from directory names altogether by passing an explicit directory name to vadduser: [coltest@stardust ~]$ echo supersecret | vadduser --directory=./users/ab.c ab.c vadduser: user 'ab.c' successfully added [coltest@stardust ~]$ ls -l users total 0 drwx------. 5 coltest coltest 39 Jun 16 10:26 ab.c drwx------. 5 coltest coltest 39 Jun 16 09:31 xy:z But still, it would be nicer if Dovecot cleanly supported the ":" in directory names, as it seems to _want_ to support it (by introducing the "::" escaping) but then only does it halfway. Could this be fixed in whatever direction, or is there any other advice how we should handle this? All the best, Jonas

1 0

Re: [EXT] Re: The meaning of "tarpit" as used in the auth policy docs?
by Hippo Man 15 Jun '22

15 Jun '22

I'm sorry. Gmail didn't include the list in the "To" header when I responded, and I didn't notice. I fixed it now. -- hippoman(a)gmail.com Take a hippopotamus to lunch today. On Tue, Jun 14, 2022 at 2:40 AM Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > Hi! > > Can you please keep responses on the list? =) > > Aki > > > On 14/06/2022 03:57 Hippo Man <hippoman(a)gmail.com> wrote: > > > > > > Thank you very much for explaining what "tarpit" means. > > > > So, if I return a huge numeric status value from the "before" check, > could I > > make the client wait for hours or even days or weeks before continuing > > with the authentication? > > > > -- > > hippoman(a)gmail.com > > Take a hippopotamus to lunch today. > > > > > > > > On Wed, Jun 8, 2022 at 12:50 AM Aki Tuomi <aki.tuomi(a)open-xchange.com> > wrote: > > > > > > > On 08/06/2022 05:34 Hippo Man <hippoman(a)gmail.com> wrote: > > > > > > > > > > > > I'm trying to understand what is meant by "tarpit" in the > ffollowing quote of the Dovecot auth policy docs for the "before" check ... > > > > First query is done before password and user databases are > consulted. This means that any userdb/passdb attributes are left empty. > > > > The command used here is ‘allow’ and will appear on the URL as > command=allow. > > > > status result values: > > > > * -1: Reject > > > > * 0: Accept > > > > * (Any other positive value): Tarpit for this number of seconds. > > > > What actually happens if a positive status value is returned here? > > > > > > > > Does it mean that the client will not receive any response for the > given number of seconds, but that the given login attempt will still > continue after that delay? Or does it mean that the client will not receive > a response for the given number of seconds, and then the login will be > rejected after that delay? > > > > > > > > What if the client has connected via pop3 and is sending multple > "user" and "pass" combinations? If the "before" check returns, for example, > the integer "5" as the status, does this mean there will be a 5-second > delay between each login attempt that is being sent to this connection? > > > > > > > > Ideally, I'd like the following to occur, but I don't know if it is > possible: > > > > > > > > * Client connects via pop3 and intends to send a large batch of > "user"/"pass" commands via this connection. > > > > > > > > * The first "user"/"pass" command is made to wait for many seconds. > > > > > > > > * Then, the connection is dropped, so that the subsequent > "user"/"pass" commands do not even hit the dovecot server. > > > > > > > > I'm thinking that this is *not* what is meant by "tarpit" in the > auth policy docs, correct? > > > > > > > > Thank you for any clarification. > > > > > > > > -- > > > > hippoman(a)gmail.com > > > > Take a hippopotamus to lunch today. > > > > > > Hi! > > > > > > When client connects, and sends authentication request dovecot > performs a policy lookup before any actual authentication. If this policy > result has positive value, tarpit occurs. > > > > > > This means we simply add a timeout to auth process and leave it > hanging, while handling other requests, and then perform the actual > authentication. The authentication is not failed, however. > > > > > > Aki > > > >

1 0

Re: doveadm-deduplicate deletes non-duplicates
by gravitini 14 Jun '22

14 Jun '22

Replying to: https://dovecot.org/pipermail/dovecot/2022-May/124816.html Hi, Looking at the code (and tested via local build from source) it looks like doveadm deduplicate in 2.3.19 can cause significant data loss. A 2022-02-11 commit removed key duplication resulting in undefined behaviour which is often truncation of a mailbox to 67 entries. (HASH_TABLE_MIN_SIZE) https://github.com/dovecot/core/commit/320844f50cd669b602d30210e2e5216f65d2… diff --git a/src/doveadm/doveadm-mail-deduplicate.c b/src/doveadm/doveadm-mail-deduplicate.c index caec758112..2152482876 100644 --- a/src/doveadm/doveadm-mail-deduplicate.c +++ b/src/doveadm/doveadm-mail-deduplicate.c @@ -63,8 +63,10 @@ cmd_deduplicate_box(struct doveadm_mail_cmd_context *_ctx, if (key != NULL && *key != '\0') { if (hash_table_lookup(hash, key) != NULL) mail_expunge(mail); - else + else { + key = p_strdup(pool, key); hash_table_insert(hash, key, POINTER_CAST(1)); + } } }

4 8

Re: dovecot Digest, Vol 230, Issue 21
by Joseph Tam 10 Jun '22

10 Jun '22

On Thu, 9 Jun 2022, Richard wrote: > Rather than simply upping the limit I think a reasonable question to > ask is why/how they are managing to do that. That's a lot of open > folders. If this mail client behaves anything like Apple mail clients, these connection storms can come about when doing global searches. The mail clients will march through each mailbox (opening a connection for each mailbox) looking for a pattern. If you can narrow this scenario for one specific user (e.g. $user), you can deep dive what's going on by enabling IMAP session logs for this user protocol imap { ... rawlog_dir = /log/dir/%u } then (Make sure this user has write permissions into this directory) mkdir /log/dir/$user After you're done, you can disable logging, rm -rf /log/dir/$user Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: Issue with one user only, exceeding connections
by Richard 09 Jun '22

09 Jun '22

> Date: Thursday, June 09, 2022 11:07:38 -0500 > From: Jeremy Schaeffer <kb9mfd(a)phonesplus.biz> > > On 6/9/2022 10:59, Richard wrote: >> >>> Date: Thursday, June 09, 2022 10:46:25 -0500 >>> From: Jeremy Schaeffer <kb9mfd(a)phonesplus.biz> >>> >>> That was the first thing I tried, I lowered the cache connections >>> in Thunderbird. Actually the max connections was 50, not 500, but >>> I could see why as I do have a lot of folders, but what is odd is >>> I have other mailboxes that have even more folders, but it's only >>> one mailbox that is trowing the error. >>> >>> "# ps -axww | grep imap" does not give me the same results - >>> >>> ..... >>> >>> 19897 ? S 0:00 dovecot/imap >>> 19900 ? S 0:02 dovecot/imap >>> 19901 ? S 0:00 dovecot/imap >>> 19902 ? S 0:00 dovecot/imap >>> ..... >>> >>> I wish it did give me the mailbox, is there a option to get it to >>> give me that information? >> >> Try "auxw" on your "ps". I.e., add in the "u" which will get you >> the user detail in the first column, otherwise you just get the >> process id. >> >> > Thank you! That worked, I piped the output to a file, grep the > username and sure enough there are 60 lines. So I guess going over > 50 was a possibility. > > Learn something new every day. I set the maximum to 100 so I should > not have any errors on that anymore. > Rather than simply upping the limit I think a reasonable question to ask is why/how they are managing to do that. That's a lot of open folders. By the way, the single command: ps auxw | grep imap | cut -d" " -f1 | sort | uniq -c will get you a nice list with the users and their connection counts.

4 4

Issue with one user only, exceeding connections
by Jeremy Schaeffer 09 Jun '22

09 Jun '22

I keep having this issue with one user, and I have to restart dovecot several times a day to clear it. What I have is a postfix / dovecot mail server (Centos 7) and about a dozen users. All mailboxes are imap ssl. I monitor about 4 mailboxes on my computer and tablet. I use Thunderbird on the computer (cache connections at 2) and K9 on the tablet, but one user of the four I keep getting "Maximum number of connections from user+IP exceeded" and I have the maximum at 50 "(mail_max_userip_connections=50)" so its hard for me to believe I am actually exceeding it unless dovecot/client is not dropping connections and keeps starting new ones until it reaches the maximum, but again, only for one user, even though I am monitoring 4 on the same devices. Any idea how to troubleshoot this? I don't know if I should be looking at dovecot or the clients, or what I need to look for. It's been going on since I put this server in use over a year ago. I also have issues with Outlook clients disconnecting, just outlook, is there any recommended settings to make Outlook work smoother? Thanks! - Jeremy Config - # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-1160.11.1.el7.x86_64 x86_64 CentOS Linux release 7.9.2009 (Core) # Hostname: *** auth_mechanisms = plain login debug_log_path = /var/log/dovecot_debug.log first_valid_gid = 500 last_valid_gid = 600 last_valid_uid = 600 listen = * mail_location = maildir:~/Maildir mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . } passdb { driver = pam } pop3_uidl_format = %f protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 1 service_count = 0 } service imap { process_limit = 1024 } service lmtp { unix_listener lmtp { mode = 0666 } } ssl = required ssl_cert = <*** ssl_cipher_list = ECDHE-RSA-CHACHA20-POLY1305:ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = passwd } protocol imap { mail_max_userip_connections = 50 }

4 6

Replicator: Panic: data stack: Out of memory
by Ivan Jurišić 09 Jun '22

09 Jun '22

Reproduce of error (Replicator: Panic: data stack: Out of memory): ================================================================== When replay on any mail I will got OOM. I try add in service replicate vsz_limit = 0 but that not help (after proces dovecot/replicator eat all ram I got OOM) Log error: ========== Jun 03 09:38:59 Warning: imap(ivan(a)myserv.local)<2533034><h7EQN4bgaJ4FvZdE>: replication(ivan(a)myserv.local): Sync failure: Timeout in 2 secs Jun 03 09:39:03 Panic: replicator: data stack: Out of memory when allocating 4294967336 bytes Jun 03 09:39:03 Error: replicator: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7feef08f0582] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7feef08f069e] -> /usr/lib/dovecot/libdovecot.so.0(+0x1022fb) [0x7feef08fd2fb] -> /usr/lib/dovecot/libdovecot.so.0(+0x102391) [0x7feef08fd391] -> /usr/lib/dovecot/libdovecot.so.0(+0x55589) [0x7feef0850589] -> /usr/lib/dovecot/libdovecot.so.0(+0x54d52) [0x7feef084fd52] -> /usr/lib/dovecot/libdovecot.so.0(+0xfb808) [0x7feef08f6808] -> /usr/lib/dovecot/libdovecot.so.0(+0x122938) [0x7feef091d938] -> /usr/lib/dovecot/libdovecot.so.0(+0xf76bf) [0x7feef08f26bf] -> /usr/lib/dovecot/libdovecot.so.0(+0xf777e) [0x7feef08f277e] -> /usr/lib/dovecot/libdovecot.so.0(buffer_append+0x61) [0x7feef08f2a21] -> dovecot/replicator(replicator_queue_push+0x15a) [0x55953ece9b7a] -> dovecot/replicator(+0x62b6) [0x55953ece92b6] -> dovecot/replicator(+0x594e) [0x55953ece894e] -> dovecot/replicator(+0x5b2c) [0x55953ece8b2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7feef0913529] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) [0x7feef0914c12] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) [0x7feef09135d0] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7feef0913790] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7feef0886353] -> dovecot/replicator(main+0x18d) [0x55953ece7cbd] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7feef0651d0a] -> dovecot/replicator(_start+0x2a) [0x55953ece7d7a] Jun 03 09:39:03 Fatal: replicator: master: service(replicator): child 2532886 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) My configuration: ========== # 2.3.19 (b3ad6004dc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 5.10.0-13-amd64 x86_64 Debian 11.3 ext4 # Hostname: mail.myserv.local auth_mechanisms = plain login auth_verbose = yes debug_log_path = /var/log/dovecot.debug default_client_limit = 5000 default_process_limit = 1000 default_vsz_limit = 2 G deliver_log_format = msgid=%m, subject=%s, from=%f, size=%p(%w), %$ dict { quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } doveadm_password = # hidden, use -P to show it hostname = myserv.local lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot.log mail_home = /var/mail/vhosts/%d/mail/%n mail_location = maildir:/var/mail/vhosts/%d/mail/%n:INDEX=/var/mail/vhosts/%d/indexes/%n mail_plugins = " quota fts fts_solr notify replication" mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = Sent autocreate2 = Drafts autocreate3 = Junk autocreate4 = Trash autosubscribe = Sent autosubscribe2 = Drafts autosubscribe3 = Junk autosubscribe4 = Trash expire = Trash expire2 = Trash/* expire3 = Spam fts = solr fts_autoindex = yes fts_solr = url=http://localhost:8983/solr/dovecot/ imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * mail_replica = tcp:imap.myserv2.local:12345 quota = dict:Mailbox::proxy::quota quota_rule = *:storage=1G quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u replication_sync_timeout = 2 sieve = file:/var/mail/vhosts/%d/sieve/%n;active=/var/mail/vhosts/%d/sieve/%n/.dovecot.sieve sieve_after = /etc/dovecot/sieve/after.d/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_quota_max_scripts = 50 } postmaster_address = postmaster@%d protocols = " imap lmtp sieve pop3" service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service dict { unix_listener dict { user = vmail } } service doveadm { inet_listener { port = 12345 ssl = no } } service imap-login { process_min_avail = 4 service_count = 1 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-warning { executable = script /var/lib/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_cert = </etc/letsencrypt/live/myserv.local/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.1 userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " quota fts fts_solr notify replication sieve" } protocol !indexer-worker { mail_vsize_bg_after_count = 100 } protocol lda { mail_plugins = " quota fts fts_solr notify replication sieve" } protocol imap { mail_max_userip_connections = 20 mail_plugins = " quota fts fts_solr notify replication imap_quota imap_sieve" } protocol sieve { mail_max_userip_connections = 20 managesieve_max_line_length = 64 k } protocol pop3 { mail_max_userip_connections = 20 mail_plugins = " quota fts fts_solr notify replication quota" }

4 14

Re: Issue with one user only, exceeding connections
by Richard 08 Jun '22

08 Jun '22

> Date: Wednesday, June 08, 2022 14:14:23 -0500 > From: Jeremy Schaeffer <kb9mfd(a)phonesplus.biz> > > I keep having this issue with one user, and I have to restart > dovecot several times a day to clear it. What I have is a postfix / > dovecot mail server (Centos 7) and about a dozen users. All > mailboxes are imap ssl. I monitor about 4 mailboxes on my computer > and tablet. I use Thunderbird on the computer (cache connections at > 2) and K9 on the tablet, but one user of the four I keep getting > "Maximum number of connections from user+IP exceeded" and I have > the maximum at 50 "(mail_max_userip_connections=50)" so its hard > for me to believe I am actually exceeding it unless dovecot/client > is not dropping connections and keeps starting new ones until it > reaches the maximum, but again, only for one user, even though I am > monitoring 4 on the same devices. Any idea how to troubleshoot > this? I don't know if I should be looking at dovecot or the > clients, or what I need to look for. It's been going on since I put > this server in use over a year ago. I also have issues with Outlook > clients disconnecting, just outlook, is there any recommended > settings to make Outlook work smoother? > > Thanks! - Jeremy > > Config - > ># 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf ># OS: Linux 3.10.0-1160.11.1.el7.x86_64 x86_64 CentOS Linux release ># 7.9.2009 (Core) Hostname: *** > } > protocol imap { > mail_max_userip_connections = 50 > } > On my centos-7 dovecot install (same versions you indicate you are using), that connection-limiting line is in: /etc/dovecot/conf.d/20-imap.conf while you appear to have it in the main dovecot.conf file. If you have that in both places I'm not certain which one takes precedence, but you might want to check. [In my setup almost all the configuration pieces are in include files, rather than the main .conf.] Starting with a "ps" (and grep &etc.) it's fairly easy to see how many connections any account has open. An open mailbox has one or two connections and then every folder (under a mailbox) that is open will have a connection associated with it too, which pushes up the count quickly if someone holds a lot of folders open.

1 0

Occasional service disruptions
by Nikolaos Milas 08 Jun '22

08 Jun '22

Hello, On a server with (Postfix and) Dovecot 2.3.18 (on a VM running CentOS 7 - 1 CPU, 5 GB RAM) with the config you will see below, we are facing occasional (infrequent) service disruptions: IMAP service seems unavailable to some users. Jun 6 12:01:25 vweb2 roundcube: <1eecb0d4> IMAP Error: Login failed for imaptester against vmail2.noa.gr from 195.251.202.xxx. Could not connect to ssl://vmail2.noa.gr:993: Connection rejected in /var/webs/webmail/rcube/program/lib/Roundcube/rcube_imap.php on line 211 (POST /?_task=login&_action=login) At that time there was no associated logged event in dovecot log. (Other users are logging in and out.) However, I see some warnings (I list the two of them closest to the above event): Jun 06 12:01:22 imap(user1)<29639><Vr0atcPg5M3BXBCl>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances ... Jun 06 12:01:26 imap(user2)<29793><rZuSt8PgztoKyVSG>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances (In above log excerpts I've only modified real usernames.) Restarting Dovecot returns things back to normal. I have tried to use "service_count = 100" in all configured services, to see how it goes. Most of the config is inherited from the past (older versions) and is not optimized. For example one can observe different "process_limit" values for different services, for no apparent reason I am aware of. Could anyone suggest changes and/or additions to the OS and/or Dovecot to resolve this issue? Any additional suggestions will also be welcome. Thanks in advance for your kind assistance. Here is the config (I've only changed postmaster address): ======================================================================= protocols = imap pop3 sieve lmtp login_greeting = Dovecot NOA ICXC-NIKA log_path = /var/log/dove.log mail_location = maildir:~/Maildir/ mail_gid = 500 mail_uid = 500 auth_mechanisms = plain login auth_username_format = %Ln auth_verbose = no auth_debug = no mail_debug = no disable_plaintext_auth = no mail_plugins = quota mail_log notify protocol imap { imap_client_workarounds = "delay-newmail" mail_plugins = quota imap_quota mail_log notify mail_max_userip_connections = 400 namespace inbox { mailbox Trash { autoexpunge = 15d } } } protocol pop3 { mail_max_userip_connections = 3 mail_plugins = quota notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv namespace inbox { mailbox Trash { autoexpunge = 15d } } } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota notify sieve postmaster_address = xxxxxxxxx(a)noa.gr sendmail_path = /usr/lib/sendmail } protocol lmtp { auth_socket_path = /var/run/dovecot/auth-master postmaster_address = xxxxxxxxx(a)noa.gr mail_plugins = quota notify sieve sendmail_path = /usr/lib/sendmail } protocol sieve { managesieve_max_line_length = 65536 mail_max_userip_connections = 10 managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 10 } userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = uid box msgid size flags vsize from subject quota = maildir:User quota quota_rule = *:storage=15G quota_rule2 = Trash:storage=+3%% quota_warning = storage=75%% quota-warning 75 %u quota_warning2 = storage=90%% quota-warning 90 %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_max_script_size = 0 sieve_max_actions = 0 sieve_max_redirects = 2 } service quota-warning { executable = script /opt/mail1.sh user = vmail unix_listener quota-warning { user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root } service imap-login { service_count = 100 vsz_limit = 64 M process_limit = 500 } service pop3-login { service_count = 100 vsz_limit = 64 M } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 100 process_min_avail = 0 vsz_limit = 64M } service managesieve { process_limit = 1024 } service imap { executable = imap postlogin process_limit = 2048 } service pop3 { executable = pop3 postlogin } service postlogin { executable = script-login -d rawlog unix_listener postlogin { } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = yes ssl_cert = </etc/pki/tls/certs/star_noa_gr-cert-with_CA-rev-754868755.crt ssl_key = </etc/pki/tls/private/star_noa_gr-1243437.key namespace inbox { separator = . prefix = inbox = yes mailbox Drafts { special_use = \Drafts auto = subscribe } mailbox Junk { special_use = \Junk auto = subscribe } mailbox Trash { special_use = \Trash auto = subscribe } mailbox Sent { special_use = \Sent auto = subscribe } } ======================================================================= Nick

3 4

Distros based on OpenSSL 3 fail on lib-dcrypt/test-crypto
by Sloane Bernstein 08 Jun '22

08 Jun '22

Hello, I am getting our Dovecot packages preliminarily ready to support Linux distributions which rely on OpenSSL 3. I notice that even the main dev branch will build, but the test suite fails (among other places) at test_password_change in src/lib-dcrypt/test-crypto.c: -- [root@al9 lib-dcrypt]# ./test-crypto test_cipher_test_vectors ............................................. : ok test_cipher_aead_test_vectors ........................................ : ok test_hmac_test_vectors ............................................... : ok test_load_v1_keys .................................................... : ok test_load_v1_key ..................................................... : ok test_load_v1_public_key .............................................. : ok test_load_v2_key ..................................................... : ok test_load_v2_public_key .............................................. : ok test_get_info_v2_key ................................................. : ok test_gen_and_get_info_rsa_pem ........................................ : ok test_get_info_rsa_private_key ........................................ : ok test_get_info_invalid_keys ........................................... : ok test_get_info_key_encrypted .......................................... : ok test_get_info_pw_encrypted ........................................... : ok test-crypto.c:827: Assert failed: ret == TRUE Panic: file dcrypt-openssl.c: line 2636 (dcrypt_openssl_private_to_public_key): assertion failed: (priv_key != NULL && pub_key_r != NULL) Error: Raw backtrace: ./test-crypto(backtrace_append+0x42) [0x445332] -> ./test-crypto(backtrace_get+0x1e) [0x44544e] -> ./test-crypto() [0x42414b] -> ./test-crypto() [0x424181] -> ./test-crypto() [0x412b69] -> .libs/libdcrypt_openssl.so(+0x5f25) [0x7fb61954df25] -> ./test-crypto() [0x41cd9a] -> ./test-crypto() [0x4200af] -> ./test-crypto(test_run+0x4c) [0x420c5c] -> ./test-crypto(main+0x4b) [0x41717b] -> /lib64/libc.so.6(+0x44e50) [0x7fb6195a3e50] -> /lib64/libc.so.6(__libc_start_main+0x7c) [0x7fb6195a3efc] -> ./test-crypto(_start+0x25) [0x417295] Aborted (core dumped) -- Looking at how various distros handle this test failure when building packages, they all seem to apply the same patch developed by Red Hat to get this test to pass, attached to https://bugzilla.redhat.com/show_bug.cgi?id=1962035: -- diff -up dovecot-2.3.14/src/lib-dcrypt/dcrypt-openssl.c.opensslv3 dovecot-2.3.14/src/lib-dcrypt/dcrypt-openssl.c --- dovecot-2.3.14/src/lib-dcrypt/dcrypt-openssl.c.opensslv3 2021-06-03 18:56:52.573174433 +0200 +++ dovecot-2.3.14/src/lib-dcrypt/dcrypt-openssl.c 2021-06-03 18:56:52.585174274 +0200 @@ -73,10 +73,30 @@ 2<tab>key algo oid<tab>1<tab>symmetric algo name<tab>salt<tab>hash algo<tab>rounds<tab>E(RSA = i2d_PrivateKey, EC=Private Point)<tab>key id **/ +#if OPENSSL_VERSION_MAJOR == 3 +static EC_KEY *EVP_PKEY_get0_EC_KEYv3(EVP_PKEY *key) +{ + EC_KEY *eck = EVP_PKEY_get1_EC_KEY(key); + EVP_PKEY_set1_EC_KEY(key, eck); + EC_KEY_free(eck); + return eck; +} + +static EC_KEY *EVP_PKEY_get1_EC_KEYv3(EVP_PKEY *key) +{ + EC_KEY *eck = EVP_PKEY_get1_EC_KEY(key); + EVP_PKEY_set1_EC_KEY(key, eck); + return eck; +} + +#define EVP_PKEY_get0_EC_KEY EVP_PKEY_get0_EC_KEYv3 +#define EVP_PKEY_get1_EC_KEY EVP_PKEY_get1_EC_KEYv3 +#else #ifndef HAVE_EVP_PKEY_get0 #define EVP_PKEY_get0_EC_KEY(x) x->pkey.ec #define EVP_PKEY_get0_RSA(x) x->pkey.rsa #endif +#endif #ifndef HAVE_OBJ_LENGTH #define OBJ_length(o) ((o)->length) -- I presume that either this patch or an equivalent is planned for eventual inclusion into upstream? -- Sloane Bernstein Developer I cPanel, L.L.C.

2 1