- dovecot - dovecot.org

doveadm http api not responding under load
by Achim Stahlberger 23 Jan '24

23 Jan '24

Hello, we see this problem when doveadm-server is under load. We enabled doveadm http api with this config: service doveadm { inet_listener { port = 50000 } inet_listener http { port = 50001 ssl = no } } When doveadm server receives at lot of connects on port 50000 the http service on port 50001 is not responding until load on port 50000 drops to zero. It seems like doveadm-server is prefering port 50000 over 50001. Looking into Recv-Q when http hangs Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port tcp LISTEN 129 128 0.0.0.0:50000 0.0.0.0:* tcp LISTEN 1 128 0.0.0.0:50001 0.0.0.0:* I used this script to produce load on port 50000 #!/bin/bash for i in {1..1000} do echo "124" | netcat localhost 50000 & done wait When this script is started it takes several seconds until a connect on port 50001 succeeds. We are using Dovecot version 2.3.20 (bdc808f24c). I think the problem is in src/lib/ioloop-epoll.c function io_loop_handler_run_internal. This might fix the problem (look for new variable rr): void io_loop_handler_run_internal(struct ioloop *ioloop) { struct ioloop_handler_context *ctx = ioloop->handler_context; struct epoll_event *events; const struct epoll_event *event; struct io_list *list; struct io_file *io; struct timeval tv; unsigned int events_count; int msecs, ret, i, j, k, rr; bool call; i_assert(ctx != NULL); /* get the time left for next timeout task */ msecs = io_loop_run_get_wait_time(ioloop, &tv); events = array_get_modifiable(&ctx->events, &events_count); if (ioloop->io_files != NULL && events_count > ctx->deleted_count) { ret = epoll_wait(ctx->epfd, events, events_count, msecs); if (ret < 0 && errno != EINTR) i_fatal("epoll_wait(): %m"); } else { /* no I/Os, but we should have some timeouts. just wait for them. */ i_assert(msecs >= 0); i_sleep_intr_msecs(msecs); ret = 0; } /* execute timeout handlers */ io_loop_handle_timeouts(ioloop); if (!ioloop->running) return; rr = ret>1 ? i_rand_limit(ret) : 0; for (k = 0; k < 2; k++) { for (i = (k==0 ? rr : 0); i < (k==0 ? ret : rr); i++) { /* io_loop_handle_add() may cause events array reallocation, so we have use array_idx() */ event = array_idx(&ctx->events, i); list = event->data.ptr; for (j = 0; j < IOLOOP_IOLIST_IOS_PER_FD; j++) { io = list->ios[j]; if (io == NULL) continue; call = FALSE; if ((event->events & (EPOLLHUP | EPOLLERR)) != 0) call = TRUE; else if ((io->io.condition & IO_READ) != 0) call = (event->events & EPOLLIN) != 0; else if ((io->io.condition & IO_WRITE) != 0) call = (event->events & EPOLLOUT) != 0; else if ((io->io.condition & IO_ERROR) != 0) call = (event->events & IO_EPOLL_ERROR) != 0; if (call) { io_loop_call_io(&io->io); if (!ioloop->running) return; } } } } } Thanks a lot. Achim https://www.tallence.com Folgt uns auf LinkedIn <https://www.linkedin.com/company/tallence-ag> // XING <https://www.xing.com/pages/tallenceag> // Twitter <https://twitter.com/tallence> // Instagram <https://www.instagram.com/tallence_ag/> // facebook <http://www.facebook.com/Tallence> Tallence AG // Sitz der Gesellschaft: Hamburg // Neue Gröningerstraße 13, 20457 Hamburg // Amtsgericht Hamburg HRB 142900 Vorstand: Frank Moll (Vorsitzender), Bernd Scherf // Aufsichtsrat: Wolf Ingomar Faecks (Vorsitzender) Die Speicherung und Verarbeitung personenbezogener Daten erfolgt entsprechend unserer Datenschutzhinweise. <https://www.tallence.com/datenschutzhinweise>

2 2

"Connection reset by peer" errors with Outlook
by Steve Dondley 23 Jan '24

23 Jan '24

I have a mail server using dovecot that has been running without issue for quite a couple of years now. It serves email for about 30 individuals. But since Jan 14th, users have been reporting spurious errors in MS Outlook: 316 Jan 21 00:38:12 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=783) failed: Connection reset by peer, session=<yVd4Hm4Pst12wSuN> 317 Jan 21 00:38:12 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=598) failed: Connection reset by peer, session=<aKOAHm4PCOB2wSuN> 318 Jan 21 00:38:13 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=598) failed: Connection reset by peer, session=<9rWIHm4PtuF2wSuN> 319 Jan 21 00:38:13 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=677) failed: Connection reset by peer, session=<NsORHm4P9OJ2wSuN> 320 Jan 21 00:38:14 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=691) failed: Connection reset by peer, session=<AACaHm4PdOR2wSuN> 321 Jan 21 00:38:15 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=610) failed: Connection reset by peer, session=<G8umHm4P2OV2wSuN> 322 Jan 21 00:38:16 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=609) failed: Connection reset by peer, session=<qSy0Hm4PuOd2wSuN> 323 Jan 21 00:38:16 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=596) failed: Connection reset by peer, session=<W427Hm4PtOl2wSuN> 324 Jan 21 00:38:17 ip-172-30-0-131 dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=118.xxx.xxx.xxx, lip=172.30.0.131, TLS handshaking: read(size=596) failed: Connection reset by peer, session=<mu/JHm4Ptup2wSuN> Some characteristics of the problem that may offer a clue: * happening with multiple users, not just the same one * happens from different IP addresses. * happens about 3 to 5 times per day and the errors come in batches like above * MS Outlook error is: reported error (0x80042109): ‘Outlook cannot conect to your outgoing SMTP email server. If you continue to receive this message….blah blah blah I googled the error code but didn’t find anything particularly helpful. I’m running Debian bullseye, version 11.8.

4 7

Migrate Dovecot 2.0.16 (mbox) to Dovecot 2.3.16 (MaiDir) and preserve POP UIDs
by Barbara M. 21 Jan '24

21 Jan '24

Already done a first migration using imapsync Good result, but there is the POP3 UIDs problem. Searching online give a Tried to understand the instructions on: https://doc.dovecot.org/settings/plugin/pop3-migration-plugin/? with limited comprension results (the link in the page is to the old wiki). Searching online give more confusion ... I am obviouly interested to resync the INBOX with somethig that give to clients the old UIDs so they don't re-download all messages in the inbox as duplicates into their mailbox. I suppose I can do: doveadm -v expunge -u <username> mailbox INBOX all rm dovecot* in the Maildir folder doveadm force-resync -u <username> INBOX then use dsync to migrate the INBOX Here I need some hints to how setup it and wich syntax use. If useful I can copy or mount the /var/mail/ folder of the old server that contains the users INBOX to the new server. Thanks, B.

2 1

Re: Migrate Dovecot 2.0.16 (mbox) to Dovecot 2.3.16 (MaiDir) and preserve POP UIDs
by Joseph Tam 21 Jan '24

21 Jan '24

From: "Barbara M." <barbara(a)rfx.it> > I am obviouly interested to resync the INBOX with somethig that give to > clients the old UIDs so they don't re-download all messages in the inbox > as duplicates into their mailbox. When I mitigated away from qpopper, I configured protocol pop3 { ... pop3_reuse_xuidl = yes } Maybe that's of use to you? Joseph Tam <jtam.home(a)gmail.com>

1 0

Uncompress compressed messages in Maildir mailbox?
by achekalin＠lazurit.com 21 Jan '24

21 Jan '24

Some time ago I enabled compression on incoming messages, so LMTP delivered messages were compressed. Good idea in terms of disk space, but now I cannot uncompress any of these messages even when I need to! I was using this setting: plugins { zlib_save = lz4 } and now I can see this string at the beginning of each message file - "Dovecot-LZ4", so no other software can uncompress them for me. Please advise, how can I uncompress message file in Maildir mailbox when I need to access its contents directly?

4 5

master-users problem
by Barbara M. 20 Jan '24

20 Jan '24

I am trying to activate master-users as usual but seems in this case I am doing something wrong and I can't have it working. It is a Rocky Linux 9.x (in Proxmox CT), installed with virtualmin and using system users: # doveconf -n # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # OS: Linux 6.5.11-4-pve x86_64 Rocky Linux release 9.3 (Blue Onyx) auth_master_user_separator = * auth_mechanisms = plain login disable_plaintext_auth = no first_valid_uid = 1000 mail_location = maildir:~/Maildir mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes result_success = continue } passdb { driver = pam } protocols = imap pop3 ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_cipher_list = PROFILE=SYSTEM ssl_key = # hidden, use -P to show it userdb { driver = passwd } Any hints on what I am doing wrong and/or how to debug it? Thanks, B.

3 4

Forcing imap authentication failure for certain IP addresses
by Hippo Man 20 Jan '24

20 Jan '24

I'm running dovecot 2.3.18 under Debian 11. I want to do something that's a bit unusual: when IMAP connections are attempted from a few specific IP addresses, I want to force an IMAP authentication failure from those connections, no matter what user ID and password are specified. I know that I can use iptables to completely block imap access from those IP addresses to the IMAP ports. However, in these specific cases, I'd prefer that the connection goes through to dovecot, but for dovecot then to always generate authentication failures for those specific connections ... even if a valid user ID and password happen to be specified. Is there a way to do this in dovecot? Thank you very much in advance. -- hippoman(a)gmail.com Take a hippopotamus to lunch today. .---------, 0__0 / ( oo'---, / oo\ ,\ | | \ ,=__/ \ / / /------| /| |__|-' |__|'

2 1

Documentation flaw: man-page link.
by Ralph Corderoy 19 Jan '24

19 Jan '24

Hi, https://doc.dovecot.org/admin_manual/doveadm_mailbox_commands/ ends with The man-pages also contain good descriptions of doveadm commands. Man-pages can be accessed online at e.g. https://wiki.dovecot.org/Tools/Doveadm That wiki page says Our primary documentation is now at https://doc.dovecot.org, please update your links. But I was at https://doc.dovecot.org to start with and there is no obvious mention of man pages there. This needs tidying up to avoid the loop and ideally provide man pages. -- Cheers, Ralph.

3 3

post-login scripting
by Alex 19 Jan '24

19 Jan '24

Hi, I'm trying to determine the last login time for my IMAP users. Some systems use mbox while others are Maildir, but not sure that matters. I'd just like to touch a file similar to how it's done in this article, but it doesn't work for me. https://doc.dovecot.org/admin_manual/post_login_scripting/ I've followed the instructions explicitly - made the required changes to 10-master.conf and created a shell script in /usr/local/bin that touches a file in the user's home directory. However, the file doesn't get created/touched. I've enabled debugging and don't see any errors that might lead to what I'm doing wrong. It doesn't say explicitly in the article that the last-login script referenced in the article is actually the contents of the postlogin.sh script, but I'm assuming it is? When I put the following in my dovecot.conf instead of conf.d/10-auth.conf, it core dumps. I thought local changes should be made in dovecot.conf? service imap-login { executable = imap imap-postlogin inet_listener imap { port = 0 } inet_listener imaps { port = 993 } process_min_avail = 6 service_count = 0 vsz_limit = 512 M } service imap-postlogin { executable = script-login /usr/local/bin/postlogin.sh user = $default_internal_user } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 } } Hi, I'm trying to determine the last login time for my IMAP users. Some systems use mbox while others are Maildir, but not sure that matters. I'd just like to touch a file similar to how it's done in this article, but it doesn't work for me. https://doc.dovecot.org/admin_manual/post_login_scripting/ I've followed the instructions explicitly - made the required changes to 10- master.conf and created a shell script in /usr/local/bin that touches a file in the user's home directory. However, the file doesn't get created/touched. I've enabled debugging and don't see any errors that might lead to what I'm doing wrong. It doesn't say explicitly in the article that the last-login script referenced in the article is actually the contents of the postlogin.sh script, but I'm assuming it is? When I put the following in my dovecot.conf instead of conf.d/10-auth.conf, it core dumps. I thought local changes should be made in dovecot.conf? service imap-login { executable = imap imap-postlogin inet_listener imap { port = 0 } inet_listener imaps { port = 993 } process_min_avail = 6 service_count = 0 vsz_limit = 512 M } service imap-postlogin { executable = script-login /usr/local/bin/postlogin.sh user = $default_internal_user } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 } }

4 7

Re: What is the difference between BEFORE, SENTBEFORE, and SAVEDBEFORE?
by Joseph Tam 19 Jan '24

19 Jan '24

From: Paul Pace <paul(a)mostlybsd.com> > BEFORE date specification > Matches messages with an internal date before date > specification. > > SENTBEFORE date specification > Matches messages with a Date: header before date specification. > > SAVEDBEFORE date specification > Matches messages, which were saved before date specification. > > I am creating a cron job to purge old messages in Trash or Spam folders, > but I discovered using doveadm search queries using savedbefore that > somehow no message in any of the folders shows up with queries starting > at 5d, even though there are much older messages. I don't know exactly the answer to your query, but the semantics of each term may not be the problem. I had this exact problem a decade ago with my own trash purging script. If you dump the above values e.g. doveadm fetch -ftab -A 'mailbox date.received' mailbox Trash BEFORE 90d you may find that many of them have the same value, and what's more, the time coincides with the first time you queried for that value (i.e. when you ran the above doveadm fetch). I think these fields may not have cached values before you ask for it, then it gets instantiated the current timestamp when you do. If you do a fetch every day, you'll eventually reach 90d, and it will work forever more (+/- 1 day). Perhaps adding those fields into these settings is a more direct and better solution: https://doc.dovecot.org/configuration_manual/mail_cache_settings/ Joseph Tam <jtam.home(a)gmail.com>

1 1