March 2024 - dovecot - dovecot.org

dovecot panic
by Michael Grant 15 Mar '24

15 Mar '24

I'm running dovecot 2.3.19.1 on debian stable. I just started seeing a slew of these in my log: Mar 14 10:02:38 strange.networkguild.org dovecot[865654]: imap(mgrant)<1939553><DcMIWJ8T6s7ZIx04>: Panic: file istream-header-filter.c: line 663 (i_stream_header_filter_snapshot_free): assertion failed: (snapshot->mstream->snapshot_pending) Mar 14 10:02:38 strange.networkguild.org dovecot[865654]: imap(mgrant)<1939553><DcMIWJ8T6s7ZIx04>: Error: Raw backtrace: #0 test_subprocess_fork[0x7f1a33d7ca20] -> #1 backtrace_append[0x7f1a33d7cc90] -> #2 backtrace_get[0x7f1a33d7ce20] -> #3 execvp_const[0x7f1a33d89f90] -> #4 i_syslog_fatal_handler[0x7f1a33d8a900] -> #5 i_panic[0x7f1a33ce00a4] -> #6 fs_wrapper_unlock[0x7f1a33cdb71d] -> #7 i_stream_snapshot_free[0x7f1a33d96110] -> #8 i_stream_unref[0x7f1a33d96180] -> #9 index_mail_get_virtual_size[0x7f1a33f2d190] -> #10 index_mail_save_finish[0x7f1a33f2d470] -> #11 index_mail_get_special[0x7f1a33f2d690] -> #12 mail_get_special[0x7f1a33eaf950] -> #13 cmd_select[0x560714f983e0] -> #14 imap_fetch_begin[0x560714f9d170] -> #15 imap_fetch_more[0x560714f9d550] -> #16 cmd_fetch[0x560714f91c00] -> #17 command_exec[0x560714f9a880] -> #18 cmd_x_cancel[0x560714fa0510] -> #19 cmd_x_cancel[0x560714fa0510] -> #20 cmd_x_cancel[0x560714fa0510] -> #21 client_handle_input[0x560714fa0880] -> #22 client_input[0x560714fa0d80] -> #23 io_loop_call_io[0x7f1a33da1c70] -> #24 io_loop_handler_run_internal[0x7f1a33da3970] -> #25 io_loop_handler_run[0x7f1a33da3b00] -> #26 io_loop_run[0x7f1a33da3cd0] -> #27 master_service_run[0x7f1a33d14180] -> #28 main[0x560714f8ca00] -> #29 __libc_init_first[0x7f1a33ac51d0] -> #30 __libc_start_main[0x7f1a33ac5280] -> #31 _start[0x560714f8d000] I realize this is a slightly old version of dovecot but this is what's in debian's stable package repository. I see 2.3.21+dfsg1-2 in testing and 2.3.21+dfsg1-3 in unstable. I really try to avoid installing this on production. I didn't recently change anything in my configs on the server. I did start recently using 'eM Client' on Windows a couple days ago, could that have caused this? Mail seems to continue to come in to my imap clients, but these errors in the log are worrying! Michael Grant I'm running dovecot 2.3.19.1 on debian stable. I just started seeing a slew of these in my log: Mar 14 10:02:38 strange.networkguild.org dovecot[865654]: imap (mgrant)<1939553><DcMIWJ8T6s7ZIx04>: Panic: file istream-header-filter.c: line 663 (i_stream_header_filter_snapshot_free): assertion failed: (snapshot- >mstream->snapshot_pending) Mar 14 10:02:38 strange.networkguild.org dovecot[865654]: imap (mgrant)<1939553><DcMIWJ8T6s7ZIx04>: Error: Raw backtrace: #0 test_subprocess_fork[0x7f1a33d7ca20] -> #1 backtrace_append[0x7f1a33d7cc90] - > #2 backtrace_get[0x7f1a33d7ce20] -> #3 execvp_const[0x7f1a33d89f90] -> #4 i_syslog_fatal_handler[0x7f1a33d8a900] -> #5 i_panic[0x7f1a33ce00a4] -> #6 fs_wrapper_unlock[0x7f1a33cdb71d] -> #7 i_stream_snapshot_free[0x7f1a33d96110] -> #8 i_stream_unref[0x7f1a33d96180] -> #9 index_mail_get_virtual_size [0x7f1a33f2d190] -> #10 index_mail_save_finish[0x7f1a33f2d470] -> #11 index_mail_get_special[0x7f1a33f2d690] -> #12 mail_get_special[0x7f1a33eaf950] -> #13 cmd_select[0x560714f983e0] -> #14 imap_fetch_begin[0x560714f9d170] - > #15 imap_fetch_more[0x560714f9d550] -> #16 cmd_fetch[0x560714f91c00] -> #17 command_exec[0x560714f9a880] -> #18 cmd_x_cancel[0x560714fa0510] -> #19 cmd_x_cancel[0x560714fa0510] -> #20 cmd_x_cancel[0x560714fa0510] -> #21 client_handle_input[0x560714fa0880] -> #22 client_input[0x560714fa0d80] -> #23 io_loop_call_io[0x7f1a33da1c70] -> #24 io_loop_handler_run_internal [0x7f1a33da3970] -> #25 io_loop_handler_run[0x7f1a33da3b00] -> #26 io_loop_run [0x7f1a33da3cd0] -> #27 master_service_run[0x7f1a33d14180] -> #28 main [0x560714f8ca00] -> #29 __libc_init_first[0x7f1a33ac51d0] -> #30 __libc_start_main[0x7f1a33ac5280] -> #31 _start[0x560714f8d000] I realize this is a slightly old version of dovecot but this is what's in debian's stable package repository. I see 2.3.21+dfsg1-2 in testing and 2.3.21+dfsg1-3 in unstable. I really try to avoid installing this on production. I didn't recently change anything in my configs on the server. I did start recently using 'eM Client' on Windows a couple days ago, could that have caused this? Mail seems to continue to come in to my imap clients, but these errors in the log are worrying! Michael Grant

2 1

Re: [EXT] Re: How to get a memory pointer in the core process
by Joseph Tam 15 Mar '24

15 Mar '24

From: Joan Moreau > I am trying to avoid closing/ reopening a file pointer to the exact same file > between each call to the plugin Can't you do an end-run around all this by having a persistent standalone process that holds all your persistent data, and the plugin collects session data and passes it to the persistent process via sockets? You'll have to have some initial handshake protocol to establish session context, but this seems the easiest way to accomplish what you want. Joseph Tam <jtam.home(a)gmail.com>

1 0

dovecot and solr clustering
by Maciej Milaszewski 13 Mar '24

13 Mar '24

Hi I have 1 director and ~8 dovecot nodes. I thinking about Solr cluster beacuse one server solr probably is not enough. I thinking about SolrCloud Mode but I don't know if it will work with Dovecot And if not SolrCloud Mode, then what? I don't have much experience in Solr clustering and I want to choose the most optimal solution with scalable aand redundancy (bare metal servers is not problem) Hi I have 1 director and ~8 dovecot nodes. I thinking about Solr cluster beacuse one server solr probably is not enough. I thinking about SolrCloud Mode but I don't know if it will work with Dovecot And if not SolrCloud Mode, then what? I don't have much experience in Solr clustering and I want to choose the most optimal solution with scalable aand redundancy (bare metal servers is not problem)

1 0

Dovecot and SNI
by steffan＠hldns.com 13 Mar '24

13 Mar '24

I have an old CentOS 7 server using dovecot 2.2.36 and OpenSSL 1.0.2k-fips that’s been fine for quite some time. Recently I started getting complaints related to SNI. I test with this: openssl s_client -connect mail.domain.com:993 -crlf -quiet On macOS using OpenSSL LibreSSL 3.3.6 I test and get the default dovecot domain “SomeWrongDomain.com” which causes issues. On Oracle Linux 9 using OpenSSL 3.0.7 I get the correct response for the domain “mail.domain.com” What could cause this? Thank you, Steffan Cline steffan(a)hldns.com<mailto:steffan@hldns.com> 602-793-0014 I have an old CentOS 7 server using dovecot 2.2.36 and OpenSSL 1.0.2k-fips that’s been fine for quite some time. Recently I started getting complaints related to SNI. I test with this: openssl s_client -connect mail.domain.com:993 -crlf -quiet On macOS using OpenSSL LibreSSL 3.3.6 I test and get the default dovecot domain “SomeWrongDomain.com” which causes issues. On Oracle Linux 9 using OpenSSL 3.0.7 I get the correct response for the domain “mail.domain.com” What could cause this? Thank you, Steffan Cline steffan(a)hldns.com 602-793-0014

4 3

quota plugin with virtual user
by noemi 13 Mar '24

13 Mar '24

Hi, I'm having some problems after activating the quota plugin: I followed the guide from the doc[0] but I'm having problems with some virtual user. I understand that,with my conf,the quota plugin check the limit of a user and then tell postfix to accept/discard the mail. i have some users under my main domain [conf in /etc/passwd] and some virtual user/domains [conf in virtusertable/virtdomains as files in postfix]. the virtual users use their local user for the login, so dovecot doesn't know they are virtual. I think the problem is here, because it is clear that the quota-status check is made by dovecot using the virtual user and not the local user[1]. understanding the problem doesn't help me in the resolution [other than burn everything and rebuild with a single management for local/virtual users using a db] but i will prefer a more soft approach. any help? my complete conf below[2] thanks, noemi [0] https://blog.sys4.de/postfix-dovecot-mailbox-quota-en.html [1] where the virtual user is mail(a)virtdomain.com and the local user is mail_virtdomain the check is done to user 'mail', thus returning a error: Nov 16 06:36:09 quota-status(other-user)<129776><OOwwMv+3VWXw+gEAKobZ8Q>: Error: Failed to lookup user mail(a)virtdomain.com: Mail access for users with UID 8 not permitted (see first_valid_uid in config file, uid from userdb lookup). [2] # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 6.1.0-10-amd64 x86_64 Debian 11.7 zfs # Hostname: mail-ng.pangea.org auth_mechanisms = plain login auth_username_format = %Ln default_client_limit = 3000 disable_plaintext_auth = no first_valid_uid = 8 log_path = /var/log/mail.log mail_location = maildir:/home/%n/Maildir mail_plugins = " quota" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to https://laweb.pangea.org/contacte/ for instructions on how to fix this. quota_grace = 10%% quota_max_mail_size = 100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full and cannot receive any more emails quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve/;active=~/.dovecot.sieve sieve_user_log = /var/log/sieve.log sieve_vacation_send_from_recipient = yes } protocols = " imap lmtp sieve pop3" service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service imap-login { process_limit = 1024 service_count = 100 } service imap { process_limit = 1024 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { process_limit = 1024 service_count = 0 } service pop3 { process_limit = 1024 } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = 127.0.0.1 port = 7777 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = dovecot } ssl_cert = </etc/letsencrypt/live/mail-ng.pangea.org/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol imap { mail_max_userip_connections = 40 mail_plugins = " quota imap_quota" } protocol lmtp { mail_plugins = " quota sieve" }

1 0

Possible bug in dovecot doc
by Petr Šorm 13 Mar '24

13 Mar '24

Dear maintainer, there is potentially conflicting information in dovecot doc about the count quota backend. According to https://doc.dovecot.org/configuration_manual/quota/quota_count/ the count quota backend is the recommended way of calculating quota on recent Dovecot installations. But according to https://github.com/dovecot/core/blob/main/doc/example-config/conf.d/90-quot… it is extremely SLOW with Maildir. It'll eat your CPU and disk I/O. I don't think the information on github is true, but I could be wrong. Could you fix/clarify this? Thank you in advance. Best regards Petr Sorm

1 0

Configure: "Can't build with MySQL support: libmysqlclient not found"
by James L. Brown 12 Mar '24

12 Mar '24

Hi all. I’m getting errors when trying to configure make for version 2.3.21 on macOS 14.3.1 Sonoma. It ends with: checking for mysql_config... mysql_config checking for mysql_init in -lmysqlclient... no configure: error: Can't build with MySQL support: libmysqlclient not found I’m using: env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config --libs libsodium CPPFLAGS=-I/opt/homebrew/Cellar/openssl(a)3/3.2.1/include LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl --with-mysql --with-sodium Looking at config.log I see: configure:28222: checking for mysql_config configure:28238: found /opt/homebrew/bin/mysql_config configure:28250: result: mysql_config configure:28285: checking for mysql_init in -lmysqlclient configure:28310: gcc -o conftest -std=gnu99 -g -O2 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/opt/homebrew/Cellar/openssl(a)3/3.2.1/include -I/opt/homebrew/Cellar/openssl(a)3/3.2.1/include -L/opt/homebrew/opt/openssl@3/lib conftest.c -lmysqlclient -L/opt/homebrew/Cellar/mysql/8.3.0_1/lib -lmysqlclient -lz -lzstd -lssl -lcrypto -lresolv -lz -lm >&5 ld: warning: ignoring duplicate libraries: '-lmysqlclient', '-lz' ld: library 'zstd' not found clang: error: linker command failed with exit code 1 (use -v to see invocation) configure:28310: $? = 1 Odd that it is using the old version mysql. Would someone be able to help me get it to find libmysqlclient? In /opt/homebrew/Cellar/mysql/8.0.32/lib/ I have: libmysqlclient.21.dylib libmysqlclient.a libmysqlclient.dylib Thanks, James. Hi all. I’m getting errors when trying to configure make for version 2.3.21 on macOS 14.3.1 Sonoma. It ends with: checking for mysql_config... mysql_config checking for mysql_init in -lmysqlclient... no configure: error: Can't build with MySQL support: libmysqlclient not found I’m using: env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config --libs libsodium CPPFLAGS=-I/opt/homebrew/Cellar/openssl(a)3/3.2.1/include LDFLAGS=-L/opt/ homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl --with-mysql --with- sodium Looking at config.log I see: configure:28222: checking for mysql_config configure:28238: found /opt/homebrew/bin/mysql_config configure:28250: result: mysql_config configure:28285: checking for mysql_init in -lmysqlclient configure:28310: gcc -o conftest -std=gnu99 -g -O2 -fstack-protector-strong - U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing- declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast - Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/opt/homebrew/Cellar/ openssl(a)3/3.2.1/include -I/opt/homebrew/Cellar/openssl(a)3/3.2.1/include -L/opt/ homebrew/opt/openssl@3/lib conftest.c -lmysqlclient -L/opt/homebrew/Cellar/ mysql/8.3.0_1/lib -lmysqlclient -lz -lzstd -lssl -lcrypto -lresolv -lz -lm >&5 ld: warning: ignoring duplicate libraries: '-lmysqlclient', '-lz' ld: library 'zstd' not found clang: error: linker command failed with exit code 1 (use -v to see invocation) configure:28310: $? =1 Odd that it is using the old version mysql. Would someone be able to help me get it to find libmysqlclient? In /opt/homebrew/Cellar/mysql/8.0.32/lib/ I have: libmysqlclient.21.dylib libmysqlclient.a libmysqlclient.dylib Thanks, James.

3 4

Re: Dovecot installation and ssl certificates
by Joseph Tam 12 Mar '24

12 Mar '24

Jerry Stuckle writes: > I'm starting with POP3 (because it's easy to handle from the CLI). I > have it working from localhost - I can telnet to port localhost 110 and > access emails (of course I can't do this from a remote system because it > requires SSL). Sure you can: (STARTTLS style) openssl s_client -starttls pop3 -connect your.pop.server:110 (SSL style) openssl s_client -connect your.pop.server:995 You can also use ncat (exercise left to the reader). "localhost", I believe, is a specific exclusion where SSL in *not* enforced during the session. > Trying to access this from a remote system with Thunderbird is not > working. /var/log/mail.log shows the following: > > 2024-03-03T22:18:54.887061-05:00 debian-server dovecot: pop3-login: > Disconnected: Connection closed: SSL_accept() failed: error:0A000412:SSL > routines::sslv3 alert bad certificate: SSL alert number 42 (no auth > attempts in 0 secs): user=<>, rip=206.223.85.12, lip=206.223.85.137, TLS > handshaking: SSL_accept() failed: error:0A000412:SSL routines::sslv3 > alert bad certificate: SSL alert number 42, session=<l6qBL80SOePO31UM> This looks like an error message stating STARTTLS was expected but the client used a plaintext session. Try enabling STARTTLS on your Thunderbird. If that is not the cause ... Ref: https://doc.dovecot.org/configuration_manual/dovecot_ssl_configuration/ The most important dovecot configurations are ssl_cert = </path/to/your/server.crt ssl_key = </path/to/your/server.key You can have protocol specific certificates e.g. protocol pop3 { ssl_cert = ... ssl_key = ... } but you'll typically put this outside the protocol section so that there is one default certificate. > I need help. First of all, an explanation as to how to configure the > dovecot-openssl.cnf file (an example with actual values - real or fake - > would be a real help). > > Second, where does this go? Maybe running doveconf will tell you where your installation expects the main configuration file to be. This file may include other config files. > Note that for right now I'm trying to just get one domain working but > eventually this will serve at least 4 domains. Once I get the first > domain working, thoughts about how to get multiple domains working would > also be appreciated. This depends on how you set up your filesystem and authentication and your security constraints. You'll have to be more specific on your setup. Confining my reply to just SSL setup, you can obtain a SSL certificate with multiple domains named listed, which makes multi-domain SSL support easier. Joseph Tam <jtam.home(a)gmail.com>

1 0

Remote IP address in dovecot
by Andre Rodier 12 Mar '24

12 Mar '24

Hello, I am using a webmail, SOGo, to access Dovecot mail server in IMAP. SOGo correctly logs the remote IP address, using the standard nginx headers, especially X-Forwarded-For. However, when I check the IMAP logs, I see my server's IP address for the "remote_ip" value. I think there is a way to override this value from the environment, but I could not find it. Thanks for your help, -- André Rodier

2 1

Dovecot installation and ssl certificates
by Jerry Stuckle 12 Mar '24

12 Mar '24

Hi, all, First of all, I will admit I am totally a noob with Dovecot. I am trying to install it on Debian and am somewhat lost. Right now this is a test system on our LAN. I'm starting with POP3 (because it's easy to handle from the CLI). I have it working from localhost - I can telnet to port localhost 110 and access emails (of course I can't do this from a remote system because it requires SSL). Trying to access this from a remote system with Thunderbird is not working. /var/log/mail.log shows the following: 2024-03-03T22:18:54.887061-05:00 debian-server dovecot: pop3-login: Disconnected: Connection closed: SSL_accept() failed: error:0A000412:SSL routines::sslv3 alert bad certificate: SSL alert number 42 (no auth attempts in 0 secs): user=<>, rip=206.223.85.12, lip=206.223.85.137, TLS handshaking: SSL_accept() failed: error:0A000412:SSL routines::sslv3 alert bad certificate: SSL alert number 42, session=<l6qBL80SOePO31UM> I tried to create a certificate with mkconf.sh but obviously don't have the right information in the configuration file and/or didn't get the files in the right place (I've seen different places they should be). I need help. First of all, an explanation as to how to configure the dovecot-openssl.cnf file (an example with actual values - real or fake - would be a real help). Second, where does this go? Note that for right now I'm trying to just get one domain working but eventually this will serve at least 4 domains. Once I get the first domain working, thoughts about how to get multiple domains working would also be appreciated. I know we noobs are a PITA but I really appreciate the help. Jerry

2 1