August 2024 - dovecot

Debug: open(/proc/self/io) failed: Permission denied
by Alex 31 Aug '24

31 Aug '24

Hi, I've seen a few emails over the years related to this error message, but they are typically associated with doveadm. In my case, it's related to dovecot itself. I've just set up dovecot-2.3.20-1.fc37.x86_64 on fedora37 and unable to receive mail. The main config was copied over from another dovecot system on fedora37 with virtually identical settings, including the same SSL cert. I've also tried to just migrate the important stuff and otherwise leave the defaults, but I'm having the same problem. imap(bcc-user)<2977522><rappeBf6vLZEw8Eq>: Debug: Loading modules from directory: /usr/lib64/dovecot imap(bcc-user)<2977522><rappeBf6vLZEw8Eq>: Debug: Module loaded: /usr/lib64/dovecot/lib20_listescape_plugin.so imap(bcc-user)<2977522><rappeBf6vLZEw8Eq>: Debug: Effective uid=1001, gid=1001, home=/home/bcc-user imap(bcc-user)<2977522><rappeBf6vLZEw8Eq>: Debug: open(/proc/self/io) failed: Permission denied imap(bcc-user)<2977522><rappeBf6vLZEw8Eq>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/spool/mail/bcc-user imap(bcc-user)<2977522><rappeBf6vLZEw8Eq>: Debug: fs: root=/home/bcc-user/mail, index=, indexpvt=, control=, inbox=/var/spool/mail/bcc-user, alt= imap(bcc-user)<2977522><rappeBf6vLZEw8Eq>: Debug: Mailbox Trash: Mailbox opened I've set the /var/spool/mail/bcc-user permissions to 600, bcc-user:mail. The /home/bcc-user directory is 611, bcc-user:bcc-user. Here is my doveconf -n: # 2.3.20 (80a5ac675d): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.20 (149edcf2) # OS: Linux 6.1.15-200.fc37.x86_64 x86_64 Fedora release 37 (Thirty Seven) auth_debug = yes auth_verbose = yes mail_debug = yes mail_gid = mail mail_home = mail/ mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_max_userip_connections = 500 mail_plugins = " listescape" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 } process_min_avail = 6 service_count = 0 vsz_limit = 512 M } ssl_cert = </etc/letsencrypt/fullchain.cer ssl_cipher_list = ALL:!LOW:!EXP:!aNULL:!RC4::!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = passwd } verbose_ssl = yes Any ideas greatly appreciated. I was very surprised to be having problems after having done this so many times before.

3 5

Connection closed (user disabled)
by Roberto J. Bolivar Plenge 28 Aug '24

28 Aug '24

Hello everyone Thank you in advance for your help. Why do I get the following error: Aug 23 10:37:51 cp-cl02 dovecot[2337577]: pop3-login: Disconnected: Connection closed (user disabled): user=<loliva(a)romaseguro.pe>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS: Connection closed, session=<8na4jlsg3HK1Q1LE>. I never observed this before: Connection closed (user disabled): I wish to know what is causing it and how can I fix it. Thank you.

2 4

Dovecot with LDAP and Solr: Force Solr user key
by Michele Giacomoli 27 Aug '24

27 Aug '24

Good morning everyone. I am configuring dovecot with ldap and solr as fts plugins. The LDAP configuration works correctly and users can log in both via the userPrincipalName attribute and via mail. In this configuration the LDAP realm and the email domain differ, and on the server the LDAP realm domain is privileged. This means that the mail dir is located in ./ldap_realm/username and in the same way postfix delivers the emails to dovecot using username@ldap_realm as envelope recipient. Consequently the emails are indexed on solr using "username@ldap_realm" as user key. However if I log in in imap using the email address as username and try to do some searches dovecot sends a select to solr using the email address as key and not the ldap user, and consequently I do not get any results from the search. So my question is: is it possible to configure dovecot to force queries to solr so that "username@ldap_realm" is always used as the key even when logging in via email? Dovecot version is: 2.3.7.2 Solr version: 7.2.1 Ldap configuration: [...] user_filter = (&(|(userPrincipalName=%u)(mail=%u))(objectClass=person)(!(userAccountControl=514))(memberOf=cn=Mail,ou=Groups,dc=example,dc=org)) pass_filter = (&(|(userPrincipalName=%u)(mail=%u))(objectClass=person)(!(userAccountControl=514))(memberOf=cn=Mail,ou=Groups,dc=example,dc=org)) pass_attrs = userPassword=password user_attrs = =home=/data/mail/%Ld/%Ln/Maildir/,=mail=maildir:/data/mail/%Ld/%Ln/Maildir/ Solr configuration: plugin { fts = solr fts_solr = url=http://solr_host:8983/solr/dovecot/ fts_autoindex=yes } Best regards -- Logo Mynet *Michele Giacomoli* Reparto IT 32.000 km di fibra ottica nel nord Italia al servizio delle aziende *www.mynet.it* <https://www.mynet.it/> - *www.vogliadifibra.it* <https://www.vogliadifibra.it/>

1 1

2.3.21(.1) build error on aarch64 (using slackware ARM -current)
by shawn.kubik＠shaw.ca 22 Aug '24

22 Aug '24

I've spent the last week trying to rebuild Dovecot on my Raspberry Pi 3 (aarch64; running Slackware ARM -current) and am running into the following build error. I suspected it may be my recent update to Slackware but I've been able to build the same source tarball on an x86 system running slackware64-current without any problems, so it *seems* specific to aarch64. I've tried both building from the 2.3.21 (and 2.3.21.1) tarball(s) as well as pulling the latest code from github and they all fail similarly, though in different spots it appears. I am building from source instead of using the available Slackware packages bc those packages don't have support for postgresql built in, but has mysql/mariadb support, which I am not using and is not installed on my systems. While it does run, when I try to connect to it, it fails citing a missing mariadb.so file. gcc -v output: Reading specs from /usr/lib64/gcc/aarch64-slackware-linux-gnu/14.2.0/specs COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-slackware-linux-gnu/14.2.0/lto-wrapper Target: aarch64-slackware-linux-gnu Configured with: ../configure --with-arch=armv8-a --verbose --prefix=/usr --mandir=/usr/man --infodir=/usr/info --libdir=/usr/lib64 --enable-bootstrap --enable-checking=release --enable-libstdcxx-dual-abi --enable-shared --enable-languages=ada,c,c++,d,fortran,go,lto,m2,objc,obj-c++,rust --enable-objc-gc --enable-threads=posix --enable-__cxa_atexit --enable-gnu-unique-object --enable-clocale=gnu --enable-plugin --enable-lto --with-arch-directory= --with-system-zlib --with-gnu-ld --with-isl --with-default-libstdcxx-abi=new --disable-libunwind-exceptions --disable-libstdcxx-pch --disable-libssp --disable-werror --disable-gtktest --disable-install-libiberty --host=aarch64-slackware-linux-gnu --build=aarch64-slackware-linux-gnu --target=aarch64-slackware-linux-gnu Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 14.2.0 (GCC) ldd --version output: ldd (GNU libc) 2.40 Copyright (C) 2024 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper. Configure is: ./configure \ --with-pgsql \ --localstatedir=/var \ --with-lucene \ --with-libcap \ --with-inotify=inotify \ --disable-static \ --build=$ARCH-slackware-linux || exit 1 And the output at the point of failure is: libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/lib -I../../../../src/lib-test -I../../../../src/lib-settings -I../../../../src/lib-mail -I../../../../src/lib-imap -I../../../../src/lib-imap-client -I../../../../src/lib-index -I../../../../src/lib-storage -I../../../../src/lib-storage/list -I../../../../src/lib-storage/index -I../../../../src/lib-ssl-iostream -std=gnu99 -g -O2 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -MT imapc-sync.lo -MD -MP -MF .deps/imapc-sync.Tpo -c imapc-sync.c -fPIC -DPIC -o .libs/imapc-sync.o free(): double free detected in tcache 2 during GIMPLE pass: slp imapc-sync.c: In function ‘imapc_initial_sync_check’: imapc-sync.c:294:1: internal compiler error: Aborted 294 | imapc_initial_sync_check(struct imapc_sync_context *ctx, bool nooped) | ^~~~~~~~~~~~~~~~~~~~~~~~ 0x1b4a3df internal_error(char const*, ...) ???:0 0x7f80a139fc pthread_kill@@GLIBC_2.34 ???:0 0x7f809c419b gsignal ???:0 0x7f809abfa7 abort ???:0 0x7f80a06943 __libc_message_impl ???:0 0x7f80a1e2c7 malloc_printerr ???:0 0x7f80a2053b _int_free ???:0 0x7f80a22b7f __free ???:0 0xfa35d7 _slp_tree::~_slp_tree() ???:0 0xfa3aa7 vect_free_slp_instance(_slp_instance*) ???:0 0xfb9a7f vect_slp_analyze_operations(vec_info*) ???:0 0xfc06a7 vect_slp_function(function*) ???:0 Please submit a full bug report, with preprocessed source (by using -freport-bug). Please include the complete backtrace with any bug report. See <https://gcc.gnu.org/bugs/> for instructions. make[5]: *** [Makefile:597: imapc-sync.lo] Error 1 make[5]: Leaving directory '/data/local/src/Dovecot/dovecot-2.3.21/src/lib-storage/index/imapc' make[4]: *** [Makefile:733: all-recursive] Error 1 make[4]: Leaving directory '/data/local/src/Dovecot/dovecot-2.3.21/src/lib-storage/index' make[3]: *** [Makefile:1107: all-recursive] Error 1 make[3]: Leaving directory '/data/local/src/Dovecot/dovecot-2.3.21/src/lib-storage' make[2]: *** [Makefile:573: all-recursive] Error 1 make[2]: Leaving directory '/data/local/src/Dovecot/dovecot-2.3.21/src' make[1]: *** [Makefile:704: all-recursive] Error 1 make[1]: Leaving directory '/data/local/src/Dovecot/dovecot-2.3.21' make: *** [Makefile:548: all] Error 2 Can someone please help me figure this out? If more output is required let me know and I can provide it. Thank You!

2 2

limits to auth of submission
by horizon＠secubox.org 21 Aug '24

21 Aug '24

Hello I am using a plaintext db as the auth db. The db content is like this line: user@mail.com:{CRYPT}$****::::::userdb_quota_rule=*:bytes=1G How can I limit some people can auth with postfix submission (port 587), and some others can't? Thank you.

1 0

Seeking Suggestions for Optimizing Dovecot Performance
by leoniemeeyr＠gmail.com 21 Aug '24

21 Aug '24

Hi everyone, I hope you’re all doing well! I’ve been working with Dovecot for a while now and I’m looking to optimize its performance on my server. I’d love to hear any tips or suggestions you might have to help me get the best performance out of it. Specifically, I’m interested in: 1. Configuration Tweaks: Any key settings that you’ve found to be particularly beneficial for performance? 2. Best Practices: Are there any common practices or tools that help in monitoring and improving performance? 3. Resource Management: How do you handle resource allocation to ensure smooth operation under heavy loads? Any advice or experiences you can share would be greatly appreciated! Thanks in advance for your help. Best Regards https://www.igmguru.com/blog/msbi-vs-power-bi

1 0

dovecot-indexer writting as root
by Joan Moreau 20 Aug '24

20 Aug '24

Hi I have that issue times to times Full case described properly here : https://github.com/grosjo/fts-xapian/issues/168 Can you help understand the root cause of that ? THank you

3 6

SQL Multiple passwords
by Aubry 19 Aug '24

19 Aug '24

Hi, From what I understood from the archive and from my tests, we cannot have multiple passwords for a given account. (I get the error: Password query returned multiple matches) But it looks like it can be done via a PAM module. Does anyone succeeded setup multiple password with PAM or any other method with a SQL backend ? Regards, Aubry

6 7

Panic with FTS_BACKEND_FLAG_TOKENIZED_INPUT and virtual mailboxes: file index-search-result.c: line 132
by markus-dovecot＠doits.de 17 Aug '24

17 Aug '24

Originally posted and debugged here: https://github.com/slusarz/dovecot-fts-flatcurve/issues/66 I have to following panic with fts flatcurve enabled: ``` Panic: file index-search-result.c: line 132 (index_search_result_update_flags): assertion failed: (result->search_args->args == &search_arg) Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x74e9890a2846] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x74e9890a2962] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d81b) [0x74e9890af81b] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d8b7) [0x74e9890af8b7] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e2e2) [0x74e9890002e2] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x518a4) [0x74e9891d38a4] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x274e) [0x74e988bf8a9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x5c) [0x74e9891eff9c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x39) [0x74e9891f0039] -> dovecot/imap(cmd_select_full+0x1df) [0x62354bd8326f] -> dovecot/imap(command_exec+0xa4) [0x62354bd8b444] -> dovecot/imap(+0x22392) [0x62354bd89392] -> dovecot/imap(+0x22444) [0x62354bd89444] -> dovecot/imap(client_handle_input+0x1bd) [0x62354bd8989d] -> dovecot/imap(client_input+0x74) [0x62354bd89e64] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d) [0x74e9890c603d] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x13a) [0x74e9890c77aa] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x74e9890c60e4] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x74e9890c62a0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x74e989036ad7] -> dovecot/imap(main+0x570) [0x62354bd7ad20] -> /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x74e988c29d90] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x74e988c29e40] -> dovecot/imap(_start+0x25) [0x62354bd7ade5] ``` It happens when accessing virtual folder with filters like this: ``` something-*/excep-dev* header x-abc-exception "1" ``` The error happens on subsequent access of the folder: - Removing dovecot.index{,.log} files allows to access the virtual folder again - dovecot.index gets recreated on access and the folder can be accessed multiple times - Marking an e-mail read triggers the error, though! Now each next access raises the error - Removing dovecot.index{,.log} restores access again and the e-mails are still marked as read (so this information is not lost), until the next e-mail is marked as read: then the folder cannot be accessed again Slusarz debugged the backtrace and wrote a detailed description at https://github.com/slusarz/dovecot-fts-flatcurve/issues/66#issuecomment-229… which I'll quote here: > This is not a flatcurve bug - it is a bug within Dovecot's indexing/fts handling. > > (Detailed debug below, so that core team can understand the problem.) > > As discussed above, this bug is triggered with virtual mailboxes on subsequent accesses. The initial access will work fine; the assert crash only happens when the Dovecot lib-storage index code reads the indexes created for the virtual mailboxes. > > `index_search_result_update_appends()` in lib-storage/index/index-search-result.c > > * Inside this function, a "temporary search parameter to limit the search only to the new messages" is added to the mail search args. Later, it is checked that the mail_search args points to this same search parameter after the search is executed. It is this assert check that is failing, because the mail args is changing during the course of the search. > > * `i_assert(result->search_args->args == &search_arg);` > > > The search args are being altered in plugins/fts/fts-search.args.c, `fts_search_args_expand()`. Specifically, in that function, either `fts_search_args_expand_language_top_level()` or `fts_search_args_expand_tree()` is being called, which can replace the current args parameter with a new pointer. > > `fts_search_args_expand()` is being called in plugins//fts/fts-search.c by `fts_search_try_lookup()` when the FTS backend has the FTS_BACKEND_FLAG_TOKENIZED_INPUT flag set. Flatcurve has this flag set. > > This flag is defined in plugins/fts/fts-api-private.h. Comments say: > > ``` > /* Tokenize all the input. update_build_more() will be called a single > directly indexable token at a time. Searching will modify the search > args so that lookup() sees only tokens that can be directly > searched. */ > ``` > > Thus, the comments explicitly say the search args will be modified if this flag is set. This modification is incompatible with the assert check in `index_search_result_update_appends()`, which requires that the search args not be modified. > > The core team will have to determine whether this behavior is correct, or if the assert check is invalid. You can find some more discussion at https://github.com/slusarz/dovecot-fts-flatcurve/issues/66, but I think the most important part (the very detailed analysis from Slusarz, thanks again!) might be enough to understand and maybe fix the problem. Relevant information about the mail system: ``` # 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21 (f6cd4b8e) # OS: Linux 6.5.0-45-generic x86_64 Ubuntu 22.04.4 LTS [...] namespace { location = virtual:~/virtual:LAYOUT=maildir++ prefix = _/ separator = / } [...] plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared fts = flatcurve fts_autoindex = yes fts_filters = normalizer-icu snowball stopwords fts_filters_en = lowercase snowball stopwords fts_languages = de en fts_tokenizer_email_address = maxlen=100 fts_tokenizer_generic = algorithm=simple maxlen=30 fts_tokenizers = generic email-address sieve = file:~/sieve;active=~/.dovecot.sieve } [...] ``` I can provide more information if you need. Thanks for looking into it!

1 1

Re: Frequency Control for IMAP connection
by Joseph Tam 16 Aug '24

16 Aug '24

From: Corey H <coreyh(a)alwaysdata.net> > $ sudo tail -10000 /var/log/mail.log|grep **(a)**mail.com|wc -l > 3544 > > logs like this: > > Aug 15 19:54:10 mx dovecot: imap-login: Login: user=<**(a)**mail.com>, > method=PLAIN, rip=**, lip=**, mpid=4151830, TLS, > session=<UPotgLcfTOjYUzTN> > > How can I implement a Frequency Control on client connections to slow > down his requests? Is this a MacOSX client? I think they have a habit of doing this when doing global mailbox searches. Instead of doing a sane serial march through mailboxes, it opens up as many mailboxes as it can, then closes them when the IMAP server denies then when they hit the max. If Aki doesn't can't offer a solution, neither can I, but maybe you can play around with mail_max_userip_connections to stop them from swamping all your connections. You might have to resort to firewall or other general network controls to actually rate limit connections. Joseph Tam <jtam.home(a)gmail.com>

2 1