Dovecot crashing when attempting to search in virtual folder with fts_squat activated

Benjamin Godbersen benjamin at godbersen.info
Wed Mar 20 22:53:44 EET 2019


Thank you for pointing me to the source. Unfortunately I am running 
Plesk on my server and thus relying on the plesk-dovecot package to 
ensure the seamless integration with the rest of the packages.

Besides being behind a few months on the updates Plesk also still 
recommends using fts_squat 
(https://support.plesk.com/hc/en-us/articles/115000532574-The-search-speed-is-slow-in-Roundcube-or-Horde) 
so it seems I'm stuck here.

Thanks and cheers
Benjamin

Am 20.03.2019 um 21:40 schrieb Aki Tuomi via dovecot:
> We provide packages at https://repo.dovecot.org - maybe you can use 
> these?
>
> Aki
>> On 20 March 2019 22:19 Benjamin Godbersen via dovecot 
>> <dovecot at dovecot.org> wrote:
>>
>>
>> Hi Michael,
>>
>> thanks for getting back to me. I understand the alternatives to squat 
>> are solr and lucene and have to be explicitly included into dovecot 
>> at compilation. Unfortunately I am not in a position to manually 
>> compile dovecot.
>>
>> What other option do I have to use full text search in dovecot 
>> without manually compilating my package?
>>
>> Thanks and cheers
>> Benjamin
>>
>> Am 20.03.2019 um 20:31 schrieb Michael Slusarz via dovecot:
>>> fts_squat was deprecated in 2.1.  There's a high likelihood it is 
>>> buggy in a variety of ways in any recent Dovecot release.
>>>
>>> michael
>>>
>>>> On March 20, 2019 at 1:21 PM Benjamin Godbersen via dovecot 
>>>> <dovecot at dovecot.org> wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> I have now updated to dovecot 2.3.4.1 - unfortunately the issue 
>>>> still persists. Can anyone help me figure out if this is due to a 
>>>> misconfiguration on my part or another error?
>>>>
>>>> Any help is greatly appreciated!
>>>>
>>>> Cheers
>>>> Benjamin
>>>>
>>>> Am 14.03.2019 um 23:28 schrieb benjamin at godbersen.info 
>>>> <mailto:benjamin at godbersen.info>:
>>>>> Hi everyone,
>>>>>
>>>>> I am running into a problem when trying to use fts_squat in a 
>>>>> virtual folder. Without fts_squat plugin the search (from, 
>>>>> subject...) works in all folders. With activated fts the search on 
>>>>> the inbox folders works expectedly well but any attempt to search 
>>>>> anything in any virtual folder leads to the following error. 
>>>>> Similarly when attempting "doveadm fts lookup". I also noticed 
>>>>> that no search index for the virtual folders gets build - is this 
>>>>> expected behaviour?
>>>>>
>>>>>     Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1].
>>>>>     Panic: file mail-storage.c: line 1913
>>>>>     (mailbox_get_open_status): assertion failed: (box->opened)
>>>>>     Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1].
>>>>>     Error: Raw backtrace:
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(+0xba731)
>>>>>     [0x7f553a7ff731] ->
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(+0xba7fa)
>>>>>     [0x7f553a7ff7fa] ->
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(i_fatal+0)
>>>>>     [0x7f553a771638] ->
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot-storage.so.0(mailbox_get_open_status+0x68)
>>>>>     [0x7f553aae4a78] ->
>>>>>     /usr/lib/dovecot/modules/lib21_fts_squat_plugin.so(+0x3684)
>>>>>     [0x7f553677a684] ->
>>>>>     /usr/lib/dovecot/modules/lib21_fts_squat_plugin.so(+0x3820)
>>>>>     [0x7f553677a820] ->
>>>>>     /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_lookup_multi+0x163)
>>>>>     [0x7f5539b016a3] ->
>>>>>     /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd728)
>>>>>     [0x7f5539b06728] ->
>>>>>     /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_lookup+0xeb)
>>>>>     [0x7f5539b06bbb] ->
>>>>>     /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xf8b8)
>>>>>     [0x7f5539b088b8] -> dovecot/imap(imap_search_start+0x6a)
>>>>>     [0x5654cb5a0d6a] -> dovecot/imap(cmd_sort+0x293)
>>>>>     [0x5654cb593553] -> dovecot/imap(command_exec+0x64)
>>>>>     [0x5654cb599874] -> dovecot/imap(+0x1bd22) [0x5654cb597d22] ->
>>>>>     dovecot/imap(+0x1bdbc) [0x5654cb597dbc] ->
>>>>>     dovecot/imap(client_handle_input+0x1b5) [0x5654cb5981c5] ->
>>>>>     dovecot/imap(client_input+0xa4) [0x5654cb5987e4] ->
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_call_io+0x69)
>>>>>     [0x7f553a8174a9] ->
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e)
>>>>>     [0x7f553a818d1e] ->
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c)
>>>>>     [0x7f553a8175ac] ->
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_run+0x38)
>>>>>     [0x7f553a8177b8] ->
>>>>>     /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(master_service_run+0x13)
>>>>>     [0x7f553a7940a3] -> dovecot/imap(main+0x339) [0x5654cb58a539]
>>>>>     -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)
>>>>>     [0x7f553a375b97] -> dovecot/imap(_start+0x2a) [0x5654cb58a71a]
>>>>>     Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1].
>>>>>     Fatal: master: service(imap): child 6436 killed with signal 6
>>>>>     (core dumps disabled)
>>>>>
>>>>> This is my config:
>>>>>
>>>>>     # 2.3.0.1 (ffd8a29): /etc/dovecot/dovecot.conf
>>>>>     # Pigeonhole version 0.5.0.1 (d33dca20)
>>>>>     # OS: Linux 4.15.0-46-generic x86_64 Ubuntu 18.04.2 LTS ext4
>>>>>     auth_mechanisms = plain login digest-md5 cram-md5 apop
>>>>>     auth_username_chars =
>>>>>     abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890&.-_@'
>>>>>     default_vsz_limit = 8096 M
>>>>>     disable_plaintext_auth = no
>>>>>     first_valid_uid = 30
>>>>>     imap_client_workarounds = delay-newmail
>>>>>     imap_logout_format = rcvd=%i, sent=%o
>>>>>     mail_home = /var/qmail/mailnames/%Ld/%Ln
>>>>>     mail_location = maildir:/var/qmail/mailnames/%Ld/%Ln/Maildir
>>>>>     mail_log_prefix = "service=%s, user=%u, ip=[%r]. "
>>>>>     mail_max_userip_connections = 100
>>>>>     mail_plugins = quota fts fts_squat virtual
>>>>>     managesieve_logout_format = rcvd=%i, sent=%o
>>>>>     managesieve_notify_capability = mailto
>>>>>     managesieve_sieve_capability = fileinto reject envelope
>>>>>     encoded-character vacation subaddress comparator-i;ascii-numeri$
>>>>>     namespace inbox {
>>>>>       inbox = yes
>>>>>       location =
>>>>>       prefix = INBOX.
>>>>>       separator = .
>>>>>     }
>>>>>     namespace virtual {
>>>>>       hidden = no
>>>>>       inbox = no
>>>>>       list = yes
>>>>>       location =
>>>>>     virtual:/var/qmail/mailnames/%Ld/%Ln/virtual/:INDEX=/var/qmail/mailnames/%Ld/%Ln/virtual/
>>>>>       prefix =
>>>>>       separator = .
>>>>>     }
>>>>>     passdb {
>>>>>       driver = plesk
>>>>>     }
>>>>>     plugin {
>>>>>       fts = squat
>>>>>       fts_squat = partial=4 full=10
>>>>>       quota = maildir:User quota
>>>>>       quota_grace = 0
>>>>>       sieve = ~/.dovecot.sieve
>>>>>       sieve_dir = ~/sieve
>>>>>       sieve_extensions = +notify +imapflags
>>>>>     }
>>>>>     pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
>>>>>     pop3_logout_format = rcvd=%i, sent=%o, top=%t/%p, retr=%r/%b,
>>>>>     del=%d/%m, size=%s
>>>>>     protocols = imap pop3 sieve
>>>>>     service auth-worker {
>>>>>       group =
>>>>>       user =
>>>>>     }
>>>>>     service auth {
>>>>>       group =
>>>>>       unix_listener auth-userdb {
>>>>>         group = popuser
>>>>>         mode = 0600
>>>>>         user = popuser
>>>>>       }
>>>>>       user =
>>>>>     }
>>>>>     service imap-login {
>>>>>       process_limit = 2048
>>>>>       service_count = 1
>>>>>     }
>>>>>     service imap {
>>>>>       process_limit = 2048
>>>>>       service_count = 1
>>>>>     }
>>>>>     service pop3-login {
>>>>>       process_limit = 2048
>>>>>       service_count = 1
>>>>>     }
>>>>>     service pop3 {
>>>>>       process_limit = 2048
>>>>>       service_count = 1
>>>>>     }
>>>>>     ssl_cert = </etc/dovecot/private/dovecot.pem
>>>>>     ssl_cipher_list = HIGH:!aNULL:!MD5
>>>>>     ssl_dh =  # hidden, use -P to show it
>>>>>     ssl_key =  # hidden, use -P to show it
>>>>>     ssl_prefer_server_ciphers = yes
>>>>>     userdb {
>>>>>       args = uid=popuser gid=popuser
>>>>>       driver = static
>>>>>     }
>>>>>     protocol imap {
>>>>>       mail_plugins = quota fts fts_squat virtual imap_quota
>>>>>     }
>>>>>     protocol pop3 {
>>>>>       pop3_uidl_format = UID%u-%v
>>>>>     }
>>>>>     protocol sieve {
>>>>>       mail_plugins = quota fts fts_squat virtual
>>>>>     }
>>>>>     protocol lda {
>>>>>       mail_plugins = quota fts fts_squat virtual sieve
>>>>>     }
>>>>>
>>>>> dovecot --version -> 2.3.0.1 (ffd8a29)
>>>>>
>>>>> I will be greatful for any tips hinting me on how to resolve this 
>>>>> behaviour. Thank you very much in advance
>>>>> Benjamin
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
> ---
> Aki Tuomi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20190320/dd5082ee/attachment.html>


More information about the dovecot mailing list