- dovecot - dovecot.org

rawlog data in a lua script
by michael.zork＠feierfighter.de 08 Aug '22

08 Aug '22

Hi, I'm searching for a possibility to have the rawlog feature in lua, which would be much easier for processing. Currently Dovecot, when activating rawlog for a user, writes everything to disk (which creates I/O), and I have to somehow read it from there. That's a bit complicated, because I have to get notified via inotify or similar when there are new files created, and then I have to start a "tail" or "epoll" mechanism on the files to get the contents in more or less real time (IMAP sessions can be multiple hours or days). It would be much easier to hook to the "raw request and response events" inside Dovecot and have the rawlog-data in a lua script, where I can prepare it and send it to another maschine for monitoring/collection/analysis/statistics or similar, for example via HTTP. Having the rawlog data available in lua would make things a lot easier. Is there any possibility at the moment to create a lua script and "hook" to those "request and response events"? If not, would it be possible to add that feature in the future? Kind regards Michael

5 8

RFC 9266: Channel Bindings for TLS 1.3 support
by * Neustradamus * 08 Aug '22

08 Aug '22

Hello all, I would like to know if it is possible to add RFC 9266: Channel Bindings for TLS 1.3? - https://datatracker.ietf.org/doc/html/rfc9266 Little details, to know easily: - tls-unique for TLS =< 1.2 - tls-exporter for TLS = 1.3 It is will be used by SCRAM-SHA-*-PLUS like SCRAM-SHA-1-PLUS/SCRAM-SHA-256-PLUS. SCRAM-SHA-1 and SCRAM-SHA-256 are already supported. Thanks in advance. Regards, Neustradamus

2 1

Re: RHEL9 Repository
by justina colmena ~biz 07 Aug '22

07 Aug '22

On August 5, 2022 3:30:57 PM AKDT, Peter <peter(a)pajamian.dhs.org> wrote: >The main site doesn't currently support https but the repositories do, also all packages are cryptographically signed and the signing keys are served off of a secure server. > >The info on the site is public information that doesn't really need to be secure. > In which case any actual content on the said site becomes injected with ultra-persistent linux-targeted adware, spyware, and pop-ups by any given third party in transit, which craps up my phone and slows down my desktop browser to a crawl, and then I have to update my adblocker, re-up all my security settings and fix all the other things that break due to spam malicious advertising on the internet. Plain old http is simply maddening these days. Leave the front door wide open for online hustlers and thieves, yeah, some people have bank accounts or manage actual money on their computers. I would highly encourage a use of basic https all around: certbot/letsencrypt is currently free and there are many other low-cost options for https in conjunction with any given hosting service or platform. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

2 1

RHEL9 Repository
by Peter 06 Aug '22

06 Aug '22

For those who have been asking, GhettoForge 9 is now released with dovecot23 packages for all EL9 distributions in the gf-plus repository. These are built against Rocky Linux 9 and should install and run on any EL9 distro including, but not limited to: * Rocky Linux 9 * Red Hat Enterprise Linux 9 * Oracle Linux 9 * Alma Linux 9 * Scientific Linux 9 ...and more This provides the latest stable version of dovecot-2.3.19.1 Please see the instructions at the following link for how to install and run packages from the gf-plus repository: http://ghettoforge.org/index.php/Usage ...and let me know if you have any difficulties or questions with these packages. Peter Ajamian

2 2

variable %w recursive expanding
by Franz Beslmeisl 02 Aug '22

02 Aug '22

In order to change the password scheme I wrote a script named updateproxy that needs the plain text password from the user. To get that I use the line password_query = SELECT username as user, password, \ '%w' as userdb_plain_pass FROM auth_user WHERE username='%n' This works nicely with almost all passwords but not with this one 1234567%&/abcd the error message being dovecot: Failed to expand plugin setting plain_pass = '1234567%&/abcd': Unknown variable '%&' It seems to me that dovecot tries to do another level of variable evaluation upon the **value** of the already evaluated variable. So I searched for ways to escape problematic characters like % and changed my line to password_query = SELECT username as user, password, \ '%E{w}' as userdb_plain_pass FROM auth_user WHERE username='%n' but this produces problems with password values containing quotes. So how can I get a plain text password containing any ascii char (or even better any utf-8 char) safely to my script? Thanks for your suggestions -------------- here the nasty details, if you want ------------- $ dovecot -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-122-generic x86_64 Ubuntu 20.04.4 LTS # Hostname: mx-10-2.bildung.hessen.de auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyz_0123456789.ABCDEFGHIJKLMNOPQRSTUVWXYZ-@ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { # the following file contains a '%w'-line args = /etc/dovecot/db1.conf driver = sql } passdb { # the following file contains a '%w'-line args = /etc/dovecot/db2.conf driver = sql } passdb { # the following file contains no '%w'-line (just for detail) args = /etc/dovecot/db3.conf driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_max_actions = 64 sieve_max_redirects = 16 sieve_max_script_size = 10M sieve_trace_debug = yes sieve_user_log = ~/sievelog sieve_vacation_dont_check_recipient = yes sieve_vacation_use_original_recipient = yes } protocols = imap sieve lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service imap { executable = imap after-login } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service after-login { executable = script-login /etc/dovecot/updateproxy user = vmail } service stats { unix_listener stats-reader { group = mail mode = 0666 } unix_listener stats-writer { group = mail mode = 0666 } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_cipher_list = ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP:-SSLv3 ssl_key = # hidden, use -P to show it ssl_min_protocol = SSLv3 ssl_prefer_server_ciphers = yes userdb { args = uid=vmail gid=vmail home=/var/vmail/%n driver = static } protocol lmtp { mail_plugins = quota sieve postmaster_address = somebody(a)somwhere.org } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address = somebody(a)somehwere.org quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 300 }

3 3

Dovecot-lda mysql look ups.
by Simon B 02 Aug '22

02 Aug '22

Hallo again. Dovecot is not doing a look up for the MailDirLoc in the mysql database after postfix hands over the mail to dovecot-lda. Postfix knows the right location # postmap -q user(a)domain.net mysql:/etc/postfix/Mail-Mailbox.cf domain.com/user/ But apparently this has not worked as intended since I moved from courier to dovecot some years back. The minute I change /etc/dovecot/dovecot-sql.conf postfix starts spitting SASL authentication errors. How can I make Dovecot-LDA query the DB in that file for the delivery location (userdb_mail)? Grateful for any help :) Regards Simon

1 0

tika 2.4.1 'Text extraction failed' errors when dovecot+fts 2.3.19.1 passes embedded *.eml (message/rfc822) files ; org.apache.tika.parser.mail.RFC822Parser or dovecot ?
by PGNet Dev 01 Aug '22

01 Aug '22

i'm running dovecot 2.3.19.1 + fts tika-server-standard 2.4.1 dovecot is feeding tika backend via fts_tika when dovecot passes data with *.eml attachments embedded, tika fails to correctly parse/extract content not clear if the issue is with tika, or what dovecot's passing in this case. other non-.eml attachments are fine. here's the current failing procedure, (1) create a simple pdf enscript -p mime.ps /etc/mime.types ps2pdf mime.ps mime.pdf (2) send an email *with* mime.pdf attachment to echo "test" | mailx -s "test" -a ./mime.pdf testuser(a)example.com tika processes OK journalctl -f -u tika ... Jul 30 19:09:24 mx-test tika[19682]: INFO [qtp2112135199-30] 19:09:24,165 org.apache.tika.server.core.resource.TikaResource /tika (application/pdf) ... save the just-received email with .pdf attachment as mime.eml (3) send an email with NO .pdf attachment save the just-received email with .pdf attachment as mime2.eml (4) send an email with mime.eml attachment, containing the embedded mime.pdf echo "test" | mailx -s "test" -a ./mime.eml testuser(a)example.com tika fails to extract message/rfc822 journalctl -f -u tika | grep -v StatusLogger ... Jul 30 19:28:00 mx-test tika[20049]: INFO [qtp2112135199-30] 19:28:00,834 org.apache.tika.server.core.resource.TikaResource /tika (message/rfc822) Jul 30 19:28:00 mx-test tika[20049]: WARN [qtp2112135199-30] 19:28:00,840 org.apache.tika.server.core.resource.TikaResource tika/: Text extraction failed (mime.eml) Jul 30 19:28:00 mx-test tika[20049]: org.apache.tika.exception.ZeroByteFileException: InputStream must have > 0 bytes Jul 30 19:28:00 mx-test tika[20049]: at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:153) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.tika.parser.ParserDecorator.parse(ParserDecorator.java:152) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.tika.parser.DigestingParser.parse(DigestingParser.java:55) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.tika.server.core.resource.TikaResource.parse(TikaResource.java:352) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.tika.server.core.resource.TikaResource.lambda$produceText$1(TikaResource.java:502) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.jaxrs.provider.BinaryDataProvider.writeTo(BinaryDataProvider.java:177) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.jaxrs.utils.JAXRSUtils.writeMessageBody(JAXRSUtils.java:1616) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.serializeMessage(JAXRSOutInterceptor.java:249) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.processResponse(JAXRSOutInterceptor.java:122) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.handleMessage(JAXRSOutInterceptor.java:84) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:90) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:247) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:79) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:00 mx-test tika[20049]: at java.lang.Thread.run(Thread.java:833) ~[?:?] Jul 30 19:28:00 mx-test tika[20049]: ERROR [qtp2112135199-30] 19:28:00,845 org.apache.cxf.jaxrs.utils.JAXRSUtils Problem with writing the data, class org.apache.tika.server.core.resource.TikaResource$$Lambda$338/0x0000000800eb4a38, ContentType: text/plain (5) send an email with mime2.eml attachment, WITHOUT an embedded .pdf echo "test" | mailx -s "test" -a ./mime.eml testuser(a)example.com again, tika fails to extract message/rfc822 journalctl -f -u tika | grep -v StatusLogger ... Jul 30 19:28:33 mx-test tika[20049]: INFO [qtp2112135199-30] 19:28:33,607 org.apache.tika.server.core.resource.TikaResource /tika (message/rfc822) Jul 30 19:28:33 mx-test tika[20049]: WARN [qtp2112135199-30] 19:28:33,616 org.apache.tika.server.core.resource.TikaResource tika/: Text extraction failed (mime2.eml) Jul 30 19:28:33 mx-test tika[20049]: org.apache.tika.exception.ZeroByteFileException: InputStream must have > 0 bytes Jul 30 19:28:33 mx-test tika[20049]: at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:153) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.tika.parser.ParserDecorator.parse(ParserDecorator.java:152) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.tika.parser.DigestingParser.parse(DigestingParser.java:55) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.tika.server.core.resource.TikaResource.parse(TikaResource.java:352) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.tika.server.core.resource.TikaResource.lambda$produceText$1(TikaResource.java:502) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.jaxrs.provider.BinaryDataProvider.writeTo(BinaryDataProvider.java:177) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.jaxrs.utils.JAXRSUtils.writeMessageBody(JAXRSUtils.java:1616) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.serializeMessage(JAXRSOutInterceptor.java:249) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.processResponse(JAXRSOutInterceptor.java:122) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.handleMessage(JAXRSOutInterceptor.java:84) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:90) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:247) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:79) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) ~[tika-server-standard-2.4.1.jar:2.4.1] Jul 30 19:28:33 mx-test tika[20049]: at java.lang.Thread.run(Thread.java:833) ~[?:?] Jul 30 19:28:33 mx-test tika[20049]: ERROR [qtp2112135199-30] 19:28:33,630 org.apache.cxf.jaxrs.utils.JAXRSUtils Problem with writing the data, class org.apache.tika.server.core.resource.TikaResource$$Lambda$338/0x0000000800eb4a38, ContentType: text/plain (6) submit mime.eml directly to tika curl -T ./mime.eml http://127.0.0.1:9998/tika journalctl -f -u tika | grep -v StatusLogger ... Jul 30 19:30:08 mx-test tika[20049]: INFO [qtp2112135199-34] 19:30:08,073 org.apache.tika.server.core.resource.TikaResource /tika (autodetecting type) (7) submit mime2.eml directly to tika curl -T ./mime2.eml http://127.0.0.1:9998/tika journalctl -f -u tika | grep -v StatusLogger ... Jul 30 19:30:52 mx-test tika[20049]: INFO [qtp2112135199-30] 19:30:52,349 org.apache.tika.server.core.resource.TikaResource /tika (autodetecting type) (8) where, cat mime.eml Return-Path: <msmtp(a)pgnd.example.com> Delivered-To: testuser(a)example.com ... From: msmtp(a)pgnd.example.com Date: Sat, 30 Jul 2022 18:53:38 -0400 To: testuser(a)example.com Subject: test User-Agent: Heirloom mailx 12.5 7/5/10 Content-Type: multipart/mixed; boundary="=_62e5b672.wAyBX+sGMbS7ZcNv8O/A1QeYuseaJ2NDRf8hfdbm/x8Vayp+" Message-Id: <4LwKS35QWSzWf3Q(a)mx-test.example.com> This is a multi-part message in MIME format. --=_62e5b672.wAyBX+sGMbS7ZcNv8O/A1QeYuseaJ2NDRf8hfdbm/x8Vayp+ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline test --=_62e5b672.wAyBX+sGMbS7ZcNv8O/A1QeYuseaJ2NDRf8hfdbm/x8Vayp+ Content-Type: application/pdf Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="mime.pdf" JVBERi0xLjQKJcfsj6IKJSVJbnZvY2F0aW9uOiBwYXRoL2dzIC1QLSAtZFNBRkVSIC1kQ29t ... Rgo= --=_62e5b672.wAyBX+sGMbS7ZcNv8O/A1QeYuseaJ2NDRf8hfdbm/x8Vayp+-- and, cat mime2.eml Return-Path: <msmtp(a)pgnd.example.com> Delivered-To: testuser(a)example.com ... From: msmtp(a)pgnd.example.com Date: Sat, 30 Jul 2022 19:14:59 -0400 To: testuser(a)example.com Subject: test User-Agent: Heirloom mailx 12.5 7/5/10 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <4LwKwh5brVzWf3Q(a)mx-test.example.com> test

2 2

Re: Dovecot folder problem while deleting via IMAP
by Christian Mack 29 Jul '22

29 Jul '22

Hello Am 29.07.22 um 10:42 schrieb Pascal Bonny: > Dovecot Version: > 2.3.17 (8414c38c5c) > > Dear, > Some troubles occur while we delete a folder via IMAP ssl command or via a IMAP > client (different clients tested). > > When we want to delete subfolder we observe following issues. > > > some examples following: > > we created a folder with subfolders x/y/z > > <IMAP command create> > a02 CREATE x.y.z > a02 OK Create completed (0.003 + 0.000 + 0.002 secs). > <IMAP command create> > > <doveadm mailbox list -u "user(a)domain.tld"> > ====response from dovecot cli==== > x > x.y > x.y.z > > then we try to delete the z subfolder of x > > <IMAP command delete> > a04 DELETE x.y.z > a05 OK Delete completed (0.002 + 0.000 + 0.001 secs). > <IMAP command delete> > > The IMAP console returns success and the dovecot listing is ok > > <doveadm mailbox list -u "user(a)domain.tld"> > ====response from dovecot cli ===== > x > x.y > > But dovecot status on x.y returns the folder is not existent > [cut] That is normal IMAP behaviour. With creating x.y.z you really create only folder z and a folder hirarchy x.y, in which you can not store emails (nor have any access). You have to create x and x.y, in order to get complete folders and subfolders usable for emails and be able to delete them :-) Kind regards, Christian Mack -- Christian Mack Universität Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Abteilung IT-Dienste Forschung, Lehre, Infrastruktur 78457 Konstanz +49 7531 88-4416

1 0

large search indexer tasks, submitted to flatcurve+tika+tesseract backend for attachment scanning, timeout even with "fts_index_timeout = 0"; how to increase/remove timeouts?
by PGNet Dev 28 Jul '22

28 Jul '22

i'm running dovecot 2.3.19.1 search indexing is enabled, via fts_flatcurve (dovecot23_fts_flatcurve) attachment processing is via fts_tika, with tika-server-standard-2.4.2 backend & tesseract5 OCR i'm working getting this optimized on a smaller, 4-core server that opt includes capping tesseract @ single-thread, Environment=OMP_THREAD_LIMIT=1 still, some attachments are OCR-scanned, and can take some time. when i exec large reindex jobs, i get occassional timeout errors on dovecot's indexer-worker connection to tiks backend, e.g., 2022-07-23 09:54:43 indexer-worker(postmaster(a)example.com)<DIcjEWb922JhXAAA+IOfAw>: Error: fts_tika: PUT http://127.0.0.1:9998/tika/ failed: Request timed out (Request queued 61.031 secs ago, 1 send attempts in 60.103 secs, 60.080 in http ioloop, 0.000 in other ioloops, connected 60.103 secs ago) 2022-07-23 09:54:43 indexer-worker(postmaster(a)example.com)<DIcjEWb922JhXAAA+IOfAw>: Error: Mailbox Sent: Precache for UID=90782 failed: Internal error occurred. Refer to server log for more information. [2022-07-23 09:54:43] (attempted to index 2 messages between UIDs 90778..90782) i don't see any fts timeout info @ https://wiki.dovecot.org/Timeouts here https://doc.dovecot.org/settings/plugin/fts-plugin/#plugin_setting-fts-fts_… " fts_index_timeout Default: 0 Values: Unsigned integer When the full text search backend detects that the index isn’t up-to-date, the indexer is told to index the messages and is given this much time to do so. If this time limit is reached, an error is returned, indicating that the search timed out during waiting for the indexing to complete: NO [INUSE] Timeout while waiting for indexing to finish A value of 0 means no timeout. " double-checking my config, /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf -n submission_relay_connect_timeout = 10 secs submission_timeout = 10 secs there's no tika/fts timeout override, so should be "0" by default. i.e., no timeout @ Patch: enhancements for solr/tika integration https://dovecot.org/list/dovecot/2022-January/123828.html a patch was proposed for similar issue with fts-solr indexer (not sure if it's in current release AND effective for my config), but that *limits* the size of attachments submitted. i _do_ want to index all/large attachments, so, iiuc, need to ensure no, or longer, timeout. where do I set that timeout to not fail, as above, on large index tasks?

2 2

RHEL9 Latest Repo?
by dovecot＠ptld.com 28 Jul '22

28 Jul '22

Any plans or timeline for when there will be a latest repo for RHEL9? [dovecot-2.3-latest] name=Dovecot 2.3 RHEL $releasever - $basearch baseurl=http://repo.dovecot.org/ce-2.3-latest/rhel/$releasever/RPMS/$basear… gpgkey=https://repo.dovecot.org/DOVECOT-REPO-GPG gpgcheck=1 enabled=1

3 2