November 2020 - dovecot

Dovecot alternatives with good i18n support?
by David Bürgin 26 Nov '20

26 Nov '20

Hello! I’m interested in exploring alternative IMAP servers that have good support for internationalised email, such as support for SMTPUTF8, IDN mail domains, Unicode local-part, etc. etc. If you used to use Dovecot, and found an alternative that does such things well, I would very much like to hear it. Thank you, -- David

2 2

Dovecot+Samba AD - authentication failure
by Odhiambo Washington 26 Nov '20

26 Nov '20

Hi, I have setup samba4 as AD and hoping to have dovecot authenticate users against it. I am facing challenges though and I am unable to figure it out. I could do with a third eye to help me spot what is wrong. root@adc0:/etc# doveadm auth test -x service=imap odhiambo(a)newideatest.local Password: passdb: odhiambo(a)newideatest.local auth failed extra fields: temp Warning: auth-client: conn unix:/var/run/dovecot/auth-client: Auth connection closed with 1 pending requests (max 0 secs, pid=10537, EOF) Fatal: Couldn't connect to auth socket A test against IMAP gives the following debug information: Nov 22 14:31:01 auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Nov 22 14:31:01 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/lib20_auth_var_expand_crypt.so Nov 22 14:31:01 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Nov 22 14:31:01 auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Nov 22 14:31:01 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libauthdb_ldap.so Nov 22 14:31:01 auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat Nov 22 14:31:01 auth: Debug: auth client connected (pid=10979) Nov 22 14:31:08 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=uPLvabC0RIh/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=34884 resp=<hidden> Nov 22 14:31:08 auth: Debug: ldap(odhiambo(a)newideatest.local,127.0.0.1,<uPLvabC0RIh/AAAB>): Performing passdb lookup Nov 22 14:31:08 auth: Debug: ldap(odhiambo(a)newideatest.local,127.0.0.1,<uPLvabC0RIh/AAAB>): bind search: base=cn=Users,dc=NEWIDEATEST,dc=LOCAL filter=(&(objectClass=user)(!(userAccountControl:1.2.840.113556.1.4.803:=2))(sAMAccountName=odhiambo@newideatest.local )) Nov 22 14:31:08 auth: Debug: ldap(odhiambo(a)newideatest.local,127.0.0.1,<uPLvabC0RIh/AAAB>): no fields returned by the server *< ====================* Nov 22 14:31:08 auth: Debug: ldap(odhiambo(a)newideatest.local,127.0.0.1,<uPLvabC0RIh/AAAB>): Finished passdb lookup Nov 22 14:31:08 auth: Debug: auth(odhiambo(a)newideatest.local,127.0.0.1,<uPLvabC0RIh/AAAB>): Auth request finished Nov 22 14:31:10 auth: Debug: client passdb out: FAIL 1 user=odhiambo(a)newideatest.local info.log: Nov 22 14:31:08 auth: Info: ldap(odhiambo(a)newideatest.local ,127.0.0.1,<uPLvabC0RIh/AAAB>):* unknown user* (given password: XXXXXXX) Nov 22 14:31:15 imap-login: Info: Aborted login (auth failed, 1 attempts in 7 secs): user=<odhiambo(a)newideatest.local>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session=<uPLvabC0RIh/AAAB> Here is my doveconf -n: https://paste.ubuntu.com/p/SPmrxZxHPx/ My dovecot-ldap.cont.ext: uris = ldap://localhost/ dn = "dovecot(a)newideatest.local" dnpass = "XXXXXXXX" sasl_bind = no tls = no ldap_version = 3 deref = never scope = subtree base = cn=Users,dc=NEWIDEATEST,dc=LOCAL auth_bind = yes user_filter = (&(objectClass=user)(!(userAccountControl:1.2.840.113556.1.4.803:=2))(|(mail=%u)(sAMAccountName=%u)(otherMailbox=%u))) user_attrs = sAMAccountName=user,userPassword=password,=mail=maildir:/home/%n/Maildir/ pass_filter = (&(objectClass=user)(!(userAccountControl:1.2.840.113556.1.4.803:=2))(sAMAccountName=%u)) pass_attrs = sAMAccountName=user,userPassword=password The use exists in the database: *root@adc0:/var/log/dovecot# samba-tool user show odhiambo* ldb_wrap open of secrets.ldb dn: CN=Odhiambo Washington,CN=Users,DC=newideatest,DC=local objectClass: top objectClass: person objectClass: organizationalPerson objectClass: user cn: Odhiambo Washington sn: Washington givenName: Odhiambo instanceType: 4 whenCreated: 20201120101420.0Z displayName: Odhiambo Washington uSNCreated: 4086 name: Odhiambo Washington objectGUID: e6969596-8b28-41af-b5d8-cea63cc97f98 badPwdCount: 0 codePage: 0 countryCode: 0 badPasswordTime: 0 lastLogoff: 0 lastLogon: 0 primaryGroupID: 513 objectSid: S-1-5-21-701866827-3355127779-3787685610-1106 accountExpires: 9223372036854775807 logonCount: 0 sAMAccountName: odhiambo sAMAccountType: 805306368 userPrincipalName: odhiambo(a)newideatest.local objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=newideatest,DC=local mail: odhiambo(a)newideatest.local loginShell: /bin/bash userAccountControl: 512 pwdLastSet: 132505181852397220 whenChanged: 20201122112945.0Z uSNChanged: 4104 distinguishedName: CN=Odhiambo Washington,CN=Users,DC=newideatest,DC=local -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

2 3

Panic: file istream-seekable.c: line 226
by Eric Mauger 26 Nov '20

26 Nov '20

Hi, I have a lot of Panic in my logs. They happen almost every day and never on the same user. When it happens, the mail is not distributed (it's always lda). I discovered it because my Sympa server had a lot of users in error. The last error (below) was a local user sending a mail to 10 users : 3 of them never get the email. I've looked into my logs and discovered it started after the upgrade from 2.2.27 to 2.3.4 in february. I'm on Debian. This week, I installed the dovecot repo and upgraded to the very last and same thing. The error below happened after the latest upgrade. An example : Nov 25 11:33:07 mx.igf.cnrs.fr dovecot: lda(username_hidden)<4653><A3S4MuIyvl8tEgAAmSH/vg>: Panic: file istream-seekable.c: line 230 (read_from_buffer): assertion failed: (*ret_r > 0) Nov 25 11:33:07 mx.igf.cnrs.fr dovecot: lda(sebastien.granier)<4653><A3S4MuIyvl8tEgAAmSH/vg>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7f49a56d4eed] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f49a56d500e] -> /usr/lib/dovecot/libdovecot.so.0(+0xf465b) [0x7f49a56de65b] -> /usr/lib/dovecot/libdovecot.so.0(+0xf46c1) [0x7f49a56de6c1] -> /usr/lib/dovecot/libdovecot.so.0(+0x514a6) [0x7f49a563b4a6] -> /usr/lib/dovecot/libdovecot.so.0(+0x53cfb) [0x7f49a563dcfb] -> /usr/lib/dovecot/libdovecot.so.0(+0x1074a1) [0x7f49a56f14a1] -> /usr/lib/dovecot/libdovecot.so.0(+0x1078f6) [0x7f49a56f18f6] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_stat+0x34) [0x7f49a56eaab4] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb1396) [0x7f49a5852396] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb1472) [0x7f49a5852472] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_physical_size+0x38) [0x7f49a57ef0c8] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x4005) [0x7f49a5907005] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver_init+0x141) [0x7f49a5908101] -> /usr/lib/dovecot/deliver(main+0x879) [0x562d1c0e6f99] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f49a543e09b] -> /usr/lib/dovecot/deliver(_start+0x2a) [0x562d1c0e76ea] My configuration : dovecot -n # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.11 (6c69c917) # OS: Linux 4.19.0-9-amd64 x86_64 Debian 10.6 # Hostname: mx.igf.cnrs.fr auth_username_format = disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_location = maildir:~/maildir mail_plugins = fts mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Ads { auto = subscribe } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . type = private } passdb { driver = pam } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/dovecot/sieve.d } protocols = " imap sieve pop3 sieve" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { process_min_avail = 16 service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service stats { unix_listener stats-reader { group = dovecot mode = 0666 user = root } unix_listener stats-writer { group = dovecot mode = 0666 user = root } } ssl_cert = </etc/dovecot/ssl/certs/mx_igf_cnrs_fr_with_CA.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it syslog_facility = local2 userdb { driver = passwd } verbose_proctitle = yes protocol lda { mail_plugins = fts sieve syslog_facility = local2 } Thanks, -- Eric Mauger | Administrateur système et réseau IGF UMR5203 | https://www.igf.cnrs.fr

1 0

Dovecot+Solr+Tika segmentation fault
by Manuel Josupeit-Walter 25 Nov '20

25 Nov '20

Hi all, I am running a mail server for family and friends based on Dovecot and Docker. Today I made a weird observation which I must have overlooked for a long time so far because they logs were written to the journal. However, I am usually only investigating the Docker logs. It seems that some e-mail indexing leads to segfaults and it seems to be related only with Solr and Tika. I have auto-indexing turned on so sometimes the indexer-worker crashes when moving mails from one folder to the other or searching for mails otherwise. Also, if I try to manually reindex using doveadm index, the doveadm process quits with a segfault. I am always running the most recent Dovecot version, in this case 2.3.11.3, but I am not neccessarily up-to-date with Solr or Tika. I am on Solr Version 7.4 and Tika 1.24.1. On the console I am receiving Tika errors and on the Solr Webinterface I see a couple of Ealry EOFs as well. I am not a Linux expert from a dev perspective, but it seems to me that the client (Dovecot) closes the connection prematurely. Does anyone have a hint? I am happy to provide more information if required. I need to know how to collect them, though. Best, Manuel The following can be found in the journald logs during the attempt to reindex my Sent mailbox with doveadm index: Nov 24 16:59:49 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:49 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name tika[988]: WARN Using fallback font LiberationSans for Helvetica Nov 24 16:59:50 server.name kernel: doveadm[400]: segfault at 0 ip 00007fb9f4fa2b64 sp 00007ffc927694f0 error 4 in libdovecot.so.0.0.0[7fb9f4f58000+db000] Nov 24 16:59:50 server.name kernel: Code: 89 f5 48 8d b8 f0 01 00 00 48 8b 52 08 4c 8b 62 48 e8 90 89 fb ff 48 8b 44 24 08 be 00 20 00 00 4c 89 ef 48 8b 80 10 01 00 00 <4c> 8b 38 e8 64 b2 fb ff 48 8b 44 24 08 4c 89 ef 48 8b b0 10 01 00 Nov 24 16:59:50 server.name tika[988]: INFO tika/ (application/pdf) Nov 24 16:59:50 server.name tika[988]: WARN tika/: Text extraction failed (null) Nov 24 16:59:50 server.name tika[988]: org.apache.tika.exception.TikaException: TIKA-198: Illegal IOException from org.apache.tika.parser.DefaultParser@4dcc5b82 Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:286) Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143) Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.server.resource.TikaResource.parse(TikaResource.java:409) Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.server.resource.TikaResource$4.write(TikaResource.java:521) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.jaxrs.provider.BinaryDataProvider.writeTo(BinaryDataProvider.java:177) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.jaxrs.utils.JAXRSUtils.writeMessageBody(JAXRSUtils.java:1472) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.serializeMessage(JAXRSOutInterceptor.java:249) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.processResponse(JAXRSOutInterceptor.java:122) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.handleMessage(JAXRSOutInterceptor.java:84) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:90) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:247) Nov 24 16:59:50 server.name tika[988]: at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:79) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.Server.handle(Server.java:500) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) Nov 24 16:59:50 server.name tika[988]: at java.lang.Thread.run(Thread.java:748) Nov 24 16:59:50 server.name tika[988]: Caused by: org.eclipse.jetty.io.EofException: Early EOF Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.HttpInput$3.getError(HttpInput.java:1141) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.HttpInput$3.noContent(HttpInput.java:1129) Nov 24 16:59:50 server.name tika[988]: at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:321) Nov 24 16:59:50 server.name tika[988]: at java.io.FilterInputStream.read(FilterInputStream.java:133) Nov 24 16:59:50 server.name tika[988]: at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) Nov 24 16:59:50 server.name tika[988]: at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) Nov 24 16:59:50 server.name tika[988]: at java.io.BufferedInputStream.read(BufferedInputStream.java:345) Nov 24 16:59:50 server.name tika[988]: at java.io.FilterInputStream.read(FilterInputStream.java:107) Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.io.ProxyInputStream.read(ProxyInputStream.java:78) Nov 24 16:59:50 server.name tika[988]: at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:78) Nov 24 16:59:50 server.name tika[988]: at org.apache.pdfbox.io.ScratchFile.createBuffer(ScratchFile.java:443) Nov 24 16:59:50 server.name tika[988]: at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1220) Nov 24 16:59:50 server.name tika[988]: at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1196) Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.parser.pdf.PDFParser.getPDDocument(PDFParser.java:190) Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:149) Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) Nov 24 16:59:50 server.name tika[988]: at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) Nov 24 16:59:50 server.name tika[988]: ... 39 more Nov 24 16:59:50 server.name tika[988]: ERROR Problem with writing the data, class org.apache.tika.server.resource.TikaResource$4, ContentType: text/plain

1 0

Dovecot authentication hangs when ldap_start_tls_s() fails for invalid certificate
by sebastiano degan 19 Nov '20

19 Nov '20

On a Debian10 I've installed postfix 3.4.14 and dovecot 2.3.4.1. I've configured multiple passdb sources and I expect that if one fails the other ones are tested. This is usually the case, except when the failure is due to an invalid certificate from the ldap server. In that case all authentication attempts from that moment on will fail. I've trimmed down the configuration as much as possible (see below), and this is the syslog output of my tests: TEST 1 - Everything is configured correctly: No output on syslog TEST 2 - Wrong hostname as a ldap server: Auth attempt 1: Nov 19 11:25:29 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-passdb.conf.ext: ldap_start_tls_s() failed: Can't contact LDAP server Nov 19 11:25:29 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-userdb.conf.ext: ldap_start_tls_s() failed: Can't contact LDAP server Nov 19 11:25:29 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-passdb.conf.ext: ldap_start_tls_s() failed: Can't contact LDAP server Nov 19 11:25:35 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-passdb.conf.ext: ldap_start_tls_s() failed: Can't contact LDAP server Auth attempt 2: Nov 19 11:25:57 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-passdb.conf.ext: ldap_start_tls_s() failed: Can't contact LDAP server Nov 19 11:25:59 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-passdb.conf.ext: ldap_start_tls_s() failed: Can't contact LDAP server Auth attempt 3: Nov 19 11:26:28 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-passdb.conf.ext: ldap_start_tls_s() failed: Can't contact LDAP server Nov 19 11:26:30 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-passdb.conf.ext: ldap_start_tls_s() failed: Can't contact LDAP server TEST 3 - Invalid certificate: Attempt 1: Nov 19 11:21:20 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-passdb.conf.ext: ldap_start_tls_s() failed: Connect error Nov 19 11:21:20 debian dovecot: auth: Error: LDAP /etc/dovecot/dovecot-ldap-userdb.conf.ext: ldap_start_tls_s() failed: Connect error No output on other attempts... CONFIGURATION FILES: ______________________________________________________ auth.conf: auth_mechanisms = plain login passdb { driver = ldap args = /etc/dovecot/dovecot-ldap-passdb.conf.ext } userdb { driver = ldap args = /etc/dovecot/dovecot-ldap-userdb.conf.ext default_fields = uid=vmail gid=vmail home=/var/mail/vmail/%d/%n } ______________________________________________________ ______________________________________________________ dovecot-ldap-passdb.conf.ext: tls = yes hosts = DC1.fv.lan base = ou=Frigoveneta,dc=fv,dc=lan auth_bind = yes auth_bind_userdn = %u ______________________________________________________ ______________________________________________________ dovecot-ldap-userdb.conf.ext: tls = yes hosts = DC1.fv.lan base = ou=Frigoveneta,dc=fv,dc=lan dn = ##removed## dnpass = ##removed## user_filter = (&(userPrincipalName=%u)) ______________________________________________________

1 0

RE: about imap-login: Error and imap: Error after Version-UP
by 'Aki Tuomi' 18 Nov '20

18 Nov '20

Hi.. there is supposed to be only at most one (1) stats process. Having multiple stats processes breaks stats. Aki > On 18/11/2020 01:48 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > Aki-san > > Last night We adjusted the "process_limit" in the stats service to get rid of the error. > > "client_limit" is in the blank (default) state. > > Thank you very much. > > morikawa > -----Original Message----- > From: 'Aki Tuomi' [mailto:aki.tuomi@open-xchange.com] > Sent: Tuesday, November 17, 2020 2:35 PM > To: 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp>; dovecot(a)dovecot.org > Subject: RE: about imap-login: Error and imap: Error after Version-UP > > You need to configure stats service client limit and imap service client limit. Those are different services than imap-login service. > > Aki > > > On 17/11/2020 01:28 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > > > > Aki-san > > > > I have already seen that page. > > Is the setting method wrong? > > > > Is there any other tuning point? > > > > morikawa > > -----Original Message----- > > From: Aki Tuomi [mailto:aki.tuomi@open-xchange.com] > > Sent: Monday, November 16, 2020 8:52 PM > > To: 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp>; dovecot(a)dovecot.org > > Subject: Re: about imap-login: Error and imap: Error after Version-UP > > > > > > > On 16/11/2020 13:44 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > > > > > > > We have upgraded dovecot from 2.0.9 (centos6) to 2.2.36 (centos8). > > > > > > If more than 1000 users log in, the following error will occur. > > > ------------------------- > > > Nov 16 11:33:00 xxxxxxxx dovecot[1361]: imap-login: Error: master(imap): > > > net_connect_unix(imap) failed: Resource temporarily unavailable - > > > http://wiki2.dovecot.org/SocketUnavailable (client-pid=1362, > > > client-id=129834, rip=10.10.10.10, created 551 msecs ago, received > > > 0/4 > > > bytes) > > > Nov 16 11:30:26 xxxxxxxx dovecot[1361]: imap: Error: > > > net_connect_unix(/var/run/dovecot/stats-writer) failed: Resource > > > temporarily unavailable > > > ------------------------- > > > > > > I set the following because I thought it was a problem with the > > > number of connections, but I get an error. > > > > > > service imap-login { > > > service_count=0 > > > client_limit = $default_client_limit > > > process_min_avail = 4 > > > process_limit = $default_process_limit > > > vsz_limit = 1G > > > } > > > > > > limits.conf > > > ------------------------- > > > * soft nofile 4096 > > > * hard nofile 4096 > > > ------------------------- > > > > > > cat /proc/sys/net/core/somaxconn > > > ------------------------- > > > 4096 > > > ------------------------- > > > > > > Is there any other tuning point? > > > Where should I look? > > > > https://wiki.dovecot.org/SocketUnavailable > > > > Aki

1 0

RE: about imap-login: Error and imap: Error after Version-UP
by 'Aki Tuomi' 18 Nov '20

18 Nov '20

You need to configure stats service client limit and imap service client limit. Those are different services than imap-login service. Aki > On 17/11/2020 01:28 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > Aki-san > > I have already seen that page. > Is the setting method wrong? > > Is there any other tuning point? > > morikawa > -----Original Message----- > From: Aki Tuomi [mailto:aki.tuomi@open-xchange.com] > Sent: Monday, November 16, 2020 8:52 PM > To: 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp>; dovecot(a)dovecot.org > Subject: Re: about imap-login: Error and imap: Error after Version-UP > > > > On 16/11/2020 13:44 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > > > > We have upgraded dovecot from 2.0.9 (centos6) to 2.2.36 (centos8). > > > > If more than 1000 users log in, the following error will occur. > > ------------------------- > > Nov 16 11:33:00 xxxxxxxx dovecot[1361]: imap-login: Error: master(imap): > > net_connect_unix(imap) failed: Resource temporarily unavailable - > > http://wiki2.dovecot.org/SocketUnavailable (client-pid=1362, > > client-id=129834, rip=10.10.10.10, created 551 msecs ago, received 0/4 > > bytes) > > Nov 16 11:30:26 xxxxxxxx dovecot[1361]: imap: Error: > > net_connect_unix(/var/run/dovecot/stats-writer) failed: Resource > > temporarily unavailable > > ------------------------- > > > > I set the following because I thought it was a problem with the number > > of connections, but I get an error. > > > > service imap-login { > > service_count=0 > > client_limit = $default_client_limit > > process_min_avail = 4 > > process_limit = $default_process_limit > > vsz_limit = 1G > > } > > > > limits.conf > > ------------------------- > > * soft nofile 4096 > > * hard nofile 4096 > > ------------------------- > > > > cat /proc/sys/net/core/somaxconn > > ------------------------- > > 4096 > > ------------------------- > > > > Is there any other tuning point? > > Where should I look? > > https://wiki.dovecot.org/SocketUnavailable > > Aki

2 1

Re: about imap-login: Error and imap: Error after Version-UP
by Aki Tuomi 17 Nov '20

17 Nov '20

> On 16/11/2020 13:44 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > We have upgraded dovecot from 2.0.9 (centos6) to 2.2.36 (centos8). > > If more than 1000 users log in, the following error will occur. > ------------------------- > Nov 16 11:33:00 xxxxxxxx dovecot[1361]: imap-login: Error: master(imap): > net_connect_unix(imap) failed: Resource temporarily unavailable - > http://wiki2.dovecot.org/SocketUnavailable (client-pid=1362, > client-id=129834, rip=10.10.10.10, created 551 msecs ago, received 0/4 > bytes) > Nov 16 11:30:26 xxxxxxxx dovecot[1361]: imap: Error: > net_connect_unix(/var/run/dovecot/stats-writer) failed: Resource temporarily > unavailable > ------------------------- > > I set the following because I thought it was a problem with the number of > connections, but I get an error. > > service imap-login { > service_count=0 > client_limit = $default_client_limit > process_min_avail = 4 > process_limit = $default_process_limit > vsz_limit = 1G > } > > limits.conf > ------------------------- > * soft nofile 4096 > * hard nofile 4096 > ------------------------- > > cat /proc/sys/net/core/somaxconn > ------------------------- > 4096 > ------------------------- > > Is there any other tuning point? > Where should I look? https://wiki.dovecot.org/SocketUnavailable Aki

2 1

about imap-login: Error and imap: Error after Version-UP
by 森川　孝司 16 Nov '20

16 Nov '20

We have upgraded dovecot from 2.0.9 (centos6) to 2.2.36 (centos8). If more than 1000 users log in, the following error will occur. ------------------------- Nov 16 11:33:00 xxxxxxxx dovecot[1361]: imap-login: Error: master(imap): net_connect_unix(imap) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable (client-pid=1362, client-id=129834, rip=10.10.10.10, created 551 msecs ago, received 0/4 bytes) Nov 16 11:30:26 xxxxxxxx dovecot[1361]: imap: Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Resource temporarily unavailable ------------------------- I set the following because I thought it was a problem with the number of connections, but I get an error. service imap-login { service_count=0 client_limit = $default_client_limit process_min_avail = 4 process_limit = $default_process_limit vsz_limit = 1G } limits.conf ------------------------- * soft nofile 4096 * hard nofile 4096 ------------------------- cat /proc/sys/net/core/somaxconn ------------------------- 4096 ------------------------- Is there any other tuning point? Where should I look?

1 0

last-login Plugin
by Andrea Gabellini 16 Nov '20

16 Nov '20

Hello, i'm using last_login plugin and store data into a MySql DB. I'm using version 2.3.10.1 and the config is at the end of the email. I would like to store also the remote IP and the session name. The config is working but other informations are written only the first time. Any update is ignored but last_login time. Debugging MySQL, I notice that dovecot is using this query: INSERT INTO last_login (last_login,userid,rip,protocol,session) VALUES (1605350938,'xxxx',x.x.x.x','imap','jLFs5A609cdSN4Wh') ON DUPLICATE KEY UPDATE last_login=1605350938 How can I modify the configuration? conf.d/95-last_login.conf: dict { lastlogin = mysql:/etc/dovecot/dovecot-last-login.conf.ext } plugin { last_login_dict = proxy::lastlogin last_login_key = last-login/%u/%r/%s/%{session} } protocol imap { mail_plugins = $mail_plugins last_login } protocol pop3 { mail_plugins = $mail_plugins last_login } dovecot-last-login.conf.ext: connect = host=/var/lib/mysql/mysql.sock dbname=dovecot user=xxx password=xxx map { pattern = shared/last-login/$userid/$rip/$service/$session table = last_login value_field = last_login value_type = uint fields { userid = $userid rip = $rip protocol = $service session = $session } } -- __________________________ UNIX is user friendly. It's just selective about who its friends are. __________________________ TIM San Marino S.p.A. Andrea Gabellini Engineering R&D TIM San Marino S.p.A. - https://www.telecomitalia.sm Via Ventotto Luglio, 212 - Piano -2 47893 - Borgo Maggiore - Republic of San Marino Tel: (+378) 0549 886237 Fax: (+378) 0549 886188 -- Informativa Privacy Questa email ha per destinatari dei contatti presenti negli archivi di TIM San Marino S.p.A.. Tutte le informazioni vengono trattate e tutelate nel rispetto della normativa vigente sulla protezione dei dati personali (Reg. EU 2016/679). Per richiedere informazioni e/o variazioni e/o la cancellazione dei vostri dati presenti nei nostri archivi potete inviare una email a privacy(a)telecomitalia.sm. Avviso di Riservatezza Il contenuto di questa e-mail e degli eventuali allegati e' strettamente confidenziale e destinato alla/e persona/e a cui e' indirizzato. Se avete ricevuto per errore questa e-mail, vi preghiamo di segnalarcelo immediatamente e di cancellarla dal vostro computer. E' fatto divieto di copiare e divulgare il contenuto di questa e-mail. Ogni utilizzo abusivo delle informazioni qui contenute da parte di persone terze o comunque non indicate nella presente e-mail potra' essere perseguito ai sensi di legge.

2 1

[patch] enhancement for tika server protected by user/password basic auth
by John Fawcett 16 Nov '20

16 Nov '20

Hi The existing code is designed for a tika server on local host, or on a remote host that is either accessable to all or for example taht is protected by ip restrictions via a proxy in front of it. I've configured a tika server behind an apache proxy which enforces basic auth, but sending basic auth credentials for a tika server is not currently supported by Dovecot. The following patch allows to have user and password specified in the fts_tika url in much the same way you can for fts_solr. fts_tika = https://user:password@tika_host/tika John --- dovecot-2.3.11.3-orig/src/plugins/fts/fts-parser-tika.c 2020-08-12 14:20:41.000000000 +0200 +++ dovecot-2.3.11.3/src/plugins/fts/fts-parser-tika.c 2020-11-15 15:18:24.351281064 +0100 @@ -57,7 +57,7 @@ tuser = p_new(user->pool, struct fts_parser_tika_user, 1); MODULE_CONTEXT_SET(user, fts_parser_tika_user_module, tuser); - if (http_url_parse(url, NULL, 0, user->pool, + if (http_url_parse(url, NULL, HTTP_URL_ALLOW_USERINFO_PART, user->pool, &tuser->http_url, &error) < 0) { i_error("fts_tika: Failed to parse HTTP url %s: %s", url, error); return -1; @@ -152,6 +152,11 @@ http_url->host.name, t_strconcat(http_url->path, http_url->enc_query, NULL), fts_tika_parser_response, parser); + if (http_url->user != NULL) { + http_client_request_set_auth_simple( + http_req, http_url->user, http_url->password); + } + http_client_request_set_port(http_req, http_url->port); http_client_request_set_ssl(http_req, http_url->have_ssl); if (parser_context->content_type != NULL)

2 9

Re: no shared cipher openssl
by Aki Tuomi 16 Nov '20

16 Nov '20

> On 16/11/2020 09:54 lists(a)lazygranch.com <lists(a)lazygranch.com> wrote: > > > On Sun, 15 Nov 2020 17:31:07 -0500 > Mike Schroeder <mikeschroe(a)gmail.com> wrote: > > > CentOS 7 > > Dovecot 2.2.36 > > > > Nov 14 07:13:08 mail dovecot: pop3-login: Disconnected (no auth > > attempts in 0 secs): > > user=<>, rip=73.0.0.0, lip=192.64.118.242, TLS handshaking: > > SSL_accept() failed: > > error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher, > > session=<> > > > > Was working fine for over a year, until the cert expired and I > > replaced it. I've tried the good cert I have for https and I used the > > Dovecot.org script to generate a self-signed certificate. > > > > 10-ssl.conf > > ## SSL settings > > #ssl = required > > ssl = yes > > #ssl = no > > ssl_cert = </etc/pki/dovecot/certs/mydomain.com.crt > > ssl_key = </etc/pki/dovecot/private/mydomain.com.key > > #ssl_ca = > > #ssl_require_crl = yes > > #ssl_client_ca_dir = > > #ssl_client_ca_file = > > #ssl_verify_client_cert = no > > #ssl_cert_username_field = commonName > > #ssl_dh_parameters_length = 1024 > > #ssl_protocols = !SSLv3 > > > > # SSL ciphers to use > > # ols values ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > > ssl_cipher_list = > > ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK: > > !RC4:!ADH:!LOW@STRENGTH > > > > # Prefer the server's order of ciphers over client's. > > #ssl_prefer_server_ciphers = no > > > > # Prefer the server's order of ciphers over client's. > > #ssl_prefer_server_ciphers = no > > # SSL crypto device to use, for valid values run "openssl engine" > > #ssl_crypto_device = > > > > # SSL extra options. Currently supported options are: > > # no_compression - Disable compression. > > # no_ticket - Disable SSL session tickets. > > #ssl_options = > > > > =========================== > > # openssl x509 -dates -in mydomain.com.crt > > notBefore=Nov 11 16:31:35 2020 GMT > > notAfter=Nov 11 16:31:35 2022 GMT > > -----BEGIN CERTIFICATE----- > > : > > =========================== > > # openssl pkey -in mydomain.com.key > > -----BEGIN PRIVATE KEY----- > > : > > > > Thanks for taking a look. Any ideas on what I should do next to > > debug? > > > > Mike > > I remembered this problem was posted and still had the reply post from > Viktor. This may or may not be relevant. A search on this text will > probably drag up the whole thread. > --------------- > Specifically, an ECDSA P-256 certificate, but some systems don't (yet?) > support ECDSA. You'd need an additional RSA certificate to interoperate > with their sending MTA's limited STARTTLS cipher/protocol repertoire. > -------------- > > When this thread went around I looked at my logs and found some no > auth complaints on my dovecot log. I believe they were trying to use > the sslv3 to hack my server. Or at least see if it is hackable. Since > my email server is a personal one and the attack was from a hosting > company, I blocked server IP space. > > The weird thing I get your error now myself but not consistently. Here > is an example. > ------------------------------- > Nov 16 04:18:37 imap-login: Info: Disconnected (no auth attempts in 0 secs): user=<>, rip=myvpn, lip=myserverip, TLS handshaking: SSL_accept() failed: error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown: SSL alert number 46, session=<rXchrDG06qvGx2p9> > Nov 16 04:18:37 imap-login: Info: Login: user=<me(a)mydomain.com>, method=PLAIN, rip=myvpn, lip=myserverip, mpid=11710, TLS, session=<DSIjrDG05KvGx2p9> > > However the problem isn't present at the moment. Dovecot supports alternative certificate if you have problems with ECDSA and need to use RSA for them. See https://doc.dovecot.org/settings/core/#ssl-alt-cert Aki

1 0

no shared cipher openssl
by Mike Schroeder 16 Nov '20

16 Nov '20

CentOS 7 Dovecot 2.2.36 Nov 14 07:13:08 mail dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=73.0.0.0, lip=192.64.118.242, TLS handshaking: SSL_accept() failed: error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher, session=<> Was working fine for over a year, until the cert expired and I replaced it. I've tried the good cert I have for https and I used the Dovecot.org script to generate a self-signed certificate. 10-ssl.conf ## SSL settings #ssl = required ssl = yes #ssl = no ssl_cert = </etc/pki/dovecot/certs/mydomain.com.crt ssl_key = </etc/pki/dovecot/private/mydomain.com.key #ssl_ca = #ssl_require_crl = yes #ssl_client_ca_dir = #ssl_client_ca_file = #ssl_verify_client_cert = no #ssl_cert_username_field = commonName #ssl_dh_parameters_length = 1024 #ssl_protocols = !SSLv3 # SSL ciphers to use # ols values ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_cipher_list = ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK: !RC4:!ADH:!LOW@STRENGTH # Prefer the server's order of ciphers over client's. #ssl_prefer_server_ciphers = no # Prefer the server's order of ciphers over client's. #ssl_prefer_server_ciphers = no # SSL crypto device to use, for valid values run "openssl engine" #ssl_crypto_device = # SSL extra options. Currently supported options are: # no_compression - Disable compression. # no_ticket - Disable SSL session tickets. #ssl_options = =========================== # openssl x509 -dates -in mydomain.com.crt notBefore=Nov 11 16:31:35 2020 GMT notAfter=Nov 11 16:31:35 2022 GMT -----BEGIN CERTIFICATE----- : =========================== # openssl pkey -in mydomain.com.key -----BEGIN PRIVATE KEY----- : Thanks for taking a look. Any ideas on what I should do next to debug? Mike

2 1

imap dumping core in libdovecot-storage with assert "(block_count == 0 || block_idx+1 == block_count)"
by Renaud Métrich 15 Nov '20

15 Nov '20

Hi, I would like to report an issue which is 100% reproducible with a specific mail available in the reproducer attached to Red Hat's bugzilla <https://bugzilla.redhat.com/show_bug.cgi?id=1894418> (https://bugzilla.redhat.com/attachment.cgi?id=1726535) -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- Return-Path: <XXXXXXXXX(a)XXXXXXX.XXX.XX> X-Original-To: XXXXXXXXX(a)XXXXXXX.XXX.XX Delivered-To: XXXXXXXXXXXXXXXX(a)XXXXXX.XX.XXX.XX Received: from XXXXXXXXXXXXXXXXX.XXXXX.XX.XX (unknown [XX.XX.XXX.XXX]) by XXXXXX.XX.XXX.XX (Postfix) with ESMTP id XXXXXXXXXXX for <XXXXXXXXX(a)XXXXXXX.XXX.XX>; Thu, 15 Oct 2020 23:11:45 +0200 (CEST) Date: Thu, 15 Oct 2020 23:11:42 +0200 Subject: zoom Message-ID: <XXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXX(a)XXXXX.XXXXXXX.XXX> From: "XXXXXXXXX(a)XXXXXXX.XXX.XX" <XXXXXXXXX(a)XXXXXXX.XXX.XX> To: XXXXXXXXX(a)XXXXXXX.XXX.XX MIME-Version: 1.0 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- The email seems completely valid to me, but I'm no email expert. When performing a "FETCH <ID> (BINARY.SIZE[1] BINARY.PEEK[1])" operation, imap crashes with following backtrace: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- (gdb) bt [...] #5 0x00007ffff7707a25 in i_panic ( format=format@entry=0x7ffff7b7b460 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:523 #6 0x00007ffff7ab5de7 in blocks_count_lines (full_input=0x5555557f1230, ctx=0x7fffffffdb80) at index-mail-binary.c:357 #7 index_mail_read_binary_to_cache (_mail=0x5555557e3b48, part=0x5555557e4678, include_hdr=<optimized out>, reason=<optimized out>, binary_r=0x7fffffffdc16, converted_r=0x7fffffffdc17) at index-mail-binary.c:395 #8 0x00007ffff7b39c8b in index_mail_get_binary_size (lines_r=0x7fffffffdcc4, size_r=0x7fffffffdd20, include_hdr=false, part=0x5555557e4678, _mail=0x5555557e3b48) at index-mail-binary.c:497 #9 index_mail_get_binary_stream (_mail=0x5555557e3b48, part=0x5555557e4678, include_hdr=<optimized out>, size_r=0x7fffffffdd20, lines_r=0x7fffffffdcc4, binary_r=<optimized out>, stream_r=0x0) at index-mail-binary.c:557 #10 0x00007ffff7abd204 in mail_get_binary_size (mail=mail@entry=0x5555557e3b48, part=0x5555557e4678, include_hdr=<optimized out>, size_r=size_r@entry=0x7fffffffdd20, lines_r=lines_r@entry=0x7fffffffdcc4) at mail.c:342 #11 0x00007ffff7b7add8 in imap_msgpart_size (mail=0x5555557e3b48, msgpart=0x5555557e0748, size_r=size_r@entry=0x7fffffffdd20) at imap-msgpart.c:755 #12 0x000055555557917c in fetch_binary_size (ctx=0x5555557c97b0, mail=<optimized out>, body=0x5555557dfa20) at imap-fetch-body.c:212 #13 0x00005555555777e2 in imap_fetch_more_int (ctx=0x5555557c97b0, cancel=false) at imap-fetch.c:562 #14 0x00005555555788ed in imap_fetch_more (ctx=0x5555557c97b0, cmd=cmd@entry=0x5555557c95d8) at imap-fetch.c:617 #15 0x00005555555694b1 in cmd_fetch (cmd=0x5555557c95d8) at cmd-fetch.c:337 [...] #28 0x0000555555566405 in main (argc=<optimized out>, argv=<optimized out>) at main.c:523 -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- This happens on latest dovecot. Best regards, Renaud METRICH

2 3

dovecot fts-solr + solr 8.7.0 upgrade: "Indexing failed: 401 Unauthorized" + "Transaction commit failed: FTS transaction commit failed: backend deinit" ?
by PGNet Dev 14 Nov '20

14 Nov '20

I've built a new dovecot + fts-solr instance; I've now picked up & am running the recently released solr 8.7.0. In a test account, I've one message -- in the 'Drafts' folder. On exec of fts 'index' doveadm index -u testuser(a)example.com -q '*' I get a 401 unauthorized indexing failure, only for the one dir+message in 'Drafts', 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<8MKVNHmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Queue 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<OHfzNHmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in testing 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<yMxJNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Junk 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<uDCbNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Trash 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<kA/uNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Sent 2020-11-13 05:43:22 indexer-worker(testuser(a)example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: fts_solr: Indexing failed: 401 Unauthorized 2020-11-13 05:43:22 indexer-worker(testuser(a)example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: fts_solr: Indexing failed: 401 Unauthorized 2020-11-13 05:43:22 indexer-worker(testuser(a)example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: Mailbox Drafts: Transaction commit failed: FTS transaction commit failed: backend deinit (attempted to index 1 messages (UIDs 3..3)) 2020-11-13 05:43:22 indexer-worker(testuser(a)example.com)<2OUMCHqNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in INBOX I get NO output atm in my solr logs @ this 'index'; just, atm, in the dovecot logs, above I've checked all my usual perms. Indexing _was_ working with a prior solr v8.6.3 ... Either I've misconfigured some perm, and am missing it, or new-solr-version v8.7.0 is causing me some grief. (1) Can anyone yet verify Doveoct/fts-solr working with solr 8.7.x? (2) What logging config in Dovecot gets me more/detailed output for the fts_solr fail? (3) Any obvious clues as to what, specifically, is the source of this prob?

2 9

Submission service capabilities listing
by Dan 13 Nov '20

13 Nov '20

Hello all, I'm attempting to implement a submission server with questions in regards to how Dovecot handles the backend hosts capabilities. To my understanding, Dovecot will connect to the host specified in either submission_host or submission_relay_host, and gather a list of capabilities. Once gathered, after the client sends the EHLO greeting Dovecot will respond back with capabilities that the backend host is capable of. My question is, When does this occur? Running a tcpdump with the filter set to the relay host IP, I don't see any traffic until after I send the EHLO command. Dovecot states that CHUNKING is available, but the backend host does not have that capability. Is there a way to disable CHUNKING explicitly? Thanks. submission_relay_host = x.x.x.x submission_host = x.x.x.x submission_relay_ssl = no submission_relay_ssl_verify = no protocol submission { passdb { driver = static args = host=x.x.x.x proxy=y nopassword=y port=587 } }

2 1

How do Cerbot files map to Dovecot?
by Raymond Herrera 12 Nov '20

12 Nov '20

I am postponing the Apache plugin issue (CentOS is not Certbot friendly) and requesting a standalone, generic certificate. After the command "1: Spin up a temporary webserver" I have the following 2 files in the folder /etc/letsencrypt: -rw-r--r-- 1 root root 924 Nov 12 11:14 csr/0000_csr-certbot.pem -rw------- 1 root root 1708 Nov 12 11:14 keys/0000_key-certbot.pem The "key" is probably a direct replacement for the file in the distribution. What about the "csr" file? It seems to be a request, not the certificate itself. TIA

3 2

Fwd: Trying to install certbot on CentOS
by Raymond Herrera 12 Nov '20

12 Nov '20

On 11/12/2020 10:44 AM, Raymond Herrera wrote: > Following the advice obtained here, I am trying to get a LetsEncrypt > certificate. > > These are the instructions: > > Step 1: > https://snapcraft.io/docs/installing-snapd > > Step 2: > https://certbot.eff.org/lets-encrypt/centosrhel7-apache > > My problem is this error message: > > # snap install --classic certbot > > error: system does not fully support snapd: cannot mount squashfs > image using "squashfs": mount: > /tmp/sanity-squashfs-881328484: failed to setup loop device: No > such file or directory > > Apparently, RedHat/CentOS are not supporters of snap. > > TIA > > OS: CentOS 7 > Thanks to Leonardo and Michael, I have made some progress. The snaps application is not supported by RedHat/CentOS. The solution is simple, just use: # yum install certbot The new error message is this: # certbot certonly --apache Saving debug log to /var/log/letsencrypt/letsencrypt.log Could not choose appropriate plugin: The requested apache plugin does not appear to be installed The requested apache plugin does not appear to be installed I am trying to find which is the "requested apache plugin". The log mentions "authenticator". I am hoping that an Apache Certificate will be useful for Dovecot.

2 1

Guide on Renewing SSL Certificate for Apache, Postfix and Dovecot on CentOS 6.8 Linux
by Turritopsis Dohrnii Teo En Ming 12 Nov '20

12 Nov '20

Good day from Singapore, Please check out the following link for my guide. https://marc.info/?l=postfix-users&m=160517863730596&w=2 Thank you very much. -----BEGIN EMAIL SIGNATURE----- The Gospel for all Targeted Individuals (TIs): [The New York Times] Microwave Weapons Are Prime Suspect in Ills of U.S. Embassy Workers Link: https://www.nytimes.com/2018/09/01/science/sonic-attack-cuba-microwave.html ******************************************************************************************** Singaporean Targeted Individual Mr. Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 14 Feb 2019 and refugee seeking attempts at the United Nations Refugee Agency Bangkok (21 Mar 2017), in Taiwan (5 Aug 2019) and Australia (25 Dec 2019 to 9 Jan 2020): [1] https://tdtemcerts.wordpress.com/ [2] https://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming -----END EMAIL SIGNATURE-----

2 1

Trying to install certbot on CentOS
by Raymond Herrera 12 Nov '20

12 Nov '20

Following the advice obtained here, I am trying to get a LetsEncrypt certificate. These are the instructions: Step 1: https://snapcraft.io/docs/installing-snapd Step 2: https://certbot.eff.org/lets-encrypt/centosrhel7-apache My problem is this error message: # snap install --classic certbot error: system does not fully support snapd: cannot mount squashfs image using "squashfs": mount: /tmp/sanity-squashfs-881328484: failed to setup loop device: No such file or directory Apparently, RedHat/CentOS are not supporters of snap. TIA OS: CentOS 7

3 2

Testing
by Turritopsis Dohrnii Teo En Ming 12 Nov '20

12 Nov '20

Testing -----BEGIN EMAIL SIGNATURE----- The Gospel for all Targeted Individuals (TIs): [The New York Times] Microwave Weapons Are Prime Suspect in Ills of U.S. Embassy Workers Link: https://www.nytimes.com/2018/09/01/science/sonic-attack-cuba-microwave.html ******************************************************************************************** Singaporean Targeted Individual Mr. Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 14 Feb 2019 and refugee seeking attempts at the United Nations Refugee Agency Bangkok (21 Mar 2017), in Taiwan (5 Aug 2019) and Australia (25 Dec 2019 to 9 Jan 2020): [1] https://tdtemcerts.wordpress.com/ [2] https://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming -----END EMAIL SIGNATURE-----

1 0

Support TLS for pure remote doveadm commands
by Vincent Brillault 11 Nov '20

11 Nov '20

Dear all, A lot of doveadm commands (e.g. backup/sync, deduplicate, mailbox) support a `-S` (a.k.a `socket-path`) argument that allows to run the command remotely. Unfortunately, I think that there is a bug in the handling of the TLS configuration for this feature. After a bit of debugging, I think it's simply due to doveadm_mail_server_user_get_host (https://github.com/dovecot/core/blob/master/src/doveadm/doveadm-mail-server…) bailing out too quickly: It's used by doveadm_mail_server_user (https://github.com/dovecot/core/blob/master/src/doveadm/doveadm-mail-server…) to set its `user`, `host`, `hostip`, `port` and `ssl_flags`. When doveadm_port is not (or set to 0): - `user` is set to `input->username`: OK - `host` is set to `ctx->set->doveadm_socket_path` (the content of the `-S` option): OK - `hostip` is not set: OK (Later on, the `host` is used in that case) - `port` is set to 0: This is OK, as when the `hostip` (or rather the `server->ip` later in the code) is not set, this is only the default, the port is extracted from the `host` (or rather `server->name`) - `ssl_flags` is not modified: this is not OK as there is nothing else modifying it! Bailing out just a few line later (as in https://patch-diff.githubusercontent.com/raw/dovecot/core/pull/145.patch) fixes the issue for me. To reproduce the issue, with dovecot-2.3.11.3-4, you will need two systems, A & B: A needs to have dovecot configured and port 9091 open in its firewall for B. B just need dovecot to be installed. - Testing without TLS: - Add the following to A's dovecot configuration (and reload it): ``` service doveadm { inet_listener { port = 9091 } } doveadm_password=VeryRandomString ``` - On B, create a file X containing (only): ``` doveadm_password=VeryRandomString ``` - On B, running `doveadm -c X mailbox list -u ${existing_user} -S ${A}:9091` should return the mailbox list - Switching to TLS: - Replace what was added to A's dovecot configuration (and reload it) with: ``` service doveadm { inet_listener { port = 9091 ssl = yes } } doveadm_password=VeryRandomString doveadm_ssl = ssl # Not strictly required ``` - On B, create a file Y containing (only): ``` doveadm_password=VeryRandomString doveadm_ssl=yes ``` - On B, running `doveadm -c Y mailbox list -u ${existing_user} -S ${A}:9091` will not work. - On A, you should see in the logs: `dovecot[16435]: doveadm(${B}): Error: SSL handshake failed: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol` Applying the patch mentioned above makes the 2nd case work. What do you think? Cheers, Vincent

1 0

Re: how to set smtp-client -> submission_relay_host for IPv4 only?
by PGNet Dev 11 Nov '20

11 Nov '20

ref https://dovecot.org/pipermail/dovecot/2020-October/120226.html > On 10/15/20 10:13 PM, Aki Tuomi wrote: >> I'm asking how/where to 'tell', via config, Dovecot's smtp-CLIENT, that's making >> to connection to the submission_relay_host, to use _only_ IPv4. > > > There is currently no (other) way to do this than using /etc/hosts or specifying > IPv4 address for the relay host. Is there a bug request opened for this, then ? Is it planned for implementation ?

1 0

Re: Recovering expunged but not purged e-mails from mdbox with zlib compression
by Aki Tuomi 11 Nov '20

11 Nov '20

> On 10/11/2020 20:07 Daniel Schütze <dms(a)cwa.uk.com> wrote: > > > Dear Aki > Thank you. Unfortunately I'm struggling to get the right syntax for this as it looks like someone else was here too > https://dovecot.org/pipermail/dovecot/2018-July/112441.html > The location in my dovecot.conf is > > mdbox:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/%%n This syntax is for accessing shared folders. You probably should try doveadm -Dv -o mail_location=mdbox_deleted:/path/to/mdbox fetch -u victim text ALL > I'm trying to fetch a message (for testing purposes now) based on it's guid as that is available from doveadm dump. > So my syntax is (based on the previous person who didn't get it to work). > > doveadm -o "mail_location=mdbox_deleted:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/%%n" fetch "body" guid (msg.guid from doveadm dump) > Doveadm does not support var expand, so this will not work. See above for syntax. > But that's getting no response. I've tried putting in the hard path in case there is any trouble with the %%h etc but that doesn't help either. > I've also tried the fetch with the subject of a known deleted message and also adding mailbox Drafts (as I know that's where it is or rather was) but to no avail. > Given doveadm dump gives msg.uid and no subject I'd prefer to use that in any case. > There are no error messages, I do with doveadm was a little more verbose to help over these bumps! > > Any chance of a helping hand? > > > > Aki > > > > Daniel Schütze > Director > > ------------------ > > CWA International Ltd > 5th Floor, 42 Trinity Square > London > EC3N 4DJ > > (t) + 44 (0)20 7242 8444 > (e) dms(a)cwa.uk.com > (w) http://www.cwa.international/ > On 10/11/20 10:53, Aki Tuomi wrote: > > > You can use mdbox_deleted driver to access mails with refcount 0. See > > https://wiki2.dovecot.org/MailboxFormat/dbox > > > > Aki > > > > On 10.11.2020 12.42, Daniel Schütze wrote: > > > > > Yesterday I had to recover an e-mail which a user had deleted. If I > > > understand this correctly the message was expunged but not purged > > > (doveadm purge had not been run). > > > > > > This e-mail was clearly still in the mdbox stored with zlib > > > compression as I could tell using the doveadm dump command (doveadm > > > dump -t dbox filename). > > > > > > I could however not reveal the e-mail with the normal doveadm fetch -u > > > username "body" guid (guid from dump output) > > > > > > In the end I was able to recover the e-mail by cutting it out from the > > > mdbox and running gunzip over it, but this method was very fiddly and > > > would not have worked for a bulk job. I appreciate I could have just > > > deleted the index files and gotten all the deleted messages back by > > > having the index rebuilt but that would have been the proverbial > > > "hammer to crack a nut". > > > > > > Fortunately this e-mail had no attachment for me to worry about, as > > > those are also detached for sis by dovecot. > > > > > > Can anyone tell me of a quick and easy way to recover one or multiple > > > e-mails marked as expunged but not purged which are stored in a mdbox > > > with zlib compression? I'm sure I'll have to do this again the future > > > and my method wouldn't work with a folder! > > > > > > Clearly if I was not using zlib compression I could just have read the > > > contents of the mdbox without any complication. > > > > > > > > > -- > > > Daniel Schütze > > > Director > > > > > > ------------------ > > > > > > CWA International Ltd > > > 5th Floor, 42 Trinity Square > > > London > > > EC3N 4DJ > > > > > > (t) + 44 (0)20 7242 8444 > > > (e) dms(a)cwa.uk.com > > > (w) http://www.cwa.international/ > > >

3 2

Recommended Protocols?
by Raymond Herrera 11 Nov '20

11 Nov '20

Hello all: For several years I have been running the following in a Linux server. Dovecot Version: 2.0.9 *IMAP:* Connection Security: SSL/TLS Port: 993 Authentication Method: Normal Password *SMTP:* Connection Security: STARTTLS Port: 587 Authentication Method: Normal Password The E-mail client is Thunderbird on Windows. I am preparing a new server, with Dovecot 2.2.36 and would like to know the currently recommended protocols. Should I stick to what I have? I would prefer to start with the easiest configuration possible, which I will revise later. This is the command that I have been using to verify the server's functionality: % openssl s_client -connect localhost:imaps TIA

8 13