November 2020 - dovecot

RE: about imap-login: Error and imap: Error after Version-UP
by 'Aki Tuomi' 18 Nov '20

18 Nov '20

You need to configure stats service client limit and imap service client limit. Those are different services than imap-login service. Aki > On 17/11/2020 01:28 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > Aki-san > > I have already seen that page. > Is the setting method wrong? > > Is there any other tuning point? > > morikawa > -----Original Message----- > From: Aki Tuomi [mailto:aki.tuomi@open-xchange.com] > Sent: Monday, November 16, 2020 8:52 PM > To: 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp>; dovecot(a)dovecot.org > Subject: Re: about imap-login: Error and imap: Error after Version-UP > > > > On 16/11/2020 13:44 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > > > > We have upgraded dovecot from 2.0.9 (centos6) to 2.2.36 (centos8). > > > > If more than 1000 users log in, the following error will occur. > > ------------------------- > > Nov 16 11:33:00 xxxxxxxx dovecot[1361]: imap-login: Error: master(imap): > > net_connect_unix(imap) failed: Resource temporarily unavailable - > > http://wiki2.dovecot.org/SocketUnavailable (client-pid=1362, > > client-id=129834, rip=10.10.10.10, created 551 msecs ago, received 0/4 > > bytes) > > Nov 16 11:30:26 xxxxxxxx dovecot[1361]: imap: Error: > > net_connect_unix(/var/run/dovecot/stats-writer) failed: Resource > > temporarily unavailable > > ------------------------- > > > > I set the following because I thought it was a problem with the number > > of connections, but I get an error. > > > > service imap-login { > > service_count=0 > > client_limit = $default_client_limit > > process_min_avail = 4 > > process_limit = $default_process_limit > > vsz_limit = 1G > > } > > > > limits.conf > > ------------------------- > > * soft nofile 4096 > > * hard nofile 4096 > > ------------------------- > > > > cat /proc/sys/net/core/somaxconn > > ------------------------- > > 4096 > > ------------------------- > > > > Is there any other tuning point? > > Where should I look? > > https://wiki.dovecot.org/SocketUnavailable > > Aki

2 1

Re: about imap-login: Error and imap: Error after Version-UP
by Aki Tuomi 17 Nov '20

17 Nov '20

> On 16/11/2020 13:44 森川　孝司 <morikawa-k6(a)itec.hankyu-hanshin.co.jp> wrote: > > > We have upgraded dovecot from 2.0.9 (centos6) to 2.2.36 (centos8). > > If more than 1000 users log in, the following error will occur. > ------------------------- > Nov 16 11:33:00 xxxxxxxx dovecot[1361]: imap-login: Error: master(imap): > net_connect_unix(imap) failed: Resource temporarily unavailable - > http://wiki2.dovecot.org/SocketUnavailable (client-pid=1362, > client-id=129834, rip=10.10.10.10, created 551 msecs ago, received 0/4 > bytes) > Nov 16 11:30:26 xxxxxxxx dovecot[1361]: imap: Error: > net_connect_unix(/var/run/dovecot/stats-writer) failed: Resource temporarily > unavailable > ------------------------- > > I set the following because I thought it was a problem with the number of > connections, but I get an error. > > service imap-login { > service_count=0 > client_limit = $default_client_limit > process_min_avail = 4 > process_limit = $default_process_limit > vsz_limit = 1G > } > > limits.conf > ------------------------- > * soft nofile 4096 > * hard nofile 4096 > ------------------------- > > cat /proc/sys/net/core/somaxconn > ------------------------- > 4096 > ------------------------- > > Is there any other tuning point? > Where should I look? https://wiki.dovecot.org/SocketUnavailable Aki

2 1

about imap-login: Error and imap: Error after Version-UP
by 森川　孝司 16 Nov '20

16 Nov '20

We have upgraded dovecot from 2.0.9 (centos6) to 2.2.36 (centos8). If more than 1000 users log in, the following error will occur. ------------------------- Nov 16 11:33:00 xxxxxxxx dovecot[1361]: imap-login: Error: master(imap): net_connect_unix(imap) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable (client-pid=1362, client-id=129834, rip=10.10.10.10, created 551 msecs ago, received 0/4 bytes) Nov 16 11:30:26 xxxxxxxx dovecot[1361]: imap: Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Resource temporarily unavailable ------------------------- I set the following because I thought it was a problem with the number of connections, but I get an error. service imap-login { service_count=0 client_limit = $default_client_limit process_min_avail = 4 process_limit = $default_process_limit vsz_limit = 1G } limits.conf ------------------------- * soft nofile 4096 * hard nofile 4096 ------------------------- cat /proc/sys/net/core/somaxconn ------------------------- 4096 ------------------------- Is there any other tuning point? Where should I look?

1 0

last-login Plugin
by Andrea Gabellini 16 Nov '20

16 Nov '20

Hello, i'm using last_login plugin and store data into a MySql DB. I'm using version 2.3.10.1 and the config is at the end of the email. I would like to store also the remote IP and the session name. The config is working but other informations are written only the first time. Any update is ignored but last_login time. Debugging MySQL, I notice that dovecot is using this query: INSERT INTO last_login (last_login,userid,rip,protocol,session) VALUES (1605350938,'xxxx',x.x.x.x','imap','jLFs5A609cdSN4Wh') ON DUPLICATE KEY UPDATE last_login=1605350938 How can I modify the configuration? conf.d/95-last_login.conf: dict { lastlogin = mysql:/etc/dovecot/dovecot-last-login.conf.ext } plugin { last_login_dict = proxy::lastlogin last_login_key = last-login/%u/%r/%s/%{session} } protocol imap { mail_plugins = $mail_plugins last_login } protocol pop3 { mail_plugins = $mail_plugins last_login } dovecot-last-login.conf.ext: connect = host=/var/lib/mysql/mysql.sock dbname=dovecot user=xxx password=xxx map { pattern = shared/last-login/$userid/$rip/$service/$session table = last_login value_field = last_login value_type = uint fields { userid = $userid rip = $rip protocol = $service session = $session } } -- __________________________ UNIX is user friendly. It's just selective about who its friends are. __________________________ TIM San Marino S.p.A. Andrea Gabellini Engineering R&D TIM San Marino S.p.A. - https://www.telecomitalia.sm Via Ventotto Luglio, 212 - Piano -2 47893 - Borgo Maggiore - Republic of San Marino Tel: (+378) 0549 886237 Fax: (+378) 0549 886188 -- Informativa Privacy Questa email ha per destinatari dei contatti presenti negli archivi di TIM San Marino S.p.A.. Tutte le informazioni vengono trattate e tutelate nel rispetto della normativa vigente sulla protezione dei dati personali (Reg. EU 2016/679). Per richiedere informazioni e/o variazioni e/o la cancellazione dei vostri dati presenti nei nostri archivi potete inviare una email a privacy(a)telecomitalia.sm. Avviso di Riservatezza Il contenuto di questa e-mail e degli eventuali allegati e' strettamente confidenziale e destinato alla/e persona/e a cui e' indirizzato. Se avete ricevuto per errore questa e-mail, vi preghiamo di segnalarcelo immediatamente e di cancellarla dal vostro computer. E' fatto divieto di copiare e divulgare il contenuto di questa e-mail. Ogni utilizzo abusivo delle informazioni qui contenute da parte di persone terze o comunque non indicate nella presente e-mail potra' essere perseguito ai sensi di legge.

2 1

[patch] enhancement for tika server protected by user/password basic auth
by John Fawcett 16 Nov '20

16 Nov '20

Hi The existing code is designed for a tika server on local host, or on a remote host that is either accessable to all or for example taht is protected by ip restrictions via a proxy in front of it. I've configured a tika server behind an apache proxy which enforces basic auth, but sending basic auth credentials for a tika server is not currently supported by Dovecot. The following patch allows to have user and password specified in the fts_tika url in much the same way you can for fts_solr. fts_tika = https://user:password@tika_host/tika John --- dovecot-2.3.11.3-orig/src/plugins/fts/fts-parser-tika.c 2020-08-12 14:20:41.000000000 +0200 +++ dovecot-2.3.11.3/src/plugins/fts/fts-parser-tika.c 2020-11-15 15:18:24.351281064 +0100 @@ -57,7 +57,7 @@ tuser = p_new(user->pool, struct fts_parser_tika_user, 1); MODULE_CONTEXT_SET(user, fts_parser_tika_user_module, tuser); - if (http_url_parse(url, NULL, 0, user->pool, + if (http_url_parse(url, NULL, HTTP_URL_ALLOW_USERINFO_PART, user->pool, &tuser->http_url, &error) < 0) { i_error("fts_tika: Failed to parse HTTP url %s: %s", url, error); return -1; @@ -152,6 +152,11 @@ http_url->host.name, t_strconcat(http_url->path, http_url->enc_query, NULL), fts_tika_parser_response, parser); + if (http_url->user != NULL) { + http_client_request_set_auth_simple( + http_req, http_url->user, http_url->password); + } + http_client_request_set_port(http_req, http_url->port); http_client_request_set_ssl(http_req, http_url->have_ssl); if (parser_context->content_type != NULL)

2 9

Re: no shared cipher openssl
by Aki Tuomi 16 Nov '20

16 Nov '20

> On 16/11/2020 09:54 lists(a)lazygranch.com <lists(a)lazygranch.com> wrote: > > > On Sun, 15 Nov 2020 17:31:07 -0500 > Mike Schroeder <mikeschroe(a)gmail.com> wrote: > > > CentOS 7 > > Dovecot 2.2.36 > > > > Nov 14 07:13:08 mail dovecot: pop3-login: Disconnected (no auth > > attempts in 0 secs): > > user=<>, rip=73.0.0.0, lip=192.64.118.242, TLS handshaking: > > SSL_accept() failed: > > error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher, > > session=<> > > > > Was working fine for over a year, until the cert expired and I > > replaced it. I've tried the good cert I have for https and I used the > > Dovecot.org script to generate a self-signed certificate. > > > > 10-ssl.conf > > ## SSL settings > > #ssl = required > > ssl = yes > > #ssl = no > > ssl_cert = </etc/pki/dovecot/certs/mydomain.com.crt > > ssl_key = </etc/pki/dovecot/private/mydomain.com.key > > #ssl_ca = > > #ssl_require_crl = yes > > #ssl_client_ca_dir = > > #ssl_client_ca_file = > > #ssl_verify_client_cert = no > > #ssl_cert_username_field = commonName > > #ssl_dh_parameters_length = 1024 > > #ssl_protocols = !SSLv3 > > > > # SSL ciphers to use > > # ols values ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > > ssl_cipher_list = > > ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK: > > !RC4:!ADH:!LOW@STRENGTH > > > > # Prefer the server's order of ciphers over client's. > > #ssl_prefer_server_ciphers = no > > > > # Prefer the server's order of ciphers over client's. > > #ssl_prefer_server_ciphers = no > > # SSL crypto device to use, for valid values run "openssl engine" > > #ssl_crypto_device = > > > > # SSL extra options. Currently supported options are: > > # no_compression - Disable compression. > > # no_ticket - Disable SSL session tickets. > > #ssl_options = > > > > =========================== > > # openssl x509 -dates -in mydomain.com.crt > > notBefore=Nov 11 16:31:35 2020 GMT > > notAfter=Nov 11 16:31:35 2022 GMT > > -----BEGIN CERTIFICATE----- > > : > > =========================== > > # openssl pkey -in mydomain.com.key > > -----BEGIN PRIVATE KEY----- > > : > > > > Thanks for taking a look. Any ideas on what I should do next to > > debug? > > > > Mike > > I remembered this problem was posted and still had the reply post from > Viktor. This may or may not be relevant. A search on this text will > probably drag up the whole thread. > --------------- > Specifically, an ECDSA P-256 certificate, but some systems don't (yet?) > support ECDSA. You'd need an additional RSA certificate to interoperate > with their sending MTA's limited STARTTLS cipher/protocol repertoire. > -------------- > > When this thread went around I looked at my logs and found some no > auth complaints on my dovecot log. I believe they were trying to use > the sslv3 to hack my server. Or at least see if it is hackable. Since > my email server is a personal one and the attack was from a hosting > company, I blocked server IP space. > > The weird thing I get your error now myself but not consistently. Here > is an example. > ------------------------------- > Nov 16 04:18:37 imap-login: Info: Disconnected (no auth attempts in 0 secs): user=<>, rip=myvpn, lip=myserverip, TLS handshaking: SSL_accept() failed: error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown: SSL alert number 46, session=<rXchrDG06qvGx2p9> > Nov 16 04:18:37 imap-login: Info: Login: user=<me(a)mydomain.com>, method=PLAIN, rip=myvpn, lip=myserverip, mpid=11710, TLS, session=<DSIjrDG05KvGx2p9> > > However the problem isn't present at the moment. Dovecot supports alternative certificate if you have problems with ECDSA and need to use RSA for them. See https://doc.dovecot.org/settings/core/#ssl-alt-cert Aki

1 0

no shared cipher openssl
by Mike Schroeder 16 Nov '20

16 Nov '20

CentOS 7 Dovecot 2.2.36 Nov 14 07:13:08 mail dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=73.0.0.0, lip=192.64.118.242, TLS handshaking: SSL_accept() failed: error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher, session=<> Was working fine for over a year, until the cert expired and I replaced it. I've tried the good cert I have for https and I used the Dovecot.org script to generate a self-signed certificate. 10-ssl.conf ## SSL settings #ssl = required ssl = yes #ssl = no ssl_cert = </etc/pki/dovecot/certs/mydomain.com.crt ssl_key = </etc/pki/dovecot/private/mydomain.com.key #ssl_ca = #ssl_require_crl = yes #ssl_client_ca_dir = #ssl_client_ca_file = #ssl_verify_client_cert = no #ssl_cert_username_field = commonName #ssl_dh_parameters_length = 1024 #ssl_protocols = !SSLv3 # SSL ciphers to use # ols values ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_cipher_list = ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK: !RC4:!ADH:!LOW@STRENGTH # Prefer the server's order of ciphers over client's. #ssl_prefer_server_ciphers = no # Prefer the server's order of ciphers over client's. #ssl_prefer_server_ciphers = no # SSL crypto device to use, for valid values run "openssl engine" #ssl_crypto_device = # SSL extra options. Currently supported options are: # no_compression - Disable compression. # no_ticket - Disable SSL session tickets. #ssl_options = =========================== # openssl x509 -dates -in mydomain.com.crt notBefore=Nov 11 16:31:35 2020 GMT notAfter=Nov 11 16:31:35 2022 GMT -----BEGIN CERTIFICATE----- : =========================== # openssl pkey -in mydomain.com.key -----BEGIN PRIVATE KEY----- : Thanks for taking a look. Any ideas on what I should do next to debug? Mike

2 1

imap dumping core in libdovecot-storage with assert "(block_count == 0 || block_idx+1 == block_count)"
by Renaud Métrich 15 Nov '20

15 Nov '20

Hi, I would like to report an issue which is 100% reproducible with a specific mail available in the reproducer attached to Red Hat's bugzilla <https://bugzilla.redhat.com/show_bug.cgi?id=1894418> (https://bugzilla.redhat.com/attachment.cgi?id=1726535) -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- Return-Path: <XXXXXXXXX(a)XXXXXXX.XXX.XX> X-Original-To: XXXXXXXXX(a)XXXXXXX.XXX.XX Delivered-To: XXXXXXXXXXXXXXXX(a)XXXXXX.XX.XXX.XX Received: from XXXXXXXXXXXXXXXXX.XXXXX.XX.XX (unknown [XX.XX.XXX.XXX]) by XXXXXX.XX.XXX.XX (Postfix) with ESMTP id XXXXXXXXXXX for <XXXXXXXXX(a)XXXXXXX.XXX.XX>; Thu, 15 Oct 2020 23:11:45 +0200 (CEST) Date: Thu, 15 Oct 2020 23:11:42 +0200 Subject: zoom Message-ID: <XXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXX(a)XXXXX.XXXXXXX.XXX> From: "XXXXXXXXX(a)XXXXXXX.XXX.XX" <XXXXXXXXX(a)XXXXXXX.XXX.XX> To: XXXXXXXXX(a)XXXXXXX.XXX.XX MIME-Version: 1.0 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- The email seems completely valid to me, but I'm no email expert. When performing a "FETCH <ID> (BINARY.SIZE[1] BINARY.PEEK[1])" operation, imap crashes with following backtrace: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- (gdb) bt [...] #5 0x00007ffff7707a25 in i_panic ( format=format@entry=0x7ffff7b7b460 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:523 #6 0x00007ffff7ab5de7 in blocks_count_lines (full_input=0x5555557f1230, ctx=0x7fffffffdb80) at index-mail-binary.c:357 #7 index_mail_read_binary_to_cache (_mail=0x5555557e3b48, part=0x5555557e4678, include_hdr=<optimized out>, reason=<optimized out>, binary_r=0x7fffffffdc16, converted_r=0x7fffffffdc17) at index-mail-binary.c:395 #8 0x00007ffff7b39c8b in index_mail_get_binary_size (lines_r=0x7fffffffdcc4, size_r=0x7fffffffdd20, include_hdr=false, part=0x5555557e4678, _mail=0x5555557e3b48) at index-mail-binary.c:497 #9 index_mail_get_binary_stream (_mail=0x5555557e3b48, part=0x5555557e4678, include_hdr=<optimized out>, size_r=0x7fffffffdd20, lines_r=0x7fffffffdcc4, binary_r=<optimized out>, stream_r=0x0) at index-mail-binary.c:557 #10 0x00007ffff7abd204 in mail_get_binary_size (mail=mail@entry=0x5555557e3b48, part=0x5555557e4678, include_hdr=<optimized out>, size_r=size_r@entry=0x7fffffffdd20, lines_r=lines_r@entry=0x7fffffffdcc4) at mail.c:342 #11 0x00007ffff7b7add8 in imap_msgpart_size (mail=0x5555557e3b48, msgpart=0x5555557e0748, size_r=size_r@entry=0x7fffffffdd20) at imap-msgpart.c:755 #12 0x000055555557917c in fetch_binary_size (ctx=0x5555557c97b0, mail=<optimized out>, body=0x5555557dfa20) at imap-fetch-body.c:212 #13 0x00005555555777e2 in imap_fetch_more_int (ctx=0x5555557c97b0, cancel=false) at imap-fetch.c:562 #14 0x00005555555788ed in imap_fetch_more (ctx=0x5555557c97b0, cmd=cmd@entry=0x5555557c95d8) at imap-fetch.c:617 #15 0x00005555555694b1 in cmd_fetch (cmd=0x5555557c95d8) at cmd-fetch.c:337 [...] #28 0x0000555555566405 in main (argc=<optimized out>, argv=<optimized out>) at main.c:523 -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- This happens on latest dovecot. Best regards, Renaud METRICH

2 3

dovecot fts-solr + solr 8.7.0 upgrade: "Indexing failed: 401 Unauthorized" + "Transaction commit failed: FTS transaction commit failed: backend deinit" ?
by PGNet Dev 14 Nov '20

14 Nov '20

I've built a new dovecot + fts-solr instance; I've now picked up & am running the recently released solr 8.7.0. In a test account, I've one message -- in the 'Drafts' folder. On exec of fts 'index' doveadm index -u testuser(a)example.com -q '*' I get a 401 unauthorized indexing failure, only for the one dir+message in 'Drafts', 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<8MKVNHmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Queue 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<OHfzNHmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in testing 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<yMxJNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Junk 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<uDCbNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Trash 2020-11-13 05:43:21 indexer-worker(testuser(a)example.com)<kA/uNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Sent 2020-11-13 05:43:22 indexer-worker(testuser(a)example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: fts_solr: Indexing failed: 401 Unauthorized 2020-11-13 05:43:22 indexer-worker(testuser(a)example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: fts_solr: Indexing failed: 401 Unauthorized 2020-11-13 05:43:22 indexer-worker(testuser(a)example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: Mailbox Drafts: Transaction commit failed: FTS transaction commit failed: backend deinit (attempted to index 1 messages (UIDs 3..3)) 2020-11-13 05:43:22 indexer-worker(testuser(a)example.com)<2OUMCHqNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in INBOX I get NO output atm in my solr logs @ this 'index'; just, atm, in the dovecot logs, above I've checked all my usual perms. Indexing _was_ working with a prior solr v8.6.3 ... Either I've misconfigured some perm, and am missing it, or new-solr-version v8.7.0 is causing me some grief. (1) Can anyone yet verify Doveoct/fts-solr working with solr 8.7.x? (2) What logging config in Dovecot gets me more/detailed output for the fts_solr fail? (3) Any obvious clues as to what, specifically, is the source of this prob?

2 9

Submission service capabilities listing
by Dan 13 Nov '20

13 Nov '20

Hello all, I'm attempting to implement a submission server with questions in regards to how Dovecot handles the backend hosts capabilities. To my understanding, Dovecot will connect to the host specified in either submission_host or submission_relay_host, and gather a list of capabilities. Once gathered, after the client sends the EHLO greeting Dovecot will respond back with capabilities that the backend host is capable of. My question is, When does this occur? Running a tcpdump with the filter set to the relay host IP, I don't see any traffic until after I send the EHLO command. Dovecot states that CHUNKING is available, but the backend host does not have that capability. Is there a way to disable CHUNKING explicitly? Thanks. submission_relay_host = x.x.x.x submission_host = x.x.x.x submission_relay_ssl = no submission_relay_ssl_verify = no protocol submission { passdb { driver = static args = host=x.x.x.x proxy=y nopassword=y port=587 } }

2 1