- dovecot - dovecot.org

Indexing fails with .. FIELDS_INDEX_EXTENSION).c_str() )' failed
by Akash 14 Oct '15

14 Oct '15

Hi, Mailing list archives suggest that this problem has been fixed in the older versions itself but I am getting the error still while trying to doveadm index a large folder of emails. Sometimes it throws error after 30000, something 40000, the latest it gave up after 111000. But it just never completes. 111000/322080doveadm: /home/stephan/packages/wheezy/i386/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:210: std::string lucene::index::DocumentsWriter::closeDocStore(): Assertion `numDocsInStore*8 == directory->fileLength( (docStoreSegment + "." + IndexFileNames::FIELDS_INDEX_EXTENSION).c_str() )' failed. Aborted I am using dovecot 2:2.2.19-1~auto+7& libclucene-core1:i386 2.3.3.4-4 from debian wheezy backports. Please advice. -Akash

3 2

Re: iterate users with passwd-file passdb?
by Joseph Tam 14 Oct '15

14 Oct '15

Eric Abrahamsen <eric(a)ericabrahamsen.net> writes: >>> Simply: Is it possible to iterate over users if I'm using the >>> passwd-file passdb driver? Do I need a SQL-based driver if I want to >>> iterate? >> >> What do you mean by "iterate"? If you mean whether you can look up a >> password entry in a multi-entry file, then yes, definitely. If you >> mean to sequentially go through it and do a first/last/best match, >> probably not. > > Basically I just mean the -A argument to the doveadm commands. For > example, iterate through all the users and run sa-learn on each user's > learn/spam folder. If it's got to be SQL, it won't kill me... Ah. Yes, you can use -A and iterate through users, subject to any constraints such as first_valid_uid, last_valid_uid, etc. Joseph Tam <jtam.home(a)gmail.com>

1 0

Dovecot - Postfix with HAproxy
by Michael JOIGNY 13 Oct '15

13 Oct '15

Hi Everyone, I wouldlike to set up a postfix-dovecot with HA using HAproxy but im facing issues. I've followed this documentation : http://wiki2.dovecot.org/HAProxy (pour dovecot) http://blog.haproxy.com/2012/06/30/efficient-smtp-relay-infrastructure-with… (pour postfix) Package's version : dovecot : 2:2.2.19 (>= 2.2.19 pour proxy protocol) haproxy : 1.5.14 postfix : 2.11.2-1 (>2.10 pour postscreen) A part of my configuration : ##HAPROXY #postfix listen smtp bind mail.xx.xx:465 balance roundrobin timeout client 1m timeout connect 5s no option http-server-close mode tcp option smtpchk option tcplog server tst tst.xxx:10465 send-proxy server tst2 tst2.xxx:10465 send-proxy server tst3 tst3.xxx:10465 send-proxy #dovecot listen imap bind mail.xxx.xx:993 timeout client 1m no option http-server-close balance leastconn stick store-request src stick-table type ip size 200k expire 30m mode tcp option tcplog server tst tst.xxx:10993 send-proxy-v2 server tst2tst2.xxx:10993 send-proxy-v2 server tst3 tst3.xxx:10993 send-proxy-v2 ##POSTFIX postix main.cf #Haproxy proxy protocol postscreen_upstream_proxy_protocol = haproxy postfix master.cf #haproxy 10465 inet n – n – 1 postscreen smtpd pass – – n – – smtpd S ##DOVECOT # 2.2.19 (719e7f8fd70b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 # OS: Linux 2.6.32-41-pve x86_64 Debian 7.9 simfs auth_debug = yes auth_verbose = yes disable_plaintext_auth = no *haproxy_timeout = 5 secs** **haproxy_trusted_networks = x.x.x.x* log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/mailbox/%d/%n mail_max_userip_connections = 0 mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /mailbox/globalsieverc } protocols = sieve pop3 imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0660 user = mailboxes } unix_listener auth-userdb { group = mail mode = 0666 user = dovecot } user = root } service imap-login { inet_listener imap { port = 0 } inet_listener imap_haproxy { haproxy = yes port = 10993 } inet_listener imaps { address = * port = 993 } process_limit = 450 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_limit = 180 } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/star.key userdb { args = uid=5000 gid=5000 home=/mailbox/%d/%n/ allow_all_users=yes driver = static } protocol imap { imap_client_workarounds = } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol sieve { mail_location = maildir:/mailbox } protocol lda { auth_socket_path = /var/run/dovecot/auth-master hostname = xxx.yyy.com log_path = /var/log/dovecot/deliver.log mail_plugin_dir = /usr/lib/dovecot/modules mail_plugins = sieve postmaster_address = postmaster(a)xxx.yyy rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail } With my mail client : With an IMAP connection, logs below, i don't understand why my login is empty ... dovecot: imap-login: Disconnected: Too many invalid commands (no auth attempts in 0 secs): *user=<>*, rip=mon_ip_publique, lip=ip_publique_haproxy, session= xxx With a SMTP connection, logs below, i have a timeout. postfix/postscreen[16654]: CONNECT from [my public ip]:49942 to [my haproxy public ip]:465 postfix/postscreen[16654]: PREGREET 166 after 0 from [mon ip publique]:49942: \22\3\1\161\1\157\3\3+0E\b\213\131\177\173>\r/\213\177i\223k”FjA#\144\145\153\vP\\\155HL\190 If someone could help me, thanks. Kind regards. --

2 1

Dualstack IPv4/IPv6 setup with directors
by Heiko Schlittermann 13 Oct '15

13 Oct '15

Hi, still using 2.2.9, I've two directors, and these directors use both IPv4/IPv6 addresses. `host directors.<domain>` returns one A and AAA for each of the two directors: directors.<domain> has address 149.x.y.96 (director1) directors.<domain> has address 149.x.y.97 (director2) directors.<domain> has IPv6 address 2001:x:y:f33::5:1 (director1) directors.<domain> has IPv6 address 2001:x:y:f33::5:2 (director2) But connecting the the neighbor director gives: Oct 13 21:23:29 director1 dovecot: director: Warning: Director 2001:x:y:f33::5:2:9090/left disconnected us with reason: Invalid input: ME 149.x.y.96 9090 Oct 13 21:23:29 director1 dovecot: director: Error: director(149.x.y.97:9090/out): connect() failed: Connection refused Oct 13 21:23:29 director1 dovecot: director: Warning: net_connect_ip(): ip->family != my_ip->family Oct 13 21:23:29 director1 dovecot: director: Error: Remote director thinks it's someone else (connected to 2001:x:y:f33::5:1:9090, remote says it's 149.x.y.96:9090) I assume it's because the director uses only one (IPv4 OR IPv6) address for it's own identity, right? Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ -

2 2

Storage Design (Regligious War, Whatever)
by Kelsey Cummings 13 Oct '15

13 Oct '15

We've been running dovecot director setup using MAILDIR++ spools on NFS served by 4 (aging) Netapp clustered filers and are considering modernizing our storage. There's nothing particularly wrong with the existing storage except the age, density and power consumption don't compare well when against new systems. Refreshing the Netapps is an option but they come at a substantial cost. The last time we were looking one of the recommended hot DIY setups was Linux NFS backed by XFS on N mirrors but this predated the availability of stable ZFS in FreeBSD or Linux as well as products like FreeNAS. A FreeBSD NFS/ZFS filer on commodity hardware with zil/l2arc on a PCIe SSD seems like an attractive, affordable and easily scaled out solution which also would allow us to leverage compression at the filesystem layer. Does anyone have any experience running ZFS spool storage? If so, how do you handle DR/HA for spool storage? Thoughts on how this might compare to using DRBD? Any pitfalls to watch out for or general pointers? Suggestions on pool configuration? -K

1 0

Unknown cache fields?
by Larry Rosenman 13 Oct '15

13 Oct '15

I assume this is expected, but just checking. I do NOT use POP3. Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring unknown cache field: pop3.order Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring unknown cache field: binary.parts Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring unknown cache field: body.snippet -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961

2 1

doveadm index assertion failed
by Nick Rosier 13 Oct '15

13 Oct '15

Hi, one of my mailboxes returns following error when I run doveadm index on it: Panic: file charset-iconv.c: line 85 (charset_to_utf8_try): assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE) OS: FreeBSD 10.2 Dovecot: 2.1.19 Tika: 1.10 SOLR: 5.3.1 Running doveadm -D index does not show any more information indicating what causes this error (mail/folder/...) How I can find what is causing this error? Rgds, N.

2 1

Re: bug report: dovecot-imapd core dump
by Tamas Papp 13 Oct '15

13 Oct '15

Hi, could not find it -- back to the list, maybe someone can suggest a fix/workaround. Best, Tamas On Wed, Oct 07 2015, Dominik Breu wrote: > Hello, > > yeah this one looks familiar to me can you search the list back in > september there was a somewhat similar bug with thunderbird imho timo > posted a notice about it. > > greetings, > > dominik > > Am Mittwoch, den 07.10.2015, 15:53 +0200 schrieb Tamas Papp: >> Hi Dominik, >> >> Thanks for the help, the backtrace is here (I did not sent it to the >> list since I am not sure if it contains password or message >> information, >> sorry but I am unfamiliar with these things). >> >> Core was generated by `dovecot/imap'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x00007f404e58e3b6 in mail_index_keywords_unref >> #(_keywords=_keywords@entry=0x7ffecb8adeb8) at mail-index.c:383 >> 383 mail-index.c: No such file or directory. >> (gdb) bt full >> #0 0x00007f404e58e3b6 in mail_index_keywords_unref >> #(_keywords=_keywords@entry=0x7ffecb8adeb8) at mail-index.c:383 >> keywords = 0x7f404e27066d <buffer_alloc+45> >> __FUNCTION__ = "mail_index_keywords_unref" >> #1 0x00007f404e559975 in mailbox_keywords_unref >> (keywords=keywords@entry=0x7ffecb8adeb8) at mailbox-keywords.c:94 >> No locals. >> #2 0x00007f404ea254ad in cmd_append_handle_args >> (nonsync_r=0x7ffecb8ade8f, >> #args=<optimized out>, cmd=0x7f4050bc3250) at cmd-append.c:602 >> ctx = 0x7f4050bc3348 >> flags_list = 0x7f4050bcc840 >> flags = 9 >> keywords_list = 0x7f4050bc33f0 >> internal_date_str = 0x7f4050bcca00 "17-Sep-2015 14:19:23 >> +0200" >> internal_date = -1 >> timezone_offset = 0 >> keywords = 0x0 >> ret = <optimized out> >> client = 0x7f4050bc2670 >> cat_list = 0x0 >> input = <optimized out> >> valid = <optimized out> >> #3 cmd_append_parse_new_msg (cmd=cmd@entry=0x7f4050bc3250) at >> cmd-append.c:770 >> client = <optimized out> >> ctx = 0x7f4050bc3348 >> args = 0x7f4050bcc488 >> msg = <optimized out> >> arg_min_count = <optimized out> >> fatal = 255 >> nonsync = true >> last_literal = <optimized out> >> ret = <optimized out> >> __FUNCTION__ = "cmd_append_parse_new_msg" >> #4 0x00007f404ea25a17 in cmd_append (cmd=0x7f4050bc3250) at >> cmd-append.c:932 >> client = 0x7f4050bc2670 >> ctx = 0x7f4050bc3348 >> mailbox = 0x7f4050bafb78 "Cemex" >> #5 0x00007f404ea30e4c in command_exec (cmd=cmd@entry=0x7f4050bc3250) >> at >> imap-commands.c:158 >> hook = 0x7f4050bad0d0 >> ret = <optimized out> >> #6 0x00007f404ea2fd62 in client_command_input (cmd=0x7f4050bc3250) >> at >> imap-client.c:780 >> client = 0x7f4050bc2670 >> command = <optimized out> >> __FUNCTION__ = "client_command_input" >> #7 0x00007f404ea2fe16 in client_command_input (cmd=0x7f4050bc3250) >> at >> imap-client.c:841 >> client = 0x7f4050bc2670 >> command = <optimized out> >> __FUNCTION__ = "client_command_input" >> #8 0x00007f404ea3014d in client_handle_next_command >> (remove_io_r=<synthetic >> pointer>, client=0x7f4050bc2670) at imap-client.c:879 >> ---Type <return> to continue, or q <return> to quit--- >> No locals. >> #9 client_handle_input (client=0x7f4050bc2670) at imap-client.c:891 >> _data_stack_cur_id = 3 >> remove_io = false >> handled_commands = false >> client = 0x7f4050bc2670 >> #10 0x00007f404ea304f5 in client_input (client=0x7f4050bc2670) at >> imap-client.c:933 >> cmd = 0xf013c >> output = 0x7f4050bc30a0 >> bytes = 135 >> __FUNCTION__ = "client_input" >> #11 0x00007f404e285247 in io_loop_call_io (io=0x7f4050bc3170) at >> ioloop.c:388 >> ioloop = 0x7f4050bac720 >> t_id = 2 >> #12 0x00007f404e286079 in io_loop_handler_run >> (ioloop=ioloop@entry=0x7f4050bac720) at ioloop-epoll.c:220 >> ctx = 0x7f4050bad3b0 >> io = <optimized out> >> tv = {tv_sec = 1799, tv_usec = 999282} >> events_count = <optimized out> >> msecs = <optimized out> >> ret = 1 >> i = 0 >> j = <optimized out> >> call = <optimized out> >> __FUNCTION__ = "io_loop_handler_run" >> #13 0x00007f404e284d88 in io_loop_run (ioloop=0x7f4050bac720) at >> ioloop.c:412 >> __FUNCTION__ = "io_loop_run" >> #14 0x00007f404e231de3 in master_service_run (service=0x7f4050bac5b0, >> callback=<optimized out>) at master-service.c:566 >> No locals. >> #15 0x00007f404ea2424e in main (argc=1, argv=0x7f4050bac390) at >> main.c:400 >> set_roots = {0x7f404ec489c0 <imap_setting_parser_info>, 0x0} >> login_set = {auth_socket_path = 0x7f4050ba4048 "\001", >> postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = >> 0x7f404ea39f40 <login_client_connected>, >> failure_callback = 0x7f404ea39c20 <login_client_failed>, >> request_auth_token = 1} >> service_flags = <optimized out> >> storage_service_flags = <optimized out> >> username = 0x0 >> c = <optimized out> >> (gdb) quit >> >> Best, >> >> Tamas >> >> On Wed, Oct 07 2015, Dominik Breu wrote: >> >> > Hello >> > >> > Tamas it would be a better backtace when you have installed the >> > dovecot >> > debug symboly mostly located under dovecote-dbg in apt. Further >> > debug >> > help can you find in the wiki. >> > >> > greetings >> > >> > dominik >> > >> > Am Mittwoch, den 07.10.2015, 15:09 +0200 schrieb Tamas Papp: >> > > Typical log message: >> > > >> > > Oct 7 11:39:36 hostname dovecot: imap(username): Fatal: master: >> > > service(imap): child 14864 killed with signal 11 (core dumped) >> > > >> > > What triggers the problem: >> > > >> > > User moving multiple messages to a subfolder via IMAP (from >> > > another >> > > account, on another server, client: Thunderbird). Moving single >> > > messages >> > > either does not trigger the problem, or with much lower >> > > probability >> > > (could not reproduce). >> > > >> > > Version: >> > > $ dpkg -l 'dovecot*' >> > > Desired=Unknown/Install/Remove/Purge/Hold >> > > > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig >> > > > -aWait/Trig-pend >> > > > / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) >> > > > > / Name Version >> > > > > Architecture Description >> > > +++-=====================================-======================= >> > > -======================= >> > > -================================================================ >> > > ==== >> > > ============ >> > > un dovecot-common <none> >> > > <none> (no description available) >> > > ii dovecot-core 1:2.2.9-1ubuntu5 >> > > amd64 secure POP3/IMAP server - core files >> > > un dovecot-gssapi <none> >> > > <none> (no description available) >> > > ii dovecot-imapd 1:2.2.9-1ubuntu5 >> > > amd64 secure POP3/IMAP server - IMAP daemon >> > > un dovecot-ldap <none> >> > > <none> (no description available) >> > > un dovecot-lmtpd <none> >> > > <none> (no description available) >> > > un dovecot-managesieved <none> >> > > <none> (no description available) >> > > un dovecot-mysql <none> >> > > <none> (no description available) >> > > un dovecot-pgsql <none> >> > > <none> (no description available) >> > > un dovecot-pop3d <none> >> > > <none> (no description available) >> > > un dovecot-sieve <none> >> > > <none> (no description available) >> > > un dovecot-solr <none> >> > > <none> (no description available) >> > > un dovecot-sqlite <none> >> > > <none> (no description available) >> > > tamas@szob:~$ lsb_release -a >> > > No LSB modules are available. >> > > Distributor ID: Ubuntu >> > > Description: Ubuntu 15.04 >> > > Release: 15.04 >> > > Codename: vivid >> > > >> > > Also tried 2.2.18 from Ubuntu wiley, same results. >> > > >> > > Backtrace (I am not sure this is useful, extraced core from >> > > Ubuntu's >> > > Apport): >> > > >> > > (gdb) bt full >> > > #0 0x00007f404e58e3b6 in mail_index_keywords_unref () from >> > > /usr/lib/dovecot/libdovecot-storage.so.0 >> > > No symbol table info available. >> > > #1 0x00007f404ea254ad in _start () >> > > No symbol table info available. >> > > >> > > Any help would be appreciated, I am happy to provide more info if >> > > necessary. >> > > >> > > Tamas

2 2

dovecot as proxy and verification of the backends certificate
by Heiko Schlittermann 13 Oct '15

13 Oct '15

Hello, I'm using a dovecot as proxy, connecting to one or more backends. The backends use X.509 certificates. The proxy's passdb returns extra fields: user=foo proxy host=backend1.<domain> ssl=yes nopassword=y Thus the proxy connects to the backend but can't verify the backends certificate. The following comment suggests using ssl_client_ca_file for that. # Directory and/or file for trusted SSL CA certificates. These are used only # when Dovecot needs to act as an SSL client (e.g. imapc backend). The # directory is usually /etc/ssl/certs in Debian-based systems and the file is # /etc/pki/tls/cert.pem in RedHat-based systems. #ssl_client_ca_dir = #ssl_client_ca_file = ssl_client_ca_file = /tmp/certs/ca-local.pem But that does not work! Instead I've to use ssl_ca # PEM encoded trusted certificate authority. Set this only if you intend to use # ssl_verify_client_cert=yes. The file should contain the CA certificate(s) # followed by the matching CRL(s). (e.g. ssl_ca = </etc/ssl/certs/ca.pem) # ssl_ca = ssl_ca = </tmp/certs/ca-local.pem Bug or feature? Mainly I'm asking because the comments do not indicate that I should have used ssl_ca for this type of operation (dovecot as a SSL client) Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ -

2 2

Dovecot auth-ldap ignores tls_* settings when using ldaps://
by Heiko Schlittermann 13 Oct '15

13 Oct '15

Hi, I'm using dovecot 2.2.9 (but after checking src/auth/db-ldap.c in 2.2.13 there seems to be the same bug/feature). The userdb and passdb use LDAP. All further configuration is done in auth-ldap.conf.ext. uri = ldaps://<host>/ # tls = tls_cert_file = /etc/ssl/certs/client-cert.pem tls_key_file = /etc/ssl/certs/client-key.file Dovecot ignores the tls_* options. If I use an ldap:// URI and switch on TLS using tls=yes it works as expected. But I do not see any reason why LDAPS should not read the tls_* settings. This small patch solved it for me --- dovecot-2.2.9/src/auth/db-ldap.c 2013-11-24 14:37:39.000000000 +0100 +++ dovecot-2.2.9.hs12/src/auth/db-ldap.c 2015-10-08 21:24:47.051446465 +0200 @@ -1043,7 +1043,7 @@ static void db_ldap_set_tls_options(struct ldap_connection *conn) { - if (!conn->set.tls) + if (!(conn->set.tls || strncmp(conn->set.uris, "ldaps:", 6) == 0)) return; #ifdef OPENLDAP_TLS_OPTIONS It would be great, if somebody can confirm this and if this or some equivalent patch could make it upstream. Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ -

2 2