- dovecot - dovecot.org

[Dovecot] What to back up from a mdbox
by Dag Nygren 26 Feb '13

26 Feb '13

Hi short question to the list: - What parts of a mdbox mail store directory can I leave out from my backup? I can see huge *.cache files for eample. Can these be rebuilt? - Can dovecot recover from a backup "on the fly"? Best Dag

2 1

[Dovecot] Dovecot SASL: SCRAM-SHA-1 Authentication Fails
by Thomas Reim 26 Feb '13

26 Feb '13

Dear all, I use Dovecot SASL (2.1.15) on Ubuntu 12.04 for IMAP authentication and Postfix SASL authentication. I tried to setup SCRAM-SHA-1 as SASL mechanism. This works well on Dovecot's client side towards my OpenLDAP server (with libsasl-2), but fails on the server side (IMAP and SMTP). In the following, there's an extract from Dovecot's log, when using mutt as SMTP client: dovecot: auth: Debug: auth client connected (pid=0) dovecot: auth: Debug: client in: AUTH#0111#011 SCRAM-SHA-1#011 service=smtp#011 nologin#011 lip=192.168.0.65#011 rip=192.168.0.65#011 secured#011 resp=<hidden> dovecot: auth: scram-sha-1(?,192.168.0.65): authzid not supported dovecot: auth: Debug: client passdb out: FAIL#0111 postfix/smtpd[7621]: warning: markaurel.gas.de[192.168.0.65]: SASL SCRAM-SHA-1 authentication failed Here's the log, when using mutt as IMAP client: dovecot: auth: Debug: auth client connected (pid=23409) dovecot: auth: Debug: client in: AUTH#0111#011 SCRAM-SHA-1#011 service=imap#011 secured#011 session=<session ID>#011 lip=192.168.0.65#011 rip=192.168.0.65#011 lport=143#011 rport=36543#011 resp=<hidden> auth: scram-sha-1(?,192.168.0.65,<session ID>): authzid not supported auth: Debug: client passdb out: FAIL#0111 In the following is mutt's output: imap_authenticate: Trying method scram-sha-1 SASL local ip: 192.168.0.65;36543, remote ip:192.168.0.65;143 External SSF: 128 External authentication name: myname mutt_sasl_cb_authname: getting authname for mail.mydomain.local:143 mutt_sasl_cb_authname: getting user for mail.gas.de:143 mutt_sasl_cb_pass: getting password for myname@mail.mydomain.local:143 Authentifiziere (SCRAM-SHA-1)... 4> a0002 AUTHENTICATE SCRAM-SHA-1 <uuencoded string> 4< a0002 NO [AUTHENTICATIONFAILED] Authentication failed. IMAP queue drained imap_auth_sasl: scram-sha-1 failed I've configured mutt, so that it immediately retries the SASL authentication using DIGEST-MD5. This 2nd try is succesful and mutt gets access to the imap/smtp service. Any hints, what's going wrong here? Output of dovecot -n: # 2.1.15 (e33fe1a7bb89): /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-37-generic x86_64 Ubuntu 12.04.2 LTS auth_debug = yes auth_mechanisms = scram-sha-1 digest-md5 plain login auth_verbose = yes base_dir = /var/run/dovecot/ log_timestamp = "%Y-%m-%d %H:%M:%S " mail_gid = vmail mail_location = mdbox:~/mdbox mail_privileged_group = mail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster@localhost protocols = imap lmtp quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail user = vmail } } service imap-login { inet_listener imap { address = 192.168.0.65 port = 143 } inet_listener imaps { address = 192.168.0.65 port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol imap { mail_max_userip_connections = 10 mail_plugins = " mail_log notify" }

2 1

[Dovecot] 2.1.12: Panic: file solr-connection.c: line 547 (solr_connection_post_more)
by Marc Schiffbauer 26 Feb '13

26 Feb '13

Hi all, I am having a problem indexing one of my mailboxes using the solr fts backend in dovecot 2.1.12 For many mailboxes it works just fine, but on one mailbox I currently always get a panic. solr setup: Java: icedtea 6.1.11.5 Solr: 3.6.2 running in tomcat 7.0.32 Command to reproduce error: doveadm index -u my@user badmailbox I already noticed that there have been some solr backend fixes just before 2.1.21 release... Is this another bug in the backend? Any idea? IMO relevant logs line parts: java.io.CharConversionException: Invalid UTF-8 start byte 0xf8 (at char #483989, byte #532382) complete dovecot log: doveadm(my@user): Error: fts_solr: Invalid XML input at 85:198: mismatched tag (near: <html><head><title>Apache Tomcat/7.0.32-gentoo - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:wh) doveadm(my@user): Panic: file solr-connection.c: line 547 (solr_connection_post_more): assertion failed: (maxfd >= 0) doveadm(my@user): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x4bcdb) [0x34fbd53ecdb] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x3a) [0x34fbd53edb8] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x34fbd50eeac] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x2a4) [0x34fbba81072] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x40a5) [0x34fbba7d0a5] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x462) [0x34fbbc8d6dc] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xd8c2) [0x34fbbc928c2] -> doveadm(+0x17a30) [0x264ecb7a30] -> doveadm(+0x13a45) [0x264ecb3a45] -> doveadm(+0x14762) [0x264ecb4762] -> doveadm(doveadm_mail_try_run+0x173) [0x264ecb4cfd] -> doveadm(main+0x3e9) [0x264ecbd0d5] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x34fbd16a48d] -> doveadm(+0x13809) [0x264ecb3809] Aborted complete java exception: Dec 31, 2012 4:12:58 AM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr-web path=/update params={} status=500 QTime=118 Dec 31, 2012 4:12:58 AM org.apache.solr.common.SolrException log SEVERE: java.lang.RuntimeException: [was class java.io.CharConversionException] Invalid UTF-8 start byte 0xf8 (at char #483989, byte #532382) at com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18) at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731) at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657) at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809) at org.apache.solr.handler.XMLLoader.readDoc(XMLLoader.java:315) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:156) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Unknown Source) at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown Source) at org.apache.catalina.core.StandardWrapperValve.invoke(Unknown Source) at org.apache.catalina.core.StandardContextValve.invoke(Unknown Source) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Unknown Source) at org.apache.catalina.core.StandardHostValve.invoke(Unknown Source) at org.apache.catalina.valves.ErrorReportValve.invoke(Unknown Source) at org.apache.catalina.valves.AccessLogValve.invoke(Unknown Source) at org.apache.catalina.core.StandardEngineValve.invoke(Unknown Source) at org.apache.catalina.connector.CoyoteAdapter.service(Unknown Source) at org.apache.coyote.http11.AbstractHttp11Processor.process(Unknown Source) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Unknown Source) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Caused by: java.io.CharConversionException: Invalid UTF-8 start byte 0xf8 (at char #483989, byte #532382) at com.ctc.wstx.io.UTF8Reader.reportInvalidInitial(UTF8Reader.java:302) at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:188) at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84) at com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57) at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:992) at com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4628) at com.ctc.wstx.sr.BasicStreamReader.readCoalescedText(BasicStreamReader.java:4126) at com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3701) at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3649) ... 25 more TIA -Marc -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich

4 9

[Dovecot] Dovecot auth works when tested with doveadm, but fails with Postfix
by Darren Pilgrim 26 Feb '13

26 Feb '13

I have three postfix 2.9.5 servers: chombo, rush, yoshi. Chombo relays to rush and yoshi for outbound email. Outbound relay requires SASL authentication. Rush and yoshi run Dovecot 2.1.12 servers with simple passwd-file backends. If I create a new password hash for chombo's user, houseloki, on either rush or yoshi: # doveadm pw -u houseloki -p <password> {CRAM-MD5}... Then I add that to rush and yoshi's passwd file: houseloki@_auth.bluerosetech.com:{CRAM-MD5}... Then `doveadm reload`, it works fine: # doveadm auth houseloki <password> passdb: houseloki auth succeeded extra fields: user=houseloki(a)_auth.bluerosetech.com So I add that username and password to the smtp_sasl_password_maps hash file on chombo, reload postfix, and then try to relay something from chombo, it fails with rush and yoshi logging warnings like: Feb 19 03:32:33 yoshi postfix/smtpd[75783]: warning: chombo.example.com[2001:db8::2]: SASL DIGEST-MD5 authentication failed: <really long string redacted> Rush and yoshi have other hashes in their passwd files, and if I configure chombo to use one of those, it works fine. Those hashes use digest-md5 and are at least a few years old. I tried that scheme instead of the default cram-md5, as well as several others, but none work. I've poured over the wiki and man pages, but can't find the problem. What am I missing? Why can I not generate new hashes correctly?

2 3

Re: [Dovecot] v2.2.rc1 released
by Michael Grimm 26 Feb '13

26 Feb '13

> Please test and report any bugs found. New in rc1: | doveconf: Error: dlopen(/.../lib/dovecot/settings/libmanagesieve_login_settings.so) failed: /.../lib/dovecot/settings/libmanagesieve_login_settings.so: Undefined symbol "hook_config_parser_begin" | doveconf: Error: dlopen(/.../lib/dovecot/settings/libmanagesieve_settings.so) failed: /.../lib/dovecot/settings/libmanagesieve_settings.so: Undefined symbol "mail_user_setting_parser_info" | doveconf: Error: protocols: Unknown protocol: sieve | doveconf: Fatal: Error in configuration file /.../etc/dovecot/dovecot.conf: protocols: Unknown protocol: sieve Regards, Michael

3 9

[Dovecot] 2.2beta2: LSUB broken
by Lutz Preßler 26 Feb '13

26 Feb '13

Hello, I have some problems with (listing) subscriptions on my test system running 2.2beta2 (compared with 2.1.15, configuration and Maildir content unchanged): With 2.1.15: 0 lsub "" "*" * LSUB () "." "INBOX.Trash" * LSUB () "." "INBOX.in2009" * LSUB () "." "INBOX.test" * LSUB () "." "INBOX.test.test2" * LSUB () "." "INBOX.in2010" * LSUB () "." "INBOX.in" * LSUB () "." "INBOX.virtual.small" * LSUB () "." "INBOX.virtual.sent" * LSUB () "." "INBOX.virtual.allin" * LSUB () "." "INBOX.virtual.week" * LSUB () "." "INBOX.virtual.gmail" 0 OK Lsub completed. With 2.2beta2 (output of doveconf -n attached): 0 lsub "" "*" 0 OK Lsub completed. ... empty list. Same with "list (subscribed)". After disableing the "virtual" namespace in the config, it gets: 0 lsub "" "*" * LSUB () "." Trash * LSUB () "." in2009 * LSUB () "." test * LSUB () "." test.test2 * LSUB () "." in2010 * LSUB () "." in * LSUB () "." INBOX.Trash * LSUB () "." INBOX.in2009 * LSUB () "." INBOX.test * LSUB () "." INBOX.test.test2 * LSUB () "." INBOX.in2010 * LSUB () "." INBOX.in 0 OK Lsub completed. Note mailboxes listed with and without INBOX. prefix. Thanks, Lutz

2 3

[Dovecot] v2.2.rc1 released
by Timo Sirainen 25 Feb '13

25 Feb '13

http://dovecot.org/releases/2.2/rc/dovecot-2.2.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.rc1.tar.gz.sig A few fixes since beta2. Also added tcp and tcps (=tcp+ssl) targets for dsync, where the destination needs to be a doveadm-server listener. I'm planning on releasing v2.2.0 in 1-2 weeks if no major problems are found. Please test and report any bugs found. BTW. I've also gone through basically all of my unread emails. If you haven't gotten a reply to something, ask again. Here's again all the biggest changes since v2.1: * When creating home directories, the permissions are copied from the parent directory if it has setgid-bit set. For full details, see http://wiki2.dovecot.org/SharedMailboxes/Permissions * "doveadm auth" command was renamed to "doveadm auth test" * IMAP: ID command now advertises server name as Dovecot by default. It was already trivial to guess this from command replies. * dovecot.index.cache files can be safely accessed only by v2.1.11+. Older versions may think they're corrupted and delete them. * LDA/LMTP: If saving a mail brings user from under quota to over quota, allow it based on quota_grace setting (default: 10% above quota limit). * pop3_lock_session=yes now uses a POP3-only dovecot-pop3-session.lock file instead of actually locking the mailbox (and causing IMAP/LDA/LMTP to wait for the POP3 session to close). * mail_shared_explicit_inbox setting's default switched to "no". * ssl_client_ca_dir setting replaced imapc_ssl_ca_dir and pop3c_ssl_ca_dir settings. + Implemented IMAP MOVE and BINARY extensions + Implemented IMAP CATENATE, URLAUTH and URLAUTH=BINARY extensions (by Stephan Bosch). + Implemented IMAP NOTIFY extension. Requires mailbox_list_index=yes to be enabled. + Redesigned and rewritten dsync. The new design makes the syncing faster, more reliable and more featureful. The new dsync protocol isn't backwards compatible with old dsync versions (but is designed to be forwards compatible with future versions). + All mailbox formats now support per-user message flags for shared mailboxes by using a private index. It can be enabled by adding :INDEXPVT=<path> to mail location. This should be used instead of :INDEX also for Maildir/mbox to improve performance. + Improved mailbox list indexes. They should be usable now, although still disabled by default. + Added LAYOUT=index. The mailbox directories are created using their GUIDs in the filesystem, while the actual GUID <-> name mapping exists only in the index. + LMTP proxy: Implemented XCLIENT extension for passing remote IP address through proxy.

1 0

[Dovecot] Bug when a duplicated file is found
by Angel L. Mateo 25 Feb '13

25 Feb '13

Hello, Last days we had to restore some mailboxes from backup. Our mailboxes are in maildir format and during restore, some duplicated files appeared. The situation is: * A user has messages in his new directory when the backup is made * In some moment after the backup is done, the user accesses to his mailbox, so the message is move to the cur directory. * We restore his mailbox directly to his maildir path, so the message is now at new directory too. When this happens, dovecot detect it and it renames one of the files. This is the reported log: Feb 13 23:08:05 myotis31 dovecot: imap(<user>): Warning: Fixed a duplicate: /home/alumnos/59/122559/Maildir/cur/1360169810.M670208P13643.myotis31,S=6282,W=6409:2, -> 1360793285.M721935P29136.myotis31 The problem with this is that we are using zlib plugin so messages are compressed. When the rename is done, the rename is to a name without the S=XXXX field (neither the W=XXX, although this field is less sensible), so the result is that user's can't access to any message after this one. I think than in this situation, rename one of the files is a good solution (although it provokes duplicate mails, but this an expected/acceptable situation after the restore), but the rename should maintain the S= and W= fields (and even other maildir flags). -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337

3 3

[Dovecot] SSL errors for just one client after updaing both dovecot and openssl
by Charles Marcus 25 Feb '13

25 Feb '13

Hi all, Ok, I have a strange problem after updating both dovecot and openssl... OpenSSL was 1.0.0j, now updated to 1.0.1c Dovecot was 2.1.13, now updated to 2.1.15 I'm getting a bunch of lines like the following: Feb 23 10:48:01 myhost dovecot: imap-login: Disconnected (no auth attempts in 29 secs): user=<>, rip=#.#.#.#, lport=993, TLS handshaking: SSL_accept() syscall failed: Connection reset by peer, session=<In+cO2bWngCthJz2> where only the session id (and number of seconds for no auth attempts) is different... This is happening for only the one client. All other clients - I've counted about 25 so far - are working fine. Anyone have any ideas? I can't believe it is a generic openssl problem, since it is only affecting the one client. I've contacted him and asked him to reboot any/all devices that connect to our mail to see if that helps... -- Best regards, */Charles/*

3 3

[Dovecot] understanding namespace configuration
by btb＠bitrate.net 25 Feb '13

25 Feb '13

i'm a little confused about namespace configuration. reading http://wiki2.dovecot.org/Namespaces , and looking at the examples, it seems that when a namespace is declared, you can either do "namespace { ..." or "namespace foo { ..." - i'm having trouble finding clarity on the significance of "foo" though. it would appear it's not required, but then what is its purpose? what does it mean? does it imply something? it's also unclear to me how to create the "default private namespace". i see that there are three types of namespaces, but the list of namespace settings doesn't seem to include a parameter to set the namespace type. what am i missing? it's also not quite clear how to denote a namespace as "default". from coming across this thread [ http://mailman2.u.washington.edu/pipermail/imap-protocol/2010-May/001077.ht… ] it would indicate in theory that a namespace is denoted as default by using an empty prefix - but i wasn't able to find corroboration of that in dovecot's documentation. thanks -ben

2 3