July 2009 - dovecot - dovecot.org

[Dovecot] mbox format and UIDVALIDITY
by Axel Luttgens 08 Jul '09

08 Jul '09

My base concern may be illustrated with the help of that simple telnet session: # telnet 127.0.0.1 imap Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE AUTH=PLAIN] Dovecot ready. a1 login testuser ****** a1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH] Logged in a2 list "" % * LIST (\NoInferiors \UnMarked) "/" "Sent" * LIST (\HasNoChildren \UnMarked) "/" "INBOX" a2 OK List completed. a3 create new1 a3 OK Create completed. a4 create new2 a4 OK Create completed. a5 select new1 * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1] UIDs valid * OK [UIDNEXT 1] Predicted next UID * OK [HIGHESTMODSEQ 1] a5 OK [READ-WRITE] Select completed. a6 select new2 * OK [CLOSED] * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1] UIDs valid * OK [UIDNEXT 1] Predicted next UID * OK [HIGHESTMODSEQ 1] a6 OK [READ-WRITE] Select completed. a7 unselect a7 OK Unselect completed. a8 delete new1 a8 OK Delete completed. a9 delete new2 a9 OK Delete completed. a10 logout * BYE Logging out a10 OK Logout completed. Connection closed by foreign host. During the whole telnet session, both files "new1" and "new2" remained empty (i.e. no pseudo-message written to them), no uidvalidity-related helper file seems to have been created and no uidvalidity-related function seems to have been called. And clearly, both mailboxes "new1" and "new2" have received the same UIDVALIDITY. Those UIDVALIDITYs appear to be persistent; messages may be added to and deleted from those mailboxes, those mailboxes may be renamed... their UIDVALIDITY remain unchanged and equal to 1. In fact, any additional mailbox is created with that unique and persistent UIDVALIDITY of 1. Is this supposed to behave that way? (note that I quickly tried with dovecot 1.1.16 and could observe the same behavior) TIA, Axel # 1.2.rc7: /usr/local/etc/dovecot.conf # OS: Darwin 9.7.0 i386 protocols: pop3 imap ssl: no disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/dovecot-1.2.rc7/libexec/dovecot/ imap-login login_executable(imap): /usr/local/dovecot-1.2.rc7/libexec/dovecot/ imap-login login_executable(pop3): /usr/local/dovecot-1.2.rc7/libexec/dovecot/ pop3-login mail_max_userip_connections(default): 4 mail_max_userip_connections(imap): 4 mail_max_userip_connections(pop3): 10 first_valid_uid: 2001 mail_location: mbox:~/_mailboxes:INBOX=~/_mailboxes/inbox:CONTROL=~/ _mboxesctrl mail_debug: yes mbox_read_locks: flock mbox_write_locks: flock dotlock mail_executable(default): /usr/local/dovecot-1.2.rc7/libexec/dovecot/ imap mail_executable(imap): /usr/local/dovecot-1.2.rc7/libexec/dovecot/imap mail_executable(pop3): /usr/local/dovecot-1.2.rc7/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/dovecot-1.2.rc7/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/dovecot-1.2.rc7/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/dovecot-1.2.rc7/lib/dovecot/pop3 pop3_lock_session(default): no pop3_lock_session(imap): no pop3_lock_session(pop3): yes pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %08Xv%08Xu auth default: debug: yes passdb: driver: pam args: * userdb: driver: passwd

2 5

[Dovecot] Wrong credential caching
by Ralf Hildebrandt 08 Jul '09

08 Jul '09

A user logged in with a captial "L" for his username: Jul 7 12:30:31 postamt dovecot: auth(default): cache(Loser,10.47.64.227): miss Jul 7 12:30:31 postamt dovecot: auth(default): shadow(Loser,10.47.64.227): lookup Jul 7 12:30:31 postamt dovecot: auth(default): shadow(Loser,10.47.64.227): unknown user Jul 7 12:30:31 postamt dovecot: auth(default): cache(Loser,10.47.64.227): miss Jul 7 12:30:31 postamt dovecot: auth-worker(default): pam(Loser,10.47.64.227): lookup service=dovecot Jul 7 12:30:31 postamt dovecot: auth-worker(default): pam(Loser,10.47.64.227): #1/1 style=1 msg=Password: Jul 7 12:30:33 postamt dovecot: auth-worker(default): pam(Loser,10.47.64.227): unknown user Then I told him that he's supposed to use an "l" instead of "L" but: Jul 7 12:42:01 postamt dovecot: imap-login: Login: user=<loser>, method=PLAIN, rip=10.47.64.227, lip=141.42.4.250, TLS Jul 7 12:42:02 postamt dovecot: auth(default): client in: ... Jul 7 12:42:02 postamt dovecot: auth(default): cache(Loser,10.47.64.227): hit: Jul 7 12:42:02 postamt dovecot: auth(default): cache(Loser,10.47.64.227): User unknown Jul 7 12:42:02 postamt dovecot: auth(default): cache(Loser,10.47.64.227): hit: Jul 7 12:42:02 postamt dovecot: auth(default): cache(Loser,10.47.64.227): User unknown Jul 7 12:43:50 postamt dovecot: imap-login: Disconnected (auth failed, 1 attempts): user=<Loser>, method=PLAIN, rip=10.47.64.227, lip=141.42.4.250, TLS: Disconnected I think the auth cache may work case-insensitive, thus making the "User invalid" response for "Loser" also valid for "loser" (which is the valid and correct username!) -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

2 2

[Dovecot] Dovecot 1.2rc7 / Sieve Plugin problem / Namespaces
by reg9009 08 Jul '09

08 Jul '09

Hi, I've got a problem with Dovecot 1.2rc7 and a Sieve script which should store eMails in a subfolder. It always fails with Jun 28 14:56:43 deliver(): Error: sieve: msgid=<4A47687D.30903(a)some.domain>: failed to store into mailbox '': Invalid mailbox name Jun 28 14:56:43 deliver(): Error: sieve: script /home/dovecot/sieve/info/dovecot.sieve failed with unsuccessful implicit keep This is with dovecot-libsieve. Same happens with cmusieve, though the message is slightly different: Jun 28 14:55:39 deliver(): Info: cmusieve: Executing script /home/dovecot/sieve/info/dovecot.sievec Jun 28 14:55:39 deliver(): Info: sieve runtime error: Keep: Generic Error Sieve script: if header :comparator "i;ascii-casemap" :contains "X-Spam-Level" "*****" { fileinto "INBOX/Junk"; stop; } I tried "INBOX.Junk", "Junk", etc., but no success. Namespace section in dovecot.conf: namespace private { separator = / prefix = INBOX/ inbox = yes #hidden = yes #list=yes } Same configuration works with cmusieve and Dovecot 1.1.16. I assume some problems with the namespace separator. Anyone got any ideas or got sieve running with namespace separator "/" and prefix "INBOX"? Regards, Sebastian

2 4

[Dovecot] Bug in MEMORY INDEX ? [v 1.1.16]
by Joel A. Chornik 08 Jul '09

08 Jul '09

When setting a CONTROL directory that does not exists (has to be created by dovecot upon first login) AND INDEX=MEMORY Dovecot POP3 will fail upon first login with 'Couldn't init INBOX: Internal error occurred. Refer to server log for more information.' Temporary fix seems to work fine patching dovecot-1.1.16/src/lib-storage/index/maildir/maildir-util.c ---- 98a99 > unsigned int k = 0; 105c106,109 < dirs[i++] = mail_storage_get_mailbox_index_dir(box->storage, --- > > /* Only create index folders if index is not MEMORY */ > if(!mail_index_is_in_memory(mbox->ibox.index)) > dirs[i++] = mail_storage_get_mailbox_index_dir(box->storage, 107c111,113 < i_assert(i == N_ELEMENTS(dirs)); --- > else k++; > > i_assert(i == N_ELEMENTS(dirs)-k); ----- Tested only on dovecot 1.1.16 Regards, Joel A. Chornik ELSERVER.COM

2 1

[Dovecot] readdir and rename don't mix; patch included
by Mike Abbott 08 Jul '09

08 Jul '09

Enclosed please find a patch to dovecot-1.1.16 that fixes a common problem we see under load on Mac OS X and HFS+. The symptom is that sometimes readdir() returns EINVAL. The problem is that readdir() and rename() don't mix well, and maildir_scan_dir() renames messages from new/ to cur/. The solution I chose is just to retry scanning the directory, since code to rescan is already present. An alternative solution would be to read the entire directory first, then process the contents; that is, to do all the readdirs before any renames. The patch is like Apple's previous ones, all tagged with /*APPLE*/ and formatted to simplify our merges. Please reformat it as you desire. It should also apply easily to 1.2.0. --- dovecot-1.1.16/src/lib-storage/index/maildir/maildir-sync.c 2009-05-20 16:42:49.000000000 -0500 +++ dovecot/src/lib-storage/index/maildir/maildir-sync.c 2009-07-07 19:50:37.000000000 -0500 @@ -354,7 +354,8 @@ return -1; } -static int maildir_scan_dir(struct maildir_sync_context *ctx, bool new_dir) +static int maildir_scan_dir(struct maildir_sync_context *ctx, bool new_dir, + bool final) /* APPLE */ { struct mail_storage *storage = &ctx->mbox->storage->storage; const char *path; @@ -486,6 +487,11 @@ } } + /* APPLE - rename can cause readdir to fail with EINVAL; force + quiet rescan unless this is the final such rescan already */ + if (errno == EINVAL && move_count && !final) + move_count = MAILDIR_RENAME_RESCAN_COUNT + 1; + else /* APPLE reduce code deltas */ if (errno != 0) { mail_storage_set_critical(storage, "readdir(%s) failed: %m", path); @@ -772,7 +778,8 @@ that new/ dir is checked as well. it's a good idea anyway. */ unsigned int count = 0; - while ((ret = maildir_scan_dir(ctx, TRUE)) > 0) { + while ((ret = maildir_scan_dir(ctx, TRUE, + count == MAILDIR_SCAN_DIR_MAX_COUNT)) > 0) { /* APPLE */ /* rename()d at least some files, which might have caused some other files to be missed. check again (see MAILDIR_RENAME_RESCAN_COUNT). */ @@ -783,7 +790,8 @@ return -1; if (cur_changed) { - if (maildir_scan_dir(ctx, FALSE) < 0) + if (maildir_scan_dir(ctx, FALSE, + FALSE) < 0) /* APPLE */ return -1; }

2 1

[Dovecot] GSSAPI cross-realm fixed
by Bryan Jacobs 08 Jul '09

08 Jul '09

Attached is a patch which in my environment (Linux/Heimdal 1.2.1) fixes cross-realm GSSAPI authentication. Changes it makes: 1. When using krb5_kuserok, do not call gss_compare_name to check that authn_name and authz_name are the same. Instead, make TWO calls to krb5_kuserok, one for each ID. If both IDs are acceptable, allow the login. 2. Disable checking that the name is a GSS_KRB5_PRINCIPAL_NAME, as this doesn't appear to be always the case for the authz_name. If I create a .k5login listing both username@REALM1 and username@REALM2, and make that file follow the appropriate security restrictions (world read, user only write permissions), this lets me use GSSAPI logins with principals from either REALM1 or REALM2. This leaves untouched the behavior in the case where krb5_kuserok is not available. Bryan Jacobs

2 3

[Dovecot] v1.2.0 released
by Timo Sirainen 08 Jul '09

08 Jul '09

http://dovecot.org/releases/1.2/dovecot-1.2.0.tar.gz http://dovecot.org/releases/1.2/dovecot-1.2.0.tar.gz.sig As promised. No changes since v1.2.rc8 (except packaged in OS X, since CentOS 5 autotools didn't add support for configure --docdir). Below are the largest changes since v1.1: * When creating files or directories to mailboxes, Dovecot now uses the mailbox directory's permissions and GID for them. Previous versions simply used 0600 mode always. For backwards compatibility dovecot-shared file's permissions still override these with Maildir. * SQL dictionary (quota) configuration file is different than in v1.1. See doc/dovecot-dict-sql-example.conf for the new format. * deliver -m: Mailbox name is now assumed to be in UTF-8 format, not modified-UTF7. Stephan Bosch's new Sieve implementation also assumes UTF-8 format in fileinto parameters. + Full support for shared mailboxes and IMAP ACL extension. The code is mainly from Sascha Wilde and Bernhard Herzog. + IMAP: Added support for extensions: CONDSTORE, QRESYNC, ESEARCH, ESORT, SEARCHRES, WITHIN, ID and CONTEXT=SEARCH. + SEARCH supports INTHREAD search key, but the rest of the INTHREAD draft isn't implemented yet so it's not advertised in capability. + THREAD REFS algorithm where threads are sorted by their latest message instead of the thread root message. There is also no base subject merging. + IMAP: Implemented imap-response-codes draft. + Thread indexes for optimizing IMAP THREAD command and INTHREAD search key. + Added userdb checkpassword (by Sascha Wilde) + Virtual mailboxes: http://wiki.dovecot.org/Plugins/Virtual + Autocreate plugin: http://wiki.dovecot.org/Plugins/Autocreate + Listescape plugin: http://wiki.dovecot.org/Plugins/Listescape

10 29

[Dovecot] Lost email where did it go...?
by Romer Ventura 08 Jul '09

08 Jul '09

Hello, I have this user that lost an email, even when the server is set up to do lazy expunge. The mail server logs show that the message was received, the user also read it and he did reply to it. here is the mail.log Jun 29 07:52:48 mx20 postfix/smtpd[19289]: 0ACD81C443B: client=usxsl050.slb.atosorigin-asp.com[199.6.139.15] Jun 29 07:52:48 mx20 postfix/cleanup[20259]: 0ACD81C443B: message- id=<791A7985471643BA9997A472B6E429AD(a)eur.slb.com> Jun 29 07:52:50 mx20 postfix/qmgr[24622]: 0ACD81C443B: from=<remoteusers(a)stonehouse.oilfield.slb.com>, size=405287, nrcpt=1 (queue active) Jun 29 07:52:50 mx20 postfix/smtpd[19289]: disconnect from usxsl050.slb.atosorigin-asp.com[199.6.139.15] Jun 29 07:52:50 mx20 postfix/smtpd[20261]: connect from localhost [127.0.0.1] Jun 29 07:52:50 mx20 postfix/smtpd[20261]: EB9C11C4443: client=localhost[127.0.0.1] Jun 29 07:52:50 mx20 postfix/cleanup[20259]: EB9C11C4443: message- id=<791A7985471643BA9997A472B6E429AD(a)eur.slb.com> Jun 29 07:52:50 mx20 postfix/qmgr[24622]: EB9C11C4443: from=< remoteuser(a)stonehouse.oilfield.slb.com>, size=406106, nrcpt=1 (queue active) Jun 29 07:52:50 mx20 postfix/smtp[20260]: 0ACD81C443B: to=<localuser(a)mydomain.com>, relay=127.0.0.1[127.0.0.1]:10025, delay=3.2, delays=2.5/0.01/0/0.7, dsn=2.0.0, status=sent (250 OK, sent 4A48B922_20407_5563_1 EB9C11C4443) Jun 29 07:52:50 mx20 postfix/qmgr[24622]: 0ACD81C443B: removed Jun 29 07:52:50 mx20 postfix/smtpd[20261]: disconnect from localhost [127.0.0.1] Jun 29 07:52:51 mx20 deliver(localuser(a)mydomain.com): msgid=<791A7985471643BA9997A472B6E429AD(a)eur.slb.com>: saved mail to INBOX But i cant find it anywhere... Is there any way of finding where it went...? I've done: grep -r 'sender@senderemail' <user>/Maildir/ And it only shows the email my user sent, not the original... any ideas of what happen as well...? Thanks.

3 4

[Dovecot] rquota RPC with netapp filer
by Ralf Becker 07 Jul '09

07 Jul '09

Hello list, with dovecot 1.2 on AIX rquota RPC calls fail with quota-fs: remote ext rquota call failed: RPC: 1832-012 Program/version mismatch when using a netapp filer. This is what netapp says: Bug ID 97288 Title Support requested for rquota version 2 Bug Severity 5 - Suggestion Bug Status Closed Product Data ONTAP Bug Type NFS Description Linux has implemented its own extensions to the rquota protocol as a version 2 of that protocol. ONTAP only supports v1 as that is the only version defined by Sun, the owner of that RPC program number. Workaround - Fixed-In This bug is not scheduled to be fixed, you may opt to Version open a technical support case if you would like to contact Network Appliance regarding the status of this bug. A complete list of releases where this bug is fixed is available here. So this should be no AIX specific problem. Using the attached rquota.x fixes the problem. In addition you have to disable group quota checking by adding "user" to your "quota" definition. Otherwise you'll see: quota-fs: rquota not compiled with group support Regards, Ralf -- ______________________________________________________________________ Dipl.-Inform. (FH) Ralf Becker Rechenzentrum (r/ft) der FH Trier (Network|Mail|Web|Firewall) University of applied sciences Administrator Schneidershof, D-54293 Trier Mail: beckerr(a)fh-trier.de Fon: +49 651 8103 499 Web: http://www.fh-trier.de/~beckerr Fax: +49 651 8103 214 PubKey: http://www.fh-trier.de/~beckerr Crypto: GnuPG, S/MIME ______________________________________________________________________ Wenn Gott gewollt haette, dass E-Mail in HTML geschrieben wuerden, endeten Gebete traditionell mit </amen>. (Tom Listen) /* * COMPONENT_NAME: onchdrs * * FUNCTIONS: none * * ORIGINS: 24,27 * * * (C) COPYRIGHT International Business Machines Corp. 1988,1993 * All Rights Reserved * Licensed Materials - Property of IBM * US Government Users Restricted Rights - Use, duplication or * disclosure restricted by GSA ADP Schedule Contract with IBM Corp. */ /* * Copyright (c) 1988 by Sun Microsystems, Inc. * (#) from SUN 1.4 */ /* * Remote quota protocol * Requires unix authentication */ const RQ_PATHLEN = 1024; struct getquota_args { string gqa_pathp<RQ_PATHLEN>; /* path to filesystem of interest */ int gqa_uid; /* inquire about quota for uid */ }; /* * remote quota structure */ struct rquota { int rq_bsize; /* block size for block counts */ bool rq_active; /* indicates whether quota is active */ unsigned int rq_bhardlimit; /* absolute limit on disk blks alloc */ unsigned int rq_bsoftlimit; /* preferred limit on disk blks */ unsigned int rq_curblocks; /* current block count */ unsigned int rq_fhardlimit; /* absolute limit on allocated files */ unsigned int rq_fsoftlimit; /* preferred file limit */ unsigned int rq_curfiles; /* current # allocated files */ unsigned int rq_btimeleft; /* time left for excessive disk use */ unsigned int rq_ftimeleft; /* time left for excessive files */ }; enum gqr_status { Q_OK = 1, /* quota returned */ Q_NOQUOTA = 2, /* noquota for uid */ Q_EPERM = 3 /* no permission to access quota */ }; union getquota_rslt switch (gqr_status status) { case Q_OK: rquota gqr_rquota; /* valid if status == Q_OK */ case Q_NOQUOTA: void; case Q_EPERM: void; }; program RQUOTAPROG { version RQUOTAVERS { /* * Get all quotas */ getquota_rslt RQUOTAPROC_GETQUOTA(getquota_args) = 1; /* * Get active quotas only */ getquota_rslt RQUOTAPROC_GETACTIVEQUOTA(getquota_args) = 2; } = 1; } = 100011;

2 2

[Dovecot] [PATCH] Fix VPATH build of RQUOTA support
by Matthias Andree 07 Jul '09

07 Jul '09

# HG changeset patch # User Matthias Andree <matthias.andree(a)gmx.de> # Date 1246993296 -7200 # Branch HEAD # Node ID c884491cb1a61ef50e227fcf4bb77d2e449c1e3a # Parent dff7312629a72f6ee90ab470ad0d50dfa68e6f71 Fix VPATH build of RQUOTA support. Some rpcgen derive #include "..." paths from the infile argument. This will be off for VPATH builds, as the generated rquota_xdr.c code will look in $(srcdir), but we'll generate the rquota.h file in $(builddir). Play safe and copy rquota.x to $(builddir) first. This fixes the build on openSUSE 11.1. diff -r dff7312629a7 -r c884491cb1a6 src/plugins/quota/Makefile.am --- a/src/plugins/quota/Makefile.am Tue Jul 07 15:00:52 2009 -0400 +++ b/src/plugins/quota/Makefile.am Tue Jul 07 21:01:36 2009 +0200 @@ -37,9 +37,10 @@ #RQUOTA_X = /usr/include/rpcsvc/rquota.x RQUOTA_X = $(srcdir)/rquota.x rquota_xdr.c: Makefile $(RQUOTA_X) + if "$(srcdir)" != "$(builddir)" ; then cp $(RQUOTA_X) $(builddir) ; fi (echo '#include "lib.h"'; \ echo '#include <rpc/rpc.h>'; \ - $(RPCGEN) -c $(RQUOTA_X) | \ + $(RPCGEN) -c $(builddir)/rquota.x | \ sed -e 's/IXDR_PUT/(void)IXDR_PUT/g' \ -e 's,/usr/include/rpcsvc/rquota.h,rquota.h,' \ -e 's/int32_t \*buf/int32_t *buf ATTR_UNUSED/' \

2 2