[Dovecot] bug report, from apple.com
Bruce includes detailed information including the imap session. Bruce works at Apple, on the Apple Mail.app product.
---------- Forwarded message ---------- From: Bruce Arthur barthur@apple.com To: dovecot@dovecot.org Cc: Bruce Arthur barthur@apple.com Date: Tue, 29 Aug 2006 09:04:35 -0700 Subject: Problem using Apple Mail with Dovecot
Server Configuration: dovecot-1.0.rc6 Built with: ./configure --prefix=/usr/local OS: FreeBSD 4.10-STABLE i386 running on ufs
Client configuration Mac OS X 10.4.7, using Apple Mail client.
Issue: Apple Mail is able to connect to the server, but it is not seeing the available folder correctly.
I did a little debugging and it looks like dovecot is failing to provide the separator in response to the LIST command. I did not spend anytime trying to figure out how other mail clients are getting this (my tcpdump is a little rusty these days). Here is what I see Mail sending to the server, and it's responses.
1 LOGIN applecot ******** 1 OK Logged in. 2 CAPABILITY
- CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS 2 OK Capability completed. 3 LIST "" "" 3 OK List completed.
The "LIST" command should be sending back the mailbox separator. The IMAP spec seems pretty clear about this (but reading RFC's is always a bit of a challenge).
From the RFC: http://www.ietf.org/rfc/rfc2060.txt 6.3..8. LIST Command
Arguments: reference name
mailbox name with possible wildcards
Responses: untagged responses: LIST
Result: OK - list completed
NO - list failure: can't list that reference or name
BAD - command unknown or arguments invalid
An empty ("" string) reference name argument indicates that the
mailbox name is interpreted as by SELECT. The returned mailbox
names MUST match the supplied mailbox name pattern. A non-empty
reference name argument is the name of a mailbox or a level of
mailbox hierarchy, and indicates a context in which the mailbox
name is interpreted in an implementation-defined manner.
An empty ("" string) mailbox name argument is a special request to
return the hierarchy delimiter and the root name of the name given
in the reference. The value returned as the root MAY be null if
the reference is non-rooted or is null. In all cases, the
hierarchy delimiter is returned. This permits a client to get the
hierarchy delimiter even when no mailboxes by that name currently
exist.
Example: C: A101 LIST "" ""
S: * LIST (\Noselect) "/" ""
S: A101 OK LIST Completed
C: A102 LIST #news.comp.mail.misc ""
S: * LIST (\Noselect) "." #news.
S: A102 OK LIST Completed
C: A103 LIST /usr/staff/jones ""
S: * LIST (\Noselect) "/" /
S: A103 OK LIST Completed
C: A202 LIST ~/Mail/ %
S: * LIST (\Noselect) "/" ~/Mail/foo
S: * LIST () "/" ~/Mail/meetings
S: A202 OK LIST completed-- Bruce
From: Jason Fesler jfesler@gigo.com To: Bruce Arthur barthur@apple.com Cc: dovecot@dovecot.org Date: Tue, 29 Aug 2006 10:41:55 -0700 (PDT) Subject: Re: Problem using Apple Mail with Dovecot
Server Configuration: dovecot-1.0.rc6 Built with: ./configure --prefix=/usr/local OS: FreeBSD 4.10-STABLE i386 running on ufs
Thanks, Bruce. Timo et all; Bruce specifically has been looking at my instance of dovecot after a friend of his reported problems with Mail.app and my server.
More information: Stock dovecot namespaces work;
This name space does not:namespace private { separator = . prefix = INBOX. inbox = yes hidden = no } namespace private { separator = / inbox = no hidden = yes prefix = "#mail/" location = mbox:/home/%u/mail }
On 6.9.2006, at 19.01, Jason Fesler wrote:
3 LIST "" "" 3 OK List completed. .. namespace private { separator = . prefix = INBOX. inbox = yes hidden = no } namespace private { separator = / inbox = no hidden = yes prefix = "#mail/" location = mbox:/home/%u/mail }
You could work around this by creating yet another hidden namespace
with empty prefix. But yea, I guess it's still a bug in Dovecot.
Although I'm not exactly sure what separator it should return in that
case. Probably either the first non-hidden namespace's, or the
inbox=yes namespace's..
BTW. for everyone reading this: I'm way too busy right now to even
read this list. Just moved to another city, school started and I'm
late in work related stuff.. Hopefully things will quiet down in a
few weeks :)
You could work around this by creating yet another hidden namespace with empty prefix. But yea, I guess it's still a bug in Dovecot. Although I'm not exactly sure what separator it should return in that case. Probably either the first non-hidden namespace's, or the inbox=yes namespace's..
Bruce believed the problem was not so much the prefix but the seperator. In any case, I'd recommend advertising the inbox=yes namespace. I'll be glad to test any code changes and report back to Bruce Arther barther@apple.com.
My workaround was to set up a second dovecot instance, "stock", on a different IP, with a different (signed, ug) certificate. In that configuration, the assumptions Apple Mail.app makes, work.
On Wed, 2006-09-06 at 16:56 -0700, Jason Fesler wrote:
You could work around this by creating yet another hidden namespace with empty prefix. But yea, I guess it's still a bug in Dovecot. Although I'm not exactly sure what separator it should return in that case. Probably either the first non-hidden namespace's, or the inbox=yes namespace's..
Bruce believed the problem was not so much the prefix but the seperator.
But the problem is that it asks for separator for the "" namespace, which doesn't exist. Dovecot does return the separator as long as it can find a namespace for it.
In any case, I'd recommend advertising the inbox=yes namespace.
Some people however set inbox=yes namespace to be /var/mail/user mbox, while everything else to be ~/Maildir with '.' separator. If those users also set ~/Maildir namespace to "INBOX.", that would break them..
My workaround was to set up a second dovecot instance, "stock", on a different IP, with a different (signed, ug) certificate. In that configuration, the assumptions Apple Mail.app makes, work.
I don't really understand how this changes anything :)
But the problem is that it asks for separator for the "" namespace, which doesn't exist. Dovecot does return the separator as long as it can find a namespace for it.
The RFC specifies that as a possibility. From what I'm reading, 'LIST "" ""' should return the root and the seperator both. (rfc2060, 6.3..8.)
Maybe we should instead put the burden on the operator to define the answer to both questions, instead of guessing.
(hours later).. Actually, what may instead be prefered, is to advise in the sample conf they need at least one name space with an empty prefix setting; and if their default name space has a non-empty prefix, to create an empty prefix hidden name space that has the same mailbox location.
On Wed, 2006-09-06 at 16:56 -0700, Jason Fesler wrote:
You could work around this by creating yet another hidden namespace with empty prefix. But yea, I guess it's still a bug in Dovecot. Although I'm not exactly sure what separator it should return in that case. Probably either the first non-hidden namespace's, or the inbox=yes namespace's..
Bruce believed the problem was not so much the prefix but the seperator. In any case, I'd recommend advertising the inbox=yes namespace. I'll be glad to test any code changes and report back to Bruce Arther barther@apple.com.
My workaround was to set up a second dovecot instance, "stock", on a different IP, with a different (signed, ug) certificate. In that configuration, the assumptions Apple Mail.app makes, work.
I tried setting up the second namespace, but I ended up getting Mail.app working, and duplicate folders everywhere else.
My solution was like yours, except I just fired courier back up on another port. :(
Rick
Hi all,
How can we reconstruct a corrupted mailbox with dovecot ?
On Thu, 2006-09-07 at 17:28 +0000, Fatou Ndiaye wrote:
Hi all,
How can we reconstruct a corrupted mailbox with dovecot ?
What is a "corrupted mailbox"? There's no such thing really with mbox or maildir. Except if mbox doesn't begin with a valid From-line, then it's treated as not a valid mbox..
Dovecot's index files might get corrupted, but normally they're fixed automatically. If not, just delete them and they get rebuilt.
We're talkink about "corrupted Mailbox" because when we try to acceed via webmail we have this message: ERREUR : Impossible d'achever la requête. Requête : SELECT "INBOX" Raison invoquée : Internal error occurred. Refer to server log for more information We had the same problem when we were using cyrus but after doing .../cyrus-imapd/reconstruct it works well.
-----Message d'origine----- De : dovecot-bounces@dovecot.org [mailto:dovecot-bounces@dovecot.org] De la part de Timo Sirainen Envoyé : jeudi 7 septembre 2006 17:45 À : Dovecot Mailing List Objet : Re: [Dovecot] Reconstruct a corrupted mailbox
On Thu, 2006-09-07 at 17:28 +0000, Fatou Ndiaye wrote:
Hi all,
How can we reconstruct a corrupted mailbox with dovecot ?
What is a "corrupted mailbox"? There's no such thing really with mbox or maildir. Except if mbox doesn't begin with a valid From-line, then it's treated as not a valid mbox..
Dovecot's index files might get corrupted, but normally they're fixed automatically. If not, just delete them and they get rebuilt.
Fatou Ndiaye schrieb:
We're talkink about "corrupted Mailbox" because when we try to acceed via webmail we have this message:
If this happens only when using some webmail client and access to the inbox via an imap mail client such as thunderbird works, the problem is probably not a corrupted inbox. What webmail app are you using ? What storage format do you use in dovecot?
Yours, Jakob Curdes
On Thu, 2006-09-07 at 18:16 +0000, Fatou Ndiaye wrote:
We're talkink about "corrupted Mailbox" because when we try to acceed via webmail we have this message: ERREUR : Impossible d'achever la requête. Requête : SELECT "INBOX" Raison invoquée : Internal error occurred. Refer to server log for more information
Show me what the server log says. Easier to solve the problem when you know the exact error message..
Here is the message in the server log:
IMAP(publi.com): Unexpected transaction log desync with index /var/mail/p/publi.com/Maildir/dovecot.index
-----Message d'origine----- De : dovecot-bounces@dovecot.org [mailto:dovecot-bounces@dovecot.org] De la part de Timo Sirainen Envoyé : jeudi 7 septembre 2006 18:42 À : Dovecot Mailing List Objet : Re: [Dovecot] Reconstruct a corrupted mailbox
On Thu, 2006-09-07 at 18:16 +0000, Fatou Ndiaye wrote:
We're talkink about "corrupted Mailbox" because when we try to acceed via webmail we have this message: ERREUR : Impossible d'achever la requête. Requête : SELECT "INBOX" Raison invoquée : Internal error occurred. Refer to server log for more information
Show me what the server log says. Easier to solve the problem when you know the exact error message..
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, 8 Sep 2006, Fatou Ndiaye wrote:
Here is the message in the server log:
IMAP(publi.com): Unexpected transaction log desync with index /var/mail/p/publi.com/Maildir/dovecot.index
Easy workaround: remove that file.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux)
iQEVAwUBRQEqoy9SORjhbDpvAQKluAf8CHSWE3QEiKeqZNOY2yB2wHiEzTbX7eRN +3G3ea8SEx5GlpjEJ0lcTdaF44owTOMQrGMn7eH8EEqlpWhnXEhR9ZGoSsIPtQP1 yHe7TPaJ1dj9UfELk4krTQz+mlyjluPKb/znuEt/jxAEKJujSaAWBkmu8jLddz0e K8oy1RISSPGs4+4erzmxDFamN2kzohb6lILdDBPQwEVQha7KT/uYc9onZf06ESCs 4hVwa2/LURwdS/SSABZVv6ZZvNql42wT6/X+lL+evGInQOb3Qw3n74b4f+VkCFe6 HcUvNeBnK8Ww3PpWtrfUaZmcux+qsS/v3I8NCrFoXa7C+v1o3uZ+cw== =dFmt -----END PGP SIGNATURE-----
Thanks all! It's Ok now.
-----Message d'origine----- De : dovecot-bounces@dovecot.org [mailto:dovecot-bounces@dovecot.org] De la part de Steffen Kaiser Envoyé : vendredi 8 septembre 2006 08:33 À : dovecot@dovecot.org Objet : Re: [Dovecot] Reconstruct a corrupted mailbox
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, 8 Sep 2006, Fatou Ndiaye wrote:
Here is the message in the server log:
IMAP(publi.com): Unexpected transaction log desync with index /var/mail/p/publi.com/Maildir/dovecot.index
Easy workaround: remove that file.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux)
iQEVAwUBRQEqoy9SORjhbDpvAQKluAf8CHSWE3QEiKeqZNOY2yB2wHiEzTbX7eRN +3G3ea8SEx5GlpjEJ0lcTdaF44owTOMQrGMn7eH8EEqlpWhnXEhR9ZGoSsIPtQP1 yHe7TPaJ1dj9UfELk4krTQz+mlyjluPKb/znuEt/jxAEKJujSaAWBkmu8jLddz0e K8oy1RISSPGs4+4erzmxDFamN2kzohb6lILdDBPQwEVQha7KT/uYc9onZf06ESCs 4hVwa2/LURwdS/SSABZVv6ZZvNql42wT6/X+lL+evGInQOb3Qw3n74b4f+VkCFe6 HcUvNeBnK8Ww3PpWtrfUaZmcux+qsS/v3I8NCrFoXa7C+v1o3uZ+cw== =dFmt -----END PGP SIGNATURE-----
Fatou Ndiaye schrieb:
Here is the message in the server log:
IMAP(publi.com): Unexpected transaction log desync with index /var/mail/p/publi.com/Maildir/dovecot.index
To me that sounds like some other application is accessing and changing the mailbox bypassing the imap server.
JC
I had something like this happen and it was because I was using spamassassin and spamfilter.sh file was moving files around and setting the file permission incorrectly...
Webmail (apache user could not access the mail and it gave an error similar to that)
But check the mail log to be sure.
Cha Yang
-------- Original Message -------- Subject: Re:[Dovecot] Reconstruct a corrupted mailbox From: Fatou Ndiaye fatou.ndiaye@sentoo.sn To: 'Dovecot Mailing List' dovecot@dovecot.org Date: 9/7/2006 11:16 AM
We're talkink about "corrupted Mailbox" because when we try to acceed via webmail we have this message: ERREUR : Impossible d'achever la requête. Requête : SELECT "INBOX" Raison invoquée : Internal error occurred. Refer to server log for more information We had the same problem when we were using cyrus but after doing .../cyrus-imapd/reconstruct it works well.
-----Message d'origine----- De : dovecot-bounces@dovecot.org [mailto:dovecot-bounces@dovecot.org] De la part de Timo Sirainen Envoyé : jeudi 7 septembre 2006 17:45 À : Dovecot Mailing List Objet : Re: [Dovecot] Reconstruct a corrupted mailbox
On Thu, 2006-09-07 at 17:28 +0000, Fatou Ndiaye wrote:
Hi all,
How can we reconstruct a corrupted mailbox with dovecot ?
What is a "corrupted mailbox"? There's no such thing really with mbox or maildir. Except if mbox doesn't begin with a valid From-line, then it's treated as not a valid mbox..
Dovecot's index files might get corrupted, but normally they're fixed automatically. If not, just delete them and they get rebuilt.
Hi there Fatou
We're talkink about "corrupted Mailbox" because when we try to acceed via webmail we have this message: ERREUR : Impossible d'achever la requête. Requête : SELECT "INBOX" Raison invoquée : Internal error occurred. Refer to server
I've had that error. It was simply a corrupted mail file for that user in his /var/spool/mail/$user mailbox. I deleted the top two (corrupted) lines of mail down to the first From: line and it resolved the problem.
Regards Chris
You could work around this by creating yet another hidden namespace with empty prefix. But yea, I guess it's still a bug in Dovecot. Although I'm not
FYI, I tried this. apple's mail works, and thunderbird works, but.. squirrelmail for some reason sees this hidden name space, and starts showing users two sets of heirarchies of the same folders (one as INBOX.*, which was prefered, plus again without a prefix).
exactly sure what separator it should return in that case. Probably either the first non-hidden namespace's, or the inbox=yes namespace's..
Maybe defer this problem to the administrator to define. What seperator to return, when list "" "" is asked for, and there is no namespace with an empty prefix defined. I suspect the number of people this affects is small - folks not using an empty prefix name space, ie idiots like me trying to seamlessly adapt the previous mail server's appearances when moving to dovecot.
I suspect the number of people this affects is small - folks not using an empty prefix name space, ie idiots like me trying to seamlessly adapt the previous mail server's appearances when moving to dovecot.
I too then am an idiot. Is the empty namespace the "preferred" way? As in,
# default namespace namespace private { separator = . inbox = yes hidden = no prefix = INBOX. }
should really "almost always and is really suggested" be:
# default namespace namespace private { separator = . inbox = yes hidden = no prefix = }
A while back I was able to get ACLs working so I started using INBOX. as the prefix. I thought it would "help" email clients by having all namespaces have a prefix.
I need to re-read this thread but is a blank namespace required? If it is then "inbox = yes" should enforce that? Maybe I am getting confused because I have been thinking about... I have an Outlook 2003 user who after I started using "INBOX." says he must use "INBOX." as the root prefix in his account preferences/options and this means he can't see any shared folders. No other email clients have this problem so I ignore him.
Jim
participants (9)
-
Cha Yang
-
Chris Kenward
-
Fatou Ndiaye
-
Jakob Curdes
-
Jason Fesler
-
Jim Horner
-
Rick Romero
-
Steffen Kaiser
-
Timo Sirainen