[Dovecot] Message display corrupted or missing in Netscape 7.0 and Mozilla 1.3
Hi,
a few days ago, we switched our site from pop3 to imap. We are running about 20 users with Netscape 7.0 on a FreeBSD-4.8 Server . I installed dovecot 0.9.99.1 via the freebsd-ports collection, which was as easy as make install and start dovecot. Our users mailboxes are still standart mbox format in /var/mail/(user), as it is default in Freebsd.
Now we suffer from a realy ugly problem, which was arising after the switch from pop3. When Netscape is newly started, it works fine as expected until Netscape displays messages corrupted or stops displaying them completly. I also cannot forward them in that state or open or delete them in any other way. The only way to get it working again is to quit Netscape, exit quickstart and restart Netscape again.
It just hangs on a bunch of messages or all of them.
The user notices it like this: There are three windows displayed in netscape mail:
- imap folder view
- message list of the current folder and
- the currently displayed message.
The Folder view and Message List in Netscape is always fine, so the Subject and sender, time, etc is always displayed in the List. The problem occurs then on some messages when i select them for reading. The Message Display below the message list just stays empty (also opening the message in a new window doesn't help) or the "title bar in the Message Display" stays empty (no subject, sender, time etc) and the raw email starting with header, raw html email and encoded attachment is shown below, each line followed by another blank line, like this:
== start corrupted email X-UIDL: 8793f170f0430909
Return-Path: frank.xx@xx.de
Delivered-To: xx@mdn.de
Received: from mailrelay1.xx.de (mailrelay.xx.de [212.114.153.xx])
by xx.de (Postfix) with ESMTP id D31B7A88F
for <xx@mdn.de>; Wed, 18 Jun 2003 08:55:07 +0200 (CEST)Received: from ntmail1.xx.de (ntmail1.xx.de [212.114.155.xx])
html and stuff following...
== end corrupted email
One could read it, but its realy ugly.
It seems, that if i delete a lot of messages in sequence (the morning spam), i can force to happen that problem earlier. It always hangs while deleting my spam.
I like dovecot a lot, it was easy to install etc and works pretty fine except that it still seems to be in beta-stage. Now i would switch to courier/uw-imap until dovecot is ready, but the mbox format dovecot is using seems to be pretty uniq? I think, the users could keep their /var/mail/Username-INBOX, but wouldn't they loose their email folders? Is there any way to switch the software in a easy way? Or will there be a near future release which will fix that? What about the cvs version?
Thanks for you help!
Best regards
Steven
----- Steven Enderle ----- m d n Huebner GmbH ---- ----- enderle@mdn.de ----- + 49 911 93 90 90 ----- ----- Digital Imaging & Documentmanagment -----
Hi Steven,
It is Netscape problem. Sad, but it is.
Looks like rendering process hangs while it tries to show SPAM. I have similar problem. Subject line with 8-bit characters is the worst i think. Sometimes rules aren't working at all, if message filter tries to process SPAM message.
Gasha
Steven Enderle wrote:
Hi,
Now we suffer from a realy ugly problem, which was arising after the switch from pop3. When Netscape is newly started, it works fine as expected until Netscape displays messages corrupted or stops displaying them completly. I also cannot forward them in that state or open or delete them in any other way. The only way to get it working again is to quit Netscape, exit quickstart and restart Netscape again.
Hi Gasha,
thanks for your prompt answer! But sadly, spam ain't our only problem (hell, i got 660 mails in my spam folder in the last 30 days!!) and it ain't the problem for message display:
It worked perfectly with popa3d! We are using that system for years now. The message display problem is limited to the yet switched workstations, all i did is configuring netscape as i have written down to do it here at our site. Some workstations are still running pop3, with no problems at all like they did always. The difference from configuring pop<->imap is following:
== Skip all pop stuff, select the same server for imap.
Messages are checked at startup and then every minute, empty trash on exit and "only show subscribed folders" is disabled! All other options are left as they are. The Netscape installation is absolutly identical over all workstations, even the signature.
I also still got my pop3 config running parallel (just in case), and the spam is displayed in excellent 16.x million colours and all details, with imap and also pop3. (Will i order the taiwanese soap or the mini fridge for my car?)
So the spam mail ain't the problem. I am sure, the problem must be somewhere between netscape, dovecot or the chair i am sitting in.
Steven
----- Steven Enderle ----- m d n Huebner GmbH ---- ----- enderle@mdn.de ----- + 49 911 93 90 90 ----- ----- Digital Imaging & Documentmanagment -----
Gasha wrote:
Hi Steven,
It is Netscape problem. Sad, but it is.
Looks like rendering process hangs while it tries to show SPAM. I have similar problem. Subject line with 8-bit characters is the worst i think. Sometimes rules aren't working at all, if message filter tries to process SPAM message.
Gasha
Steven Enderle wrote:
Hi,
Now we suffer from a realy ugly problem, which was arising after the switch from pop3. When Netscape is newly started, it works fine as expected until Netscape displays messages corrupted or stops displaying them completly. I also cannot forward them in that state or open or delete them in any other way. The only way to get it working again is to quit Netscape, exit quickstart and restart Netscape again.
On Thu, 2003-06-26 at 11:54, Steven Enderle wrote:
a few days ago, we switched our site from pop3 to imap. We are running about 20 users with Netscape 7.0 on a FreeBSD-4.8 Server .
Isn't Netscape 7's mail pretty much the same as Mozilla's mail?
Now we suffer from a realy ugly problem, which was arising after the switch from pop3. When Netscape is newly started, it works fine as expected until Netscape displays messages corrupted or stops displaying them completly. I also cannot forward them in that state or open or delete them in any other way. The only way to get it working again is to quit Netscape, exit quickstart and restart Netscape again.
Hmm. It might help if you showed what exactly Dovecot was sending to Netscape. See http://dovecot.procontrol.fi/bugreport.html#sniffing
If it's Dovecot's fault, I can think of only two reasons:
a) Dovecot doesn't detect that mbox was changed and doesn't synchronize it, and it starts reading messages from wrong location. I haven't noticed problems with this myself and I've been using Dovecot with mbox for almost a year now. Maybe some other program touches the mbox in a way Dovecot doesn't understand.
b) mbox support still doesn't behave fully as IMAP RFC requires. Message headers may change (not permitted) and so message size may not be exactly correct (not permitted either). I doubt Netscape really cares about these though.
Or maybe it's something completely different. I can't say really.
email starting with header, raw html email and encoded attachment is shown below, each line followed by another blank line, like this:
== start corrupted email X-UIDL: 8793f170f0430909
Return-Path: frank.xx@xx.de
Was there any other corruption than extra empty lines? Is there ever any other corruption?
I like dovecot a lot, it was easy to install etc and works pretty fine except that it still seems to be in beta-stage. Now i would switch to courier/uw-imap until dovecot is ready, but the mbox format dovecot is using seems to be pretty uniq? I think, the users could keep their /var/mail/Username-INBOX, but wouldn't they loose their email folders? Is there any way to switch the software in a easy way? Or will there be a near future release which will fix that? What about the cvs version?
You should be able to switch between UW-IMAP and Dovecot easily. Only difference is that Dovecot saves mailbox subscriptions to ~/mail/.subscriptions and UW-IMAP to ~/.mailboxlist. And also that Dovecot shows only mailboxes under ~/mail by default, UW-IMAP shows everything under ~/. The ".imap" directories that Dovecot creates can be deleted without any data loss.
Hello Timo!
yes, Mozilla and Netscape are almost the same. There is hardly any difference except logos and instant messaging and stuff...
I will sniff the communication as soon as i have time to install a sniffer.
If it's Dovecot's fault, I can think of only two reasons:
a) Dovecot doesn't detect that mbox was changed and doesn't synchronize it, and it starts reading messages from wrong location. I haven't noticed problems with this myself and I've been using Dovecot with mbox for almost a year now. Maybe some other program touches the mbox in a way Dovecot doesn't understand.
There are two other programms working with the mbox directly (postfix though procmail and popa3d - popa3d never at the same time as dovecot), but the corruption also happens in my subfolders, where only dovecot is working.
b) mbox support still doesn't behave fully as IMAP RFC requires. Message headers may change (not permitted) and so message size may not be exactly correct (not permitted either). I doubt Netscape really cares about these though.
I just had some heavy problems with email attachments. It saved just a few bytes of my attachments, sometimes 5KB or 20KB of 66KB, sometimes it saved a null file. i can't really say why, but that happened again and again, i had to open netscape and forward the mail to my second workstation without doing something else or i had to restart netscape completly (e.g. when opening the attachment for reading or saving it). What makes me wondering is, the other workstation is also using the same imap server.
Ok, that was another corruption i saw, but i guess its related to the corrupted email thing.
Steven
----- Steven Enderle ----- m d n Huebner GmbH ---- ----- enderle@mdn.de ----- + 49 911 93 90 90 ----- ----- Digital Imaging & Documentmanagment -----
Timo Sirainen wrote:
On Thu, 2003-06-26 at 11:54, Steven Enderle wrote:
a few days ago, we switched our site from pop3 to imap. We are running about 20 users with Netscape 7.0 on a FreeBSD-4.8 Server .
Isn't Netscape 7's mail pretty much the same as Mozilla's mail?
Now we suffer from a realy ugly problem, which was arising after the switch from pop3. When Netscape is newly started, it works fine as expected until Netscape displays messages corrupted or stops displaying them completly. I also cannot forward them in that state or open or delete them in any other way. The only way to get it working again is to quit Netscape, exit quickstart and restart Netscape again.
Hmm. It might help if you showed what exactly Dovecot was sending to Netscape. See http://dovecot.procontrol.fi/bugreport.html#sniffing
If it's Dovecot's fault, I can think of only two reasons:
a) Dovecot doesn't detect that mbox was changed and doesn't synchronize it, and it starts reading messages from wrong location. I haven't noticed problems with this myself and I've been using Dovecot with mbox for almost a year now. Maybe some other program touches the mbox in a way Dovecot doesn't understand.
b) mbox support still doesn't behave fully as IMAP RFC requires. Message headers may change (not permitted) and so message size may not be exactly correct (not permitted either). I doubt Netscape really cares about these though.
Or maybe it's something completely different. I can't say really.
email starting with header, raw html email and encoded attachment is shown below, each line followed by another blank line, like this:
== start corrupted email X-UIDL: 8793f170f0430909
Return-Path: frank.xx@xx.de
Was there any other corruption than extra empty lines? Is there ever any other corruption?
I like dovecot a lot, it was easy to install etc and works pretty fine except that it still seems to be in beta-stage. Now i would switch to courier/uw-imap until dovecot is ready, but the mbox format dovecot is using seems to be pretty uniq? I think, the users could keep their /var/mail/Username-INBOX, but wouldn't they loose their email folders? Is there any way to switch the software in a easy way? Or will there be a near future release which will fix that? What about the cvs version?
You should be able to switch between UW-IMAP and Dovecot easily. Only difference is that Dovecot saves mailbox subscriptions to ~/mail/.subscriptions and UW-IMAP to ~/.mailboxlist. And also that Dovecot shows only mailboxes under ~/mail by default, UW-IMAP shows everything under ~/. The ".imap" directories that Dovecot creates can be deleted without any data loss.
Hello Timo, Hello List,
I just sniffed the communication between Netscape and Dovecot and I catched the error! It seems like dovecot returns different sizes for the same mail:
This mail is displayed correct and the dovecot returns it like this:
- 268 FETCH (UID 542 RFC822.SIZE 2989 BODY[] {2989}..Return-Path: The corrupted one starts like this:
- 268 FETCH (UID 542 FLAGS (\Seen) RFC822.SIZE 2975 BODY[] {2975}.. continuing in next packet (There is data missing): eturn-Path: root@mdn.de.Delivered-To: enderle@mdn.de.Received: by
Also, sometimes there seems to be data from the imap protocol appended to the email, i think its the next byte after the email [one ")"]! I saw it often appended to someones signatures and was wondering if it belongs there. As i have seen the raw text going over the line now, i am sure it doesn't belong to the email.
Can i do something more to discribe that problem? Is it a bug in Netscape or Dovecot? I attached you the stuff i sniffed after the mail...
Regards
Steven
START CORRUPT MESSAGE
# ./ngrep -A tcp port 143 interface: xl0 (212.202.153.176/255.255.255.248) filter: ip and ( port 143 ) # T 10.0.0.122:49510 -> 10.0.2.179:143 [AP] 5 UID fetch 542 (UID RFC822.SIZE BODY[])..
# T 10.0.2.179:143 -> 10.0.0.122:49510 [AP]
- 268 FETCH (UID 542 FLAGS (\Seen) RFC822.SIZE 2975 BODY[] {2975}..
T 10.0.2.179:143 -> 10.0.0.122:49510 [A] eturn-Path: root@mdn.de.Delivered-To: enderle@mdn.de.Received: by mail.mdn.de (Postfix, fro m userid 0)..id 50682A898; Fri, 27 Jun 2003 03:04:18 +0200 (CEST).To: root@mdn.de.Subject: md n.de security check output.Message-Id: 20030627010418.50682A898@mail.mdn.de.Date: Fri, 27 J un 2003 03:04:18 +0200 (CEST).From: root@mdn.de (root).Content-Length: 2485.Status: O.X-UID: 542.X-Keywords:
..Checking setuid files and devices:......Checking for uids of 0:..root 0..toor 0......Checking for passwordless accounts:......mdn.de kernel log messages:..> Jun 2 6 11:04:06 mdn login: ROOT LOGIN (root) ON ttyv0..> Jun 26 13:33:27 mdn su: enderle to root o n /dev/ttyp0..> Jun 26 15:03:49 mdn login: ROOT LOGIN (root) ON ttyv0..> Jun 26 15:30:16 mdn su: BAD SU enderle to root on /dev/ttyp2..> Jun 26 15:30:18 mdn su: enderle to root on /dev/t typ2..> xl0: promiscuous mode enabled..> Jun 26 15:30:58 mdn /kernel: xl0: promiscuous mode e nabled..> xl0: promiscuous mode disabled..> Jun 26 15:31:00 mdn /kernel: xl0: promiscuous mod e disabled..> xl0: promiscuous mode enabled..> Jun 26 15:31:23 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mode disabled..> Jun 26 15:32:24 mdn /kernel: xl0: promiscuo us mode disabled..> xl0: promiscuous mode enabled..> Jun 26 15:33:23 mdn /kernel: xl0: promis cuous mode enabled..> xl0: promiscuous mode disabled.
# T 10.0.2.179:143 -> 10.0.0.122:49510 [A] .> Jun 26 15:33:39 mdn /kernel: xl0: promiscuous mode disabled..> xl0: promiscuous mode enabl ed..> Jun 26 15:33:42 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mode dis abled..> Jun 26 15:33:48 mdn /kernel: xl0: promiscuous mode disabled..> xl0: promiscuous mode enabled..> Jun 26 15:33:51 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mo de disabled..> Jun 26 15:33:52 mdn /kernel: xl0: promiscuous mode disabled..> xl0: promiscuou s mode enabled..> Jun 26 15:34:37 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscu ous mode disabled..> Jun 26 15:34:47 mdn /kernel: xl0: promiscuous mode disabled..> xl0: prom iscuous mode enabled..> Jun 26 15:35:03 mdn /kernel: xl0: promiscuous mode enabled..> xl0: pr omiscuous mode disabled..> Jun 26 15:35:07 mdn /kernel: xl0: promiscuous mode disabled..> xl0 : promiscuous mode enabled..> Jun 26 15:35:36 mdn /kernel: xl0: promiscuous mode enabled..> x l0: promiscuous mode disabled..> Jun 26 15:35:40 mdn /kernel: xl0: promiscuous mode disabled. .> xl0: promiscuous mode enabled..> Jun 26 15:35:41 mdn /kernel: xl0: promiscuous mode enable d..> xl0: promiscuous mode disabled..> Jun 26 15:42:08 mdn /kernel: xl0: promiscuous mode dis abled..> Jun 26 15:44:11 mdn su: enderle to root on /dev/ttyp3..> xl0: promiscuous mode enabl ed..> Jun 26 15:44:14 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mode dis abled..> Jun 26 15:45:37 mdn /kernel: xl0: promiscuou
### T 10.0.2.179:143 -> 10.0.0.122:49510 [AP] s mode disabled......mdn.de login failures:......mdn.de refused connections:.. ## T 10.0.2.179:143 -> 10.0.0.122:49510 [AP] )..5 OK Fetch completed...
## T 10.0.0.122:64953 -> 10.0.2.179:143 [AP] 1885 noop..
# T 10.0.0.122:64931 -> 10.0.2.179:143 [AP] 2013 noop..
# T 10.0.2.179:143 -> 10.0.0.122:64953 [AP] 1885 OK NOOP completed...
# T 10.0.2.179:143 -> 10.0.0.122:64931 [AP] 2013 OK NOOP completed...
# T 10.0.0.122:64953 -> 10.0.2.179:143 [AP] 1886 UID fetch 60:* (FLAGS)..
# T 10.0.0.122:64931 -> 10.0.2.179:143 [AP] 2014 UID fetch 725:* (FLAGS)..
# T 10.0.2.179:143 -> 10.0.0.122:64953 [AP]
- 29 FETCH (UID 59 FLAGS (\Recent))..1886 OK Fetch completed...
T 10.0.2.179:143 -> 10.0.0.122:64931 [AP]
- 121 FETCH (UID 724 FLAGS (\Recent))..2014 OK Fetch completed...
##^Cexit 152 received, 0 dropped END CORRUPT MESSAGE
START CORRECT MESSAGE
# ./ngrep -A tcp port 143 interface: xl0 (212.202.153.176/255.255.255.248) filter: ip and ( port 143 ) # T 10.0.0.122:49520 -> 10.0.2.179:143 [AP] 4 UID fetch 542 (UID RFC822.SIZE BODY[])..
## T 10.0.2.179:143 -> 10.0.0.122:49520 [A] promiscuous mode disabled..> xl0:
268 FETCH (UID 542 RFC822.SIZE 2989 BODY[] {2989}..Return-Path: root@mdn.de..Delivered-To : enderle@mdn.de..Received: by mail.mdn.de (Postfix, from userid 0)...id 50682A898; Fri, 27 J un 2003 03:04:18 +0200 (CEST)..To: root@mdn.de..Subject: mdn.de security check output..Messag e-Id: 20030627010418.50682A898@mail.mdn.de..Date: Fri, 27 Jun 2003 03:04:18 +0200 (CEST)..F rom: root@mdn.de (root)..Content-Length: 2485..Status: RO..X-UID: 542..X-Keywords:
....Checking setuid files and devices:......Checking for uids of 0:..root 0..toor 0......Checking for passwordless accounts:......mdn.de kernel log messages:..> Jun 26 11:04:06 mdn login: ROO T LOGIN (root) ON ttyv0..> Jun 26 13:33:27 mdn su: enderle to root on /dev/ttyp0..> Jun 26 15 :03:49 mdn login: ROOT LOGIN (root) ON ttyv0..> Jun 26 15:30:16 mdn su: BAD SU enderle to roo t on /dev/ttyp2..> Jun 26 15:30:18 mdn su: enderle to root on /dev/ttyp2..> xl0: promiscuous mode enabled..> Jun 26 15:30:58 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuou s mode disabled..> Jun 26 15:31:00 mdn /kernel: xl0: promiscuous mode disabled..> xl0: promis cuous mode enabled..> Jun 26 15:31:23 mdn /kernel: xl0: promiscuous mode enabled..> xl0: prom iscuous mode disabled..> Jun 26 15:32:24 mdn /kernel: xl0:
promiscuous mode enabled..> Jun 26 15:33:23 mdn /kern
Jun 26 15:35:07 mdn /kernel: xl0: promiscuous mode disabled..> xl0:
# T 10.0.2.179:143 -> 10.0.0.122:49520 [A] el: xl0: promiscuous mode enabled..> xl0: promiscuous mode disabled..> Jun 26 15:33:39 mdn /k ernel: xl0: promiscuous mode disabled..> xl0: promiscuous mode enabled..> Jun 26 15:33:42 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mode disabled..> Jun 26 15:33:48 mdn /kernel: xl0: promiscuous mode disabled..> xl0: promiscuous mode enabled..> Jun 26 15:33: 51 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mode disabled..> Jun 26 15: 33:52 mdn /kernel: xl0: promiscuous mode disabled..> xl0: promiscuous mode enabled..> Jun 26 15:34:37 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mode disabled..> Jun 26 15:34:47 mdn /kernel: xl0: promiscuous mode disabled..> xl0: promiscuous mode enabled..> J un 26 15:35:03 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mode disabled.. promiscuous mode enable d..> Jun 26 15:35:36 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mode disa bled..> Jun 26 15:35:40 mdn /kernel: xl0: promiscuous mode disabled..> xl0: promiscuous mode enabled..> Jun 26 15:35:41 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promiscuous mod e disabled..> Jun 26 15:42:08 mdn /kernel: xl0: promiscuous mode disabled..> Jun 26 15:44:11 mdn su: enderle to root on /dev/ttyp3..> xl0: promiscuous mode enabled..> Jun 26 15:44:14 mdn /kernel: xl0: promiscuous mode enabled..> xl0: promi
# T 10.0.2.179:143 -> 10.0.0.122:49520 [AP] scuous mode disabled..> Jun 26 15:45:37 mdn /kernel: xl0: promiscuous mode disabled......mdn. de login failures:......mdn.de refused connections:..)..4 OK Fetch completed... ##^Cexit 49 received, 0 dropped END CORRECT MESSAGE
Steven Enderle wrote:
Hello Timo!
yes, Mozilla and Netscape are almost the same. There is hardly any difference except logos and instant messaging and stuff...
I will sniff the communication as soon as i have time to install a sniffer.
If it's Dovecot's fault, I can think of only two reasons:
a) Dovecot doesn't detect that mbox was changed and doesn't synchronize it, and it starts reading messages from wrong location. I haven't noticed problems with this myself and I've been using Dovecot with mbox for almost a year now. Maybe some other program touches the mbox in a way Dovecot doesn't understand.
There are two other programms working with the mbox directly (postfix though procmail and popa3d - popa3d never at the same time as dovecot), but the corruption also happens in my subfolders, where only dovecot is working.
b) mbox support still doesn't behave fully as IMAP RFC requires. Message headers may change (not permitted) and so message size may not be exactly correct (not permitted either). I doubt Netscape really cares about these though.
I just had some heavy problems with email attachments. It saved just a few bytes of my attachments, sometimes 5KB or 20KB of 66KB, sometimes it saved a null file. i can't really say why, but that happened again and again, i had to open netscape and forward the mail to my second workstation without doing something else or i had to restart netscape completly (e.g. when opening the attachment for reading or saving it). What makes me wondering is, the other workstation is also using the same imap server.
Ok, that was another corruption i saw, but i guess its related to the corrupted email thing.
Steven
--
----- Steven Enderle ----- m d n Huebner GmbH ---- ----- enderle@mdn.de ----- + 49 911 93 90 90 ----- ----- Digital Imaging & Documentmanagment -----
On Fri, 2003-06-27 at 08:53, Steven Enderle wrote:
- 268 FETCH (UID 542 FLAGS (\Seen) RFC822.SIZE 2975 BODY[] {2975}.. continuing in next packet (There is data missing): eturn-Path: root@mdn.de.Delivered-To: enderle@mdn.de.Received: by
If it's missing one character, that actually looks like that one CRLF-bug that I fixed after 0.99.9.1.. Except your mbox probably doesn't have any CRs? Maybe it's still the same, see if 0.99.10 helps.
participants (3)
-
Gasha
-
Steven Enderle
-
Timo Sirainen