[Dovecot] emails getting mangled when dragging from Exchange account to IMAP shared folders
I'm having the most frustrating issue, and I'm at a loss for what is happening. I'm not even sure it is with Dovecot, but that's why I'm posting this here... if it isn't a dovecot issue, maybe someone can get me headed in the right direction? I've posted relevant dovecot and config info at the bottom of this post.
Here's the scenario: We recently migrated our email from Postfix+Dovecot+Amavis+SA to Exchange (not my choice, unfortunately). Now, users have to use Outlook (as opposed to before, when they could choose between Outlook and Thunderbird), and have the new Exchange account set up, as well as their old IMAP account, which they still use for shared project folders. So, emails come in to Exchange, then they drag the emails from the Exchange mailbox to the shared branch under their IMAP account, and into the appropriate job folder.
Emails to one specific user in my organization (let's call him Roy@Ocean.org) is having his emails mangled when dragging certain items from his Exchange mailbox to the IMAP account. I can't reproduce the issue with any other user. Also, this only seems to happens with emails from one other specific domain (let's call them tornado.org). So, in summary, user1@Tornado.orgsend an email to Roy@Ocean.organd Sue@Ocean.org. Both drag the email from their Exchange box into any folder (shared or otherwise) in the IMAP tree. Sue's copy makes it in just fine, while Roy's copy gets mangled. Here's the headers of the resultant email after Roy has dragged it into the shared folder:
========================================
From: "User1" User1@tornado.org
To: "Roy",
"Sue"
References: B3457C1920D7A2438CCC19EF2A527293372B11@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293372D9A@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933A2C3D@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933A310E@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933A3683@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933A3B97@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933D103B@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933D192F@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933D1B12@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933FD341@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933FD8F8@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933FD9A0@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272933FDFDD@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272934C8927@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272934C8B46@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272934C8D76@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272934C8D7A@abc027.abc.local
And here's the email body:
========================================
D7A2438CCC19EF2A5272934C8D95@abc027.abc.local>
B3457C1920D7A2438CCC19EF2A5272934C8DB6@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272934C8F0D@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935008B5@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293500C27@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935011BC@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293525B1D@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293525D81@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293526139@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293526416@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935524A6@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935524C9@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293589E1C@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935BF6A9@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935E7414@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935E75E2@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935E75E8@abc027.abc.local
D7A2438CCC19EF2A5272935E75F4@abc027.abc.local>
B3457C1920D7A2438CCC19EF2A5272935E800E@abc027.abc.local
Subject: RE: Some Subject
Date: Tue, 26 Oct 2010 15:11:01 -0700
Message-ID: B3457C1920D7A2438CCC19EF2A52729364D0A0@abc027.abc.local
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_0118_01CB7755.52354760"
X-Mailer: Microsoft Office Outlook 12.0
Thread-Index:
AcrhmOPG84jptRwERxazrnj4Ne2YAQAACMtAAAAjcPAAAAu+gAAAE1hAAAAZWCAAABMaEAAADGzwACO3fdAACN0HsAEpVUrgAPt2jvAAAcrbcABrFxCQAAAXwjAAABGv8AAAI35QAYbDMHAAAPjDYAADNREgAA5XR4AAHoOFcAFtPWTAAIubCCAALwf/MAADBptQACzN6nAAAC61kAAAEN+AAACfkjAAO7MG4AAHLrMQALmKX6AAAUlCcAAAq9SgAAuoo+AAABqqMABXq+FwAD5jAAABYqZYoAOISoLgAAAd//AAAKVPYAApG8DwAAt6dsAAKxpd8AEy9g6gAADrZVAAAE4WgAAlXdkQAPY4tNAAAWY+kAFsS/RgAFtG7qAAAGJT8AAAPLUwAAA2xsAAACamEAAC4ufAADaZyfAAKizZ0AAAZ9EAAAWRHAAAAOgkgAAAUUOgAAQWBBAAij+IkAAE2k1wAAPgAgAANGrwsAACt+wgAC0EW9AAAEDpoAA12EJgAAAyAtAAAyATYAAgyfBQAKS/7/AAAEPWEAABNsTAAAAZujAALLkhkAAwwG5AAAMVt8AAAE3iwAEhzNqgAA8qbEAAIqDaMAA0UTmgAEE0hxAAlsz9IAABAdrAAFIwRcAAZxd5oAAOziPQAJEu45AAOaFlIAAeeCygADwtnbAD9zXHwAApeNOAACk/pfAAABxl0AAAWgGAAAB7d4AACGbyQACNOVmwABF68KAAKuKysAAsiR5gADDcY1AAOMAK8ACO561QABCk+7AAI5IqsAAJGVEgAFjaIkAANWSCQAACuaAwAAAKUQAABugScAAAbmTQAADspwAAzNO+MAAiFH8wAAEA4mAAMv6E0AEtz2wQADWsQHAAO4pvkACIRgpAAAApFcAAABO4UABmP4/QAAEJ/lABNOByMAAF28awAACwfDA=
Content-Language: en-us
X-OlkEid: C664F3259FA9393D4124CD409B5CAE330696EE82
content-class: urn:content-classes:message
This is a multi-part message in MIME format.
------=_NextPart_000_0118_01CB7755.52354760
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
<SNIP>
===========================================================================
Obviously, it's breaking the message in the References header, and I've
tracked down the following byte sequence at the breaking point:
0D:09:0A:09
...which is CR,TAB,CR,LF.
What puzzles me is why Roy's gets broken here, when the others' do not;
they're the exact same email, so you'd think they would all get broken. I
can't figure out A) where that byte sequence is coming from (our Exchange
server, Tornado.net's exchange server, or one of the two of our external
spam filtering service... they use BigFish.com, and we use MxLogic), or B)
if that's even the issue.
I've rebuilt Roy's entire machine, reconfigured Outlook from scratch, still
happening. When he logs onto another machine and sets up his Exchange
account on another outlook, it still happens. Our Exchange administrator
(I don't have access to Exchange) says there is nothing wrong with his
Exchange account, for what it's worth.
I don't expect anyone to have the solution here, since there are many
variables, and there is other info that would probably be needed. What I DO
want is to be pointed in the right direction on how I might track this down.
For example, what is the path that this mangled email would take in this
case? Even with full logging in Dovecot and postfix, I can't see any info
on the path that this email follows in our system, and where it might be
getting mangled. Does it actually go through the same path as an email that
was actually sent to the IMAP account, or is it different when it is dragged
in from another account in the same mail client? Is this more likely a
problem with Postfix, or Sieve, or....? (I've disabled Amavis and SA to get
them out of the picture, and same problem).
Here's the dovecot config info; please let me know if you want any other
config info, and thanks ahead of time!
# dovecot --version
1.1.19
# dovecot -n
# 1.1.19: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.28-hardened-r9 x86_64 Gentoo Base System release 1.12.13
protocols: imaps imap managesieve
listen(default): 127.0.0.1:143
listen(imap): 127.0.0.1:143
listen(managesieve): 127.0.0.1:2000
ssl_listen(default): *:993
ssl_listen(imap): *:993
ssl_listen(managesieve):
ssl_cert_file: /etc/ssl/dovecot/imapd.crt
ssl_key_file: /etc/ssl/dovecot/imapd.key
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(managesieve): /usr/libexec/dovecot/managesieve-login
login_greeting_capability(default): yes
login_greeting_capability(imap): yes
login_greeting_capability(managesieve): no
valid_chroot_dirs: /var/mail
first_valid_uid: 206
last_valid_uid: 206
first_valid_gid: 206
last_valid_gid: 206
maildir_copy_preserve_filename: yes
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(managesieve): /usr/libexec/dovecot/managesieve
mail_plugins(default): acl
mail_plugins(imap): acl
mail_plugins(managesieve):
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(managesieve): /usr/lib64/dovecot/managesieve
imap_client_workarounds(default): outlook-idle delay-newmail
imap_client_workarounds(imap): outlook-idle delay-newmail
imap_client_workarounds(managesieve):
sieve_storage(default):
sieve_storage(imap):
sieve_storage(managesieve): ~
sieve(default):
sieve(imap):
sieve(managesieve): ~/.dovecot.sieve
namespace:
type: private
separator: .
location: maildir:~/Maildir
inbox: yes
list: yes
subscriptions: yes
namespace:
type: public
separator: .
prefix: shared.
location: maildir:/var/mail/shared
list: yes
lda:
postmaster_address: postmaster@jensenmaritime.com
hostname: mail.jensenmaritime.com
mail_plugins: cmusieve
mail_plugin_dir: /usr/lib/dovecot/lda
sieve_global_path: /var/mail/.dovecot.sieve
sieve: ~/.dovecot.sieve
sendmail_path: /usr/lib/sendmail
auth_socket_path: /var/run/dovecot/deliver-auth
auth lda:
default_realm: jensenmaritime.com
user: postmaster
passdb:
driver: ldap
args: /etc/dovecot/lda-ldap.conf
userdb:
driver: static
args: allow_all_users=yes user=%Lu uid=206 gid=206 home=/var/mail/%Lu
socket:
type: listen
master:
path: /var/run/dovecot/deliver-auth
mode: 384
user: vmail
group: vmail
auth default:
mechanisms: PLAIN LOGIN
default_realm: jensenmaritime.com
user: postmaster
passdb:
driver: ldap
args: /etc/dovecot/default-ldap.conf
userdb:
driver: static
args: user=%Lu uid=206 gid=206 home=/var/mail/%Lu
socket:
type: listen
client:
path: /var/spool/postfix/private/auth
mode: 432
user: postfix
group: postfix
plugin:
acl: vfile
# grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-ldap.conf
uris = ldap://127.0.0.1:389/
ldap_version = 3
base = ou=users,ou=accounts,dc=jensenmaritime,dc=com
scope = onelevel
auth_bind = yes
pass_filter =
(&(objectClass=CourierMailAccount)(!(disableimap=*))(mail=%Lu))
pass_attrs = auth_username_format=%Lu
# uname -a
Linux milne 2.6.28-hardened-r9 #4 SMP Mon Jun 7 10:50:28 PDT 2010 x86_64
Intel(R) Xeon(R) CPU E5405 @ 2.00GHz GenuineIntel GNU/Linux
(Using Gentoo 64-bit Linux)
And here's the email body:
========================================
D7A2438CCC19EF2A5272934C8D95@abc027.abc.local>
B3457C1920D7A2438CCC19EF2A5272934C8DB6@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272934C8F0D@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935008B5@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293500C27@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935011BC@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293525B1D@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293525D81@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293526139@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293526416@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935524A6@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935524C9@abc027.abc.local
B3457C1920D7A2438CCC19EF2A527293589E1C@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935BF6A9@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935E7414@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935E75E2@abc027.abc.local
B3457C1920D7A2438CCC19EF2A5272935E75E8@abc027.abc.local B3457C1920D7A2438CCC19EF2A5272935E800E@abc027.abc.local Subject: RE: Some Subject Date: Tue, 26 Oct 2010 15:11:01 -0700 Message-ID: B3457C1920D7A2438CCC19EF2A52729364D0A0@abc027.abc.local MIME-Version: 1.0 Content-Type: multipart/alternative; X-Mailer: Microsoft Office Outlook 12.0 Thread-Index:
AcrhmOPG84jptRwERxazrnj4Ne2YAQAACMtAAAAjcPAAAAu+gAAAE1hAAAAZWCAAABMaEAAADGzwACO3fdAACN0HsAEpVUrgAPt2jvAAAcrbcABrFxCQAAAXwjAAABGv8AAAI35QAYbDMHAAAPjDYAADNREgAA5XR4AAHoOFcAFtPWTAAIubCCAALwf/MAADBptQACzN6nAAAC61kAAAEN+AAACfkjAAO7MG4AAHLrMQALmKX6AAAUlCcAAAq9SgAAuoo+AAABqqMABXq+FwAD5jAAABYqZYoAOISoLgAAAd//AAAKVPYAApG8DwAAt6dsAAKxpd8AEy9g6gAADrZVAAAE4WgAAlXdkQAPY4tNAAAWY+kAFsS/RgAFtG7qAAAGJT8AAAPLUwAAA2xsAAACamEAAC4ufAADaZyfAAKizZ0AAAZ9EAAAWRHAAAAOgkgAAAUUOgAAQWBBAAij+IkAAE2k1wAAPgAgAANGrwsAACt+wgAC0EW9AAAEDpoAA12EJgAAAyAtAAAyATYAAgyfBQAKS/7/AAAEPWEAABNsTAAAAZujAALLkhkAAwwG5AAAMVt8AAAE3iwAEhzNqgAA8qbEAAIqDaMAA0UTmgAEE0hxAAlsz9IAABAdrAAFIwRcAAZxd5oAAOziPQAJEu45AAOaFlIAAeeCygADwtnbAD9zXHwAApeNOAACk/pfAAABxl0AAAWgGAAAB7d4AACGbyQACNOVmwABF68KAAKuKysAAsiR5gADDcY1AAOMAK8ACO561QABCk+7AAI5IqsAAJGVEgAFjaIkAANWSCQAACuaAwAAAKUQAABugScAAAbmTQAADspwAAzNO+MAAiFH8wAAEA4mAAMv6E0AEtz2wQADWsQHAAO4pvkACIRgpAAAApFcAAABO4UABmP4/QAAEJ/lABNOByMAAF28awAACwfDA= Content-Language: en-us X-OlkEid: C664F3259FA9393D4124CD409B5CAE330696EE82 content-class: urn:content-classes:message This is a multi-part message in MIME format. ------=_NextPart_000_0118_01CB7755.52354760 Content-Type: text/plain; Content-Transfer-Encoding: 7bit <SNIP>
=========================================================================== Obviously, it's breaking the message in the References header, and I've
tracked down the following byte sequence at the breaking point:
0D:09:0A:09
...which is CR,TAB,CR,LF.
What puzzles me is why Roy's gets broken here, when the others' do not;
they're the exact same email, so you'd think they would all get broken. I
can't figure out A) where that byte sequence is coming from (our Exchange
server, Tornado.net's exchange server, or one of the two of our external
spam filtering service... they use BigFish.com, and we use MxLogic), or B)
if that's even the issue. I've rebuilt Roy's entire machine, reconfigured Outlook from scratch, still
happening. When he logs onto another machine and sets up his Exchange
account on another outlook, it still happens. Our Exchange administrator
(I don't have access to Exchange) says there is nothing wrong with his
Exchange account, for what it's worth. I don't expect anyone to have the solution here, since there are many
variables, and there is other info that would probably be needed. What I DO
want is to be pointed in the right direction on how I might track this down.
For example, what is the path that this mangled email would take in this
case? Even with full logging in Dovecot and postfix, I can't see any info
on the path that this email follows in our system, and where it might be
getting mangled. Does it actually go through the same path as an email that
was actually sent to the IMAP account, or is it different when it is dragged
in from another account in the same mail client? Is this more likely a
problem with Postfix, or Sieve, or....? (I've disabled Amavis and SA to get
them out of the picture, and same problem).
Here's the dovecot config info; please let me know if you want any other
config info, and thanks ahead of time! # dovecot --version
1.1.19 # dovecot -n
# 1.1.19: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.28-hardened-r9 x86_64 Gentoo Base System release 1.12.13
protocols: imaps imap managesieve
listen(default): 127.0.0.1:143
listen(imap): 127.0.0.1:143
listen(managesieve): 127.0.0.1:2000
ssl_listen(default): *:993
ssl_listen(imap): *:993
ssl_listen(managesieve):
ssl_cert_file: /etc/ssl/dovecot/imapd.crt
ssl_key_file: /etc/ssl/dovecot/imapd.key
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(managesieve): /usr/libexec/dovecot/managesieve-login
login_greeting_capability(default): yes
login_greeting_capability(imap): yes
login_greeting_capability(managesieve): no
valid_chroot_dirs: /var/mail
first_valid_uid: 206
last_valid_uid: 206
first_valid_gid: 206
last_valid_gid: 206
maildir_copy_preserve_filename: yes
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(managesieve): /usr/libexec/dovecot/managesieve
mail_plugins(default): acl
mail_plugins(imap): acl
mail_plugins(managesieve):
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(managesieve): /usr/lib64/dovecot/managesieve
imap_client_workarounds(default): outlook-idle delay-newmail
imap_client_workarounds(imap): outlook-idle delay-newmail
imap_client_workarounds(managesieve):
sieve_storage(default):
sieve_storage(imap):
sieve_storage(managesieve): ~
sieve(default):
sieve(imap):
sieve(managesieve): ~/.dovecot.sieve
namespace:
type: private
separator: .
location: maildir:~/Maildir
inbox: yes
list: yes
subscriptions: yes
namespace:
type: public
separator: .
prefix: shared.
location: maildir:/var/mail/shared
list: yes
lda:
postmaster_address: postmaster@jensenmaritime.com
hostname: mail.jensenmaritime.com
mail_plugins: cmusieve
mail_plugin_dir: /usr/lib/dovecot/lda
sieve_global_path: /var/mail/.dovecot.sieve
sieve: ~/.dovecot.sieve
sendmail_path: /usr/lib/sendmail
auth_socket_path: /var/run/dovecot/deliver-auth
auth lda:
default_realm: jensenmaritime.com
user: postmaster
passdb:
driver: ldap
args: /etc/dovecot/lda-ldap.conf
userdb:
driver: static
args: allow_all_users=yes user=%Lu uid=206 gid=206 home=/var/mail/%Lu
socket:
type: listen
master:
path: /var/run/dovecot/deliver-auth
mode: 384
user: vmail
group: vmail
auth default:
mechanisms: PLAIN LOGIN
default_realm: jensenmaritime.com
user: postmaster
passdb:
driver: ldap
args: /etc/dovecot/default-ldap.conf
userdb:
driver: static
args: user=%Lu uid=206 gid=206 home=/var/mail/%Lu
socket:
type: listen
client:
path: /var/spool/postfix/private/auth
mode: 432
user: postfix
group: postfix
plugin:
acl: vfile # grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-ldap.conf
uris = ldap://127.0.0.1:389/
ldap_version = 3
base = ou=users,ou=accounts,dc=jensenmaritime,dc=com
scope = onelevel
auth_bind = yes
pass_filter =
(&(objectClass=CourierMailAccount)(!(disableimap=*))(mail=%Lu))
pass_attrs = auth_username_format=%Lu # uname -a
Linux milne 2.6.28-hardened-r9 #4 SMP Mon Jun 7 10:50:28 PDT 2010 x86_64
Intel(R) Xeon(R) CPU E5405 @ 2.00GHz GenuineIntel GNU/Linux
(Using Gentoo 64-bit Linux) D7A2438CCC19EF2A5272935E75F4@abc027.abc.local>
boundary="----=_NextPart_000_0118_01CB7755.52354760"
charset="iso-8859-1"
On 3.11.2010, at 19.53, Scott Goodwin wrote:
Emails to one specific user in my organization (let's call him Roy@Ocean.org) is having his emails mangled when dragging certain items from his Exchange mailbox to the IMAP account.
Is the client using SSL? If not, looking at the actual IMAP traffic would be helpful. Best would be to look it at the Outlook machine, but I don't know what Windows tools exist for that. You could also enable Dovecot's rawlog, which works also with SSL enabled. If in the output you see the same corruption then it's caused by Outlook (or some antivirus/firewall in the middle). http://wiki.dovecot.org/Debugging/Rawlog
Thanks for the reply, Timo. Yes, clients use SSL, which is required, but I could disable it, and may do so. (Since our Exchange migration, none of the IMAP traffic happens outside of our firewall). I missed the rawlog on the debugging page earlier -- I'll report back my findings. --scott
On Wed, Nov 3, 2010 at 1:32 PM, Timo Sirainen tss@iki.fi wrote:
On 3.11.2010, at 19.53, Scott Goodwin wrote:
Emails to one specific user in my organization (let's call him Roy@Ocean.org) is having his emails mangled when dragging certain items from his Exchange mailbox to the IMAP account.
Is the client using SSL? If not, looking at the actual IMAP traffic would be helpful. Best would be to look it at the Outlook machine, but I don't know what Windows tools exist for that. You could also enable Dovecot's rawlog, which works also with SSL enabled. If in the output you see the same corruption then it's caused by Outlook (or some antivirus/firewall in the middle). http://wiki.dovecot.org/Debugging/Rawlog
FYI, I got rawlog working and it shows the same break in the raw logs as in the broken headers. Below is a snippet from the rawlog (names and other identifiers redacted). The offending sequence is always in the References headers section, and you can see the line breaks there that show this. So it sounds like this can't be an issue with Dovecot, am I right? I wish I could get this figured out. Such a headache.... *sigh*
Thanks!
========================================================================
poht IDLE
DONE
slwf LIST "" "INBOX"
36oi LSUB "" "*"
2by6 IDLE
DONE
p5dx SELECT "shared.IT"
uz85 FETCH 6 (UID)
f925 UID FETCH 1:51 (UID FLAGS)
kd5i IDLE
DONE
y9u7 UID FETCH 51 (UID FLAGS BODY.PEEK[])
qoyj IDLE
DONE
rfxa UID STORE 51 +FLAGS (\Deleted \Seen)
h8ly IDLE
DONE
fe4g EXPUNGE
fauq IDLE
DONE
1wa2 UID FETCH 49 (UID FLAGS BODY.PEEK[])
6403 IDLE
DONE
luq8 UID FETCH 50 (UID FLAGS BODY.PEEK[])
700u IDLE
DONE
jl6b APPEND "shared.IT" (\Seen \Answered) " 2-Nov-2010 11:46:19 -0700"
{12898938}
From: "User" User@xxx.com
To: "zzz, Roy",
"zzz, Mandred",
"zzz, Jerry"
Cc: "zzz, Johan",
"Bob zzz" bobw@xxx.com,
"Brandon zzz" brandonh@xxx.com,
"Mark zzz" MarkBe@xxx.com,
"Cassie zzz" CassieE@xxx.com,
"Erin zzz" erinb@xxx.com
References: B3457C1920D7A2438CCC19EF2A527293372B11@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A527293372D9A@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933A2C3D@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933A310E@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933A3683@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933A3B97@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933D103B@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933D192F@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933D1B12@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933FD341@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933FD8F8@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933FD9A0@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272933FDFDD@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272934C8927@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272934C8B46@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272934C8D76@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272934C8D7A@yyy027.yyy.local On Wed, Nov 3, 2010 at 1:32 PM, Timo Sirainen tss@iki.fi wrote: On 3.11.2010, at 19.53, Scott Goodwin wrote: Emails to one specific user in my organization (let's call him
Roy@Ocean.org)
is having his emails mangled when dragging certain items from his
Exchange
mailbox to the IMAP account. Is the client using SSL? If not, looking at the actual IMAP traffic would
be helpful. Best would be to look it at the Outlook machine, but I don't
know what Windows tools exist for that. You could also enable Dovecot's
rawlog, which works also with SSL enabled. If in the output you see the same
corruption then it's caused by Outlook (or some antivirus/firewall in the
middle). http://wiki.dovecot.org/Debugging/RawlogD7A2438CCC19EF2A5272934C8D95@yyy027.yyy.local>
B3457C1920D7A2438CCC19EF2A5272934C8DB6@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272934C8F0D@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272935008B5@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A527293500C27@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272935011BC@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A527293525B1D@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A527293525D81@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A527293526139@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A527293526416@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272935524A6@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272935524C9@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A527293589E1C@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272935BF6A9@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272935E7414@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272935E75E2@yyy027.yyy.local
B3457C1920D7A2438CCC19EF2A5272935E75E8@yyy027.yyy.local
On Wed, 2010-11-03 at 16:18 -0700, Scott Goodwin wrote:
FYI, I got rawlog working and it shows the same break in the raw logs as in the broken headers. Below is a snippet from the rawlog (names and other identifiers redacted). The offending sequence is always in the References headers section, and you can see the line breaks there that show this. So it sounds like this can't be an issue with Dovecot, am I right?
Yeah, sounds like Outlook breaks with huge headers. That's one huge References header you have.
Looks like there was some more discussion of this via the digest, but to keep the original thread intact, I'll just reply here. I'm still a little confused over whether or not my issue is an Outlook/Exchange problem or something else, since I'm not certain how I should be interpreting this raw log. I've attached the rawlog to this email (domain names and a few surnames redacted, and about 40K crap between the <HEAD></HEAD> tags removed, but everything else intact). Can anyone take a peek at this and see if it indicates a problem upstream or downstream from the rawlog? Thanks much. Also note the other oddity that stands out is that some of the envelope recipients are broken (actually, all the recipients in my domain). For example, lines 19 - 21:
To: "Roy",
"Mandred",
"Jerry"
...don't show any email addresses -- just the names. I know that this is a problem, but I'm just not sure if it is a separate issue, or related to the email mangling problem. At any rate, line 28 shows the ~1134 character References header, with the breakage at the end there.
Note that this is the rawlog.out, not the rawlog.in. My confusion here lies in the fact that the breakage seems to be in the out-log only. I just don't know how to read these logs. The in-log is basically exactly the same as the out-log, except that it doesn't contain the first 38 lines that the out-log contains... so does that mean the email came in just fine, but didn't go "out" ok? Or does this still confirm that the email was mangled from outside of Dovecot? Thanks ahead of time. I can send the in-log if you want.
On Thu, Nov 4, 2010 at 8:08 AM, Timo Sirainen tss@iki.fi wrote:
On Wed, 2010-11-03 at 16:18 -0700, Scott Goodwin wrote:
FYI, I got rawlog working and it shows the same break in the raw logs as in the broken headers. Below is a snippet from the rawlog (names and other identifiers redacted). The offending sequence is always in the References headers section, and you can see the line breaks there that show this. So it sounds like this can't be an issue with Dovecot, am I right?
Yeah, sounds like Outlook breaks with huge headers. That's one huge References header you have.
Le 9 nov. 2010 à 01:57:17, Scott Goodwin a écrit :
Looks like there was some more discussion of this via the digest, but to keep the original thread intact, I'll just reply here. I'm still a little confused over whether or not my issue is an Outlook/Exchange problem or something else, since I'm not certain how I should be interpreting this raw log.
Hello Scott,
If I understood you correctly, the raw log shows fetches against a problematic message already stored in the IMAP mailbox; I would be tempted to say that Dovecot just returns the garbage it has been asked to store...
Perhaps would it be nice to trace the whole history of such a problematic message; quoting your initial post:
Le 3 nov. 2010 à 20:53:01, Scott Goodwin a écrit :
[...] So, in summary, user1@Tornado.org send an email to Roy@Ocean.org and Sue@Ocean.org. Both drag the email from their Exchange box into any folder (shared or otherwise) in the IMAP tree. Sue's copy makes it in just fine, while Roy's copy gets mangled. [...]
it might prove useful to have, for example:
- the raw source of the message posted by user1@Tornado.org as stored in user1's MUA 2a. the raw source of the message received by Roy@Ocean.org as displayed by Roy's Outlook when received in the Exchange account 2b. the raw source of the message received by Sue@Ocean.org as displayed by Sues's Outlook when received in the Exchange account 3a. the raw log describing what happens when Roy drags the message into his IMAP mailbox 3b. the raw log describing what happens when Sue drags the message into her IMAP mailbox
HTH, Axel
On Mon, 2010-11-08 at 16:57 -0800, Scott Goodwin wrote:
Note that this is the rawlog.out, not the rawlog.in. My confusion here lies in the fact that the breakage seems to be in the out-log only. I just don't know how to read these logs. The in-log is basically exactly the same as the out-log, except that it doesn't contain the first 38 lines that the out-log contains... so does that mean the email came in just fine, but didn't go "out" ok? Or does this still confirm that the email was mangled from outside of Dovecot? Thanks ahead of time. I can send the in-log if you want.
Yeah, the in-log would have been much more useful than out-log. The important question is if in the in-log the References: header is truncated at the same point.
in-log attached. I don't see any truncation in there, though you can see that there is the byte sequence 0D090A09 in there, which I think is what is breaking it (whatever "it" is). Still not sure if this indicates the mangling happens somewhere on my linux server, or somewhere upstream in Outlook/Exchange. God I hate having to deal with Outlook.
Also, to Axel: I don't have access to any of the raw email source on the Exchange/Outlook end, which is frustrating, but unavoidable. If I find the mangling happening on the Postfix/Dovecot end of things, this is actually good news to me, since I have full access to that portion of the email trail.
On Tue, Nov 9, 2010 at 9:54 AM, Timo Sirainen tss@iki.fi wrote:
On Mon, 2010-11-08 at 16:57 -0800, Scott Goodwin wrote:
Note that this is the rawlog.out, not the rawlog.in. My confusion here lies in the fact that the breakage seems to be in the out-log only. I just don't know how to read these logs. The in-log is basically exactly the same as the out-log, except that it doesn't contain the first 38 lines that the out-log contains... so does that mean the email came in just fine, but didn't go "out" ok? Or does this still confirm that the email was mangled from outside of Dovecot? Thanks ahead of time. I can send the in-log if you want.
Yeah, the in-log would have been much more useful than out-log. The important question is if in the in-log the References: header is truncated at the same point.
On 9.11.2010, at 22.16, Scott Goodwin wrote:
in-log attached. I don't see any truncation in there, though you can see that there is the byte sequence 0D090A09 in there, which I think is what is breaking it (whatever "it" is).
Yeah. Outlook adds the extra empty line there in the middle.
Still not sure if this indicates the mangling happens somewhere on my linux server, or somewhere upstream in Outlook/Exchange. God I hate having to deal with Outlook.
You'd have to look at the outgoing IMAP traffic on the Outlook machine. (I don't know how.)
participants (3)
-
Axel Luttgens
-
Scott Goodwin
-
Timo Sirainen