[Dovecot] PDF corruption issue
Greetings,
I have several users reporting that PDF attachments are coming in
corrupted. They are using either thunderbird or outlook on windows xp.
I have flushed the cache files for the users, but the problem still
persists. Could anyone offer some troubleshooting tips?
Thanks in advance for any assistance.
Below is my version and config.
Regards,
Mike
dovecot --version 1.1.7
dovecot -n
# 1.1.7: /etc/dovecot.conf
Warning: fd limit 1024 is lower than what Dovecot can use under full
load (more than 1792). Either grow the limit or change
login_max_processes_count and max_mail_processes settings
# OS: Linux 2.6.18-92.1.13.el5 x86_64 Red Hat Enterprise Linux Server
release 5.2 (Tikanga)
base_dir: /var/run/dovecot/
log_path: /var/log/dovecot
listen: [*]
ssl_key_file: /etc/pki/dovecot/certs/dovecot.pem
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(pop3): /usr/libexec/dovecot/pop3-login
login_max_processes_count: 384
login_max_connections: 384
max_mail_processes: 1024
verbose_proctitle: yes
first_valid_uid: 10
mail_privileged_group: mail
mail_location: mbox:%h/mail:INBOX=/var/spool/mail/%u
mail_debug: yes
mail_full_filesystem_access: yes
mmap_disable: yes
mail_nfs_index: yes
lock_method: dotlock
mbox_read_locks: dotlock
mbox_write_locks: dotlock
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(pop3): /usr/libexec/dovecot/pop3
mail_plugin_dir(default): /usr/lib64/dovecot/imap
mail_plugin_dir(imap): /usr/lib64/dovecot/imap
mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3
mail_log_max_lines_per_sec: 25
imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep
imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep
imap_client_workarounds(pop3):
namespace:
type: private
separator: /
inbox: yes
list: yes
subscriptions: yes
namespace:
type: private
separator: /
prefix: mail/
hidden: yes
subscriptions: yes
namespace:
type: private
separator: /
prefix: ~/mail/
hidden: yes
subscriptions: yes
namespace:
type: private
separator: /
prefix: ~%u/mail/
hidden: yes
subscriptions: yes
auth default:
mechanisms: plain login digest-md5
verbose: yes
debug: yes
passdb:
driver: ldap
args: /etc/dovecot-ldap.conf
userdb:
driver: ldap
args: /etc/dovecot-ldap.conf
socket:
type: listen
client:
path: /var/spool/postfix/private/auth
mode: 432
user: postfix
group: postfix
Michael Dunne schrieb:
Greetings,
I have several users reporting that PDF attachments are coming in corrupted. They are using either thunderbird or outlook on windows xp. I have flushed the cache files for the users, but the problem still persists. Could anyone offer some troubleshooting tips?
Thanks in advance for any assistance.
Below is my version and config.
Regards,
Mike
dovecot --version 1.1.7
dovecot -n # 1.1.7: /etc/dovecot.conf Warning: fd limit 1024 is lower than what Dovecot can use under full load (more than 1792). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: Linux 2.6.18-92.1.13.el5 x86_64 Red Hat Enterprise Linux Server release 5.2 (Tikanga) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot listen: [*] ssl_key_file: /etc/pki/dovecot/certs/dovecot.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_max_processes_count: 384 login_max_connections: 384 max_mail_processes: 1024 verbose_proctitle: yes first_valid_uid: 10 mail_privileged_group: mail mail_location: mbox:%h/mail:INBOX=/var/spool/mail/%u mail_debug: yes mail_full_filesystem_access: yes mmap_disable: yes mail_nfs_index: yes lock_method: dotlock mbox_read_locks: dotlock mbox_write_locks: dotlock mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 mail_log_max_lines_per_sec: 25 imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(pop3): namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~/mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~%u/mail/ hidden: yes subscriptions: yes auth default: mechanisms: plain login digest-md5 verbose: yes debug: yes passdb: driver: ldap args: /etc/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix
Hi Michael, the chances that dovecot or postfix has something to do with this are nearly null, corruption of spez. file types may result by broken virus scanners , or what i never had expected i had this issue one time on a windows exchange machine with pop3 downloader which had a broken interface bonding, so it was the nic drivers fault, so there may be some equal nic or network problem on the clients side and or virus scanners , proxies etc on their side, a good test is to use some webmail ( i.e squirrelmail ) on the dovecot server itself open some example mails ( which are reported to be broken ) with it and look if they currupted or not, if they arent the problem is on the client side and you dont have to care, if they are allready corrupted on your mailserver check antivirus solution you have installted in the server, disable it for further tests.
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Greetings,
Thanks to everyone that replied. I have not been able to isolate the
cause, has this happened to anyone else?
Thanks,
Mike
On Jan 5, 2009, at 5:43 PM, Michael Dunne wrote:
Greetings,
I have several users reporting that PDF attachments are coming in
corrupted. They are using either thunderbird or outlook on windows
xp. I have flushed the cache files for the users, but the problem
still persists. Could anyone offer some troubleshooting tips?Thanks in advance for any assistance.
Below is my version and config.
Regards,
Mike
dovecot --version 1.1.7
dovecot -n # 1.1.7: /etc/dovecot.conf Warning: fd limit 1024 is lower than what Dovecot can use under full
load (more than 1792). Either grow the limit or change
login_max_processes_count and max_mail_processes settings # OS: Linux 2.6.18-92.1.13.el5 x86_64 Red Hat Enterprise Linux
Server release 5.2 (Tikanga) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot listen: [*] ssl_key_file: /etc/pki/dovecot/certs/dovecot.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_max_processes_count: 384 login_max_connections: 384 max_mail_processes: 1024 verbose_proctitle: yes first_valid_uid: 10 mail_privileged_group: mail mail_location: mbox:%h/mail:INBOX=/var/spool/mail/%u mail_debug: yes mail_full_filesystem_access: yes mmap_disable: yes mail_nfs_index: yes lock_method: dotlock mbox_read_locks: dotlock mbox_write_locks: dotlock mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 mail_log_max_lines_per_sec: 25 imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(pop3): namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~/mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~%u/mail/ hidden: yes subscriptions: yes auth default: mechanisms: plain login digest-md5 verbose: yes debug: yes passdb: driver: ldap args: /etc/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix
I think the main question here is if this problem is specifically with PDFs, or if you're just not sending other attachments as much? v1.1 also does have some mbox bugs that will be fixed in v1.1.8 (I think I'll do the release now).
On Wed, 2009-01-07 at 11:58 -0800, Michael Dunne wrote:
Greetings,
Thanks to everyone that replied. I have not been able to isolate the
cause, has this happened to anyone else?Thanks,
Mike
On Jan 5, 2009, at 5:43 PM, Michael Dunne wrote:
Greetings,
I have several users reporting that PDF attachments are coming in
corrupted. They are using either thunderbird or outlook on windows
xp. I have flushed the cache files for the users, but the problem
still persists. Could anyone offer some troubleshooting tips?Thanks in advance for any assistance.
Below is my version and config.
Regards,
Mike
dovecot --version 1.1.7
dovecot -n # 1.1.7: /etc/dovecot.conf Warning: fd limit 1024 is lower than what Dovecot can use under full
load (more than 1792). Either grow the limit or change
login_max_processes_count and max_mail_processes settings # OS: Linux 2.6.18-92.1.13.el5 x86_64 Red Hat Enterprise Linux
Server release 5.2 (Tikanga) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot listen: [*] ssl_key_file: /etc/pki/dovecot/certs/dovecot.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_max_processes_count: 384 login_max_connections: 384 max_mail_processes: 1024 verbose_proctitle: yes first_valid_uid: 10 mail_privileged_group: mail mail_location: mbox:%h/mail:INBOX=/var/spool/mail/%u mail_debug: yes mail_full_filesystem_access: yes mmap_disable: yes mail_nfs_index: yes lock_method: dotlock mbox_read_locks: dotlock mbox_write_locks: dotlock mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 mail_log_max_lines_per_sec: 25 imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(pop3): namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~/mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~%u/mail/ hidden: yes subscriptions: yes auth default: mechanisms: plain login digest-md5 verbose: yes debug: yes passdb: driver: ldap args: /etc/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix
On 1/7/2009 2:58 PM, Michael Dunne wrote:
Greetings,
Thanks to everyone that replied. I have not been able to isolate the cause, has this happened to anyone else?
I don't know if it helps, but I had a pdf corruption problem a while back that drove me crazy for about 2 weeks, to the point I was beginning to suspect the mail server too, even though it didn't make sense...
Then I accidentally stumbled on the cause...
We use a centrally located/shared version of FoxitReader (standalone version, not the installer version).
At some point I updated it, and sometime later started getting occasional complaints of corrupt pdfs - some of the corrupt pdfs were generated internally, and some weren't. They would be emailed/forwarded once or more times and would be ok up to a certain point. I had narrowed the problem down to PDFs that would open ok in FoxitReader, but not Adobe Reader - but didn't connect the foxitreader update and the corruption issues until I just happened to watch someone open a PDF from an email, click the 'File' menu in FoxitReader, then click 'Save As', and save a copy.
Anyway, careful testing revealed that this particular build of FoxitReader had a weird bug, where doing this - opening a PDF then clickng 'File' > 'Save As' would create a PDF that would open ok in FoxitReader, but not Adobe Reader. I also confirmed this as a known bug with the Foxit Software guys. Sorry, but don't remember the build #...
Anyway, for what it worth...
--
Best regards,
Charles
Michael Dunne wrote:
Greetings,
Thanks to everyone that replied. I have not been able to isolate the cause, has this happened to anyone else?
Just a few days ago I sent a file to someone on my same mailserver who then reported the file corrupt and sent it back to me (where I also verified it corrupt). I use Thunderbird to send and they use OE to receive. Postfix + Dovecot 1.1.3
I personally suspect Thunderbird, but it's possible that it is related to Dovecot
Certainly I occasionally receive emails from external people which also appear to have corrupt attachments until resent (usually PDF, but that's also mainly what I receive). These would be sent from probably a MS Exchange using Outlook, and received here by Postfix+Dovecot+Thunderbird
I can basically confirm the same intermittent issue, but not where it's coming from
Oh, some wild thought occurred to me - check the actual files on disk. ie go into the server and grab the actual "corrupt" message from disk and call it something.eml - then you can double click it on a client machine and it should open in the default email program and you can then prove that it wasn't the imap/pop delivery which was corrupting the message. This still means it could be either the MTA or the delivery agent, but I think it will vindicate Dovecot?
Regards
Ed W
On 1/8/2009, Ed W (lists@wildgooses.com) wrote:
I personally suspect Thunderbird, but it's possible that it is related to Dovecot
I have also seen TBird have a corrupt local cache of the file... it seems to usually happen to people who I know are the 'impatient' type, and will click on a message with a large attachment, then when it doesn't respond *immediately*, start clicking on other messages/folders...
I know it is a local cache problem because if I delete the local copy of the folder in question and force TBird to re-download the message from the IMAP server, the PDF (or whatever) is fine... so yes, TBird has a problem in this area...
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi,
On 2009-01-08 18:29, Charles Marcus wrote: | On 1/8/2009, Ed W (lists@wildgooses.com) wrote: |> I personally suspect Thunderbird, but it's possible that it is related |> to Dovecot | | I have also seen TBird have a corrupt local cache of the file... it | seems to usually happen to people who I know are the 'impatient' type, | and will click on a message with a large attachment, then when it | doesn't respond *immediately*, start clicking on other messages/folders... | | I know it is a local cache problem because if I delete the local copy of | the folder in question and force TBird to re-download the message from | the IMAP server, the PDF (or whatever) is fine... so yes, TBird has a | problem in this area...
Have a look at https://bugzilla.mozilla.org/show_bug.cgi?id=253711 - may be related.
STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/ Phone: +86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779
PGP key: E883A005 https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iEYEARECAAYFAklmxLYACgkQ7yMg/OiDoAVXVACfez6XxMKh/bGM6k39wnnvTYbe nxwAni+bqI+rYDC26TbDywnW3spn2jb4 =YCO2 -----END PGP SIGNATURE-----
Patrick Nagel wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi,
On 2009-01-08 18:29, Charles Marcus wrote: | On 1/8/2009, Ed W (lists@wildgooses.com) wrote: |> I personally suspect Thunderbird, but it's possible that it is related |> to Dovecot | | I have also seen TBird have a corrupt local cache of the file... it | seems to usually happen to people who I know are the 'impatient' type, | and will click on a message with a large attachment, then when it | doesn't respond *immediately*, start clicking on other messages/folders... | | I know it is a local cache problem because if I delete the local copy of | the folder in question and force TBird to re-download the message from | the IMAP server, the PDF (or whatever) is fine... so yes, TBird has a | problem in this area...
Have a look at https://bugzilla.mozilla.org/show_bug.cgi?id=253711 - may be related.
I just came across the corruption issue with Thunderbird today trying to download a 4 meg PDF. The trick was to sit and stare at it while it thought about the attachment after quitting and reopening. Clicking around will anger it.
~Seth
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Ed,
I can basically confirm the same intermittent issue, but not where it's coming from Same to me. I found it sometimes at JPGs included at emails. The most of the image shows correct but about the last 30 percent is only grey.
Stefan
CaC, Computer and Communication Inhaber Stefan Klatt Triftstrasse 9 60528 Frankfurt Germany
Tel.: +49-(0)172-6807809 Tel.: +49-(0)69-67808-900
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAklqawEACgkQ+AkeCg4UPr5G6gCg7xLiuJfeRvIcuDoBZwuarCqM SOUAmwTwBEjT2/KWxRQAUxc+6DkT1JKm =Vyyk -----END PGP SIGNATURE-----
On Sun, Jan 11, 2009 at 10:56 PM, Stefan Klatt stefan.klatt@cac-netzwerk.de wrote:
Hi Ed,
I can basically confirm the same intermittent issue, but not where it's coming from Same to me. I found it sometimes at JPGs included at emails. The most of the image shows correct but about the last 30 percent is only grey.
Could you please have a look if you are seeing messages like this in your dovecot.log?
Error: IMAP(username): FETCH for mailbox Sent UID 2375 got too little data: 4096 vs 5247
(with different numbers)
Thanks, Diego Liziero.
Greetings,
We were able to isolate the problem to user(s) error. Thanks again for all of the responses!
Regards,
MIke
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 7 Jan 2009, Michael Dunne wrote:
has this happened to anyone else?
Yes. The PDFs in question were "binary" PDFs with embedded ^Z. On Windows this is EOF marker for text files. I suppose that there is some issue interpreting text/pdf or application/pdf as text file or something - I don't remember the Content-Type header neither the disposition setting. I copied the Maildir file into my account and could open the PDF with pine fine.
I repacked the PDF into a Zip, which worked then.
BTW: The sender never used Windows, but Linux and FreeBSD. Funnily the recipient was the same person all the time.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBSWXfEnWSIuGy1ktrAQLQsAf/f3Fj4hEy8L6ooTAzBB8bG4MUA7nQkFSf 4y/1PF8cb3wux69RJou1fLvs3/mlvrepUdZJrXQpmEFPPfZ6ouLq9hDmYCxGB1qK UBtN+6DPivTas4Bo7t465lihze7ZhgS79a2tyP2raoCRJ3X4oLK/cpe0EvIbt3zl q2TtGRqFEERi+xNXDtV8Gxg1nd5V/a19ww/VDVnesnb+0y98Ht9niJLr1+yFLP1s EZyEmoftXxXvQGQSlXiwOg52jCt12SaXKwbKhKwLCaCFkdRa0pMlSsamvioPgVxl +TRHUHtmc0Ha5/l/0E16/CHJfE2IvxQtlZwLg0l0dJpMpY/raaDl/Q== =y7++ -----END PGP SIGNATURE-----
participants (10)
-
Charles Marcus
-
Diego Liziero
-
Ed W
-
Michael Dunne
-
Patrick Nagel
-
Robert Schetterer
-
Seth Mattinen
-
Stefan Klatt
-
Steffen Kaiser
-
Timo Sirainen