[Dovecot] pop3+leave messages on server
hi All,
I have a srange problem.
Before dovecot there was a courier imap and it worked just fine. The client(s) are the same then before, so I think, this is an server side problem.
I have a user with Outlook and he uses pop3 and leaves messages on server.
Sometimes the messages get downloaded again, when he starts the outlook. Not all, but 7-9 hours back.
uidl format is like it's suggested:
pop3_uidl_format = %08Xu%08Xv
Any idea, what's wrong and what I can do?
Thank you very much,
tamas
On Tue, 29 Dec 2009 09:48:46 +0100, Papp Tamas tompos@martos.bme.hu wrote:
hi All,
I have a srange problem.
Before dovecot there was a courier imap and it worked just fine. The client(s) are the same then before, so I think, this is an server side problem.
I have a user with Outlook and he uses pop3 and leaves messages on server.
Sometimes the messages get downloaded again, when he starts the outlook. Not all, but 7-9 hours back.
uidl format is like it's suggested:
pop3_uidl_format = %08Xu%08Xv
Any idea, what's wrong and what I can do?
Thank you very much,
tamas
Is that a problem with other MUAs too? Or is Outlook the only affected one? You tried Thunderbird or Windows Live Mail to verify?
Regards
tobi
Tobias wrote:
Is that a problem with other MUAs too? Or is Outlook the only affected one? You tried Thunderbird or Windows Live Mail to verify?
No, the users are using only Outlook. Should I try it with TB? It would be a bit hard, because the failure does not depend on anything, I don't know, how to cause it.
Thanks,
tamas
On Tue, 29 Dec 2009 11:27:49 +0100, Papp Tamas tompos@martos.bme.hu wrote:
Tobias wrote:
Is that a problem with other MUAs too? Or is Outlook the only affected one? You tried Thunderbird or Windows Live Mail to verify?
No, the users are using only Outlook. Should I try it with TB? It would be a bit hard, because the failure does not depend on anything, I don't know, how to cause it.
Thanks,
tamas
Hi Tamas
yes I would try with another MUA just to verify whether it's a MUA- or server problem. Btw: No chance to persuade the user to use IMAP instead of POP3?
Cheers
tobi
Tobias wrote:
yes I would try with another MUA just to verify whether it's a MUA- or server problem.
Well as I wrote, it will be hard, but I'll do it. Until I cannot, do you have any other idea?
Btw: No chance to persuade the user to use IMAP instead of POP3?
No there is no chance.
I had another thread here in the list (imap + TB + two Trash folders). Until I don't have a final user and client independent solution, I recommend imap for users. But this is not the biggest problem. The protocoll imap is not the same as using pop3+leave messages on server. He wants to use this scenario anyway as he did for many years with no problems.
Thank you,
tamas
Papp Tamas wrote:
Tobias wrote:
yes I would try with another MUA just to verify whether it's a MUA- or server problem.
Well as I wrote, it will be hard, but I'll do it. Until I cannot, do you have any other idea?
BTW, I think we can be pretty sure, it's not a MUA problem, because it was working just fine, until it was connected to an courier server (something very old version from 2004).
Thanks,
tamas
On 2009-12-29, Papp Tamas (tompos@martos.bme.hu) wrote:
The protocoll imap is not the same as using pop3+leave messages on server.
That is correct. The POP protocol is designed to delete the messages from the server once they have been POPPED. The IMAP protocol is designed to leave the messages on the server all the time.
He wants to use this scenario anyway as he did for many years with no problems.
The fact that he has had 'no problems' in many years is purely the luck of the draw. One minor bug in either the IMAP/POP server or the mail client during an upgrade or other maintenance, and boom - he will *really* be surprised when *all* of his messages from the last few *years* are downloaded again, not just the last few hours worth.
One real worl example of how this can happen: his Outlook profile gets corrupted, and he still has all of his mail, but he has to recreate his account settings - boom, all of his mail downloads again from the date he started leaving them on the server.
Same thing goes for when he install Outlook on a new computer - which, in some cases, might be desirable, and in others, not...
The bottom line is, POP is simply not designed to work this way. The fact that it can be *manipulated* to work this way doesn't change the nature of the protocol, or the inherent problems.
--
Best regards,
Charles
Not to pick nits but pop3+leave on server does not mean you have all message from the dawn of time stored on the server. Outlook and presumably other MUAs have "remove from server after X time" and "remove from server when message is deleted" options when leaving pop3 messages on the server is enabled.
I agree with the general principal though that if you wish to keep messages on server as general practice you should use IMAP. IMO pop3+leave is really only ideal while traveling using other clients/computers or maybe if IMAP is not offered at all.
-----Original Message----- From: dovecot-bounces+jkrejci=usinternet.com@dovecot.org [mailto:dovecot-bounces+jkrejci=usinternet.com@dovecot.org] On Behalf Of Charles Marcus Sent: Tuesday, December 29, 2009 6:19 AM To: Dovecot Mailing List Subject: Re: [Dovecot] pop3+leave messages on server
On 2009-12-29, Papp Tamas (tompos@martos.bme.hu) wrote:
The protocoll imap is not the same as using pop3+leave messages on server.
That is correct. The POP protocol is designed to delete the messages from the server once they have been POPPED. The IMAP protocol is designed to leave the messages on the server all the time.
He wants to use this scenario anyway as he did for many years with no problems.
The fact that he has had 'no problems' in many years is purely the luck of the draw. One minor bug in either the IMAP/POP server or the mail client during an upgrade or other maintenance, and boom - he will *really* be surprised when *all* of his messages from the last few *years* are downloaded again, not just the last few hours worth.
One real worl example of how this can happen: his Outlook profile gets corrupted, and he still has all of his mail, but he has to recreate his account settings - boom, all of his mail downloads again from the date he started leaving them on the server.
Same thing goes for when he install Outlook on a new computer - which, in some cases, might be desirable, and in others, not...
The bottom line is, POP is simply not designed to work this way. The fact that it can be *manipulated* to work this way doesn't change the nature of the protocol, or the inherent problems.
--
Best regards,
Charles
On 2009-12-29, Justin Krejci (jkrejci@usinternet.com) wrote:
Not to pick nits but pop3+leave on server does not mean you have all message from the dawn of time stored on the server. Outlook and presumably other MUAs have "remove from server after X time" and "remove from server when message is deleted" options when leaving pop3 messages on the server is enabled.
It depends on how it is configured. You can also configure it to never delete them.
Justin Krejci wrote, On 2009. 12. 29. 16:27:
Not to pick nits but pop3+leave on server does not mean you have all message from the dawn of time stored on the server. Outlook and presumably other MUAs have "remove from server after X time" and "remove from server when message is deleted" options when leaving pop3 messages on the server is enabled.
I agree with the general principal though that if you wish to keep messages on server as general practice you should use IMAP. IMO pop3+leave is really only ideal while traveling using other clients/computers or maybe if IMAP is not offered at all.
This is not an ideal situation, we are agreed.
tamas
Charles Marcus wrote, On 2009. 12. 29. 13:18:
The fact that he has had 'no problems' in many years is purely the luck of the draw. One minor bug in either the IMAP/POP server or the mail client during an upgrade or other maintenance, and boom - he will *really* be surprised when *all* of his messages from the last few *years* are downloaded again, not just the last few hours worth.
No, he makes the messages deleted from time to time.
tamas
Tobias wrote, On 2009. 12. 29. 11:45:
Hi Tamas
yes I would try with another MUA just to verify whether it's a MUA- or server problem. Btw: No chance to persuade the user to use IMAP instead of POP3?
Well, the news is that every other user's client is working fine. This error only for one user. Of course, this user is the boss:)
tamas
On 2009-12-29 3:48 AM, Papp Tamas wrote:
I have a user with Outlook and he uses pop3 and leaves messages on server
Sometimes the messages get downloaded again, when he starts the outlook. Not all, but 7-9 hours back.
uidl format is like it's suggested:
pop3_uidl_format = %08Xu%08Xv
Any idea, what's wrong and what I can do?
Syandard procedure for asking for help on a email support list is to provide at least some basic info on your system, like, dovecot version/output of dovecot -n, platform, filesystem, etc...
--
Best regards,
Charles
On Tue, 2009-12-29 at 09:48 +0100, Papp Tamas wrote:
I have a user with Outlook and he uses pop3 and leaves messages on server.
Sometimes the messages get downloaded again, when he starts the outlook. Not all, but 7-9 hours back.
uidl format is like it's suggested:
pop3_uidl_format = %08Xu%08Xv
dovecot -n output could be helpful. Also did you look at logs? Maybe there are errors. http://wiki.dovecot.org/Logging
Anyway, the possible reasons:
Dovecot changed the UIDLs because something weird happened. In that case it logged an error.
Did you use the courier migration script so that the UIDLs got converted to Dovecot? If not, maybe Outlook got confused because the UIDLs changed, and fixing it would require recreating the Outlook account or something..
Outlook just doesn't like the new UIDLs and messes up itself. I think Courier usually uses maildir filenames as UIDLs, you could have kept doing that with pop3_uidl_format=%f. But changing it now would cause users to redownload everything again.. With v1.2.5+ you could use pop3_save_uidl=yes to make it possible to change UIDLs on the fly.
Timo Sirainen wrote, On 2009. 12. 29. 21:08:
dovecot -n output could be helpful. Also did you look at logs? Maybe
Yes, I looked logs, I didn't found an error.
This is dovecot -n:
# 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.28-15-server x86_64 Ubuntu 9.04 xfs protocols: imap imaps pop3 pop3s managesieve ssl_cert_file: /data/apps/dovecot/mail.exampledomain.hu.pem ssl_key_file: /data/apps/dovecot/mail.exampledomain.hu.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_user: _dovecot verbose_proctitle: yes first_valid_uid: 1 mail_privileged_group: dovecot mail_uid: dovecot mail_gid: dovecot mail_location(default): maildir:/data/mail/%d/%n/Maildir/ mail_location(imap): maildir:/data/mail/%d/%n/Maildir/ mail_location(pop3): maildir:/data/mail/%d/%n/Maildir/ mail_location(managesieve): maildir:/data/mail/%d/%n maildir_stat_dirs: yes maildir_copy_preserve_filename: yes maildir_very_dirty_syncs: yes mbox_write_locks: fcntl dotlock mbox_dirty_syncs: no mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota deleted_to_trash mail_plugins(imap): quota imap_quota deleted_to_trash mail_plugins(pop3): mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): imap_client_workarounds(managesieve): pop3_enable_last(default): no pop3_enable_last(imap): no pop3_enable_last(pop3): yes pop3_enable_last(managesieve): no pop3_reuse_xuidl(default): no pop3_reuse_xuidl(imap): no pop3_reuse_xuidl(pop3): yes pop3_reuse_xuidl(managesieve): no pop3_save_uidl(default): no pop3_save_uidl(imap): no pop3_save_uidl(pop3): yes pop3_save_uidl(managesieve): no pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh pop3_client_workarounds(managesieve): lda: postmaster_address: root@mail.exampledomain.hu hostname: mail0.exampledomain.hu mail_plugins: quota sieve deliver_log_format: msgid=%m: From=<%f> %$ auth default: mechanisms: plain login default_realm: exampledomain.hu user: dovecot passdb: driver: sql args: /data/apps/dovecot/dovecot-mysql.conf userdb: driver: sql args: /data/apps/dovecot/dovecot-mysql.conf socket: type: listen client: path: /data/postfix/private/dovecot-auth-client mode: 438 user: dovecot group: dovecot master: path: /var/run/dovecot/auth-master mode: 432 user: dovecot group: dovecot plugin: sieve: /data/mail/%d/%n/.sieve.dovecot sieve_dir: /data/mail/%d/%n quota: dict:User quota::proxy::quota quota_rule: *:storage=10GB dict: quota: mysql:/data/apps/dovecot/dict-sql.conf
I didn't want to send it, because I don't know, which part is useful in this case.
there are errors. http://wiki.dovecot.org/Logging
Actually I've juest enabled all debugging related options, except password. I'm afraid, there will be too much messages.
Anyway, the possible reasons:
- Dovecot changed the UIDLs because something weird happened. In that case it logged an error.
No error.
- Did you use the courier migration script so that the UIDLs got converted to Dovecot? If not, maybe Outlook got confused because the UIDLs changed, and fixing it would require recreating the Outlook account or something..
No, there was no conversation. I was thinking about recreating the account, but it's not possible right now. It will be actual in 1-2 weeks.
Anyway, even if it help, there is an error, which does not happen all the time, still I think it's server related.
- Outlook just doesn't like the new UIDLs and messes up itself. I think Courier usually uses maildir filenames as UIDLs, you could have kept doing that with pop3_uidl_format=%f. But changing it now would cause users to redownload everything again.. With v1.2.5+ you could use pop3_save_uidl=yes to make it possible to change UIDLs on the fly.
Are you sure, it can be enabled on the fly and if I use it, the mail won't be donwload again after changing the uidl format?
Thank you,
tamas
On Sun, 2010-01-03 at 23:57 +0100, Papp Tamás wrote:
pop3_reuse_xuidl(pop3): yes
If a mail has X-UIDL: header it's used. Is it possible that some mails have it, and that's the reason the UIDL changed?
there are errors. http://wiki.dovecot.org/Logging
Actually I've juest enabled all debugging related options, except password. I'm afraid, there will be too much messages.
Errors are logged regardless of any debugging options. And the debugging options don't really help in detecting these kind of problems.
- Did you use the courier migration script so that the UIDLs got converted to Dovecot? If not, maybe Outlook got confused because the UIDLs changed, and fixing it would require recreating the Outlook account or something..
No, there was no conversation. I was thinking about recreating the account, but it's not possible right now. It will be actual in 1-2 weeks.
Anyway, even if it help, there is an error, which does not happen all the time, still I think it's server related.
I think it's unlikely. Outlook is known to have had lots of problems with POP3 + leave mail on server..
- Outlook just doesn't like the new UIDLs and messes up itself. I think Courier usually uses maildir filenames as UIDLs, you could have kept doing that with pop3_uidl_format=%f. But changing it now would cause users to redownload everything again.. With v1.2.5+ you could use pop3_save_uidl=yes to make it possible to change UIDLs on the fly.
Are you sure, it can be enabled on the fly and if I use it, the mail won't be donwload again after changing the uidl format?
User needs to have been logged in while still running with older uidl format, so that the UIDLs get written to dovecot-uidlist file. After that they don't change when you change pop3_uidl_format.
participants (6)
-
Charles Marcus
-
Justin Krejci
-
Papp Tamas
-
Papp Tamás
-
Timo Sirainen
-
Tobias