[Dovecot] questions/problems virtual plugin pop3 leave copies on the server for x days then delete
Hi Timo, i allready asked about this , and did some fixes , but it isnt working yet ,as far i have a setup like
namespace default { # type = private separator = / prefix = "" list = yes # hidden = no subscriptions = yes }
namespace virtual { # type = private prefix = "virtual/" separator = / location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual hidden = yes list = no subscriptions= no }
namespace real { # type = private prefix = "RealMails/" separator = / list = no hidden = yes }
pop3_enable_last = yes pop3_reuse_xuidl = no pop3_uidl_format = %g pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s mail_max_userip_connections = 100 mail_plugins = expire virtual quota mail_log fts fts_squat notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
RealMails/ RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* unseen
normally this works as expected
some users want to use leave copies messages on the server for x days then delete via pop3
i tested this with thunderbird and looked into the maildir, it seems that the mails stay in "new" folder of the maildir after i downloaded via pop3 ( i am not sure is this expected ? , i would thought they get to cur ) thunderbird does not download this mails again ( good ), but it doesnt delete them from maildir new ever( bad ) , so the whole leave copy and delete after x days function pop3 is not working
i am not quite sure if there is relation to virtual namespaces ( which i like to use in future too ) , but have to renable this feature working any idea?
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Am 13.03.2011 12:40, schrieb Robert Schetterer:
Hi Timo, i allready asked about this , and did some fixes , but it isnt working yet ,as far i have a setup like
namespace default { # type = private separator = / prefix = "" list = yes # hidden = no subscriptions = yes }
namespace virtual { # type = private prefix = "virtual/" separator = / location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual hidden = yes list = no subscriptions= no }
namespace real { # type = private prefix = "RealMails/" separator = / list = no hidden = yes }
pop3_enable_last = yes pop3_reuse_xuidl = no pop3_uidl_format = %g pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s mail_max_userip_connections = 100 mail_plugins = expire virtual quota mail_log fts fts_squat notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
RealMails/ RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* unseen
normally this works as expected
some users want to use leave copies messages on the server for x days then delete via pop3
i tested this with thunderbird and looked into the maildir, it seems that the mails stay in "new" folder of the maildir after i downloaded via pop3 ( i am not sure is this expected ? , i would thought they get to cur ) thunderbird does not download this mails again ( good ), but it doesnt delete them from maildir new ever( bad ) , so the whole leave copy and delete after x days function pop3 is not working
i am not quite sure if there is relation to virtual namespaces ( which i like to use in future too ) , but have to renable this feature working any idea?
ok i have compared this with a nearly equal setupped server but without virtual plugin, but with same thunderbird functions here the mail gets into mail cur after pop3 download, and gets deleted on the server if i delete them in tb pop3 inbox ( wich is another option which should work equal as delete after x days ) so i am nearly sure there is a problem with virtual plugin or in my using it speculate mail cant be found to delete if it stays in maildir new
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Am 13.03.2011 13:02, schrieb Robert Schetterer:
Am 13.03.2011 12:40, schrieb Robert Schetterer:
Hi Timo, i allready asked about this , and did some fixes , but it isnt working yet ,as far i have a setup like
namespace default { # type = private separator = / prefix = "" list = yes # hidden = no subscriptions = yes }
namespace virtual { # type = private prefix = "virtual/" separator = / location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual hidden = yes list = no subscriptions= no }
namespace real { # type = private prefix = "RealMails/" separator = / list = no hidden = yes }
pop3_enable_last = yes pop3_reuse_xuidl = no pop3_uidl_format = %g pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s mail_max_userip_connections = 100 mail_plugins = expire virtual quota mail_log fts fts_squat notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
RealMails/ RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* unseen
normally this works as expected
some users want to use leave copies messages on the server for x days then delete via pop3
i tested this with thunderbird and looked into the maildir, it seems that the mails stay in "new" folder of the maildir after i downloaded via pop3 ( i am not sure is this expected ? , i would thought they get to cur ) thunderbird does not download this mails again ( good ), but it doesnt delete them from maildir new ever( bad ) , so the whole leave copy and delete after x days function pop3 is not working
i am not quite sure if there is relation to virtual namespaces ( which i like to use in future too ) , but have to renable this feature working any idea?
ok i have compared this with a nearly equal setupped server but without virtual plugin, but with same thunderbird functions here the mail gets into mail cur after pop3 download, and gets deleted on the server if i delete them in tb pop3 inbox ( wich is another option which should work equal as delete after x days ) so i am nearly sure there is a problem with virtual plugin or in my using it speculate mail cant be found to delete if it stays in maildir new
speculate next virtual plugin ignore and/or sets another value with
protocol pop3 { # Don't try to set mails non-recent or seen with POP3 sessions. This is # mostly intended to reduce disk I/O. With maildir it doesn't move files # from new/ to cur/, with mbox it doesn't write Status-header. #pop3_no_flag_updates = no
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
On Sun, 2011-03-13 at 12:40 +0100, Robert Schetterer wrote:
RealMails/ RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* unseen .. that the mails stay in "new" folder of the maildir after i downloaded via pop3 ( i am not sure is this expected ? , i would thought they get to cur )
They get moved to cur/ when the \Recent flag gets cleared.
thunderbird does not download this mails again ( good ), but it doesnt delete them from maildir new ever( bad ) , so the whole leave copy and delete after x days function pop3 is not working
What if that mailbox was an IMAP mailbox called "All mails" and user opened it, should that clear the \Recent flags from all the mailboxes? I'm not entirely sure what the right answer should be, but I'm thinking probably no.. Especially since such mailbox is used by some webmails to implement "search in all mailboxes".
So when should the \Recent flags be cleared with virtual mailboxes, and when not? Is it simply "clear with POP3, don't clear with others"? Then again others have wanted to create a virtual INBOX that shows full threads, i.e. messages also from Sent mailbox. That basically acts like an INBOX, so it should clear \Recent flags from INBOX.
Maybe this should be configurable. For example:
+a b +b/*
would keep \Recent flags in mailbox "b", but clear them from "a" and "b/*". Hmmh. Yeah, I guess that's the right solution. See if this works:
Am 16.03.2011 18:18, schrieb Timo Sirainen:
On Sun, 2011-03-13 at 12:40 +0100, Robert Schetterer wrote:
RealMails/ RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* unseen .. that the mails stay in "new" folder of the maildir after i downloaded via pop3 ( i am not sure is this expected ? , i would thought they get to cur )
They get moved to cur/ when the \Recent flag gets cleared.
thunderbird does not download this mails again ( good ), but it doesnt delete them from maildir new ever( bad ) , so the whole leave copy and delete after x days function pop3 is not working
What if that mailbox was an IMAP mailbox called "All mails" and user opened it, should that clear the \Recent flags from all the mailboxes? I'm not entirely sure what the right answer should be, but I'm thinking probably no.. Especially since such mailbox is used by some webmails to implement "search in all mailboxes".
nobody reported imap problems if users use both pop3 ( with stay mails on the server ) and imap mails get to cur wtih seen flag ( as far i know by small tests ) if they connect via imap the problem users mostly use outlook 2010/2007 no webmail etc and my test client is thunderbird ( current stable )
So when should the \Recent flags be cleared with virtual mailboxes, and when not? Is it simply "clear with POP3, don't clear with others"? Then again others have wanted to create a virtual INBOX that shows full threads, i.e. messages also from Sent mailbox. That basically acts like an INBOX, so it should clear \Recent flags from INBOX.
Maybe this should be configurable. For example:
+a b +b/*
would keep \Recent flags in mailbox "b", but clear them from "a" and "b/*". Hmmh. Yeah, I guess that's the right solution. See if this works:
i will give this a try and report
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
On Wed, 2011-03-16 at 18:29 +0100, Robert Schetterer wrote:
if users use both pop3 ( with stay mails on the server ) and imap mails get to cur wtih seen flag ( as far i know by small tests ) if they connect via imap the problem users mostly use outlook 2010/2007 no webmail etc and my test client is thunderbird ( current stable )
Are you talking about the feature of POP3 clients deleting mails after x days (not some server cronjob)? I don't think mail being in new/ or cur/ should affect that, because POP3 doesn't even know about it.
I wonder how POP3 clients would even know how long the mail has been in server..
Am 16.03.2011 18:33, schrieb Timo Sirainen:
On Wed, 2011-03-16 at 18:29 +0100, Robert Schetterer wrote:
if users use both pop3 ( with stay mails on the server ) and imap mails get to cur wtih seen flag ( as far i know by small tests ) if they connect via imap the problem users mostly use outlook 2010/2007 no webmail etc and my test client is thunderbird ( current stable )
Are you talking about the feature of POP3 clients deleting mails after x days (not some server cronjob)? I don't think mail being in new/ or cur/ should affect that, because POP3 doesn't even know about it.
I wonder how POP3 clients would even know how long the mail has been in server..
youre right difficult to say , might ask some developer of mail clients this option exists since ever in most pop3 clients, and was wide used before imap comes more public known
i only know for getmail, getmail is building a list about what allready downloaded via pop3 so it should be independent whatever the server reports, no idea how its done by thunderbird etc, perhaps some stuff with date of mails, or hash comparing...
thunderbirds default on pop3 is leave copy on the server for 14 days also there is an option for delete mails at once , when they are deleted from inbox to trash of the pop3 local account trash
where you have the option to use one local pop3 box for all accounts or use seperate for each account ( some pseudo virtual config )
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Am 16.03.2011 20:56, schrieb Robert Schetterer:
I wonder how POP3 clients would even know how long the mail has been in
server..
here it is for tb
http://kb.mozillazine.org/Popstate.dat
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Am 16.03.2011 22:59, schrieb Robert Schetterer:
Am 16.03.2011 20:56, schrieb Robert Schetterer:
I wonder how POP3 clients would even know how long the mail has been in
server..
here it is for tb
ok , i tested a little the popstate.dat files stores what to do with pop3 mail identified by UIDL
this tb function looks good on both servers
if i.e a mail was deleted from local inbox it gets declared something like d uidl in the popstate.dat
the difference between the servers is that in the server without virtual plugin the mail gets in deed deleted at pop3 reconnect
sending pop3(...@....): expunge: box=INBOX, uid=172, msgid=<4D813935.9070606@....>, size=1836
with my virtual plugin setup it isnt deleted cant see any expunge with this tb account
miracle i can see onther account ( think he has old netscape ) which sends expunge over pop3
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Am 16.03.2011 23:52, schrieb Robert Schetterer:
Am 16.03.2011 22:59, schrieb Robert Schetterer:
Am 16.03.2011 20:56, schrieb Robert Schetterer:
I wonder how POP3 clients would even know how long the mail has been in
server..
here it is for tb
ok , i tested a little the popstate.dat files stores what to do with pop3 mail identified by UIDL
this tb function looks good on both servers
if i.e a mail was deleted from local inbox it gets declared something like d uidl in the popstate.dat
the difference between the servers is that in the server without virtual plugin the mail gets in deed deleted at pop3 reconnect
sending pop3(...@....): expunge: box=INBOX, uid=172, msgid=<4D813935.9070606@....>, size=1836
with my virtual plugin setup it isnt deleted cant see any expunge with this tb account
miracle i can see onther account ( think he has old netscape ) which sends expunge over pop3
checked this with outlook 2010 too it doesnt work either, see no expunge in log
so the virtual plugin or my setup within simply broke this pop3 features
Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Am 17.03.2011 00:07, schrieb Robert Schetterer:
Am 16.03.2011 23:52, schrieb Robert Schetterer:
Am 16.03.2011 22:59, schrieb Robert Schetterer:
Am 16.03.2011 20:56, schrieb Robert Schetterer:
I wonder how POP3 clients would even know how long the mail has been in
server..
here it is for tb
ok , i tested a little the popstate.dat files stores what to do with pop3 mail identified by UIDL
this tb function looks good on both servers
if i.e a mail was deleted from local inbox it gets declared something like d uidl in the popstate.dat
the difference between the servers is that in the server without virtual plugin the mail gets in deed deleted at pop3 reconnect
sending pop3(...@....): expunge: box=INBOX, uid=172, msgid=<4D813935.9070606@....>, size=1836
with my virtual plugin setup it isnt deleted cant see any expunge with this tb account
miracle i can see onther account ( think he has old netscape ) which sends expunge over pop3
checked this with outlook 2010 too it doesnt work either, see no expunge in log
so the virtual plugin or my setup within simply broke this pop3 features hi Timo
i did more debugging, used telnet with pop3 the list and dele function are working,
the problem is based on that if tb downloads the mail ( with option leave copies on the server ) mail stays in maildir cur now, but it cant be seen via pop3 list anymore so its clear, whatever tb sends dele later connect it fails cause the mail cant be seen via pop3 anymore
Am 18.03.2011 23:39, schrieb Robert Schetterer:
Am 17.03.2011 00:07, schrieb Robert Schetterer:
Am 16.03.2011 23:52, schrieb Robert Schetterer:
Am 16.03.2011 22:59, schrieb Robert Schetterer:
Am 16.03.2011 20:56, schrieb Robert Schetterer:
I wonder how POP3 clients would even know how long the mail has been in
server..
here it is for tb
ok , i tested a little the popstate.dat files stores what to do with pop3 mail identified by UIDL
this tb function looks good on both servers
if i.e a mail was deleted from local inbox it gets declared something like d uidl in the popstate.dat
the difference between the servers is that in the server without virtual plugin the mail gets in deed deleted at pop3 reconnect
sending pop3(...@....): expunge: box=INBOX, uid=172, msgid=<4D813935.9070606@....>, size=1836
with my virtual plugin setup it isnt deleted cant see any expunge with this tb account
miracle i can see onther account ( think he has old netscape ) which sends expunge over pop3
checked this with outlook 2010 too it doesnt work either, see no expunge in log
so the virtual plugin or my setup within simply broke this pop3 features hi Timo
i did more debugging, used telnet with pop3 the list and dele function are working,
the problem is based on that if tb downloads the mail ( with option leave copies on the server ) mail stays in maildir cur now, but it cant be seen via pop3 list anymore so its clear, whatever tb sends dele later connect it fails cause the mail cant be seen via pop3 anymore
for very small tests this looks like solved i have to wait for wide user reaction/test to be sure to see mails via pop3 and leave copies on the server
this cant work
+RealMails/ RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* unseen
it must be
+RealMails/ RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* all
which is example in the wiki too, but not related special to this pop3 leave copies on the server feature
so timos patch worked , also it was a configration/mythinking bug dont know if the patch should be reverted , dont think so cause it looks better to have mails in maildir cur when the got downloaded then in new as it was before
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
On Sun, 2011-03-20 at 12:27 +0100, Robert Schetterer wrote:
this cant work
+RealMails/ RealMails/* .. unseen
Oh, I hadn't noticed this before. Yes, "unseen" doesn't work nicely with POP3.
so timos patch worked , also it was a configration/mythinking bug dont know if the patch should be reverted , dont think so cause it looks better to have mails in maildir cur when the got downloaded then in new as it was before
For POP3 it doesn't really matter, because it's not visible to clients in any way. You would get a bit better performance if you didn't include the '+' because then it wouldn't rename the mails to cur/ (i.e. same idea as pop3_no_flag_updates setting).
The '+' feature is more useful with IMAP though, because there it has visible effects to clients. So in any case it wasn't a waste to implement it.
BTW. If you set "+RealMails/" it moves mails from new/ to cur/ in INBOX, but you also need to set "+RealMails/*" if you want that to happen for non-INBOXes too.
Am 21.03.2011 17:22, schrieb Timo Sirainen:
On Sun, 2011-03-20 at 12:27 +0100, Robert Schetterer wrote:
this cant work
+RealMails/ RealMails/* .. unseen
Oh, I hadn't noticed this before. Yes, "unseen" doesn't work nicely with POP3.
should be noted in the wiki *g
so timos patch worked , also it was a configration/mythinking bug dont know if the patch should be reverted , dont think so cause it looks better to have mails in maildir cur when the got downloaded then in new as it was before
For POP3 it doesn't really matter, because it's not visible to clients in any way. You would get a bit better performance if you didn't include the '+' because then it wouldn't rename the mails to cur/ (i.e. same idea as pop3_no_flag_updates setting).
The '+' feature is more useful with IMAP though, because there it has visible effects to clients. So in any case it wasn't a waste to implement it.
ok
BTW. If you set "+RealMails/" it moves mails from new/ to cur/ in INBOX, but you also need to set "+RealMails/*" if you want that to happen for non-INBOXes too.
yes did this now too, looks fine for small tests
much thx !
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Am 16.03.2011 18:18, schrieb Timo Sirainen:
So when should the \Recent flags be cleared with virtual mailboxes, and when not? Is it simply "clear with POP3, don't clear with others"? Then again others have wanted to create a virtual INBOX that shows full threads, i.e. messages also from Sent mailbox. That basically acts like an INBOX, so it should clear \Recent flags from INBOX.
Maybe this should be configurable. For example:
+a b +b/*
would keep \Recent flags in mailbox "b", but clear them from "a" and "b/*". Hmmh. Yeah, I guess that's the right solution. See if this works:
used your patch, and tested but i am not sure if i did the right
with
+RealMails/ RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* unseen
good : mails are now going to maildir cur after pop3 download good : they are not going downloaded twice
bad: as configured in thunderbird and working in a setup without virtual plugin, it should be deleted from maildir cur ( server ) at deletion from tb local inbox at pop3 reconnect
tbs shortest option for delete after x days is one day, cant wait this, but i expect this working if option delete from server if deleted from local inbox ( after reconnect ) works
did i missconfig something?
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
participants (2)
-
Robert Schetterer
-
Timo Sirainen