[Dovecot] qpopper UIDLs and migrating to dovecot/maildir
hi all,
I am hoping to convert a 350K user qpopper installation using mbox to dovecot and maildir (pop3 only).
many of the users "leave mail on server", and qpopper's UIDL format isn't (yet) supported by dovecot. since the UIDLs have been written by qpoppper to the existing mbox files (and will be preserved when the mboxes are converted to maildir) it would be possible to do this seamlessly if dovecot could read the UIDLs from the messages and populate the index files as needed.
is anything like this already possible? it doesn't matter what UIDLs new messages get, since they would never have been downloaded by the user via qpopper.
any ideas would be appreciated.
grant.
On Thu, Sep 01, 2005 at 10:23:28AM +1000, grant beattie wrote:
is anything like this already possible? it doesn't matter what UIDLs new messages get, since they would never have been downloaded by the user via qpopper.
one possibility is to create the dovecot indexes at mbox -> maildir conversion time, preserving the UIDLs from the mbox file.
perhaps someone has done this before?
grant.
Hmm, interesting idea. I hadn't noticed that qpopper sticks in X-UIDL headers (as I never use POP myself).
I might have a go at coding it in Dovecot-1.0-stable and alpha1 (if Timo doesn't beat me to it and do it properly :) )
We're migrating from qpopper/uw-imap to Dovecot-1.0-stable, probably on the 14th of September and I was resigned to qpopper users getting all their left-on-server mail again (I did have a good look at emulating qpopper UIDLs, but failed).
grant beattie wrote:
On Thu, Sep 01, 2005 at 10:23:28AM +1000, grant beattie wrote:
is anything like this already possible? it doesn't matter what UIDLs new messages get, since they would never have been downloaded by the user via qpopper.
one possibility is to create the dovecot indexes at mbox -> maildir conversion time, preserving the UIDLs from the mbox file.
No need, I think. Dovecot will create indexes as it needs them. I think just have an option "preserve_x_uidl" or some such and then use the specified format ("%u.%v" etc.) for new UIDLs.
perhaps someone has done this before?
grant.
Best Wishes, Chris
-- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
Right! Here's a patch against dovecot-1.0.alpha1 which adds an extra option: pop3_reuse_xuidl = yes which if set will use the contents of the (first) X-UIDL header, if it exists, instead of the result of expanding pop3_uidl_format. It will use pop3_uidl_format if there is no X-UIDL header. This is typically for people migrating from Qpopper who don't want their users to get all the mail they've left on the server a second time, but should also work for any other POP server that happens to write X-UIDL headers, if any such exist ;) I've only done rudimentary testing, but it seems to work! I also have a version of the patch for dovecot-1.0-stable (which is what we'll be using next week!) if anybody's interested (only minor differences, use 'mail->get_header(mail, "X-UIDL")' instead of 'mail_get_first_header(ctx->mail, "X-UIDL")') Best Wishes, Chris Chris Wakelin wrote:
Hmm, interesting idea. I hadn't noticed that qpopper sticks in X-UIDL headers (as I never use POP myself).
I might have a go at coding it in Dovecot-1.0-stable and alpha1 (if Timo doesn't beat me to it and do it properly :) )
We're migrating from qpopper/uw-imap to Dovecot-1.0-stable, probably on the 14th of September and I was resigned to qpopper users getting all their left-on-server mail again (I did have a good look at emulating qpopper UIDLs, but failed).
grant beattie wrote:
On Thu, Sep 01, 2005 at 10:23:28AM +1000, grant beattie wrote:
is anything like this already possible? it doesn't matter what UIDLs new messages get, since they would never have been downloaded by the user via qpopper.
one possibility is to create the dovecot indexes at mbox -> maildir conversion time, preserving the UIDLs from the mbox file.
No need, I think. Dovecot will create indexes as it needs them. I think just have an option "preserve_x_uidl" or some such and then use the specified format ("%u.%v" etc.) for new UIDLs.
perhaps someone has done this before?
grant.
Best Wishes, Chris
-- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094 diff -ruN dovecot-1.0.alpha1.orig/src/master/mail-process.c dovecot-1.0.alpha1/src/master/mail-process.c --- dovecot-1.0.alpha1.orig/src/master/mail-process.c Mon Jul 4 12:30:08 2005 +++ dovecot-1.0.alpha1/src/master/mail-process.c Sun Sep 4 00:14:48 2005 @@ -227,6 +227,8 @@ env_put("FULL_FILESYSTEM_ACCESS=1"); if (set->pop3_no_flag_updates) env_put("POP3_NO_FLAG_UPDATES=1"); + if (set->pop3_reuse_xuidl) + env_put("POP3_REUSE_XUIDL=1"); if (set->pop3_enable_last) env_put("POP3_ENABLE_LAST=1"); if (set->mbox_dirty_syncs) diff -ruN dovecot-1.0.alpha1.orig/src/master/master-settings.c dovecot-1.0.alpha1/src/master/master-settings.c --- dovecot-1.0.alpha1.orig/src/master/master-settings.c Sun Aug 7 13:50:32 2005 +++ dovecot-1.0.alpha1/src/master/master-settings.c Sun Sep 4 00:14:48 2005 @@ -131,6 +131,7 @@ /* pop3 */ DEF(SET_BOOL, pop3_no_flag_updates), DEF(SET_BOOL, pop3_enable_last), + DEF(SET_BOOL, pop3_reuse_xuidl), DEF(SET_STR, pop3_uidl_format), DEF(SET_STR, pop3_client_workarounds), DEF(SET_STR, pop3_logout_format), @@ -327,6 +328,7 @@ /* pop3 */ MEMBER(pop3_no_flag_updates) FALSE, MEMBER(pop3_enable_last) FALSE, + MEMBER(pop3_reuse_xuidl) FALSE, MEMBER(pop3_uidl_format) "%v.%u", MEMBER(pop3_client_workarounds) NULL, MEMBER(pop3_logout_format) "top=%t/%T, retr=%r/%R, del=%d/%m, size=%s", diff -ruN dovecot-1.0.alpha1.orig/src/master/master-settings.h dovecot-1.0.alpha1/src/master/master-settings.h --- dovecot-1.0.alpha1.orig/src/master/master-settings.h Sat May 14 20:58:29 2005 +++ dovecot-1.0.alpha1/src/master/master-settings.h Sun Sep 4 00:14:48 2005 @@ -100,6 +100,7 @@ /* pop3 */ int pop3_no_flag_updates; int pop3_enable_last; + int pop3_reuse_xuidl; const char *pop3_uidl_format; const char *pop3_client_workarounds; const char *pop3_logout_format; diff -ruN dovecot-1.0.alpha1.orig/src/pop3/commands.c dovecot-1.0.alpha1/src/pop3/commands.c --- dovecot-1.0.alpha1.orig/src/pop3/commands.c Tue Jun 28 21:58:17 2005 +++ dovecot-1.0.alpha1/src/pop3/commands.c Sun Sep 4 00:15:27 2005 @@ -504,6 +504,7 @@ }; struct var_expand_table *tab; string_t *str; + const char *xuidl; int ret, found = FALSE; tab = t_malloc(sizeof(static_tab)); @@ -548,7 +549,15 @@ str_truncate(str, 0); str_printfa(str, ctx->message == 0 ? "%u " : "+OK %u ", ctx->mail->seq); - var_expand(str, uidl_format, tab); + if (reuse_xuidl) { + xuidl=mail_get_first_header(ctx->mail, "X-UIDL"); + if (xuidl != NULL) + str_append(str, xuidl); + else + var_expand(str, uidl_format, tab); + } + else + var_expand(str, uidl_format, tab); ret = client_send_line(client, "%s", str_c(str)); t_pop(); diff -ruN dovecot-1.0.alpha1.orig/src/pop3/common.h dovecot-1.0.alpha1/src/pop3/common.h --- dovecot-1.0.alpha1.orig/src/pop3/common.h Sat May 14 20:34:32 2005 +++ dovecot-1.0.alpha1/src/pop3/common.h Sun Sep 4 00:14:48 2005 @@ -18,7 +18,7 @@ extern struct ioloop *ioloop; extern enum client_workarounds client_workarounds; -extern int enable_last_command, no_flag_updates; +extern int enable_last_command, no_flag_updates, reuse_xuidl; extern const char *uidl_format, *logout_format; extern enum uidl_keys uidl_keymask; diff -ruN dovecot-1.0.alpha1.orig/src/pop3/main.c dovecot-1.0.alpha1/src/pop3/main.c --- dovecot-1.0.alpha1.orig/src/pop3/main.c Sat May 14 21:16:41 2005 +++ dovecot-1.0.alpha1/src/pop3/main.c Sun Sep 4 00:14:48 2005 @@ -41,6 +41,7 @@ enum client_workarounds client_workarounds = 0; int enable_last_command = FALSE; int no_flag_updates = FALSE; +int reuse_xuidl = FALSE; const char *uidl_format, *logout_format; enum uidl_keys uidl_keymask; @@ -174,6 +175,7 @@ parse_workarounds(); enable_last_command = getenv("POP3_ENABLE_LAST") != NULL; no_flag_updates = getenv("POP3_NO_FLAG_UPDATES") != NULL; + reuse_xuidl = getenv("POP3_REUSE_XUIDL") != NULL; uidl_format = getenv("POP3_UIDL_FORMAT"); if (uidl_format == NULL)
participants (2)
-
Chris Wakelin
-
grant beattie