[Dovecot] UID order vs date order
Hi
i have recently moved a mailbox to dovecot and intend to use it via imap and i have run into a problem.
when i imported the mails apparently the UID order got reversed compared to the date order, result is that some clients like on android devices download the oldest mails instead of newest. it is not so interesting to have several years old mail show up at the top of the mailbox.
is there any way of convincing dovecot to renumber the mails by date order instead? i have tried to delete the dovecot-uidlist and dovecont.index* files in an unimportant folder and the only thing that happens was that all of the mails got renumbered in the wrong order again (lower numbers but still same problem)
any advice? i cant be the only one that have run into this.
Am 17.11.2013 11:20, schrieb Torbjorn Jansson:
when i imported the mails
how did you did this ? from what server to what version of dove, what mailbox format
meanwhile read i.e for possible solutions
http://wiki2.dovecot.org/Tools/Doveadm/Mailbox
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 2013-11-17 11:45, Robert Schetterer wrote:
Am 17.11.2013 11:20, schrieb Torbjorn Jansson:
when i imported the mails
how did you did this ? from what server to what version of dove, what mailbox format
i used thunderbirds import function to import mail from outlook then copy over the folders to the imap server with thunderbird. (tbird imports to "Local Folders" and not directly to imap)
dovecot is version 2.2.6 and dovecot uses maildir format for the mailbox.
Am 17.11.2013 12:06, schrieb Torbjorn Jansson:
(tbird imports to "Local Folders" and not directly to imap)
beside other dovecot solving ,this perhaps is the source of the problem, you have better choosen i.e imapsync or dsync on the server , is there a chance to retry your procedure with imapsync , which might be the fastest solution.
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 2013-11-17 15:17, Robert Schetterer wrote:
Am 17.11.2013 12:06, schrieb Torbjorn Jansson:
(tbird imports to "Local Folders" and not directly to imap)
beside other dovecot solving ,this perhaps is the source of the problem, you have better choosen i.e imapsync or dsync on the server , is there a chance to retry your procedure with imapsync , which might be the fastest solution.
the problem is that the source mailbox was a local pst file. i suspect both of those tools are for converting between existing imap accounts and/or mbox/maildir format and not pst file.
with a bit of trouble i can probably redo the import but i still don't know what i could have done differently to affect the sort order of the UID numbers.
only thing i can think of now is to take the dovecot-uidlist in each folder and then renumber them by taking the first "column" and reverse it (first value and last value gets swapped and so on) probably not the right way but what other choice do i have?
Am 17.11.2013 16:02, schrieb Torbjorn Jansson:
the problem is that the source mailbox was a local pst file.
have you tried open mailbox with imap in outlook, additional open pst file and simply copy from pst to imap, i see no reason why you have to use thunderbird for that job, however that might not solve your problem , but one source of bug chance is gone then
for server side solution i only see some doveadm stuff
http://wiki2.dovecot.org/Tools/Doveadm
but i am too short in time, for now ,to think about more or exact advice, perhaps wait for other advice from the list
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 17 Nov 2013, at 03:20 , Torbjorn Jansson <torbjorn.jansson@mbox200.swipnet.se> wrote:
Hi
i have recently moved a mailbox to dovecot and intend to use it via imap and i have run into a problem.
when i imported the mails apparently the UID order got reversed compared to the date order, result is that some clients like on android devices download the oldest mails instead of newest. it is not so interesting to have several years old mail show up at the top of the mailbox.
is there any way of convincing dovecot to renumber the mails by date order instead?
Probably the easiest solution is to figure out what you did in the import to cause this and reimport the mails. Short of that, the only thing you can do (assuming you are using Maildir, right?) is to get the date out of the mail and rewrite the filename with the correct epoch timestamp.
It would be great if there was a way to use doveadm to do this, but short of that you just have to fix it yourself. The good news is, with gnu date it’s pretty simple.
$ date --date='2013-11-17 01:23:45 +0700' +%s 1384626225 $ gdate --date='November 17, 2013 01:23:45 +0700' +%s 1384626225 $ gdate --date='17 Nov 2013 01:23:45 +0700' +%s 1384626225
i cant be the only one that have run into this.
I had a similar problem, but I ignored it and as new mail came in, the problem went away (mostly) on its own. :/
-- A cubicle is simply a cell without a door.
Am 17.11.2013 19:27, schrieb LuKreme:
On 17 Nov 2013, at 03:20 , Torbjorn Jansson <torbjorn.jansson@mbox200.swipnet.se> wrote:
Hi
i have recently moved a mailbox to dovecot and intend to use it via imap and i have run into a problem.
when i imported the mails apparently the UID order got reversed compared to the date order, result is that some clients like on android devices download the oldest mails instead of newest. it is not so interesting to have several years old mail show up at the top of the mailbox.
is there any way of convincing dovecot to renumber the mails by date order instead?
Probably the easiest solution is to figure out what you did in the import to cause this and reimport the mails. Short of that, the only thing you can do (assuming you are using Maildir, right?) is to get the date out of the mail and rewrite the filename with the correct epoch timestamp.
It would be great if there was a way to use doveadm to do this, but short of that you just have to fix it yourself. The good news is, with gnu date it’s pretty simple.
$ date --date='2013-11-17 01:23:45 +0700' +%s 1384626225 $ gdate --date='November 17, 2013 01:23:45 +0700' +%s 1384626225 $ gdate --date='17 Nov 2013 01:23:45 +0700' +%s 1384626225
i cant be the only one that have run into this.
I had a similar problem, but I ignored it and as new mail came in, the problem went away (mostly) on its own. :/
http://www.five-ten-sg.com/libpst/ might be an idea too
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 2013-11-17 19:27, LuKreme wrote:
On 17 Nov 2013, at 03:20 , Torbjorn Jansson <torbjorn.jansson@mbox200.swipnet.se> wrote:
Hi
i have recently moved a mailbox to dovecot and intend to use it via imap and i have run into a problem.
when i imported the mails apparently the UID order got reversed compared to the date order, result is that some clients like on android devices download the oldest mails instead of newest. it is not so interesting to have several years old mail show up at the top of the mailbox.
is there any way of convincing dovecot to renumber the mails by date order instead?
Probably the easiest solution is to figure out what you did in the import to cause this and reimport the mails. Short of that, the only thing you can do (assuming you are using Maildir, right?) is to get the date out of the mail and rewrite the filename with the correct epoch timestamp.
It would be great if there was a way to use doveadm to do this, but short of that you just have to fix it yourself. The good news is, with gnu date it’s pretty simple.
$ date --date='2013-11-17 01:23:45 +0700' +%s 1384626225 $ gdate --date='November 17, 2013 01:23:45 +0700' +%s 1384626225 $ gdate --date='17 Nov 2013 01:23:45 +0700' +%s 1384626225
i cant be the only one that have run into this.
I had a similar problem, but I ignored it and as new mail came in, the problem went away (mostly) on its own. :/
i have fixed the problem. initially during my testing i did try the outlook import to imap folder and it worked but for some reason it aborted half way thru, probably due to some badly formed mail from many years ago so i chose not to use outlook for the conversion.
but i had to use outlook anyway and what i did was to copy one folder at a time via outlook from the local pst to imap folder, some folders i lost a few mails but not many. end result is that i have a good mailbox now.
to sum it up:
Tbird import works, no problem with character sets but screws up the order of the mails.
Outlook import works but more likely to stop on bad mails, some character set issues (a few mails ended up with ??? in the sender instead of the right name) and sort order is correct.
i did run into another issue and that was with android email client, it decided to use up all 10 simultaneous connections and then displayed an error. had to increase the connection limit several times and at one point the client used 65 connections at the same time. compare this with tbird that uses only 5. but i believe this is a known issue.
Am 20.11.2013 07:56, schrieb Torbjorn Jansson:
On 2013-11-17 19:27, LuKreme wrote:
On 17 Nov 2013, at 03:20 , Torbjorn Jansson <torbjorn.jansson@mbox200.swipnet.se> wrote:
Hi
i have recently moved a mailbox to dovecot and intend to use it via imap and i have run into a problem.
when i imported the mails apparently the UID order got reversed compared to the date order, result is that some clients like on android devices download the oldest mails instead of newest. it is not so interesting to have several years old mail show up at the top of the mailbox.
is there any way of convincing dovecot to renumber the mails by date order instead?
Probably the easiest solution is to figure out what you did in the import to cause this and reimport the mails. Short of that, the only thing you can do (assuming you are using Maildir, right?) is to get the date out of the mail and rewrite the filename with the correct epoch timestamp.
It would be great if there was a way to use doveadm to do this, but short of that you just have to fix it yourself. The good news is, with gnu date it’s pretty simple.
$ date --date='2013-11-17 01:23:45 +0700' +%s 1384626225 $ gdate --date='November 17, 2013 01:23:45 +0700' +%s 1384626225 $ gdate --date='17 Nov 2013 01:23:45 +0700' +%s 1384626225
i cant be the only one that have run into this.
I had a similar problem, but I ignored it and as new mail came in, the problem went away (mostly) on its own. :/
i have fixed the problem. initially during my testing i did try the outlook import to imap folder and it worked but for some reason it aborted half way thru, probably due to some badly formed mail from many years ago so i chose not to use outlook for the conversion.
but i had to use outlook anyway and what i did was to copy one folder at a time via outlook from the local pst to imap folder, some folders i lost a few mails but not many. end result is that i have a good mailbox now.
to sum it up:
Tbird import works, no problem with character sets but screws up the order of the mails.
Outlook import works but more likely to stop on bad mails, some character set issues (a few mails ended up with ??? in the sender instead of the right name) and sort order is correct.
i did run into another issue and that was with android email client, it decided to use up all 10 simultaneous connections and then displayed an error. had to increase the connection limit several times and at one point the client used 65 connections at the same time. compare this with tbird that uses only 5. but i believe this is a known issue.
thx for info, so it looks like many client problems here , but none by dovecot, anyway good to know about cause migration/restore from pst file might happen in many places
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
participants (3)
-
LuKreme
-
Robert Schetterer
-
Torbjorn Jansson