Help with migration to Dovecot from raw files

[Stefano Antonelli]

On 2020-03-28 23:54, Stefano Antonelli wrote:
> I've attached my script in case it's helpful to anyone in a similar
> situation.  It took quite a bit of effort to get this right, highly
> depends on the format of the original mailstore, and I've only tested
> this on one smaller mailbox (<400 messages).  And I wasn't concerned
> with any client having to download everything again since this was a
> fresh import.
> 
> In any case, it worked for me.

Err had a few issues on a larger 4gb mailbox:

1. small bug in the script, new script attached.  Change:
	# Grab the flags from the filename.
	F=`echo $fname | cut -c 26-27`
to:
	# Grab the flags from the filename.
	F=0x`echo $fname | cut -c 26-27`

   without the 0x in front, the masking operation interprets $F as octal.

2. there are cases where the old mail server wrote files with identical 
timestamps even though the dates of the email are clearly different.  I 
presume this was just buggy behaviour and could explain why I couldn't 
successfully download via imap.

201011072323050332-000001060000028C3D2E9D97.imap
1289200985.M000000P12345.icewarp,S=5614,W=5614:2,RS
Received:  Tue, 14 Sep 2010 19:44:47 -0700

201011072323050330-000001060000028A3D2C9676.imap
1289200985.M000000P12345.icewarp,S=2317,W=2317:2,RS
Received:  Sun, 12 Sep 2010 18:51:44 -0700

I may consider fixing this.

3. for some reason there were some off by one file sizes that resulted 
in errors like this:
dovecot.index.cache: UID 278: Broken physical size in mailbox ...
failed: Cached message size smaller than expected (1972 < 1973 
,box=INBOX, UID=278)

In my case 298 out of 22023 files were affected.  Script to fix this is 
after using the import-from-icewarp-files.sh script is attached.  I 
couldn't find a good way to find only maildir messages by filename.  
Seems difficult to parse.  When I renamed the files, I used icewarp as 
the server name and that's how I searched for files:

FILES=`find /path/to/mail/user -name \*\.icewarp\,\*`

If there's a better way to do this, I would like to know for future 
reference.

Thanks,
Stef


-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-file-size.sh
Type: text/x-shellscript
Size: 980 bytes
Desc: not available
URL: <https://dovecot.org/pipermail/dovecot/attachments/20200329/74496a75/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: import-from-icewarp-files.sh
Type: text/x-shellscript
Size: 4856 bytes
Desc: not available
URL: <https://dovecot.org/pipermail/dovecot/attachments/20200329/74496a75/attachment-0001.bin>