[Dovecot] Duplicated (but only for unread) messages after second rsync pass...
Ok, I know I should probably be using dsync for this, but I'm more familiar with rsync, and didn't think this would be a problem.
Here is what I am doing, and what is apparently happening (only done this once, but I'm guessing it will happen each time):
rsync -avHP /mnt/vmail/example.com/user1/ /var/vmail/example.com/user1/
chown -R vmail:vmail /var/vmail/example.com
Ok, so far so good, everything works fine accessing the mails (I'm using Thunderbird).
Now I do another rsync, but this time I did the entire mailstore:
rsync -avHP /mnt/vmail/example.com/ /var/vmail/example.com/
chown -R vmail:vmail /var/vmail/example.com
Right after I started up dovecot again, I hot a bunch of these in the logs:
2013-12-22T09:52:51-05:00 newhost dovecot: imap(user1@example.com): Warning: Maildir /var/vmail/example.com/user1/: Expunged message reappeared, giving a new UID (old uid=45516, file=1387468268.Vfe02Ic1343fM845754.oldhost.sub.example.com:2,ST) 2013-12-22T09:52:51-05:00 newhost dovecot: imap(user1@example.com): Warning: Maildir /var/vmail/example.com/user1/: Expunged message reappeared, giving a new UID (old uid=45518, file=1387468645.Vfe02Ic73e0eM530539.oldhost.sub.example.com:2,ST)
The problem is, it looks like every single message that was flagged as unread is now duplicated on the new system.
Also - not that this is a problem, but all of the old messages have the fqdn of the old server in the individual email file names - oldhost.sub.example.com - while the new ones only have the local hostname (newhost)
Did I do something wrong? Or is this going to be an unavoidable problem if I want to use rsync to do the final migration that using dsync would avoid?
Thanks,
--
Best regards,
*/Charles /*
On 22.12.2013, at 17.13, Charles Marcus <CMarcus@Media-Brokers.com> wrote:
Ok, I know I should probably be using dsync for this, but I'm more familiar with rsync, and didn't think this would be a problem.
Here is what I am doing, and what is apparently happening (only done this once, but I'm guessing it will happen each time):
rsync -avHP /mnt/vmail/example.com/user1/ /var/vmail/example.com/user1/
chown -R vmail:vmail /var/vmail/example.com
Ok, so far so good, everything works fine accessing the mails (I'm using Thunderbird).
Now I do another rsync, but this time I did the entire mailstore:
rsync -avHP /mnt/vmail/example.com/ /var/vmail/example.com/
chown -R vmail:vmail /var/vmail/example.com
Right after I started up dovecot again, I hot a bunch of these in the logs:
2013-12-22T09:52:51-05:00 newhost dovecot: imap(user1@example.com): Warning: Maildir /var/vmail/example.com/user1/: Expunged message reappeared, giving a new UID (old uid=45516, file=1387468268.Vfe02Ic1343fM845754.oldhost.sub.example.com:2,ST) 2013-12-22T09:52:51-05:00 newhost dovecot: imap(user1@example.com): Warning: Maildir /var/vmail/example.com/user1/: Expunged message reappeared, giving a new UID (old uid=45518, file=1387468645.Vfe02Ic73e0eM530539.oldhost.sub.example.com:2,ST)
The problem is, it looks like every single message that was flagged as unread is now duplicated on the new system.
Also - not that this is a problem, but all of the old messages have the fqdn of the old server in the individual email file names - oldhost.sub.example.com - while the new ones only have the local hostname (newhost)
The hostname doesn't matter.
Did I do something wrong? Or is this going to be an unavoidable problem if I want to use rsync to do the final migration that using dsync would avoid?
Use rsync --delete to get rid of the extra mails in destination. And of course don't deliver any new mails to destination before that is done. :)
On 2013-12-22 10:37 AM, Timo Sirainen <tss@iki.fi> wrote:
Use rsync --delete to get rid of the extra mails in destination.
I wondered if that would do it... thanks.
And of course don't deliver any new mails to destination before that is done.:)
Well, at least I knew not to do that... ;)
Thanks Timo.
--
Best regards,
*/Charles/*
Am 22.12.2013 16:44, schrieb Charles Marcus:
On 2013-12-22 10:37 AM, Timo Sirainen <tss@iki.fi> wrote:
Use rsync --delete to get rid of the extra mails in destination.
I wondered if that would do it... thanks
in case of a 1:1 rsync you always want the parameters below to include any sort of links, permissions and attributes
[harry@srv-rhsoft:~]$ which rsync.sh /usr/local/bin/rsync.sh
[harry@srv-rhsoft:~]$ cat /usr/local/bin/rsync.sh
#!/bin/bash
# -z compress
# -t timestamps
# -P progress
# -r recursive
# -l links
# -H hard-links
# -p permissions
# -o owner
# -g group
# -E executability
# -A acls
# -X xtended attributes
# Sicherstellen dass Source UND Target uebergeben wurden
if [ "$1" == "" ] || [ "$2" == "" ] || [ "$1" == "$2" ]; then
echo "USAGE: rsync.sh <source> <target> [bwlimit]"
exit
fi
# Standard-Parameter
RSYNC_PARAMS="--no-motd --force --delete-after --devices --specials --sparse -tPrlHpogEAX"
# Wenn in einem der beiden Paramneter ein @ vorkommt Komprimierung einschalten
# Ansonsten handelt es sich um zwei lokale Ordner und rsync wuerde die
# Daten ohne Sinn komprimieren
if [ grep '@' <<< "$1" ] || [ grep '@' <<< "$2" ]; then
RSYNC_PARAMS="--compress --sockopts=SO_SNDBUF=32768,SO_RCVBUF=32768 $RSYNC_PARAMS"
fi
if [ "$3" != "" ]; then
RSYNC_PARAMS="--bwlimit=$3 $RSYNC_PARAMS"
fi
# Eigentliches Kommando ausfuehren
nice -n 19 rsync $RSYNC_PARAMS --rsync-path='nice -n 19 rsync' "$1" "$2"
On 2013-12-22 10:37 AM, Timo Sirainen <tss@iki.fi> wrote:
Use rsync --delete to get rid of the extra mails in destination. And of course don't deliver any new mails to destination before that is done. :)
Thanks Timo - but I'm curious why this only affected UNREAD messages? Even ones that were not new since the first rsync (some were over a month old, but still marked as unread, and those were duplicated too).
--
Best regards,
*/Charles/*
participants (3)
-
Charles Marcus
-
Reindl Harald
-
Timo Sirainen