[Dovecot] Problems using GFS2 and clustered dovecot
I am trying to use dovecot. I've got a GFS2 shared volume on two servers with dovecot running on both. On one server at a time, it works.
The test I am trying is to attach two mail programs (MUA) via IMAPS (Thunderbird and Evolution as it happens). I've attached one mail program to each IMAPS server. I am trying to move emails around in one program (from folder to folder), and then checking it on the other program. I'm testing in effect how it would work if my users log on twice and end up on different servers.
I seem to suffer from never ending index file corruption:
Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Corrupted transaction log file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex.log: end_offset (284) > current sync_offset (104) Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): broken sync positions in index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): fscking index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Fixed index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex: log file sync pos 1,284 -> 1, 104 Apr 4 16:55:50 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Unexpected transaction log desync with index /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:56:18 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Transaction log file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex.log: marked corrupted
This happens pretty quickly. I can also get message duplication- if I move a message from one server to another, I can get a message in the original folder and the destination folder.
I am using RC28. I have tried using fcntl locking and dotlocking with equal lack of success.
Can anyone offer any advice?
David
Griffin Information Systems Ltd 20 St Christopher's Way, Pride Park, Derby DE24 8JY www.griffin.com http://www.griffin.com/
Registered in England and Wales. Company Registration Number: 3676297. VAT Number: 716604053
On 095, 04 05, 2007 at 09:58:44AM +0100, David Craigon wrote:
I am trying to use dovecot. I've got a GFS2 shared volume on two servers with dovecot running on both. On one server at a time, it works.
The test I am trying is to attach two mail programs (MUA) via IMAPS (Thunderbird and Evolution as it happens). I've attached one mail program to each IMAPS server. I am trying to move emails around in one program (from folder to folder), and then checking it on the other program. I'm testing in effect how it would work if my users log on twice and end up on different servers.
I seem to suffer from never ending index file corruption:
Where are index files located ?
Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Corrupted transaction log file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex.log: end_offset (284) > current sync_offset (104) Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): broken sync positions in index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): fscking index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Fixed index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex: log file sync pos 1,284 -> 1, 104 Apr 4 16:55:50 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Unexpected transaction log desync with index /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:56:18 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Transaction log file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex.log: marked corrupted
This happens pretty quickly. I can also get message duplication- if I move a message from one server to another, I can get a message in the original folder and the destination folder.
I am using RC28. I have tried using fcntl locking and dotlocking with equal lack of success.
Can anyone offer any advice?
David
Griffin Information Systems Ltd 20 St Christopher's Way, Pride Park, Derby DE24 8JY www.griffin.com http://www.griffin.com/
Registered in England and Wales. Company Registration Number: 3676297. VAT Number: 716604053
-- Andrey Panin | Linux and UNIX system administrator pazke@donpac.ru | PGP key: wwwkeys.pgp.net
David Craigon wrote:
I am trying to use dovecot. I've got a GFS2 shared volume on two servers with dovecot running on both. On one server at a time, it works.
Sorry to not answer your question directly, but do you have a good reason to be using GFS instead of NFS? We experimented with GFS last year (granted, not GFS2), and found its performance pretty horrible and its stability comically bad... had we been paying for it, however, it wouldn't have been funny. OCFS2 was better, but still unreliable. Lustre was the best we found (it worked, at least), but it required a lot of hardware/admin overhead. We tried other clustering filesystems as well, and none of them was worth the trouble.
The test I am trying is to attach two mail programs (MUA) via IMAPS (Thunderbird and Evolution as it happens). I've attached one mail program to each IMAPS server. I am trying to move emails around in one program (from folder to folder), and then checking it on the other program. I'm testing in effect how it would work if my users log on twice and end up on different servers.
I seem to suffer from never ending index file corruption:
Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Corrupted transaction log file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex.log: end_offset (284) > current sync_offset (104) Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): broken sync positions in index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): fscking index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:55:49 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Fixed index file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex: log file sync pos 1,284 -> 1, 104 Apr 4 16:55:50 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Unexpected transaction log desync with index /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex Apr 4 16:56:18 mailstore-b dovecot: IMAP(david@everybody-loves.craigon.co.uk): Transaction log file /mnt/mailstore/e/everybody-loves.craigon.co.uk/d/david/.fish/dovecot.ind ex.log: marked corrupted
This happens pretty quickly. I can also get message duplication- if I move a message from one server to another, I can get a message in the original folder and the destination folder.
I am using RC28. I have tried using fcntl locking and dotlocking with equal lack of success.
Can anyone offer any advice?
David
Griffin Information Systems Ltd 20 St Christopher's Way, Pride Park, Derby DE24 8JY www.griffin.com http://www.griffin.com/
Registered in England and Wales. Company Registration Number: 3676297. VAT Number: 716604053
On 5.4.2007, at 11.58, David Craigon wrote:
I am trying to use dovecot. I've got a GFS2 shared volume on two
servers with dovecot running on both. On one server at a time, it works.
I guess GFS2 has some caching problems then..
end_offset (284) > current sync_offset (104) Apr 4 16:55:49
mailstore-b dovecot:
This means that dovecot.index file said that transaction log is
synced up to offset 284, but when Dovecot was trying read the
dovecot.index.log file, it found data only up to offset 104. So
either GFS2 said that the file ended at offset 104, or it decided to
return old data from cache. I guess it's the latter, because the log
file reads are done without locking.
Probably the easiest way is to just put the index files into local disk.
participants (5)
-
Andrey Panin
-
David Craigon
-
Justin McAleer
-
Timo Sirainen
-
Tomi Hakala