Timo Sirainen writes:
Why not go with a pure log replication scheme? this way you basically have 3 processes.
1- The normal, currently existing programs. Add logs to the process 2- A Master replication process which listens for clients requesting for info. 3- The slave processes that request infomation and write it to the slave machines.
With this approach you can basically break it down into logical units of code which can be tested and debugged. Also helps when you need to worry about security and the level at which each component needs to work.
I'm not completely sure what you mean by these. Basically the same as what I said, except just have imap/deliver simply send the changes without any waiting?
I am basically suggesting to log all the changes to a log(s) and have a separate program handle passing on the information to the slaves.
But isn't the point of the master/slave that the slave would switch on if the master dies?
Yes.
If you switch slave to be the new master, it doesn't matter if the logs were written to master's disk. Sure the message could come back when the master is again brought back
I was thinking that once a master dies, next it it comes back it would be a slave and no longer master. This would, unfortunately imply, that any transactions that were not copied over would be lost.
I don't understand what you mean. Sure the logs could timeout at some point (or shouldn't there be some size limits anyway?), but you'd still need to keep track of what different slaves have seen.
I was thinking that it would be the slaves job to ask for data. Example pseudo transactions.
Master gets transactions 1 through 100 Slave(s) start from scratch and ask from transactions starting at 1. Say one slave, let's call it A, dies at transaction 50 and another slave, say B, continues and goes all the way to 100.
More transactions come and now we are up to 150. Slave B will ask for anything after 100. When Slave A comes back it would ask for anything after 50.
The master simply gets a request for transactions after a given number so it doesn't need to keep track the status of each slave.
possible. It would be easier to implement much simpler master-slave replication, but in error conditions that would almost guarantee that some messages get lost. I want to avoid that.
If you think Synchronous replication is doable.. go for it. :-) it is a tradeoff of reliability vs speed. Specially as the number of slaves grow the communication will grow.
Sure. I wasn't planning on implementing multi-slave or multi-master before the master/slave was fully working and stress testing showing that no mails get lost even if master is killed every few seconds (and each crash causing master/slave to switch roles randomly).
Great idea.