I'm going to deploy postfix + dovecot + CephFS( as Mail Storage). Basically I want to use two servers for them, which is kind of HA.
you may consider dovecot's builtin dsync replication which works great with two servers (while there still is one little bug that may duplicate mails upon deletion with pop3 only under specific conditions)
My idea is that using keepalived or Pacemaker to host a VIP, which could fail over the other server once one is down. And I'll use Haproxy or Nginx to schedule connections to one of those server based on source IP( Session stickiness), I'll use VIP as DNS record.etc, is my plan doable?
I know MX could be server ones with different priority. But I think it brings along shortage that DNS couldn't know Email server is up or down, it just returns results to MUA, right?
DNS just returns your servers' ip addresses/mx records and does not know if they are up or down. You could combine that with an external monitoring system that modifies your dns entries but this is overkill (keep ttl in mind). DNS resolvers return records in a round robin fashion so that you get 50/50 active/active loadbalancing. SMTP does cope with delivery errors very well (e.g. greylisting is a temporary delivery error). MTAs just connect to the second MX and try to deliver the mail. Even MUAs like Outlook, Apple Mail or Thunderbird are capable to use more than one ip - if the connection fails they connect to the second ip returned via DNS, without any user interaction.
Best regards Gerald