On 2022-10-21 13:25, dovecot@ptld.com wrote:
the problem that prevents most load balancers from handling the backend imap/pop traffic is that the load balancer needs to be aware of the context of each connection. which all boils down to the index files (only a single dovecot server can access a set of index files concurrently, else the indexes will get corrupted)
As someone else asked on this thread, what prevents two clients, both being directed to the same server, from fighting over index files? Wouldn't file locks over NFS prevent this problem? And if so, doesn't that also prevent two dovecot installations from fighting over index files?
i believe the dovecot processes have some sort of interprocess communication when they are running on the same host that they use to negotiate writes to the index files. i don't really know the details, other than that the index files get corrupted very quickly if multiple hosts are accessing them at once. the index files are fine if a users' multiple imap sessions are on a single host.
iirc, dovecot does use file locks when moving/deleting maildir+ message files. but that's not really the issue - it's all about the index files.
those index files just weren't designed to have parallel access from multiple machines.
What is a way to test your system to know if dovecot is using the default fcntl file locks over NFS4 and they actually work? Or is it better/safer to use dotlock on NFS4 without director?
nfs locks do work, as long as your nfs server supports them well. dotlocks don't require any nfs server support, but they are slower. but, for the most part if you are redirecting users sessions to the same server it doesn't matter. we've used both locking types, though dotlocks were more reliable on some nfs servers.
as long as you aren't using DBOX for mail storage, having the indexes get corrupted isn't the end of the world - dovecot will just regenerate them (though you might have to remove the broken files and kill the users' sessions to force this).
there's some dovecot documentation with suggestions: https://doc.dovecot.org/configuration_manual/nfs/ and some older docs: https://wiki1.dovecot.org/NFS