On Aug 9, 2006, at 2:06 PM, Ben Schumacher wrote:
On 8/9/06, James Berry <james@jberry.us> wrote:
maildir-save.c: [NEED HELP]
In maildir_file_move(), there seems to be no fallbackcase, and I'm unsure of the desired symantics, so I could use some suggestions here: can we just fall back into something like the code in file- copy.c?
James-
This probably has to do with the way that Maildir's work. Check out this links for details:
http://cr.yp.to/proto/maildir.html http://en.wikipedia.org/wiki/Maildir
The use of hardlinks give you a level of atomicity and efficiency that I'm not sure you're going to achieve using a standard copy, but I suppose if HFS+ doesn't handle hardlinks well then it's not doing much for you. Basically the hardlink solution gives you good atomicity even through a process crash/kill... something that a straight data copy and delete can't easily ensure.
Hmm looks to me like rename() would give the same level of atomicity
as link?
So instead of doing link/unlink, we could just do rename()?
It's really unfortunate that Apple made such an enormously poor decision in the implementation of HFS+. Any chance you could just use a UFS1 or UFS2 formatted drive for your mail storage?
Yes. Unfortunately hardlinks were grafted on to HFS+ after it was
designed. They are a hack. They work fine if there are not too many
of them. But they suffer if you create many, many of them.
Hardinked files on HFS+ live in a single special hidden directory.
Any file that becomes multiply linked is moved into that directory.
And the only way for it to leave that directory is for (a) the link
count to go to zero and (b) for the volume to be remounted. What this
means is that even just using link/unlink to move a file will cause
the file to accumulate in that hidden directory (with linkcount=1).
So in the MailDir case, _all_ mail files will ultimately end up in
that hidden directory. Making the matter worse, temporary files, such
as .lock files, will accumulate at linkcount=0 in the hidden
directory until the volume is remounted (usually at reboot). So as
you can image, the performance of hardlinks suffers due to the size
of this directory as soon as you get very many files, and grows worse
the longer the machine is up.
So, hardlinks were grafted on to HFS+ anticipating that unix-like
systems needed to hardlink _some_ files. The design did not
anticipate the sort of usage of hardlinks by dovecot. And it fails
miserably.
It certainly looks possible to me that we can make the needed changes
to get very reasonable performance without hardlinks, and without
having to use UFS, which has its own problems with many files such as
we get with MailDir.
James