[Dovecot] Avoiding Hardlinks (was: Something other than dotlock for uidlist locking?)

[James Berry]

Hi Timo,

Just to follow up, I've been running with these patches for the last  
24 hours or so. With no problems on my end. And with great benefit  
wrt performance. Many operations have improved by several orders of  
magnitude.

Could we get this patch into rc7?

James

On Aug 9, 2006, at 4:03 PM, James Berry wrote:

> Timo,
>
> Thanks for your comments and attention. I'm attaching a patch which  
> is a bit rough, but it's what I'm going to begin testing with.  
> Basically in all cases ifdef'd based on AVOID_HARDLINKS. Hopefully  
> you might be able to improve the configurability of the dotlock case.
>
> Thanks for such a great product.
>
> James
>
> On Aug 9, 2006, at 3:05 PM, Timo Sirainen wrote:
>
>> On 9.8.2006, at 23.44, James Berry wrote:
>>> Timo, would you mind commenting on these to make sure my  
>>> understanding is correct? And for the last one, I need a  
>>> suggestion on how to proceed.
>>>
>>> file-copy.c:  [EASY]
>>>
>>> 	Looks like it would be easy to take the already coded case for  
>>> when try_hardlink is false.
>>
>> Yes, easy. This case could even be #ifdefed. Although currently  
>> the only user of file_copy() is copying ssl-parameters.dat from / 
>> var/lib/dovecot to /var/run/dovecot, and I don't see it being used  
>> elsewhere anytime soon.
>
> #ifdef'd
>
>>> file-dotlock.c: [EASY]
>>>
>>> 	In dotlock_create, use try_create_lock_excl rather than  
>>> try_create_lock_hardlink.
>>
>> This really should rather be configurable from dovecot.conf, but  
>> there's no simple way to do that, except something like a getenv()  
>> hack inside the file-dotlock.c code..
>>
>> I don't think this should be #ifdefed since O_EXCL could break if  
>> you're using NFS.
>
> #ifdef'd to always take the O_EXCL case if AVOID_HARDLINKS is defined.
>
>>> maildir-copy.c: [LOOKS EASY]
>>>
>>> 	In maildir_copy(), take the fallback case using mail_storage_copy.
>>
>> This is the default (maildir_copy_with_hardlinks=no) so no need to  
>> do anything.
>
> Okay, nothing done.
>
>>> maildir-save.c: [NEED HELP]
>>>
>>> 	In maildir_file_move(), there seems to be no fallback case, and  
>>> I'm unsure of the desired  symantics, so I could use some  
>>> suggestions here: can we just fall back into something like the  
>>> code in file-copy.c?
>>
>> Like you noticed, this can be changed to just rename(). The only  
>> reason it's link()+unlink() is because maildir spec says it should  
>> be done like that. And in case the maildir filenames aren't really  
>> unique the link()+unlink() method makes sure that mails aren't  
>> overwritten, but that practically can't happen with Dovecot. This  
>> code could perhaps also be #ifdefed..
>
> #ifdef'd to always use rename() if AVOID_HARDLINKS is defined.
>
> Patch attached:
>
> <patch-avoid-hardlinks.diff>