[Dovecot] Something other than dotlock for uidlist locking?

[James Berry]

On Aug 8, 2006, at 5:43 PM, James Berry wrote:

> Hi Timo, (By the way, this seems to relate to a performance issue  
> on Mac OS X with HFS+).

I think the problem may be more general than this particular case.  
This fix speeds up some accesses, but there are still bottlenecks, I  
believe. I have a feeling the performance issues will manifest in any  
cases where hardlinks are used heavily.

Can you tell me, Timo, in what other cases hard links are used (file  
copy, seemingly being one) and whether there is any way to turn them  
off or rework the code to avoid them?

Thanks!

James

> On Aug 28, 2005, at 12:30 PM, Timo Sirainen wrote:
>> On Thu, 2005-08-25 at 10:26 -0700, James Berry wrote:
>>> If I read the code correctly, only dotlocks are supported for  
>>> locking
>>> of dovecot-uidlist. And I don't see any settings in the config file
>>> for it.
>>>
>>> As it turns out, dotlocks are very slow on my system, due to their
>>> implementation via hardlinks.
>>>
>>> Would it be possible to support other locking mechanisms for locking
>>> of the uidlist file?
>>
>> They are locked only when they are modified, and the modification  
>> works
>> by creating a new file and renaming it over the dovecot-uidlist file.
>>
>> Is it only hardlinks that are problematic? It shouldn't be too  
>> difficult
>> to change the dotlock code to rely on working O_EXCL instead of
>> hardlinks.
>>
>> Hmm. Actually I just did that in CVS. I guess I should also put  
>> out some
>> setting to enable it in config file. For now you could just change  
>> this:
>>
>> 	ret = set->use_excl_lock ?
>> 		try_create_lock_excl(&lock_info, write_pid) :
>> 		try_create_lock_hardlink(&lock_info, write_pid);
>>
>> to
>>
>> 	ret = try_create_lock_excl(&lock_info, write_pid);
>>
>> and see if it works faster.
>>
>
> Well, a year later I finally got around to trying this out. On a  
> Mac system. And guess what? It seems much, much snappier. Sorry  
> about the delay in testing it. One of the symptoms I've been seeing  
> on the server is that performance gradually degrades over time,  
> even if dovecot is restarted. The problem seems better after a  
> server restart, which clears out an area on disk which tracks  
> unlinked and hardlinked files. I believe this option will fix that  
> performance issue.
>
> Any chance of getting a configuration option for this?
>
> James