[Dovecot] Running on HPUX

[Rich Rauenzahn]

Timo Sirainen wrote:
> On Fri, 2006-06-23 at 16:13 -0700, Rich Rauenzahn wrote:
>   
>> I think multiple mmap's of the same file are verboten --
>>
>>          +  In most cases, two separate calls to mmap() cannot map
>>               overlapping ranges in a file.  The virtual address range
>>               reserved for a file range is determined at the time of the
>>               initial mapping of the file range into a process address
>>               space.  The system allocates only the virtual address range
>>               necessary to represent the initial mapping.  As long as the
>>               initial mapping exists, subsequent attempts to map a different
>>               file range that includes any portion of the initial range may
>>               fail with an ENOMEM error if an extended contiguous address
>>               range that preserves the mappings of the initial range cannot
>>               be allocated.
>>     
>
> Well, I don't see anything like that described in Linux or UNIX98 man
> pages..
>   

I'm sure the limitation predates UNIX98 and has something to do with a 
non-unified buffer cache and was a design/performance tradeoff long 
ago.  Soon we'll have a unified cache.

>> So, I was trying to look at why the index file is being mapped twice, 
>> and why the addr isn't saved and reused.
>>     
>
> The problem has something to do with how index file code works
> internally. A "view" is (supposed to be) a snapshot of the mailbox at a
> given state. It's not updated until the view is synchronized. Each view
> points to a "map" which contains the index file's mmap pointer and other
> information, including records_count which tells how many messages it
> holds.
>
> mail_index_map() is used to get the latest mapping of the index file. If
> the existing latest map is already used by views, Dovecot simply
> discards the existing map and mmap()s the file all over again.
>   

I was pondering an alternate work around from the bottom, but I think 
there's a problem with it.  I could write a mmap64_hpux that would 
remember the first mapping's addr (also verifies the fd against the 
inode #), and return the same addr again.  I think the problem occurs if 
you want some mappings to be RO and some RW.

> One way to handle this would be to unmap the existing area always,
> mmap() it and update all the existing maps' pointer to the new one. But
> since the maps may have different files mmapped (the index file could
> have been rebuilt and the maps point to different files), it's not that
> simple to implement.
>   

Would knowing the inode# associated with the fd help?  And would the 
RW/RO problem occur here as well?

> Another workaround would be to move the map to memory and let the views
> use that. That would be pretty easy to implement, but if HP-UX is the
> only OS where this breaks, I think I'll just leave it as it is since
> it's faster this way..
>
> Anyway, I think something like this should fix it (not tested):\
>   
Which workaround is this below?  Must be the 2nd.  It looks like you 
clone/copy the older memory map, and then unmap it.

> diff -u -r1.230.2.5 mail-index.c
> --- src/lib-index/mail-index.c  17 Jun 2006 13:42:36 -0000      1.230.2.5
> +++ src/lib-index/mail-index.c  24 Jun 2006 19:35:30 -0000
> @@ -1031,6 +1031,15 @@
>                 /* this map is already used by some views and they may have
>                    pointers into it. leave them and create a new mapping. */
>                 if (!index->mmap_disable) {
> +                       struct mail_index_map *old_map;
> +
> +                       old_map = index->map;
> +                       map = mail_index_map_clone(index->map,
> +                                                  index->map->hdr.record_size);
> +                       map->refcount = index->map->refcount;
> +                       mail_index_unmap(index, &index->map);
> +                       *old_map = *map;
> +                       i_free(map);
>                         map = NULL;
>                 } else {
>                         /* create a copy of the mapping instead so we don't
>
>