Timo Sirainen wrote:
On Fri, 2006-06-23 at 16:13 -0700, Rich Rauenzahn wrote:
I think multiple mmap's of the same file are verboten --
+ In most cases, two separate calls to mmap() cannot map overlapping ranges in a file. The virtual address range reserved for a file range is determined at the time of the initial mapping of the file range into a process address space. The system allocates only the virtual address range necessary to represent the initial mapping. As long as the initial mapping exists, subsequent attempts to map a different file range that includes any portion of the initial range may fail with an ENOMEM error if an extended contiguous address range that preserves the mappings of the initial range cannot be allocated.
Well, I don't see anything like that described in Linux or UNIX98 man pages..
I'm sure the limitation predates UNIX98 and has something to do with a non-unified buffer cache and was a design/performance tradeoff long ago. Soon we'll have a unified cache.
So, I was trying to look at why the index file is being mapped twice, and why the addr isn't saved and reused.
The problem has something to do with how index file code works internally. A "view" is (supposed to be) a snapshot of the mailbox at a given state. It's not updated until the view is synchronized. Each view points to a "map" which contains the index file's mmap pointer and other information, including records_count which tells how many messages it holds.
mail_index_map() is used to get the latest mapping of the index file. If the existing latest map is already used by views, Dovecot simply discards the existing map and mmap()s the file all over again.
I was pondering an alternate work around from the bottom, but I think there's a problem with it. I could write a mmap64_hpux that would remember the first mapping's addr (also verifies the fd against the inode #), and return the same addr again. I think the problem occurs if you want some mappings to be RO and some RW.
One way to handle this would be to unmap the existing area always, mmap() it and update all the existing maps' pointer to the new one. But since the maps may have different files mmapped (the index file could have been rebuilt and the maps point to different files), it's not that simple to implement.
Would knowing the inode# associated with the fd help? And would the RW/RO problem occur here as well?
Another workaround would be to move the map to memory and let the views use that. That would be pretty easy to implement, but if HP-UX is the only OS where this breaks, I think I'll just leave it as it is since it's faster this way..
Anyway, I think something like this should fix it (not tested):\
Which workaround is this below? Must be the 2nd. It looks like you clone/copy the older memory map, and then unmap it.
diff -u -r1.230.2.5 mail-index.c --- src/lib-index/mail-index.c 17 Jun 2006 13:42:36 -0000 1.230.2.5 +++ src/lib-index/mail-index.c 24 Jun 2006 19:35:30 -0000 @@ -1031,6 +1031,15 @@ /* this map is already used by some views and they may have pointers into it. leave them and create a new mapping. */ if (!index->mmap_disable) { + struct mail_index_map *old_map; + + old_map = index->map; + map = mail_index_map_clone(index->map, + index->map->hdr.record_size); + map->refcount = index->map->refcount; + mail_index_unmap(index, &index->map); + *old_map = *map; + i_free(map); map = NULL; } else { /* create a copy of the mapping instead so we don't