[Dovecot] Compressing existing maildirs

[Stan Hoeppner]

On 12/31/2011 12:56 AM, Jan-Frode Myklebust wrote:
> On Fri, Dec 30, 2011 at 06:38:28PM -0600, Stan Hoeppner wrote:
>>
>> Roger that.  Good strategy.  You using SAN storage or local RAID?  What
>> filesystem do you plan to use for the new mailbox location?  What OS is
>> the Dovecot host?
> 
> IBM DS4800 SAN-storage. Filesystem is IBM GPFS, which stripe all I/O
> over all the RAID5 LUNs it has assigned. Kind of like RAID5+0. To guard
> against disaster if one RAID5 array should fail, we plan on replicating
> the filesystem metadata on different sets for LUNs.

Nice setup.  I've mentioned GPFS for cluster use on this list before,
but I think you're the only operator to confirm using it.  I'm sure
others would be interested in hearing of your first hand experience:
pros, cons, performance, etc.  And a ball park figure on the licensing
costs, whether one can only use GPFS on IBM storage or if storage from
others vendors is allowed in the GPFS pool.

To this point IIRC everyone here doing clusters is using NFS, GFS, or
OCFS.  Each has its downsides, mostly because everyone is using maildir.
 NFS has locking issues with shared dovecot index files.  GFS and OCFS
have filesystem metadata performance issues.  How does GPFS perform with
your maildir workload?

> OS is RHEL (currently RHEL4 and RHEL5, but new servers are implemented
> on RHEL6).
> 
>> Lastly, how many users you have?  Sorry for prying,
> 
> I'd rather not say.. but we're an ISP, with about 250.000 residential
> customers and multiple mailboxes per customer.
> 
>> I'm always really curious about system details when someone states they
>> have 12TB of mailbox data. ;)
> 
> 	$ df -h /usr/local/atmail/users
> 	Filesystem            Size  Used Avail Use% Mounted on
> 	/dev/atmailusers       14T   12T  2.1T  85% /usr/local/atmail/users
> 	$ df -hi /usr/local/atmail/users
> 	Filesystem            Inodes   IUsed   IFree IUse% Mounted on
> 	/dev/atmailusers        145M    109M     37M   75% /usr/local/atmail/users
> 
> Looking forward to reducing the number of inodes when we finally move to
> mdbox.. Should do wonders to the backup process.

That will depend to a large degree on your mdbox_rotate_size value.  The
default is 2MB, which means you'll get multiple ~2MB mdbox files.  If we
assume the average email size including headers and attachments is 32KB,
Dovecot will place ~64 such emails in a single mdbox file with the
default 2MB setting.  32KB may be a high or low average depending on
your particular users.

Considering there is no inherent performance downside to going larger
than the default, and significant gains to be made, consider a setting
of 8MB to 16MB.  This will dramatically reduce both inode consumption
and filesystem metadata IOPS vs maildir.  Reducing IOPS on a shared SAN
is always a plus, especially if you're going to be adding some extra
GPFS replication traffic.

Timo, is there any technical or sanity based upper bound on mdbox size?
 Anything wrong with using 64MB, 128MB, or even larger for
mdbox_rotate_size?

-- 
Stan