[Dovecot] Performance-Tuning
Hi,
I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA.
I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that.
At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff).
I have to change my setup, maybe somebody else here have hard facts:
*) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout?
*) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to?
Thanks for any hints an tricks,
Peer
--
Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de
Tel: 030/405051-42 Fax: 030/405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
What is the setup on the EVA, FC or iSCSI? Sent from my BlackBerry® wireless device
-----Original Message----- From: Peer Heinlein p.heinlein@heinlein-support.de Sender: dovecot-bounces@dovecot.org Date: Tue, 8 Nov 2011 14:50:25 To: dovecot@dovecot.org Subject: [Dovecot] Performance-Tuning
Hi,
I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA.
I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that.
At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff).
I have to change my setup, maybe somebody else here have hard facts:
*) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout?
*) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to?
Thanks for any hints an tricks,
Peer
--
Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de
Tel: 030/405051-42 Fax: 030/405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
On 08.11.2011 14:50, Peer Heinlein wrote:
*) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to?
Hi Peer,
We have switched our mailbox storage format from maildir to mdbox!
Maildir is a disaster. (too many small files) After the migration to mdbox the performance has improved significantly.
Conclusion: mdbox is great and much better performance than maildir! I would also recommend ext4.
Best regards,
Morten
- Morten Stevens mstevens@imt-systems.com:
We have switched our mailbox storage format from maildir to mdbox!
I wonder how I can incrementally change over from Maildir to mdbox? I can of course use dsync to mirror Maildir: to mdbox:, but how can I make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or vice versa).
That would allow for a smooth transition...
-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de
Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt:
Hi,
I wonder how I can incrementally change over from Maildir to mdbox?
If you have double diskspace:
Just use "dsync mirror" in the background to prepare the change. After that it's just a short downtime to migrate incremental the last changes, or it's just a question of a short login-script.
if [ -d ~/Maildir ] ; then dsync mirror voodoo-magic rm -R ~/Maildr fi
I can of course use dsync to mirror Maildir: to mdbox:, but how can I make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or vice versa).
I wonder about that problem too. Even the last-last-last-quick sync would be so much IO, that I can't handle it in realtime in the morning at 9 a.m.
Looks like a nightly downtime for the last incremental run.
It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP attributes. In this case the whole migration process could be split up into groups. Unfortunately we have shared folders and I don't know a way to read the *remote* mailbox-format from LDAP... So having users with maildir and mdbox mixed up will break their shared folders...
Peer
--
Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de
Tel: 030/405051-42 Fax: 030/405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
- Peer Heinlein p.heinlein@heinlein-support.de:
Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt:
Hi,
I wonder how I can incrementally change over from Maildir to mdbox?
If you have double diskspace:
haha :) no.
I thought of a per-user migration, that way I don't need extra space.
-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de
On 8.11.2011, at 16.34, Peer Heinlein wrote:
I can of course use dsync to mirror Maildir: to mdbox:, but how can I make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or vice versa).
I wonder about that problem too. Even the last-last-last-quick sync would be so much IO, that I can't handle it in realtime in the morning at 9 a.m.
Looks like a nightly downtime for the last incremental run.
It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP attributes.
Easy!
In this case the whole migration process could be split up into groups. Unfortunately we have shared folders and I don't know a way to read the *remote* mailbox-format from LDAP... So having users with maildir and mdbox mixed up will break their shared folders...
Not so easy.. Only the home directory can be currently looked up from userdb for shared folders.
There is also automatic detection of Maildir and mbox when mail_location isn't set, but no such code for mdbox. It could be added without much trouble though. But for shared folders, assuming you'd want per-user \seen flags, it would also need something like:
mail_location = auto::INDEX=~/shared-indexes
This "auto" doesn't exist yet either. And then there's the biggest problem: You can't have per-user \seen flags with mdbox, because you can't change the index file path without breaking mdbox.
Quoting Peer Heinlein p.heinlein@heinlein-support.de:
It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP attributes. In this case the whole migration process could be split up into groups. Unfortunately we have shared folders and I don't know a way to read the *remote* mailbox-format from LDAP... So having users with maildir and mdbox mixed up will break their shared folders...
May not work for you, but...
The way I did this when I migrated was to run two dovecot instances, and have perdition software on a front-end (could be on the same machine instead of a front-end, I just happen to have a front-end machine to do it).
Perdition will query ldap for the info per user/connection, and send the connection to the correct dovecot instance based on the ldap lookup. Worked for me, your milage may vary...
-- Eric Rostetter The Department of Physics The University of Texas at Austin
Go Longhorns!
Am Dienstag, 8. November 2011, 17:53:32 schrieb Eric Rostetter:
May not work for you, but...
The way I did this when I migrated was to run two dovecot instances, and have perdition software on a front-end (could be on the same machine instead of a front-end, I just happen to have a front-end machine to do it).
You could do that with Dovecot, too.
Perdition will query ldap for the info per user/connection, and send the connection to the correct dovecot instance based on the ldap lookup. Worked for me, your milage may vary...
The problem is: You're running in problems with shared folders. You can't read your neighbors storage-engine from ldap.
It's easy to read the user's storage engine from ldap. So there's no need to use perdition for that :-) But you can't read or proxy the storage engine from somebody who shared you his folders.
That's my problem :-(
Peer
--
Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de
Tel: 030/405051-42 Fax: 030/405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Quoting Peer Heinlein p.heinlein@heinlein-support.de:
The problem is: You're running in problems with shared folders. You can't read your neighbors storage-engine from ldap.
Yes, but I didn't have any shared folders, so it worked. Your milage may vary, as I said... :)
-- Eric Rostetter The Department of Physics The University of Texas at Austin
Go Longhorns!
On 08.11.2011 15:16, Ralf Hildebrandt wrote:
- Morten Stevens mstevens@imt-systems.com:
We have switched our mailbox storage format from maildir to mdbox!
I never tried it, but it should be possible to provide the mail_location from the user repsoitory (LDAP, SQL, whatever) So you can keep your global config, and use a script to convert one mailbox after another, and add a mail_location extra userdb field in the user repository to overwrite the global setting on a per-user-basis.
Regards, Oliver
Hi there,
I never tried it, but it should be possible to provide the mail_location from the user repsoitory (LDAP, SQL, whatever)
Actually this works :-) Our userdb looks similar to:
account1:xyz:000:000::/account1s/home/dir::userdb_mail=maildir:/account1s/home/dir/Maildir account2:xyz:000:000::/account2s/home/dir::userdb_mail=mdbox:/ account2s/home/dir
http://wiki2.dovecot.org/UserDatabase/ExtraFields
Concerning Maildir backups: what about a backup-to-disc-to-tape scheme using snapshots for the "to-disc" part and something like perpetual incrementals afterwards for the "top-tape" (secondary store) ?
Regards Dirk
Dirk Jahnke-Zumbusch Deutsches Elektronen-Synchrotron DESY IT Information Fabrics Member of the Helmholtz Association D-22603 Hamburg Notkestrasse 85 / 22607 Hamburg T: +49-40-899.81760 F: +49-40-899.41760 dirk.jahnke-zumbusch@desy.de
So you can keep your global config, and use a script to convert one mailbox after another, and add a mail_location extra userdb field in the user repository to overwrite the global setting on a per-user-basis.
Regards, Oliver
On 8.11.2011, at 16.16, Ralf Hildebrandt wrote:
- Morten Stevens mstevens@imt-systems.com:
We have switched our mailbox storage format from maildir to mdbox!
I wonder how I can incrementally change over from Maildir to mdbox? I can of course use dsync to mirror Maildir: to mdbox:, but how can I make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or vice versa).
That would allow for a smooth transition...
If you don't have shared folders (as explained in previous mail) and you can have per-user mail_location in the userdb, this should be pretty easy. The man page for dsync lists the steps that can be used for online migration.
On 8.11.2011, at 16.03, Morten Stevens wrote:
We have switched our mailbox storage format from maildir to mdbox!
Maildir is a disaster. (too many small files) After the migration to mdbox the performance has improved significantly.
Conclusion: mdbox is great and much better performance than maildir! I would also recommend ext4.
You don't happen to have any specific numbers/graphs that can be used to compare maildir vs. mdbox in the same hardware? I'd be interested in seeing those, such as a graph of disk iops spanning a month before/after mdbox switch.
We are very happy with mdbox+zlib+ext4 + iSCSI SAN (HP Lefthand in our setup)
If you have CPU to spare, consider using zlib with mdbox. You are trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Mdbox has halved our backup windows (2,8 TB uncompressed mailboxes, 2 TB compressed) and backup software is happier because there are few (100.000+ files with mdbox) to backup instead of several millions (Maildir)
Regards
JavierHi,
I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA.
I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that.
At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff).
I have to change my setup, maybe somebody else here have hard facts:
*) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout?
*) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to?
Thanks for any hints an tricks,
Peer
Am Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodríguez:
Hi,
If you have CPU to spare, consider using zlib with mdbox. You aretrading CPU power (cheap) to get fewer IOPS (IOPS count is expensive).
Hey. This point is great. I hadn't realized that.
Sure. zlib will save IOPS and 2x6-CPUs aren't a problem. Good point -thanks.
compressed) and backup software is happier because there are few (100.000+ files with mdbox) to backup instead of several millions (Maildir)
Yes, that#s the main reason why I want to switch to mbox. At the moment our roundtrip-time for the backup is > 24h...
Peer
--
Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de
Tel: 030/405051-42 Fax: 030/405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Other important thing to consider is message expunging. With mdbox you are "delaying" the I/O associated with deleting e-mails. We have a nightly cronjob that expunge messages from mdboxes.
If you have en EVA (wich one? 4.400? 6.400? ) you also can consider RAID 1+0 or SSD for indexes. Indexes are hammered in mdbox.
Regards
JavierAm Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodríguez:
Hi,
If you have CPU to spare, consider using zlib with mdbox. You aretrading CPU power (cheap) to get fewer IOPS (IOPS count is expensive).
Hey. This point is great. I hadn't realized that.
Sure. zlib will save IOPS and 2x6-CPUs aren't a problem. Good point -thanks.
compressed) and backup software is happier because there are few (100.000+ files with mdbox) to backup instead of several millions (Maildir) Yes, that#s the main reason why I want to switch to mbox. At the moment our roundtrip-time for the backup is> 24h...
Peer
On 8.11.2011, at 15.50, Peer Heinlein wrote:
At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff).
A few more ideas for Maildir if you haven't done yet:
- maildir_very_dirty_syncs = yes
- pop3_no_flag_updates = yes
- Switching to dict file quota instead of Maildir++ quota.
On 11/8/2011 7:50 AM, Peer Heinlein wrote:
I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA.
That's a lot of mail (likely a large user base--not given), on a filesystem not designed for such, on a decent SAN controller--LUN RAID configuration not given.
I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that.
If you're going to switch filesystems, for this size dataset and concurrent workload, you're moving in the wrong direction.
At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff).
EXT3/4 are not designed, nor optimized, for high concurrency workloads.
I have to change my setup, maybe somebody else here have hard facts:
*) Is ext4 faster? How much faster?
Simulated maildir workload test on 2.6.35-rc5, 128 threads (No data published for newer kernels):
http://btrfs.boxacle.net/repository/raid/2.6.35-rc5/2.6.35-rc5/2.6.35-rc5_Ma...
As you can see EXT4 shows a small gain over EXT3, ~20%. If you really want high performance it's time to move to XFS, properly configured to match the underlying RAID characteristics of the LUN(s) you're mounting. You'll prefer kernel 2.6.39+, 2.6.36 at minimum, so you get the delayed logging feature (2.6.35 had delayed logging but had problems in other areas).
I'll assume with a >10TB mail store that you're seeing greater than 128 concurrent user operations regularly. As you can see from the graph, XFS will give you ~50% greater ops/s than EXT4 and ~90% greater than EXT3--yes, almost double that of EXT3. As the concurrency increases, so will this performance gap, as XFS was designed from day 1 for high concurrency workloads.
This is a simulated mail server benchmark. However you should see similar gains with Dovecot. The XFS delayed logging feature will dramatically reduce the number of physical IOs required for journal writes (i.e. metadata IO), as will delayed allocation, a feature of XFS since its inception in 1994. EXT4 was the first of its lineage to gain delayed allocation, some 10+ years later, after Ted T'so studied the XFS code.
In short, if you want an 'enterprise caliber' production Linux filesystem tailor made for high IO concurrency, XFS is it. JFS yields similar performance, but hasn't been actively developed for 8 years or so. XFS has substantial ongoing feature and fix development.
*) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout?
AIUI, the bulk of the EXT4 performance advantage over EXT3 is the delayed allocation logic. The new EXT4 extent based on disk layout yields little in the way of additional performance, but much in free space management, fragmentation mitigation, etc.
*) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to?
mdbox will substantially decrease physical IOs to your storage back end due to dramatically less metadata operations compared to maildir. You've stated you currently have a storage IOPS bottleneck, so I'd have to assume mdbox will seriously increase your overall performance. Good old mbox will do so as well, but everyone shuns it for various reasons, some valid, some not so valid.
If you have an appropriate LUN available (sufficient size and spindle speed/count of member disks), properly create an XFS filesystem on it (read much before creating it), and moved to mdbox atop that, I think you'll be really surprised by how much you gain from simply changing filesystems and mailbox storage formats. If you double the size of the LUN you could potentially carry twice as many users with, fewer IOPS than you're seeing now, on essentially the same hardware platform.
-- Stan
Am Montag, 14. November 2011, 00:31:24 schrieb Patrick Westenberg:
I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA.
You have 11 TB of mails on a non cluster filesystem?
Yes.
I don't believe a clustered filesystem would have more performance and would be more rock solid.
I don't have a problem on my frontend server. Why should I have two or more of them? I have a problem in my backend. My SAN has too much to do. Why should a cluster filesystem be better for my SAN?
Is it only accessed from one server or how does it work?
Yes.
peer
--
Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting
http://www.heinlein-support.de Tel: 030 / 40 50 51 - 0 Fax: 030 / 40 50 51 - 19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
On Mon, Nov 14, 2011 at 10:34:02AM +0100, Peer Heinlein wrote:
I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA.
You have 11 TB of mails on a non cluster filesystem?
Yes.
I don't believe a clustered filesystem would have more performance and would be more rock solid.
I don't have a problem on my frontend server. Why should I have two or more of them? I have a problem in my backend. My SAN has too much to do. Why should a cluster filesystem be better for my SAN?
Agree. A non-clustered fs should give you better performance, and probably also be more reliable, if you can live with the SPoF and full downtime during patching/upgrades/maintenance. But I would expect xfs to be a better choice than ext*.
We have about the same storage size as you (12TB/115M-inodes), with the backup-process almost biting itself in the tail every day, but I can't quite imagine running it all on a single local fs with no scale-out options if we should want/need more processing power for dovecot. I'm looking forward to moving to mdbox soonish.. to reduce the number of files and speed up the backup process.
-jf
On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote:
Agree. A non-clustered fs should give you better performance, and probably also be more reliable, if you can live with the SPoF and full downtime during patching/upgrades/maintenance. But I would expect xfs to be a better choice than ext*.
Depends on the workload characteristics and how well the XFS filesystem is tuned to the storage hardware. If setup properly, using many allocation groups with fast spindles, a decent amount of BBWC, and a high concurrency maildir workload (dozens to hundreds of delivery and IMAP operations), XFS will runs circles around EXTx as it can create/write/read to every AG in parallel. Much of EXT4's operation is still serialized. This is why XFS outruns all other filesystems in the highly parallel mail workload benchmarks I posted previously, EXTx by a factor of 2-3.
For smaller hosts that don't see parallelism, for example SOHO servers, XFS will likely be slower than EXTx as the workload will be serialized.
-- Stan
Jan-Frode
Wich clustered filesytem do you have?
I used to have ocfs2 but had problems with performance. So had to get back to ext4 and it solve the performance problem...
My ocfs2 setup had some problems... but still.. Some numbers:
OCFS2 1TB of maildir files. Full backup 36 Hours Incremental 15 hours
Ext4 1TB of maildir files. Full backup 16 Hours Incremental 1 hour
Same LUN on storage.
[]'sf.rique
On Mon, Nov 14, 2011 at 4:42 PM, Stan Hoeppner stan@hardwarefreak.comwrote:
On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote:
Agree. A non-clustered fs should give you better performance, and probably also be more reliable, if you can live with the SPoF and full downtime during patching/upgrades/maintenance. But I would expect xfs to be a better choice than ext*.
Depends on the workload characteristics and how well the XFS filesystem is tuned to the storage hardware. If setup properly, using many allocation groups with fast spindles, a decent amount of BBWC, and a high concurrency maildir workload (dozens to hundreds of delivery and IMAP operations), XFS will runs circles around EXTx as it can create/write/read to every AG in parallel. Much of EXT4's operation is still serialized. This is why XFS outruns all other filesystems in the highly parallel mail workload benchmarks I posted previously, EXTx by a factor of 2-3.
For smaller hosts that don't see parallelism, for example SOHO servers, XFS will likely be slower than EXTx as the workload will be serialized.
-- Stan
On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote:
Wich clustered filesytem do you have?
We use IBM GPFS, with currently 7 servers working against shared LUNs from an IBM DS4800.
My ocfs2 setup had some problems... but still.. Some numbers:
OCFS2 1TB of maildir files. Full backup 36 Hours Incremental 15 hours
Ext4 1TB of maildir files. Full backup 16 Hours Incremental 1 hour
Wow, local fs's are fast!
We have split the backup process up to run on 6 of the servers, with each server processing only a part of the filesystem (/a-f, /g-l, /m-p, etc..). The backup processing time varies quite a bit every day, but are mostly somewhere between 14-24 hours on each server. This sounds like something between 1.5x to 2x the incremental-performance you're seeing:
15 hours/incremental of 1TB ocfs2 = 15h/TB
6x 15 hours for incremental of 12 TB GPFS = 7.5h/TB
6x 20 hours for incremental of 12 TB GPFS = 10h/TB All our backups are incremental.
-jf
[]'sf.rique
How many servers have access to your maildir on ext4 so that you could
switch from ocfs2 to ext4? I use ocfs2 in my test environment for four servers (2 MX and 2 IMAP)
I used have 3 serves one mailman and 2 imaps. I experence that if take my loadbalancer and takes it all to just one server, itwould go faster becasue of cache. But once thsi server fails, the loadbalaner send it to the other one, but it simple does not work, clients experence timeouts, because the filesystem were too slow.
On Thu, Nov 17, 2011 at 7:34 AM, Jan-Frode Myklebust janfrode@tanso.netwrote:
On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote:
Wich clustered filesytem do you have?
We use IBM GPFS, with currently 7 servers working against shared LUNs from an IBM DS4800.
My ocfs2 setup had some problems... but still.. Some numbers:
OCFS2 1TB of maildir files. Full backup 36 Hours Incremental 15 hours
Ext4 1TB of maildir files. Full backup 16 Hours Incremental 1 hour
Wow, local fs's are fast!
Much faster! I could not run a "du" before, if i did would crash the system, now i can!
We have split the backup process up to run on 6 of the servers, with each server processing only a part of the filesystem (/a-f, /g-l, /m-p, etc..). The backup processing time varies quite a bit every day, but are mostly somewhere between 14-24 hours on each server. This sounds like something between 1.5x to 2x the incremental-performance you're seeing:
15 hours/incremental of 1TB ocfs2 = 15h/TB 6x 15 hours for incremental of 12 TB GPFS = 7.5h/TB 6x 20 hours for incremental of 12 TB GPFS = 10h/TBAll our backups are incremental.
-jf
participants (13)
-
Eric Rostetter
-
Henrique Santos Fernandes
-
Jahnke-Zumbusch, Dirk
-
Jan-Frode Myklebust
-
Javier de Miguel Rodríguez
-
Morten Stevens
-
Oliver Eales
-
Patrick Westenberg
-
Peer Heinlein
-
Ralf Hildebrandt
-
Ricardo Branco
-
Stan Hoeppner
-
Timo Sirainen