On Thu, Jul 5, 2012 at 12:35 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
On 7/5/2012 2:44 AM, Adrian M wrote:
Hi Stan, I know how to add drives to the storage and how to grow the existing filesystem, but such big filesystems are somehow new to mainstream linux. Yes, I know some university out there already have pentabytes filesystems, but right now stable linux systems have trouble formatting ext4 partition over 16T. All this is telling me that is safer to have two or tree smaller filesystems than a big one. Dovecot has a nice feature for this "Directory hashing" http://wiki.dovecot.org/MailLocation/
At 16TB+ scale with maildir you should be using XFS on kernel 3.x, not EXT4. Your performance will be significantly better, as in 30% or much more. The typical XFS filesystem in the wild today is 50TB+ and there are hundreds of XFS filesystems well over 100TB deployed around the world.
NASA has XFS filesystems of 380TB and 535TB, and also has multiple 1PB+ CXFS (cluster XFS) filesystems. 20TB is a tiny snack for XFS, 500TB is lunch, 1PB is a big supper. A single XFS can scale to 16 Exabytes, or 1 million terabytes, though the largest deployed so far that I'm aware of is NASA's 535TB XFS. It'll scale to anything you'll ever throw at it, and much more.
What I don't know is a nice way to migrate from a single directory no hashing to more than one and hashing.
It's a good time to migrate to XFS.
-- Stan
Hi Stan, I already have xfs, and kernel 3.x. Switching from kernel 2.6.x to 3.x a couple months ago did indeed decrease the iops number.