[Dovecot] Dovecot imaptest on RHEL4/GFS1, RHEL6/GFS2, NFS and local storage results
We have done some benchmarking tests using dovecot 2.0.12 to find the best shared filesystem for hosting many users, here I share with you the results, notice the bad perfomance of all the shared filesystems against the local storage. Is there any specific optimization/tunning on dovecot for use GFS2 on rhel6??, we have configured the director to make the user mailbox persistent in a node, we will thank's any help from you. we are interested in using GFS2 or NFS, we believe the problem is the locks, how can we improve this??
best regards, Aliet
The results
rhel 4.8 x86_64/GFS1 two nodes, shared FC lun on a SAN
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 2608 1321 1311 2608 2508 3545 547 2001 2493 2702 5282 2- 2810 1440 1430 2810 2688 3835 403 2154 2679 2925 5706 3- 2913 1457 1441 2908 2778 3913 417 2253 2773 3034 5924 4- 2814 1448 1412 2812 2695 3910 401 2186 2686 2929 5712 5- 2789 1464 1432 2787 2652 3774 427 2112 2649 2879 5676 6- 2843 1460 1444 2839 2722 3948 422 2164 2713 2957 5778
rhel6 x86_64/GFS2 two nodes, shared FC lun on a SAN(Used RDM in VMWare
vSphere for GFS2 lun)
Tunned cluster suite cluster.conf + <dlm plock_ownership="1"
plock_rate_limit="0"/>
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 2730 1340 1356 2704 2644 3748 522 2125 2643 2662 5422 2- 3309 1618 1659 3294 3223 4658 531 2563 3221 3239 6596 3- 2131 1046 1017 2055 2025 2911 381 1608 2024 2052 4256 4- 2176 1055 1039 2082 2058 2947 377 1671 2058 2078 4344 5- 1859 928 931 1859 1800 2626 304 1454 1799 1801 3706 6- 2672 1322 1329 2672 2607 3758 464 2097 2606 2615 5326
rhel6 x86_64/GFS2 two nodes, shared FC lun on a SAN(Used RDM in VMWare vSphere for GFS2 lun) Cluster suite defaults configs for plocks
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 1417 644 676 1325 1305 1872 308 1048 1302 1318 2824 2- 837 378 392 742 726 1050 117 588 722 734 1658 3- 803 363 347 752 745 1069 153 597 744 750 1658 4- 1682 802 811 1587 1569 2261 291 1299 1569 1585 3360 5- 1146 583 564 1146 1037 1500 213 811 1037 1049 2290 6- 838 403 366 744 734 1057 152 561 731 736 1664
rhel6 x86_64 two nodes used NFS(NAS Freenas 0.8, nfsvers 3)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 1382 699 691 1357 1063 1500 224 861 1053 1313 2694 2- 1634 785 799 1610 1459 2120 311 1192 1451 1570 3204 2- 1635 826 806 1611 1463 2088 345 1159 1459 1568 3190 3- 1574 758 781 1537 1403 2060 324 1135 1396 1504 3090 4- 1685 842 807 1653 1506 2135 349 1215 1504 1634 3344 5- 1766 850 893 1737 1582 2289 335 1288 1579 1705 3480 6- 1597 797 769 1572 1423 2007 313 1133 1420 1536 3142
rhel6 x86_64 local storage
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 7798 3868 3889 7706 7566 10713 1080 6089 7559 7688 15562 2- 7806 3879 3874 7716 7585 10873 1114 6018 7578 7696 15572 3- 7866 3910 3855 7773 7748 11053 1076 6253 7747 7761 15710 4- 7893 3978 3931 7802 7772 10988 1117 6197 7767 7789 15760 5- 7775 3853 3809 7683 7654 10897 1081 6142 7651 7675 15534 6- 7877 3919 3872 7789 7758 10986 1085 6218 7755 7773 15720
We have used sdbox as mailbox format, and all the user data is configured in LDAP Servers
2011/5/5 Aliet Santiesteban Sifontes alietsantiesteban@gmail.com
We have done some benchmarking tests using dovecot 2.0.12 to find the best shared filesystem for hosting many users, here I share with you the results, notice the bad perfomance of all the shared filesystems against the local storage. Is there any specific optimization/tunning on dovecot for use GFS2 on rhel6??, we have configured the director to make the user mailbox persistent in a node, we will thank's any help from you. we are interested in using GFS2 or NFS, we believe the problem is the locks, how can we improve this??
best regards, Aliet
The results
rhel 4.8 x86_64/GFS1 two nodes, shared FC lun on a SAN
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 2608 1321 1311 2608 2508 3545 547 2001 2493 2702 5282 2- 2810 1440 1430 2810 2688 3835 403 2154 2679 2925 5706 3- 2913 1457 1441 2908 2778 3913 417 2253 2773 3034 5924 4- 2814 1448 1412 2812 2695 3910 401 2186 2686 2929 5712 5- 2789 1464 1432 2787 2652 3774 427 2112 2649 2879 5676 6- 2843 1460 1444 2839 2722 3948 422 2164 2713 2957 5778
rhel6 x86_64/GFS2 two nodes, shared FC lun on a SAN(Used RDM in VMWare vSphere for GFS2 lun) Tunned cluster suite cluster.conf + <dlm plock_ownership="1" plock_rate_limit="0"/>
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 2730 1340 1356 2704 2644 3748 522 2125 2643 2662 5422 2- 3309 1618 1659 3294 3223 4658 531 2563 3221 3239 6596 3- 2131 1046 1017 2055 2025 2911 381 1608 2024 2052 4256 4- 2176 1055 1039 2082 2058 2947 377 1671 2058 2078 4344 5- 1859 928 931 1859 1800 2626 304 1454 1799 1801 3706 6- 2672 1322 1329 2672 2607 3758 464 2097 2606 2615 5326
rhel6 x86_64/GFS2 two nodes, shared FC lun on a SAN(Used RDM in VMWare vSphere for GFS2 lun) Cluster suite defaults configs for plocks
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 1417 644 676 1325 1305 1872 308 1048 1302 1318 2824 2- 837 378 392 742 726 1050 117 588 722 734 1658 3- 803 363 347 752 745 1069 153 597 744 750 1658 4- 1682 802 811 1587 1569 2261 291 1299 1569 1585 3360 5- 1146 583 564 1146 1037 1500 213 811 1037 1049 2290 6- 838 403 366 744 734 1057 152 561 731 736 1664
rhel6 x86_64 two nodes used NFS(NAS Freenas 0.8, nfsvers 3)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 1382 699 691 1357 1063 1500 224 861 1053 1313 2694 2- 1634 785 799 1610 1459 2120 311 1192 1451 1570 3204 2- 1635 826 806 1611 1463 2088 345 1159 1459 1568 3190 3- 1574 758 781 1537 1403 2060 324 1135 1396 1504 3090 4- 1685 842 807 1653 1506 2135 349 1215 1504 1634 3344 5- 1766 850 893 1737 1582 2289 335 1288 1579 1705 3480 6- 1597 797 769 1572 1423 2007 313 1133 1420 1536 3142
rhel6 x86_64 local storage
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 7798 3868 3889 7706 7566 10713 1080 6089 7559 7688 15562 2- 7806 3879 3874 7716 7585 10873 1114 6018 7578 7696 15572 3- 7866 3910 3855 7773 7748 11053 1076 6253 7747 7761 15710 4- 7893 3978 3931 7802 7772 10988 1117 6197 7767 7789 15760 5- 7775 3853 3809 7683 7654 10897 1081 6142 7651 7675 15534 6- 7877 3919 3872 7789 7758 10986 1085 6218 7755 7773 15720
On 2011-05-05 7:56 PM, Aliet Santiesteban Sifontes wrote:
We have used sdbox as mailbox format, and all the user data is configured in LDAP Servers
It might help Timo to provide some suggestions if you also provide dovecot -n output... ;)
--
Best regards,
Charles
the configs:
[root@n02 ~]# dovecot -n # 2.0.12: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-71.24.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.0 (Santiago) auth_cache_size = 15 M auth_default_realm = test.com auth_mechanisms = plain login auth_worker_max_count = 60 disable_plaintext_auth = no login_greeting = Server ready. mail_fsync = never mail_location = sdbox:~/sdbox:INDEX=/vmail/index/%n mail_plugins = quota zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl mmap_disable = yes namespace { inbox = yes location = prefix = separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Sent autocreate2 = Trash autocreate3 = Drafts autocreate4 = Junk autocreate5 = Archives autocreate6 = Templates autosubscribe = Sent autosubscribe2 = Trash autosubscribe3 = Drafts autosubscribe4 = Junk autosubscribe5 = Archives autosubscribe6 = Templates quota = dict:User quota::file:%h/sdbox/dovecot-quota quota_rule = *:storage=250M quota_rule2 = Trash:storage=+50M quota_rule3 = Spam:storage=+25M quota_rule4 = Sent:ignore sieve = ~/.dovecot.sieve sieve_before = /var/vmail/sievescripts/before.d sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@test.com protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail mode = 0660 user = root } } service imap-login { service_count = 0 }
best regards
2011/5/6 Charles Marcus CMarcus@media-brokers.com
On 2011-05-05 7:56 PM, Aliet Santiesteban Sifontes wrote:
We have used sdbox as mailbox format, and all the user data is configured in LDAP Servers
It might help Timo to provide some suggestions if you also provide dovecot -n output... ;)
--
Best regards,
Charles
New results, now with all plugins disabled:
os rhel6 x86_64, GFS2 Lun
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 4514 2189 2250 4514 4264 6163 709 3403 4260 4292 8726 2- 2827 1409 1389 2827 2765 3951 495 2168 2765 2777 5644 3- 2711 1409 1368 2711 2649 3833 512 2145 2647 2662 5396 4- 1799 912 890 1799 1720 2492 360 1370 1719 1735 3592 5- 3817 1869 1896 3760 3717 5313 575 3026 3715 3737 7616 6- 3296 1583 1628 3296 3215 4585 523 2600 3215 3238 6584
2011/5/6 Aliet Santiesteban Sifontes alietsantiesteban@gmail.com
the configs:
[root@n02 ~]# dovecot -n # 2.0.12: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-71.24.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.0 (Santiago) auth_cache_size = 15 M auth_default_realm = test.com auth_mechanisms = plain login auth_worker_max_count = 60 disable_plaintext_auth = no login_greeting = Server ready. mail_fsync = never mail_location = sdbox:~/sdbox:INDEX=/vmail/index/%n mail_plugins = quota zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl mmap_disable = yes namespace { inbox = yes location = prefix = separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Sent autocreate2 = Trash autocreate3 = Drafts autocreate4 = Junk autocreate5 = Archives autocreate6 = Templates autosubscribe = Sent autosubscribe2 = Trash autosubscribe3 = Drafts autosubscribe4 = Junk autosubscribe5 = Archives autosubscribe6 = Templates quota = dict:User quota::file:%h/sdbox/dovecot-quota quota_rule = *:storage=250M quota_rule2 = Trash:storage=+50M quota_rule3 = Spam:storage=+25M quota_rule4 = Sent:ignore sieve = ~/.dovecot.sieve sieve_before = /var/vmail/sievescripts/before.d sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@test.com protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail mode = 0660 user = root } } service imap-login { service_count = 0 }
best regards
2011/5/6 Charles Marcus CMarcus@media-brokers.com
On 2011-05-05 7:56 PM, Aliet Santiesteban Sifontes wrote:
We have used sdbox as mailbox format, and all the user data is configured in LDAP Servers
It might help Timo to provide some suggestions if you also provide dovecot -n output... ;)
--
Best regards,
Charles
I don't think those results look too bad, even the original ones. imaptest doesn't measure real world performance anyway. Some ideas:
Try mdbox instead of sdbox. Cluster filesystems apparently like a few bigger files better than many small ones.
Try imaptest with logout=0 (or =1 or something). Now you're measuring way too much the login performance.
autocreate plugin sucks, especially with logout=100 because it has to check that all of the mailboxes exist. In v2.1 autocreate plugin is redesigned to not do any disk I/O.
On Fri, 2011-05-06 at 23:01 -0400, Aliet Santiesteban Sifontes wrote:
New results, now with all plugins disabled:
os rhel6 x86_64, GFS2 Lun
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 4514 2189 2250 4514 4264 6163 709 3403 4260 4292 8726 2- 2827 1409 1389 2827 2765 3951 495 2168 2765 2777 5644 3- 2711 1409 1368 2711 2649 3833 512 2145 2647 2662 5396 4- 1799 912 890 1799 1720 2492 360 1370 1719 1735 3592 5- 3817 1869 1896 3760 3717 5313 575 3026 3715 3737 7616 6- 3296 1583 1628 3296 3215 4585 523 2600 3215 3238 6584
2011/5/6 Aliet Santiesteban Sifontes alietsantiesteban@gmail.com
the configs:
[root@n02 ~]# dovecot -n # 2.0.12: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-71.24.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.0 (Santiago) auth_cache_size = 15 M auth_default_realm = test.com auth_mechanisms = plain login auth_worker_max_count = 60 disable_plaintext_auth = no login_greeting = Server ready. mail_fsync = never mail_location = sdbox:~/sdbox:INDEX=/vmail/index/%n mail_plugins = quota zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl mmap_disable = yes namespace { inbox = yes location = prefix = separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Sent autocreate2 = Trash autocreate3 = Drafts autocreate4 = Junk autocreate5 = Archives autocreate6 = Templates autosubscribe = Sent autosubscribe2 = Trash autosubscribe3 = Drafts autosubscribe4 = Junk autosubscribe5 = Archives autosubscribe6 = Templates quota = dict:User quota::file:%h/sdbox/dovecot-quota quota_rule = *:storage=250M quota_rule2 = Trash:storage=+50M quota_rule3 = Spam:storage=+25M quota_rule4 = Sent:ignore sieve = ~/.dovecot.sieve sieve_before = /var/vmail/sievescripts/before.d sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@test.com protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail mode = 0660 user = root } } service imap-login { service_count = 0 }
best regards
2011/5/6 Charles Marcus CMarcus@media-brokers.com
On 2011-05-05 7:56 PM, Aliet Santiesteban Sifontes wrote:
We have used sdbox as mailbox format, and all the user data is configured in LDAP Servers
It might help Timo to provide some suggestions if you also provide dovecot -n output... ;)
--
Best regards,
Charles
Timo, thank's for your answer, we finally found the problem, it was cluster related. We have a rhel6-x86_64 cluster using Redhat Cluster Suite and GFS2, the third node was located in an external location for Disaster Recovery, the ethernet links and fiber channel links of that facility are experimenting high latency, this was affecting cluster intercomunicate, many packets were retransmited, after we removed the third node from that facility results improved a lot. Righ now we have all the node in the same place, two shared FC luns using GFS2, one for indexes and the other for mailbox data, here the new results:
Using local storage(local hard driver ext4 filesystems)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 7798 3868 3889 7706 7566 10713 1080 6089 7559 7688 15562 7806 3879 3874 7716 7585 10873 1114 6018 7578 7696 15572 7866 3910 3855 7773 7748 11053 1076 6253 7747 7761 15710 7893 3978 3931 7802 7772 10988 1117 6197 7767 7789 15760 7775 3853 3809 7683 7654 10897 1081 6142 7651 7675 15534 7877 3919 3872 7789 7758 10986 1085 6218 7755 7773 15720
GFS2-mdbox, (no plugins)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 7547 3739 3749 7455 7421 10605 1053 5931 7417 7443 15074 7480 3702 3724 7387 7367 10558 1064 5874 7366 7378 14946 7523 3759 3711 7428 7394 10560 1126 5898 7390 7412 15014 7455 3736 3621 7364 7326 10561 1088 5854 7324 7349 14880 7431 3712 3686 7337 7312 10406 1017 5882 7311 7328 14844 7426 3704 3671 7334 7296 10364 1076 5791 7296 7325 14834 7517 3673 3782 7425 7406 10554 1103 5913 7404 7414 15008
GFS2-mdbox( using plugins)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 5464 2713 2697 5371 5201 7503 733 4152 5201 5361 10910 5649 2757 2781 5555 5500 7814 810 4397 5500 5549 11286 5303 2589 2583 5211 5147 7398 783 4067 5147 5201 10590 5446 2633 2721 5353 5280 7465 799 4272 5278 5336 10860 5628 2781 2865 5536 5467 7867 792 4317 5466 5520 11224 5699 2837 2797 5605 5543 7771 809 4416 5542 5599 11382
GFS2-sdbox(using plugins)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 6102 3008 3018 6008 5885 8395 851 4705 5882 5999 12186 6140 2963 3085 6045 6013 8534 845 4798 6011 6035 12260 6063 2997 3021 5970 5929 8568 894 4719 5926 5955 12100 5747 2805 2890 5651 5599 7956 799 4434 5598 5638 11470 6025 3000 3014 5931 5901 8476 869 4697 5898 5917 12022 5899 2863 2890 5807 5762 8249 839 4610 5761 5802 11792
We will continue the tests with your suggestions. Best regards and thank you all for a great work!! Aliet
2011/5/10 Timo Sirainen tss@iki.fi
I don't think those results look too bad, even the original ones. imaptest doesn't measure real world performance anyway. Some ideas:
Try mdbox instead of sdbox. Cluster filesystems apparently like a few bigger files better than many small ones.
Try imaptest with logout=0 (or =1 or something). Now you're measuring way too much the login performance.
autocreate plugin sucks, especially with logout=100 because it has to check that all of the mailboxes exist. In v2.1 autocreate plugin is redesigned to not do any disk I/O.
On Fri, 2011-05-06 at 23:01 -0400, Aliet Santiesteban Sifontes wrote:
New results, now with all plugins disabled:
os rhel6 x86_64, GFS2 Lun
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 4514 2189 2250 4514 4264 6163 709 3403 4260 4292 8726 2- 2827 1409 1389 2827 2765 3951 495 2168 2765 2777 5644 3- 2711 1409 1368 2711 2649 3833 512 2145 2647 2662 5396 4- 1799 912 890 1799 1720 2492 360 1370 1719 1735 3592 5- 3817 1869 1896 3760 3717 5313 575 3026 3715 3737 7616 6- 3296 1583 1628 3296 3215 4585 523 2600 3215 3238 6584
2011/5/6 Aliet Santiesteban Sifontes alietsantiesteban@gmail.com
the configs:
[root@n02 ~]# dovecot -n # 2.0.12: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-71.24.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.0 (Santiago) auth_cache_size = 15 M auth_default_realm = test.com auth_mechanisms = plain login auth_worker_max_count = 60 disable_plaintext_auth = no login_greeting = Server ready. mail_fsync = never mail_location = sdbox:~/sdbox:INDEX=/vmail/index/%n mail_plugins = quota zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl mmap_disable = yes namespace { inbox = yes location = prefix = separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Sent autocreate2 = Trash autocreate3 = Drafts autocreate4 = Junk autocreate5 = Archives autocreate6 = Templates autosubscribe = Sent autosubscribe2 = Trash autosubscribe3 = Drafts autosubscribe4 = Junk autosubscribe5 = Archives autosubscribe6 = Templates quota = dict:User quota::file:%h/sdbox/dovecot-quota quota_rule = *:storage=250M quota_rule2 = Trash:storage=+50M quota_rule3 = Spam:storage=+25M quota_rule4 = Sent:ignore sieve = ~/.dovecot.sieve sieve_before = /var/vmail/sievescripts/before.d sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@test.com protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail mode = 0660 user = root } } service imap-login { service_count = 0 }
best regards
2011/5/6 Charles Marcus CMarcus@media-brokers.com
On 2011-05-05 7:56 PM, Aliet Santiesteban Sifontes wrote:
We have used sdbox as mailbox format, and all the user data is configured in LDAP Servers
It might help Timo to provide some suggestions if you also provide dovecot -n output... ;)
--
Best regards,
Charles
On 11/05/2011 00:00, Aliet Santiesteban Sifontes wrote:
Using local storage(local hard driver ext4 filesystems)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 7798 3868 3889 7706 7566 10713 1080 6089 7559 7688 15562 7806 3879 3874 7716 7585 10873 1114 6018 7578 7696 15572 7866 3910 3855 7773 7748 11053 1076 6253 7747 7761 15710 7893 3978 3931 7802 7772 10988 1117 6197 7767 7789 15760 7775 3853 3809 7683 7654 10897 1081 6142 7651 7675 15534 7877 3919 3872 7789 7758 10986 1085 6218 7755 7773 15720
GFS2-mdbox, (no plugins)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 7547 3739 3749 7455 7421 10605 1053 5931 7417 7443 15074 7480 3702 3724 7387 7367 10558 1064 5874 7366 7378 14946 7523 3759 3711 7428 7394 10560 1126 5898 7390 7412 15014 7455 3736 3621 7364 7326 10561 1088 5854 7324 7349 14880 7431 3712 3686 7337 7312 10406 1017 5882 7311 7328 14844 7426 3704 3671 7334 7296 10364 1076 5791 7296 7325 14834 7517 3673 3782 7425 7406 10554 1103 5913 7404 7414 15008
Hi, this performance seems excellent!
There is no reason at all why you might try this, but as someone on lower end hardware I would be fascinated to learn how the performance changes is:
- Switch FC to gig ethernet? (expecting substantial performance hit?)
- Reverting to maildir (suspecting much less of a hit based on your numbers above?)
- OCFS vs GFS (although probably not sensible in your architecture since you have a support contract for GFS, some have suggested OCFS can be faster?)
Please do post any other performance results - seems like you have found an excellent cluster setup?
Ed W
Thanks Ed, right now we are finishing the setup, next week we will continue the tests and will let you know the results... best regards
2011/5/23 Ed W lists@wildgooses.com
On 11/05/2011 00:00, Aliet Santiesteban Sifontes wrote:
Using local storage(local hard driver ext4 filesystems)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 7798 3868 3889 7706 7566 10713 1080 6089 7559 7688 15562 7806 3879 3874 7716 7585 10873 1114 6018 7578 7696 15572 7866 3910 3855 7773 7748 11053 1076 6253 7747 7761 15710 7893 3978 3931 7802 7772 10988 1117 6197 7767 7789 15760 7775 3853 3809 7683 7654 10897 1081 6142 7651 7675 15534 7877 3919 3872 7789 7758 10986 1085 6218 7755 7773 15720
GFS2-mdbox, (no plugins)
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 7547 3739 3749 7455 7421 10605 1053 5931 7417 7443 15074 7480 3702 3724 7387 7367 10558 1064 5874 7366 7378 14946 7523 3759 3711 7428 7394 10560 1126 5898 7390 7412 15014 7455 3736 3621 7364 7326 10561 1088 5854 7324 7349 14880 7431 3712 3686 7337 7312 10406 1017 5882 7311 7328 14844 7426 3704 3671 7334 7296 10364 1076 5791 7296 7325 14834 7517 3673 3782 7425 7406 10554 1103 5913 7404 7414 15008
Hi, this performance seems excellent!
There is no reason at all why you might try this, but as someone on lower end hardware I would be fascinated to learn how the performance changes is:
- Switch FC to gig ethernet? (expecting substantial performance hit?)
- Reverting to maildir (suspecting much less of a hit based on your numbers above?)
- OCFS vs GFS (although probably not sensible in your architecture since you have a support contract for GFS, some have suggested OCFS can be faster?)
Please do post any other performance results - seems like you have found an excellent cluster setup?
Ed W
Timo, Can you recommend us some benchmarking tools to test the dovecot cluster setup?? Best regards
2011/5/10 Timo Sirainen tss@iki.fi
I don't think those results look too bad, even the original ones. imaptest doesn't measure real world performance anyway. Some ideas:
Try mdbox instead of sdbox. Cluster filesystems apparently like a few bigger files better than many small ones.
Try imaptest with logout=0 (or =1 or something). Now you're measuring way too much the login performance.
autocreate plugin sucks, especially with logout=100 because it has to check that all of the mailboxes exist. In v2.1 autocreate plugin is redesigned to not do any disk I/O.
On Fri, 2011-05-06 at 23:01 -0400, Aliet Santiesteban Sifontes wrote:
New results, now with all plugins disabled:
os rhel6 x86_64, GFS2 Lun
Totals: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 1- 4514 2189 2250 4514 4264 6163 709 3403 4260 4292 8726 2- 2827 1409 1389 2827 2765 3951 495 2168 2765 2777 5644 3- 2711 1409 1368 2711 2649 3833 512 2145 2647 2662 5396 4- 1799 912 890 1799 1720 2492 360 1370 1719 1735 3592 5- 3817 1869 1896 3760 3717 5313 575 3026 3715 3737 7616 6- 3296 1583 1628 3296 3215 4585 523 2600 3215 3238 6584
2011/5/6 Aliet Santiesteban Sifontes alietsantiesteban@gmail.com
the configs:
[root@n02 ~]# dovecot -n # 2.0.12: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-71.24.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.0 (Santiago) auth_cache_size = 15 M auth_default_realm = test.com auth_mechanisms = plain login auth_worker_max_count = 60 disable_plaintext_auth = no login_greeting = Server ready. mail_fsync = never mail_location = sdbox:~/sdbox:INDEX=/vmail/index/%n mail_plugins = quota zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl mmap_disable = yes namespace { inbox = yes location = prefix = separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Sent autocreate2 = Trash autocreate3 = Drafts autocreate4 = Junk autocreate5 = Archives autocreate6 = Templates autosubscribe = Sent autosubscribe2 = Trash autosubscribe3 = Drafts autosubscribe4 = Junk autosubscribe5 = Archives autosubscribe6 = Templates quota = dict:User quota::file:%h/sdbox/dovecot-quota quota_rule = *:storage=250M quota_rule2 = Trash:storage=+50M quota_rule3 = Spam:storage=+25M quota_rule4 = Sent:ignore sieve = ~/.dovecot.sieve sieve_before = /var/vmail/sievescripts/before.d sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@test.com protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail mode = 0660 user = root } } service imap-login { service_count = 0 }
best regards
2011/5/6 Charles Marcus CMarcus@media-brokers.com
On 2011-05-05 7:56 PM, Aliet Santiesteban Sifontes wrote:
We have used sdbox as mailbox format, and all the user data is configured in LDAP Servers
It might help Timo to provide some suggestions if you also provide dovecot -n output... ;)
--
Best regards,
Charles
On Sun, 2011-05-22 at 08:46 -0400, Aliet Santiesteban Sifontes wrote:
Timo, Can you recommend us some benchmarking tools to test the dovecot cluster setup??
See http://www.imapwiki.org/Benchmarking - the bottom lists all the benchmarking tools I know of.
participants (4)
-
Aliet Santiesteban Sifontes
-
Charles Marcus
-
Ed W
-
Timo Sirainen