Awfully slow dovecot
Hi,
We’re using dovecot 1.0.7, which seems to be the latest version available on CentOS 5.
Downloading emails are dead slow. Really small emails goes quickly, but normal emails and emails with attachments are so slow to download it’s almost ridiculous. I’ve googled some and found that it could be related to quota, but I disabled the quota plugins on imap with no difference.
dovecot -n: # 1.0.7: /etc/dovecot.conf ssl_cert_file: /etc/dovecot/cert.pem ssl_key_file: /etc/dovecot/key.pem disable_plaintext_auth: yes verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login valid_chroot_dirs: /var/mail/domains verbose_proctitle: yes last_valid_uid: 500 mail_location: maildir:/var/mail/domains/%d/%n/mail mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): mail_plugins(imap): mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 auth default: mechanisms: plain login passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: mail plugin: convert_mail: maildir:/var/mail/domains/%d/%u/mail
dovecot-ldap.conf: auth_bind = yes hosts = example.com ldap_version = 3 base = o=hosting,dc=example,dc=com dn = cn=phamm,o=hosting,dc=example,dc=com dnpass = password pass_attrs = mail pass_filter = (&(objectClass=VirtualMailAccount)(accountActive=TRUE)(mail=%u)) user_attrs = mail,,,mail,,,quota=quota=maildir:storage user_filter = (&(objectClass=VirtualMailAccount)(accountActive=TRUE)(mail=%u)) deref = never scope = subtree default_pass_scheme = MD5 user_global_uid = 500 user_global_gid = 12
Am 18.12.2014 um 17:56 schrieb Robin Helgelin:
We’re using dovecot 1.0.7
that version is total out of date , update to recent version
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Am 25.12.2014 um 21:09 schrieb Benny Pedersen:
Robert Schetterer skrev den 2014-12-25 19:49:
We’re using dovecot 1.0.7
Am 18.12.2014 um 17:56 schrieb Robin Helgelin: that version is total out of date , update to recent version
centos is a precompiled problem :=)
no it is not
do you realy think the RPMS are falling from heaven or is it more likely be able to use rpmbuild as i do on Fedora for packages like dovecot-2.2.15-3.fc20.20141025.rh.x86_64 or postfix-2.11.3-1.fc20.20141020.rh.x86_64?
your Gentoo is nice in a small environment
on larger setups someone is using binary packages and can setup his own repo with overrides while maintain *testable* setups
On Dec 25, 2014 3:15 PM, "Reindl Harald" h.reindl@thelounge.net wrote:
Am 25.12.2014 um 21:09 schrieb Benny Pedersen:
Robert Schetterer skrev den 2014-12-25 19:49:
Am 18.12.2014 um 17:56 schrieb Robin Helgelin:
We’re using dovecot 1.0.7
that version is total out of date , update to recent version
centos is a precompiled problem :=)
no it is not
do you realy think the RPMS are falling from heaven or is it more likely
be able to use rpmbuild as i do on Fedora for packages like dovecot-2.2.15-3.fc20.20141025.rh.x86_64 or postfix-2.11.3-1.fc20.20141020.rh.x86_64?
your Gentoo is nice in a small environment
on larger setups someone is using binary packages and can setup his own
repo with overrides while maintain *testable* setups
Just to point out, it is possible to set up a binary Gentoo setup with a single server compiling packages then made available to downstream computers -- I ran such a setup for a few years. Can also have multiple of these in an overlay fashion for testing. Pros and cons vs. normal binary distros, but it can be done.
Anyways, regarding the OP's problem, 1.0.7 is only the latest available package from RedHat/CentOS. It's so out of date and so many bugs have been squashed that it makes little sense for anyone to spend much time trying to figure out the problem. Even Red Hat doesn't support it in production anymore.
Might be time to break out the compiler.
On 12/26/14, Jeff Mitchell jeffrey.mitchell@gmail.com wrote:
On Dec 25, 2014 3:15 PM, "Reindl Harald" h.reindl@thelounge.net wrote:
your Gentoo is nice in a small environment
on larger setups someone is using binary packages and can setup his own
repo with overrides while maintain *testable* setups
Just to point out, it is possible to set up a binary Gentoo setup with a single server compiling packages then made available to downstream computers -- I ran such a setup for a few years. Can also have multiple of these in an overlay fashion for testing. Pros and cons vs. normal binary distros, but it can be done.
As we do today for some 417 servers (real servers, not virtual crap), its very easy to do, even my previous employer who used slackware with a few hundred servers used almost identical fashion.
Amazing at how rpm and deb users think they are the only ones in this world who can manage large enterprise server farms, just shows how narrow sighted and ill-informed they are.
Am 26.12.2014 um 02:20 schrieb Edwardo Garcia:
On 12/26/14, Jeff Mitchell jeffrey.mitchell@gmail.com wrote:
On Dec 25, 2014 3:15 PM, "Reindl Harald" h.reindl@thelounge.net wrote:
your Gentoo is nice in a small environment
on larger setups someone is using binary packages and can setup his own
repo with overrides while maintain *testable* setups
Just to point out, it is possible to set up a binary Gentoo setup with a single server compiling packages then made available to downstream computers -- I ran such a setup for a few years. Can also have multiple of these in an overlay fashion for testing. Pros and cons vs. normal binary distros, but it can be done.
As we do today for some 417 servers (real servers, not virtual crap), its very easy to do, even my previous employer who used slackware with a few hundred servers used almost identical fashion.
Amazing at how rpm and deb users think they are the only ones in this world who can manage large enterprise server farms, just shows how narrow sighted and ill-informed they are.
narrow sighted are people thinking others are ill-informed or as Benny thinking outdated RPM packages are a persistent problem not easily solveable
sure, you can manage anything if you write enough tools to automate things, nothing new for me as software developer, but don't you think there is a reason why advanced package management exists and 95% of all production environments are uusing them?
and if it is only to have a *formal verification* based on the rpm database that there are no dep errors and compare 100, 200, 1000 machine setups automated with a single click
On 12/26/14, Reindl Harald h.reindl@thelounge.net wrote:
Am 26.12.2014 um 02:20 schrieb Edwardo Garcia:
On 12/26/14, Jeff Mitchell jeffrey.mitchell@gmail.com wrote:
On Dec 25, 2014 3:15 PM, "Reindl Harald" h.reindl@thelounge.net wrote:
your Gentoo is nice in a small environment
on larger setups someone is using binary packages and can setup his own
repo with overrides while maintain *testable* setups
Just to point out, it is possible to set up a binary Gentoo setup with a single server compiling packages then made available to downstream computers -- I ran such a setup for a few years. Can also have multiple of these in an overlay fashion for testing. Pros and cons vs. normal binary distros, but it can be done.
As we do today for some 417 servers (real servers, not virtual crap), its very easy to do, even my previous employer who used slackware with a few hundred servers used almost identical fashion.
Amazing at how rpm and deb users think they are the only ones in this world who can manage large enterprise server farms, just shows how narrow sighted and ill-informed they are.
narrow sighted are people thinking others are ill-informed or as Benny thinking outdated RPM packages are a persistent problem not easily solveable
sure, you can manage anything if you write enough tools to automate things, nothing new for me as software developer, but don't you think there is a reason why advanced package management exists and 95% of all production environments are uusing them?
it takes no more than a few minutes to write a perl script to handle all. and you can not claim 95% of anything in real world, even if so, there is no difference to automated tools, than yum or apt, they can do the same thing and as every machine is identical, if work on dev box, there is no way it not work on production.
its simple, if it is not work on rpm, erase rpm and use source.
it is silly and time waste to try log bug problem with version not supported in years
Am 26.12.2014 um 17:16 schrieb Nick Edwards:
On 12/26/14, Reindl Harald h.reindl@thelounge.net wrote:
sure, you can manage anything if you write enough tools to automate things, nothing new for me as software developer, but don't you think there is a reason why advanced package management exists and 95% of all production environments are uusing them?
it takes no more than a few minutes to write a perl script to handle all. and you can not claim 95% of anything in real world, even if so, there is no difference to automated tools, than yum or apt, they can do the same thing and as every machine is identical, if work on dev box, there is no way it not work on production.
deployment yes
versioned, clean downgrades and preserve permissions, get rid of obsolete files to keep the system clean over many years take more effort
its simple, if it is not work on rpm, erase rpm and use source.
it is silly and time waste to try log bug problem with version not supported in years
hence i recommended use rpmbuild and build a *override* from recent source, in case of dovecot just build from source may be easy, if it comes to dependencies rpm become the easier and safer way because it would refuse to override incompatible libraries until you take care of the dependencie tree which does not come from rpm itself but is managed by using it
binary packages vs. compiled has nothing to do with the op issue. The problem is an outdated version of the software. Update by whatever means necessary and get on with your day. On Dec 25, 2014, at 7:11 PM, Jeff Mitchell wrote:
On Dec 25, 2014 3:15 PM, "Reindl Harald" h.reindl@thelounge.net wrote:
Am 25.12.2014 um 21:09 schrieb Benny Pedersen:
Robert Schetterer skrev den 2014-12-25 19:49:
Am 18.12.2014 um 17:56 schrieb Robin Helgelin:
We’re using dovecot 1.0.7
that version is total out of date , update to recent version
centos is a precompiled problem :=)
no it is not
do you realy think the RPMS are falling from heaven or is it more likely
be able to use rpmbuild as i do on Fedora for packages like dovecot-2.2.15-3.fc20.20141025.rh.x86_64 or postfix-2.11.3-1.fc20.20141020.rh.x86_64?
your Gentoo is nice in a small environment
on larger setups someone is using binary packages and can setup his own
repo with overrides while maintain *testable* setups
Just to point out, it is possible to set up a binary Gentoo setup with a single server compiling packages then made available to downstream computers -- I ran such a setup for a few years. Can also have multiple of these in an overlay fashion for testing. Pros and cons vs. normal binary distros, but it can be done.
Anyways, regarding the OP's problem, 1.0.7 is only the latest available package from RedHat/CentOS. It's so out of date and so many bugs have been squashed that it makes little sense for anyone to spend much time trying to figure out the problem. Even Red Hat doesn't support it in production anymore.
Might be time to break out the compiler.
Am 25.12.2014 um 21:07 schrieb Benny Pedersen:
We’re using dovecot 1.0.7, which seems to be the latest version available on CentOS 5.
update to centos 7
if dovecot still not working, report again
And if it is then still slow because some people have folders with shitloads of emails, you may consider switching your mail storage over to mdbox with enabled compression.
Marc Stürmer skrev den 2014-12-26 09:42:
And if it is then still slow because some people have folders with shitloads of emails, you may consider switching your mail storage over to mdbox with enabled compression.
using dovecot 1.2.17 here with maildir, still have to see performance issues, but yes will move to ssd soon, i will just fix problems that are here, not things that are not a problem, the above will kill my server, so need to have an intel i7 with 25MB L1 cache, so far i just keep it simple
Zitat von Benny Pedersen me@junc.eu:
using dovecot 1.2.17 here with maildir, still have to see
performance issues, but yes will move to ssd soon, i will just fix
problems that are here, not things that are not a problem, the above
will kill my server, so need to have an intel i7 with 25MB L1 cache,
so far i just keep it simple
Look... if you want to get a real speedup in I/O on the same hardware
without getting a SSD, you should grab yourself a recent dovecot
version (2.2.X), enable gzip compression and switch over to mdbox.
Maildir is fine as long as you don't have too much mail on your
storage, but there comes a point when you are getting big enough where
Maildir really isn't going to behave really nicely anymore, because
too many files and way too many seeks. Mdbox is quite different and
scales beyond the point of Maildir with ease.
On 12/26/14, Marc Stürmer mail@marc-stuermer.de wrote:
Maildir is fine as long as you don't have too much mail on your storage, but there comes a point when you are getting big enough where Maildir really isn't going to behave really nicely anymore, because too many files and way too many seeks. Mdbox is quite different and scales beyond the point of Maildir with ease.
that may only be true depending om your filesystem
it may be no more supported for obvious reasons, but reiserfs will never be beaten for maildir and maildir is time tested and proven in very large environments with millions of users, with many GB quotas runs perfect with depuping on netapps too.
Am 26.12.2014 um 17:21 schrieb Nick Edwards:
Maildir is fine as long as you don't have too much mail on your storage, but there comes a point when you are getting big enough where Maildir really isn't going to behave really nicely anymore, because too many files and way too many seeks. Mdbox is quite different and scales beyond the point of Maildir with ease.
that may only be true depending om your filesystem
it may be no more supported for obvious reasons, but reiserfs will never be beaten for maildir and maildir is time tested and proven in very large environments with millions of users, with many GB quotas runs perfect with depuping on netapps too.
Wrong. ReiserFS is something you should really avoid nowadays on servers. V3, while stable and not getting much love nor features anyway, has some serious known quirks. There's a reason why Reiser4 was invented back then.
ReiserFS4 will almost certainly never find its way in the kernel and goesn't get much love either, especially since its author went into jail for murder.
Because Reiser4 is being maintained outside the kernel you have either to compile the kernels yourself or get community kernels. Aside that, Reiser4 is still experimental and not stable.
Aside from some still raging and alive fanboys who didn't get the message ReiserFS doesn't really matter anymore. It had some nice ideas, but people moved on to other systems and are happy with those.
The thing is, that Maildir means every mail is one file and you've got to maintain some index files to get a decent speed out of it. Robust, also yes, because all data is in the file system itself.
This means for every modern file system, regardless if you are using XFS, ext4 or Reiser, that you need to lookup that file in the folder and read/write it. All modern file systems use for that some kind of binary tree algorithm.
Some older file systems tend do slow down drastically if you put enough mails in one folder. Just consider one mail folder with the contents of the LKML of one year, this folder alone would be around 100.000 files or even more.
And if your machines are getting big enough, Maildir plainly sucks and making backups takes more and more time, because reading 100000 files for a backup means many seeks and lots of work for the HDD, it means of course much more protocol overhead e.g. on rsync.
For example my mailbox had a lkml archive with over > 30000 mails and I switched to mdbox. On the pro side you get much more speed out of your hardware, because you can do more with less I/O operations. On the con side you cannot read mails on the file system level anymore and need to learn the dovecot tool chain, which though IMHO is absolutely worth it, backups need proper preparations and losing the mapping files would be a desaster.
Now instead of maybe around 60000 files I had and taking over 800 MB my mail storage consists of 347 files and takes 485 MB and I didn't throw any mail away.
How comes? Simple, I enabled compressed saving with gzip. CPU power is cheap, getting more memory and hdd speed is most time not. Because of all my mails are now compressed in much less files on the HDD, looking them up in that folder is blazingly fast, because it's quite a small tree needed to maintain those.
And it also means that I can use the same amount of memory to cache more mails in the file system cache (roughly around 30-40% more), because the data itself is compressed. That's the beauty of it, and much faster backups again.
If you really run millions of accounts, you wouldn't want Maildir anymore when you can have mdbox.
If you want to build a really large imap server on Linux, either take ext4 or XFS as file system.
participants (9)
-
Benny Pedersen
-
Edgar Pettijohn III
-
Edwardo Garcia
-
Jeff Mitchell
-
Marc Stürmer
-
Nick Edwards
-
Reindl Harald
-
Robert Schetterer
-
Robin Helgelin