[Dovecot] Out of memory after upgrading to dovecot 2.0.21
Hello,
I've upgraded our servers and now dovecot seems to be running out of memory when accessing my own mailbox (and only mine, which is in a way pretty fortunate):
dovecot: imap(abelard-a): Error: mmap() failed with file /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory
imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory
/vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is 72% full of 8GB.
any idea of what is happening?
Thanks in advance,
Arnaud
-- Arnaud Abélard jabber: arnaud.abelard@univ-nantes.fr / twitter: ArnY Administrateur Système DSI Université de Nantes
Hrm... exactly when I pressed sent, dovecot started giving me more information:
master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it))
Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were running...).
Arnaud
On 02/05/2013 06:33 PM, Arnaud Abélard wrote:
Hello,
I've upgraded our servers and now dovecot seems to be running out of memory when accessing my own mailbox (and only mine, which is in a way pretty fortunate):
dovecot: imap(abelard-a): Error: mmap() failed with file /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory
imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory
/vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is 72% full of 8GB.
any idea of what is happening?
Thanks in advance,
Arnaud
-- Arnaud Abélard jabber: arnaud.abelard@univ-nantes.fr / twitter: ArnY Administrateur Système DSI Université de Nantes
hard to say without config-details
shared prcoess or one per connection? in case of shared 256 MB is really really low
Am 05.02.2013 18:38, schrieb Arnaud Abélard:
Hrm... exactly when I pressed sent, dovecot started giving me more information:
master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it))
Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were running...).
Arnaud
On 02/05/2013 06:33 PM, Arnaud Abélard wrote:
Hello,
I've upgraded our servers and now dovecot seems to be running out of memory when accessing my own mailbox (and only mine, which is in a way pretty fortunate):
dovecot: imap(abelard-a): Error: mmap() failed with file /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory
imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory
/vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is 72% full of 8GB.
any idea of what is happening?
On 02/05/2013 06:44 PM, Reindl Harald wrote:
hard to say without config-details
shared prcoess or one per connection?
One per connection, around 1000 connections right now.
in case of shared 256 MB is really really low
I changed the vsz_limit to 512MB and it seems better, but I'm still surprised my mailbox actually hit the memory limit since I doubt it's most used one.
Arnaud
Am 05.02.2013 18:38, schrieb Arnaud Abélard:
Hrm... exactly when I pressed sent, dovecot started giving me more information:
master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it))
Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were running...).
Arnaud
On 02/05/2013 06:33 PM, Arnaud Abélard wrote:
Hello,
I've upgraded our servers and now dovecot seems to be running out of memory when accessing my own mailbox (and only mine, which is in a way pretty fortunate):
dovecot: imap(abelard-a): Error: mmap() failed with file /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory
imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory
/vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is 72% full of 8GB.
any idea of what is happening?
-- Arnaud Abélard jabber: arnaud.abelard@univ-nantes.fr / twitter: ArnY Administrateur Système DSI Université de Nantes
On 2/5/2013 12:15 PM, Arnaud Abélard wrote:
I changed the vsz_limit to 512MB and it seems better, but I'm still surprised my mailbox actually hit the memory limit since I doubt it's most used one.
According to the wiki, vsz_limit only affects login processes, not IMAP processes, which is odd given the second error you posted, which seems to indicate a relationship between vsz_limit and imap.
dovecot: imap(abelard-a): Error: mmap() failed with file /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory
imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory
This error suggests your mail_process_size variable is set too low. If it is currently set to 128MB, increase it to 512MB. The bulk of this memory is used for mmap()ing files, and is virtual, not physical. Thus you don't actually need 51GB of RAM to support 100 users.
Also, a dovecot.index.cache file of size 224MB seems rather large. My largest list mail folder is 150MB, contains 17,647 msgs, and has a dovecot.index.cache file of only 16MB. I use mbox. This would seem to suggest you may have a single folder with tens of thousands of msgs in it. Or maybe indexes are handled differently for your mailbox format--I've not used any others. If the former, I suggest you cull your large folders to get them down to manageable size.
-- Stan
On 02/05/2013 09:25 PM, Stan Hoeppner wrote:
On 2/5/2013 12:15 PM, Arnaud Abélard wrote:
I changed the vsz_limit to 512MB and it seems better, but I'm still surprised my mailbox actually hit the memory limit since I doubt it's most used one.
According to the wiki, vsz_limit only affects login processes, not IMAP processes, which is odd given the second error you posted, which seems to indicate a relationship between vsz_limit and imap.
Actually, service { vsz_limit } replaced mail_process_size (at least according to what dovecot said upon restart...) so service imap { vsz_limit=512MB } which i added earlier actually does what you suggested.
dovecot: imap(abelard-a): Error: mmap() failed with file /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory
imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory
This error suggests your mail_process_size variable is set too low. If it is currently set to 128MB, increase it to 512MB. The bulk of this memory is used for mmap()ing files, and is virtual, not physical. Thus you don't actually need 51GB of RAM to support 100 users.
Also, a dovecot.index.cache file of size 224MB seems rather large. My largest list mail folder is 150MB, contains 17,647 msgs, and has a dovecot.index.cache file of only 16MB. I use mbox. This would seem to suggest you may have a single folder with tens of thousands of msgs in it. Or maybe indexes are handled differently for your mailbox format--I've not used any others. If the former, I suggest you cull your large folders to get them down to manageable size.
Yep I have a huge INBOX folder right now, I haven't archived my 2011 mail yet (I keep the previous year in my INBOX).
Thanks,
Arnaud
-- Arnaud Abélard jabber: arnaud.abelard@univ-nantes.fr / twitter: ArnY Administrateur Système DSI Université de Nantes
participants (3)
-
Arnaud Abélard
-
Reindl Harald
-
Stan Hoeppner