mremap_anon() failed: Not enough space
Hey folks. Suddenly I'm getting lots and lots of messages like this in my logs:
Jun 19 14:47:31 <hostname> dovecot: [ID 583609 local0.error] imap(<address>): Error: mremap_anon(/var/mail/<domain>/<uname>/mailboxes/INBOX/Trash/dbox-Mails/dovecot.index.cache, 255557632) failed: Not enough space
I'm running 2.2.36.1 under Solaris 10 (patched to current) on UltraSPARC. There's plenty of memory, plenty of swap, and plenty of disk, but this is a fairly busy mail server.
Can anyone point me in the right direction? I'm guessing I have to increase a vsz_limit somewhere, but where? It's not clear to me exactly what is running out of what here. The output of doveconf -n is pasted below.
Thanks,
-Dave$ doveconf -n # 2.2.36.1 (5d621cf65): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: SunOS 5.10 sun4v # Hostname: mail.neurotica.com auth_failure_delay = 5 secs auth_mechanisms = plain login base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug disable_plaintext_auth = no first_valid_uid = 200 login_access_sockets = tcpwrap mail_fsync = never mail_plugins = fts fts_solr managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader mdbox_rotate_size = 64 M mmap_disable = yes namespace inbox { inbox = yes location = mailbox uncaught-spam { auto = subscribe special_use = \Junk } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ sieve = /var/sieve-scripts/%u.sieve sieve_dir = ~/sieve sieve_extensions = +editheader sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0600 user = vmail } user = root } service imap-login { process_min_avail = 10 service_count = 10 vsz_limit = 512 M } service imap { executable = /usr/local/libexec/dovecot/imap vsz_limit = 512 M } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 1 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 10 } service pop3-login { process_min_avail = 10 service_count = 10 vsz_limit = 512 M } service pop3 { executable = /usr/local/libexec/dovecot/pop3 } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl_cert =
-- Dave McGuire, AK4HZ New Kensington, PA
Hey folks. Suddenly I'm getting lots and lots of messages like this in my logs:
Jun 19 14:47:31 <hostname> dovecot: [ID 583609 local0.error] imap(<address>): Error: mremap_anon(/var/mail/<domain>/<uname>/mailboxes/INBOX/Trash/dbox-Mails/dovecot.index.cache, 255557632) failed: Not enough space
Hi,
Maybe inodes are exhausted (df -o i ) ?
Petri
On 19 Jun 2019, at 13:13, Dave McGuire via dovecot dovecot@dovecot.org wrote:
Jun 19 14:47:31 <hostname> dovecot: [ID 583609 local0.error] imap(<address>): Error: mremap_anon(/var/mail/<domain>/<uname>/mailboxes/INBOX/Trash/dbox-Mails/dovecot.index.cache, 255557632) failed: Not enough space
I'm running 2.2.36.1 under Solaris 10 (patched to current) on UltraSPARC. There's plenty of memory, plenty of swap, and plenty of disk, but this is a fairly busy mail server.
Are you sure there is plenty of space on /var/mail/ ? Because that should only show up when the mount point is full.
Also, what is your definition of “plenty”? (I have some index files in the 50MB range, and I am sure there are people with indexes much larger than that).
-- Lead me not into temptation, I can find the way.
On 6/20/19 6:07 AM, @lbutlr via dovecot wrote:
Jun 19 14:47:31 <hostname> dovecot: [ID 583609 local0.error] imap(<address>): Error: mremap_anon(/var/mail/<domain>/<uname>/mailboxes/INBOX/Trash/dbox-Mails/dovecot.index.cache, 255557632) failed: Not enough space
I'm running 2.2.36.1 under Solaris 10 (patched to current) on UltraSPARC. There's plenty of memory, plenty of swap, and plenty of disk, but this is a fairly busy mail server.
Are you sure there is plenty of space on /var/mail/ ? Because that should only show up when the mount point is full.
Yes, there's about 3TB available there.
Also, what is your definition of “plenty”? (I have some index files in the 50MB range, and I am sure there are people with indexes much larger than that).
The index file in question was 255MB. In desperation, and watching the other thread going on about index files, I moved dovecot.index.cache aside and let it be re-created. No further issues have appeared in the log, and the new dovecot.index.cache file (created yesterday) has grown to about 120KB.
The spool in question was about 3GB, with about 100K messages in it. I've never seen a dovecot.index.cache file grow so large; does that seem reasonable to you?
-Dave-- Dave McGuire, AK4HZ New Kensington, PA
On 20 Jun 2019, at 07:46, Dave McGuire via dovecot dovecot@dovecot.org wrote:
On 6/20/19 6:07 AM, @lbutlr via dovecot wrote:
Jun 19 14:47:31 <hostname> dovecot: [ID 583609 local0.error] imap(<address>): Error: mremap_anon(/var/mail/<domain>/<uname>/mailboxes/INBOX/Trash/dbox-Mails/dovecot.index.cache, 255557632) failed: Not enough space
I'm running 2.2.36.1 under Solaris 10 (patched to current) on UltraSPARC. There's plenty of memory, plenty of swap, and plenty of disk, but this is a fairly busy mail server.
Are you sure there is plenty of space on /var/mail/ ? Because that should only show up when the mount point is full.
Yes, there's about 3TB available there.
Also, what is your definition of “plenty”? (I have some index files in the 50MB range, and I am sure there are people with indexes much larger than that).
The index file in question was 255MB. In desperation, and watching the other thread going on about index files, I moved dovecot.index.cache aside and let it be re-created. No further issues have appeared in the log, and the new dovecot.index.cache file (created yesterday) has grown to about 120KB.
The spool in question was about 3GB, with about 100K messages in it. I've never seen a dovecot.index.cache file grow so large; does that seem reasonable to you?
They seem to grow. As I meant to say, but typoed, I have some in the range of 500MB here (not 50MB, though I have those too). I think deleting the index.cache every now and then is probably reasonable but I don’t know what a reasonable time frame is. Yearly? I might start with yearly. I removed them all yeterday shortly after posting to see how things would go.
# exa -l /usr/local/virtual/*/*/.*/dove* | sort -k 2 -h
They are much much smaller (under 10MB), but I am sure they will grow as users access the larger folders on their mail store (it’s not unusual for user to have 4-5GB of email in their .Archive mail store).
It’s also possible that the cache files USED to grow quite large in previous versions and that now they are much more reasonable, but the only way to reset their size appears to be to delete them and let dovecot rebuild them.
Anyway, if you get any sort of index errors in the future the first trouble shooting step is to delete the index file. If that doesn’t work, delete all the index files for that folder. Then if that still didn’t fix it you start looking for other causes.
participants (4)
-
unknown＠example.com
-
＠lbutlr
-
Dave McGuire
-
Petri Mäenpää