[Dovecot] Quota over NFS with inode limit
I tried searching this list, but couldn't find anything related to
inode limits except for linux quotas.
I wanted to add sieve support for our users, so was attempting to get
our email system to use dovecot's deliver lda to handle email, instead
of postfix's virtual lda, using maildir mailstore.
When I did this, I noticed that emails would get stuck in the queue if
a user was over quota, this got fixed when I added in quota support,
and set it up to check it using the line:
quota = fs:User quota:user:inode_per_mail
Now if a user excedes disk space via quota, it's bounced with a message.
But if a user excedes their inode quota, it's still tempfail and hangs
in the queue till it expires.
I just wanted to know if anyone knows about this issue, if I made a
stupid mistake or anything else, before I go digging into the source.
Looking into this some more, all the log entries that cause this have
to do with rename and EFBIG.
deliver(xxx):
rename(/var/mail/virtual/xxx/tmp/1244513712.M557487P17017.5011.mail,
/var/mail/virtual/xxx/new/1244513712.M557487P17017.5011.mail,W=2019)
failed: File too large
I do see other accounts that are at/over quota for blocksize and
inodes, and they are all failing on open or fsync with DIsk quota
exceeded (EDQUOT I assume)
Quoting Patrick Domack patrickdk@patrickdk.com:
I tried searching this list, but couldn't find anything related to inode limits except for linux quotas.
I wanted to add sieve support for our users, so was attempting to get our email system to use dovecot's deliver lda to handle email, instead of postfix's virtual lda, using maildir mailstore.
When I did this, I noticed that emails would get stuck in the queue if a user was over quota, this got fixed when I added in quota support, and set it up to check it using the line:
quota = fs:User quota:user:inode_per_mail
Now if a user excedes disk space via quota, it's bounced with a message. But if a user excedes their inode quota, it's still tempfail and hangs in the queue till it expires.
I just wanted to know if anyone knows about this issue, if I made a stupid mistake or anything else, before I go digging into the source.
On Jun 10, 2009, at 1:51 PM, Patrick Domack wrote:
Looking into this some more, all the log entries that cause this
have to do with rename and EFBIG.deliver(xxx): rename(/var/mail/virtual/xxx/tmp/ 1244513712.M557487P17017.5011.mail, /var/mail/virtual/xxx/new/ 1244513712.M557487P17017.5011.mail,W=2019) failed: File too large
That looks weird. So this is with Linux as NFS client? What do you use
as NFS server? To me this feels like NFS server bug..
Opps, relized I didn't include the list.
I'm using a netapp filer with ontap 7.2.2 software.
I searched netapp's bug database but can't find anything related to
this being reported (so far).
Quoting Timo Sirainen tss@iki.fi:
On Jun 10, 2009, at 1:51 PM, Patrick Domack wrote:
Looking into this some more, all the log entries that cause this
have to do with rename and EFBIG.deliver(xxx):
rename(/var/mail/virtual/xxx/tmp/1244513712.M557487P17017.5011.mail,
/var/mail/virtual/xxx/new/1244513712.M557487P17017.5011.mail,W=2019)
failed: File too largeThat looks weird. So this is with Linux as NFS client? What do you use as NFS server? To me this feels like NFS server bug..
Hmm, looking at how postfix handles this, it hardlinks the files, so I
guess I could turn on hardlinking that way rename isn't used, and
won't get this error.
But probably still something to look into?
Quoting Timo Sirainen tss@iki.fi:
On Jun 10, 2009, at 1:51 PM, Patrick Domack wrote:
Looking into this some more, all the log entries that cause this
have to do with rename and EFBIG.deliver(xxx):
rename(/var/mail/virtual/xxx/tmp/1244513712.M557487P17017.5011.mail,
/var/mail/virtual/xxx/new/1244513712.M557487P17017.5011.mail,W=2019)
failed: File too largeThat looks weird. So this is with Linux as NFS client? What do you use as NFS server? To me this feels like NFS server bug..
Dovecot always renames files when moving from tmp/, you can't make it
use hard links. The maildir_copy_with_hardlinks is only about how to
copy messages from one mailbox to another, and it's enabled by default
in v1.1+.
Wonder if this affects rename() only when moving from one directory to
another, or also within a directory? So if you're out of inodes and
you change a message flag, does rename() succeed?
On Jun 11, 2009, at 8:34 AM, Patrick Domack wrote:
Hmm, looking at how postfix handles this, it hardlinks the files, so
I guess I could turn on hardlinking that way rename isn't used, and
won't get this error.But probably still something to look into?
Quoting Timo Sirainen tss@iki.fi:
On Jun 10, 2009, at 1:51 PM, Patrick Domack wrote:
Looking into this some more, all the log entries that cause this
have to do with rename and EFBIG.deliver(xxx): rename(/var/mail/virtual/xxx/tmp/ 1244513712.M557487P17017.5011.mail, /var/mail/virtual/xxx/new/ 1244513712.M557487P17017.5011.mail,W=2019) failed: File too large
That looks weird. So this is with Linux as NFS client? What do you
use as NFS server? To me this feels like NFS server bug..
Hmm, there are no log messages for the last two weeks that say a
rename failed for imap services at all. The user account this was
happening on has had their quota problem fixed, and now I am unable to
replicate it :(
The only thing I did notice is the user was at 1 inode below thier
hardlimit, and wasn't anywhere near their block limit.
I guess this is going on hold till I can replicate the issue again,
but here is some log info, and I didn't see any notice in postfix
about looking for this error code, but it sees it and bounces on it,
so maybe I should dig into it's code some more.
deliver(xxx): msgid=<xxx>: save failed to INBOX: Internal error
occurred. Refer to server log for more information. [2009-06-08
22:15:12]
deliver(xxx):
rename(/var/mail/xxx/tmp/1244513712.M557487P17017.5011.mail,
/var/mail/xxx/new/1244513712.M557487P17017.5011.mail,W=2019) failed:
File too large
postfix/pipe[17014]: 68DFC7066D: to=<xxx>, relay=dovecot, delay=72,
delays=72/0.02/0/0.04, dsn=4.3.0, status=deferred (temporary failure)
mail_bulk/virtual[13809]: 552706F2AD: to=<xxx>, relay=virtual,
delay=0.04, delays=0.01/0.01/0/0.02, dsn=5.2.2, status=bounced
(maildir delivery failed: create maildir file
/var/mail/xxx/new/1244513678.V15I307e5e5M366990.5012.mail: File too
large)
Quoting Timo Sirainen tss@iki.fi:
Dovecot always renames files when moving from tmp/, you can't make it use hard links. The maildir_copy_with_hardlinks is only about how to copy messages from one mailbox to another, and it's enabled by default in v1.1+.
Wonder if this affects rename() only when moving from one directory to another, or also within a directory? So if you're out of inodes and you change a message flag, does rename() succeed?
On Jun 11, 2009, at 8:34 AM, Patrick Domack wrote:
Hmm, looking at how postfix handles this, it hardlinks the files,
so I guess I could turn on hardlinking that way rename isn't used,
and won't get this error.But probably still something to look into?
Quoting Timo Sirainen tss@iki.fi:
On Jun 10, 2009, at 1:51 PM, Patrick Domack wrote:
Looking into this some more, all the log entries that cause this
have to do with rename and EFBIG.deliver(xxx):
rename(/var/mail/virtual/xxx/tmp/1244513712.M557487P17017.5011.mail,
/var/mail/virtual/xxx/new/1244513712.M557487P17017.5011.mail,W=2019)
failed: File too largeThat looks weird. So this is with Linux as NFS client? What do you use as NFS server? To me this feels like NFS server bug..
And as is always the case, you find the real cause of the problem when
you give up :)
[wafl.dir.size.max:warning]: Directory /var/mail/xxx/new/ reached the
maxdirsize limit. Reduce the number of files or use the vol options
command to increase this limit.
So it's a netapp max files per directory limit, that causes EFBIG
error to be returned, not a quota limit, just a chance that they where
at both at.
Quoting Patrick Domack patrickdk@patrickdk.com:
Hmm, there are no log messages for the last two weeks that say a rename failed for imap services at all. The user account this was happening on has had their quota problem fixed, and now I am unable to replicate it :(
The only thing I did notice is the user was at 1 inode below thier hardlimit, and wasn't anywhere near their block limit.
I guess this is going on hold till I can replicate the issue again, but here is some log info, and I didn't see any notice in postfix about looking for this error code, but it sees it and bounces on it, so maybe I should dig into it's code some more.
deliver(xxx): msgid=<xxx>: save failed to INBOX: Internal error occurred. Refer to server log for more information. [2009-06-08 22:15:12] deliver(xxx): rename(/var/mail/xxx/tmp/1244513712.M557487P17017.5011.mail, /var/mail/xxx/new/1244513712.M557487P17017.5011.mail,W=2019) failed: File too large postfix/pipe[17014]: 68DFC7066D: to=<xxx>, relay=dovecot, delay=72, delays=72/0.02/0/0.04, dsn=4.3.0, status=deferred (temporary failure)
mail_bulk/virtual[13809]: 552706F2AD: to=<xxx>, relay=virtual, delay=0.04, delays=0.01/0.01/0/0.02, dsn=5.2.2, status=bounced (maildir delivery failed: create maildir file /var/mail/xxx/new/1244513678.V15I307e5e5M366990.5012.mail: File too large)
Quoting Timo Sirainen tss@iki.fi:
Dovecot always renames files when moving from tmp/, you can't make it use hard links. The maildir_copy_with_hardlinks is only about how to copy messages from one mailbox to another, and it's enabled by default in v1.1+.
Wonder if this affects rename() only when moving from one directory to another, or also within a directory? So if you're out of inodes and you change a message flag, does rename() succeed?
On Jun 11, 2009, at 8:34 AM, Patrick Domack wrote:
Hmm, looking at how postfix handles this, it hardlinks the files,
so I guess I could turn on hardlinking that way rename isn't
used, and won't get this error.But probably still something to look into?
Quoting Timo Sirainen tss@iki.fi:
On Jun 10, 2009, at 1:51 PM, Patrick Domack wrote:
Looking into this some more, all the log entries that cause this
have to do with rename and EFBIG.deliver(xxx):
rename(/var/mail/virtual/xxx/tmp/1244513712.M557487P17017.5011.mail,
/var/mail/virtual/xxx/new/1244513712.M557487P17017.5011.mail,W=2019)
failed: File too largeThat looks weird. So this is with Linux as NFS client? What do you use as NFS server? To me this feels like NFS server bug..
participants (2)
-
Patrick Domack
-
Timo Sirainen