How to understand NFS lookup (requests) spike?
Hi, I'm are running a classic Dovecot setup:
About ten thousand connected users mailbox in Maildir format shared via NFS (NetApp) Director for POP/IMAP Delivery via Dovecot LDA
All works fine but sometimes I see a spike on the load of POP/IMAP servers and high disk usage (close to 100%) on NFS NetApp.
When this happens on NFS stats (of POP/IMAP) I can see an high volume of "lookup, remove, rename" requests.
Example (avg is during normal load, max is request number during the spike):
Lookup avg 100 max 700 Remove avg 50 max 300 Rename avg 50 max 300 Getattr avg 200 max 250 Total NFS avg 600 max 1800
I think that some users are doing some kinds of "intensive" operations on their mailbox but what and who?
I am currently using "iotop" to monitor the activity of individual users but I can't figure out who is causing the high number of I/O requests.
Have you any suggestions? Thanks
Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
On Wed, Feb 17, 2016 at 12:49 AM, Alessio Cecchi alessio@skye.it wrote:
Hi, I'm are running a classic Dovecot setup:
About ten thousand connected users mailbox in Maildir format shared via NFS (NetApp) Director for POP/IMAP Delivery via Dovecot LDA
All works fine but sometimes I see a spike on the load of POP/IMAP servers and high disk usage (close to 100%) on NFS NetApp.
When this happens on NFS stats (of POP/IMAP) I can see an high volume of "lookup, remove, rename" requests.
Example (avg is during normal load, max is request number during the spike):
Lookup avg 100 max 700 Remove avg 50 max 300 Rename avg 50 max 300 Getattr avg 200 max 250 Total NFS avg 600 max 1800
I think that some users are doing some kinds of "intensive" operations on their mailbox but what and who?
I am currently using "iotop" to monitor the activity of individual users but I can't figure out who is causing the high number of I/O requests.
One suggestions is that I'd walk those directories and see if someone has a mailbox with 100k files in it.
participants (2)
-
Alessio Cecchi
-
Mark Moseley