[Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning <dir> took <X> seconds)
Hello,
We have a dovecot servers farm with storage in NFS and indexes in local files. Under heavy load conditions we a log of messages in the form:
Sep 3 12:35:49 myotis32 dovecot: imap(<imapuser>): Warning: Maildir: Scanning <maildir cur directory> took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80)
We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers).
So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause.
PS: I have attached my dovecot -n output
On 5.9.2012, at 14.40, Angel L. Mateo wrote:
Sep 3 12:35:49 myotis32 dovecot: imap(<imapuser>): Warning: Maildir: Scanning <maildir cur directory> took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80)
We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers).
So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause.
That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations).
El 05/09/12 13:50, Timo Sirainen escribió:
On 5.9.2012, at 14.40, Angel L. Mateo wrote:
Sep 3 12:35:49 myotis32 dovecot: imap(<imapuser>): Warning: Maildir: Scanning <maildir cur directory> took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80)
We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers).
So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause.
That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations).
Ok, I'll try this.
But... is there any way to know how a readdir would be done?On 5.9.2012, at 15.07, Angel L. Mateo wrote:
El 05/09/12 13:50, Timo Sirainen escribió:
On 5.9.2012, at 14.40, Angel L. Mateo wrote:
Sep 3 12:35:49 myotis32 dovecot: imap(<imapuser>): Warning: Maildir: Scanning <maildir cur directory> took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80)
We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers).
So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause.
That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations).
Ok, I'll try this.
But... is there any way to know how a readdir would be done?
What do you mean by "how"? Without that setting readdir() is called any time the cur/ directory's mtime changes. With that setting it's done only when the mtime isn't what Dovecot expected it to be (saved in index). Also it's done if Dovecot can't open maildir file by guessing its name based on the flags in index file, which can happen if multiple clients access the mailbox at the same time.
El 11/09/12 18:07, Timo Sirainen escribió:
On 5.9.2012, at 15.07, Angel L. Mateo wrote:
El 05/09/12 13:50, Timo Sirainen escribió:
On 5.9.2012, at 14.40, Angel L. Mateo wrote:
Sep 3 12:35:49 myotis32 dovecot: imap(<imapuser>): Warning: Maildir: Scanning <maildir cur directory> took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80)
We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers).
So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause.
That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations).
Ok, I'll try this.
But... is there any way to know how a readdir would be done?
What do you mean by "how"? Without that setting readdir() is called any time the cur/ directory's mtime changes. With that setting it's done only when the mtime isn't what Dovecot expected it to be (saved in index). Also it's done if Dovecot can't open maildir file by guessing its name based on the flags in index file, which can happen if multiple clients access the mailbox at the same time.
I meant "when", not how, sorry. But my question is answer with this information.
participants (2)
-
Angel L. Mateo
-
Timo Sirainen