[Dovecot] getdents() with 4KB buffer - seems slow (Maildir, large inbox)

[Marinos Yannikos]

Hi,

I have a very large inbox (~146K mails) in Maildir format and dovecot
seems to spend a lot of time rescanning the directory, especially when
the server is loaded. I'm not sure whether this is triggered by
Thunderbird or done regularly, but it takes longer when the server is
loaded, so sometimes it seems that it is scanning continuously. Since
it takes around 2000 getdents64() syscalls to scan my inbox, I'm
wondering whether the buffer size is a bit small with 4KB and might
benefit from being configurable:

...
getdents64(8, /* 73 entries */, 4096)   = 4088
getdents64(8, /* 73 entries */, 4096)   = 4088
getdents64(8, /* 73 entries */, 4096)   = 4088
getdents64(8, /* 73 entries */, 4096)   = 4088
getdents64(8, /* 73 entries */, 4096)   = 4088
...

(strace output of an imap process belonging to me)

The behaviour is identical on 1.0.15 and 1.2.13 (I just upgraded).

Regards,
 Marinos

PS. I know that the main reason for slowness is my large inbox and the
server being loaded (disk I/O), but still, so many syscalls just to
rescan the inbox seems wasteful.

# dovecot -n
# 1.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-6-xen-686 i686 Debian 4.0
log_timestamp: %Y-%m-%d %H:%M:%S
ssl_listen: *
ssl_cert_file: XXXXX (censored)
ssl_key_file: XXXXX
login_dir: /var/run/dovecot/login
login_executable: /usr/lib/dovecot/imap-login
login_process_size: 32
login_processes_count: 8
verbose_proctitle: yes
mail_privileged_group: mail
mail_location: maildir:~/Maildir
mbox_write_locks: fcntl dotlock
mail_process_size: 2048
imap_client_workarounds: outlook-idle
auth default:
  passdb:
    driver: pam
  userdb:
    driver: passwd
  socket:
    type: listen
    client:
      path: /var/spool/postfix/private/auth
      mode: 432
      user: postfix
      group: postfix