Hello

I have some performance problem with

doveadm quota get -u user@domain

after migrating from 2.3.21 to 2.4.1 and enabling fts-flatcurve plugin for mail account with 1330 folders and subfolders.

Quota on 2.3.21 was stored in maildirsize. After upgrading dovecot to 2.4.2 quota driver is set to count

When I run doveadm quota get -u user@domain command it takes about 15 minutes. Mail account quota is used about 6.7GB and about 10k messeges. Doveadm process use 100% CPU when it try to count quota values.

When I run command doveadm with "strace" to trace callbacks it looks like:

14:51:00.577061 newfstatat(AT_FDCWD, "/maildovecot/*******/Maildir/Foldery przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714", {st_mode=S_IFDIR|0700, st_size=4096, ...}, 0) = 0 14:51:00.577295 openat(AT_FDCWD, "/maildovecot/*******/Maildir/Foldery przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 13 14:51:00.577680 rmdir("/maildovecot/*******/Maildir/Foldery przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714") = -1 ENOTEMPTY (Directory not empty) 14:51:00.577765 rmdir("/maildovecot/*******/Maildir/Foldery przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714") = -1 ENOTEMPTY (Directory not empty) 14:51:00.577847 rmdir("/maildovecot/*******/Maildir/Foldery przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714") = -1 ENOTEMPTY (Directory not empty) 14:51:00.577929 rmdir("/maildovecot/*******Maildir/Foldery przechowywania/Zakupy/drogerie/rossmann/fts-flatcurve/index.714") = -1 ENOTEMPTY (Directory not empty)

or

14:33:44.529146 newfstatat(AT_FDCWD, "/maildovecot//******//Maildir/Foldery przechowywania/programy lojalnosciowe/vitay/fts-flatcurve/cur", 0x7fff0d9cf270, 0) = -1 ENOENT (No such file or directory)

and it loops 2374 times

grep fts-flatcurve strace_quota | grep rossmann | wc -l 2374

My configuration:

quota "User quota" { driver = count quota_storage_grace = 10M quota_mail_size = 50M quota_message_count = 1500000

quota_warning warn-80-storage {
	quota_storage_percentage = 80
            quota_warning_resource = storage
            quota_warning_threshold = over
	execute quota-warning {
                    args = 80 size %{user} %{userdb:quota_storage_size} 
	}
}

quota_warning warn-90-storage {
	quota_storage_percentage = 90
            quota_warning_resource = storage
            quota_warning_threshold = over
	execute quota-warning {
                    args = 90 size %{user} %{userdb:quota_storage_size} 
 	}
}

    quota_warning warn-90-message {
	quota_message_percentage = 95
            quota_warning_resource = message
            quota_warning_threshold = over
            execute quota-warning {
		args = 90 inodes 
            }
    }

}

service quota-warning { executable = script /dovecot/bin/quota.sh user = mail unix_listener quota-warning { user = mail } }

cat conf.d/90-fts.conf

language_tokenizers = generic email-address language_tokenizer_generic_algorithm = simple language_filters = lowercase snowball stopwords fts_autoindex = yes fts_autoindex_max_recent_msgs = 100 fts_message_max_size = 20M fts_driver = flatcurve fts_decoder_driver = script fts_decoder_script_socket_path = decode2text

service decode2text { executable = script /dovecot/bin/attachment2text.sh unix_listener decode2text { mode = 0666 user = dovecot group = dovecot } }

fts_header_excludes { Authentication-Results = yes Received = yes DKIM-* = yes X-* = yes Comments = yes Date = yes Message-ID = yes Received = yes }

mailbox TRASH { fts_autoindex = no }

mailbox SPAM { fts_autoindex = no }

mailbox confirmed-spam { fts_autoindex = no }

mailbox confirmed-ham { fts_autoindex = no }

language en { default = yes filters = lowercase snowball english-possessive stopwords }

fts flatcurve { commit_limit = 500 min_term_size = 2 optimize_limit = 10 rotate_count = 5000 rotate_time = 5s substring_search = no }

My questions are:

Why "doveadm quota get" tries to open "fts-flatcurve" directories and tries to find "cur" or "ndex.714" directory? Is there any options to exclude all "fts-flatcurve" directories from quota ? Do you observe performance hit for mail accounts with many folders and subfolders after enabling fts-flatcurve?