December 2007 - dovecot

[Dovecot] quota_wrning not working for me (quota_rewrite patch for dovecot 1.0.8)
by Juan C. Blanco 20 Jan '08

20 Jan '08

Hello, We are using dovecot 1.0.8 with LDAP authentication and extra variebles (like quota limit) storage. Since some of our users are asking for warning of quota use I have been playing around with the quota rewrite patch for dovecot 1.0.8 and found that it does not work for us as it is now. I've applied the patch recompiled dovecot reconfigured using the new "quota_warning" options for a test user with a 10MB quota and warning limits in the 45 and 50 percent. The script for the warnings was never executed and then I've done some debugging with the code and found two different problems (well, at least I think that there are two problems but I may be wrong) The first problem is with the parsing of percentage limits in the quota_warning options. The function "quota_rule_parse_percentage" was using the same value for the variables "percentage" and "*limit" and then the calculated value is not a percentage of the default quota rule limit but of the percentage itself (i.e. using a 80% limit returns a value of 64 for the internal saved limit, instead of 80% of the default rule value). In my sample a 50% limit returned a value of 25 bytes for the limit instead of 5,120. The other problem is when checking the limits to execute the scripts, the "quota_warnings_execute" function is checking if the warning limit is reached with this comparison: if ((bytes_current < warnings[i].bytes_limit && bytes_current + ctx->bytes_used >= warnings[i].bytes_limit) || ... My debug have reported that at the point of the execution of this function the "bytes_current" variable already includes the size of the current transaction (at least it was in all my tests) and then the first part of the comparison (that we are under the limit prior to this transaction)is false. I've changed this to: if ((bytes_current - ctx->bytes_used < warnings[i].bytes_limit && bytes_current >= warnings[i].bytes_limit) || ... I've included the modified patch in this messag but since this is my first contact with dovecot code I'm not sure enough if the problems was with the code or if it was my fault. There is someone using the quota-rewrite patch and can confirm if it was working fine?. Can the author of the original patch take a look at it and confirm if I'm wrong or my changes are valid I don't have a Dovecot 1.1 working here and can not verify if the problems occurs also with this vesion, however the code seems the same after a quick look at it and I think that may be usefull to recheck the code prior to the RC stage. Regards Juan C. Blanco -- +----------------------------------------------------------------+ | Juan C. Blanco | | | | Centro de Calculo | | | Facultad de Informatica U.P.M. | E-mail: jcblanco(a)fi.upm.es | | Campus de Montegancedo | | | Boadilla del Monte | Tel.: (+34) 91 336 7466 | | 28660 MADRID (Spain) | Fax : (+34) 91 336 6913 | +----------------------------------------------------------------+ diff -ru dovecot-1.0/dovecot-example.conf dovecot-1.0-quotarewrite/dovecot-example.conf --- dovecot-1.0/dovecot-example.conf 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/dovecot-example.conf 2007-11-27 05:42:11.000000000 +0200 @@ -1031,6 +1031,27 @@ # dict: Keep quota stored in dictionary (eg. SQL) # maildir: Maildir++ quota # fs: Read-only support for filesystem quota + # + # Quota limits are set using "quota_rule" parameters, either in here or in + # userdb. It's also possible to give mailbox-specific limits, for example: + # quota_rule = *:storage=1048576 + # quota_rule2 = Trash:storage=102400 + # User has now 1GB quota, but when saving to Trash mailbox the user gets + # additional 100MB. + # + # Multiple quota roots are also possible, for example: + # quota = dict:user::proxy::quota + # quota2 = dict:domain:%d:proxy::quota_domain + # quota_rule = *:storage=102400 + # quota2_rule = *:storage=1048576 + # Gives each user their own 100MB quota and one shared 1GB quota within + # the domain. + # + # You can execute a given command when user exceeds a specified quota limit. + # Each quota root has separate limits. Only the command for the first + # exceeded limit is excecuted, so put the highest limit first. + # quota_warning = storage=95% /usr/local/bin/quota-warning.sh 95 + # quota_warning2 = storage=80% /usr/local/bin/quota-warning.sh 80 #quota = maildir # ACL plugin. vfile backend reads ACLs from "dovecot-acl" file from maildir diff -ru dovecot-1.0/src/plugins/imap-quota/imap-quota-plugin.c dovecot-1.0-quotarewrite/src/plugins/imap-quota/imap-quota-plugin.c --- dovecot-1.0/src/plugins/imap-quota/imap-quota-plugin.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/imap-quota/imap-quota-plugin.c 2007-07-27 13:18:23.000000000 +0300 @@ -30,7 +30,7 @@ str_append(str, " ("); list = quota_root_get_resources(root); for (i = 0; *list != NULL; list++) { - ret = quota_get_resource(root, *list, &value, &limit); + ret = quota_get_resource(root, "", *list, &value, &limit); if (ret > 0) { if (i > 0) str_append_c(str, ' '); @@ -39,8 +39,8 @@ (unsigned long long)limit); i++; } else if (ret < 0) { - client_send_line(cmd->client, t_strconcat( - "* BAD ", quota_last_error(quota_set), NULL)); + client_send_line(cmd->client, + "* BAD Internal quota calculation error"); } } str_append_c(str, ')'); @@ -86,7 +86,7 @@ str_append(str, "* QUOTAROOT "); imap_quote_append_string(str, orig_mailbox, FALSE); - iter = quota_root_iter_init(box); + iter = quota_root_iter_init(quota_set, box); while ((root = quota_root_iter_next(iter)) != NULL) { str_append_c(str, ' '); imap_quote_append_string(str, quota_root_get_name(root), FALSE); @@ -95,7 +95,7 @@ client_send_line(cmd->client, str_c(str)); /* send QUOTA reply for each quotaroot */ - iter = quota_root_iter_init(box); + iter = quota_root_iter_init(quota_set, box); while ((root = quota_root_iter_next(iter)) != NULL) quota_send(cmd, root); quota_root_iter_deinit(iter); @@ -135,7 +135,7 @@ { struct quota_root *root; struct imap_arg *args, *arg; - const char *root_name, *name; + const char *root_name, *name, *error; uint64_t value; /* <quota root> <resource limits> */ @@ -169,9 +169,8 @@ } value = strtoull(IMAP_ARG_STR_NONULL(&arg[1]), NULL, 10); - if (quota_set_resource(root, name, value) < 0) { - client_send_command_error(cmd, - quota_last_error(quota_set)); + if (quota_set_resource(root, name, value, &error) < 0) { + client_send_command_error(cmd, error); return TRUE; } } diff -ru dovecot-1.0/src/plugins/imap-quota/Makefile.am dovecot-1.0-quotarewrite/src/plugins/imap-quota/Makefile.am --- dovecot-1.0/src/plugins/imap-quota/Makefile.am 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/imap-quota/Makefile.am 2007-07-27 13:18:23.000000000 +0300 @@ -8,12 +8,12 @@ imap_moduledir = $(moduledir)/imap -lib11_imap_quota_plugin_la_LDFLAGS = -module -avoid-version +lib12_imap_quota_plugin_la_LDFLAGS = -module -avoid-version imap_module_LTLIBRARIES = \ - lib11_imap_quota_plugin.la + lib12_imap_quota_plugin.la -lib11_imap_quota_plugin_la_SOURCES = \ +lib12_imap_quota_plugin_la_SOURCES = \ imap-quota-plugin.c noinst_HEADERS = \ diff -ru dovecot-1.0/src/plugins/Makefile.am dovecot-1.0-quotarewrite/src/plugins/Makefile.am --- dovecot-1.0/src/plugins/Makefile.am 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/Makefile.am 2007-07-27 13:18:23.000000000 +0300 @@ -2,4 +2,4 @@ ZLIB = zlib endif -SUBDIRS = acl convert quota imap-quota lazy-expunge mail-log trash $(ZLIB) +SUBDIRS = acl convert quota imap-quota lazy-expunge mail-log $(ZLIB) diff -ru dovecot-1.0/src/plugins/quota/Makefile.am dovecot-1.0-quotarewrite/src/plugins/quota/Makefile.am --- dovecot-1.0/src/plugins/quota/Makefile.am 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/Makefile.am 2007-07-27 13:18:23.000000000 +0300 @@ -14,7 +14,7 @@ lib10_quota_plugin_la_SOURCES = \ quota.c \ - quota-count.c \ + quota-count.c \ quota-fs.c \ quota-dict.c \ quota-dirsize.c \ diff -ru dovecot-1.0/src/plugins/quota/quota.c dovecot-1.0-quotarewrite/src/plugins/quota/quota.c --- dovecot-1.0/src/plugins/quota/quota.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota.c 2007-07-27 13:25:18.000000000 +0300 @@ -6,6 +6,19 @@ #include "quota-private.h" #include "quota-fs.h" +#include <ctype.h> +#include <stdlib.h> +#include <sys/wait.h> + +#define RULE_NAME_ALL_MAILBOXES "*" + +struct quota_root_iter { + struct quota *quota; + struct mailbox *box; + + unsigned int i; +}; + unsigned int quota_module_id = 0; extern struct quota_backend quota_backend_dict; @@ -13,7 +26,7 @@ extern struct quota_backend quota_backend_fs; extern struct quota_backend quota_backend_maildir; -static struct quota_backend *quota_backends[] = { +static const struct quota_backend *quota_backends[] = { #ifdef HAVE_FS_QUOTA &quota_backend_fs, #endif @@ -23,449 +36,745 @@ }; #define QUOTA_CLASS_COUNT (sizeof(quota_backends)/sizeof(quota_backends[0])) -void (*hook_quota_root_created)(struct quota_root *root); +static int quota_default_test_alloc(struct quota_transaction_context *ctx, + uoff_t size, bool *too_large_r); struct quota *quota_init(void) { struct quota *quota; quota = i_new(struct quota, 1); - ARRAY_CREATE(&quota->setups, default_pool, struct quota_setup *, 4); + quota->test_alloc = quota_default_test_alloc; + quota->debug = getenv("DEBUG") != NULL; + ARRAY_CREATE(&quota->roots, system_pool, struct quota_root *, 4); + ARRAY_CREATE(&quota->storages, system_pool, struct mail_storage *, 8); + return quota; } void quota_deinit(struct quota *quota) { - while (array_count(&quota->setups) > 0) { - struct quota_setup *const *setup; + struct quota_root **root; - setup = array_idx(&quota->setups, 0); - quota_setup_deinit(*setup); + while (array_count(&quota->roots) > 0) { + root = array_idx_modifyable(&quota->roots, 0); + quota_root_deinit(*root); } - array_free(&quota->setups); + array_free(&quota->roots); + array_free(&quota->storages); i_free(quota); } -struct quota_setup * -quota_setup_init(struct quota *quota, const char *data, bool user_root) +static const struct quota_backend *quota_backend_find(const char *name) { - struct quota_setup *setup; - const char *backend_name, *p; unsigned int i; - setup = i_new(struct quota_setup, 1); - setup->quota = quota; - setup->data = i_strdup(data); - setup->user_root = user_root; - ARRAY_CREATE(&setup->roots, default_pool, struct quota_root *, 4); + for (i = 0; i < QUOTA_CLASS_COUNT; i++) { + if (strcmp(quota_backends[i]->name, name) == 0) + return quota_backends[i]; + } + + return NULL; +} + +struct quota_root *quota_root_init(struct quota *quota, const char *root_def) +{ + struct quota_root *root; + const struct quota_backend *backend; + const char *p, *args, *backend_name; t_push(); - p = strchr(setup->data, ':'); + + /* <backend>[:<quota root name>[:<backend args>]] */ + p = strchr(root_def, ':'); if (p == NULL) { - backend_name = setup->data; - data = ""; + backend_name = root_def; + args = NULL; } else { - backend_name = t_strdup_until(setup->data, p); - data = p+1; + backend_name = t_strdup_until(root_def, p); + args = p + 1; } - for (i = 0; i < QUOTA_CLASS_COUNT; i++) { - if (strcmp(quota_backends[i]->name, backend_name) == 0) { - setup->backend = quota_backends[i]; - break; + + backend = quota_backend_find(backend_name); + if (backend == NULL) + i_fatal("Unknown quota backend: %s", backend_name); + + t_pop(); + + root = backend->v.alloc(); + root->quota = quota; + root->backend = *backend; + root->pool = pool_alloconly_create("quota root", 512); + + if (args != NULL) { + /* save root's name */ + p = strchr(args, ':'); + if (p == NULL) { + root->name = p_strdup(root->pool, args); + args = NULL; + } else { + root->name = p_strdup_until(root->pool, args, p); + args = p + 1; } + } else { + root->name = ""; } - if (setup->backend == NULL) - i_fatal("Unknown quota backend: %s", backend_name); + ARRAY_CREATE(&root->rules, system_pool, struct quota_rule, 4); + ARRAY_CREATE(&root->warning_rules, system_pool, + struct quota_warning_rule, 4); + array_create(&root->quota_module_contexts, default_pool, + sizeof(void *), 5); - t_pop(); + array_append(&quota->roots, &root, 1); - array_append(&quota->setups, &setup, 1); - return setup; + if (backend->v.init != NULL) { + if (backend->v.init(root, args) < 0) { + quota_root_deinit(root); + return NULL; + } + } + return root; } -void quota_setup_deinit(struct quota_setup *setup) +void quota_root_deinit(struct quota_root *root) { - struct quota_setup *const *setups; + pool_t pool = root->pool; + struct quota_root *const *roots; + struct quota_warning_rule *warnings; unsigned int i, count; - setups = array_get(&setup->quota->setups, &count); + roots = array_get(&root->quota->roots, &count); for (i = 0; i < count; i++) { - if (setups[i] == setup) { - array_delete(&setup->quota->setups, i, 1); - break; - } + if (roots[i] == root) + array_delete(&root->quota->roots, i, 1); } - i_assert(i != count); - while (array_count(&setup->roots) > 0) { - struct quota_root *const *root; + warnings = array_get_modifyable(&root->warning_rules, &count); + for (i = 0; i < count; i++) + i_free(warnings[i].command); + array_free(&root->warning_rules); - root = array_idx(&setup->roots, 0); - quota_root_deinit(*root); - } + array_free(&root->rules); + array_free(&root->quota_module_contexts); - array_free(&setup->roots); - i_free(setup->data); - i_free(setup); + root->backend.v.deinit(root); + pool_unref(pool); } -struct quota_root * -quota_root_init(struct quota_setup *setup, const char *name) +static struct quota_rule * +quota_root_rule_find(struct quota_root *root, const char *name) { - struct quota_root *root; - - root = setup->backend->v.init(setup, name); - root->setup = setup; - ARRAY_CREATE(&root->storages, default_pool, struct mail_storage *, 8); - array_create(&root->quota_module_contexts, - default_pool, sizeof(void *), 5); - array_append(&setup->roots, &root, 1); + struct quota_rule *rules; + unsigned int i, count; - if (hook_quota_root_created != NULL) - hook_quota_root_created(root); - return root; + rules = array_get_modifyable(&root->rules, &count); + for (i = 0; i < count; i++) { + if (strcmp(rules[i].mailbox_name, name) == 0) + return &rules[i]; + } + return NULL; } -void quota_root_deinit(struct quota_root *root) -{ - /* make a copy, since root is freed */ - array_t module_contexts = root->quota_module_contexts; - struct mail_storage *const *storage_p; - struct quota_root *const *roots; - unsigned int i, count; +static int +quota_rule_parse_percentage(struct quota_root *root, struct quota_rule *rule, + int64_t *limit, const char **error_r) +{ + struct quota_rule *defrule; + int64_t percentage = *limit; + + if (percentage < 0) { + *error_r = p_strdup_printf(root->pool, + "Invalid rule percentage: %lld", (long long)percentage); + return -1; + } - /* remove from all storages */ - while (array_count(&root->storages) > 0) { - storage_p = array_idx(&root->storages, 0); - quota_mail_storage_remove_root(*storage_p, root); + if (rule == &root->default_rule) { + *error_r = "Default rule can't be a percentage"; + return -1; + } + defrule = &root->default_rule; + if (defrule == NULL) { + *error_r = "Default rule not defined"; + return -1; } - /* remove from setup */ - roots = array_get(&root->setup->roots, &count); - for (i = 0; i < count; i++) { - if (roots[i] == root) { - array_delete(&root->setup->roots, i, 1); + if (limit == &rule->bytes_limit) + *limit = root->default_rule.bytes_limit * percentage / 100; + else if (limit == &rule->count_limit) + *limit = root->default_rule.count_limit * percentage / 100; + else + i_unreached(); + return 0; +} + +static int +quota_rule_parse_limits(struct quota_root *root, struct quota_rule *rule, + const char *limits, const char **error_r) +{ + const char **args; + char *p; + uint64_t multiply; + int64_t *limit; + + args = t_strsplit(limits, ":"); + for (; *args != NULL; args++) { + multiply = 1; + limit = NULL; + if (strncmp(*args, "storage=", 8) == 0) { + multiply = 1024; + limit = &rule->bytes_limit; + *limit = strtoll(*args + 8, &p, 10); + } else if (strncmp(*args, "bytes=", 6) == 0) { + limit = &rule->bytes_limit; + *limit = strtoll(*args + 6, &p, 10); + } else if (strncmp(*args, "messages=", 9) == 0) { + limit = &rule->count_limit; + *limit = strtoll(*args + 9, &p, 10); + } else { + *error_r = p_strdup_printf(root->pool, + "Unknown rule limit name: %s", *args); + return -1; + } + + switch (i_toupper(*p)) { + case '\0': + /* default */ + break; + case 'B': + multiply = 1; + break; + case 'K': + multiply = 1024; + break; + case 'M': + multiply = 1024*1024; break; + case 'G': + multiply = 1024*1024*1024; + break; + case 'T': + multiply = 1024ULL*1024*1024*1024; + break; + case '%': + multiply = 1; + if (quota_rule_parse_percentage(root, rule, limit, + error_r) < 0) + return -1; + break; + default: + *error_r = p_strdup_printf(root->pool, + "Invalid rule limit value: %s", *args); + return -1; } + *limit *= multiply; } - i_assert(i != count); - - array_free(&root->storages); - root->v.deinit(root); - array_free(&module_contexts); + return 0; } -void quota_add_user_storage(struct quota *quota, struct mail_storage *storage) +int quota_root_add_rule(struct quota_root *root, const char *rule_def, + const char **error_r) { - struct quota_setup *const *setups; - struct quota_root *const *roots; - unsigned int i, j, setup_count, root_count; - bool found = FALSE; + struct quota_rule *rule; + const char *p, *mailbox_name; + int ret = 0; - setups = array_get(&quota->setups, &setup_count); - for (i = 0; i < setup_count; i++) { - roots = array_get(&setups[i]->roots, &root_count); - for (j = 0; j < root_count; j++) { - if (!roots[j]->user_root) - continue; + p = strchr(rule_def, ':'); + if (p == NULL) { + *error_r = "Invalid rule"; + return -1; + } - if (quota_mail_storage_add_root(storage, roots[j])) - found = TRUE; + /* <mailbox name>:<quota limits> */ + t_push(); + mailbox_name = t_strdup_until(rule_def, p++); + + rule = quota_root_rule_find(root, mailbox_name); + if (rule == NULL) { + if (strcmp(mailbox_name, RULE_NAME_ALL_MAILBOXES) == 0) + rule = &root->default_rule; + else { + rule = array_append_space(&root->rules); + rule->mailbox_name = p_strdup(root->pool, mailbox_name); } } - if (!found && setup_count > 0) { - /* create a new quota root for the storage */ - struct quota_root *root; + if (strncmp(p, "backend=", 8) == 0) { + if (!root->backend.v.parse_rule(root, rule, p, error_r)) + ret = -1; + } else { + if (quota_rule_parse_limits(root, rule, p, error_r) < 0) + ret = -1; + } - root = quota_root_init(setups[0], ""); /* FIXME: name? */ - found = quota_mail_storage_add_root(storage, root); - i_assert(found); + if (root->quota->debug) { + i_info("Quota rule: root=%s mailbox=%s " + "storage=%lldkB messages=%lld", root->name, + rule->mailbox_name != NULL ? rule->mailbox_name : "", + (long long)rule->bytes_limit / 1024, + (long long)rule->count_limit); } + + t_pop(); + return ret; } -struct quota_root *quota_root_lookup(struct quota *quota, const char *name) +static bool quota_root_get_rule_limits(struct quota_root *root, + const char *mailbox_name, + uint64_t *bytes_limit_r, + uint64_t *count_limit_r) +{ + struct quota_rule *rule; + int64_t bytes_limit, count_limit; + bool found; + + bytes_limit = root->default_rule.bytes_limit; + count_limit = root->default_rule.count_limit; + + /* if default rule limits are 0, this rule applies only to specific + mailboxes */ + found = bytes_limit != 0 || count_limit != 0; + + rule = quota_root_rule_find(root, mailbox_name); + if (rule != NULL) { + bytes_limit += rule->bytes_limit; + count_limit += rule->count_limit; + found = TRUE; + } + + *bytes_limit_r = bytes_limit <= 0 ? 0 : bytes_limit; + *count_limit_r = count_limit <= 0 ? 0 : count_limit; + return found; +} + +void quota_add_user_storage(struct quota *quota, struct mail_storage *storage) { - struct quota_setup *const *setups; struct quota_root *const *roots; - unsigned int i, j, setup_count, root_count; + struct mail_storage *const *storages; + struct quota_backend **backends; + const char *path, *path2; + unsigned int i, j, count; + bool is_file; + + /* first check if there already exists a storage with the exact same + path. we don't want to count them twice. */ + path = mail_storage_get_mailbox_path(storage, "", &is_file); + if (path != NULL) { + storages = array_get(&quota->storages, &count); + for (i = 0; i < count; i++) { + path2 = mail_storage_get_mailbox_path(storages[i], "", + &is_file); + if (path2 != NULL && strcmp(path, path2) == 0) { + /* duplicate */ + return; + } + } + } - setups = array_get(&quota->setups, &setup_count); - for (i = 0; i < setup_count; i++) { - roots = array_get(&setups[i]->roots, &root_count); - for (j = 0; j < root_count; j++) { - if (strcmp(roots[j]->name, name) == 0) - return roots[j]; + array_append(&quota->storages, &storage, 1); + + roots = array_get(&quota->roots, &count); + /* @UNSAFE: get different backends into one array */ + backends = t_new(struct quota_backend *, count + 1); + for (i = 0; i < count; i++) { + for (j = 0; backends[j] != NULL; j++) { + if (backends[j]->name == roots[i]->backend.name) + break; } + if (backends[j] == NULL) + backends[j] = &roots[i]->backend; } - return NULL; -} -const char *quota_root_get_name(struct quota_root *root) -{ - return root->name; + for (i = 0; backends[i] != NULL; i++) { + if (backends[i]->v.storage_added != NULL) + backends[i]->v.storage_added(quota, storage); + } } -const char *const *quota_root_get_resources(struct quota_root *root) +void quota_remove_user_storage(struct quota *quota, + struct mail_storage *storage) { - return root->v.get_resources(root); + struct mail_storage *const *storages; + unsigned int i, count; + + storages = array_get(&quota->storages, &count); + for (i = 0; i < count; i++) { + if (storages[i] == storage) { + array_delete(&quota->storages, i, 1); + break; + } + } } -int quota_get_resource(struct quota_root *root, const char *name, - uint64_t *value_r, uint64_t *limit_r) +int quota_root_add_warning_rule(struct quota_root *root, const char *rule_def, + const char **error_r) { - return root->v.get_resource(root, name, value_r, limit_r); -} + struct quota_warning_rule *warning; + struct quota_rule rule; + const char *p; + int ret; -int quota_set_resource(struct quota_root *root, - const char *name, uint64_t value) -{ - return root->v.set_resource(root, name, value); + p = strchr(rule_def, ' '); + if (p == NULL) { + *error_r = "No command specified"; + return -1; + } + + memset(&rule, 0, sizeof(rule)); + t_push(); + ret = quota_rule_parse_limits(root, &rule, t_strdup_until(rule_def, p), + error_r); + t_pop(); + if (ret < 0) + return -1; + + if (rule.bytes_limit < 0) { + *error_r = "Bytes limit can't be negative"; + return -1; + } + if (rule.count_limit < 0) { + *error_r = "Count limit can't be negative"; + return -1; + } + + warning = array_append_space(&root->warning_rules); + warning->command = i_strdup(p+1); + warning->bytes_limit = rule.bytes_limit; + warning->count_limit = rule.count_limit; + return 0; } -struct quota_transaction_context *quota_transaction_begin(struct mailbox *box) +struct quota_root_iter * +quota_root_iter_init(struct quota *quota, struct mailbox *box) { - struct quota_transaction_context *ctx; - struct quota_root_transaction_context *root_ctx; struct quota_root_iter *iter; - struct quota_root *root; - ctx = i_new(struct quota_transaction_context, 1); - ARRAY_CREATE(&ctx->root_transactions, default_pool, - struct quota_root_transaction_context *, 4); + iter = i_new(struct quota_root_iter, 1); + iter->quota = quota; + iter->box = box; + return iter; +} + +struct quota_root *quota_root_iter_next(struct quota_root_iter *iter) +{ + struct quota_root *const *roots, *root = NULL; + unsigned int count; + uint64_t value, limit; + int ret; - iter = quota_root_iter_init(box); - while ((root = quota_root_iter_next(iter)) != NULL) { - root_ctx = root->v.transaction_begin(root, ctx, box); - array_append(&ctx->root_transactions, &root_ctx, 1); + roots = array_get(&iter->quota->roots, &count); + if (iter->i >= count) + return NULL; + + for (; iter->i < count; iter->i++) { + ret = quota_get_resource(roots[iter->i], "", + QUOTA_NAME_STORAGE_KILOBYTES, + &value, &limit); + if (ret == 0) { + ret = quota_get_resource(roots[iter->i], "", + QUOTA_NAME_MESSAGES, + &value, &limit); + } + if (ret > 0) { + root = roots[iter->i]; + break; + } } - quota_root_iter_deinit(iter); - return ctx; + + iter->i++; + return root; } -static void quota_transaction_free(struct quota_transaction_context *ctx) +void quota_root_iter_deinit(struct quota_root_iter *iter) { - array_free(&ctx->root_transactions); - i_free(ctx); + i_free(iter); } -int quota_transaction_commit(struct quota_transaction_context *ctx) +struct quota_root *quota_root_lookup(struct quota *quota, const char *name) { - struct quota_root_transaction_context *const *root_transactions; + struct quota_root *const *roots; unsigned int i, count; - int ret = 0; - root_transactions = array_get(&ctx->root_transactions, &count); + roots = array_get(&quota->roots, &count); for (i = 0; i < count; i++) { - struct quota_root_transaction_context *t = - root_transactions[i]; - - if (t->root->v.transaction_commit(t) < 0) - ret = -1; + if (strcmp(roots[i]->name, name) == 0) + return roots[i]; } - - quota_transaction_free(ctx); - return ret; + return NULL; } -void quota_transaction_rollback(struct quota_transaction_context *ctx) +const char *quota_root_get_name(struct quota_root *root) { - struct quota_root_transaction_context *const *root_transactions; - unsigned int i, count; - - root_transactions = array_get(&ctx->root_transactions, &count); - for (i = 0; i < count; i++) { - struct quota_root_transaction_context *t = - root_transactions[i]; - - t->root->v.transaction_rollback(t); - } + return root->name; +} - quota_transaction_free(ctx); +const char *const *quota_root_get_resources(struct quota_root *root) +{ + return root->backend.v.get_resources(root); } -int quota_try_alloc(struct quota_transaction_context *ctx, - struct mail *mail, bool *too_large_r) +int quota_get_resource(struct quota_root *root, const char *mailbox_name, + const char *name, uint64_t *value_r, uint64_t *limit_r) { - struct quota_root_transaction_context *const *root_transactions; - unsigned int i, count; - int ret = 1; + uint64_t bytes_limit, count_limit; + bool kilobytes = FALSE; + int ret; - root_transactions = array_get(&ctx->root_transactions, &count); - for (i = 0; i < count; i++) { - struct quota_root_transaction_context *t = - root_transactions[i]; + if (strcmp(name, QUOTA_NAME_STORAGE_KILOBYTES) == 0) { + name = QUOTA_NAME_STORAGE_BYTES; + kilobytes = TRUE; + } + + (void)quota_root_get_rule_limits(root, mailbox_name, + &bytes_limit, &count_limit); + if (strcmp(name, QUOTA_NAME_STORAGE_BYTES) == 0) + *limit_r = bytes_limit; + else if (strcmp(name, QUOTA_NAME_MESSAGES) == 0) + *limit_r = count_limit; + else + *limit_r = 0; + + ret = root->backend.v.get_resource(root, name, value_r, limit_r); + if (kilobytes && ret > 0) { + *value_r /= 1024; + *limit_r /= 1024; + } + return ret <= 0 ? ret : + (*limit_r == 0 ? 0 : 1); +} + +int quota_set_resource(struct quota_root *root __attr_unused__, + const char *name __attr_unused__, + uint64_t value __attr_unused__, const char **error_r) +{ + /* the quota information comes from userdb (or even config file), + so there's really no way to support this until some major changes + are done */ + *error_r = MAIL_STORAGE_ERR_NO_PERMISSION; + return -1; +} - ret = t->root->v.try_alloc(t, mail, too_large_r); - if (ret <= 0) - break; - } - return ret; +struct quota_transaction_context *quota_transaction_begin(struct quota *quota, + struct mailbox *box) +{ + struct quota_transaction_context *ctx; + + ctx = i_new(struct quota_transaction_context, 1); + ctx->quota = quota; + ctx->box = box; + ctx->bytes_left = (uint64_t)-1; + ctx->count_left = (uint64_t)-1; + return ctx; } -int quota_try_alloc_bytes(struct quota_transaction_context *ctx, - uoff_t size, bool *too_large_r) +static int quota_transaction_set_limits(struct quota_transaction_context *ctx) { - struct quota_root_transaction_context *const *root_transactions; + struct quota_root *const *roots; + const char *mailbox_name; unsigned int i, count; - int ret = 1; + uint64_t current, limit, left; + int ret; - root_transactions = array_get(&ctx->root_transactions, &count); - for (i = 0; i < count; i++) { - struct quota_root_transaction_context *t = - root_transactions[i]; + ctx->limits_set = TRUE; + mailbox_name = mailbox_get_name(ctx->box); - ret = t->root->v.try_alloc_bytes(t, size, too_large_r); - if (ret <= 0) - break; + /* find the lowest quota limits from all roots and use them */ + roots = array_get(&ctx->quota->roots, &count); + for (i = 0; i < count; i++) { + ret = quota_get_resource(roots[i], mailbox_name, + QUOTA_NAME_STORAGE_BYTES, + &current, &limit); + if (ret > 0) { + current += ctx->bytes_used; + left = limit < current ? 0 : limit - current; + if (ctx->bytes_left > left) + ctx->bytes_left = left; + } else if (ret < 0) { + ctx->failed = TRUE; + return -1; + } + + ret = quota_get_resource(roots[i], mailbox_name, + QUOTA_NAME_MESSAGES, &current, &limit); + if (ret > 0) { + current += ctx->count_used; + left = limit < current ? 0 : limit - current; + if (ctx->count_left > left) + ctx->count_left = left; + } else if (ret < 0) { + ctx->failed = TRUE; + return -1; + } } - return ret; + return 0; } -int quota_test_alloc_bytes(struct quota_transaction_context *ctx, - uoff_t size, bool *too_large_r) +static void quota_warning_execute(const char *cmd) { - struct quota_root_transaction_context *const *root_transactions; - unsigned int i, count; - int ret = 1; - - root_transactions = array_get(&ctx->root_transactions, &count); - for (i = 0; i < count; i++) { - struct quota_root_transaction_context *t = - root_transactions[i]; + int ret = system(cmd); - ret = t->root->v.test_alloc_bytes(t, size, too_large_r); - if (ret <= 0) - break; + if (ret < 0) { + i_error("system(%s) failed: %m", cmd); + } else if (WIFSIGNALED(ret)) { + i_error("system(%s) died with signal %d", cmd, WTERMSIG(ret)); + } else if (!WIFEXITED(ret) || WEXITSTATUS(ret) != 0) { + i_error("system(%s) exited with status %d", + cmd, WIFEXITED(ret) ? WEXITSTATUS(ret) : ret); } - return ret; } -void quota_alloc(struct quota_transaction_context *ctx, struct mail *mail) +static void quota_warnings_execute(struct quota_root *root, + struct quota_transaction_context *ctx) { - struct quota_root_transaction_context *const *root_transactions; + struct quota_warning_rule *warnings; unsigned int i, count; + uint64_t bytes_current, bytes_limit, count_current, count_limit; - root_transactions = array_get(&ctx->root_transactions, &count); - for (i = 0; i < count; i++) { - struct quota_root_transaction_context *t = - root_transactions[i]; + warnings = array_get_modifyable(&root->warning_rules, &count); + if (count == 0) + return; + + if (quota_get_resource(root, "", QUOTA_NAME_STORAGE_BYTES, + &bytes_current, &bytes_limit) < 0) + return; + if (quota_get_resource(root, "", QUOTA_NAME_MESSAGES, + &count_current, &count_limit) < 0) + return; - t->root->v.alloc(t, mail); + for (i = 0; i < count; i++) { + if ((bytes_current - + ctx->bytes_used < warnings[i].bytes_limit && + bytes_current >= warnings[i].bytes_limit) || + (count_current - + ctx->count_used < warnings[i].count_limit && + count_current >= warnings[i].count_limit)) { + quota_warning_execute(warnings[i].command); + break; + } } } -void quota_free(struct quota_transaction_context *ctx, struct mail *mail) +int quota_transaction_commit(struct quota_transaction_context **_ctx) { - struct quota_root_transaction_context *const *root_transactions; + struct quota_transaction_context *ctx = *_ctx; + struct quota_root *const *roots; unsigned int i, count; + int ret = 0; - root_transactions = array_get(&ctx->root_transactions, &count); - for (i = 0; i < count; i++) { - struct quota_root_transaction_context *t = - root_transactions[i]; + *_ctx = NULL; - t->root->v.free(t, mail); + if (ctx->failed) + ret = -1; + else if (ctx->bytes_used != 0 || ctx->count_used != 0 || + ctx->recalculate) { + roots = array_get(&ctx->quota->roots, &count); + for (i = 0; i < count; i++) { + quota_warnings_execute(roots[i], ctx); + if (roots[i]->backend.v.update(roots[i], ctx) < 0) + ret = -1; + } } -} -const char *quota_last_error(struct quota *quota) -{ - return quota->last_error != NULL ? quota->last_error : - "Unknown quota error"; + i_free(ctx); + return ret; } -void quota_set_error(struct quota *quota, const char *errormsg) +void quota_transaction_rollback(struct quota_transaction_context **_ctx) { - i_free(quota->last_error); - quota->last_error = i_strdup(errormsg); -} + struct quota_transaction_context *ctx = *_ctx; -void -quota_default_transaction_rollback(struct quota_root_transaction_context *ctx) -{ + *_ctx = NULL; i_free(ctx); } -int quota_default_try_alloc_bytes(struct quota_root_transaction_context *ctx, - uoff_t size, bool *too_large_r) +int quota_try_alloc(struct quota_transaction_context *ctx, + struct mail *mail, bool *too_large_r) { int ret; - ret = quota_default_test_alloc_bytes(ctx, size, too_large_r); - if (ret <= 0 || ctx->disabled) + ret = quota_test_alloc(ctx, mail_get_physical_size(mail), too_large_r); + if (ret <= 0) return ret; - ctx->count_diff++; - ctx->bytes_diff += size; + quota_alloc(ctx, mail); return 1; } -int quota_default_test_alloc_bytes(struct quota_root_transaction_context *ctx, - uoff_t size, bool *too_large_r) +int quota_test_alloc(struct quota_transaction_context *ctx, + uoff_t size, bool *too_large_r) { - if (ctx->disabled) { - *too_large_r = FALSE; - return 1; - } - if (ctx->bytes_current == (uint64_t)-1) { - /* failure in transaction initialization */ + if (ctx->failed) return -1; - } - *too_large_r = size > ctx->bytes_limit; - - if (ctx->bytes_current + ctx->bytes_diff + size > ctx->bytes_limit) - return 0; - if (ctx->count_current + ctx->count_diff + 1 > ctx->count_limit) - return 0; - return 1; + if (!ctx->limits_set) { + if (quota_transaction_set_limits(ctx) < 0) + return -1; + } + return ctx->quota->test_alloc(ctx, size, too_large_r); } -int quota_default_try_alloc(struct quota_root_transaction_context *ctx, - struct mail *mail, bool *too_large_r) +static int quota_default_test_alloc(struct quota_transaction_context *ctx, + uoff_t size, bool *too_large_r) { - uoff_t size; + struct quota_root *const *roots; + unsigned int i, count; + + *too_large_r = FALSE; - if (ctx->disabled) + if (ctx->count_left != 0 && ctx->bytes_left >= ctx->bytes_used + size) return 1; - size = mail_get_physical_size(mail); - if (size == (uoff_t)-1) { - mail_storage_set_critical(mail->box->storage, - "Quota: Couldn't get new message's size"); - return -1; + roots = array_get(&ctx->quota->roots, &count); + for (i = 0; i < count; i++) { + uint64_t bytes_limit, count_limit; + + if (!quota_root_get_rule_limits(roots[i], + mailbox_get_name(ctx->box), + &bytes_limit, &count_limit)) + continue; + + /* if size is bigger than any limit, then + it is bigger than the lowest limit */ + if (size > bytes_limit) { + *too_large_r = TRUE; + break; + } } - return quota_default_try_alloc_bytes(ctx, size, too_large_r); + return 0; } -void quota_default_alloc(struct quota_root_transaction_context *ctx, - struct mail *mail) +void quota_alloc(struct quota_transaction_context *ctx, struct mail *mail) { uoff_t size; - if (ctx->disabled) - return; - size = mail_get_physical_size(mail); if (size != (uoff_t)-1) - ctx->bytes_diff += size; - ctx->count_diff++; + ctx->bytes_used += size; + + ctx->count_used++; } -void quota_default_free(struct quota_root_transaction_context *ctx, - struct mail *mail) +void quota_free(struct quota_transaction_context *ctx, struct mail *mail) { uoff_t size; - if (ctx->disabled) - return; - size = mail_get_physical_size(mail); - if (size != (uoff_t)-1) - ctx->bytes_diff -= size; - ctx->count_diff--; + if (size == (uoff_t)-1) + quota_recalculate(ctx); + else + quota_free_bytes(ctx, size); +} + +void quota_free_bytes(struct quota_transaction_context *ctx, + uoff_t physical_size) +{ + ctx->bytes_used -= physical_size; + ctx->count_used--; +} + +void quota_recalculate(struct quota_transaction_context *ctx) +{ + ctx->recalculate = TRUE; } diff -ru dovecot-1.0/src/plugins/quota/quota-count.c dovecot-1.0-quotarewrite/src/plugins/quota/quota-count.c --- dovecot-1.0/src/plugins/quota/quota-count.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota-count.c 2007-07-27 13:18:36.000000000 +0300 @@ -1,6 +1,7 @@ /* Copyright (C) 2006 Timo Sirainen */ #include "lib.h" +#include "array.h" #include "mail-search.h" #include "mail-storage.h" #include "quota-private.h" @@ -46,24 +47,20 @@ return ret; } -int quota_count_storage(struct mail_storage *storage, - uint64_t *bytes_r, uint64_t *count_r) +static int quota_count_storage(struct mail_storage *storage, + uint64_t *bytes, uint64_t *count) { struct mailbox_list_context *ctx; - struct mailbox_list *list; + struct mailbox_list *info; int ret = 0; - *bytes_r = *count_r = 0; - ctx = mail_storage_mailbox_list_init(storage, "", "*", - MAILBOX_LIST_FAST_FLAGS | - MAILBOX_LIST_INBOX); - while ((list = mail_storage_mailbox_list_next(ctx)) != NULL) { - if ((list->flags & (MAILBOX_NONEXISTENT | - MAILBOX_PLACEHOLDER | + MAILBOX_LIST_FAST_FLAGS); + while ((info = mail_storage_mailbox_list_next(ctx)) != NULL) { + if ((info->flags & (MAILBOX_NONEXISTENT | MAILBOX_NOSELECT)) == 0) { - ret = quota_count_mailbox(storage, list->name, - bytes_r, count_r); + ret = quota_count_mailbox(storage, info->name, + bytes, count); if (ret < 0) break; } @@ -73,3 +70,20 @@ return ret; } + +int quota_count(struct quota *quota, uint64_t *bytes_r, uint64_t *count_r) +{ + struct mail_storage *const *storages; + unsigned int i, count; + int ret = 0; + + *bytes_r = *count_r = 0; + + storages = array_get(&quota->storages, &count); + for (i = 0; i < count; i++) { + ret = quota_count_storage(storages[i], bytes_r, count_r); + if (ret < 0) + break; + } + return ret; +} diff -ru dovecot-1.0/src/plugins/quota/quota-dict.c dovecot-1.0-quotarewrite/src/plugins/quota/quota-dict.c --- dovecot-1.0/src/plugins/quota/quota-dict.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota-dict.c 2007-07-27 13:18:23.000000000 +0300 @@ -1,7 +1,6 @@ /* Copyright (C) 2005-2006 Timo Sirainen */ #include "lib.h" -#include "array.h" #include "str.h" #include "dict.h" #include "quota-private.h" @@ -15,291 +14,164 @@ struct dict_quota_root { struct quota_root root; struct dict *dict; - - uint64_t message_bytes_limit; - uint64_t message_count_limit; - - unsigned int counting:1; }; extern struct quota_backend quota_backend_dict; -static struct quota_root * -dict_quota_init(struct quota_setup *setup, const char *name) +static struct quota_root *dict_quota_alloc(void) { struct dict_quota_root *root; - struct dict *dict; - const char *uri, *const *args; - unsigned long long message_bytes_limit = 0, message_count_limit = 0; - uri = strchr(setup->data, ' '); - if (uri == NULL) { - i_fatal("dict quota: URI missing from parameters: %s", - setup->data); - } + root = i_new(struct dict_quota_root, 1); + return &root->root; +} - t_push(); - args = t_strsplit(t_strdup_until(setup->data, uri++), ":"); - for (; *args != '\0'; args++) { - if (strncmp(*args, "storage=", 8) == 0) { - message_bytes_limit = - strtoull(*args + 8, NULL, 10) * 1024; - } else if (strncmp(*args, "messages=", 9) == 0) - message_count_limit = strtoull(*args + 9, NULL, 10); - } - t_pop(); +static int dict_quota_init(struct quota_root *_root, const char *args) +{ + struct dict_quota_root *root = (struct dict_quota_root *)_root; + const char *username, *p; - if (getenv("DEBUG") != NULL) { - i_info("dict quota: uri = %s", uri); - i_info("dict quota: byte limit = %llu", message_bytes_limit); - i_info("dict quota: count limit = %llu", message_count_limit); + p = args == NULL ? NULL : strchr(args, ':'); + if (p == NULL) { + i_error("dict quota: URI missing from parameters"); + return -1; } - dict = dict_init(uri, getenv("USER")); - if (dict == NULL) - i_fatal("dict quota: dict_init() failed"); + username = t_strdup_until(args, p); + args = p+1; - root = i_new(struct dict_quota_root, 1); - root->root.name = i_strdup(name); - root->root.v = quota_backend_dict.v; - root->dict = dict; - - root->message_bytes_limit = - message_bytes_limit == 0 ? (uint64_t)-1 : message_bytes_limit; - root->message_count_limit = - message_count_limit == 0 ? (uint64_t)-1 : message_count_limit; - return &root->root; + if (*username == '\0') + username = getenv("USER"); + + if (getenv("DEBUG") != NULL) + i_info("dict quota: user = %s, uri = %s", username, args); + + /* FIXME: we should use 64bit integer as datatype instead but before + it can actually be used don't bother */ + root->dict = dict_init(args, username); + return root->dict != NULL ? 0 : -1; } static void dict_quota_deinit(struct quota_root *_root) { struct dict_quota_root *root = (struct dict_quota_root *)_root; - i_free(root->root.name); + if (root->dict != NULL) + dict_deinit(&root->dict); i_free(root); } -static bool -dict_quota_add_storage(struct quota_root *root __attr_unused__, - struct mail_storage *storage __attr_unused__) -{ - return TRUE; -} - -static void -dict_quota_remove_storage(struct quota_root *root __attr_unused__, - struct mail_storage *storage __attr_unused__) -{ -} - static const char *const * dict_quota_root_get_resources(struct quota_root *root __attr_unused__) { - static const char *resources[] = { QUOTA_NAME_STORAGE, NULL }; + static const char *resources[] = { + QUOTA_NAME_STORAGE_KILOBYTES, QUOTA_NAME_MESSAGES, NULL + }; return resources; } -static struct mail_storage * -dict_quota_root_get_storage(struct quota_root *root) -{ - /* FIXME: figure out how to support multiple storages */ - struct mail_storage *const *storages; - unsigned int count; - - storages = array_get(&root->storages, &count); - i_assert(count > 0); - - return storages[0]; -} - -static int dict_quota_lookup(struct dict_quota_root *root, const char *path, - uint64_t *value_r) +static int +dict_quota_count(struct dict_quota_root *root, + bool want_bytes, uint64_t *value_r) { struct dict_transaction_context *dt; - const char *value; uint64_t bytes, count; - int ret; - i_assert(!root->counting); - - t_push(); - ret = dict_lookup(root->dict, unsafe_data_stack_pool, path, &value); - if (ret > 0) { - long long tmp; - - tmp = strtoll(value, NULL, 10); - if (tmp >= 0) { - *value_r = tmp; - t_pop(); - return 0; - } - /* negative quota. recalculate it. we don't track expunges - entirely correctly, so this can happen if two processes - expunge at the same time. */ - } - t_pop(); - - if (ret < 0) - return -1; - - /* not found, recalculate the quota */ - root->counting = TRUE; - ret = quota_count_storage(dict_quota_root_get_storage(&root->root), - &bytes, &count); - root->counting = FALSE; - - if (ret < 0) + if (quota_count(root->root.quota, &bytes, &count) < 0) return -1; t_push(); dt = dict_transaction_begin(root->dict); - if (root->message_bytes_limit != (uint64_t)-1) - dict_set(dt, DICT_QUOTA_CURRENT_BYTES_PATH, dec2str(bytes)); - if (root->message_count_limit != (uint64_t)-1) - dict_set(dt, DICT_QUOTA_CURRENT_COUNT_PATH, dec2str(count)); + dict_set(dt, DICT_QUOTA_CURRENT_BYTES_PATH, dec2str(bytes)); + dict_set(dt, DICT_QUOTA_CURRENT_COUNT_PATH, dec2str(count)); t_pop(); if (dict_transaction_commit(dt) < 0) i_error("dict_quota: Couldn't update quota"); - if (strcmp(path, DICT_QUOTA_CURRENT_BYTES_PATH) == 0) - *value_r = bytes; - else { - i_assert(strcmp(path, DICT_QUOTA_CURRENT_COUNT_PATH) == 0); - *value_r = count; - } - return 0; + *value_r = want_bytes ? bytes : count; + return 1; } static int dict_quota_get_resource(struct quota_root *_root, const char *name, - uint64_t *value_r, uint64_t *limit_r) + uint64_t *value_r, uint64_t *limit __attr_unused__) { struct dict_quota_root *root = (struct dict_quota_root *)_root; + const char *value; + bool want_bytes; + int ret; - if (strcmp(name, QUOTA_NAME_STORAGE) == 0) { - if (root->message_bytes_limit == (uint64_t)-1) - return 0; - - *limit_r = root->message_bytes_limit / 1024; - if (dict_quota_lookup(root, DICT_QUOTA_CURRENT_BYTES_PATH, - value_r) < 0) - return -1; - *value_r /= 1024; - } else if (strcmp(name, QUOTA_NAME_MESSAGES) == 0) { - if (root->message_count_limit == (uint64_t)-1) - return 0; - - *limit_r = root->message_count_limit; - if (dict_quota_lookup(root, DICT_QUOTA_CURRENT_COUNT_PATH, - value_r) < 0) - return -1; - } else { + if (strcmp(name, QUOTA_NAME_STORAGE_BYTES) == 0) + want_bytes = TRUE; + else if (strcmp(name, QUOTA_NAME_MESSAGES) == 0) + want_bytes = FALSE; + else return 0; - } - return 1; -} - -static int -dict_quota_set_resource(struct quota_root *root, - const char *name __attr_unused__, - uint64_t value __attr_unused__) -{ - quota_set_error(root->setup->quota, MAIL_STORAGE_ERR_NO_PERMISSION); - return -1; -} - -static struct quota_root_transaction_context * -dict_quota_transaction_begin(struct quota_root *_root, - struct quota_transaction_context *_ctx, - struct mailbox *box __attr_unused__) -{ - struct dict_quota_root *root = (struct dict_quota_root *)_root; - struct quota_root_transaction_context *ctx; + t_push(); + ret = dict_lookup(root->dict, unsafe_data_stack_pool, + want_bytes ? DICT_QUOTA_CURRENT_BYTES_PATH : + DICT_QUOTA_CURRENT_COUNT_PATH, &value); + if (ret < 0) + *value_r = 0; + else if (ret == 0) + ret = dict_quota_count(root, want_bytes, value_r); + else { + long long tmp; - ctx = i_new(struct quota_root_transaction_context, 1); - ctx->root = _root; - ctx->ctx = _ctx; - - ctx->bytes_limit = root->message_bytes_limit; - ctx->count_limit = root->message_count_limit; - - if (root->counting) { - /* created by quota_count_storage(), we don't care about - the quota there */ - ctx->bytes_limit = (uint64_t)-1; - ctx->count_limit = (uint64_t)-1; - return ctx; + /* recalculate quota if it's negative or if it wasn't found */ + tmp = ret == 0 ? -1 : strtoll(value, NULL, 10); + if (tmp < 0) + ret = dict_quota_count(root, want_bytes, value_r); + else + *value_r = tmp; } - t_push(); - if (ctx->bytes_limit != (uint64_t)-1) { - if (dict_quota_lookup(root, DICT_QUOTA_CURRENT_BYTES_PATH, - &ctx->bytes_current) < 0) - ctx->bytes_current = 0; - } - if (ctx->count_limit != (uint64_t)-1) { - if (dict_quota_lookup(root, DICT_QUOTA_CURRENT_COUNT_PATH, - &ctx->count_current) < 0) - ctx->bytes_current = 0; - } t_pop(); - return ctx; + return ret; } static int -dict_quota_transaction_commit(struct quota_root_transaction_context *ctx) +dict_quota_update(struct quota_root *_root, + struct quota_transaction_context *ctx) { - struct dict_quota_root *root = (struct dict_quota_root *)ctx->root; + struct dict_quota_root *root = (struct dict_quota_root *) _root; struct dict_transaction_context *dt; dt = dict_transaction_begin(root->dict); - if (ctx->bytes_limit != (uint64_t)-1) { - dict_atomic_inc(dt, DICT_QUOTA_CURRENT_BYTES_PATH, - ctx->bytes_diff); - } - if (ctx->count_limit != (uint64_t)-1) { - dict_atomic_inc(dt, DICT_QUOTA_CURRENT_COUNT_PATH, - ctx->count_diff); + + if (ctx->recalculate) { + dict_set(dt, DICT_QUOTA_CURRENT_BYTES_PATH, "-1"); + dict_set(dt, DICT_QUOTA_CURRENT_COUNT_PATH, "-1"); + } else { + if (ctx->bytes_used != 0) { + dict_atomic_inc(dt, DICT_QUOTA_CURRENT_BYTES_PATH, + ctx->bytes_used); + } + if (ctx->count_used != 0) { + dict_atomic_inc(dt, DICT_QUOTA_CURRENT_COUNT_PATH, + ctx->count_used); + } } + if (dict_transaction_commit(dt) < 0) - i_error("dict_quota: Couldn't update quota"); - - i_free(ctx); + return -1; return 0; } -static void -dict_quota_transaction_rollback(struct quota_root_transaction_context *ctx) -{ - i_free(ctx); -} - struct quota_backend quota_backend_dict = { "dict", { + dict_quota_alloc, dict_quota_init, dict_quota_deinit, - - dict_quota_add_storage, - dict_quota_remove_storage, - + NULL, + NULL, dict_quota_root_get_resources, - dict_quota_get_resource, - dict_quota_set_resource, - - dict_quota_transaction_begin, - dict_quota_transaction_commit, - dict_quota_transaction_rollback, - - quota_default_try_alloc, - quota_default_try_alloc_bytes, - quota_default_test_alloc_bytes, - quota_default_alloc, - quota_default_free + dict_quota_update } }; diff -ru dovecot-1.0/src/plugins/quota/quota-dirsize.c dovecot-1.0-quotarewrite/src/plugins/quota/quota-dirsize.c --- dovecot-1.0/src/plugins/quota/quota-dirsize.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota-dirsize.c 2007-11-27 05:42:11.000000000 +0200 @@ -13,66 +13,27 @@ #include <dirent.h> #include <sys/stat.h> -struct dirsize_quota_root { - struct quota_root root; - - uint64_t storage_limit; +struct quota_count_path { + const char *path; + bool is_file; }; extern struct quota_backend quota_backend_dirsize; -static struct quota_root * -dirsize_quota_init(struct quota_setup *setup, const char *name) +static struct quota_root *dirsize_quota_alloc(void) { - struct dirsize_quota_root *root; - const char *const *args; - - root = i_new(struct dirsize_quota_root, 1); - root->root.name = i_strdup(name); - root->root.v = quota_backend_dirsize.v; - - t_push(); - args = t_strsplit(setup->data, ":"); - - for (; *args != '\0'; args++) { - if (strncmp(*args, "storage=", 8) == 0) - root->storage_limit = strtoull(*args + 8, NULL, 10); - } - t_pop(); - - if (getenv("DEBUG") != NULL) { - i_info("dirsize quota limit = %llukB", - (unsigned long long)root->storage_limit); - } - - return &root->root; + return i_new(struct quota_root, 1); } static void dirsize_quota_deinit(struct quota_root *_root) { - struct dirsize_quota_root *root = (struct dirsize_quota_root *)_root; - - i_free(root->root.name); - i_free(root); -} - -static bool -dirsize_quota_add_storage(struct quota_root *root __attr_unused__, - struct mail_storage *storage __attr_unused__) -{ - return TRUE; -} - -static void -dirsize_quota_remove_storage(struct quota_root *root __attr_unused__, - struct mail_storage *storage __attr_unused__) -{ + i_free(_root); } static const char *const * dirsize_quota_root_get_resources(struct quota_root *root __attr_unused__) { - static const char *resources[] = { QUOTA_NAME_STORAGE, NULL }; + static const char *resources[] = { QUOTA_NAME_STORAGE_KILOBYTES, NULL }; return resources; } @@ -133,8 +94,7 @@ return ret; } -static int get_usage(struct dirsize_quota_root *root, - const char *path, bool is_file, uint64_t *value_r) +static int get_usage(const char *path, bool is_file, uint64_t *value_r) { struct stat st; @@ -148,21 +108,14 @@ } *value_r += st.st_size; } else { - if (get_dir_usage(path, value_r) < 0) { - quota_set_error(root->root.setup->quota, - "Internal quota calculation error"); + if (get_dir_usage(path, value_r) < 0) return -1; - } } return 0; } -struct quota_count_path { - const char *path; - bool is_file; -}; - -static void quota_count_path_add(array_t *paths, const char *path, bool is_file) +static void quota_count_path_add(array_t *paths, + const char *path, bool is_file) { ARRAY_SET_TYPE(paths, struct quota_count_path); struct quota_count_path *count_path; @@ -173,7 +126,7 @@ for (i = 0; i < count; ) { if (strncmp(count_path[i].path, path, strlen(count_path[i].path)) == 0) { - /* this path is already being counted */ + /* this path has already been counted */ return; } if (strncmp(count_path[i].path, path, path_len) == 0 && @@ -193,7 +146,7 @@ } static int -get_quota_root_usage(struct dirsize_quota_root *root, uint64_t *value_r) +get_quota_root_usage(struct quota_root *root, uint64_t *value_r) { struct mail_storage *const *storages; array_t ARRAY_DEFINE(paths, struct quota_count_path); @@ -205,7 +158,7 @@ t_push(); ARRAY_CREATE(&paths, pool_datastack_create(), struct quota_count_path, 8); - storages = array_get(&root->root.storages, &count); + storages = array_get(&root->quota->storages, &count); for (i = 0; i < count; i++) { path = mail_storage_get_mailbox_path(storages[i], "", &is_file); quota_count_path_add(&paths, path, is_file); @@ -217,9 +170,10 @@ } /* now sum up the found paths */ + *value_r = 0; count_paths = array_get(&paths, &count); for (i = 0; i < count; i++) { - if (get_usage(root, count_paths[i].path, count_paths[i].is_file, + if (get_usage(count_paths[i].path, count_paths[i].is_file, value_r) < 0) { t_pop(); return -1; @@ -227,98 +181,40 @@ } t_pop(); - return 0; } static int dirsize_quota_get_resource(struct quota_root *_root, const char *name, - uint64_t *value_r, uint64_t *limit_r) + uint64_t *value_r, uint64_t *limit __attr_unused__) { - struct dirsize_quota_root *root = (struct dirsize_quota_root *)_root; - - *value_r = 0; - *limit_r = 0; - - if (strcasecmp(name, QUOTA_NAME_STORAGE) != 0) + if (strcasecmp(name, QUOTA_NAME_STORAGE_BYTES) != 0) return 0; - if (get_quota_root_usage(root, value_r) < 0) + if (get_quota_root_usage(_root, value_r) < 0) return -1; - *value_r /= 1024; - *limit_r = root->storage_limit; return 1; } -static int -dirsize_quota_set_resource(struct quota_root *root, - const char *name __attr_unused__, - uint64_t value __attr_unused__) -{ - quota_set_error(root->setup->quota, MAIL_STORAGE_ERR_NO_PERMISSION); - return -1; -} - -static struct quota_root_transaction_context * -dirsize_quota_transaction_begin(struct quota_root *_root, - struct quota_transaction_context *_ctx, - struct mailbox *box __attr_unused__) -{ - struct dirsize_quota_root *root = (struct dirsize_quota_root *)_root; - struct quota_root_transaction_context *ctx; - - ctx = i_new(struct quota_root_transaction_context, 1); - ctx->root = _root; - ctx->ctx = _ctx; - - /* Get dir usage only once at the beginning of transaction. - When copying/appending lots of mails we don't want to re-read the - entire directory structure after each mail. */ - if (get_quota_root_usage(root, &ctx->bytes_current) < 0 || - ctx->bytes_current == (uint64_t)-1) { - ctx->bytes_current = (uint64_t)-1; - quota_set_error(_root->setup->quota, - "Internal quota calculation error"); - } - - ctx->bytes_limit = root->storage_limit * 1024; - ctx->count_limit = (uint64_t)-1; - return ctx; -} - -static int -dirsize_quota_transaction_commit(struct quota_root_transaction_context *ctx) +static int +dirsize_quota_update(struct quota_root *root __attr_unused__, + struct quota_transaction_context *ctx __attr_unused__) { - int ret = ctx->bytes_current == (uint64_t)-1 ? -1 : 0; - - i_free(ctx); - return ret; + return 0; } struct quota_backend quota_backend_dirsize = { "dirsize", { - dirsize_quota_init, + dirsize_quota_alloc, + NULL, dirsize_quota_deinit, - - dirsize_quota_add_storage, - dirsize_quota_remove_storage, - + NULL, + NULL, dirsize_quota_root_get_resources, - dirsize_quota_get_resource, - dirsize_quota_set_resource, - - dirsize_quota_transaction_begin, - dirsize_quota_transaction_commit, - quota_default_transaction_rollback, - - quota_default_try_alloc, - quota_default_try_alloc_bytes, - quota_default_test_alloc_bytes, - quota_default_alloc, - quota_default_free + dirsize_quota_update } }; diff -ru dovecot-1.0/src/plugins/quota/quota-fs.c dovecot-1.0-quotarewrite/src/plugins/quota/quota-fs.c --- dovecot-1.0/src/plugins/quota/quota-fs.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota-fs.c 2007-07-27 13:18:23.000000000 +0300 @@ -56,22 +56,13 @@ struct fs_quota_mountpoint *mount; }; -struct fs_quota_root_iter { - struct quota_root_iter iter; - - bool sent; -}; - extern struct quota_backend quota_backend_fs; -static struct quota_root * -fs_quota_init(struct quota_setup *setup __attr_unused__, const char *name) +static struct quota_root *fs_quota_alloc(void) { struct fs_quota_root *root; root = i_new(struct fs_quota_root, 1); - root->root.name = i_strdup(name); - root->root.v = quota_backend_fs.v; root->uid = geteuid(); return &root->root; @@ -99,7 +90,6 @@ if (root->mount != NULL) fs_quota_mountpoint_free(root->mount); - i_free(root->root.name); i_free(root); } @@ -120,42 +110,66 @@ return mount; } -static bool fs_quota_add_storage(struct quota_root *_root, - struct mail_storage *storage) +static struct fs_quota_root * +fs_quota_root_find_mountpoint(struct quota *quota, + const struct fs_quota_mountpoint *mount) +{ + struct quota_root *const *roots; + struct fs_quota_root *empty = NULL; + unsigned int i, count; + + roots = array_get(&quota->roots, &count); + for (i = 0; i < count; i++) { + if (roots[i]->backend.name == quota_backend_fs.name) { + struct fs_quota_root *root = + (struct fs_quota_root *)roots[i]; + + if (root->mount == NULL) + empty = root; + else if (strcmp(root->mount->mount_path, + mount->mount_path) == 0) + return root; + } + } + return empty; +} + +static void fs_quota_storage_added(struct quota *quota, + struct mail_storage *storage) { - struct fs_quota_root *root = (struct fs_quota_root *)_root; struct fs_quota_mountpoint *mount; + struct quota_root *_root; + struct fs_quota_root *root; const char *dir; bool is_file; dir = mail_storage_get_mailbox_path(storage, "", &is_file); - - if (getenv("DEBUG") != NULL) - i_info("fs quota add storage dir = %s", dir); - mount = fs_quota_mountpoint_get(dir); - if (root->mount == NULL) { - if (mount == NULL) { - /* Not found */ - return TRUE; - } - root->mount = mount; - } else { - bool match = strcmp(root->mount->mount_path, - mount->mount_path) == 0; + if (getenv("DEBUG") != NULL) { + i_info("fs quota add storage dir = %s", dir); + i_info("fs quota block device = %s", mount->device_path); + i_info("fs quota mount point = %s", mount->mount_path); + } + root = fs_quota_root_find_mountpoint(quota, mount); + if (root != NULL && root->mount != NULL) { + /* already exists */ fs_quota_mountpoint_free(mount); - if (!match) { - /* different mountpoints, can't use this */ - return FALSE; - } - mount = root->mount; + return; } - if (getenv("DEBUG") != NULL) { - i_info("fs quota block device = %s", mount->device_path); - i_info("fs quota mount point = %s", mount->mount_path); + if (root == NULL) { + /* create a new root for this mountpoint */ + _root = quota_root_init(quota, quota_backend_fs.name); + root = (struct fs_quota_root *)_root; + root->root.name = + p_strdup_printf(root->root.pool, "%s%d", + quota_backend_fs.name, + array_count(&quota->roots)); + } else { + /* this is the default root. */ } + root->mount = mount; #ifdef HAVE_Q_QUOTACTL if (mount->path == NULL) { @@ -165,19 +179,12 @@ i_error("open(%s) failed: %m", mount->path); } #endif - return TRUE; -} - -static void -fs_quota_remove_storage(struct quota_root *root __attr_unused__, - struct mail_storage *storage __attr_unused__) -{ } static const char *const * fs_quota_root_get_resources(struct quota_root *root __attr_unused__) { - static const char *resources[] = { QUOTA_NAME_STORAGE, NULL }; + static const char *resources[] = { QUOTA_NAME_STORAGE_KILOBYTES, NULL }; return resources; } @@ -195,7 +202,8 @@ *value_r = 0; *limit_r = 0; - if (strcasecmp(name, QUOTA_NAME_STORAGE) != 0 || root->mount == NULL) + if (strcasecmp(name, QUOTA_NAME_STORAGE_BYTES) != 0 || + root->mount == NULL) return 0; #if defined (HAVE_QUOTACTL) && defined(HAVE_SYS_QUOTA_H) @@ -210,14 +218,12 @@ root->uid, (caddr_t)&xdqblk) < 0) { i_error("quotactl(Q_XGETQUOTA, %s) failed: %m", root->mount->device_path); - quota_set_error(_root->setup->quota, - "Internal quota error"); return -1; } /* values always returned in 512 byte blocks */ - *value_r = xdqblk.d_bcount >> 1; - *limit_r = xdqblk.d_blk_softlimit >> 1; + *value_r = xdqblk.d_bcount * 512; + *limit_r = xdqblk.d_blk_softlimit * 512; } else #endif { @@ -233,17 +239,15 @@ "(--with-linux-quota configure option)", _LINUX_QUOTA_VERSION); } - quota_set_error(_root->setup->quota, - "Internal quota error"); return -1; } #if _LINUX_QUOTA_VERSION == 1 - *value_r = dqblk.dqb_curblocks; + *value_r = dqblk.dqb_curblocks * 1024; #else - *value_r = dqblk.dqb_curblocks / 1024; + *value_r = dqblk.dqb_curblocks; #endif - *limit_r = dqblk.dqb_bsoftlimit; + *limit_r = dqblk.dqb_bsoftlimit * 1024; } #elif defined(HAVE_QUOTACTL) /* BSD, AIX */ @@ -251,11 +255,10 @@ root->uid, (void *)&dqblk) < 0) { i_error("quotactl(Q_GETQUOTA, %s) failed: %m", root->mount->mount_path); - quota_set_error(_root->setup->quota, "Internal quota error"); return -1; } - *value_r = (uint64_t)dqblk.dqb_curblocks * DEV_BSIZE / 1024; - *limit_r = (uint64_t)dqblk.dqb_bsoftlimit * DEV_BSIZE / 1024; + *value_r = (uint64_t)dqblk.dqb_curblocks * DEV_BSIZE; + *limit_r = (uint64_t)dqblk.dqb_bsoftlimit * DEV_BSIZE; #else /* Solaris */ if (root->mount->fd == -1) @@ -266,42 +269,18 @@ ctl.addr = (caddr_t)&dqblk; if (ioctl(root->mount->fd, Q_QUOTACTL, &ctl) < 0) { i_error("ioctl(%s, Q_QUOTACTL) failed: %m", root->mount->path); - quota_set_error(_root->setup->quota, "Internal quota error"); return -1; } - *value_r = (uint64_t)dqblk.dqb_curblocks * 1024 / DEV_BSIZE; - *limit_r = (uint64_t)dqblk.dqb_bsoftlimit * 1024 / DEV_BSIZE; + *value_r = (uint64_t)dqblk.dqb_curblocks * DEV_BSIZE; + *limit_r = (uint64_t)dqblk.dqb_bsoftlimit * DEV_BSIZE; #endif return 1; } -static int -fs_quota_set_resource(struct quota_root *root, - const char *name __attr_unused__, - uint64_t value __attr_unused__) -{ - quota_set_error(root->setup->quota, MAIL_STORAGE_ERR_NO_PERMISSION); - return -1; -} - -static struct quota_root_transaction_context * -fs_quota_transaction_begin(struct quota_root *root, - struct quota_transaction_context *ctx, - struct mailbox *box __attr_unused__) -{ - struct quota_root_transaction_context *root_ctx; - - root_ctx = i_new(struct quota_root_transaction_context, 1); - root_ctx->root = root; - root_ctx->ctx = ctx; - root_ctx->disabled = TRUE; - return root_ctx; -} - -static int -fs_quota_transaction_commit(struct quota_root_transaction_context *ctx) +static int +fs_quota_update(struct quota_root *root __attr_unused__, + struct quota_transaction_context *ctx __attr_unused__) { - i_free(ctx); return 0; } @@ -309,26 +288,16 @@ "fs", { - fs_quota_init, + fs_quota_alloc, + NULL, fs_quota_deinit, + NULL, - fs_quota_add_storage, - fs_quota_remove_storage, + fs_quota_storage_added, fs_quota_root_get_resources, - fs_quota_get_resource, - fs_quota_set_resource, - - fs_quota_transaction_begin, - fs_quota_transaction_commit, - quota_default_transaction_rollback, - - quota_default_try_alloc, - quota_default_try_alloc_bytes, - quota_default_test_alloc_bytes, - quota_default_alloc, - quota_default_free + fs_quota_update } }; diff -ru dovecot-1.0/src/plugins/quota/quota.h dovecot-1.0-quotarewrite/src/plugins/quota/quota.h --- dovecot-1.0/src/plugins/quota/quota.h 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota.h 2007-07-27 13:24:10.000000000 +0300 @@ -5,7 +5,9 @@ struct mailbox; /* Message storage size kilobytes. */ -#define QUOTA_NAME_STORAGE "STORAGE" +#define QUOTA_NAME_STORAGE_KILOBYTES "STORAGE" +/* Message storage size bytes. This is used only internally. */ +#define QUOTA_NAME_STORAGE_BYTES "STORAGE_BYTES" /* Number of messages. */ #define QUOTA_NAME_MESSAGES "MESSAGES" @@ -14,25 +16,24 @@ struct quota_root_iter; struct quota_transaction_context; -extern void (*hook_quota_root_created)(struct quota_root *root); - struct quota *quota_init(void); void quota_deinit(struct quota *quota); -/* Create a new quota setup under which quota roots are created. - user_root is TRUE if this quota points to user's own mailboxes instead of - shared mailboxes. */ -struct quota_setup * -quota_setup_init(struct quota *quota, const char *data, bool user_root); -void quota_setup_deinit(struct quota_setup *setup); - /* Create a new quota root. */ -struct quota_root * -quota_root_init(struct quota_setup *setup, const char *name); +struct quota_root *quota_root_init(struct quota *quota, const char *root_def); void quota_root_deinit(struct quota_root *root); +/* Add a new rule too the quota root. Returns 0 if ok, -1 if rule is invalid. */ +int quota_root_add_rule(struct quota_root *root, const char *rule_def, + const char **error_r); +/* Add a new warning rule for the quota root. Returns 0 if ok, -1 if rule is + invalid. */ +int quota_root_add_warning_rule(struct quota_root *root, const char *rule_def, + const char **error_r); + /* List all quota roots. Returned quota roots are freed by quota_deinit(). */ -struct quota_root_iter *quota_root_iter_init(struct mailbox *box); +struct quota_root_iter * +quota_root_iter_init(struct quota *quota, struct mailbox *box); struct quota_root *quota_root_iter_next(struct quota_root_iter *iter); void quota_root_iter_deinit(struct quota_root_iter *iter); @@ -45,34 +46,34 @@ const char *const *quota_root_get_resources(struct quota_root *root); /* Returns 1 if quota value was found, 0 if not, -1 if error. */ -int quota_get_resource(struct quota_root *root, +int quota_get_resource(struct quota_root *root, const char *mailbox_name, const char *name, uint64_t *value_r, uint64_t *limit_r); /* Returns 0 if OK, -1 if error (eg. permission denied, invalid name). */ -int quota_set_resource(struct quota_root *root, - const char *name, uint64_t value); +int quota_set_resource(struct quota_root *root, const char *name, + uint64_t value, const char **error_r); /* Start a new quota transaction. */ -struct quota_transaction_context *quota_transaction_begin(struct mailbox *box); +struct quota_transaction_context *quota_transaction_begin(struct quota *quota, + struct mailbox *box); /* Commit quota transaction. Returns 0 if ok, -1 if failed. */ -int quota_transaction_commit(struct quota_transaction_context *ctx); +int quota_transaction_commit(struct quota_transaction_context **ctx); /* Rollback quota transaction changes. */ -void quota_transaction_rollback(struct quota_transaction_context *ctx); +void quota_transaction_rollback(struct quota_transaction_context **ctx); /* Allocate from quota if there's space. Returns 1 if updated, 0 if not, -1 if error. If mail size is larger than even maximum allowed quota, too_large_r is set to TRUE. */ int quota_try_alloc(struct quota_transaction_context *ctx, struct mail *mail, bool *too_large_r); -int quota_try_alloc_bytes(struct quota_transaction_context *ctx, - uoff_t size, bool *too_large_r); -/* Like quota_try_alloc_bytes(), but don't actually update the quota. */ -int quota_test_alloc_bytes(struct quota_transaction_context *ctx, - uoff_t size, bool *too_large_r); +/* Like quota_try_alloc(), but don't actually allocate anything. */ +int quota_test_alloc(struct quota_transaction_context *ctx, + uoff_t size, bool *too_large_r); /* Update quota by allocating/freeing space used by mail. */ void quota_alloc(struct quota_transaction_context *ctx, struct mail *mail); void quota_free(struct quota_transaction_context *ctx, struct mail *mail); - -/* Returns the last error message. */ -const char *quota_last_error(struct quota *quota); +void quota_free_bytes(struct quota_transaction_context *ctx, + uoff_t physical_size); +/* Mark the quota to be recalculated */ +void quota_recalculate(struct quota_transaction_context *ctx); #endif diff -ru dovecot-1.0/src/plugins/quota/quota-maildir.c dovecot-1.0-quotarewrite/src/plugins/quota/quota-maildir.c --- dovecot-1.0/src/plugins/quota/quota-maildir.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota-maildir.c 2007-11-27 05:42:11.000000000 +0200 @@ -16,14 +16,13 @@ #include <dirent.h> #include <sys/stat.h> -#define MAILDIR_TRASH_MAILBOX "Trash" #define MAILDIRSIZE_FILENAME "maildirsize" #define MAILDIRSIZE_STALE_SECS (60*15) struct maildir_quota_root { struct quota_root root; - char *ignore; + const char *maildirsize_path; uint64_t message_bytes_limit; uint64_t message_count_limit; @@ -31,13 +30,13 @@ uint64_t total_count; int fd; + time_t recalc_last_stamp; + unsigned int limits_initialized:1; unsigned int master_message_limits:1; }; struct maildir_list_context { - struct maildir_quota_root *root; - struct mailbox_list_context *ctx; struct mailbox_list *list; @@ -60,8 +59,8 @@ MEMBER(use_excl_lock) FALSE }; -static int maildir_sum_dir(struct mail_storage *storage, const char *dir, - uint64_t *total_bytes, uint64_t *total_count) +static int maildir_sum_dir(const char *dir, uint64_t *total_bytes, + uint64_t *total_count) { DIR *dirp; struct dirent *dp; @@ -75,8 +74,7 @@ if (dirp == NULL) { if (errno == ENOENT || errno == ESTALE) return 0; - mail_storage_set_critical(storage, "opendir(%s) failed: %m", - dir); + i_error("opendir(%s) failed: %m", dir); return -1; } @@ -115,28 +113,25 @@ *total_bytes += st.st_size; *total_count += 1; } else if (errno != ENOENT && errno != ESTALE) { - mail_storage_set_critical(storage, - "stat(%s) failed: %m", str_c(path)); + i_error("stat(%s) failed: %m", str_c(path)); ret = -1; } } } if (closedir(dirp) < 0) { - mail_storage_set_critical(storage, "closedir(%s) failed: %m", - dir); + i_error("closedir(%s) failed: %m", dir); return -1; } return ret; } static struct maildir_list_context * -maildir_list_init(struct maildir_quota_root *root, struct mail_storage *storage) +maildir_list_init(struct mail_storage *storage) { struct maildir_list_context *ctx; ctx = i_new(struct maildir_list_context, 1); - ctx->root = root; ctx->path = str_new(default_pool, 512); ctx->ctx = mail_storage_mailbox_list_init(storage, "", "*", MAILBOX_LIST_FAST_FLAGS | @@ -158,10 +153,6 @@ return NULL; } - if (ctx->root->ignore != NULL && - strcmp(ctx->list->name, ctx->root->ignore) == 0) - continue; - t_push(); path = mail_storage_get_mailbox_path(ctx->ctx->storage, ctx->list->name, @@ -179,8 +170,7 @@ /* ignore if the directory got lost, stale or if it was actually a file and not a directory */ if (errno != ENOENT && errno != ESTALE && errno != ENOTDIR) { - mail_storage_set_critical(ctx->ctx->storage, - "stat(%s) failed: %m", str_c(ctx->path)); + i_error("stat(%s) failed: %m", str_c(ctx->path)); ctx->state = 0; } } @@ -199,14 +189,13 @@ } static int -maildirs_check_have_changed(struct maildir_quota_root *root, - struct mail_storage *storage, time_t latest_mtime) +maildirs_check_have_changed(struct mail_storage *storage, time_t latest_mtime) { struct maildir_list_context *ctx; time_t mtime; int ret = 0; - ctx = maildir_list_init(root, storage); + ctx = maildir_list_init(storage); while (maildir_list_next(ctx, &mtime) != NULL) { if (mtime > latest_mtime) { ret = 1; @@ -218,8 +207,7 @@ return ret; } -static int maildirsize_write(struct maildir_quota_root *root, - struct mail_storage *storage, const char *path) +static int maildirsize_write(struct maildir_quota_root *root, const char *path) { struct dotlock *dotlock; string_t *str; @@ -227,8 +215,7 @@ i_assert(root->fd == -1); - dotlock_settings.use_excl_lock = - (storage->flags & MAIL_STORAGE_FLAG_DOTLOCK_USE_EXCL) != 0; + dotlock_settings.use_excl_lock = getenv("DOTLOCK_USE_EXCL") != NULL; fd = file_dotlock_open(&dotlock_settings, path, DOTLOCK_CREATE_FLAG_NONBLOCK, &dotlock); if (fd == -1) { @@ -237,8 +224,7 @@ return 1; } - mail_storage_set_critical(storage, - "file_dotlock_open(%s) failed: %m", path); + i_error("file_dotlock_open(%s) failed: %m", path); return -1; } @@ -257,8 +243,7 @@ (unsigned long long)root->total_bytes, (unsigned long long)root->total_count); if (write_full(fd, str_data(str), str_len(str)) < 0) { - mail_storage_set_critical(storage, - "write_full(%s) failed: %m", path); + i_error("write_full(%s) failed: %m", path); file_dotlock_delete(&dotlock); return -1; } @@ -266,38 +251,34 @@ /* keep the fd open since we might want to update it later */ if (file_dotlock_replace(&dotlock, DOTLOCK_REPLACE_FLAG_DONT_CLOSE_FD) < 0) { - mail_storage_set_critical(storage, - "file_dotlock_replace(%s) failed: %m", path); + i_error("file_dotlock_replace(%s) failed: %m", path); return -1; } root->fd = fd; return 0; } -static const char *maildirsize_get_path(struct mail_storage *storage) +static void maildirsize_recalculate_init(struct maildir_quota_root *root) { - return t_strconcat(mail_storage_get_mailbox_control_dir(storage, ""), - "/"MAILDIRSIZE_FILENAME, NULL); + root->total_bytes = root->total_count = 0; + root->recalc_last_stamp = 0; } -static int maildirsize_recalculate(struct maildir_quota_root *root, - struct mail_storage *storage) +static int maildirsize_recalculate_storage(struct maildir_quota_root *root, + struct mail_storage *storage) { struct maildir_list_context *ctx; - const char *dir, *path; - time_t mtime, last_stamp = 0; + const char *dir; + time_t mtime; int ret = 0; - root->total_bytes = root->total_count = 0; - - ctx = maildir_list_init(root, storage); + ctx = maildir_list_init(storage); while ((dir = maildir_list_next(ctx, &mtime)) != NULL) { - if (mtime > last_stamp) - last_stamp = mtime; + if (mtime > root->recalc_last_stamp) + root->recalc_last_stamp = mtime; t_push(); - if (maildir_sum_dir(storage, dir, - &root->total_bytes, + if (maildir_sum_dir(dir, &root->total_bytes, &root->total_count) < 0) ret = -1; t_pop(); @@ -305,59 +286,107 @@ if (maildir_list_deinit(ctx) < 0) ret = -1; - if (ret == 0) - ret = maildirs_check_have_changed(root, storage, last_stamp); + return ret; +} - t_push(); - path = maildirsize_get_path(storage); +static int maildirsize_recalculate_finish(struct maildir_quota_root *root, + int ret) +{ if (ret == 0) { /* maildir didn't change, we can write the maildirsize file */ - ret = maildirsize_write(root, storage, path); + ret = maildirsize_write(root, root->maildirsize_path); } if (ret != 0) { /* make sure it gets rebuilt later */ - if (unlink(path) < 0 && errno != ENOENT && errno != ESTALE) { - mail_storage_set_critical(storage, - "unlink(%s) failed: %m", path); + if (unlink(root->maildirsize_path) < 0 && + errno != ENOENT && errno != ESTALE) { + i_error("unlink(%s) failed: %m", + root->maildirsize_path); } } - t_pop(); return ret; } -static int maildirsize_parse(struct maildir_quota_root *root, - int fd, const char *const *lines) +static int maildirsize_recalculate(struct maildir_quota_root *root) +{ + struct mail_storage *const *storages; + unsigned int i, count; + int ret = 0; + + maildirsize_recalculate_init(root); + + /* count mails from all storages */ + storages = array_get(&root->root.quota->storages, &count); + for (i = 0; i < count; i++) { + if (maildirsize_recalculate_storage(root, storages[i]) < 0) { + ret = -1; + break; + } + } + + if (ret == 0) { + /* check if any of the directories have changed */ + for (i = 0; i < count; i++) { + ret = maildirs_check_have_changed(storages[i], + root->recalc_last_stamp); + if (ret != 0) + break; + } + } + + return maildirsize_recalculate_finish(root, ret); +} + +static bool +maildir_parse_limit(const char *str, uint64_t *bytes_r, uint64_t *count_r) { - unsigned long long bytes; - uint64_t message_bytes_limit, message_count_limit; - long long bytes_diff, total_bytes; - int count_diff, total_count; - unsigned int line_count = 0; const char *const *limit; + unsigned long long value; char *pos; + bool ret = TRUE; - if (*lines == NULL) - return -1; + *bytes_r = (uint64_t)-1; + *count_r = (uint64_t)-1; - /* first line contains the limits. 0 value mean unlimited. */ - message_bytes_limit = (uint64_t)-1; - message_count_limit = (uint64_t)-1; - for (limit = t_strsplit(lines[0], ","); *limit != NULL; limit++) { - bytes = strtoull(*limit, &pos, 10); + /* 0 values mean unlimited */ + for (limit = t_strsplit(str, ","); *limit != NULL; limit++) { + value = strtoull(*limit, &pos, 10); if (pos[0] != '\0' && pos[1] == '\0') { switch (pos[0]) { case 'C': - if (bytes != 0) - message_count_limit = bytes; + if (value != 0) + *count_r = value; break; case 'S': - if (bytes != 0) - message_bytes_limit = bytes; + if (value != 0) + *bytes_r = value; + break; + default: + ret = FALSE; break; } + } else { + ret = FALSE; } } + return ret; +} + +static int maildirsize_parse(struct maildir_quota_root *root, + int fd, const char *const *lines) +{ + uint64_t message_bytes_limit, message_count_limit; + long long bytes_diff, total_bytes; + int count_diff, total_count; + unsigned int line_count = 0; + + if (*lines == NULL) + return -1; + + /* first line contains the limits */ + (void)maildir_parse_limit(lines[0], &message_bytes_limit, + &message_count_limit); if (!root->master_message_limits) { /* we don't know the limits, use whatever the file says */ @@ -409,32 +438,26 @@ return 1; } -static int maildirsize_read(struct maildir_quota_root *root, - struct mail_storage *storage) +static int maildirsize_read(struct maildir_quota_root *root) { - const char *path; char buf[5120+1]; unsigned int i, size; int fd, ret = 0; t_push(); - path = maildirsize_get_path(storage); if (root->fd != -1) { - if (close(root->fd) < 0) { - mail_storage_set_critical(storage, - "close(%s) failed: %m", path); - } + if (close(root->fd) < 0) + i_error("close(%s) failed: %m", root->maildirsize_path); root->fd = -1; } - fd = nfs_safe_open(path, O_RDWR | O_APPEND); + fd = nfs_safe_open(root->maildirsize_path, O_RDWR | O_APPEND); if (fd == -1) { if (errno == ENOENT) ret = 0; else { ret = -1; - mail_storage_set_critical(storage, - "open(%s) failed: %m", path); + i_error("open(%s) failed: %m", root->maildirsize_path); } t_pop(); return ret; @@ -447,8 +470,7 @@ if (ret < 0) { if (errno == ESTALE) break; - mail_storage_set_critical(storage, "read(%s) failed: %m", - path); + i_error("read(%s) failed: %m", root->maildirsize_path); } size += ret; } @@ -488,12 +510,26 @@ return ret; } -static int maildirquota_refresh(struct maildir_quota_root *root, - struct mail_storage *storage) +static void maildirquota_init_limits(struct maildir_quota_root *root) +{ + root->limits_initialized = TRUE; + + if (root->root.default_rule.bytes_limit != 0 || + root->root.default_rule.count_limit != 0) { + root->master_message_limits = TRUE; + root->message_bytes_limit = root->root.default_rule.bytes_limit; + root->message_count_limit = root->root.default_rule.count_limit; + } +} + +static int maildirquota_refresh(struct maildir_quota_root *root) { int ret; - ret = maildirsize_read(root, storage); + if (!root->limits_initialized) + maildirquota_init_limits(root); + + ret = maildirsize_read(root); if (ret == 0) { if (root->message_bytes_limit == (uint64_t)-1 && root->message_count_limit == (uint64_t)-1) { @@ -501,13 +537,12 @@ return 0; } - ret = maildirsize_recalculate(root, storage); + ret = maildirsize_recalculate(root); } return ret < 0 ? -1 : 0; } static int maildirsize_update(struct maildir_quota_root *root, - struct mail_storage *storage, int count_diff, int64_t bytes_diff) { const char *str; @@ -529,52 +564,22 @@ if (errno == ESTALE) { /* deleted/replaced already, ignore */ } else { - mail_storage_set_critical(storage, - "write_full(%s) failed: %m", - maildirsize_get_path(storage)); + i_error("write_full(%s) failed: %m", + root->maildirsize_path); } } t_pop(); return ret; } -static struct quota_root * -maildir_quota_init(struct quota_setup *setup, const char *name __attr_unused__) +static struct quota_root *maildir_quota_alloc(void) { struct maildir_quota_root *root; - const char *const *args; - unsigned long long size; root = i_new(struct maildir_quota_root, 1); - root->root.name = i_strdup(name); - root->root.v = quota_backend_maildir.v; root->fd = -1; root->message_bytes_limit = (uint64_t)-1; root->message_count_limit = (uint64_t)-1; - - t_push(); - args = t_strsplit(setup->data, ":"); - - for (args++; *args != '\0'; args++) { - if (strncmp(*args, "storage=", 8) == 0) { - size = strtoull(*args + 8, NULL, 10) * 1024; - if (size != 0) - root->message_bytes_limit = size; - root->master_message_limits = TRUE; - } else if (strncmp(*args, "messages=", 9) == 0) { - size = strtoull(*args + 9, NULL, 10); - if (size != 0) - root->message_count_limit = size; - root->master_message_limits = TRUE; - } else if (strncmp(*args, "ignore=", 7) == 0) { - i_free(root->ignore); - root->ignore = i_strdup(*args + 7); - } else { - i_error("maildir quota: Unknown setting: %s", *args); - } - } - t_pop(); - return &root->root; } @@ -584,37 +589,69 @@ if (root->fd != -1) (void)close(root->fd); - - i_free(root->ignore); - i_free(root->root.name); i_free(root); } static bool -maildir_quota_add_storage(struct quota_root *root __attr_unused__, - struct mail_storage *_storage) +maildir_quota_parse_rule(struct quota_root *root __attr_unused__, + struct quota_rule *rule, + const char *str, const char **error_r) { - if (strcmp(_storage->name, "maildir") == 0) { - struct maildir_storage *storage = - (struct maildir_storage *)_storage; + uint64_t bytes, count; - /* For newly generated filenames add ,S=size. */ - storage->save_size_in_filename = TRUE; + if (!maildir_parse_limit(str, &bytes, &count)) { + *error_r = "Invalid Maildir++ quota rule"; + return FALSE; } + + rule->bytes_limit = bytes; + rule->count_limit = count; return TRUE; } static void -maildir_quota_remove_storage(struct quota_root *root __attr_unused__, - struct mail_storage *storage __attr_unused__) +maildir_quota_root_storage_added(struct quota_root *_root, + struct mail_storage *storage) { + struct maildir_quota_root *root = (struct maildir_quota_root *)_root; + const char *control_dir; + + if (root->maildirsize_path != NULL) + return; + + control_dir = mail_storage_get_mailbox_control_dir(storage, ""); + root->maildirsize_path = + p_strconcat(_root->pool, control_dir, + "/"MAILDIRSIZE_FILENAME, NULL); +} + +static void +maildir_quota_storage_added(struct quota *quota, + struct mail_storage *_storage) +{ + struct maildir_storage *storage = + (struct maildir_storage *)_storage; + struct quota_root **roots; + unsigned int i, count; + + if (strcmp(_storage->name, "maildir") != 0) + return; + + roots = array_get_modifyable(&quota->roots, &count); + for (i = 0; i < count; i++) { + if (roots[i]->backend.name == quota_backend_maildir.name) + maildir_quota_root_storage_added(roots[i], _storage); + } + + /* For newly generated filenames add ,S=size. */ + storage->save_size_in_filename = TRUE; } static const char *const * maildir_quota_root_get_resources(struct quota_root *root __attr_unused__) { static const char *resources_both[] = { - QUOTA_NAME_STORAGE, + QUOTA_NAME_STORAGE_KILOBYTES, QUOTA_NAME_MESSAGES, NULL }; @@ -622,129 +659,50 @@ return resources_both; } -static struct mail_storage * -maildir_quota_root_get_storage(struct quota_root *root) -{ - /* FIXME: figure out how to support multiple storages */ - struct mail_storage *const *storages; - unsigned int count; - - storages = array_get(&root->storages, &count); - i_assert(count > 0); - - return storages[0]; -} - static int maildir_quota_get_resource(struct quota_root *_root, const char *name, - uint64_t *value_r, uint64_t *limit_r) + uint64_t *value_r, uint64_t *limit __attr_unused__) { struct maildir_quota_root *root = (struct maildir_quota_root *)_root; - if (maildirquota_refresh(root, - maildir_quota_root_get_storage(_root)) < 0) + if (maildirquota_refresh(root) < 0) return -1; - if (strcmp(name, QUOTA_NAME_STORAGE) == 0) { - if (root->message_bytes_limit == (uint64_t)-1) - return 0; - - *limit_r = root->message_bytes_limit / 1024; - *value_r = root->total_bytes / 1024; - } else if (strcmp(name, QUOTA_NAME_MESSAGES) == 0) { - if (root->message_count_limit == (uint64_t)-1) - return 0; - - *limit_r = root->message_count_limit; + if (strcmp(name, QUOTA_NAME_STORAGE_BYTES) == 0) + *value_r = root->total_bytes; + else if (strcmp(name, QUOTA_NAME_MESSAGES) == 0) *value_r = root->total_count; - } else { + else return 0; - } return 1; } static int -maildir_quota_set_resource(struct quota_root *root, - const char *name __attr_unused__, - uint64_t value __attr_unused__) -{ - quota_set_error(root->setup->quota, MAIL_STORAGE_ERR_NO_PERMISSION); - return -1; -} - -static struct quota_root_transaction_context * -maildir_quota_transaction_begin(struct quota_root *_root, - struct quota_transaction_context *_ctx, - struct mailbox *box) -{ - struct maildir_quota_root *root = (struct maildir_quota_root *)_root; - struct quota_root_transaction_context *ctx; - - ctx = i_new(struct quota_root_transaction_context, 1); - ctx->root = _root; - ctx->ctx = _ctx; - - if (root->ignore != NULL && - strcmp(mailbox_get_name(box), root->ignore) == 0) { - ctx->bytes_limit = (uint64_t)-1; - ctx->count_limit = (uint64_t)-1; - ctx->ignored = TRUE; - return ctx; - } - - if (maildirquota_refresh(root, - maildir_quota_root_get_storage(_root)) < 0) { - /* failed calculating the current quota */ - ctx->bytes_current = (uint64_t)-1; - } else { - ctx->bytes_limit = root->message_bytes_limit; - ctx->count_limit = root->message_count_limit; - ctx->bytes_current = root->total_bytes; - ctx->count_current = root->total_count; - } - return ctx; -} - -static int -maildir_quota_transaction_commit(struct quota_root_transaction_context *ctx) +maildir_quota_update(struct quota_root *_root, + struct quota_transaction_context *ctx) { struct maildir_quota_root *root = - (struct maildir_quota_root *)ctx->root; - int ret = ctx->bytes_current == (uint64_t)-1 ? -1 : 0; + (struct maildir_quota_root *) _root; - if (root->fd != -1 && ret == 0 && !ctx->ignored) { + if (root->fd != -1) { /* if writing fails, we don't care all that much */ - (void)maildirsize_update(root, - maildir_quota_root_get_storage(ctx->root), - ctx->count_diff, ctx->bytes_diff); + (void)maildirsize_update(root, ctx->count_used, + ctx->bytes_used); } - i_free(ctx); - return ret; + return 0; } struct quota_backend quota_backend_maildir = { "maildir", { - maildir_quota_init, + maildir_quota_alloc, + NULL, maildir_quota_deinit, - - maildir_quota_add_storage, - maildir_quota_remove_storage, - + maildir_quota_parse_rule, + maildir_quota_storage_added, maildir_quota_root_get_resources, - maildir_quota_get_resource, - maildir_quota_set_resource, - - maildir_quota_transaction_begin, - maildir_quota_transaction_commit, - quota_default_transaction_rollback, - - quota_default_try_alloc, - quota_default_try_alloc_bytes, - quota_default_test_alloc_bytes, - quota_default_alloc, - quota_default_free + maildir_quota_update } }; diff -ru dovecot-1.0/src/plugins/quota/quota-plugin.c dovecot-1.0-quotarewrite/src/plugins/quota/quota-plugin.c --- dovecot-1.0/src/plugins/quota/quota-plugin.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota-plugin.c 2007-07-27 13:24:10.000000000 +0300 @@ -1,4 +1,4 @@ -/* Copyright (C) 2005 Timo Sirainen */ +/* Copyright (C) 2005 Timo Sirainen & Tianyan Liu */ #include "lib.h" #include "mail-storage.h" @@ -15,20 +15,95 @@ const char *quota_plugin_version = PACKAGE_VERSION; struct quota *quota_set; +static void quota_root_add_rules(const char *root_name, + struct quota_root *root) +{ + const char *rule_name, *rule, *error; + unsigned int i; + + t_push(); + + rule_name = t_strconcat(root_name, "_RULE", NULL); + for (i = 2;; i++) { + rule = getenv(rule_name); + + if (rule == NULL) + break; + + if (quota_root_add_rule(root, rule, &error) < 0) { + i_fatal("Quota root %s: Invalid rule: %s", + root_name, rule); + } + rule_name = t_strdup_printf("%s_RULE%d", root_name, i); + } + + t_pop(); +} + +static void quota_root_add_warning_rules(const char *root_name, + struct quota_root *root) +{ + const char *rule_name, *rule, *error; + unsigned int i; + + t_push(); + + rule_name = t_strconcat(root_name, "_WARNING", NULL); + for (i = 2;; i++) { + rule = getenv(rule_name); + + if (rule == NULL) + break; + + if (quota_root_add_warning_rule(root, rule, &error) < 0) { + i_fatal("Quota root %s: Invalid warning rule: %s", + root_name, rule); + } + rule_name = t_strdup_printf("%s_WARNING%d", root_name, i); + } + + t_pop(); +} + void quota_plugin_init(void) { + struct quota_root *root; + unsigned int i; const char *env; env = getenv("QUOTA"); - if (env != NULL) { - quota_set = quota_init(); - /* Currently we support only one quota setup */ - (void)quota_setup_init(quota_set, env, TRUE); - - quota_next_hook_mail_storage_created = - hook_mail_storage_created; - hook_mail_storage_created = quota_mail_storage_created; + if (env == NULL) + return; + + quota_set = quota_init(); + + root = quota_root_init(quota_set, env); + if (root == NULL) + i_fatal("Couldn't create quota root: %s", env); + quota_root_add_rules("QUOTA", root); + quota_root_add_warning_rules("QUOTA", root); + + t_push(); + for (i = 2;; i++) { + const char *root_name; + + root_name = t_strdup_printf("QUOTA%d", i); + env = getenv(root_name); + + if (env == NULL) + break; + + root = quota_root_init(quota_set, env); + if (root == NULL) + i_fatal("Couldn't create quota root: %s", env); + quota_root_add_rules(root_name, root); + quota_root_add_warning_rules(root_name, root); } + t_pop(); + + quota_next_hook_mail_storage_created = + hook_mail_storage_created; + hook_mail_storage_created = quota_mail_storage_created; } void quota_plugin_deinit(void) diff -ru dovecot-1.0/src/plugins/quota/quota-private.h dovecot-1.0-quotarewrite/src/plugins/quota/quota-private.h --- dovecot-1.0/src/plugins/quota/quota-private.h 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota-private.h 2007-07-27 13:24:29.000000000 +0300 @@ -9,129 +9,93 @@ extern unsigned int quota_module_id; struct quota { - array_t ARRAY_DEFINE(setups, struct quota_setup *); - char *last_error; + array_t ARRAY_DEFINE(roots, struct quota_root *); + array_t ARRAY_DEFINE(storages, struct mail_storage *); + + int (*test_alloc)(struct quota_transaction_context *ctx, + uoff_t size, bool *too_large_r); + + unsigned int debug:1; }; -struct quota_setup { - struct quota *quota; +struct quota_rule { + char *mailbox_name; - struct quota_backend *backend; - char *data; + int64_t bytes_limit, count_limit; +}; - /* List of quota roots. It's array because there shouldn't be many. */ - array_t ARRAY_DEFINE(roots, struct quota_root *); +struct quota_warning_rule { + uint64_t bytes_limit; + uint64_t count_limit; - unsigned int user_root:1; + char *command; }; struct quota_backend_vfuncs { - struct quota_root *(*init)(struct quota_setup *setup, const char *name); + struct quota_root *(*alloc)(void); + int (*init)(struct quota_root *root, const char *args); void (*deinit)(struct quota_root *root); - bool (*add_storage)(struct quota_root *root, - struct mail_storage *storage); - void (*remove_storage)(struct quota_root *root, - struct mail_storage *storage); + bool (*parse_rule)(struct quota_root *root, struct quota_rule *rule, + const char *str, const char **error_r); + + /* called once for each backend */ + void (*storage_added)(struct quota *quota, + struct mail_storage *storage); const char *const *(*get_resources)(struct quota_root *root); + /* the limit is set by default, so it shouldn't normally need to + be changed. */ int (*get_resource)(struct quota_root *root, const char *name, - uint64_t *value_r, uint64_t *limit_r); - int (*set_resource)(struct quota_root *root, - const char *name, uint64_t value); - - struct quota_root_transaction_context * - (*transaction_begin)(struct quota_root *root, - struct quota_transaction_context *ctx, - struct mailbox *box); - int (*transaction_commit)(struct quota_root_transaction_context *ctx); - void (*transaction_rollback) - (struct quota_root_transaction_context *ctx); - - int (*try_alloc)(struct quota_root_transaction_context *ctx, - struct mail *mail, bool *too_large_r); - int (*try_alloc_bytes)(struct quota_root_transaction_context *ctx, - uoff_t size, bool *too_large_r); - int (*test_alloc_bytes)(struct quota_root_transaction_context *ctx, - uoff_t size, bool *too_large_r); - void (*alloc)(struct quota_root_transaction_context *ctx, - struct mail *mail); - void (*free)(struct quota_root_transaction_context *ctx, - struct mail *mail); + uint64_t *value_r, uint64_t *limit); + + int (*update)(struct quota_root *root, + struct quota_transaction_context *ctx); }; struct quota_backend { + /* quota backends equal if backend1.name == backend2.name */ const char *name; struct quota_backend_vfuncs v; }; struct quota_root { - struct quota_setup *setup; + pool_t pool; /* Unique quota root name. */ - char *name; + const char *name; - struct quota_backend_vfuncs v; + /* pointer to the quota that owns this root */ + struct quota *quota; + + struct quota_backend backend; + struct quota_rule default_rule; + array_t ARRAY_DEFINE(rules, struct quota_rule); + array_t ARRAY_DEFINE(warning_rules, struct quota_warning_rule); - /* Mail storages using this quota root. */ - array_t ARRAY_DEFINE(storages, struct mail_storage *); /* Module-specific contexts. See quota_module_id. */ array_t ARRAY_DEFINE(quota_module_contexts, void); - - unsigned int user_root:1; -}; - -struct quota_root_iter { - struct quota_mail_storage *qstorage; - unsigned int idx; }; struct quota_transaction_context { - array_t ARRAY_DEFINE(root_transactions, - struct quota_root_transaction_context *); -}; - -struct quota_root_transaction_context { - struct quota_root *root; - struct quota_transaction_context *ctx; + struct quota *quota; + struct mailbox *box; - int count_diff; - int64_t bytes_diff; + int64_t bytes_used, count_used; + uint64_t bytes_left, count_left; - uint64_t bytes_limit, count_limit; - uint64_t bytes_current, count_current; + struct mail *tmp_mail; - unsigned int ignored:1; - unsigned int disabled:1; + unsigned int limits_set:1; + unsigned int failed:1; + unsigned int recalculate:1; }; /* Register storage to all user's quota roots. */ void quota_add_user_storage(struct quota *quota, struct mail_storage *storage); +void quota_remove_user_storage(struct quota *quota, + struct mail_storage *storage); -/* Likn root and storage together. Returns TRUE if successful, FALSE if it - can't be done (eg. different filesystems with filesystem quota) */ -bool quota_mail_storage_add_root(struct mail_storage *storage, - struct quota_root *root); -void quota_mail_storage_remove_root(struct mail_storage *storage, - struct quota_root *root); - -void quota_set_error(struct quota *quota, const char *errormsg); - -/* default simple implementations for bytes/count updating */ -void -quota_default_transaction_rollback(struct quota_root_transaction_context *ctx); -int quota_default_try_alloc(struct quota_root_transaction_context *ctx, - struct mail *mail, bool *too_large_r); -int quota_default_try_alloc_bytes(struct quota_root_transaction_context *ctx, - uoff_t size, bool *too_large_r); -int quota_default_test_alloc_bytes(struct quota_root_transaction_context *ctx, - uoff_t size, bool *too_large_r); -void quota_default_alloc(struct quota_root_transaction_context *ctx, - struct mail *mail); -void quota_default_free(struct quota_root_transaction_context *ctx, - struct mail *mail); - -int quota_count_storage(struct mail_storage *storage, - uint64_t *bytes_r, uint64_t *count_r); +int quota_count(struct quota *quota, uint64_t *bytes_r, uint64_t *count_r); #endif diff -ru dovecot-1.0/src/plugins/quota/quota-storage.c dovecot-1.0-quotarewrite/src/plugins/quota/quota-storage.c --- dovecot-1.0/src/plugins/quota/quota-storage.c 2007-11-27 05:41:59.000000000 +0200 +++ dovecot-1.0-quotarewrite/src/plugins/quota/quota-storage.c 2007-11-27 05:44:09.000000000 +0200 @@ -16,10 +16,6 @@ struct quota_mail_storage { struct mail_storage_vfuncs super; - struct quota *quota; - - /* List of quota roots this storage belongs to. */ - array_t ARRAY_DEFINE(roots, struct quota_root *); }; struct quota_mailbox { @@ -58,7 +54,7 @@ struct quota_transaction_context *qt; t = qbox->super.transaction_begin(box, flags); - qt = quota_transaction_begin(box); + qt = quota_transaction_begin(quota_set, box); array_idx_set(&t->module_contexts, quota_storage_module_id, &qt); return t; @@ -72,10 +68,12 @@ struct quota_transaction_context *qt = QUOTA_CONTEXT(ctx); if (qbox->super.transaction_commit(ctx, flags) < 0) { - quota_transaction_rollback(qt); + quota_transaction_rollback(&qt); return -1; } else { - (void)quota_transaction_commit(qt); + if (qt->tmp_mail != NULL) + mail_free(&qt->tmp_mail); + (void)quota_transaction_commit(&qt); return 0; } } @@ -87,7 +85,10 @@ struct quota_transaction_context *qt = QUOTA_CONTEXT(ctx); qbox->super.transaction_rollback(ctx); - quota_transaction_rollback(qt); + + if (qt->tmp_mail != NULL) + mail_free(&qt->tmp_mail); + quota_transaction_rollback(&qt); } static struct mail * @@ -124,8 +125,8 @@ mail_storage_set_error(t->box->storage, "Quota exceeded"); return -1; } else { - mail_storage_set_error(t->box->storage, "%s", - quota_last_error(quota_set)); + mail_storage_set_critical(t->box->storage, + "Internal quota calculation error"); return -1; } } @@ -135,26 +136,25 @@ enum mail_flags flags, struct mail_keywords *keywords, struct mail *dest_mail) { + struct quota_transaction_context *qt = QUOTA_CONTEXT(t); struct quota_mailbox *qbox = QUOTA_CONTEXT(t->box); - struct mail *copy_dest_mail; - int ret; - if (dest_mail != NULL) - copy_dest_mail = dest_mail; - else - copy_dest_mail = mail_alloc(t, MAIL_FETCH_PHYSICAL_SIZE, NULL); + if (dest_mail == NULL) { + /* we always want to know the mail size */ + if (qt->tmp_mail == NULL) { + qt->tmp_mail = mail_alloc(t, MAIL_FETCH_PHYSICAL_SIZE, + NULL); + } + dest_mail = qt->tmp_mail; + } qbox->save_hack = FALSE; - if (qbox->super.copy(t, mail, flags, keywords, copy_dest_mail) < 0) + if (qbox->super.copy(t, mail, flags, keywords, dest_mail) < 0) return -1; /* if copying used saving internally, we already checked the quota and set qbox->save_hack = TRUE. */ - ret = qbox->save_hack ? 0 : quota_check(t, copy_dest_mail); - - if (copy_dest_mail != dest_mail) - mail_free(&copy_dest_mail); - return ret; + return qbox->save_hack ? 0 : quota_check(t, dest_mail); } static int @@ -183,14 +183,14 @@ full mail. */ bool too_large; - ret = quota_test_alloc_bytes(qt, st->st_size, &too_large); + ret = quota_test_alloc(qt, st->st_size, &too_large); if (ret == 0) { mail_storage_set_error(t->box->storage, "Quota exceeded"); return -1; } else if (ret < 0) { - mail_storage_set_error(t->box->storage, "%s", - quota_last_error(quota_set)); + mail_storage_set_critical(t->box->storage, + "Internal quota calculation error"); return -1; } } @@ -300,22 +300,8 @@ static void quota_storage_destroy(struct mail_storage *storage) { struct quota_mail_storage *qstorage = QUOTA_CONTEXT(storage); - struct quota_root *const *roots; - struct mail_storage *const *storages; - unsigned int i, j, root_count, storage_count; - - /* remove the storage from all roots' storages list */ - roots = array_get(&qstorage->roots, &root_count); - for (i = 0; i < root_count; i++) { - storages = array_get(&roots[i]->storages, &storage_count); - for (j = 0; j < storage_count; j++) { - if (storages[j] == storage) { - array_delete(&roots[i]->storages, j, 1); - break; - } - } - i_assert(j != storage_count); - } + + quota_remove_user_storage(quota_set, storage); qstorage->super.destroy(storage); } @@ -333,8 +319,6 @@ storage->v.mailbox_open = quota_mailbox_open; storage->v.mailbox_delete = quota_mailbox_delete; - ARRAY_CREATE(&qstorage->roots, storage->pool, struct quota_root *, 4); - if (!quota_storage_module_id_set) { quota_storage_module_id = mail_storage_module_id++; quota_storage_module_id_set = TRUE; @@ -348,74 +332,3 @@ quota_add_user_storage(quota_set, storage); } } - -bool quota_mail_storage_add_root(struct mail_storage *storage, - struct quota_root *root) -{ - struct quota_mail_storage *qstorage = QUOTA_CONTEXT(storage); - - if (!root->v.add_storage(root, storage)) - return FALSE; - - array_append(&root->storages, &storage, 1); - array_append(&qstorage->roots, &root, 1); - return TRUE; -} - -void quota_mail_storage_remove_root(struct mail_storage *storage, - struct quota_root *root) -{ - struct quota_mail_storage *qstorage = QUOTA_CONTEXT(storage); - struct mail_storage *const *storages; - struct quota_root *const *roots; - unsigned int i, count; - - storages = array_get(&root->storages, &count); - for (i = 0; i < count; i++) { - if (storages[i] == storage) { - array_delete(&root->storages, i, 1); - break; - } - } - i_assert(i != count); - - roots = array_get(&qstorage->roots, &count); - for (i = 0; i < count; i++) { - if (roots[i] == root) { - array_delete(&qstorage->roots, i, 1); - break; - } - } - i_assert(i != count); - - root->v.remove_storage(root, storage); -} - -struct quota_root_iter *quota_root_iter_init(struct mailbox *box) -{ - struct quota_mail_storage *qstorage = QUOTA_CONTEXT(box->storage); - struct quota_root_iter *iter; - - iter = i_new(struct quota_root_iter, 1); - iter->qstorage = qstorage; - return iter; -} - -struct quota_root *quota_root_iter_next(struct quota_root_iter *iter) -{ - struct quota_root *const *roots; - unsigned int count; - - roots = array_get(&iter->qstorage->roots, &count); - i_assert(iter->idx <= count); - - if (iter->idx >= count) - return NULL; - - return roots[iter->idx++]; -} - -void quota_root_iter_deinit(struct quota_root_iter *iter) -{ - i_free(iter); -}

2 3

[Dovecot] Deliver, sendmail, virtual users, and catch-all addresses
by Eric Loyd 17 Jan '08

17 Jan '08

I have spent a long time looking into this and can't find the answer I'm looking for, so I'm asking here. I'm cutting the example down to just the bare minimums. This is NOT my actual configuration, but is sufficient for asking this question: How can I use dovecot's deliver LDA to deliver to virtual users, but then have unknown virtual users fall back to catch-all addresses defined by sendmail? Background: * sendmail is the MTA. I do not wish to change this. * procmail is the default MDA. * deliver is set up per the dovecot documentation as another MDA. * mailertable is configured to send mydomain.com to dovecot:dovecot * SQL backend for user/passwords I have it working the way I want with the exception of catch-alls. I thought that it might work by adding @mydomain.com to virtusertable, but sendmail sees the "user not found" return from deliver as the MDA and bounces the mail to postmaster. So what I'm trying to accomplish is this: Mail from user(a)external.com to me(a)mydomain.com Deliver to me Mail from user(a)external.com to sales(a)mydomain.com (virtual) Deliver via deliver to sales(a)mydomain.com Mail from user(a)external.com to foobar(a)mydomain.com (invalid) Deliver to catchall - spam(a)otherdomain.com Can someone point me in the right direction. RTFM is fine if you'll tell me what FM to R, because I've run out of ideas... Thanks! -- Eric Loyd

2 1

[Dovecot] Indexes
by Daniel L. Miller 13 Jan '08

13 Jan '08

When something "bad" happens to the indexes, my e-mail client (Thunderbird) reports an "unable to succeed" error on opening a mailbox. Leaving that mailbox and coming back works fine. Is this expected behaviour? -- Daniel

3 6

[Dovecot] Automatically subscribe to folder?
by Christian Flothmann 11 Jan '08

11 Jan '08

Hello everybody, i am filtering e-mails using the cmusieve-plugin. A global script filters all mails tagged by SpamAssasin and moves them into the folder Spam: if header :is "X-Spam-Flag" "YES" { fileinto "Spam"; } My problem is that users who did not had a folder Spam until the first time of filtering a spam mail are not automatically subscribed to this new created folder. They have to do this step manually in their mail client. Is it possible to automate this script and subscribe the account to this folder when it is created by the sieve script? Regards, Christian

7 14

[Dovecot] Too many open files
by Andrew Hutchings 10 Jan '08

10 Jan '08

Hi Guys, Just got this on one of my test72 dovecot servers: - Jun 10 12:40:28 svr21 dovecot: auth(default): net_connect_unix(/var/run/dovecot/auth-worker.18818) failed: Too many open files Jun 10 12:40:28 svr21 dovecot: pop3-login: Disconnected: method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx Jun 10 12:40:28 svr21 dovecot: pop3-login: Disconnected: method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx Jun 10 12:40:28 svr21 dovecot: child 18818 (auth) returned error 89 Jun 10 12:40:29 svr21 dovecot: Auth process died too early - shutting down Jun 10 12:40:29 svr21 dovecot: auth(default): mysql: Connected to localhost (maildb) Jun 10 12:40:29 svr21 dovecot: auth(default): net_connect_unix(/var/run/dovecot/auth-worker.13784) failed: No such file or directory Jun 10 12:40:29 svr21 dovecot: child 13784 (auth) returned error 89 - Is dovecot leaving files open? Or is it a large mailbox? Regards Andrew -- Andrew Hutchings (A-Wing) Linux Guru - Netserve Consultants Ltd. - http://www.domaincity.co.uk/ Admin - North Wales Linux User Group - http://www.nwlug.org.uk/ BOFH excuse 122: because Bill Gates is a Jehovah's witness and so nothing can work on St. Swithin's day.

5 5

[Dovecot] maildir_uidlist_create assertion failure
by Andrew Garner 10 Jan '08

10 Jan '08

I've been getting the following error fairly often, which tends to result in a corrupted dovecot-uidlist. dovecot: IMAP(example(a)example.com): file maildir-uidlist.c: line 1009 (maildir_uidlist_recreate): assertion failed: (file_size == (uoff_t)st.st_size) dovecot: IMAP(example(a)example.com): Raw backtrace: imap [0x80cb740] -> imap [0x80cb64a] -> imap [0x8070e58] -> imap(maildir _uidlist_sync_deinit+0xe8) [0x8072d18] -> imap [0x806e9e2] -> imap(maildir_storage_sync_init+0x66) [0x806edf6] -> imap(imap_sync_init+0x4a) [0x80657da] -> ima p(cmd_sync+0x9d) [0x806590d] -> imap(cmd_store+0x33d) [0x805e1ed] -> imap(cmd_uid+0x57) [0x805e737] -> imap [0x805f009] -> imap [0x805f0ae] -> imap(client_inp ut+0x65) [0x805f6c5] -> imap(io_loop_handler_run+0x162) [0x80d3022] -> imap(io_loop_run+0x28) [0x80d2368] -> imap(main+0x4ab) [0x806723b] -> /lib/tls/i686/cmo v/libc.so.6(__libc_start_main+0xc8) [0xb7dfdea8] -> imap [0x8059551] dovecot: child 17997 (imap) killed with signal 6 dovecot: IMAP(example(a)example.com): rename(/home/vmail/domains/o/example.com/U/example/Maildir/dovecot-uidlist.…, /home/vmail/domains/o/example.com/U/example/Maildir/dovecot-uidlist) failed: No such file or directory dovecot: IMAP(example(a)example.com): unlink(/home/vmail/domains/o/example.com/U/example/Maildir/dovecot-uidlist.… failed: No such file or directory dovecot: IMAP(example(a)example.com): rename(/home/vmail/domains/o/example.com/U/example/Maildir/dovecot-uidlist.…, /home/vmail/domains/o/example.com/U/example/Maildir/dovecot-uidlist) failed: No such file or directory dovecot: IMAP(example(a)example.com): unlink(/home/vmail/domains/o/example.com/U/example/Maildir/dovecot-uidlist.… failed: No such file or directory Followed sometime later by: dovecot: IMAP(example(a)example.com): Invalid data in file /home/vmail/domains/o/example.com/U/example/Maildir/dovecot-uidlist Example of some corruption: ~/Maildir/dovecot-uidlist ... 7879 W193997 1195126451.4452.server023,S=191173:2,aopr :1195126451.4452.server023,S=191173:2,aopr ... 9628 W665454 1197354124.5254.server019,S=651976:2,pr <A0>% :1197354124.5254.server019,S=651976:2,pr # dovecot -n # 1.1.beta11: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ protocols: imap imaps pop3 pop3s login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login verbose_proctitle: yes mail_uid: 89 mail_gid: 89 mail_location: maildir:~/Maildir mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_executable(default): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): fts fts_squat mail_plugins(imap): fts fts_squat mail_plugins(pop3): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap/ mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap/ mail_plugin_dir(pop3): /usr/lib/dovecot/modules/ imap_client_workarounds(default): delay-newmail imap_client_workarounds(imap): delay-newmail imap_client_workarounds(pop3): pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %f pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: login plain cram-md5 digest-md5 username_format: %Lu verbose: yes debug: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth user: postfix group: postfix master: path: /var/run/dovecot/auth-master user: vmail group: vmail plugin: quota: maildir quota_rule: *:storage=102400:messages=30000 fts: squat

5 9

[Dovecot] What's the difference between mbox and maildir
by Andrew Falanga 08 Jan '08

08 Jan '08

I'm a little confused about some of these terms. What is: .imap/.imap/ (other than directories) mbox INBOX (I think I have this one) maildir ??????? I'm not sure how all of these things work together. Thanks, Andy

2 2

[Dovecot] What would you tell the CIO in an "ABCs of Email" overview?
by Esther Schindler 07 Jan '08

07 Jan '08

One of the key points that came up when I researched and wrote the "Five Things CIOs Should Know about Fighting Spam" article (http:// www.cio.com/article/28830) was that they should know the basics of how email works. Otherwise, said plenty of techies, the CIO won't have the first idea of what the email admin is complaining about. So I'm going to do my part. I'm going to write an "ABCs of Email" article (to accompany the many other ABCs articles we have on CIO.com, at http://www.cio.com/article/40242 ). I'd like your input on the topics that should be included, keeping in mind the fact that the target reader is a CIO, IT manager, or someone who wants to understand the basics, *not* actively get involved in email management. You don't need to write an essay for me or inundate me with links (though hey, if you want to make my life easier I shall not complain). What I'm looking for, primarily, are the categories of information that I should cover. In other words, if your CIO had an email ephiphany and asked you to give a half-hour presentation, what would you include? This won't be an "Expert says..." article nor will it be "Geek on the street says...." I intend to compile and research the "least you need to know" for the not-necessarily-techie bosses out there. And hopefully the end result will be that you have one less dumb question to deal with in your life. So: any suggestions? (You can reply privately if you prefer.) Esther Schindler senior online editor, CIO.com her blog: http://advice.cio.com/taxonomy/term/34

2 3

[Dovecot] v1.1.beta13 released
by Timo Sirainen 06 Jan '08

06 Jan '08

http://dovecot.org/releases/1.1/beta/dovecot-1.1.beta13.tar.gz http://dovecot.org/releases/1.1/beta/dovecot-1.1.beta13.tar.gz.sig Changes since beta12: - Half of LDAP code was rewritten to fix reconnect/queuing problems. I did test that the new code works with and without auth binds, but be careful. - \Recent flags should finally work correctly with maildir. - last_used field wasn't updated for cache fields that were permanently cached. This caused them to be dropped after a month. TODO for RC1: - When pipelining commands, do only one sync after all of them. Besides improving performance, this fixes some rare problems when multiple commands are running simultaneously. - SEARCH/SORT: Replace invalid input with unicode "replacement character" instead of just skipping it. - Squat: Handle expunges. NFS support. - Fix Lucene support.

4 7

[Dovecot] FETCH for mailbox INBOX UID 39641 got too little data: 4823 vs 4825
by Mikhail Teterin 04 Jan '08

04 Jan '08

Whenever I try to access an e-mail in the middle (rather than at the end) of the mailbox, my mail-client (KMail) gets an error a message like this is logged in the maillog: IMAP(mi): FETCH for mailbox INBOX UID 39641 got too little data: 4823 vs 4825 IMAP(mi): Disconnected: Disconnected The actual numbers vary, but it is always X vs X+2 The problem started when I upgraded to 1.0.8. Unfortunately, it is still here with 1.0.9. The mailbox in question is the standard Unix mbox (/var/mail/mi in my case). The server runs FreeBSD/amd64. Please, advise. Thanks! -mi

5 11