- dovecot - dovecot.org

[Dovecot] too many open files and v1.2
by Michael Moritz 20 Oct '11

20 Oct '11

Hi I've found an older thread http://www.dovecot.org/list/dovecot/2010-March/047886.html which mentions that this is a bug in version 1. I'm on Debian squeeze (1:1.2.15-7) and got errors like these recently Oct 5 11:29:29 mail dovecot: dovecot: pipe() failed: Too many open files Oct 5 11:29:32 mail dovecot: dovecot: pipe() failed: Too many open files Oct 5 11:29:32 mail dovecot: dovecot: Temporary failure in creating login processes, slowing down for now As a work around I've increased login_max_processes_count from 256 to 512 and it seems to have helped (as I suspected stale opened files). My colleague thinks it's hitting the kernel open file limit ulimit -n (currently at 1024) but I don't see how that could happen if the number of process is nowhere near that. Any ideas? Thanks, Michael

2 2

[Dovecot] Is '*:bytes=" quota specification the quota 'Value', or 'Limit'?
by Rich 19 Oct '11

19 Oct '11

Docs for quota configuration http://wiki2.dovecot.org/Quota/Configuration#Limit_configuration state that the 'bytes=' specification defines the quota **LIMIT** "bytes: Quota limit in bytes, 0 means unlimited" as used in plugin { ... quota_rule = *:bytes=10000000 ... BUT, documentation for quota dict http://wiki2.dovecot.org/Quota/Dict states "MySQL uses the following queries to update the quota" and provides an SQL example of Dovecot decrementing the 'bytes' value, "UPDATE table SET bytes=bytes-14433,messages=messages-2 ..." I'm working on a current problem (http://dovecot.org/list/dovecot/2011-October/061538.html) with 'doveadm' reporting mixing up quote Value & Limit. Question: Is 'bytes' supposed to contain the Value, or the Limit? Rich

1 0

[Dovecot] 'doveadm quota get' dictionary SQL query ignores specified '@domain' part of username. bad config or bug?
by Rich 19 Oct '11

19 Oct '11

Hi, I'm setting up SQL quota usage in Dovecot2. I've created a MAILBOX parameter table, 'PARAMS', that contains unique <user>@<domain> pairs, with quota data. mysql> select * from PARAMS; +----+------------------------+------------+-------------+------------+ | ai | MAILBOX_user_domain | quota | quota_bytes | quota_msgs | +----+------------------------+------------+-------------+------------+ | 1 | myuser(a)domain1.com | 1073741824 | 0 | 0 | | 2 | myuser(a)domain2.com | 1000000000 | 0 | 0 | +----+------------------------+------------+-------------+------------+ 2 rows in set (0.00 sec) I've specified dict usage for quota, /etc/dovecot/dovecot.conf !include conf.d/*.conf protocols = imap lmtp dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } created the maps, /etc/dovecot/dovecot-dict-sql.conf.ext connect = host=/var/run/mysql/mysql.sock dbname=my_db user=my_user password=my_pass map { pattern = priv/quota/storage table = PARAMS username_field = MAILBOX_user_domain value_field = quota_bytes } map { pattern = priv/quota/messages table = PARAMS username_field = MAILBOX_user_domain value_field = quota_msgs } assigned the global values and backend, /etc/dovecot/conf.d/90-quota.conf plugin { quota = dict:User Quota::proxy::quota quota_rule = *:storage=1GB:messages=10000 quota_rule2 = Trash:storage=+10%% } and specified the user iteration query /etc/dovecot/sql/virtmail-userdb-sql.cf driver = mysql connect = host=/var/run/mysql/mysql.sock dbname=my_db user=my_user password=my_pass user_query = CALL UserDBQuery('%n','%d'); iterate_query = SELECT `MAILBOX_user_domain` AS user FROM `PARAMS`; Verifying operation with with `doveadm`, I get a result from a !error query, doveadm quota get -u myuser(a)domain1.com Quota name Type Value Limit % User quota STORAGE 0 1048576 0 User quota MESSAGE 0 10000 0 doveadm quota get -u myuser(a)domain2.com Quota name Type Value Limit % User quota STORAGE 0 1048576 0 User quota MESSAGE 0 10000 0 But notice that there's no specificity by *domain*. Despite being provided the full <user>@<domain>, matched to the map, username_field = MAILBOX_user_domain the Dovecot dict query seems to ignore the <domain> part, and simply matches on the first of the tow identical <user> parts. Specifying "-A" doesn't help, doveadm quota get -A Username Quota name Type Value Limit % myuser(a)domain1.com User quota STORAGE 0 1048576 0 myuser(a)domain1.com User quota MESSAGE 0 10000 0 myuser(a)domain2.com User quota STORAGE 0 1048576 0 myuser(a)domain2.com User quota MESSAGE 0 10000 0 What needs to be done to get domain-specificity in the 'doveadm quota get' query? Is my configuration off, or incomplete, or is this a bug? Thanks, Rich

1 1

Re: [Dovecot] "doveadm log reopen" don't reopen separate lmtp log
by Henrik Larsson 18 Oct '11

18 Oct '11

I still see the issue below. Is there anyone running separate LMTP logging that could check if they experience the same issue? Best regards Henrik Larsson -------- Original Message -------- Subject: "doveadm log reopen" don't reopen separate lmtp log Date: Sun, 25 Sep 2011 00:23:49 +0200 From: Henrik Larsson <dovecot-user(a)spambox.dk> To: dovecot(a)dovecot.org Dear all I have setup separate pop3/imap log "/var/log/dovecot" and lmtp delivery log "/var/log/dovecot-deliver". After rotating logfiles, i run "doveadm log reopen". I see that my pop3/imap log "/var/log/dovecot" is used straight away, but my lmpt log "/var/log/dovecot-deliver" isn't used at first. After some time, usually a few minutes, logs are being written to this file anyway. But if I compare this to my maillog, it misses some deliveries just after the log rotation. Is there any problems with this configuration that should be corrected? --cut-- # doveconf -n # 2.0.15: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.2-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 first_valid_uid = 125 hostname = mail.larsson.it listen = * log_path = /var/log/dovecot mail_plugins = fts fts_squat zlib mail_privileged_group = postfix mail_temp_dir = /var/db/dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = squat fts_squat = partial=4 full=10 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { unix_listener /home/mail/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service lmtp { executable = lmtp -L unix_listener /home/mail/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/mail-chained.crt ssl_key = </etc/ssl/mail.key userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { log_path = /var/log/dovecot-deliver mail_plugins = fts fts_squat zlib sieve } protocol lda { mail_plugins = fts fts_squat zlib sieve } protocol imap { mail_plugins = fts fts_squat zlib imap_zlib } --cut-- Best regards Henrik Larsson

2 3

[Dovecot] mail_location in LDAP and variables
by Joseba Torre 18 Oct '11

18 Oct '11

Hi, Is it posible to use variables in mail_location when taken from ldap? How? I've tried mdbox:/buzones2/'%n'/mdbox mdbox:/buzones2/"%n"/mdbox mdbox:/buzones2/%n/mdbox but %n is never replaced by its value. Thanks! -- Joseba Torre. Vicegerencia de TICs, área de Explotación

2 2

[Dovecot] LDA/Postfix with quotas and without LMTP
by Steve Fatula 18 Oct '11

18 Oct '11

Reading the wiki page for dovecot2 and LDA/Postfix, the system users section (all mail delivery goes to system users)... It shows you can simply use postfix mailbox_command. There is this additional comment: "This command doesn't do a userdb lookup. If you want that (e.g. for per-user quota lookups) you need to add -d "$USER" parameter." So, i added that. Once adding it, I find I get an error: dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied Using mailbox_command in Postfix means that dovecot-lda is running as the system user getting the mail. Which means it can't access the socket file. Of course, one can simply use mode 666 on it, but, then people can use it to find out information according to the doc, which I do not want. So, I am not sure why the doc says to add -d for per user quotas, without also mentioning the problem with doing that. I don't see how it can work? Assuming I need per user quotas, and I don't want to use LMTP, is there a good way around this without the security issue? Steve

2 2

[Dovecot] Syntax to specify sdbox: mail_location path?
by mephistopheles＠operamail.com 18 Oct '11

18 Oct '11

I'm implementing sdbox. In conf.d/10-mail.conf, with "maildir:" mail_location = maildir:/my_mailbox_path/%d/%n mail gets stored in /my_mailbox_path/%d/%n/new/ With "sdbox:" mail_location = sdbox:/my_mailbox_path/%d/%n mail gets stored in /my_mailbox_path/%d/%n/mailboxes/INBOX/dbox-Mails. How do I get rid of the '/mailboxes/' and '/dbox-Mails/' path components so sdbox stores in: /my_mailbox_path/%d/%n/INBOX @ wiki2.dovecot.org/MailboxFormat/dbox#Alternate_storage points out that 'ALT=' can change the *lookup* location, and messages can be *moved* to ALT storage using 'doveadm altmove'. Unclear what TAGS are available/used for 'sdbox'.

2 1

[Dovecot] Rare error with concurrent users searching with fts-solr
by Antonio Perez-Aranda 18 Oct '11

18 Oct '11

Hi all, If I start two o more connections with diferent users that make a search text to solr I get this message many times: Error: fts_solr: Query didn't return uid If I repeat the test about 2000 times, with only one user concurrent, with 60 diferent users, without concurrent, I get between 0. If I repeat the test about 2000 times, with 2 users concurrent, I get between 3 or 10 errors. If I repeat the test about 2000 times, with 4 users concurrent, I get between 10-20 errors. After see this error, I set a tcpdump to try to watch the SOLR XML and GET, I can watch that Solr send a xml like this: <response></response> with 200 as http status When the normal response is like this <response> <lst name="responseHeader"> <int name="status">0</int> <int name="QTime">1</int> <lst name="params"> <str name="fl">uid,score</str> <str name="sort">uid asc</str> <str name="q">BLABLABLA</str> <str name="fq">+user:"ANYUSER" +uidv:1208469916 +box:"ANYFOLDER" -ns:[* TO *]</str> <str name="rows">1</str> </lst> </lst> <result name="response" numFound="0" start="0" maxScore="0.0"/> </response> At first, I think that the problems it's in Solr, but after get all GET from tcpdump, I make a python script with urllib2 and send it all to Solr with the same concurrency and I can't get any error or "empty response" Have someone made a intensive concurrency tests over Dovecot + Solr ? -- Antonio Pérez-Aranda Alcaide aperezaranda(a)yaco.es Yaco Sistemas S.L. http://www.yaco.es/ C/ Rioja 5, 41001 Sevilla Teléfono +34 954 50 00 57 Fax +34 954 50 09 29

2 1

[Dovecot] 2.1 Plans
by Steve Fatula 18 Oct '11

18 Oct '11

Back in early 2010, Timo posted about a potential 2.1 feature to in essence reduce the number of imap/pop3 processes using various techniques since the code already supports multiple connections. It even went as far as moving all idle connections. I don't see this on the 2.1 roadmap. Is it still planned perhaps for 2.1? Thousands of imap processes going away is a great thing for me! So, just curious. Reference: http://dovecot.org/list/dovecot/2010-January/046148.html Steve

2 1

[Dovecot] Convert-tool maildir > mbox
by Bertrand Jacquin 18 Oct '11

18 Oct '11

Timo, Is seems that this commit broke convertion from Maildir to Mbox using dovecot 1.2 convert-tool binary : 2009-07-09 Timo Sirainen <tss(a)iki.fi> * src/plugins/convert/convert-storage.c: convert plugin: Fixed changing hierarchy separators in mailbox names when alt_hierarchy_char isn't set. [a9f719f9a509] <HEAD> Since, I get the following : $ convert-tool user1 /var/mail/example.com/user1 Maildir:/var/mail/example.com/user1 mbox:/var/mail/example.com/user1-mbox alt_hierarchy_char=_ Error: Mailbox conversion: Couldn't create mailbox Sent/test: Mailbox doesn't allow inferior mailboxes Error: Internal failure Strace give show me a failure after : access("/var/mail/example.com/user1/.Sent.test/cur", W_OK) = 0 stat("/var/mail/example.com/user1-mbox/Sent/test", 0x7fff843e73b0) = -1 ENOTDIR (Not a directory) write(2, "Error: Mailbox conversion: Couldn"..., 103Error: Mailbox conversion: Couldn't create mailbox Sent/test: Mailbox doesn't allow inferior mailboxes ) = 103 lstat("/var/mail/example.com/user1/.dovecot.convert.lock", {st_mode=S_IFREG|0644, st_size=12, ...}) = 0 unlink("/var/mail/example.com/user1/.dovecot.convert.lock") = 0 Is that somethinh known ? Didn't find anything interesting in archives. Maybe that kind of conversion have evolved since 2.0 ? Thanks, Beber

2 1