- dovecot - dovecot.org

[Dovecot] Expire plugin not executing for sieve plugin?
by Dan Swartzendruber 20 Oct '11

20 Oct '11

Running dovecot 1.2.12 on ubuntu 10.10. Here is the problem: I have mysql backend for expire. I am using the dovecot-postfix package which creates /etc/dovecot/conf.d/01-mail-stack-delivery.conf which looks like this: # Some general options protocols = imap imaps managesieve disable_plaintext_auth = no ssl = yes ssl_cert_file = /etc/ssl/certs/ssl-mail.pem ssl_key_file = /etc/ssl/private/ssl-mail.key ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM mail_location = maildir:~/Maildir auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ # IMAP configuration protocol imap { mail_max_userip_connections = 10 imap_client_workarounds = outlook-idle delay-newmail mail_plugins = expire } # LDA configuration protocol lda { postmaster_address = postmaster mail_plugins = sieve expire quota_full_tempfail = yes deliver_log_format = msgid=%m: %$ rejection_reason = Your message to <%t> was automatically rejected:%n%r } # Plugins configuration plugin { sieve=~/.dovecot.sieve sieve_dir=~/sieve # If the user has no personal active script (i.e. if the file # indicated in sieve= does not exist), use this one: sieve_global_path = /var/lib/dovecot/sieve/default.sieve } The sieve is: require "fileinto"; require "imap4flags"; if header :contains "X-Spam-Flag" "YES" { setflag "\\Seen"; fileinto "SPAM"; stop; } Originally, this did not have the setflag or stop - I added them hoping to fix the problem, but no luck. The problem is this: when spam assassin has flagged a message as spam, the sieve runs and the message is in fact filed into the SPAM folder, but no expire entry is added to the database. If I move such a message from the SPAM folder to the Inbox and then back to SPAM, voila, I *do* get the entry created. Here is the mysql table before and after I did the move/move: mysql> select * from expires; +----------+---------------+--------------+ | username | mailbox | expire_stamp | +----------+---------------+--------------+ | dswartz | Deleted Items | 1319720163 | | dswartz | Sent | 1319120165 | | denise | Trash | 1319139920 | | samantha | Trash | 1319136146 | | denise | Sent | 1319655872 | | danielle | Sent | 1319192230 | | samantha | Sent | 1319268855 | | dswartz | Trash | 1319725884 | | samantha | SPAM | 1321504531 | +----------+---------------+--------------+ 9 rows in set (0.00 sec) mysql> select * from expires; +----------+---------------+--------------+ | username | mailbox | expire_stamp | +----------+---------------+--------------+ | dswartz | Deleted Items | 1319720163 | | dswartz | Sent | 1319120165 | | denise | Trash | 1319139920 | | samantha | Trash | 1319136146 | | denise | Sent | 1319655872 | | danielle | Sent | 1319192230 | | samantha | Sent | 1319268855 | | dswartz | Trash | 1319725884 | | samantha | SPAM | 1321504531 | | dswartz | SPAM | 1321714666 | +----------+---------------+--------------+ 10 rows in set (0.00 sec) It's almost like the fileinto stuff is not running the plugin code at all, or am I totally on drugs? If I can't get a fix for this, I at least now can think of a hack to "fix" it. e.g. every night, run a script that adds a "now" timestamp for everyone's SPAM folder to trick the periodic expire tool code into checking that folder, but obviously I'd rather have a real fix :) If this is just 'how it works', is it better in 2.0?

1 0

[Dovecot] too many open files and v1.2
by Michael Moritz 20 Oct '11

20 Oct '11

Hi I've found an older thread http://www.dovecot.org/list/dovecot/2010-March/047886.html which mentions that this is a bug in version 1. I'm on Debian squeeze (1:1.2.15-7) and got errors like these recently Oct 5 11:29:29 mail dovecot: dovecot: pipe() failed: Too many open files Oct 5 11:29:32 mail dovecot: dovecot: pipe() failed: Too many open files Oct 5 11:29:32 mail dovecot: dovecot: Temporary failure in creating login processes, slowing down for now As a work around I've increased login_max_processes_count from 256 to 512 and it seems to have helped (as I suspected stale opened files). My colleague thinks it's hitting the kernel open file limit ulimit -n (currently at 1024) but I don't see how that could happen if the number of process is nowhere near that. Any ideas? Thanks, Michael

2 2

[Dovecot] Is '*:bytes=" quota specification the quota 'Value', or 'Limit'?
by Rich 19 Oct '11

19 Oct '11

Docs for quota configuration http://wiki2.dovecot.org/Quota/Configuration#Limit_configuration state that the 'bytes=' specification defines the quota **LIMIT** "bytes: Quota limit in bytes, 0 means unlimited" as used in plugin { ... quota_rule = *:bytes=10000000 ... BUT, documentation for quota dict http://wiki2.dovecot.org/Quota/Dict states "MySQL uses the following queries to update the quota" and provides an SQL example of Dovecot decrementing the 'bytes' value, "UPDATE table SET bytes=bytes-14433,messages=messages-2 ..." I'm working on a current problem (http://dovecot.org/list/dovecot/2011-October/061538.html) with 'doveadm' reporting mixing up quote Value & Limit. Question: Is 'bytes' supposed to contain the Value, or the Limit? Rich

1 0

[Dovecot] 'doveadm quota get' dictionary SQL query ignores specified '@domain' part of username. bad config or bug?
by Rich 19 Oct '11

19 Oct '11

Hi, I'm setting up SQL quota usage in Dovecot2. I've created a MAILBOX parameter table, 'PARAMS', that contains unique <user>@<domain> pairs, with quota data. mysql> select * from PARAMS; +----+------------------------+------------+-------------+------------+ | ai | MAILBOX_user_domain | quota | quota_bytes | quota_msgs | +----+------------------------+------------+-------------+------------+ | 1 | myuser(a)domain1.com | 1073741824 | 0 | 0 | | 2 | myuser(a)domain2.com | 1000000000 | 0 | 0 | +----+------------------------+------------+-------------+------------+ 2 rows in set (0.00 sec) I've specified dict usage for quota, /etc/dovecot/dovecot.conf !include conf.d/*.conf protocols = imap lmtp dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } created the maps, /etc/dovecot/dovecot-dict-sql.conf.ext connect = host=/var/run/mysql/mysql.sock dbname=my_db user=my_user password=my_pass map { pattern = priv/quota/storage table = PARAMS username_field = MAILBOX_user_domain value_field = quota_bytes } map { pattern = priv/quota/messages table = PARAMS username_field = MAILBOX_user_domain value_field = quota_msgs } assigned the global values and backend, /etc/dovecot/conf.d/90-quota.conf plugin { quota = dict:User Quota::proxy::quota quota_rule = *:storage=1GB:messages=10000 quota_rule2 = Trash:storage=+10%% } and specified the user iteration query /etc/dovecot/sql/virtmail-userdb-sql.cf driver = mysql connect = host=/var/run/mysql/mysql.sock dbname=my_db user=my_user password=my_pass user_query = CALL UserDBQuery('%n','%d'); iterate_query = SELECT `MAILBOX_user_domain` AS user FROM `PARAMS`; Verifying operation with with `doveadm`, I get a result from a !error query, doveadm quota get -u myuser(a)domain1.com Quota name Type Value Limit % User quota STORAGE 0 1048576 0 User quota MESSAGE 0 10000 0 doveadm quota get -u myuser(a)domain2.com Quota name Type Value Limit % User quota STORAGE 0 1048576 0 User quota MESSAGE 0 10000 0 But notice that there's no specificity by *domain*. Despite being provided the full <user>@<domain>, matched to the map, username_field = MAILBOX_user_domain the Dovecot dict query seems to ignore the <domain> part, and simply matches on the first of the tow identical <user> parts. Specifying "-A" doesn't help, doveadm quota get -A Username Quota name Type Value Limit % myuser(a)domain1.com User quota STORAGE 0 1048576 0 myuser(a)domain1.com User quota MESSAGE 0 10000 0 myuser(a)domain2.com User quota STORAGE 0 1048576 0 myuser(a)domain2.com User quota MESSAGE 0 10000 0 What needs to be done to get domain-specificity in the 'doveadm quota get' query? Is my configuration off, or incomplete, or is this a bug? Thanks, Rich

1 1

Re: [Dovecot] "doveadm log reopen" don't reopen separate lmtp log
by Henrik Larsson 18 Oct '11

18 Oct '11

I still see the issue below. Is there anyone running separate LMTP logging that could check if they experience the same issue? Best regards Henrik Larsson -------- Original Message -------- Subject: "doveadm log reopen" don't reopen separate lmtp log Date: Sun, 25 Sep 2011 00:23:49 +0200 From: Henrik Larsson <dovecot-user(a)spambox.dk> To: dovecot(a)dovecot.org Dear all I have setup separate pop3/imap log "/var/log/dovecot" and lmtp delivery log "/var/log/dovecot-deliver". After rotating logfiles, i run "doveadm log reopen". I see that my pop3/imap log "/var/log/dovecot" is used straight away, but my lmpt log "/var/log/dovecot-deliver" isn't used at first. After some time, usually a few minutes, logs are being written to this file anyway. But if I compare this to my maillog, it misses some deliveries just after the log rotation. Is there any problems with this configuration that should be corrected? --cut-- # doveconf -n # 2.0.15: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.2-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 first_valid_uid = 125 hostname = mail.larsson.it listen = * log_path = /var/log/dovecot mail_plugins = fts fts_squat zlib mail_privileged_group = postfix mail_temp_dir = /var/db/dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = squat fts_squat = partial=4 full=10 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { unix_listener /home/mail/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service lmtp { executable = lmtp -L unix_listener /home/mail/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/mail-chained.crt ssl_key = </etc/ssl/mail.key userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { log_path = /var/log/dovecot-deliver mail_plugins = fts fts_squat zlib sieve } protocol lda { mail_plugins = fts fts_squat zlib sieve } protocol imap { mail_plugins = fts fts_squat zlib imap_zlib } --cut-- Best regards Henrik Larsson

2 3

[Dovecot] mail_location in LDAP and variables
by Joseba Torre 18 Oct '11

18 Oct '11

Hi, Is it posible to use variables in mail_location when taken from ldap? How? I've tried mdbox:/buzones2/'%n'/mdbox mdbox:/buzones2/"%n"/mdbox mdbox:/buzones2/%n/mdbox but %n is never replaced by its value. Thanks! -- Joseba Torre. Vicegerencia de TICs, área de Explotación

2 2

[Dovecot] LDA/Postfix with quotas and without LMTP
by Steve Fatula 18 Oct '11

18 Oct '11

Reading the wiki page for dovecot2 and LDA/Postfix, the system users section (all mail delivery goes to system users)... It shows you can simply use postfix mailbox_command. There is this additional comment: "This command doesn't do a userdb lookup. If you want that (e.g. for per-user quota lookups) you need to add -d "$USER" parameter." So, i added that. Once adding it, I find I get an error: dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied Using mailbox_command in Postfix means that dovecot-lda is running as the system user getting the mail. Which means it can't access the socket file. Of course, one can simply use mode 666 on it, but, then people can use it to find out information according to the doc, which I do not want. So, I am not sure why the doc says to add -d for per user quotas, without also mentioning the problem with doing that. I don't see how it can work? Assuming I need per user quotas, and I don't want to use LMTP, is there a good way around this without the security issue? Steve

2 2

[Dovecot] Syntax to specify sdbox: mail_location path?
by mephistopheles＠operamail.com 18 Oct '11

18 Oct '11

I'm implementing sdbox. In conf.d/10-mail.conf, with "maildir:" mail_location = maildir:/my_mailbox_path/%d/%n mail gets stored in /my_mailbox_path/%d/%n/new/ With "sdbox:" mail_location = sdbox:/my_mailbox_path/%d/%n mail gets stored in /my_mailbox_path/%d/%n/mailboxes/INBOX/dbox-Mails. How do I get rid of the '/mailboxes/' and '/dbox-Mails/' path components so sdbox stores in: /my_mailbox_path/%d/%n/INBOX @ wiki2.dovecot.org/MailboxFormat/dbox#Alternate_storage points out that 'ALT=' can change the *lookup* location, and messages can be *moved* to ALT storage using 'doveadm altmove'. Unclear what TAGS are available/used for 'sdbox'.

2 1

[Dovecot] Rare error with concurrent users searching with fts-solr
by Antonio Perez-Aranda 18 Oct '11

18 Oct '11

Hi all, If I start two o more connections with diferent users that make a search text to solr I get this message many times: Error: fts_solr: Query didn't return uid If I repeat the test about 2000 times, with only one user concurrent, with 60 diferent users, without concurrent, I get between 0. If I repeat the test about 2000 times, with 2 users concurrent, I get between 3 or 10 errors. If I repeat the test about 2000 times, with 4 users concurrent, I get between 10-20 errors. After see this error, I set a tcpdump to try to watch the SOLR XML and GET, I can watch that Solr send a xml like this: <response></response> with 200 as http status When the normal response is like this <response> <lst name="responseHeader"> <int name="status">0</int> <int name="QTime">1</int> <lst name="params"> <str name="fl">uid,score</str> <str name="sort">uid asc</str> <str name="q">BLABLABLA</str> <str name="fq">+user:"ANYUSER" +uidv:1208469916 +box:"ANYFOLDER" -ns:[* TO *]</str> <str name="rows">1</str> </lst> </lst> <result name="response" numFound="0" start="0" maxScore="0.0"/> </response> At first, I think that the problems it's in Solr, but after get all GET from tcpdump, I make a python script with urllib2 and send it all to Solr with the same concurrency and I can't get any error or "empty response" Have someone made a intensive concurrency tests over Dovecot + Solr ? -- Antonio Pérez-Aranda Alcaide aperezaranda(a)yaco.es Yaco Sistemas S.L. http://www.yaco.es/ C/ Rioja 5, 41001 Sevilla Teléfono +34 954 50 00 57 Fax +34 954 50 09 29

2 1

[Dovecot] 2.1 Plans
by Steve Fatula 18 Oct '11

18 Oct '11

Back in early 2010, Timo posted about a potential 2.1 feature to in essence reduce the number of imap/pop3 processes using various techniques since the code already supports multiple connections. It even went as far as moving all idle connections. I don't see this on the 2.1 roadmap. Is it still planned perhaps for 2.1? Thousands of imap processes going away is a great thing for me! So, just curious. Reference: http://dovecot.org/list/dovecot/2010-January/046148.html Steve

2 1