- dovecot - dovecot.org

Cache unseen count
by Jorge Carling 11 Feb '16

11 Feb '16

Dear all, Lately I am working on PHP mail account management application. It has, among others, Unread messages count column. To get the number of unread messages, I execute "doveadm search" command for each database row. The search time for single account is about 0.03 seconds. Our system has around 750 accounts, so the total loading time is currently around 20 seconds. When setting up Dovecot, I made the user_query, which is also used to store the message quota, and total number of messages stored in particular mailbox. Is there a keyword that can be included in the user_query, or some other way to maintain the unread messages count in the database? Maybe there is a way to execute a script every time a message is received, or becomes read, or unread. Kind regards,JC

2 1

Help migrating a mailbox...
by Willy Gardiol 11 Feb '16

11 Feb '16

Hi! i have a question for you guys, i have been playing with this issue for a while and i am a bit lost. I have dovecot 2.2.16 on a Gentoo linux server, which run a couple of email domains, linked with postfix. I have a huge (>4gb) Outlook PST file which contains a backup coming from a new user, he used to have this backup on his old email system and i need to dump it into his new email account on my server. This is just a "old" backup which needs to be kept for reference, there is no need to preserve any ID/UID/GUID whatever. So, i found the "readpst" util which is capable ot reading the PST file and dump it's contents as mbox structure. (see http://bit-vonameln.de/email-migration-outlook-dovecot-imap/) I did it and i got a 4.4gb folder with some sub-folders, attachments, and more files inside. Then i spent most of a few days trying to have dovecot digest this and import it on top of the user new email address. With no luck. I have being trying with doveadm sync but i cannot find how i need to do it. Note that: 1) there is no local user, only the mail user (user(a)hisdomain.org) 2) i use maildir, the PST has been converted to mbox 3) dovecot is compiled with maildir and mbox support I tried, for example: server # export mail_location=mbox:/path/to/mbox/converted/emails server # doveadm sync -u user(a)hisdomain.org -d doveadm(user(a)hisdomain.org): Fatal: User has no mail_replica in userdb Then i tried: server # export mail_location=/home/vmail/hisdomain.org/user/maildir server # doveadm sync -u user(a)hisdomain.org mbox:/path/to/mbox/converted/emails dsync(user(a)hisdomain.org): Error: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) Also tried: server # export mail_location=/home/vmail/hisdomain.org/user/maildir server # doveadm sync -u user(a)hisdomain.org /path/to/mbox/converted/emails doveadm(user(a)hisdomain.org): Fatal: execvp(/home/vmail/hisdomain.org/user/maildir) failed: Permission denied dsync-local(user(a)hisdomain.org): Error: read(remote) failed: EOF (version not received) My dovecot is configured to use "maildir", the most relevant part of my dovecot.conf is: auth_debug = no login_greeting = Dovecot ready. mail_debug = no auth_debug_passwords = no auth_default_realm = mydomain.org auth_mechanisms = plain login auth_realms = <list of my domains here> auth_verbose = no base_dir = /var/run/dovecot/ disable_plaintext_auth = no mail_uid=999 mail_gid=999 listen = * mail_location = maildir:/home/vmail/%d/%n/maildir protocols = imap sieve lmtp (i can add omitted lines if needed) either i am not undesrtanding anything, which is totally possible, or i am missing something big. If anybody on this list has any insight or suggestion, pelase advise! ps: the destination mailbox is perfectly new and empty, it's already existing and the user can successfully login, send and receive emails. Dovecot works great, and i am operating locally on the server where i uploaded the converted mbox folders. -- Willy Gardiol willy(a)gardiol.org www.gardiol.org www.trackaway.org -> Track YOUR way the way you want!

1 0

Pigeonhole Sieve fails on date-based filter rules - bug?
by Clemens Ender 10 Feb '16

10 Feb '16

Hi list! I’m running dovecot + pigeonhole on a debian wheezy based VPS. Here is the output of „dovecot -n“ # 2.2.18: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.9 disable_plaintext_auth = no imap_idle_notify_interval = 29 mins mail_plugins = acl quota fts fts_lucene managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace { location = maildir:%%h/maildir:INDEX=~/maildir/shared/%%u prefix = shared/%%u/ separator = / type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = username_format=%u /home/dovecot/auth.d/%d/users driver = passwd-file } plugin { acl_shared_dict = file:/home/dovecot/vmail/%d/shared-mailboxes.db fts = lucene fts_autoindex = yes fts_decoder = decode2text fts_lucene = whitespace_chars=@ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service decode2text { executable = script /usr/local/libexec/dovecot/decode2text.sh unix_listener decode2text { mode = 0666 } user = dovecot } service lmtp { executable = lmtp -L unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/ssl/certs/dovecot.crt ssl_key = </etc/ssl/private/dovecot.key userdb { args = username_format=%u /home/dovecot/auth.d/%d/users default_fields = uid=vmail gid=vmail home=/home/dovecot/vmail/%d/%n userdb_mail=maildir:~/maildir quota_rule=*:storage=2G driver = passwd-file } protocol lmtp { mail_plugins = acl quota fts fts_lucene sieve postmaster_address = postmaster(a)en-der.at } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log mail_plugins = acl quota fts fts_lucene sieve } I’m trying to filter emails based on the following rule: if allof (date :value "lt" "Date" "date" "2015-01-01") { fileinto "Archiv/2014"; } Some mails which should match are not filtered. Please look at what sieve-filter -e -W -v ... puts out: >> Filtering message: ID: <0.0.6.888.1CFAFF845D5E634.77992(a)mta144.avaaz.org> Date: Size: 17781 bytes Subject: =?utf-8?q?Klimafreunde_-_der_Plan_ste... Performed actions: * store message in folder: Archiv/2014 Implicit keep: (none) >> Filtering message: ID: <1527093595.86163601420098718229.JavaMail.email(a)email.apple.com> Date: Thu, 1 Jan 2015 07:51:58 +0000 (GMT) Size: 19573 bytes Subject: =?ISO-8859-1?Q?Ihre_Rechnung,_Nr.=A01... Performed actions: (none) Implicit keep: * store message in folder: INBOX ... So it appears that the „Date“ header is not properly extracted for the affected emails. Here are the headers of an particular email (the one without „Date:“ listed above): Return-Path: <bounces-43375-33947798(a)bounces.avaaz.org> Delivered-To: <clemens(a)en-der.at> Received: from vps1.en-der.at by vps1 (Dovecot) with LMTP id Vrb9JNSl31M5aQAAwqatwA for <clemens(a)en-der.at>; Mon, 04 Aug 2014 15:25:08 +0000 Received: from mta144.avaaz.org (mta144.avaaz.org [69.60.9.144]) by vps1.en-der.at (Postfix) with ESMTP id 61715ACDAA for <clemens(a)en-der.at>; Mon, 4 Aug 2014 15:25:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=av_key; d=avaaz.org; h=Content-Type:MIME-Version:Subject:From:To:List-Unsubscribe:Message-ID; i=avaaz(a)avaaz.org; bh=aNrRG1oOw57rU4AT/0miGg8aWAI=; b=erOsRngaWCaW7R0sK8COa4k97QpCISFa6/0ZKfZJHAEHKJu3FbUTExJb/L6OQez/V26UgwpbKJqU Py0l8dDpu68tUlhR42AUnZV+M+lpqI6LdoiTt/QMC0TklQnqdKZt0E4CG+Kz7plluYwhXlZHVSXc +apdPqllFZeYAcsaN5k= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=av_key; d=avaaz.org; b=YdIs06lkIN6pxGKoLORIDjbBCLU/5vwP1QCVocIa0qB4EgT1uKwf+bDlpbQVWmSsjPzekeZmWbpZ um+paKzqq7UGtJMVb1eGjod/bkuOGeXXdeZG0tGJz8wfLdNog6DbfChRBvMciS/Wsz+xrK3yVbX6 bQuKbeD/gurWPhEDfFc=; Received: by mta144.avaaz.org id hruita1u62ku for <clemens(a)en-der.at>; Mon, 4 Aug 2014 11:25:07 -0400 (envelope-from <bounces-43375-33947798(a)bounces.avaaz.org>) Content-Type: multipart/alternative; boundary="===============1522543492113580631==" MIME-Version: 1.0 Subject: =?utf-8?q?Klimafreunde_-_der_Plan_steht!?= From: Ricken Patel - Avaaz.org <avaaz(a)avaaz.org> To: "clemens(a)en-der.at" <clemens(a)en-der.at> List-Unsubscribe: <https://secure.avaaz.org/act/?r=unsub&cl=5597832458&email=clemens@en-der.at…> X-Campaign-ID: avaazCLIMATEeventhostsrecruitment201407301851073204 X-Version-ID: 43375 Message-ID: <0.0.6.888.1CFAFF845D5E634.77992(a)mta144.avaaz.org> --===============1522543492113580631== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Liebe alle, tausende von uns haben ... Any idea what is wrong here? Best regards, Clemens Ender

1 0

Re: Question on setting up dovecot
by Joseph Tam 10 Feb '16

10 Feb '16

Jeffery Small writes: > I'm new to dovecot and have a configuration question. I'm attempting to > have a Windows 7 system on my secure LAN connect to the Ubuntu server > using POP3 on channel 110 and retrieve mail from the /var/mail directory. > The user is defined on the Ubuntu machine and when running the Windows > Internet E-Mail Settings tool, the Test Account Settings Logs onto the > server successfully and sends a test message. Sending message is a distinct service from retrieving Email. > However, when I perform a > Send/Receive operation from the user's Outlook, all steps complete but no > mail is pulled down from the server. > > I have tried setting: mail_location = /var/mail/%u > > and restarted dovecot, but nothing changes. How can I debug this? I'm > unsure where to look for clues as to what's happening under the hood. Any > pointers would be appreciated. Log files would be a good start. Maybe start with something like log_path = /path/to/logfile auth_verbose = yes auth_debug = yes mail_debug = yes verbose_ssl = yes Joseph Tam <jtam.home(a)gmail.com>

1 0

user password in mail-filter plugin
by Thomas Möhle 10 Feb '16

10 Feb '16

Hello, I would like to use the users password within a mail-filter script. Dovecots %w variable is only available within dovecot-auth, is there any way to access the password for a mail-filter? Thanks, Thomas

4 11

Sieve question
by ＠lbutlr 10 Feb '16

10 Feb '16

Is it possible to use redirect in a sieve to redirect to multiple addresses? if header :contains “from” “test(a)example.xn--com-9o0a { redirect “user1@example.com,user2@example.com”; } I thought this would work, But I am seeing a “redirect address is invalid: not a single addres [sic] (found ‘,’). -- I'll have what the gentleman on the floor is having.

3 3

Question on setting up dovecot
by Jeffery Small 10 Feb '16

10 Feb '16

Server OS: Ubuntu 15.10 Sendmail: 8.14.9-4 # doveconf -n ----------------------------------------------------------- # 2.2.18: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 4.2.0-27-generic x86_64 Ubuntu 15.10 disable_plaintext_auth = no mail_location = mbox:~/lib/Mail:INBOX=/var/mail/%u passdb { driver = pam } protocols = imap pop3 ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { driver = passwd } userdb { driver = passwd } ----------------------------------------------------------- I'm new to dovecot and have a configuration question. I'm attempting to have a Windows 7 system on my secure LAN connect to the Ubuntu server using POP3 on channel 110 and retrieve mail from the /var/mail directory. The user is defined on the Ubuntu machine and when running the Windows Internet E-Mail Settings tool, the Test Account Settings Logs onto the server successfully and sends a test message. However, when I perform a Send/Receive operation from the user's Outlook, all steps complete but no mail is pulled down from the server. I have tried setting: mail_location = /var/mail/%u and restarted dovecot, but nothing changes. How can I debug this? I'm unsure where to look for clues as to what's happening under the hood. Any pointers would be appreciated. Regards, -- Jeff Small

1 0

Per-user special folder?
by Richard Platel 10 Feb '16

10 Feb '16

Hi It's possible to mark some folders as special use for IMAP in the config like: namespace inbox { mailbox Spam { special_use = \Junk } } Our webmail allows users to use an arbitrary folder for Spam, and we have this settings and we'd like to return it in from our UserDB (which is a custom dict proxy). For testing were able to set a namespace parameter like "separator" by returning: "namespace/inbox/separator" : "=", from UserDB, but can't figure out a way to set mailbox settings, "namespace/inbox/mailbox Spam/special_use" : "\Junk", for example doesn't work. Is there a way to return this setting from a UserDB query?

2 1

Load increase and imap processes in state D
by Alessio Cecchi 09 Feb '16

09 Feb '16

Hi, in recent weeks I have notice a sudden increase in the load average of our Dovecot (2.2.19) IMAP/POP servers (and consequently in the NFS storage). After an investigation I found some dovecot/imap processes in D state and running UID FETCH or UID SORT from Webmail (Roundcube) without that user was actually more connected to webmail. Some minutes after the dovecot/imap processes stop to run FETCH/SORT but in "ps aux" I can still see it. If the user try to login in Webmail again the same process start again to run FETCH/SORT (and it consumes a lot of I/O resources) but the webmail does not load any message and return a "FETCH 1: UID INTERNALDATE BODY.PEEK ..." error. The only way to restore the webmail access is to kill the related dovecot/imap process (or with doveadm kick username). I noticed that these processes in D state had started many hours before. Example, now are 17:30 (important, see the time of the dovecot/imap processes) the user alessio@skye try to login in webmail without success and close the webmail, processes are still running: # doveadm who | grep alessio@skye username # proto (pids) (ips) alessio@skye 3 imap (10440 4624) (192.168.5.215) # ps aux | grep SORT vmail 4624 0.0 0.0 108792 16048 ? D 13:32 0:01 dovecot/imap [alessio@skye 192.168.5.215 UID SORT] vmail 10440 0.0 0.1 108520 17832 ? D 12:32 0:02 dovecot/imap [alessio@skye 192.168.5.215 UID SORT] # doveadm kick alessio@skye kicked connections from the following users: alessio@skye Processes are now died, and user can login in Webmail easily. Can be a "reusing processes" issue since I have set service_count=100 for imap and pop3 services or there may be other problems related to my configuration? Thanks my configuration: # 2.2.19.1 (16dc61d3dab3): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (5d26f470efe6) # OS: Linux 2.6.32-573.12.1.el6.x86_64 x86_64 CentOS release 6.7 (Final) auth_cache_negative_ttl = 5 mins auth_cache_size = 10 M auth_cache_ttl = 20 mins auth_master_user_separator = * auth_worker_max_count = 50 deliver_log_format = msgid=%m, from=%f, subject="%s": %$ dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags imap_idle_notify_interval = 29 mins imap_logout_format = in=%i out=%o session=<%{session}> last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = 10.96.3.156 login_trusted_networks = 192.168.5.0/24 mail_fsync = always mail_location = maildir:~/Maildir mail_plugins = quota acl expire fts fts_solr zlib maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate vnd.dovecot.duplicate mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-deny-sql.conf.ext deny = yes driver = sql } passdb { args = /etc/dovecot/extra/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl antispam_backend = mailtrain antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_spam = --spam antispam_spam = Spam antispam_trash = Trash expire = Trash expire2 = Spam expire_dict = proxy::expire fts = solr fts_solr = url=http://10.0.0.5:8983/solr/ quota = maildir:UserQuota quota2 = dict:Quota Usage::noenforcing:proxy::sqlquota quota_grace = 10M quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/before.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.duplicate -vacation zlib_save = gz zlib_save_level = 6 } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o, session=<%{session}> protocols = imap pop3 sieve sendmail_path = /bin/sendmail service auth { client_limit = 6524 unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { process_limit = 500 unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { process_min_avail = 4 service_count = 0 } service imap-postlogin { executable = script-login /etc/dovecot/imap-postlogin.sh unix_listener imap-postlogin { group = vmail mode = 0660 user = vmail } user = vmail } service imap { executable = imap imap-postlogin process_limit = 5000 service_count = 100 vsz_limit = 384 M } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { process_min_avail = 4 service_count = 0 } service pop3-postlogin { executable = script-login /etc/dovecot/pop3-postlogin.sh unix_listener pop3-postlogin { group = vmail mode = 0660 user = vmail } user = vmail } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 service_count = 100 } service quota-warning { executable = script /etc/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } service stats { fifo_listener stats-mail { group = vmail mode = 0660 user = vmail } } ssl_cert = </etc/dovecot/wildcard.pem ssl_key = </etc/dovecot/wildcard.key ssl_protocols = !SSLv2 !SSLv3 submission_host = 127.0.0.1 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lda { mail_location = maildir:~/Maildir:INDEX=MEMORY mail_plugins = quota acl expire fts fts_solr zlib sieve } protocol imap { mail_max_userip_connections = 25 mail_plugins = quota acl expire fts fts_solr zlib imap_quota imap_acl imap_zlib } protocol sieve { mail_max_userip_connections = 2 } -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

1 0

Bug: The "listescape" plugin does not work with the option "mail_shared_explicit_inbox = no". [with patch]
by Rudolf Körner 09 Feb '16

09 Feb '16

Hello, there seems to be a problem with the listescape plugin and shared INBOX: === Preconditions: - Dovecot 2.2.21 (or 2.2.13) - IMAP - Separator / - Namespace-prefixes "INBOX/" and "shared/%%u/" - Maildir(++) - option mail_plugins = acl listescape - option mail_shared_explicit_inbox = no - User "owner" shares the INBOX - ACLs set for another user "reader", so "reader" is able to access the shared INBOX of "owner" === Steps to reproduce by using IMAP commands at a telnet session: - a1 login reader password - a2 select shared/owner === Result: Selecting the shared INBOX fails with a response like: a2 NO Mailbox doesn't exist: shared/owner === Expected: The "select" command works. === Analysis: The function "mailbox_list_default_get_storage_name" (lib-storage/mailbox-list.c) does not work as expected when using the mail_plugin "listescape". It compares the mailbox name to the namespace prefix. Since the mailbox name is escaped a few lines before, the comparison yields "strings differ", which makes accessing the shared INBOX fail. For example: for a string vname (mailbox name) "shared/owner", it compares "shared.owner" (escaped string) to the list->ns->prefix which is "shared/owner". Asuming that the stored prefix should not be escaped, there should be a comparison of the unescaped name to the unescaped prefix (or the escaped name to the escaped prefix?). === Patch: Attached is a small patch which shows a possible solution. When applied, the problem is solved, i.e. the shared INBOX is accessible then. Important note: this patch is experimental, it works for the described special case but might not work with a different configuration or might even break things. Best Regards, Rudolf Körner

1 0