November 2012 - dovecot

[Dovecot] Missing Messages on IOS device
by David Morsberger 29 Nov '12

29 Nov '12

Resending because I didn't get any responses from my first post: I just converted my old server running UW/Panda to a Mac Mini Server running Mountain Lion (ML) Server. I'm using the postfix / dovecot setup provided by Apple. My IMAP clients are not showing all my emails. The Mail App on my iPhone and iPad is behaving the worse. I can see the emails that are not showing up in the cur directory. The ML Mail App is stating there are 3854 emails in my INBOX. There are 3948 files in the 'cur' directory ('ls cur | wc -l' returns 3948). Other mailboxes and directories have the same behavior. I'm not sure what to expect here. My method of conversion was to bulk copy (drag and drop) all the messages from one the UW server to the Dovecot server using the Apple Desktop Mail App in Mountain Lion. Any idea why these messages are not showing up on my devices and how I can get them to show up? bash-3.2# /Applications/Server.app/Contents/ServerRoot/usr/sbin/dovecotd --version 2.0.19apple1 bash-3.2# /Applications/Server.app/Contents/ServerRoot/usr/sbin/dovecotd -n # 2.0.19apple1: /Library/Server/Mail/Config/dovecot/dovecot.conf # OS: Darwin 12.2.1 x86_64 hfs aps_topic = com.apple.mail.XServer.fc3bb7c3-c8ef-4fe2-aff3-95ecd927fb11 auth_mechanisms = cram-md5 x-plain-submit plain login apop digest-md5 auth_socket_path = /var/run/dovecot/auth-userdb auth_username_format = %n default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 6 first_valid_uid = 6 mail_access_groups = mail mail_location = maildir:/Library/Server/Mail/Data/mail/%u mail_log_prefix = "%s(pid %p user %u): " mail_plugins = quota zlib fts fts_sk managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_size = 200 M passdb { args = /Library/Server/Mail/Config/dovecot/submit.passdb driver = passwd-file pass = yes submit = yes } passdb { driver = od } plugin { fts = sk quota = maildir:User quota quota_warning = storage=100%% quota-exceeded %u sieve = /Library/Server/Mail/Data/rules/%u/dovecot.sieve sieve_dir = /Library/Server/Mail/Data/rules/%u } postmaster_address = postmaster(a)mini.mmpcrofton.com protocols = imap pop3 lmtp sieve quota_full_tempfail = yes service auth { extra_groups = _keytabusers idle_kill = 15 mins unix_listener auth-userdb { user = _dovecot } } service dns_client { unix_listener dns-client { mode = 0600 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 0 } service imap { client_limit = 5 process_limit = 200 service_count = 0 } service lmtp { unix_listener lmtp { mode = 0600 } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { client_limit = 5 process_limit = 200 service_count = 0 } service quota-exceeded { executable = script /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/quota-exceeded.sh unix_listener quota-exceeded { group = mail mode = 0660 user = _dovecot } user = _dovecot } service quota-warning { executable = script /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/quota-warning.sh unix_listener quota-warning { group = mail mode = 0660 user = _dovecot } user = _dovecot } ssl_ca = </etc/certificates/*.chain.pem ssl_cert = </etc/certificates/*.cert.pem ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL:!ADH:!eNULL ssl_key = </etc/certificates/*.key.pem ssl_key_path = /etc/certificates/*.key.pem syslog_facility = local6 userdb { args = partition=/Library/Server/Mail/Config/dovecot/partition_map.conf enforce_quotas=no driver = od } verbose_proctitle = yes protocol lmtp { mail_plugins = quota zlib fts fts_sk sieve push_notify } protocol lda { mail_plugins = quota zlib fts fts_sk sieve push_notify } protocol imap { imap_id_log = * imap_id_send = "name" * "version" * mail_max_userip_connections = 20 mail_plugins = quota zlib fts fts_sk imap_quota imap_zlib imap_fts urlauth } protocol pop3 { mail_max_userip_connections = 6 }

3 3

[Dovecot] Problem with mail_location and INDEX location
by Matt Jones 29 Nov '12

29 Nov '12

HI, I'm pulling my hair out a little trying to get dovecot to save it's index locally rather than in the NFS mount. No matter what I do it seems to save the indices in the Maildir on the NFS. I'm using dovecot 2.0.18 on CentOS 6.3. The relevant config I'm using: mmap_disable = no dotlock_use_excl = no # only needed with NFSv2, NFSv3+ supports O_EXCL and it's faster mail_fsync = always mail_nfs_storage = yes mail_nfs_index = no lock_method = dotlock # Mail location and mailbox format. mail_location = maildir:/%Lh/Maildir/:INDEX=/var/indexes/%Ld/%Lu/ Dovecot runs under the user:group vmail:vmail, and the user has write access to those directories. [root@tay var]# ls -al total 96 drwxr-xr-x. 24 root root 4096 Nov 29 09:36 . dr-xr-xr-x. 23 root root 4096 Nov 28 15:26 .. drwxr-x--- 5 amavis amavis 4096 Nov 29 09:21 amavis drwxr-xr-x. 10 root root 4096 Nov 28 15:14 cache drwxr-xr-x. 2 clamav clamav 4096 Nov 29 03:49 clamav drwxr-xr-x. 3 root root 4096 Nov 28 14:21 db drwxr-xr-x. 3 root root 4096 Nov 28 13:35 empty drwxr-xr-x. 2 root root 4096 Sep 23 2011 games drwx------ 2 vmail vmail 4096 Nov 29 10:42 indexes drwxr-xr-x. 25 root root 4096 Nov 28 15:15 lib drwxr-xr-x. 2 root root 4096 Sep 23 2011 local drwxrwxr-x. 5 root lock 4096 Nov 29 03:49 lock drwxr-xr-x. 5 root root 4096 Nov 29 04:10 log lrwxrwxrwx. 1 root root 10 Nov 28 13:34 mail -> spool/mail drwxr-xr-x. 2 root root 4096 Sep 23 2011 nis drwxr-xr-x. 2 root root 4096 Sep 23 2011 opt drwxr-xr-x. 2 root root 4096 Sep 23 2011 preserve drwxr-xr-x. 20 root root 4096 Nov 29 09:51 run drwxr-xr-x. 9 root root 4096 Nov 28 15:14 spool drwxrwxrwt. 3 root root 4096 Nov 29 09:22 tmp drwxr-x--- 2 amavis amavis 4096 Jun 8 15:39 virusmails drwx------. 5 vmail vmail 4096 Nov 29 09:35 vmail drwx------. 3 root bin 4096 Nov 29 09:22 webmin drwxr-xr-x. 12 root root 4096 Nov 28 15:17 www drwxr-xr-x. 2 root root 4096 Sep 23 2011 yp After restarting dovecot, and logging in via imap, I see the following under the Maildir: [root@tay Maildir]# ls -al /var/vmail/vmail1/orphans.co.uk/m/a/t/matt-2011.07.06.09.41.05/Maildir total 56 drwx------ 9 vmail vmail 4096 Nov 29 10:57 . drwx------ 3 vmail vmail 4096 Nov 29 10:17 .. drwx------ 2 vmail vmail 4096 Nov 29 10:41 cur -rw------- 1 vmail vmail 248 Nov 29 10:41 dovecot.index.log -rw------- 1 vmail vmail 120 Nov 29 10:41 dovecot.mailbox.log -rw------- 1 vmail vmail 51 Nov 29 10:41 dovecot-uidlist -rw------- 1 vmail vmail 8 Nov 29 10:41 dovecot-uidvalidity -r--r--r-- 1 vmail vmail 0 Nov 29 10:41 dovecot-uidvalidity.50b73be3 drwx------ 5 vmail vmail 4096 Nov 29 10:41 .Drafts drwx------ 5 vmail vmail 4096 Nov 29 10:41 .Junk drwx------ 2 vmail vmail 4096 Nov 29 10:41 new drwx------ 5 vmail vmail 4096 Nov 29 10:41 .Sent -rw------- 1 vmail vmail 29 Nov 29 10:41 subscriptions drwx------ 2 vmail vmail 4096 Nov 29 10:41 tmp drwx------ 5 vmail vmail 4096 Nov 29 10:41 .Trash [root@tay Maildir]# ls -al /var/indexes/ total 8 drwx------ 2 vmail vmail 4096 Nov 29 10:42 . drwxr-xr-x. 24 root root 4096 Nov 29 09:36 .. Any ideas? Matt

2 1

[Dovecot] Missing quotes in dovecot-config causes pigeonhole to failed.
by nanashi 29 Nov '12

29 Nov '12

I had to manually quote in dovecot-config LIBDOVECOT_LOGIN='-ldovecot-login -lssl -lcrypto ' by default was LIBDOVECOT_LOGIN=-ldovecot-login -lssl -lcrypto That cause pigeonhole to failed. Regards nanashi

2 1

[Dovecot] Shutting down my Dovecot server?
by Bjørn T Johansen 29 Nov '12

29 Nov '12

I am planning to shut down my own Dovecot server and start using the other Dovecot server I am administering together with a friend.. And I was thinking that I don't want to copy the emails I have on my server so I want to start from scratch on the other server.. But I would like to be able to look at the old emails if I need to, so was wondering if there is a way to do this without using an imap server? Can I convert my mails stored in the maildir format to mbox or is there a better way to solve my problem? Regards, BTJ -- ----------------------------------------------------------------------------------------------- Bjørn T Johansen btj(a)havleik.no ----------------------------------------------------------------------------------------------- Someone wrote: "I understand that if you play a Windows CD backwards you hear strange Satanic messages" To which someone replied: "It's even worse than that; play it forwards and it installs Windows" ----------------------------------------------------------------------------------------------- -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

5 10

Re: [Dovecot] Umask of Homedir
by sobotka karel 29 Nov '12

29 Nov '12

Hi, I know it's a while since you've had this issue, but still do you remember by any chance how you solved this? I'm having the same problems, when all the directories are created with 700 and I need a group permissions to be at least 750. Thanks! regards,

1 0

Re: [Dovecot] panic fts_solr for bad attachment
by Daniel L. Miller 28 Nov '12

28 Nov '12

On 11/28/2012 8:49 AM, Daniel L. Miller wrote: > On 11/28/2012 12:55 AM, Timo Sirainen wrote: >> On 28.11.2012, at 10.50, Daniel L. Miller wrote: >> >>> On 11/27/2012 6:45 PM, Timo Sirainen wrote: >>>> On 28.11.2012, at 4.43, Daniel L. Miller wrote: >>>> >>>>>> I did go through the code looking for that a few times already >>>>>> but didn't notice anything. I went through it once more, and >>>>>> finally found the problem. :) >>>>>> http://hg.dovecot.org/dovecot-2.1/rev/6a97faf3e500 >>>>>> >>>>> :( Mine still breaks. Both UTF-8 and Control-Char errors. >>>> Can you grab the network traffic between Dovecot and Solr and find >>>> the problematic stream? >>>> >>> Tell me how and I'll be happy to! >> Maybe the easiest would be to use tcpflow. It outputs different TCP >> streams to different files. From them you can then grep for the error >> and look closer into it. I guess something like wireshark would work >> too, but I've never been able to use its GUI in a useful way. >> > Would I just do "tcpflow -i lo port 8983"? Or something else? > Stream capture sent to you. -- Daniel

1 0

Re: [Dovecot] panic fts_solr for bad attachment
by Daniel L. Miller 28 Nov '12

28 Nov '12

On 11/27/2012 7:28 AM, Daniel L. Miller wrote: > On 11/26/2012 10:08 PM, Timo Sirainen wrote: >> On 27.11.2012, at 7.50, Timo Sirainen wrote: >> >>>> Nov 26, 2012 8:49:29 PM org.apache.solr.common.SolrException log >>>> SEVERE: org.apache.solr.common.SolrException: Illegal character >>>> ((CTRL-CHAR, code 8)) >>>> at [row,col {unknown-source}]: [1011144,197790] >>> Something's wrong. The Solr code was already supposed to catch all >>> of these. >> I was taking a brief scan of the code - and as usual I'm probably wrong - but I believe the protection comes from the xml_encode functions. Could it be that there are some solr writes that don't go through that function - because it is assumed that the data in question doesn't need that processing? Like mailbox names, field names, or uids - that SHOULDN'T have any garbage but maybe something is creeping in? -- Daniel

2 6

[Dovecot] shared mailboxes and indexes
by Angel L. Mateo 28 Nov '12

28 Nov '12

Hello, We are deploying shared mailboxes in our mail system. We are running 2.1.9 and mail backend is maildir. As described at http://wiki.dovecot.org/SharedMailboxes/Shared when shared namespace is configured as namespace shared { separator = / prefix = shared/%%u/ location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u subscriptions = no list = children } each user accessing this folder has its own index, stored in ~/Maildir/shared/%%u/<shared mailbox>, hasn't it? Our mail is store in NFS disks so we are very concerned about indexes optimizations (we had performance problems until we got all of our indexes and nfs tunned). So, is there any way so those indexes could be shared for all users (and they always would be updated). Regarding this... if we'd use dbox instead of maildir, indexes are a really important part of the mailbox and they can't be re-constructed when they are outdated. So, how do shared mailboxes work with dbox backend? Do I have to configure indexes in any particular way? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337

4 10

[Dovecot] problems with quota and clients
by Marco Sacchetto 28 Nov '12

28 Nov '12

Hi, I had configured dovecot in a pretty standard way and had it working nicely enough with maildir quotas (no tests done apart for reaching the quota and looking at mails coming back). Still I wanted to have the quota and the actual quota usage displayed in postfixadmin (3.5) but I think I messed with something. Now I can connect to the mailbox, send mail, receive mail, see the quota usage in postfixadmin but: * thunderbird: can create a subfolder if i do so by hand from the client. But if I try sending a mail i get a [TRYCREATE] mailbox error as it seems that thunderbird can't create the Sent box anymore by its own; * thunderbird: when i delete mail, it goes in the trash folder. I try then to delete it from there. In thunderbird indeed I see no mail anymore but the quota level remain the same. Then I try to browser the mail directory and... yep, all the messages are still there! How come? This is the output of dovecot -n and the relevant sql configuration : # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-33-generic i686 Ubuntu 12.04.1 LTS ext4 auth_mechanisms = plain login auth_realms = yyyyyyyyy.it aaaaaaa.com xxxxx.it auth_verbose_passwords = plain debug_log_path = syslog dict { quotadict = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_uid = 150 info_log_path = /var/log/syslog last_valid_uid = 150 listen = * mail_debug = yes mail_gid = mail mail_location = maildir:/var/vmail/%d/%u mail_plugins = " quota" mail_uid = vmail passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:User quota::proxy::sqlquota quota_rule = *:storage=50M quota_rule2 = Trash:storage=+10M } postmaster_address = xxxxxxxx(a)yyyyyyyyy.it protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = mail mode = 0600 user = vmail } } service dict { unix_listener dict { group = mail mode = 0660 user = vmail } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = quota } protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota } >>grep -v '^ *$#.*$\?$' dovecot-dict-sql.conf.ext connect = host=127.0.0.1 dbname=mail user=mail password=xxxxxxx map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } >>grep -v '^ *$#.*$\?$' dovecot-sql.conf.ext driver = mysql connect = host=localhost dbname=mail user=mail password=xxxxxxx default_pass_scheme = MD5-CRYPT password_query = \ SELECT username as user, password, '/var/vmail/%d/%u' as userdb_home, \ 'maildir:/var/vmail/%d/%u' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid \ FROM mailbox WHERE username = '%u' AND active = '1' user_query = \ SELECT '/var/vmail/%d/%u' as home, 'maildir:/var/vmail/%d/%u' as mail, \ 150 AS uid, 8 AS gid, CONCAT('*:storage=',ROUND(mailbox.quota / 1024)) AS quota_rule \ FROM mailbox WHERE username = '%u' AND active = '1' iterate_query = SELECT username AS user FROM mailbox -- -- Questo messaggio è di carattere riservato ed è indirizzato esclusivamente al destinatario specificato. L'accesso, la divulgazione, la copia o la diffusione sono vietate a chiunque altro ai sensi delle normative vigenti, e possono costituire una violazione penale. Nel caso abbiate ricevuto questo messaggio per errore siete tenuti a cancellarlo immediatamente confermando al mittente, a mezzo e-mail, l'avvenuta cancellazione. (Legge Italiana 196/2003).

1 0

[Dovecot] Vacation messages come from POSTMASTER, not user
by cfowler 28 Nov '12

28 Nov '12

Problem : All vacation autoreplies come from the postmaster address. Expected behavior : vacation notice comes from the user who set the vacation. Platform : Dovecot 2.1.7 on Debian Squeeze ( See below for example reply, sieve script and dovecot -n output) Any help appreciated! ---------------------------------------------------------------------------------------- Sieve script used: if true { vacation :days 2 :subject "Out of Office" "I am currently out of the office"; stop; } ---------------------------------------------------------------------------------------- Return-Path: <> X-Original-To: teststff(a)mydomain.com Delivered-To: teststff(a)mydomain.com Received: from localhost (localhost [127.0.0.1]) by quicksilver.mydomain.com (Postfix) with ESMTP id 12591BE16 for <teststff(a)mydomain.com>; Mon, 26 Nov 2012 16:54:16 +0000 (GMT) X-Virus-Scanned: Debian amavisd-new at mydomain.com X-Spam-Flag: NO X-Spam-Score: -0.501 X-Spam-Level: X-Spam-Status: No, score=-0.501 required=5 tests=[BAYES_05=-0.5, NO_RELAYS=-0.001] autolearn=no Received: from quicksilver.mydomain.com ([127.0.0.1]) by localhost (quicksilver.mydomain.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DNM8HKzR4d0t for <teststff(a)mydomain.com>; Mon, 26 Nov 2012 16:54:06 +0000 (GMT) Received: by quicksilver.mydomain.com (Postfix, from userid 15099) id DA3DABE1B; Mon, 26 Nov 2012 16:54:06 +0000 (GMT) X-Sieve: Pigeonhole Sieve 0.3.0 Message-ID: <dovecot-sieve-1353948846-837692-0(a)quicksilver.mydomain.com> Date: Mon, 26 Nov 2012 16:54:06 +0000 From: Postmaster <root(a)mydomain.com> To: <teststff(a)mydomain.com> Subject: Out of Office In-Reply-To: <ac0fe0e6de81129e7499aba9aa67697a@localhost> References: <ac0fe0e6de81129e7499aba9aa67697a@localhost> Auto-Submitted: auto-replied (vacation) Precedence: bulk MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit I am currently out of the office ---------------------------------------------------------------------------------------- # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 ext3 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes listen = *, [::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/maildirs/%n/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { home = /var/maildirs/%u mail_debug = yes sieve = /var/maildirs/%u/deliver.sieve sieve_dir = /var/maildirs/%n/sieve sieve_global_dir = /etc/sieve/ sieve_global_path = /etc/sieve/deliver.sieve sieve_vacation_dont_check_recipient = yes } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 } user = root } service imap-login { client_limit = 256 process_min_avail = 16 service_count = 0 vsz_limit = 256 M } service managesieve-login { client_limit = 256 process_min_avail = 16 service_count = 0 vsz_limit = 256 M } service pop3-login { client_limit = 256 process_min_avail = 16 service_count = 0 vsz_limit = 256 M } ssl_cert = </var/certs/wildcard.mydomain.ie-including-chain.pem ssl_key = </var/certs/wildcard.mydomain.ie.key userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 20 } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_debug = yes mail_plugins = sieve postmaster_address = root(a)mydomain.ie }

5 7