June 2014 - dovecot - dovecot.org

A solution : dsync(someuser) Error: Mailbox GUIDs are not permanent without index files
by James Rogers 12 Jun '14

12 Jun '14

While migrating from mbox to maildir using dsync, I kept running into a brick wall. Some folders would not sync. The message in stderr was: Error message GUID: Mailbox GUIDs are not permanent without index files. This is what solved things for me. I hope this might help someone. Let me know if I'm completely off sides. When working with dsync to migrate from mbox to maildir format mailboxes, I kept running into the error: dsync(someuser): Error: Couldn't get mailbox Drafts GUID: Mailbox GUIDs are not permanent without index files dsync(someuser): Error: Couldn't get mailbox Junk E-mail1 GUID: Mailbox GUIDs are not permanent without index files dsync(soemuser): Error: Couldn't get mailbox TO DO LATER - US GUID: Mailbox GUIDs are not permanent without index files dsync(someuser): Error: Couldn't get mailbox Notes GUID: Mailbox GUIDs are not permanent without index files dsync(someuser): Error: Couldn't get mailbox Orders Pending GUID: Mailbox GUIDs are not permanent without index files dsync(someuser): Error: Couldn't get mailbox Junk E-mail GUID: Mailbox GUIDs are not permanent without index files dsync(someuser): Error: Couldn't get mailbox Server Mgmt+Config GUID: Mailbox GUIDs are not permanent without index files The error messages were helpful, but it would be nice to know what prevented the creation of the indexes in the errors (permissions, settings, disk space, etc.). I finally figured it out semi-randomly when I dumped my doveconf file using doveconf -n, and saw: mbox_min_index_size = 100 k I felt very um... happy? And not too bright. The following made my problems go away, and the sync worked flawlessly. mbox_min_index_size = 0 I guess it's best to set this using dsync -o'mbox_min_index_size=0' Mail now converts quickly and completely even for 0 length mailboxes (i.e. empty folders are created). Also, I would like to note that mb2md.pl should not be used to convert mbox to maildir. The regex used to split mboxes will fragment messages. mb2md.pl uses: if ( /^From / And in the very least, the following would be less harmful: if ( /^From\s+(\S+.*(a)\S+.*|MAILER.DAEMON)\s+.*$/ I love dovecot. I knew it must be something bone-headed I just wasn't seeing. And now I can go get some rest. --James W. Rogers

1 0

lazy_expunge mangles dovecot-acl-list
by Christoph Bußenius 12 Jun '14

12 Jun '14

Hi, I think I found a bug in Dovecot 2.1.17 and 2.2.13. In our setup, sometimes ACLs stop working because "dovecot-acl-list" is replaced by an empty file. We found that lazy_expunge is connected to this. To reproduce, create ACLs for "user1" in a folder. Put a mail in that folder and expunge it, so that the folder will be created in the "expunged" namespace. For instance, # cat user1/mail/mailboxes/folder/dbox-Mails/dovecot-acl user=user2 keilrwts # cat user1/mail/dovecot-acl-list 1350914868 folder # doveadm -f flow fetch -u "user1" 'guid' mailbox _EXPUNGED.\* # ls -l user1/mail/dovecot-acl-list -rw------- 1 vmail vmail 0 2014-06-12 11:40 user1/mail/dovecot-acl-list You see that we have used doveadm to list the expunged namespace, which has emptied the "dovecot-acl-list" file. Cheers, Christoph # 2.2.13: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-57-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mail mail_plugins = acl mail_uid = vmail namespace { inbox = no list = children location = mdbox:%%h/mail prefix = INBOX.shared.%%u. separator = . subscriptions = no type = shared } namespace default { inbox = yes location = prefix = INBOX. separator = . type = private } namespace expunged { hidden = yes list = no location = mdbox:~/mail:MAILBOXDIR=expunged:SUBSCRIPTIONS=expunged-subscriptions prefix = _EXPUNGED. separator = . subscriptions = yes } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/mail/shared-mailboxes lazy_expunge = _EXPUNGED. } protocols = imap pop3 service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = /usr/local/dovecot/etc/dovecot/users driver = passwd-file } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_max_userip_connections = 20 mail_plugins = acl imap_acl acl } -- Christoph Bußenius Rechnerbetriebsgruppe Informatik und Mathematik Technische Universität München

2 2

Parse dovecot 2.2 logs with logstash
by Bruno Galindro da Costa 12 Jun '14

12 Jun '14

Guys, I need to parse my dovecot log files with logstash grok patterns. Is there any document specifying the patterns used by dovecot to write it's logs? I need to find all the log possibilities that could be writed to log files by dovecot. So, if a document like that exists or if anyone could answer my question, I'll could make the parser with less difficult. -- Att. Bruno Galindro da Costa

3 3

Problems with dsync and global acls
by Michał Węgrzynek 12 Jun '14

12 Jun '14

Hi all! I have 2 servers running dsync tcp replication between them. After setting global ACLs to * user=admin lrwstipekxa on both hosts, I get the following errors during replication for every folder: dsync-local(mwegrzynek): Error: Mailbox Trash: Failed to set attribute vendor/vendor.dovecot/pvt/acl/user=admin: Invalid right '' This error first manifested in version 2.2.13, there were no such errors in 2.2.9 (stock Ubuntu 14.04) version. Am I doing something wrong? My configuration: # 2.2.13.3 (6dab0352ccb3+): /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-29-generic x86_64 Ubuntu 14.04 LTS ext4 auth_cache_size = 10 M auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/dovecot/krb5.keytab auth_master_user_separator = * auth_mechanisms = plain login gssapi auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@() auth_username_format = %Ln auth_verbose = yes default_client_limit = 10000 default_process_limit = 1000 default_vsz_limit = 2 G doveadm_port = 12345 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_rcpt_check_quota = yes lmtp_save_to_detail_mailbox = yes mail_gid = vmail mail_home = /var/mail/users/%u mail_location = mdbox:/var/mail/users/%u/mdbox mail_plugins = acl quota virtual zlib listescape fts fts_squat notify replication mail_prefetch_count = 20 mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate mdbox_preallocate_space = yes namespace dzielone { list = children location = mdbox:%%h/mdbox:INDEXPVT=~/shared/%%u prefix = Dzielone/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } namespace publiczne { list = children location = mdbox:/var/mail/public:INDEXPVT=~/public prefix = Publiczne/ separator = / subscriptions = no type = public } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.passdb.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = file:/var/mail/config/shared-mailboxes fts = squat fts_squat = partial=4 full=10 mail_replica = tcps:xxx.yyy.zzz quota = dict:User quota::file:%h/dovecot-quota sieve = ~/.dovecot.sieve sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } pop3_uidl_format = %v.%u protocols = imap pop3 lmtp sieve quota_full_tempfail = yes replication_dsync_parameters = -d -l 60 -n inbox -U service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail user = vmail } } service doveadm { inet_listener { port = 12345 ssl = yes } process_limit = 10 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl_ca = </etc/ssl/certs/xxx.pem ssl_cert = </etc/ssl/certs/default.pem ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM ssl_client_ca_file = /etc/ssl/certs/xxx.pem ssl_key = </etc/ssl/private/default.key userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.userdb.ext driver = ldap } protocol lmtp { mail_plugins = acl quota virtual zlib listescape fts fts_squat notify replication sieve } protocol lda { mail_plugins = acl quota virtual zlib listescape fts fts_squat notify replication sieve } protocol imap { mail_plugins = acl quota virtual zlib listescape fts fts_squat notify replication imap_acl imap_zlib imap_quota } Thanks in advance for your help, -- *Michał Węgrzynek*

1 0

Converting mailboxes from one Maildir format to another
by Patrick C 12 Jun '14

12 Jun '14

I am currently running dovecot 2.0.9 on a CentOS 6 server with the following layout for mailboxes: mail_location = maildir:~/Maildir I would like to change the layout to this format to match the output of other software packages such as offlineimap, mbsync, etc: mail_location = maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX Is there a way to convert the mailbox layout for all mail accounts currently on the server? I was looking at dsync, but there aren't any Maildir <--> Maildir examples. I was also looking at potentially writing a script to manually rename directories/move files around to match the new format, but wasn't sure whether that would mess with UIDs or other dovecot internals. pat patrickc81(a)gmail.com

2 1

Maildir index
by Eduardo Ramos 11 Jun '14

11 Jun '14

Hi all! I store my maildir mailboxes in a ocfs2 volume. By default, my indexes are stored with the inbox. If I change now to a local disk, could I have trouble? That's my production server. Thanks!

1 0

Safe to downgrade 2.2 to 2.1?
by Florian Tischler 11 Jun '14

11 Jun '14

Hi, is it safe to downgrade Dovecot 2.2 to 2.1 (in my case 2.2.13 to 2.1.17 with mdbox format) or will there be troubles with index files and backup/restore is recommended? Unfortunately with 2.2+ it is not to be possible to use acl + lazy_expunge together anymore, so downgrading is the only option for me... @Timo, please fix acl + lazy_expunge in 2.2 if you find some time. Florian -- Florian Tischler System Administrator *Johann Radon Institute for Computational and Applied Mathematics (RICAM) http://www.ricam.oeaw.ac.at/

1 0

Change index location problem
by igord 11 Jun '14

11 Jun '14

Hi All, I was unable to change location of Dovecot indexes. Currently they are under /var/vmail/%d/%u, but I want to change them to /media/ephemeral0/%u I've changed conf line in /etc/dovecot/dovecot.conf: mail_location = maildir:/var/vmail/%d/%u:INDEX=/media/ephemeral0/%u and setup permissions of /media/ephemeral0 to dovecot:dovecot, tried also vmail:mail (which is default user for e-mails) and also tried with 777 on all dirs, but Dovecot is keeping creating indexes on the old location. I tried moving them from old to new location, but Dovecot recreates them on the old location. Details: # dovecot --version 2.0.9 # doveconf -a | grep location mail_location = maildir:/var/vmail/%d/%u:INDEX=/media/ephemeral0/%u Has anyone experienced something similar, any solution for this? Thanks in advance! -- View this message in context: http://dovecot.2317879.n4.nabble.com/Change-index-location-problem-tp48468.… Sent from the Dovecot mailing list archive at Nabble.com.

2 4

Re: Subject tag [Dovecot] is gone
by Professa Dementia 11 Jun '14

11 Jun '14

On 6/9/2014 7:26 PM, Timo Sirainen wrote: > The main reason is DKIM, which is starting to be a real problem. I have not used DKIM much. My mail server and client mostly deal with SPF. I have a filter that colorizes messages that have no SPF or a missing DKIM or bad DKIM signature. I *have* noticed that a lot of messages from the list get marked in such manner, but it never really bothered me and I never thought about it much. Now I understand why that happens (the [Dovecot] identifier in the subject). When trying to solve a problem, the first thing is to correctly identify the problem. You cannot solve a problem if you do not even know what it is. The underlying problem is to identify and classify emails as ones you want and ones you do not want. This is not easy and involves reading a person's mind. A person may, depending on their mood, classify the same email differently at different times, which complicates things. DKIM assumes that you can, in many cases, classify emails this way based on authenticating the *domain* of the sender. This has some serious flaws in that it does not address this issue, even though it purports to. One way to classify an email as "wanted" is if it comes from someone you know and want to communicate with. Signing based on a domain does nothing to address this. If my girlfriend is judy(a)yahoo.com, I want to receive her emails. That does not means I want to receive all emails from the yahoo.com domain. I do not want someone else to impersonate her. If later, we break up and I no longer want to receive her emails, DKIM does nothing to help with that, either. That could be OK if such functionality is beyond its scope. DKIM erroneously bundles sender authentication with message validation. I want to know that it really was judy(a)yahoo.com that sent me the message and not someone trying to impersonate her. However, as a separate function, I would like to know that the message I received is not the one she sent. These functions should not be integrated. As it is now, if the signature does not verify, I do not know why. Was the sender spoofed? Was some part of the message modified in some way? And just for the record, I believe that the subject line should conceptually be treated as part of the message, along with the date. DKIM is too strict. If I want to present a legal document (email) in court, I may want to prove that the document I present to the court is exactly as it was when it was sent to me. However, this is not a common occurrence. The real world is messy and imperfect and often, changes to emails are innocuous and legitimate. Mailing lists are an example of this. A mailing list or anti-virus scanner *should* be able to add a footer or add a mailing list identifier to the subject line, as long as those changes can be marked as later additions that the original sender is not accountable for. An email program should make it clear to the recipient which parts are not accountable to the original sender. I am not proposing a new standard, simply pointing out that breaking an established protocol (by removing the [Dovecot] subject identifier) because of a flawed anti-spam system is not in people's best interest. Can a spammer spoof messages from the list? Sure. Has it happened? Not that I am aware of. Is it a problem? Not so far. So why, then, make people go through all this trouble of setting up new filters and rules, mail routing, software upgrades, etc, just to appease a standard that is clearly broken? Dem

5 4

replication with plugin won´t work
by Florian Schaal 11 Jun '14

11 Jun '14

Hi all, i try to setup a master-master replication with dovecot / dsync. I´m using virtual-users. When I call doveadm sync -A remote:vmail@server2 new mails are copied to the second server, but new messages from server 2 not on Server 1. To fix this i run doveadm sync -A remote:vmail@server on the 2nd server But I want that it works with dovecot / dsync (see my config at the end - identical on bot servers, only mail_replica is different). I added some services regarding http://wiki2.dovecot.org/Replication If i send a local mail on one server it will not be replicated to the other server until i run doveadm sync. Maybe i just missed some steps in my setup? doveadm user '*' displays the right users on both servers. === docecot.conf === # 2.2.13 (7b25994a8cb7): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab076.8 x86_64 Debian 7.5 auth_mechanisms = plain login ntlm disable_plaintext_auth = no dsync_remote_cmd = ssh -p 4711 -l%{login} %{host} doveadm dsync -u%u listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_plugins = " notify replication" mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = remote:vmail@server2 quota = dict:user::file:/var/vmail/%d/%n/.quotausage replication_full_sync_interval = 1 hours sieve = /var/vmail/%d/%n/.sieve } protocols = imap pop3 service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service config { unix_listener config { user = vmail } } service doveadm { user = vmail } service imap-login { client_limit = 1000 process_limit = 500 } service replicator { process_min_avail = 1 } ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve quota } === docecot.conf === regards Florian

1 0