- dovecot - dovecot.org

dsync full sync
by Atıf CEYLAN 29 Sep '14

29 Sep '14

Hi all, I have 2 question. First: I use dovecot (version 2.2.9) with mdbox mail format. When I run dsync tool with "mirror" or "backup" parameters my source and destination directory synchronize correctly but if I delete some messages in user mailbox, deleted messages does not synced to destination. For example : atif(a)domain.com path is /mail/domain.com/atif/ and its size is 1GB. after first running the "dsync -u atif(a)domain.com backup mdbox:/backup/domain.com/atif/" command, size of "/backup/domain.com/atif/" is 1GB I deleted 300mb messages in atif mailbox then /mail/domain.com/atif/ is decreased to 700MB and I launched dsync command again. But /backup/domain.com/atif/ is not decreased to 700Mb. My backup directory still 1GB . I tried dsync with "-f , backup, mirror" parameters. Second problem, the first time the backup atif's quota size is increasing to x2. before backup : atif(a)domain.com quota size 1GB after backup: atif(a)domain.com quota size 2GB so I must run "doveadm quota recalc" command. Why? What is correct usage of mdbox mail format and dsync tool? -- M.Atıf CEYLAN

2 1

dsync mirror does not maintain changes between mailboxes
by Alan McGinlay 28 Sep '14

28 Sep '14

Hi, running: dsync -D -v -o mail_fsync=never mirror -R -f -u user@domain imapc: Where "domain" is a cyrus imap server does not do what the documentation says it does, not even close. After the first run to an empty mailbox on the dovecot 2.2.13 server all the mails (1.1GB) seem to be correctly synced. I send a test email to the dovecot server and it shows up correctly. Second run of the above dsync command, first thing that happens is all mails are deleted from the dovecot mailbox (yet the storage used by output of "du -m" doesn't change) and the sync appears to happen again from scratch however the disk space increases again! Now up to about 2GB and another run will take it up a few hundred mb more! I have tried with multiple different mail storage formats, maildir++, mdbox sdbox, all show the same (except maildir which just complains that INBOX can't be deleted on the second dsync run). This is frustrating beyond belief, please please anyone who has any insight let me know, the dsync debug output is a little huge to sanitize and post to the list but i can send it individually if anyone has any ideas! I am ready to give up totally on dovecot but at the same time I just wish it would work! doveconf -n # 2.2.13 (6bb26098a45c): /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-67-virtual x86_64 Ubuntu 14.04.1 LTS auth_debug = yes auth_master_user_separator = * default_vsz_limit = 512 M disable_plaintext_auth = yes imapc_features = rfc822.size imapc_host = oldmailhost.domain.com imapc_master_user = cyradmin imapc_password = xxxxxxxx imapc_user = %u mail_gid = vmail mail_home = /var/vmail/domains/%d/%1n/%n mail_location = sdbox:~/sdbox mail_plugins = zlib notify quota fts fts_solr mail_prefetch_count = 30 mail_uid = vmail mailbox_list_index = yes managesieve_sieve_capability = fileinto imapflags namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = solr fts_solr = url=http://localhost:8080/solr/ quota = dirsize:User quota quota_grace = 10%% quota_rule = *:storage=20G sieve = ~/.dovecot.sieve sieve_dir = /var/vmail/sieve/domain/%1d/%d/%1n/%n/ sieve_extensions = imapflags fileinto zlib_save = bz2 zlib_save_level = 5 } protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } vsz_limit = 512 M } service imap { process_limit = 1024 vsz_limit = 512 M } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_ssl = yes protocol lmtp { mail_plugins = zlib notify quota fts fts_solr quota sieve } protocol imap { mail_plugins = zlib notify quota fts fts_solr imap_quota } protocol sieve { managesieve_implementation_string = Cyrus timsievd v2.2.13 }

1 0

Re: Dovecot Sieve and Postfix header_checks Issue
by Joseph Tam 27 Sep '14

27 Sep '14

"Klaipedaville on Google" <klaipedaville(a)gmail.com> writes: > However, my header_checks file has just 5 lines of regexp as follows: > ... > /^From:.*\@.*\.tw/ REJECT Sorry, Taiwanese mail is not allowed. Can't speak about the other issues you are having, but is this regexp pattern what you want? Unless Postfix PCRE automatically right-anchors these regexps, aren't you rejecting mail from someone(a)mail.twinpeaks.org, or even twitter notifications (from *(a)bounce.twitter.com) Joseph Tam <jtam.home(a)gmail.com>

4 7

Dovecot Sieve and Postfix header_checks Issue
by Klaipedaville on Google 27 Sep '14

27 Sep '14

Hello List, I tried to subscribe but it's taking forever for the confirmation email to arrive so I thought I would ask away by emailing directly. My apologies in advance should this question appear twice. It may seem real simple to experts but I cannot really figure it out. I'll try to be concise: Dovecot version is 2.1.7. Its dovecot –n is real short one and follows right after my question. I have my Postfix 2.9.6 properties set like this: header_checks = /etc/path/to/myfile. Then I have Dovecot Sieve also configured and working fine. Now the trouble is that these two cannot be combined together can they? Dovecot Sieve and Postfix's header_checks? If I turn off header_checks in Postfix then Dovecot Sieve is working fine. If I vise versa turn off Dovecot Sieve then Postfix's header-checks are also working fine. However, when I keep them both turned on it also works but only partially and in this case Dovecot Sieve never bounces back any rejected massages remotely, the bounce happens only locally. That is I can see it in my logs that it was rejected locally but the message is never sent back to the sender saying something like, "Spam is rejected here". Thus the reject turns into a silent discard without telling the sender anything at all. Any ideas, help, advices how do I fix that? Would be really grateful for any suggestions / assistance at all. Many thanks in advance! P.S. I would like to use both because Dovecot Sieve is a very powerful and great plugin that uses Spamassassin's added tags and headers based on which I create my rules. Postfix in its turn has a very neat feature of checking the headers via regexp but Postfix does not see / detect Spamassassin's added headers at all because I passed delivery and authentication (which is a way better than Postfix's) to dovecot deliver. # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4 auth_mechanisms = plain login cram-md5 scram-sha-1 auth_verbose = yes hostname = WindTalker info_log_path = /var/log/dovecot-sieve.log log_path = /var/log/dovecot-sieve-errors.log mail_location = maildir:/home/mvail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/passwd driver = passwd-file } plugin { sieve = ~/.dovecot.sieve sieve_default = /etc/dovecot/default.sieve sieve_dir = ~/sieve } pop3_uidl_format = %g postmaster_address = postmaster(a)example.com protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl_ca = </etc/ssl/ca.crt ssl_cert = </etc/ssl//new.crt ssl_key = </etc/ssl/new.key userdb { args = uid=vmail gid=vmail home=/home/vmail/%u driver = static } protocol lda { mail_debug = yes mail_plugins = sieve } Regards, Dennis.

4 22

ALT storage question
by Ralf Hildebrandt 27 Sep '14

27 Sep '14

I want to migrate from a non-hashed storage to a hashed storage. So I set this up: mail_location = mdbox:/srv/vmail/%2.256Hu/%Lu/mdbox:ALT=/srv/vmail/%Ld/%Ln/mdbox I rsynced the "old" mdbox-style mailbox from another machine to to /srv/vmail/%Ld/%Ln/mdbox (/srv/vmail/charite.de/ralf.hildebrandt/mdbox in my case) The mail_location /srv/vmail/c5/ralf.hildebrandt(a)charite.de was non-existing. Then I connected using IMAP and tried to have a look at the INBOX: Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Debug: Effective uid=10000, gid=10000, home=/srv/vmail/c5/ralf.hildebrandt(a)charite.de Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Debug: Home dir not found: /srv/vmail/c5/ralf.hildebrandt(a)charite.de Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Debug: Namespace inbox: type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/srv/vmail/c5/ralf.hildebrandt@charite.de/mdbox:ALT=/srv/vmail/charite.de/ralf.hildebrandt/mdbox Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Debug: fs: root=/srv/vmail/c5/ralf.hildebrandt(a)charite.de/mdbox, index=, indexpvt=, control=, inbox=, alt=/srv/vmail/charite.de/ralf.hildebrandt/mdbox Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Debug: Namespace : /srv/vmail/c5/ralf.hildebrandt(a)charite.de/mdbox doesn't exist yet, using default permissions Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Debug: Namespace : Using permissions from /srv/vmail/c5/ralf.hildebrandt(a)charite.de/mdbox: mode=0700 gid=default Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Debug: Namespace : /srv/vmail/c5/ralf.hildebrandt(a)charite.de/mdbox/mailboxes/INBOX doesn't exist yet, using default permissions Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Debug: Namespace : Using permissions from /srv/vmail/c5/ralf.hildebrandt(a)charite.de/mdbox: mode=0700 gid=default Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Error: Mailbox INBOX has existing files in alt path, rebuilding storage to avoid losing messages Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Error: Failed to autocreate mailbox INBOX: Internal error occurred. Refer to server log for more information. [2014-09-27 11:15:15] Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Warning: fscking index file /srv/vmail/c5/ralf.hildebrandt(a)charite.de/mdbox/storage/dovecot.map.index Sep 27 11:15:15 imap(ralf.hildebrandt(a)charite.de): Info: Disconnected: Logged out in=58 out=623 What is "Mailbox INBOX has existing files in alt path, rebuilding storage to avoid losing messages" referring to? And why is this an error? After all that I reconnect, just to find an empty inbox. WTF? Why is the ALT storage being ignored? The ALT storage contains: total 8 drwxr-xr-x 6 vmail vmail 4096 Aug 18 15:25 mailboxes drwxr-xr-x 2 vmail vmail 4096 Sep 24 00:37 storage and those two directories contain: mailboxes: total 16 drwxr-xr-x 3 vmail vmail 4096 Aug 18 15:25 Drafts drwxr-xr-x 3 vmail vmail 4096 Aug 18 15:25 INBOX drwxr-xr-x 3 vmail vmail 4096 Aug 18 15:25 Sent drwxr-xr-x 3 vmail vmail 4096 Aug 18 15:25 Trash storage: -rw-r--r-- 1 vmail vmail 3552 Sep 25 14:47 dovecot.map.index.log -rw-r--r-- 1 vmail vmail 20015 Sep 25 13:51 m.1 On the other hand, just moving /srv/vmail/charite.de/ralf.hildebrandt/mdbox to /srv/vmail/c5/ralf.hildebrandt(a)charite.de worked like a charm! -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

1 0

Re: Is dovecot vulnerable to the shellshock/CVE-2014-6271 exploit?
by Joseph Tam 27 Sep '14

27 Sep '14

Timo Sirainen writes: >> Although I don't use it, it's plausible the checkpassword hook is also vulnerable >> via the MASTER_USER environment variable: >> >> http://wiki2.dovecot.org/AuthDatabase/CheckPassword > > This is one possibility, and it's the worst one because it could happen > before login. But it requires two things: > > 1. auth_username_chars setting must include the characters required in > the exploit, so "(){;" at least I guess. None of these characters are > enabled by default. But I think some people may have set this setting > to empty to allow all characters. > > 2. checkpassword must call bash, which also isn't done by default. > > Another possibility is is that in some setups the password (%w) may be > added to userdb fields, which ends up being exported to environment if > post-login scripts are used. Again Dovecot doesn't execute shell > automatically, but it may end up being executed by the configuration. > So this requires a valid username + password, and ability to change the > password to the bash exploit. Thanks for confirming what I suspected, Timo. The latter, if it is at all feasible, appears to be an "inside job" type of exploit, where the exploiter already has an account but can arbitrarily change their password. It's not as serious as the pre-login one, but worth addressing if the narrow circumstances of using post-login bash scripts apply. Joseph Tam <jtam.home(a)gmail.com>

1 0

doveadm penalty: who is 0.0.0.0?
by Peer Heinlein 27 Sep '14

27 Sep '14

Using doveadm penalty I just noticed a penalty for 0.0.0.0. Is this Postfix querying the auth-socket for his SASL-requests? I tried adding 0.0.0.0 to login_trusted_networks, but that doesn't help. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 2

Re: director
by Chris Lasater 26 Sep '14

26 Sep '14

The configuration and run/base_dir directories can be where ever you like. The locations Eduardo stated are the most appropriate for dovecot installed with an RPM. Mine were compiled and set in the home directory, so they are in a slightly different location. As long as you use the -c to point to the correct configuration file you should be good. For mine I kept my dovecot config file in the default location and put the director one elsewhere and created an alias, so that I can start the director by just typing director. This might make things easier for you to manage, then you can just stop it by running "doveadm -i <instance_name> stop" alias director="sudo -E PATH=${PATH} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} /home/user/dovecot/sbin/dovecot -c /home/user/dovecot/etc/dovecot/director.conf" On 09/23/2014 05:00 PM, jjhoffart wrote: >> Thanks for the replies, >> I am still fairly new to dovecot and still learning. This is a quick run down of my setup: 3 servers running iRedMail (this is a package that installs dovecot 2.1.7, postfix, and LDAP). All three servers share an NFS backend. The same 3 servers are also sitting behind a load balancer. What I want to do is use these three servers as directors and mail servers. So from the replies I have gotten I understand that I need to create 2 config files 1 for director and 1 for dovecot. Now my next question is do these config files need to be stored in /etc/dovecot or /var/run/dovecot? Also is creating the second config file as simple as copying and renaming the dovecot.conf file? >> >> On Sep 23, 2014, at 2:26 PM, Eduardo Ramos <eduardo(a)freedominterface.org> wrote: >> >>> Hi, >>> >>> In fact you don't NEED a director. You can use two dovecot servers sharing a NFS storage with a normal (backend) implementation. It's good to see that page: http://wiki2.dovecot.org/NFS for some good recomendations. >>> >>> You must have a base_dir for each instance of dovecot you want to run, and a dovecot.conf as well. One could be director and other backend. I suggest you have two separated configuration directories, /etc/dovecot-director and /etc/dovecot-backend. Each with the respective configuration. Start dovecot with "dovecot -c /etc/dovecot-director/dovecot.conf". Backend is the same. >>> >>> In order to help more, provide us more details about your environment and doulbts. >>> >>> On 09/23/2014 03:57 PM, jjhoffart wrote: >>>> Hello, >>>> I am currently trying to setup a cluster of mail servers. I currently have two servers I am trying to cluster together. These servers will also have a shared NFS backend. I have read on the site that I will need to configure director since I am using NFS. I have read through all of the documentation on director but still need a little guidance to get things up and running. One of the questions I have is about the base_dir on the Multiple Invocations section of the wiki. It states that I will need to change the bas_dir to the new run directory, not sure what needs to happen here, can anyone help me out?

3 3

Namespace configuration over SQL
by Ain 26 Sep '14

26 Sep '14

Hello! I had working setup, where namespace settings (including prefix) was queried over SQL. Now, after installing new Dovecot to new server (using same configuration), this is not working anymore. Log shows following error: """ Initialization failed: namespace configuration error: list=yes requires prefix=yes to end with separator """ relevant part from dovecot-sql.conf.ext: """ user_query = \ SELECT \ 1002 AS uid \ , 1003 AS gid \ , CONCAT('/www/home/vmail/domains/', SUBSTRING(d.name, 1, 1), '/', d.name, '/', u.username) AS home \ , namespace_prefix AS 'namespace/default/prefix' \ , 'yes' AS 'namespace/default/inbox' \ , '%s' AS 'mail_service' \ FROM vm_domain d \ INNER JOIN vm_user u ON u.vm_domain_id = d.id \ WHERE d.name = '%d' AND u.username = '%n' AND u.active = 1 AND d.active = 1 """ namespace_prefix can be empty or INBOX. doveconf -n output: """ # 2.2.13: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.3-RELEASE amd64 auth_debug = yes auth_verbose = yes disable_plaintext_auth = no listen = * mail_debug = yes mail_location = maildir:~/Maildir mmap_disable = yes namespace default { inbox = yes list = yes location = prefix = separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size mail_log_group_events = no } service imap { process_limit = 1480 } ssl_cert = </usr/local/etc/ssl/server.crt ssl_key = </usr/local/etc/ssl/server.key.unsecure userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE mail_max_userip_connections = 900 } protocol pop3 { mail_max_userip_connections = 300 } """ Seems like "prefix = " is interpreted as "prefix = yes". How I can solve this?

2 5

Is dovecot vulnerable to the shellshock/CVE-2014-6271 exploit?
by Joseph Tam 26 Sep '14

26 Sep '14

I'm right now handling this beach-ball sized grenade, and trying to figure out which of our services need to be locked down right away. Since dovecot passes values via environment variables based on user input (e.g. username, password, mailbox?) to auxilliary executables (including possibly bash shell scripts), is dovecot vulnerable to this exploit? (This is not a fault of dovecot, but rather bash's inadequate handling of environment variables.) For example, injection of this sort 1 LOGIN (){x;}exploit-code whatever I guess auth_username_chars would mitigate this particular attempt (assuming it can work), but other values such as mailbox names could also be injected post authentication. Can someone with working knowlegde of dovecot's internals confirm/deny whether this is a something that needs to be addressed? Joseph Tam <jtam.home(a)gmail.com>

4 4