September 2014 - dovecot

Indexing of mailf fts clucene aborts
by Rainer M Krug 25 Sep '14

25 Sep '14

Hi I am trying lucene to index my email. To create the index, I run the following command and the indexing aborts: ,---- | 09:42:20 ~$ doveadm -Dv index -u rainerkrug 'RMKrugGMAIL.*' | doveadm(rainerkrug): Debug: Loading modules from directory: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot | doveadm(rainerkrug): Debug: Module loaded: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/lib20_fts_plugin.so | doveadm(rainerkrug): Debug: Module loaded: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/lib21_fts_lucene_plugin.so | doveadm(rainerkrug): Debug: Loading modules from directory: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm | doveadm(rainerkrug): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: dlopen(/usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so, 10): Symbol not found: _acl_user_module | Referenced from: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | Expected in: flat namespace | in /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so (this is usually intentional, so just ignore this message) | doveadm(rainerkrug): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: dlopen(/usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so, 10): Symbol not found: _expire_set_deinit | Referenced from: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so | Expected in: flat namespace | in /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so (this is usually intentional, so just ignore this message) | doveadm(rainerkrug): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: dlopen(/usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so, 10): Symbol not found: _quota_user_module | Referenced from: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so | Expected in: flat namespace | in /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so (this is usually intentional, so just ignore this message) | doveadm(rainerkrug): Debug: Module loaded: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so | doveadm(rainerkrug): Debug: Module loaded: /usr/local/Cellar/dovecot/2.2.13_1/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so | doveadm(rainerkrug): Debug: Effective uid=501, gid=20, home=/Users/rainerkrug | doveadm(rainerkrug): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir | doveadm(rainerkrug): Debug: maildir++: root=/Users/rainerkrug/Maildir, index=, indexpvt=, control=, inbox=/Users/rainerkrug/Maildir, alt= | doveadm(rainerkrug): Debug: Ignoring unknown cache field: pop3.order | doveadm(rainerkrug): Debug: Ignoring unknown cache field: binary.parts | doveadm(rainerkrug): Info: RMKrugGMAIL.[Gmail].All Mail: Caching mails seq=2..49011 | 48700/49010Assertion failed: (numDocsInStore*8 == directory->fileLength( (docStoreSegment + "." + IndexFileNames::FIELDS_INDEX_EXTENSION).c_str() )), function closeDocStore, file /tmp/clucene-M0PS6G/src/core/CLucene/index/DocumentsWriter.cpp, line 210. | Abort trap: 6 `---- Any suggestions? I am using dovecot on a Mac, Maverich=ks, installed via homebrew. If you need any further info, please let me know. Thanks, Rainer -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982

1 0

Quota Woes
by Art Stephens 25 Sep '14

25 Sep '14

Trying to get quota to show up in IMP - Horde 5.x Seems it is not working with dovecot Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready. root@mail:~# dovecot --version 2.0.19 root@mail:~# dovecot -n # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.11.0-26-generic x86_64 Ubuntu 12.04.4 LTS auth_mechanisms = plain login disable_plaintext_auth = no mail_location = mbox:~/mail/:INBOX=/var/mail/%u mail_plugins = " quota" passdb { driver = pam } plugin { quota = fs:User quota quota_rule = *:storage=300M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd } protocol imap { mail_plugins = " quota imap_quota" } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } root@mail:~# quotaon -pa group quota on / (/dev/mapper/mail--vg-root) is on user quota on / (/dev/mapper/mail--vg-root) is on What am I missing? -- Arthur Stephens Senior Networking Technician Ptera Inc. PO Box 135 24001 E Mission Suite 50 Liberty Lake, WA 99019 509-927-7837 ptera.com facebook.com/PteraInc | twitter.com/Ptera ----------------------------------------------------------------------------- "This message may contain confidential and/or propriety information, and is intended for the person/entity to whom it was originally addressed. Any use by others is strictly prohibited. Please note that any views or opinions presented in this email are solely those of the author and are not intended to represent those of the company."

2 1

SSL issues when proxying
by Ralf Hildebrandt 25 Sep '14

25 Sep '14

I'm getting this in the log when proxying IMAP (three "valid certificate" messages, two "Invalid certificate" messages) Why is dovecot (acting as a proxy to another dovecot instance here) not recognizing the StartCom Extended Validation Server CA? . LOGIN ralf.hildebrandt(a)charite.de mypassword Sep 25 14:13:04 auth-worker(30859): Info: mysql(sql.charite.de): Connected to database mailservice Sep 25 14:13:04 imap-login: Debug: SSL: where=0x10, ret=1: before/connect initialization [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: before/connect initialization [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: unknown state [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1002, ret=-1: unknown state [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 read server hello A [127.0.0.1] Sep 25 14:13:04 imap-login: Info: Invalid certificate: unable to get local issuer certificate: /C=IL/O=StartCom Ltd./OU=StartCom Certification Authority/CN=StartCom Extended Validation Server CA Sep 25 14:13:04 imap-login: Info: Invalid certificate: certificate not trusted: /C=IL/O=StartCom Ltd./OU=StartCom Certification Authority/CN=StartCom Extended Validation Server CA Sep 25 14:13:04 imap-login: Info: Valid certificate: /C=DE/ST=Berlin/L=Berlin/postalCode=12205/street=Charitestrasse 1/O=Charite Universitaetsmedizin/CN=imap.charite.de/emailAddress=postmaster(a)charite.de/serialNumber=HRAxxxx/businessCategory=Private Organization/1.3.6.1.4.1.311.60.2.1.1=Mitte/1.3.6.1.4.1.311.60.2.1.2=Berlin/1.3.6.1.4.1.311.60.2.1.3=DE Sep 25 14:13:04 imap-login: Info: Valid certificate: /C=IL/O=StartCom Ltd./OU=StartCom Certification Authority/CN=StartCom Extended Validation Server CA Sep 25 14:13:04 imap-login: Info: Valid certificate: /C=DE/ST=Berlin/L=Berlin/postalCode=12205/street=Charitestrasse 1/O=Charite Universitaetsmedizin/CN=imap.charite.de/emailAddress=postmaster(a)charite.de/serialNumber=HRAxxxx/businessCategory=Private Organization/1.3.6.1.4.1.311.60.2.1.1=Mitte/1.3.6.1.4.1.311.60.2.1.2=Berlin/1.3.6.1.4.1.311.60.2.1.3=DE Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 read server certificate A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 read server key exchange A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 read server done A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 write client key exchange A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 write change cipher spec A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 write finished A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 flush data [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1002, ret=-1: SSLv3 read server session ticket A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1002, ret=-1: SSLv3 read server session ticket A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 read server session ticket A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1001, ret=1: SSLv3 read finished A [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [127.0.0.1] Sep 25 14:13:04 imap-login: Debug: SSL: where=0x1002, ret=1: SSL negotiation finished successfully [127.0.0.1] . OK [CAPABILITY ... -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

3 3

vsz_limit is 18,447 PB
by Dan Langille 25 Sep '14

25 Sep '14

I encountered this error on FreeBSD 9.3 with dovecot2-2.2.13_3 Short version; vsz_limit is 18,447 PB and it ran out of RAM. Suggestions for change? Sep 6 03:39:32 mailjail dovecot: imap(dan): Panic: file imap-fetch.c: line 556 (imap_fetch_more): assertion failed: (ctx->client->output_cmd_lock == NULL || ctx->client->output_cmd_lock == cmd) Sep 6 03:39:32 mailjail dovecot: imap(dan): Fatal: master: service(imap): child 71153 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes }) Sep 6 03:59:41 mailjail dovecot: imap(dan): Fatal: pool_system_realloc(2097152): Out of memory Sep 6 03:59:41 mailjail dovecot: imap(dan): Fatal: master: service(imap): child 67732 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) Background: I’m in the only user on this system, but this server is accessed by my phone, my laptop, my tablet, and perhaps a web interface. Here are the non-default values: $ doveconf -n # 2.2.13: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.3-RELEASE amd64 auth_mechanisms = plain login first_valid_gid = 1001 first_valid_uid = 1001 mail_location = maildir:~/Maildir mail_max_userip_connections = 80 mail_privileged_group = mail passdb { args = scheme=SHA512-CRYPT /var/db/dovecot.users driver = passwd-file } protocols = imap service imap-login { inet_listener imap { address = 10.0.0.1 } inet_listener imaps { port = 0 } } ssl = required ssl_ca = </usr/local/etc/ssl/ca.pem ssl_cert = </usr/local/etc/ssl/server.pem ssl_key = </usr/local/etc/ssl/mailjail.example.org.nopassword.key userdb { args = /var/db/dovecot.users driver = passwd-file } verbose_proctitle = yes verbose_ssl = yes But there are some interesting values when I look at the output of doveconf. Specifically, vsz_limit is 18,447 PB… yeah, that’s pretty big. service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = 10.0.0.1 port = 143 reuse_port = no ssl = no } inet_listener imaps { address = port = 0 reuse_port = no ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B }

4 7

Confusing wording on the wiki
by Alan McGinlay 24 Sep '14

24 Sep '14

Hi, I am confused by the wording of this Wiki entry: sieve = ~/.dovecot.sieve The path to the user's main active Sieve script. When ManageSieve is used, this is a symbolic link pointing to the active script in the sieve_dir directory. Avoid placing the active Sieve script inside the sieve_dir directory. So this should be a link *to* the active script in the sieve_dir directory however the active script shouldn't actually be in that directory???? http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration Confused :S /A

2 1

LMTP proxying
by Ralf Hildebrandt 24 Sep '14

24 Sep '14

How exactly is LMTP proxying supposed to work? According to the example on http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy I'm only seeing "proxy" in the password_query statements. But LMTP never sends a password. IMAP Proxying is working OK: Sep 24 12:51:09 imap-login: Info: proxy(ralf.hildebrandt(a)charite.de): started proxying to dovecot.charite.de:993: user=<ralf.hildebrandt(a)charite.de>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Sep 24 12:51:13 imap-login: Info: proxy(ralf.hildebrandt(a)charite.de): disconnecting 127.0.0.1 (Disconnected by server): user=<ralf.hildebrandt(a)charite.de>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured But LMTP is logging this: Sep 24 13:13:25 lmtp(17727): Info: Connect from 127.0.0.1 Sep 24 13:13:51 auth-worker(19603): Info: mysql(sql.posteo.local): Connected to database mailservice Sep 24 13:14:11 lmtp(17727, ralf.hildebrandt(a)charite.de) Info: save: box=INBOX, uid=1, msgid=, size=304 Sep 24 13:14:11 lmtp(17727, ralf.hildebrandt(a)charite.de) Info: pH2yN1WnIlQ/RQAAoQzodQ: msgid=unspecified: saved mail to INBOX Sep 24 13:14:11 dict: Info: mysql(sql.charite.de): Connected to database mailservice Sep 24 13:14:13 lmtp(17727): Info: Disconnect from 127.0.0.1: Client quit (in reset) Sep 24 13:15:29 master: Warning: Killed with signal 15 (by pid=25892 uid=0 code=kill) I read that there's no "default port for LMTP", but how can I adapt the query in such a way that the default port 24 is being used ONLY FOR LMTP? I'm using (snippet): ... NULL AS password, \ 'Y' AS nopassword, \ 'Y' AS proxy, 'dovecot.charite.de' AS host, 'any-cert' AS 'ssl', \ ... -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

3 3

Empty mailboxes
by Daniel Miller 24 Sep '14

24 Sep '14

This is driving me nuts. I need to rebuild a server - got a notification the main drive is failing, and it's been a year or two, so it's time. Got a new drive, fresh install of Ubuntu Tantric, started installing everything needed. Data stored on a RAID array - nicely available separate from the boot/operation drive. Did a fresh compile of Dovecot, copied over the configs from the old drive, mounted the RAID...and... Dovecot starts, authenticates users - and shows empty mailboxes. Nothing shown. Full panic - reboot on old drive. All mails present. Change back - no mail. Executing "doveadm user -u username" seems to give valid results. Mailbox location appears correct (using mdbox). I setup uid/gid the same - I thought it was a rights issue but everything looks correct. What can I do to try to find the stupid mistake I'm making? -- Daniel

3 4

quota recalc problem for big maildirs
by Dawid Stawiarski - nazwa.pl 24 Sep '14

24 Sep '14

hi, we use dovecot 2.2.13, and we've lately started using maildir quota (previously fs quota was used, right now it's "quota = maildir:User quota" changed from "quota = fs:User quota:user:inode_per_mail"). When the change was applied we didn't manually forced generating maildirsize files. So, when the first deliver happens (or the file grows larger then 5KB) it starts quota recalc. However for users with big maildirs, receiving email every couple of seconds, there are tens of delivery processes each doing its own recalc (as they open file for APPEND and there's no locking). What's even worse, after all the stats and getdents (it takes a lot of IO for 60GB account with 2M emails), deliver makes stat on 'new' dir and detects changes after which it unlinks freshly created maildirsize. So possibly this will never be finished as every process drops its own work. The same thing happened when we checked the native dovecot LMTP server (every connection has its own forked process - each running independently of the others). Is it a known unwanted "feature" of maildir quota (and we should use dict quota instead) or a bug that will be fixed? cheers, Dawid Stawiarski

2 2

Re: director
by Eduardo Ramos 23 Sep '14

23 Sep '14

The configuration use to be in /etc/dovecot, and the working dir is /var/run/dovecot. You should store your configuration file in the /etc/dovecot. The dovecot.conf file can be copied to dovecot-backend.conf and dovecot-director, for example, and you will use '-c' argument of dovecot binary to load on or another. If you do this, you will need create two separated working dir, that will be defined in base_dir on each configuration file. On 09/23/2014 05:39 PM, jjhoffart wrote: > Thanks for the replies, > I am still fairly new to dovecot and still learning. This is a quick run down of my setup: 3 servers running iRedMail (this is a package that installs dovecot 2.1.7, postfix, and LDAP). All three servers share an NFS backend. The same 3 servers are also sitting behind a load balancer. What I want to do is use these three servers as directors and mail servers. So from the replies I have gotten I understand that I need to create 2 config files 1 for director and 1 for dovecot. Now my next question is do these config files need to be stored in /etc/dovecot or /var/run/dovecot? Also is creating the second config file as simple as copying and renaming the dovecot.conf file? > > On Sep 23, 2014, at 2:26 PM, Eduardo Ramos <eduardo(a)freedominterface.org> wrote: > >> Hi, >> >> In fact you don't NEED a director. You can use two dovecot servers sharing a NFS storage with a normal (backend) implementation. It's good to see that page: http://wiki2.dovecot.org/NFS for some good recomendations. >> >> You must have a base_dir for each instance of dovecot you want to run, and a dovecot.conf as well. One could be director and other backend. I suggest you have two separated configuration directories, /etc/dovecot-director and /etc/dovecot-backend. Each with the respective configuration. Start dovecot with "dovecot -c /etc/dovecot-director/dovecot.conf". Backend is the same. >> >> In order to help more, provide us more details about your environment and doulbts. >> >> On 09/23/2014 03:57 PM, jjhoffart wrote: >>> Hello, >>> I am currently trying to setup a cluster of mail servers. I currently have two servers I am trying to cluster together. These servers will also have a shared NFS backend. I have read on the site that I will need to configure director since I am using NFS. I have read through all of the documentation on director but still need a little guidance to get things up and running. One of the questions I have is about the base_dir on the Multiple Invocations section of the wiki. It states that I will need to change the bas_dir to the new run directory, not sure what needs to happen here, can anyone help me out?

1 0

director
by jjhoffart 23 Sep '14

23 Sep '14

Hello, I am currently trying to setup a cluster of mail servers. I currently have two servers I am trying to cluster together. These servers will also have a shared NFS backend. I have read on the site that I will need to configure director since I am using NFS. I have read through all of the documentation on director but still need a little guidance to get things up and running. One of the questions I have is about the base_dir on the Multiple Invocations section of the wiki. It states that I will need to change the bas_dir to the new run directory, not sure what needs to happen here, can anyone help me out?

3 2