- dovecot - dovecot.org

[Dovecot] fts solr : out of memory
by tonio＠starbridge.org 31 Jul '12

31 Jul '12

Hi Timo, since I've upgraded to 2.1.8, solr indexer doesnt work correctly. I see a lot of these errors in logs: Fatal: master: service(indexer-worker): child 9582 returned error 83 (Out of memory (service indexer-worker { vsz_limit=256 MB }, you may need to increase it)) I've tried to set vsz_limit to a lot more, (around 4096 MB) but same problem occurs. thank for your help ! Regards Tonio Java(TM) SE Runtime Environment (build 1.6.0_31-b04) doveconf -n # 2.1.8: /etc/dovecot/dovecot.conf # OS: Linux 3.2.13-xxxx-std-ipv6-64 x86_64 Debian wheezy/sid xfs auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_mechanisms = plain login dict { quota = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf } hostname = maildir2.spamguard.fr log_timestamp = %Y-%m-%d %H:%M:%S mail_gid = vmail mail_location = sdbox:/home/virtual/%u mail_plugins = quota fts fts_solr acl zlib mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = / type = private } namespace { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX/ separator = / type = private } namespace { list = children location = sdbox:/home/virtual/%%u:INDEX=/home/virtual/%u/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = file:/home/virtual/shared-%d-mailboxes.db autocreate = Spam autocreate2 = SpamFalse autocreate3 = SpamToLearn autocreate4 = Sent autosubscribe = Spam autosubscribe2 = SpamFalse autosubscribe3 = SpamToLearn autosubscribe4 = Sent fts = solr fts_solr = url=http://localhost:8081/solr/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename save mailbox_create mail_log_fields = uid box msgid size quota = dict:User quota::proxy::quota quota2 = dict:User quota::file:/home/virtual/%u/dovecot-quota quota_warning = storage=85%% quota-warning 85 %u sieve = ~/dovecot.sieve sieve_before = /home/virtual/sieve/ sieve_dir = ~/sieve zlib_save = bz2 zlib_save_level = 9 } protocols = imap pop3 sieve lmtp service auth { unix_listener /var/spool/postfix-in2/private/auth { group = mail mode = 0660 user = postfix } unix_listener /var/spool/postfix-maildir2/private/auth { group = mail mode = 0660 user = postfix } unix_listener /var/spool/postfix-msa/private/auth { group = mail mode = 0660 user = postfix } unix_listener /var/spool/postfix-smtp06/private/auth { group = mail mode = 0660 user = postfix } unix_listener /var/spool/postfix/private/auth { group = mail mode = 0660 user = postfix } unix_listener auth-in2-master { mode = 0660 user = vmail } unix_listener auth-maildir2-master { mode = 0660 user = vmail } unix_listener auth-master { mode = 0660 user = vmail } unix_listener auth-msa-master { mode = 0660 user = vmail } unix_listener auth-out-master { mode = 0660 user = vmail } unix_listener auth-smtp06-master { mode = 0660 user = vmail } unix_listener auth-userdb { mode = 0660 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } process_limit = 256 process_min_avail = 4 service_count = 0 vsz_limit = 256 M } service lmtp { inet_listener lmtp { address = 46.105.108.210 port = 24 } user = vmail vsz_limit = 368 M } service managesieve-login { inet_listener sieve { port = 4190 } process_limit = 256 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 } process_limit = 256 process_min_avail = 4 service_count = 0 vsz_limit = 256 M } service quota-warning { executable = script /usr/bin/quota-warning.sh unix_listener quota-warning { mode = 0600 user = vmail } user = vmail } ssl = required ssl_ca = </etc/dovecot/ca-bundle.crt ssl_cert = </etc/dovecot/mx2.eole-its.com.certchained.pem ssl_key = </etc/postfix/tls/mx2.eole-its.com.key.pem ssl_verify_client_cert = yes userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { imap_client_workarounds = delay-newmail imap_max_line_length = 64 k mail_max_userip_connections = 60 mail_plugins = quota imap_quota acl imap_acl autocreate mail_log notify fts fts_solr zlib } protocol pop3 { mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol sieve { managesieve_logout_format = bytes ( in=%i : out=%o ) } protocol lda { info_log_path = log_path = mail_plugins = sieve quota autocreate zlib postmaster_address = postmaster(a)spamguard.fr quota_full_tempfail = yes } protocol lmtp { info_log_path = log_path = mail_plugins = sieve quota autocreate fts fts_solr zlib postmaster_address = postmaster(a)spamguard.fr quota_full_tempfail = yes } local 46.105.108.210 { protocol imap { ssl = required ssl_cert = </etc/dovecot/imap.spamguard.fr.certchained2.pem ssl_key = </etc/dovecot/imap.spamguard.fr.key.pem } } local 87.98.168.xx { protocol imap { ssl = required ssl_cert = </etc/dovecot/mail.ovxx.fr.certchained.pem ssl_key = </etc/dovecot/mail.ovxx.fr.key.pem } }

3 7

[Dovecot] FETCH Error
by Micha Krause 31 Jul '12

31 Jul '12

Hi, I have the following problem: Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: FETCH [] for mailbox INBOX.Sent UID 166 got too little data: 120860 vs 120867 Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: Maildir filename has wrong W value: /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003,W=284007:2,S Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: Corrupted index cache file /var/mail/mailboxes/users/micha/.Sent/dovecot.index.cache: Broken virtual size for mail UID 166 so far i tried deleting: /var/mail/mailboxes/users/micha/.Sent/dovecot.index.cache /var/mail/mailboxes/users/micha/.Sent/dovecot.index /var/mail/mailboxes/users/micha/.Sent/dovecot.index.log I tried to renaming: /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003,W=284007:2,S to /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003:2,S I tried doveadm: doveadm force-resync -u micha INBOX.Sent Dovecot version is: 2.0.9 Micha Krause

2 1

[Dovecot] mdbox with separate index location results in many emptydirectories
by Jürgen Obermann 31 Jul '12

31 Jul '12

Hallo, I use dovecot 2.1.7 with mdbox format and I specified a separate location for mailbox index files with mail_location = mdbox:/dovecot/storage%h:INDEX=/dovecot/index%h With this setting I get for every mailbox a directory under both filesystems /dovecot/index and /dovecot/storage, e.g. for the inbox it looks like that: # ls -lR /dovecot/index/home/hrz/myuser/mailboxes/INBOX /dovecot/storage/home/hrz/myuser/mailboxes/INBOX /dovecot/index/home/hrz/myuser/mailboxes/INBOX: total 49 -rw------- 1 myuser hrz 20480 Jul 20 11:12 dovecot.index.cache -rw------- 1 myuser hrz 1280 Jul 20 08:00 dovecot.index.log -rw------- 1 myuser hrz 301 Jul 19 16:45 dovecot.index.search -rw------- 1 myuser hrz 20 Jul 19 16:45 dovecot.index.search.uids /dovecot/storage/home/hrz/myuser/mailboxes/INBOX: total 3 drwx------ 2 myuser hrz 2 Jul 19 15:20 dbox-Mails /dovecot/storage/home/hrz/myuser/mailboxes/INBOX/dbox-Mails: total 0 As expected the index files now reside under /dovecot/index%h/mailboxes/, but the path /dovecot/storage%h/mailboxes/ also contains for every user and every mailbox name a directory which in turn contains only an empty subdirectory 'dbox-Mails'. These directories seem to me to sum up to a lot of unneded infrstructure objects. I tried to delete these empty directories, but dovecot did not like that and stopped to work. So somehow they are necessary, in spite of setting a separate index file location. So my question is, could these empty directories be avoided, perhaps in a future version? Greetings, Juergen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann(a)hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009

2 1

[Dovecot] improve error messages with quotes
by Radim Kolar 31 Jul '12

31 Jul '12

I propose to improve error messages but inserting quotes. I spent good time until I looked into source code to find right meaning of error message. I would like them in following format: namespace configuration error no 'hidden=no' namespaces

2 1

[Dovecot] Empty argument in "executable = script ..." service
by Florian Wagner 31 Jul '12

31 Jul '12

Hi, what is the point of adding an empty argument to the command called by the "executable = script ..." setting in a service? I guess that would be the call to array_append_space at line 57 in util/script.c. Background is that I'm trying to run bogofilter through the extprograms Pigeonhole plugin. So I've defined a service entry with a unix_listener and wondered why my test shell script works but bogofilter exits with a cryptic error. Turns out that this empty argument at the end trips up the bogofilter argument parsing code. If I manually call bogofilter with execv and add that empty argument I get the same cryptic error. Regards Florian

2 1

[Dovecot] Dovecot Director behaviour when two or more fail
by Dominic Malolepszy 31 Jul '12

31 Jul '12

Hi, On the Dovecot wiki here http://wiki2.dovecot.org/Director it says the following: "The directors are going to connect to each others in a ring. For example if you have servers called A, B and C, director will create connections A->B, B->C and C->A" I am curious what happens in a scenario when two or more directors fail and potentially break a ring in half, do the active directors detect this and reform a new ring? Dominic

2 1

[Dovecot] Help needed with configuration giving dict error No such file
by Ajax 31 Jul '12

31 Jul '12

Each time a mail message is delivered to a mailbox records like the following appear in /var/log/dovecot.warn: Jul 29 10:02:05 iris dovecot: lmtp(29320, jax(a)XXX.com) Error: file dict commit: file_dotlock_open(/var/vmail/XXX.com/jax/home/maildir/dovecot-quota) failed: No such file or directory Jul 29 10:02:05 iris dovecot: lmtp(29320, jax(a)XXX.com) Error: file dict commit: file_dotlock_open(/var/vmail/XXX.com/jax/home/maildir/dovecot-quota) failed: No such file or directory Jul 29 10:02:05 iris dovecot: lmtp(29320, jax(a)XXX.com) Error: dict quota: Quota update failed, it's now desynced This is with Dovecot 2.1.7 installed by Debian Squeeze binary packages, dovecot-core v 1:2.1.7-2~bpo60+1, and friends. The host is running GNU/Linux 3.2.0-0.bpo.2-amd64 and on a xfs filesystem. Entries in /etc/dovecot/users look like this: jax@XXX.com:{plain}password The output from 'doveconf -n' is in the attachment to this mail message. I'm a beginner without experience with either IMAP servers or clients. Can you give me pointers to see what I'm missing that causes the errors logged as above? The user data base looks like this: userdb { args = uid=5000 gid=5000 home=/var/vmail/%d/%n/home mail=maildir:/var/vmail/%d/%n driver = static } Being a beginner, I wonder if is necessary to manually make home directories for each '/var/vmail/%d/%n/home'? I've tried test accounts with and without empty "home" directories at the indicated locations without seeing any difference in the error indications. Is it necessary to manually populate the "home" directories? Besides having no clue about the errors logged, I don't understand what it takes to make quotas work nor do I understand what options are available or advisable. The distribution wiki/QuotaMaildir.txt seems clear that the Maildif+++ backend is indicated but beyond that I remain in the dark. FWIW, I've yet to try a real IMAP client. Thank you for any pointers you can give me.

2 1

[Dovecot] Problem with running two dovecot instances on the same server
by Alexandr Sabitov 31 Jul '12

31 Jul '12

Hello All, Have anyone met the problem I am struggling with at the moment ? The problem is when I run two instances of dovecot 2.1.8 which are one for proxy-director and another one is a regular one on the same server then I have big amount of strange entries in the /var/log/mail.err: Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs Jul 26 11:52:20 dovecot-test-1 dovecot: director: Error: Empty server list Jul 26 11:52:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting Jul 26 11:52:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs That's for sure set->director_mail_servers variable in main_preinit of ./src/director/main.c But after dumping it it turns out that ppid of the process which failed to use main_preinit function is a "regular" dovecot not the proxy one. When main_preinit is used by dovecot-proxy then set->director_mail_servers = ip address of a mail server and it is all good but if main_preinit is used by dovecot then set->director_mail_servers = empty and I get fatal errors in logs. So here is a question: How come that "non-proxy" dovecot requests director_mail_servers variable and writes fatal error in the logs if it doesn't find it ? Thank you very much for any assistance! Please find my configs: DOVECOT: root@dovecot-test-1:~/software/dovecot/dovecot-2.1.8# /usr/local/dovecot/sbin/dovecot -n # 2.1.8: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_mechanisms = plain digest-md5 cram-md5 apop auth_verbose = yes base_dir = /var/run/dovecot/ default_internal_user = webmail disable_plaintext_auth = no doveadm_password = secret listen = 192.168.0.42 mail_fsync = always mail_location = maildir:~/ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " quota fts fts_solr" mmap_disable = yes passdb { args = /usr/local/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8080/solr/ quota = maildir:User quota } service doveadm { inet_listener { port = 8282 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap { executable = imap } service lmtp { inet_listener lmtp { port = 24 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { executable = pop3 } ssl_cert = </usr/local/dovecot/ssl/dovecot.pem ssl_key = </usr/local/dovecot/ssl/dovecot.pem userdb { args = /usr/local/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol imap { mail_plugins = " quota fts fts_solr imap_quota" } DOVECOT-PROXY: root@dovecot-test-1:~/software/dovecot/dovecot-2.1.8# /usr/local/dovecot/sbin/dovecot -c /usr/local/dovecot/etc/dovecot/dovecot-proxy.conf -n # 2.1.8: /usr/local/dovecot/etc/dovecot/dovecot-proxy.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_verbose = yes base_dir = /var/run/dovecot-proxy default_internal_user = webmail director_mail_servers = 192.168.0.42 director_servers = 192.168.0.41 disable_plaintext_auth = no doveadm_password = secret doveadm_proxy_port = 8282 instance_name = dovecot-proxy listen = 192.168.0.41 login_greeting = Dovecot Proxy ready. mail_location = maildir:~/ passdb { args = proxy=y nopassword=y driver = static } service auth-worker { user = webmail } service auth { client_limit = 2400 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 8181 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { inet_listener { port = 8282 } } service imap-login { executable = imap-login director inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap { service_count = 0 } service lmtp { inet_listener lmtp { port = 24 } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { service_count = 0 } ssl_cert = </usr/local/dovecot/ssl/dovecot.pem ssl_key = </usr/local/dovecot/ssl/dovecot.pem userdb { args = /usr/local/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol imap { mail_max_userip_connections = 100 } protocol doveadm { auth_socket_path = director-userdb } -- Regards, Alexandr Sabitov

3 6

[Dovecot] maildir_copy_with_hardlinks on v.2.0.19
by mailing list subscriber 31 Jul '12

31 Jul '12

Hi, I'm trying to get the so-called "single instance store" (I think cyrus has got the name for the first time) with dovecot --version = 2.0.19 binary package installed from ubuntu 12.04 lts official repo. I have checked that "maildir_copy_with_hardlinks" is enabled ("dovecot -a|grep hard" shows "yes") then I have installed and enabled the lmtp component of dovecot. The configuration "dovecot -n" is pasted here: http://paste.lug.ro/131180 Also in the same paste is a strace against dovecot and childrent showing evidence of the MTA delivering a single copy of the message via LMTP with multiple RCPT TO: headers. However when looking in the Maildir, I see the mail break down into three separate files instead of expected hardlinked files ("stat" and "ls" shows one single link count, inodes are different) Given the above data, what (am I | dovecot is) doing wrong? Please cc me if you need additional input when replying as I'm not subscribed to the list (I'll watch the thread online only) Many thanks in advance.

3 6

Re: [Dovecot] [GoLugTech] thunderbird
by Steve Litt 31 Jul '12

31 Jul '12

On Mon, 30 Jul 2012 23:43:49 -0400, Chris Clifton said: > Hi all, > > Do I read it correctly that thunderbird funding is being cut back by > mozilla? > > I'm not on a linux mail client yet, wondering what others are using in > ubuntu for mail client, I need multiple pop3 profiles, good html > support, fast indexing and searching, solid gpg support and all the > usual bells and whistles. Evolution seems to be in pretty widespread > use. Wondering what others like and use heavily. Hi Chris, I tried Evolution once about 10 years ago, and it was too complex and comprehensive for me to understand. The centerpiece of my emailing activity is a Dovecot IMAP server on my daily driver desktop computer. The way it works is that fetchmail grabs all my email from my various POP accounts, pushes them through procmail, whose filters distribute the emails into the proper Dovecot IMAP directories (and also /dev/null stuff from PITA people). As far as I know, my Dovecot has never malfunctioned. Currently, I access my local IMAP with Claws-Mail. Claws is capable, robust, and does what I need without being an ecological disaster like Kmail or intermittently pig slow like Thunderbird. Claws' configuration is difficult to wrap your head around, and very much in need of better documentation, but so far, thanks to a knowledgeable and helpful user community on the mailing list, I've been able to make almost every config change to speed my interaction with Claws. My one complaint about Claws that can't be cured by documentation is its propensity to do everything with one process and one thread, so while you're sending an email or scanning your IMAP, you cannot look at other emails. I also wanted their powerful search to be able to recurse folders, and when nobody volunteered to do it, I tried to do it myself. Making something recursive sounds like simply a matter of writing a simple loop around the existing functionality, but what I found out is, at least this part of Claws was written in a manner resembling MVC, with process and UI completely commingled. For that reason I was unable to make it recursive. I have Claws on my daily driver and all my laptops. Thanks to Kevin Korb's instructions, on the laptops I was able to tell Claws its IMAP server was at 127.0.0.1. I pinholed my OpenBSD/pf firewall appliance to port forward incoming ssh to my daily driver. Then, on my laptop, if I'm at home and on the LAN, I run the following inhouse.sh sudo ssh -NTL 993:127.0.0.1:993 slitt(a)192.168.1.88 While on the road I run this travelling.sh: sudo ssh -NTL 993:127.0.0.1:993 slitt(a)99.99.99.99 In the preceding, 99.99.99.99 is the IP address my ISP gave me, while 192.168.1.88 is the LAN IP address of my daily driver. This makes 127.0.0.1:993 get pushed through ssh to the ISP supplied internet address or the LAN address. So I can use Claws to interact with my daily driver desktop anywhere I go. I'm extremely pleased with this setup. My IMAP tree is trivial to back up, and gives me the assurance that if Claws-Mail ever commits suicide the way Kmail did when they went to Kmail2, I can simply plug in a different IMAP enabled email client. Meanwhile, Claws gives me an extremely productive and easy to use user interface. With my client/server setup, I no longer need to copy all my email too and from the laptop when travelling. I'm going to present on my remote use of my daily driver desktop Dovecot, using Claws and ssh, at tomorrow night's GoLUG meeting. HTH, SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance

2 1