- dovecot - dovecot.org

[Dovecot] Help needed with SOLR integration
by Nigel Smith 05 Aug '13

05 Aug '13

Hi, Is there anything wrong with this config for getting SOLR working ? I'm on Ubuntu 12.04LTS and using the bundled Jetty and SOLR. Calling "http://localhost:8080/solr/update?optimize=true" doesn't return an error. And I'm pretty sure I've replaced the schema ? Help & thanks in advance Nigel # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.5.0-37-generic x86_64 Ubuntu 12.04.2 LTS mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii -numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { fts = solr fts_solr = url=http://localhost:8080/solr/ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = its-virtmail user = its-virtmail } } service imap-login { process_min_avail = 3 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = its-virtmail } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 3 } ssl = required ssl_cert = </etc/ssl/certs/mx_example_com.pem ssl_cipher_list = TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!3DES:!PSK:@STRENGTH ssl_key = </etc/ssl/private/mx_example_com.key userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 10 mail_plugins = fts fts_solr } protocol pop3 { mail_max_userip_connections = 10 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address = postmaster quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r } protocol lmtp { mail_plugins = sieve postmaster_address = bp(a)example.com }

1 0

[Dovecot] Weird log entries when some users are composing messages in Thunderbird
by Charles Marcus 05 Aug '13

05 Aug '13

Hi all/Timo, When some of our users are composing a message in Thunderbird, I see a ton of these in the logs, and they keep repeating the entire time they are typing until the message gets sent... > 2013-08-05T10:34:42-04:00 myhost dovecot: imap-login: Login: user=<user(a)media-brokers.com>, method=PLAIN, rip=###.###.###.###, lport=993, mpid=8152, TLS, session=<El2GNTTjNgDAqAIc> > 2013-08-05T10:34:42-04:00 myhost dovecot: imap(user(a)media-brokers.com): Disconnected: Disconnected in IDLE in=149 out=12449 > 2013-08-05T10:34:42-04:00 myhost dovecot: imap-login: Login: user=<user(a)media-brokers.com>, method=PLAIN, rip=###.###.###.###, lport=993, mpid=8153, TLS, session=<lJedNTTjNwDAqAIc> > 2013-08-05T10:34:42-04:00 myhost dovecot: imap(user(a)media-brokers.com): Disconnected: Disconnected in IDLE in=149 out=12449 I confirmed that their 'Auto-Save' settings are the defaults (5 minutes, same as mine), and when I'm composing a message, there are no entries like this. The only other option I could think of that might affect t his is the 'spell check as you type', which is also enabled on mine. Also - I'm not sure this *always* happens to these users... all I know is, I see these flooding the logs sometimes, and when I remote in, that user is composing a message. Anyone have any idea why this happens to only a few of our users, and possibly only sometimes? Could it be something in the HTML of the quoted text when they are replying to certain messages? Thanks, -- Best regards, Charles

2 4

Re: [Dovecot] problems with doveadm
by listserv 05 Aug '13

05 Aug '13

> > On 3.8.2013, at 14.25, listserv <listserv(a)xtlv.cn> wrote: > >> doveadm expunge -A mailbox Trash savedbefore 7d >> => kernel: doveadm[11609]: segfault at 10 ip b75f146a sp bf856ad0 error 4 in >> libdovecot.so.0.0.0[b7585000+c8000] > > gdb backtrace would be useful, for example: > > gdb --args doveadm expunge -A mailbox Trash savedbefore 7d > run > bt full > Thank you for your advice. Here the backtrace: gdb --args doveadm expunge -A mailbox Trash savedbefore 7d GNU gdb (GDB) 7.6 (Debian 7.6-5) Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i486-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/bin/doveadm...Reading symbols from /usr/lib/debug/usr/bin/doveadm...done. done. (gdb) run Starting program: /usr/bin/doveadm expunge -A mailbox Trash savedbefore 7d warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. i_stream_next_line (stream=0x0) at istream.c:382 382 istream.c: Datei oder Verzeichnis nicht gefunden. (gdb) bt full #0 i_stream_next_line (stream=0x0) at istream.c:382 _stream = <optimized out> pos = <optimized out> #1 0xb7dfbd01 in auth_master_user_list_next (ctx=0x80bf668) at auth-master.c:694 line = <optimized out> #2 0xb7ef72cb in mail_storage_service_all_next (ctx=0x80b8f60, username_r=0xbffff550) at mail-storage-service.c:1294 __FUNCTION__ = "mail_storage_service_all_next" #3 0x08057db5 in doveadm_mail_all_users (wildcard_user=0x0, argv=<optimized out>, ctx=0x80b53f0) at doveadm-mail.c:368 ret = <optimized out> input = {module = 0x0, service = 0x8092f63 "doveadm", username = 0x80bf070 "admin(a)xtlv.jp", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0} user_idx = 0 user = 0x80bf070 "admin(a)xtlv.jp" error = 0xb7e6f2f0 "" #4 doveadm_mail_cmd (argv=<optimized out>, argc=<optimized out>, cmd=0xbffff5ac) at doveadm-mail.c:523 ctx = 0x80b53f0 getopt_args = <optimized out> ret = <optimized out> c = <optimized out> wildcard_user = 0x0 error = <optimized out> #5 doveadm_mail_try_run (cmd_name=cmd_name@entry=0x80b1209 "expunge", argc=argc@entry=6, argv=argv@entry=0x80b11dc) at doveadm-mail.c:608 cmd = 0xbffff5ac #6 0x08056b26 in main (argc=6, argv=0x80b11dc) at doveadm.c:398 cmd_name = <optimized out> i = <optimized out> quick_init = false c = <optimized out> -- 中華人民共和國

1 0

[Dovecot] dovecot fts solr plugin
by Thomas Baumann 05 Aug '13

05 Aug '13

Hello list, * apache-solr-3.6.2 * dovecot-2.1.7 I configured fts_solr for dovecot, but I have two issues When doing a search request within telnet 127.0.0.1 143 it connects to solr. Jul 21, 2013 4:38:15 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={rows=1&q=box:10361d30f8c4ea51ad7c00002ecaff3b+user:" tom(a)tirism.support.tiri.li"&sort=uid+desc&fl=uid} hits=0 status=0 QTime=10 Jul 21, 2013 4:38:15 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={rows=3&q=(hdr:"test"+OR+body:"test")&fq=%2Bbox:10361d30f8c4ea51ad7c00002ecaff3b+%2Buser:" tom(a)tirism.support.tiri.li"&sort=uid+asc&fl=uid,score} hits=0 status=0 QTime=7 But (1) doveadm fts fails: $ doveadm fts rescan -u tom(a)domain.com Error message: doveadm(tom(a)tirism.support.tiri.li): Error: fts not enabled for user's namespace (null) (2) solr schema has errors Jul 21, 2013 4:35:46 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1330) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getAnalyzer(IndexSchema.java:408) at org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:383) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:574) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:206) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1436) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1319) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1245) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1234) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206) at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:79) at org.apache.solr.search.QParser.getQuery(QParser.java:143) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:105) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:59) at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1182) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) at java.lang.Thread.run(Thread.java:736) Hope somebody can help! Best regards, Thomas # dovecot --version 2.1.7 # dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.0.74-0.6.10-default x86_64 SUSE Linux Enterprise Server 11 (x86_64) xfs base_dir = /var/run/dovecot/ dict { expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_uid = 5000 listen = * login_greeting = tiri Mail ready. login_trusted_networks = 127.0.0.1 mail_location = maildir:/var/vmail/%u/ mail_plugins = fts fts_solr managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } } protocol pop3 { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } }

3 4

[Dovecot] Design: Adding checksums to index files
by Timo Sirainen 05 Aug '13

05 Aug '13

I've been planning on adding these for years. Maybe it's about time soon. I guess they could be added already to v2.2, but enabled only by a new setting because it requires file format changes that old Dovecots can't then read. I could probably patch v2.1 also so it is able to at least read the new format without failing. For v2.3 this new format could then be made the default. And what would the checksums be exactly? Would the standard CRC32 and CRC8 work fine, or are there any better ones? 1. dovecot.index v2.1+ always only fully recreates this file, never overwrites data to it. So the checksums could be written only when the dovecot.index is being recreated. There are 3 possible things to checksum: - header (32bit checksum) - all of the mail records (32bit checksum) - each mail record independently (8bit checksum per mail) The header's checksum could be verified every time the index is opened. The full mail record checksum could be verified when something appears to be wrong, but it's probably a waste of time to check it in normal operation. I'm not really sure about the per-mail checksums. It would be easy to create them while dovecot.index is being created, but after reading the file into memory the records are updated in many ways in many places. It's probably not worth the complexity and extra slowness to verify and/or update the checksums in all the different places. So is it worth it to even have them? In error conditions when fixing up indexes it could be useful to skip over records with broken checksums (and check if the mail is in dovecot.index.backup with correct checksum). Maybe that's enough to be worth 1 byte per message?.. 2. dovecot.index.log This file is only appended to. Each committed transaction could be prefixed in the new format with <transaction size><transaction 32bit checksum>. With the new format this wouldn't actually increase the log file size much, because there is already some space wasted for a compatibility "boundary" record that could be removed now. 3. dovecot.index.cache Cache file is the most complex file. Its headers get overwritten once in a while. Probably not worth the trouble to checksum the header itself, and there's not a lot that could be done even if a broken checksum was found. But each mail_cache_record could have its own checksum. A 8bit checksum could be added without increasing the file's size. Maybe that would be enough? 4. dovecot.index.thread This is a rather simple file and a 32bit checksum could be added to its header, and verified every time the file is read (because it's fully read anyway). 5. dovecot.mailbox.log This file doesn't even have a header. There are 3 unused bytes in each record currently. One of them could be used for a new "flags" parameter, with the only flag being "checksum added". There would still be space left for 8bit or 16bit checksum. 6. Other files There are also some text files, like dovecot-acl, subscriptions, quota usage and Sieve scripts. They probably have to be without checksums for now.

2 2

[Dovecot] How to configure sendmail using dovecot lmtp
by alfdc 05 Aug '13

05 Aug '13

Hi, I can't seems to find any posting on how to configure sendmail to use dovecot as lmtp. All I see is using it as LDA. I am particularly interested on how to tell sendmail to ask dovecot to deliver the message to the user. In the wiki for lda it says to add the following in the sendmail config: FEATURE(`local_procmail', `/usr/local/libexec/dovecot/deliver',`/usr/local/libexec/dovecot/deliver -d $u') MODIFY_MAILER_FLAGS(`LOCAL', `-f') MAILER(procmail) Is this the same for lmtp? If so how does it know when to use lda or lmtp? Beside, is procmail playing any role in any of these? Can the I just replace the whole thing as: FEATURE(`local_lmtp',`/usr/local/libexec/dovecot/deliver',`/usr/local/libexec/dovecot/deliver -d $u) MAILER(`local') Thanks. -- View this message in context: http://dovecot.2317879.n4.nabble.com/How-to-configure-sendmail-using-doveco… Sent from the Dovecot mailing list archive at Nabble.com.

3 2

[Dovecot] Maildir Synchronization warnings
by Kavish Karkera 05 Aug '13

05 Aug '13

Hi, We are repeatedly getting these below warnings for some of our users, al though we have no complaints from them yet, we need to know why these warning occurs. So it would be help full if some one explain these warning msg in detail. ----------------------------------------------------------------------------------------------------------------------------------------------------- Aug 2 12:52:55 blade8 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 94 seconds (23191 readdir()s, 0 rename()s to cur/, why=0x1) Aug 2 12:52:55 blade8 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir: Synchronization took 94 seconds (0 new msgs, 0 flag change attempts, 0 expunge attempts) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 154 seconds (43129 readdir()s, 0 rename()s to cur/, why=0x1) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 162 seconds (43129 readdir()s, 0 rename()s to cur/, why=0xc) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir: Synchronization took 162 seconds (0 new msgs, 0 flag change attempts, 0 expunge attempts) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Locking transaction log file /indexes//mail/v3store/example.com/kavish.karkera(a)example.com/.INBOX/dovecot.index.log took 50 seconds ----------------------------------------------------------------------------------------------------------------------------------------------------- Thanks In advance, Regards, Kavish Karkera

3 3

[Dovecot] Antispam folder names
by Jost Krieger 05 Aug '13

05 Aug '13

Is there anywhere a collection of folder names for the antispam_trash and antispam_spam configuration? Our problem is that in academia, you have a vast amount of different clients, and some, like Microsoft, use language-specific folder names *on the server*. So we're now collecting folder names to better handle the rare cases. Two more problems: 1. We needed to fix UTF-8 handling in the plugin, so that we kow canb say antispam_trash_pattern_ignorecase = gelöscht* in place of :set sp antispam_trash_pattern_ignorecase = gel&APY-scht* 2. Some clients allow prefixing the IMAP folders with a fixed name, often "INBOX". So we would like to specify something like antispam_trash_pattern_ignorecase = Junk*;*/Junk* which is not supported. Working on a patch ... Yours Jost -- | Helft Spam ausrotten! HTML in Mail ist unhöflich. | | Postmaster, JAPH, manchmal Wahrsager am RZ der RUB | | Wahre Worte sind nicht gefällig, gefällige Worte sind nicht wahr.| | Lao Tse, Tao Te King 81 |

2 1

[Dovecot] Fileoperations in Maildir – problematic or okay?
by Martin Burgraf 05 Aug '13

05 Aug '13

Hi, I'm wondering: Is it a problem to move and delete files inside a dovecot-managed maildir? For example: I have a folder ~/.maildir/.Sub1.Start/cur and a folder ~/.maildir/.Sub1.Start.old/cur and like to regularly move old mails from the first to the second one. Can you create a cron job saying something like: find ~/.maildir/.Sub1.Start/cur -mtime +5 -exec mv {} ~/.maildir/.Sub1.Start.old/cur \; ? I guess this would be some sort of problem, wouldn't it? IIRC dovecot keeps indexes of the files inside a folder and moving them around like this may confuse the software, am I right? bye Martin

3 2

[Dovecot] ./configure dovecot
by LuKreme 03 Aug '13

03 Aug '13

When I do a ./configure in dovecot (2.2.1) I get this notice (first line): configure: WARNING: you should use --build, --host, --target Then, much later on: checking sys/ucred.h presence... yes configure: WARNING: sys/ucred.h: present but cannot be compiled configure: WARNING: sys/ucred.h: check for missing prerequisite headers? configure: WARNING: sys/ucred.h: see the Autoconf documentation configure: WARNING: sys/ucred.h: section "Present But Cannot Be Compiled" configure: WARNING: sys/ucred.h: proceeding with the compiler's result configure: WARNING: ## ---------------------------------- ## configure: WARNING: ## Report this to dovecot(a)dovecot.org ## configure: WARNING: ## ---------------------------------- ## checking for sys/ucred.h... no at the end I get: configure: creating ./config.status Install prefix . : /usr/local File offsets ... : 64bit I/O polling .... : kqueue I/O notifys .... : kqueue SSL ............ : yes (OpenSSL) GSSAPI ......... : no passdbs ........ : static passwd passwd-file pam checkpassword : -shadow -bsdauth -sia -ldap -sql -vpopmail userdbs ........ : static prefetch passwd passwd-file checkpassword nss : -ldap -sql -vpopmail SQL drivers .... : : -pgsql -mysql -sqlite Full text search : squat : -lucene -solr but there is no makefile created. ./configure --help just runs the configure script again, and none of --build --host or --target have any effect on the errors shown, including the first line error. So I think, ok, fine, I'll grab the latest build (2.2.4). Same result. OK, fine. Let's look at 1.2.17. Same thing, only without the cured.h warning. Still the same error on the first line after ./configure and still no make file. There's no big flashing warning anywhere that I can see that says, "Hey, dummy, this isn't going to work." config.log even ends with a "configure: exit 0" (The system already has dovecot 1.2.17 installed, but I was looking at installing $LATEST). -- May you live in interesting times

2 9