- dovecot - dovecot.org

[Dovecot] authentication to IMAP
by punit_j 07 May '09

07 May '09

HI All, Is there a way to do admin authentication for a normal user in dovecot IMAP ? I have a user using PAM to store password in /etc/shadow file. I need to authenticate the user in IMAP using that password. I tried using : - I tried all combinations as below [root@mail ~]# cat /etc/shadow| grep -i ajay ajay.lal:CM730q8foSA9A:13545:0:99999:7::: [root@mail ~]# telnet 0 143 Trying 0.0.0.0... Connected to 0 (0.0.0.0). Escape character is '^]'. * OK dovecot ready. a1 login ajay.lal CM730q8foSA9A a1 NO Authentication failed. a1 login ajay.lal "{crypt}CM730q8foSA9A" a1 NO Authentication failed. Also i tried [root@mail ~]# perl -MMIME::Base64 -e 'print encode_base64("\0ajay.lal\0{crypt}CM730q8foSA9A");' AGFqYXkubGFsAHtjcnlwdH1DTTczMHE4Zm9TQTlB [root@mail ~]# telnet 0 143 Trying 0.0.0.0... Connected to 0 (0.0.0.0). Escape character is '^]'. * OK dovecot ready. a1 AUTHENTICATE PLAIN + AGFqYXkubGFsAHtjcnlwdH1DTTczMHE4Zm9TQTlB a1 NO Authentication failed. My dovecot server does PLAIN authentication as can auth_mechanisms = plain Is there a way to do PLAIN auth with crypt password or any means of doing admin authentication on behalf of normal user ? Regards, Punit

2 1

[Dovecot] Solr FTS issues (Was fts-solr plugin issue (Marked invalid))
by Dror Matalon 06 May '09

06 May '09

Hi, Sorry for the change of thread, I just signed up to the list so I couldn't reply to the earlier message. Let me clarify the issue that Nikolai was describing. We're running dovecot 1.1.11 and solr 1.4. The issue is quite simple. 1. I run a search. 2. Dovecot sends a list of emails to solr 3. Solr starts indexing them 4. Solr runs into a "bad" email and we get: SEVERE: java.io.IOException: Mark invalid org.apache.solr.common.SolrException log SEVERE: java.io.IOException: Mark invalid at java.io.BufferedReader.reset(BufferedReader.java:485) at org.apache.solr.analysis.HTMLStripReader.restoreState(HTMLStripReader.java:171) at org.apache.solr.analysis.HTMLStripReader.read(HTMLStripReader.java:728) at org.apache.solr.analysis.HTMLStripReader.read(HTMLStripReader.java:742) at java.io.Reader.read(Reader.java:123) at org.apache.lucene.analysis.CharTokenizer.next(CharTokenizer.java:109) at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:159) at org.apache.lucene.index.DocFieldConsumersPerField.processFields(DocFieldConsumersPerField.java:36) at org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocFieldProcessorPerThread.java:234) at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:765) ... 5. All the above email messages that were indexed are rolled back, and we're stuck. I think that the solution should be simple too :-). When solr runs into a bad email, it should just ignore it and keep indexing. This seems much more robust since the emails are coming from a variety of sources, and we can assume that some of them are going to generate badly formated emails. Seems like having a few bad emails not indexed is much better than the current situation of a bad email stopping all searching. I don't understand the architecture enough between fts, fts_solr and solr to know where this should be solved. Ideally, it would be a simple directive to solr. Regards, Dror ----- Dror Matalon President Zapatec Inc 866 522-7941 X 704 1700 MLK Way Berkeley, CA 94709 http://www.6zap.com http://twitter.com/drormata http://www.zapatec.com

1 0

[Dovecot] [OT] two-way IMAP synchronization
by Wolfram Schlich 06 May '09

06 May '09

Hi, I am looking for a tool that does a true two-way synchronization (synchronization of expunges/deletes) of IMAP mailboxes, kind of a mixture of unison and imapsync. Does anyone have any ideas how to accomplish that? TIA. -- Regards, Wolfram Schlich <wschlich(a)gentoo.org> Gentoo Linux * http://dev.gentoo.org/~wschlich/

2 3

Re: [Dovecot] Some POP3 questions
by Nikolai Derzhak 06 May '09

06 May '09

By the way, you can download ready to use open-source mail, calendar, contacts, etc: http://www.6zap.com. I'am sysadm in this project and we use dovecot to. On Wed, 05/06/2009 at 5:28pm, "Brandon Lamb" <brandonlamb(a)gmail.com> wrote: > On Wed, May 6, 2009 at 7:23 AM, Brandon Lamb <brandonlamb(a)gmail.com> wrote: >> On Wed, May 6, 2009 at 5:25 AM, Kenneth Kalmer <kenneth.kalmer(a)gmail.com> >> wrote: >>> On Tue, May 5, 2009 at 11:39 PM, Timo Sirainen <tss(a)iki.fi> wrote: >>> >>>> On Tue, 2009-05-05 at 23:32 +0200, Kenneth Kalmer wrote: >>>>>>> * NFS server which is the same server that houses single Courier-IMAP >>>>>>> installation (Gentoo, 3GB memory) >>>>>>> * 2x Postfix instances delivering to NFS >>>>>>> >>>>>> >>>>>> NFS may be problematic. Dovecot expects a perfectly working NFS setup, >>>>>> which seems to be a bit rare to find. http://wiki.dovecot.org/NFS >>>>>> >>>>> >>>>> Just for the record, dovecot will be access the mail directly on the >>>> disks, >>>>> it will be the LDA's on the postfix instances coming in over NFS. I'm >>>>> reading the NFS again to make sure I understand the risks >>>> >>>> If the LDA is Dovecot deliver, it's the same problem since it also >>>> updates index/control files that cause the problems. But if you're using >>>> something else that does nothing but write the maildir files, then it's >>>> fine. >>>> >>> >>> I'm looking to use the Dovecot LDA so the caches are updated and the >>> filenames are correct, to get good POP3 performance. >>> >>> I'm looking forward to giving Dovecot a go, but I'll rig up a staging >>> environment first. >>> >>> Thanks for the encouraging responses everyone. >>> >>> Best >>> >>> -- >>> Kenneth Kalmer >>> kenneth.kalmer(a)gmail.com >>> http://opensourcery.co.za >>> @kennethkalmer >> >> I moved from a 4 gig nfs server with eight mailheads running exim as >> lda, courier imap/pop3 (over nfs) to an 8 gig nfs server and moved to >> dovecot imap/pop3 running directly on the nfs server and dropped to >> four mailheads running exim with dovecot lda. >> >> At first I tried to keep my imap/pop3 services running on the 4 >> mailheads but over nfs had major weird 500 load spikes randomly, >> moving these to the nfs server solved all of that. The 4 mailheads >> still deliver to maildir over nfs using dovecot as lda. I have been >> pleasantly pleased. Our webmail app in our old setup used to take >> 10-30 seconds to load and now it is instantaneous. Some of it might be >> from upgrading hardware, but I definitely had a HUGE noticeable >> difference and you couldnt convince me to go back. >> >> We did have about a week of sucktastic traffic and my support >> department hating me because all our pop3 leave-messages-on-server >> users had to redownload all their mail, but it was worth the headache. >> Dovecot is just too sweet, with its performance and plugins to ever >> have to even think about a choice between it or software X. >> >> Just my $0.02 > > Oh yea, and I just recently discovered the virtual mailbox plugin and > fell even more in love. I am developing a Gmail clone inhouse for our > new webmail (we're an ISP). It is cool plugins like this that make me > glad I switched to Dovecot. Being able to consolidate various Deleted, > Trash, Deleted Items, Deleted Mails folders all into a single > virtual/Trash folder is GREAT. Powered by the 6zap. Sign up at http://www.6zap.com for an account that provides advanced e-mail, calendar and contacts capabilities.

1 0

[Dovecot] Some POP3 questions
by Kenneth Kalmer 06 May '09

06 May '09

Hi everyone Been spending the whole day on the wiki and in the list archives on using dovecot for POP3 mail. I'd like to present my findings and ask for some feedback. Our current software setup: * Courier-IMAP running POP3/IMAP * Postfix MTA * MySQL backend for user and login info As for the system layout, it looks like this: * NFS server which is the same server that houses single Courier-IMAP installation (Gentoo, 3GB memory) * 2x Postfix instances delivering to NFS * Mail directory structure: /mail/e/ex/example.com/user Usage: * Currently reaching in excess of 1,200 concurrent connections, traffic roughly 36GB per day * 90% POP3 (mostly pop+delete), 10% IMAP * ~170GB of mail on disk Current problems experienced: * Massive load averages (>500) * Memory exhaustion (3GB) * Server looses network connectivity (related to massive loads) I've got a new server ready for action, well after mail directories have been rsync'd over. Will have 8GB of memory and much fast discs. I'm looking at giving Dovecot + Postfix/LDA a go. From my research today it seems that running a POP server without LDA (or without the W=(\d+) filename field) is a futile business. I'm running Gentoo, and the stable release in portage is currently dovecot 1.1.7, with 1.1.14 being available but masked. Long stories short, will Dovecot with the Postfix/LDA give performance equal than Courier-IMAP or improved (because of the cached message sizes)? Kind regards -- Kenneth Kalmer kenneth.kalmer(a)gmail.com http://opensourcery.co.za @kennethkalmer

5 9

[Dovecot] fts-solr plugin issue (Marked invalid)
by Nikolai Derzhak 06 May '09

06 May '09

---------- Forwarded message ---------- From: "Nikolai Derzhak" <nikolai(a)6zap.com> To: "Timo Sirainen" <tss(a)iki.fi> CC: "Dovecot Mailing" <dovecot(a)dovecot.org> Date: 05/05/2009 3:01pm Subject: Re: [Dovecot] fts-solr plugin issue (Marked invalid) On Mon, 05/04/2009 at 6:23pm, "Timo Sirainen" <tss(a)iki.fi> wrote: > On May 4, 2009, at 10:16 AM, Nikolai Derzhak wrote: > >> OK. Concentrating problem in one question. >> How to ignore "bad" message and index next one in indexing procedure >> (fts plugin) ?. >> Now, one "error 500" from solr and dovecot (# 1.1.11: /etc/dovecot/ >> dovecot.conf >> # OS: Linux 2.6.21.7-2.fc8xen i686 Ubuntu 8.04.2 ext3 >> ) stop and each next search query repeat the story. > > Maybe simply using 1.1.14 would help? I already fixed one Solr issue: http:// > hg.dovecot.org/dovecot-1.1/rev/678c3252a454 > I've merge back this commit to 1.1.11 code, cause this is my request about special chars ;). > If that's not the problem, it would help to have one of those mails > that breaks it. Yes. Now i have two new issue: 1. "Content-Type:" header in mail wrong - so it's TEXT/mspowerpoint for example, but it's binary *.ppt file. 2. many html from repository change-set(many commit's to html) in text/plain mail In both cases we have "Error 500" from solr (HTMLStripReader class) (in solr log: "Marked inlalid"). And in both cases we can not detect this in dovecot code (or it's hard to do). So for now i need some method to ignore "bad" messages. Because we can not catch all variants, but indexing die on first error like this. Powered by the 6zap. Sign up at http://www.6zap.com for an account that provides advanced e-mail, calendar and contacts capabilities.

1 0

[Dovecot] NTLM configuration
by Cédric Laruelle 06 May '09

06 May '09

Hi all, Im actually running samba 3 as a primary domain controller. The workstations of my network are all running windows (some xp, some vista) and use Outlook 2007 as a web client. Samba is set up with the following parameters : security = user passdb backend = tdbsam unix password sync = Yes domain master = yes domain logons = yes Everything works fine on samba side, meaning that the machines are part of the domain, and users do authenticate correctly. For the moment, the users are getting their mail (pop3 or imap) through dovecot (running on the same box as samba). Dovecot is configured with mechanisms = plain. I would like to secure it by using mechanisms = ntlm, but still using the system users. I found much information on how to authenticate against an active directory, but nothing that allows me to keep my actual authentication against system users. Indeed, if I just change the mechanisms to ntlm, dovecot does not start anymore complaining that NTLM mechanism can't be supported with given passdbs Any help would be much appreciated as I already tried the whole day yesterday to get it working Best regards, Cédric Laruelle

2 2

[Dovecot] Testing POP3 with telnet - password always wrong
by Timothy Legg 06 May '09

06 May '09

Hello, I just installed the postfix and dovecot packages according to the document, http://www.debianadmin.com/debian-mail-server-setup-with-postfix-dovecot-sa… And it has been quite helpful. It looks like Dovecot works because I can communicate with it on port 110. I can't help but feel I am doing something stupid though... The web page listed above told me to test the POP3 server using Outlook Express, and well, I am just not going to do that. Not when I can do it by hand... legg@debian:~$ telnet 192.168.10.171 110 Trying 192.168.10.171... Connected to 192.168.10.171. Escape character is '^]'. +OK Dovecot ready. USER test(a)192.168.10.171 +OK PASS password1 -ERR Authentication failed. But why do I always get the password wrong? I twice set the password for the test user and also restarted Dovecot, but the password is always wrong. Note that disable_plaintext_auth=no. I'm sure there must be something stupid I have overlooked. Any suggestions? Thank you very much!!!! Tim Legg Here is some additional data that I am volunteering... # dovecot -n # 1.0.15: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: passdb: driver: pam userdb: driver: passwd # dovecot --version 1.0.15

3 2

[Dovecot] Unable To Send Mail
by Carlos Williams 05 May '09

05 May '09

I just installed CentOS 5 + Postfix + Dovecot. Now When I send mail via telnet from one user to another via Postfix (MTA), I have no problem. The recipient gets the message perfectly. Now this tells me that Postfix is working fine (I assume) but then I configured Dovecot to use IMAP4 and then configured my Thunderbird client. Now all the messages I previously sent using Telnet showed up in Thunderbird Inbox but then when I attempt to send a message via Thunderbird, the message appears to send fine however the recipient never gets it and I get no delivery failrures back of any kind. I check the Postfix (MTA) mail queue and it shows empty. What is causing this anyone? Does anyone know why this is not working? Here is my postconf -n for Postfix (MTA) [root@nat home]# postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases command_directory = /usr/sbin config_directory = /etc/postfix daemon_directory = /usr/libexec/postfix debug_peer_level = 2 home_mailbox = mail/ html_directory = no inet_interfaces = all mail_owner = postfix mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain mydomain = wtf.com myhostname = nat.wtf.com myorigin = $mydomain newaliases_path = /usr/bin/newaliases.postfix queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix-2.3.3/README_FILES sample_directory = /usr/share/doc/postfix-2.3.3/samples sendmail_path = /usr/sbin/sendmail.postfix setgid_group = postdrop smtpd_banner = $myhostname ESMTP $mail_name unknown_local_recipient_reject_code = 550 ================================================================ Here is what is 'UNCOMMENTED' in my /etc/dovecot.conf protocols = imap imaps mail_location = maildir:~/mail protocol imap { imap_client_workarounds = delay-newmail outlook-idle netscape-eoh } login_process_size = 64 protocol pop3 { protocol lda { protocol lda { postmaster_address = carlos(a)wtf.com } auth default { mechanisms = plain passdb pam { } userdb passwd { } user = root } dict { } plugin { } Anyone know what is causing this problem above? Sending internal email works fine via Telnet however when I try using my mail client / imap4, nothing happens.

4 9

[Dovecot] index_mailbox_set_recent_uid error on 1.1.14
by Mike Abbott 05 May '09

05 May '09

Seen this error a few times from dovecot-1.1.14, Mac OS X, 64-bit Intel. Pretty sure NFS was not used. Panic: IMAP(user): file index-sync.c: line 39 (index_mailbox_set_recent_uid): assertion failed: seq_range_exists(&ibox->recent_flags, uid) The backtrace is: 0 libSystem.B.dylib 0x00007fff85025166 __kill + 10 1 libSystem.B.dylib 0x00007fff850a80e0 abort + 83 2 imap 0x0000000100069f3e default_fatal_finish + 70 3 imap 0x0000000100069f8c i_syslog_fatal_handler + 0 4 imap 0x00000001000696eb i_info + 0 5 imap 0x000000010003b86b index_mailbox_set_recent_uid + 74 6 imap 0x000000010003b8cb index_mailbox_set_recent_seq + 58 7 imap 0x0000000100018f5d maildir_sync_index + 2163 8 imap 0x0000000100017ba5 maildir_sync_context + 662 9 imap 0x0000000100017d8e maildir_storage_sync_init + 104 10 imap 0x000000010000c037 imap_sync_init + 132 11 imap 0x000000010000c289 cmd_sync_delayed + 477 12 imap 0x0000000100006ae8 client_handle_input + 409 13 imap 0x0000000100006bc2 client_input + 148 14 imap 0x0000000100070f83 io_loop_handler_run + 272 15 imap 0x0000000100070348 io_loop_run + 55 16 imap 0x000000010000db95 main + 1635 17 imap 0x0000000100001010 start + 52

2 2

[Dovecot] mail_index_sync_map error on 1.1.14
by Mike Abbott 05 May '09

05 May '09

Seen this error a few times from dovecot-1.1.14, Mac OS X, 64-bit Intel. NFS may have been used. Panic: IMAP(user): file mail-index-sync-update.c: line 843 (mail_index_sync_map): assertion failed: (map->hdr.indexid == index- >indexid || map->hdr.indexid == 0) Two different backtraces to this error are: 0 libSystem.B.dylib 0x00007fff87b775ea __kill + 10 1 libSystem.B.dylib 0x00007fff87bf9fc8 abort + 83 2 imap 0x0000000100069f3e default_fatal_finish + 70 3 imap 0x0000000100069f8c i_syslog_fatal_handler + 0 4 imap 0x00000001000696eb i_info + 0 5 imap 0x0000000100051d3a mail_index_sync_map + 144 6 imap 0x000000010004a22c mail_index_map + 212 7 imap 0x0000000100046a0f mail_index_refresh + 14 8 imap 0x0000000100016e4b maildir_sync_header_refresh + 24 9 imap 0x0000000100016ee6 maildir_sync_quick_check + 61 10 imap 0x000000010001796a maildir_sync_context + 91 11 imap 0x0000000100017d8e maildir_storage_sync_init + 104 12 imap 0x000000010003eb1b mailbox_sync + 31 13 imap 0x00000001000048a8 cmd_select_full + 224 14 imap 0x00000001000049f3 cmd_select + 11 15 imap 0x000000010000677e client_command_input + 33 16 imap 0x000000010000693f client_command_input + 482 17 imap 0x0000000100006a3e client_handle_input + 239 18 imap 0x0000000100006bc2 client_input + 148 19 imap 0x0000000100070f83 io_loop_handler_run + 272 20 imap 0x0000000100070348 io_loop_run + 55 21 imap 0x000000010000db95 main + 1635 22 imap 0x0000000100001010 start + 52 0 libSystem.B.dylib 0x00007fff863125ea __kill + 10 1 libSystem.B.dylib 0x00007fff86394fc8 abort + 83 2 deliver 0x000000010005f82d default_fatal_finish + 70 3 deliver 0x000000010005f8fb i_set_failure_syslog + 0 4 deliver 0x000000010005efda i_info + 0 5 deliver 0x000000010004867e mail_index_sync_map + 144 6 deliver 0x0000000100040b70 mail_index_map + 212 7 deliver 0x000000010003d353 mail_index_refresh + 14 8 deliver 0x0000000100044219 mail_index_transaction_commit_v + 988 9 deliver 0x0000000100032e1c index_transaction_finish_commit + 90 10 deliver 0x000000010000fc4d maildir_transaction_commit + 130 11 deliver 0x0000000100032cb9 index_transaction_commit + 63 12 lib10_quota_plugin.so 0x000000010021efd9 quota_get_mail_user_quota + 1907 13 deliver 0x00000001000354ac mailbox_transaction_commit + 48 14 deliver 0x0000000100001844 deliver_save + 572 15 deliver 0x0000000100003244 main + 6266 16 deliver 0x0000000100000948 start + 52

2 2

[Dovecot] Disabling users whilst still allowing logins with a 'master password'
by Dominic Hargreaves 05 May '09

05 May '09

Hi, I'm having trouble coming up with an authentication configuration that does what we want with dovecot, and thought it would be worth passing it by the list in case anyone can spot an easy fix, or confirm my suspicions that code changes are required. You should probably be aware that this is to support a migration away from Dovecot to a certain Microsoft-provided mail platform, so if this makes you feel uncomfortable, you may choose not to respond ;) We have the following configuration: - the passdb in use is pam (for pam_krb5) - the userdb in use is passwd (for nss_ldap) - GSSAPI is also supported - We support both IMAP and POP - Dovecot 1.0.9 - users have their own uid/gid and the following requirement, in order to support the migration process: - Access to mailstore should be allowed by means of a master password or user (ideally configurable, so we have a fine degree of control over which users are exposed to the migration process at a given time) - Migration process accesses mailstore via IMAP - During migration, users should not be allowed to log in with their Kerberos password, nor with GSSAPI We've achieved the first requirement with master passwords roughly as described at the bottom of http://wiki.dovecot.org/Authentication/MasterUsers by configuring another passdb with the master password hash in place for the users we want to be enabled to migrate. Our standard auth config looks like: auth default { gssapi_hostname = imap210.herald.ox.ac.uk mechanisms = plain login gssapi passdb pam { } userdb passwd { args = blocking=yes } socket listen { master { path = /var/run/dovecot/auth-master } } } and my first try at satisfying my requirements was: auth default { gssapi_hostname = imap-dev.herald.ox.ac.uk mechanisms = plain login gssapi passdb sql { args = /etc/dovecot/dovecot-sql-master.conf } passdb sql { args = /etc/dovecot/dovecot-sql-deny.conf deny = yes } passdb pam { } userdb passwd { args = blocking=yes } socket listen { master { path = /var/run/dovecot/auth-master } } } where /etc/dovecot/dovecot-sql-master.conf describes a query representing those users accessible by the master password (and a hash of the master password) and /etc/dovecot/dovecot-deny.conf describes a query representing those users who should not be allowed to log in with their normal password (or GSSAPI). This almost works, but unfortunately (as documented) the GSSAPI mechanism does not check any passdbs, so does not get checked against the deny passdb. Normally to disable logins for a user we chmod their mail store to 0 so they can't access it, which has the same effect, but this obviously also prevents the master password from being able to access it. It would seem to be generally useful to be able to check GSSAPI logins against a deny passdb, but is this a valid code change? Alternatively, could the *userdbs* could be extended to cope with the idea of a deny form, since the GSSAPI login process already uses them (although it would have to return USERDB_RESULT_USER_UNKNOWN). Alternatively, does anyone have any other ideas about how I could achieve the same effect? Thanks for listening. Dominic. -- Dominic Hargreaves, Systems Development and Support Team Computing Services, University of Oxford

3 5

Re: [Dovecot] Unexplainable problem with POP3 quotas
by Marco Marongiu 05 May '09

05 May '09

Hello Rick, and thanks for replying First of all, the attached script had an error (I attached the wrong one :-\, the "export" line reads: export PATH="/usr/bin:/bin" 2009/4/30 <dovecot(a)dovecot.org>: > > It's a dumb question, but you are running the script via cron or manually > right? (dovecot doesn't execute it in it's own). >From the documentation, I'd expect it does. In http://wiki.dovecot.org/Quota/1.1 I read: >>>You can configure Dovecot to run an external command when user's quota exceeds a specified limit. so I expect that it will run the script. Am I wrong? Ciao --bronto

2 4

Re: [Dovecot] Problem with BODYSTRUCTURE (wrong charset)
by Craig Craig 05 May '09

05 May '09

From: Timo Sirainen<tss(a)iki.fi> > On Mon, 2009-05-04 at 16:18 +0200, Craig Craig wrote: > > Content-Type: multipart/mixed; > > Message-ID:=<d2788bc14788ff16e67e63941937c1e1(a)foo.de>; > > boundary="000XMAIL000" > > The problem is the Message-ID parameter. It's not legal to have ':' > before '='. I added a workaround to v1.2+, but v1.1's code is different > and I'd rather not change it much anymore. > > http://hg.dovecot.org/dovecot-1.2/rev/05a98aaf0aaf > Thanks, I had already figured out that it somehow depended on the headers and that helped a lot! :) For the archive: another problem was, that the Message-ID was not unique...and putting it between "Content-Type" and "boundary" does not seem to be a good idea at all... Thanks, Craig

1 0

Re: [Dovecot] fts-solr plugin issue (Marked invalid)
by Nikolai Derzhak 05 May '09

05 May '09

Send mail body with wrong Content-Type. If put this in maildir indexing raise 500 error. If fix Content-Type to "application/octet-stream" all OK. Because TEXT/*; content dovecot try to index and solr raise 500, on each search. On Tue, 05/05/2009 at 3:01pm, "Nikolai Derzhak" <nikolai(a)6zap.com> wrote: > On Mon, 05/04/2009 at 6:23pm, "Timo Sirainen" <tss(a)iki.fi> wrote: >> On May 4, 2009, at 10:16 AM, Nikolai Derzhak wrote: >> >>> OK. Concentrating problem in one question. >>> How to ignore "bad" message and index next one in indexing procedure >>> (fts plugin) ?. >>> Now, one "error 500" from solr and dovecot (# 1.1.11: /etc/dovecot/ >>> dovecot.conf >>> # OS: Linux 2.6.21.7-2.fc8xen i686 Ubuntu 8.04.2 ext3 >>> ) stop and each next search query repeat the story. >> >> Maybe simply using 1.1.14 would help? I already fixed one Solr issue: http:/ >> / >> hg.dovecot.org/dovecot-1.1/rev/678c3252a454 >> > I've merge back this commit to 1.1.11 code, cause this is my request about > special chars ;). > >> If that's not the problem, it would help to have one of those mails >> that breaks it. > Yes. Now i have two new issue: > 1. "Content-Type:" header in mail wrong - so it's TEXT/mspowerpoint for > example, but it's binary *.ppt file. > 2. many html from repository change-set(many commit's to html) in text/plain > mail > > In both cases we have "Error 500" from solr (HTMLStripReader class) (in solr > log: "Marked inlalid"). > And in both cases we can not detect this in dovecot code (or it's hard to do) > . > So for now i need some method to ignore "bad" messages. > Because we can not catch all variants, but indexing die on first error like > this. > > > > Powered by the 6zap. Sign up at http://www.6zap.com for an account that > provides advanced e-mail, calendar and contacts capabilities. Powered by the 6zap. Sign up at http://www.6zap.com for an account that provides advanced e-mail, calendar and contacts capabilities.

1 0

Re: [Dovecot] fts-solr plugin issue (Marked invalid)
by Nikolai Derzhak 05 May '09

05 May '09

On Mon, 05/04/2009 at 6:23pm, "Timo Sirainen" <tss(a)iki.fi> wrote: > On May 4, 2009, at 10:16 AM, Nikolai Derzhak wrote: > >> OK. Concentrating problem in one question. >> How to ignore "bad" message and index next one in indexing procedure >> (fts plugin) ?. >> Now, one "error 500" from solr and dovecot (# 1.1.11: /etc/dovecot/ >> dovecot.conf >> # OS: Linux 2.6.21.7-2.fc8xen i686 Ubuntu 8.04.2 ext3 >> ) stop and each next search query repeat the story. > > Maybe simply using 1.1.14 would help? I already fixed one Solr issue: http:// > hg.dovecot.org/dovecot-1.1/rev/678c3252a454 > I've merge back this commit to 1.1.11 code, cause this is my request about special chars ;). > If that's not the problem, it would help to have one of those mails > that breaks it. Yes. Now i have two new issue: 1. "Content-Type:" header in mail wrong - so it's TEXT/mspowerpoint for example, but it's binary *.ppt file. 2. many html from repository change-set(many commit's to html) in text/plain mail In both cases we have "Error 500" from solr (HTMLStripReader class) (in solr log: "Marked inlalid"). And in both cases we can not detect this in dovecot code (or it's hard to do). So for now i need some method to ignore "bad" messages. Because we can not catch all variants, but indexing die on first error like this. Powered by the 6zap. Sign up at http://www.6zap.com for an account that provides advanced e-mail, calendar and contacts capabilities.

1 0

[Dovecot] Maildir bugs
by Timo Sirainen 05 May '09

05 May '09

I just found out that when using deliver or saving mails with IMAP APPEND command there was all kinds of race condition bugs that could cause e.g.: dovecot-uidlist: Duplicate file entry at line 18: 1241457213.M690227P32531.host,W=3359:2,T (uid 170019 -> 170020) And who knows what else. If you've run into those problems, these patches should help: http://hg.dovecot.org/dovecot-1.1/rev/6bbe27c32cb8 http://hg.dovecot.org/dovecot-1.1/rev/003bebac2c76 http://hg.dovecot.org/dovecot-1.1/rev/96c8632775df They are pretty large though, hopefully won't break anything. But at least my stress tests can no longer break it. I'll probably release v1.1.15 RC in a few days so at least some people can try to break it before the final v1.1.15 :)

1 0

[Dovecot] questions about compiling Sieve plugin
by MercuryShadow 04 May '09

04 May '09

I'm using the info on the wiki to try to compile the sieve plugin for dovecot on a server running Debian Etch, but I'm getting nowhere because I'm stuck trying to find the directory to use in the "--with-dovecot=" argument when running the configure script. The wiki says: Compilation is identical among the CMUSieve and the new Sieve plugin. Use --with-dovecot=<path> to point to dovecot-config file's directory. There are two possibilities where this could exist: 1. If you configured Dovecot with --enable-header-install, you'll have dovecot-config installed in $prefix/lib/dovecot/ directory. 2. Compiled Dovecot sources' root directory. The problem is that there is no such file called "dovecot-config" on the server, and dovecot was installed from the Debian repositories, not compiled. There is no /usr/lib/dovecot or /usr/local/lib/dovecot. I figured that, as is the case with many Debian packages, the libraries needed to compile would be included in a "dovecot-dev" package, but I checked and there is no such package in the Debian repos. Does anyone know what I need to do in order to get this plugin compiled? -- View this message in context: http://www.nabble.com/questions-about-compiling-Sieve-plugin-tp23285989p232… Sent from the Dovecot mailing list archive at Nabble.com.

2 8

[Dovecot] Problem with BODYSTRUCTURE (wrong charset)
by Craig Craig 04 May '09

04 May '09

Hi, I've changed a program that fetches mail via IMAP from an Exchange Server to use our Dovecot machine. But now I'm experiencing strange problems with seem to be caused by a wrong BODYSTRUCTURE answer that causes wrong charset handling. When fetchting the BODYSTRUCTURE I see this: Dovecot: 88 UID fetch 32 (BODYSTRUCTURE) * 1 FETCH (UID 32 BODYSTRUCTURE ("text" "plain" ("charset" "us-ascii") NIL NIL "7bit" 725664 9385 NIL NIL NIL NIL)) 88 OK Fetch completed. Exchange: 90 UID fetch 10 (BODYSTRUCTURE) * 2 FETCH (BODYSTRUCTURE (("TEXT" "PLAIN" ("charset" "iso-8859-1") NIL NIL "QUOTED-PRINTABLE" 2784 47 NIL NIL NIL)("IMAGE" "JPEG" ("name" "197254.jpg") NIL NIL "BASE64" 498806 NIL ("attachment" ("filename" "197254.jpg")) NIL)("IMAGE" "JPEG" ("name" "197254_1.jpg") NIL NIL "BASE64" 188424 NIL ("attachment" ("filename" "197254_1.jpg")) NIL)("IMAGE" "JPEG" ("name" "197254_2.jpg") NIL NIL "BASE64" 31878 NIL ("attachment" ("filename" "197254_2.jpg")) NIL)("TEXT" "PLAIN" ("name" "pix.xml") NIL NIL "QUOTED-PRINTABLE" 2934 49 NIL ("attachment" ("filename" "pix.xml")) NIL)("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 0 1 NIL ("attachment" NIL) NIL) "mixed" ("Message-ID:" "<d2788bc14788ff16e67e63941937c1e1(a)foo.de>" "boundary" "000XMAIL000") NIL NIL) UID 10) 90 OK FETCH completed. Mail contents were: [...] To: blah(a)foo.de Subject: Pictures From: blah2(a)foo.de CC: Errors-To: blah2(a)foo.de Sender: foobar(a)foo.de MIME-Version: 1.0 Content-Type: multipart/mixed; Message-ID:=<d2788bc14788ff16e67e63941937c1e1(a)foo.de>; boundary="000XMAIL000" Message-Id: <20090504120233.58AFC346FC(a)foo.de> Date: Mon, 4 May 2009 14:02:33 +0200 (CEST) --000XMAIL000 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: Quoted-Printable <?xml version=3D"1.0" encoding=3D"ISO-8859-1" ?> <foo> [...] --000XMAIL000 Content-Type: image/jpeg; name="197254.jpg" Content-Transfer-Encoding: Base64 Content-Disposition: attachment; filename="197254.jpg" [...] --000XMAIL000 Content-Type: text/plain; name="pix.xml" Content-Transfer-Encoding: Quoted-Printable Content-Disposition: attachment; filename="pix.xml" <?xml version=3D"1.0" encoding=3D"ISO-8859-1" ?>=0D <blah>=0D [...] Here is my dovecot -n output: # 1.1.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.21-gentoo-r1 i686 Gentoo Base System release 1.12.11.1 ext3 log_path: /var/log/dovecot protocols: imap imaps pop3 pop3s listen: [::] ssl_cert_file: /etc/ssl/dovecot/server.pem ssl_key_file: /etc/ssl/dovecot/server.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_privileged_group: vmail mail_location: maildir:/var/mail/%d/%n mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 auth default: mechanisms: plain login digest-md5 cram-md5 rpa apop username_translation: AaBbCcDdEeFfGgHhIiJjKkLlMmNnOoPpQqRrSsTtUuVvWwXxYyZz verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/passwd.dovecot userdb: driver: passwd-file args: /etc/passwd.dovecot socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: root group: vmail So is this a parsing error in dovecot or was the Exchange Server just parsing incorrect MIME-Headers?! Any hint would be appreciated... Thanks in advance, Craig

2 1

Re: [Dovecot] fts-solr plugin issue (Marked invalid)
by Nikolai Derzhak 04 May '09

04 May '09

OK. Concentrating problem in one question. How to ignore "bad" message and index next one in indexing procedure (fts plugin) ?. Now, one "error 500" from solr and dovecot (# 1.1.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.21.7-2.fc8xen i686 Ubuntu 8.04.2 ext3 ) stop and each next search query repeat the story. I've explored fts, and ftp-solr directories in src, without success for now. Timo, you understand code much bettter, can you help me and point to place in code, or probably create some patch, if possible ?. On Fri, 05/01/2009 at 5:56pm, "Nikolai Derzhak" <nikolai(a)6zap.com> wrote: > Hi. > > We user ftp-solr plugin and have problem with solr-1.3+ > with HTMLStripWhitespaceTokenizerFactory (Solr schema in attachments). > In some maildir's present messages with wrong "Content-Type: " fields in > attachments. > For example: > " > Content-Type: TEXT/mspowerpoint; name="Zapatec_6zap_netvibes_1.ppt" > " > Indexing for this messages is stop with "fts_solr: Indexing failed: 500 > Internal Server Error". > In solr log is: > " > SEVERE: java.io.IOException: Mark invalid > at java.io.BufferedReader.reset(BufferedReader.java:485) > " > (mail list with discussion: http://markmail.org/message/2fnfiwygvehjngyr# > query:SEVERE%3A%20java.io.IOException%3A%20Mark%20invalid%20lucene+page:1+ > mid:2fnfiwygvehjngyr+state:results) > > Look's like dovecot try to index attachments like this. > Also for some messages we have same error. > Dovecot stop indexing of box and each search we have lag and CPU load on > server. > > So we need to make dovecot more "stable" to this error. > For first time , will be good, just ignore problematic messages with error > from solr. > > Let's discuss this issue, because this is general problem. > We ready to explore code where needed , etc. > > Regards, > Nikolai > > Powered by the 6zap. Sign up at http://www.6zap.com for an account that > provides advanced e-mail, calendar and contacts capabilities. Powered by the 6zap. Sign up at http://www.6zap.com for an account that provides advanced e-mail, calendar and contacts capabilities.

3 2

Re: [Dovecot] fts-solr plugin issue (Marked invalid)
by Nikolai Derzhak 04 May '09

04 May '09

On Mon, 05/04/2009 at 5:36pm, "Rui Carneiro" <rui.carneiro(a)portugalmail.net> wrote: > I do not have sure if I understood your problem correctly. > > Are you trying to index attachments from messages? Or Dovecot is indexing > some "bad" parts and you just do not know why? > Read first post, please, with description. But in sum: when dovecot try to index some mail's, that solr tokenizer not eat (error 500, Marked invalid), dovecot stop indexing of box and retry attempts in each next search with same result. > Regards, > Rui Carneiro > -- > Portugalmail, Comunicações S.A. > www.portugalmail.net > > Citando Nikolai Derzhak <nikolai(a)6zap.com>: > >> OK. Concentrating problem in one question. >> How to ignore "bad" message and index next one in indexing procedure (fts >> plugin) ?. >> Now, one "error 500" from solr and dovecot (# 1.1.11: >> /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.21.7-2.fc8xen i686 Ubuntu 8.04.2 ext3 >> ) stop and each next search query repeat the story. >> I've explored fts, and ftp-solr directories in src, without success for now. >> >> Timo, you understand code much bettter, can you help me and point to place >> in >> code, >> or probably create some patch, if possible ?. Powered by the 6zap. Sign up at http://www.6zap.com for an account that provides advanced e-mail, calendar and contacts capabilities.

2 1

[Dovecot] login_max_processes_count and logs
by Dominique Fournier 04 May '09

04 May '09

Hello, Our server reach today the login_max_processes_count. Dovecot works well and don't accept any new connection. But it doesn't log anything about this problem : we are blind why the connection is refused for the clients. Could you add a log when the connection limit is reached or (maybe better) a log with the occupation for the beginning of each connection ? Thanks Dom Dovecot version 1.0.15 on Debian AMD64

3 3

[Dovecot] Strange virtual users issues
by Ovi C 02 May '09

02 May '09

Hi i'm using dovecot with virtual users from a postgresql database backend on a Debian Lenny system. I'm using qmail as MTA. Mostly everything seems to work ok , except the fact that the mail delivered never reaches virtual user's INBOX. Here's the dovecot config: dovecot -n # 1.0.15: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_disable: yes login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_greeting: Server ready to process requests. login_process_per_connection: no max_mail_processes: 512 verbose_proctitle: yes first_valid_gid: 61 last_valid_gid: 2003 mail_privileged_group: mail mail_location: maildir:~/Maildir mail_debug: yes mail_plugins: quota autocreate imap_client_workarounds: outlook-idle delay-newmail auth default: mechanisms: plain login digest-md5 passdb: driver: pam passdb: driver: passwd passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 master: path: /var/run/dovecot/auth-master mode: 384 user: root group: mail plugin: quota: maildir acl: vfile:/etc/dovecot-acls trash: /etc/dovecot-trash.conf lazy_expunge: .EXPUNGED/ .DELETED/ .DELETED/.EXPUNGED/ autocreate: Trash autocreate2: Spam The setup is as follows: I've created a system user named vmail in /home with a group also called vmail. The virtual users are created in /home/vmail/username when i configure an account from thunderbird. If I send an message from a virtual user to an system user everythings is ok. The system user get's the mail in his Maildir. But if I send a message from the system user to the virtual user the MTA's log says that the message has been delivered but it never reaches virtual user's INBOX. Here's a snip from logs both qmail's and dovecot's: cat /var/log/qmail.log status: local 0/10 remote 0/20 new msg 155041 info msg 155041: bytes 263 from <caretaker@localhost.> qp 4560 uid 1000 starting delivery 1: msg 155041 to remote akkersson@localhost. status: local 0/10 remote 1/20 new msg 155147 info msg 155147: bytes 438 from <caretaker@localhost.> qp 4563 uid 64011 starting delivery 2: msg 155147 to local akkersson@localhost status: local 1/10 remote 1/20 delivery 1: success: 127.0.0.1_accepted_message./Remote_host_said:_250_ok_1241272738_qp_4563/ status: local 1/10 remote 0/20 end msg 155041 delivery 2: deferral: status: local 0/10 remote 0/20 cat /var/log/dovecot.log dovecot: 2009-05-02 16:30:46 Info: Dovecot v1.0.15 starting up dovecot: 2009-05-02 16:30:46 Info: auth(default): pgsql: Connected to maildb dovecot: 2009-05-02 16:32:00 Info: IMAP(akkersson): Loading modules from directory: /usr/lib/dovecot/modules/imap dovecot: 2009-05-02 16:32:00 Info: IMAP(akkersson): Module loaded: /usr/lib/dovecot/modules/imap/lib10_quota_plugin.so dovecot: 2009-05-02 16:32:00 Info: IMAP(akkersson): Module loaded: /usr/lib/dovecot/modules/imap/lib11_autocreate_plugin.so dovecot: 2009-05-02 16:32:00 Info: IMAP(akkersson): Effective uid=2003, gid=61, home=/home/vmail/akkersson dovecot: 2009-05-02 16:32:00 Info: IMAP(akkersson): maildir: data=/home/vmail/akkersson/Maildir dovecot: 2009-05-02 16:32:00 Info: IMAP(akkersson): maildir: root=/home/vmail/akkersson/Maildir, index=/home/vmail/akkersson/Maildir, control=, inbox= dovecot: 2009-05-02 16:32:00 Info: imap-login: Login: user=<akkersson>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured dovecot: 2009-05-02 16:58:11 Info: IMAP(akkersson): Disconnected: Logged out but akkersson's INBOX is empty.. Any hints ??

1 0

[Dovecot] deliver_log_format not working with 1.2.rc3?
by Anton Dollmaier 01 May '09

01 May '09

Hi list, just a small issue with deliver_log_format. Although, the parameter is altered in the config and is correctly shown with "dovecot -n", deliver uses only its default log_format. > root@terra:~# dovecot -n | grep -e deliver -e "^#" > # 1.2.rc3: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.26-1-686 i686 Debian 5.0.1 > deliver_log_format: %m:%$:%f:%s this is from a recent mail: > deliver(antondollmaier(a)aditsystems.de): 2009-04-30 16:13:45 Info: sieve: msgid=<20090430141309.GN16305(a)katherina.student.utwente.nl>: stored mail into mailbox 'Dovecot-List' Subject and From are obviously not logged. Any hints? thanks in advance. best regards, Anton

2 1

[Dovecot] mbox-snarf plugin and istream-mail-stats.c revision 9000:b02c642b4e51
by Jonathan Siegle 01 May '09

01 May '09

I'm getting this error: May 1 09:09:30 tr27n12.aset.psu.edu syslog: PSU mbox-snarf name is INBOX May 1 09:09:30 tr27n12.aset.psu.edu dovecot: Panic: IMAP(tstem38): file istream-mail-stats.c: line 75: assertion failed: (ret != -1 || stream->istream.eof || stream->istream.stream_errno != 0) Will this error go away before the general 1.2 release? Thanks, Jonathan

2 4