- dovecot - dovecot.org

[Dovecot] Full text search indexing
by Timo Sirainen 12 Apr '06

12 Apr '06

SEARCH is probably the slowest command that Dovecot has, especially when searching texts from message bodies. There exists a lot of text search engines, but unfortunately as far as I know there exists only one that can be used with IMAP. The reason why IMAP is special is because it requires that when searching eg. "ello", it must match also mails which contain eg. word "hello". Most search engines support only matching either complete words or matching from the beginning of the words. So, the one usable engine which I know of is Cyrus's squat indexing. It works basically by indexing the mails in 4 byte blocks and storing them in a trie. So if there exists a string "he llo", it indexes "he l", "e ll" and " llo" strings. When searching it then gets the first 4 bytes of the search keyword, looks them up from the squat index and then once it gets a list of matched mails, it reads the mails and does a proper search for only them. So the squat index doesn't give exact results, but only narrows down the number of mails which have to be searched. The biggest problem with those squat indexes is that it doesn't support incremental updates, so there has to be some nightly runs to rebuild the indexes from scratch. And with lots of users that's not practical, so I think that's the reason why the squat indexes aren't all that commonly used with Cyrus. I was thinking about taking the basic ideas from the Cyrus squat indexer but making it possible to update it incrementally so Dovecot could do it automatically. Also to reduce space I was thinking about sorting the 4 byte strings so instead of eg. "abcd", "dcba" and "abdc" all being indexed, they could all be stored as "abcd" (so the search key would also have to be sorted before use). But that might cause larger result sets making the search slower, so I'll have to test if it's worth it. Unicode characters might also be a problem. If the characters are stored in 16bit characters, it would waste space. If they're stored in UTF-8 the 4 bytes might not be enough to give a useful index (as a single character might take even 5 bytes). One thing I was thinking was to not index in 4 byte blocks, but 4 character blocks in UTF-8, so the trie could be 4..20 levels deep depending on what unicode characters are used (in finnish text it would probably be max. 7 levels deep). So, now would be a good time to tell if you can think of some better ideas how to do this, especially if you know about some new algorithms specifically designed for this kind of searching. :) Pretty much all the algorithms I've found about this were suffix trees/arrays and this basic trie thing.

6 14

[Dovecot] beta7: TLS, Success!!
by Jeff A. Earickson 12 Apr '06

12 Apr '06

Timo, With my setup (Solaris 9, IMAP only, TLS, plain auth, mbox format), beta 7 is now working correctly. Thanks for your efforts! Jeff Earickson Colby College

1 0

Re: [Dovecot] strip domains]
by Michael Baird 12 Apr '06

12 Apr '06

-------- Forwarded Message -------- From: Michael Baird <mike(a)tc3net.com> To: Timo Sirainen <tss(a)iki.fi> Subject: Re: [Dovecot] strip domains Date: Wed, 12 Apr 2006 10:42:43 -0400 On Wed, 2006-04-12 at 17:17 +0300, Timo Sirainen wrote: > On Wed, 2006-04-12 at 09:48 -0400, Michael Baird wrote: > > I don't > > know why there is so much resistance to supporting functionality that is > > mostly a standard feature with other pop3/imap servers, and which also > > makes it *much* easier to migrate to dovecot from other solutions which > > support the domain stripping. > > I don't remember having seen all that much requests for this feature. > Perhaps lately several people have asked for it, but for the last 3 > years there haven't been all that many people. > > If I added a new setting for every single feature people asked, > dovecot-example.conf would be much larger than it currently is. And it > already is too large I think. I'm not asking for every new feature, just one that I think fits in with the current design of dovecot already and is a standard feature provided by other servers. You already provide the ability built in to "strip domains" per say, with the %u and %d values built in. To accomplish what I would like to do, I only need to be able to pass whatever combination of these values I wish to my passdb and userdb functions of choice. Regards Michael Baird

1 0

[Dovecot] [patch] MD5 variable modifier
by Ben Winslow 12 Apr '06

12 Apr '06

Hello, I've taken a different (better) approach to getting tpop3d-compatible UIDLs: the first attached patch (...md5_modifier.diff) adds a variable expansion modifier (%M) to return the md5sum of the expanded variable in hex. With the patch, you can get tpop3d-compatible UIDLs by setting pop3_uidl_format to %Mf. While testing interactions with other modifiers (e.g. %MUf to get the md5sum in uppercase), I noticed that %U is broken -- it lowercases the input just as %L does. The second patch (...ucase_modifier_fix.diff) is a one-liner to fix that. Finally, the third attached patch (...apop_aesthetics.diff) is another trivial patch to add a space between the server greeting and the APOP challenge. I think it looks better when telnetting to the server to diagnose something, and since spaces are stripped from the end of the login_greeting, editing the code is the only way to add the space. Cheers, -- Ben Winslow <rain(a)bluecherry.net>

2 2

[Dovecot] Beta 7 fixes the Horde issue!
by interfaSys sàrl - Rich Internet Applications 12 Apr '06

12 Apr '06

Thank you for beta 7. Everything works well this time (for us at least). Best regards, Olivier Paroz interfaSys sàrl

1 0

[Dovecot] Postfix dovecot SASL and LDAP not working...
by Brice Figureau 12 Apr '06

12 Apr '06

Hi, I'm trying to setup a new mail server where postfix 2.3-20060405 would use Dovecot's SASL and users located in LDAP. It's with dovecot-1.0-beta5. On the postfix side, I followed the SASL_README and have the following in main.cf: smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_auth_enable = yes On the dovecot side I have the following related to auth: auth_verbose = yes auth_debug = yes auth_debug_passwords = yes auth default { mechanisms = plain digest-md5 cram-md5 # LDAP database passdb ldap { # Path for LDAP configuration file, see /etc/dovecot/dovecot-ldap.conf for # example args = /etc/dovecot/dovecot-ldap.conf } # LDAP database userdb ldap { # Path for LDAP configuration file, see /etc/dovecot/dovecot-ldap.conf for # example args = /etc/dovecot/dovecot-ldap.conf } userdb prefetch { } user = root ## ## dovecot-lda & postfix SASL specific settings ## socket listen { master { path = /var/run/dovecot-auth-master mode = 0666 } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } pop or imap authentification with LDAP works fine: Apr 11 17:43:08 localhost dovecot: auth(default): client in: AUTH^I1^IPLAIN^Iservice=IMAP^Isecured^Ilip=127.0.0.1^Irip=127.0.0.1^Iresp=AGJyaWNlAHRvdG8= Apr 11 17:43:08 localhost dovecot: auth(default): ldap(raymond,127.0.0.1): base=ou=people, dc=daysofwonder, dc=com scope=subtree filter=(&(objectClass=CourierMailAccount)(uid=raymond)) fields=uid,userPassword,homeDirectory,uidNumber,gidNumber,mailbox Apr 11 17:43:08 localhost dovecot: auth(default): ldap(raymond,127.0.0.1): uid(user)=raymond userPassword(password)={PLAIN}toto homeDirectory(userdb_home)=/home/raymond uidNumber(userdb_uid)=1000 gidNumber(userdb_gid)=1000 mailbox(userdb_mail)=/home/raymond/Maildir Apr 11 17:43:08 localhost dovecot: auth(default): client out: OK^I1^Iuser=raymond^I Apr 11 17:43:08 localhost dovecot: auth(default): master in: REQUEST^I1^I15439^I1 Apr 11 17:43:08 localhost dovecot: auth(default): ldap(raymond,127.0.0.1): base=ou=people, dc=daysofwonder, dc=com scope=subtree filter=(&(objectClass=CourierMailAccount)(uid=raymond)) fields=homeDirectory,uidNumber,gidNumber,uid,mailbox Apr 11 17:43:08 localhost dovecot: auth(default): master out: USER^I1^Iraymond^Isystem_user=raymond^Ihome=/home/raymond^Iuid=1000^Igid=1000^Imail=/home/raymond/Maildir Apr 11 17:43:08 localhost dovecot: imap-login: Login: user=<raymond>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured postfix SASL doesn't work, but more important don't print any debug messages, nor try to contact the LDAP server at all: Apr 11 18:00:59 localhost postfix/smtpd[15637]: connect from localhost.localdomain[127.0.0.1] Apr 11 18:01:13 localhost dovecot: auth(default): client in: AUTH^I3^Iplain^Iservice=smtp^Iresp=YnJpY2UAYnJpY2UAdG90bw== Apr 11 18:01:14 localhost dovecot: auth(default): client out: FAIL^I3^Iuser=raymond Apr 11 18:01:14 localhost postfix/smtpd[15637]: warning: localhost.localdomain[127.0.0.1]: SASL plain authentication failed: I tried several variations of the config, like without userdb prefetch, moving postfix socket config block in another auth block... Nothing worked. Since the samples are mentioning only pam as userdb and passdb source, maybe it can't work with LDAP ? Any idea on how to solve this issue ? Thanks, Regards, -- Brice Figureau

1 1

[Dovecot] Horde problem not fixed in beta6
by interfaSys sàrl - Rich Internet Applications 12 Apr '06

12 Apr '06

We experience the same problem between Horde and Dovecot 1.0 beta 6. Beta 3 is still the only working solution right now. Best regards, Olivier Paroz interfaSys sàrl

2 2

[Dovecot] DOVECOT+SQLITE
by Alexandre Busquets 12 Apr '06

12 Apr '06

Has somebody experience with DOVECOT+SQLITE? Which will be the config variables? auth_userdb = mysql /etc/dovecot/dovecot-mysql.conf ------------------------------------------------------ db_host = localhost db_port = 3306 db_unix_socket = /var/run/mysqld/mysqld.sock db = test db_user = test db_passwd = test db_client_flags = 0 default_pass_scheme = PLAIN password_query = SELECT clear FROM users WHERE username = '%u' user_query = SELECT pop as home, uid, gid FROM users WHERE username = '%u' ------------------------------------------------------ Is possible use sqlrelay

2 2

[Dovecot] DON'T DELETE THIS MESSAGE?
by Knute Johnson 12 Apr '06

12 Apr '06

I have a new install of FC5 running dovecot 1.0.0 beta 2.7. I get my mail with POP3 but I have a user using IMAP. They both work great but I keep getting a message from dovecot in my mail saying DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA. I think it is there to help keep track of IMAP messages. Since I use POP3 is there any way not to have it show up all the time? I've deleted it, I've deleted my account and re-added it but it keeps coming back. Dovecot is also making a 'mail' directory in my ~ that I don't think I need. What can I do about this? Thanks, -- Knute Johnson Molon Labe...

3 2

[Dovecot] IMAPS - timeout while moving mails
by Petar Bogdanovic 12 Apr '06

12 Apr '06

Hi, I'm using Dovecot v1.0.beta5 (NetBSD 3.0, i386) and Mozilla Thunderbird 1.5 (XP, i386). My dovecot.conf is pretty simple, because I just use dovecot for me as the only user. However, when I try to move a 'larger' amount of mails (>200) from my local TB-mbox to the remote IMAPS-storage, the process hangs after about 150 - 160 transferred mails (the 'blocking' mail is _never_ the same). I don't have the slightest clue why - there is nothing in the logs - even if I turn 'verbose_ssl' on. After a while, TB tells me, that the server timed out and dovecot says: "imap(petar): Disconnected" Usually - if I close my MUA correctly - dovecot says: "imap(petar): Disconnected: Logged out" Is somebody maybe experiencing something similar? Thank you very much & with kind regards, Petar Bogdanovic

4 6