June 2007 - dovecot - dovecot.org

[Dovecot] [PATCH, RFC] add APOP authentication mechanism
by Andrey Panin 14 Aug '17

14 Aug '17

Hello all, this patch add APOP authentication mechanism to dovecot 1.0-test23. Please take a look. Best regards. -- Andrey Panin | Linux and UNIX system administrator pazke(a)donpac.ru | PGP key: wwwkeys.pgp.net

3 2

[Dovecot] compressed mbox patch
by Axel Thimm 16 Jun '09

16 Jun '09

Some ages ago there existed a compressed mbox patch, but now all I can find are posting about it. Does such a beast exist and would it remotely apply to today's dovecot? If not, is there any hope that something like that emerges? Thanks! -- Axel.Thimm at ATrpms.net

3 5

[Dovecot] Feature Missing OUTBOX Folder
by Frank Doege 01 Jun '09

01 Jun '09

Hi Dovecot People, i have switched my system from courier-imap to dovecot and iam quite happy with it, the performance is well and i had nearly no problems. One little feature iam missing is in Courier i had a folder called "Outbox" there i could paste mails which i want to sent out. In some networks its not always available to sent mails via port 21, i know i could just set up my email server to another port but it was a nice feature. Is this a big afford or is it just alittle hack ? could it done via a little plugin ? Thanks Timo & all others for the great work ! Frank

5 7

[Dovecot] BUG: messages created with permissions not respecting umask
by Thomas Vander Stichele 20 Jul '08

20 Jul '08

Using dovecot 1.0rc15, together with postfix and dovecot-lda. umask is set to 0007. This should ensure directories and files get created with read/write permissions for both user and group. However, dovecot-lda writes files with 600 permissions, instead of 660. So dovecot does not seem to respect the umask configuration property for local mail delivery. In my particular case, I have vmail:vmail own the mail directories, and a backup user part of the vmail group. However the backup user cannot read the mails and thus back them up properly. Suggestions: - respect the umask for file delivery as well - alternatively, take the parent directory's permissions, strip executable bits, and apply to file Thanks Thomas

3 3

[Dovecot] Postfix & Dovecot LDA
by Jochen Schulz 18 May '08

18 May '08

Hi, before I start to write a lengthy email about something that isn't really possible anyway: can I make Postfix use Dovecot's LDA and start it with different user IDs? My scenario: Dovecot authenticates users for Postfix and itself using Postgresql. Mails for two domains should be stored under /srv/<domain>/<user> (which is the location returned by my user_sql query and mail_location). When my clients login (with usernames of the form 'user@domain'), Dovecot creates/opens the correct mailboxes for them, but I can't get LDA to deliver to these mailboxes. Whatever I try, I am always running into some kind of permission problems (either for the mailboxes, or for auth_socket). I want to use a unique UID for every virtual domain, so I guess LDA needs to have permissions for every corresponding mailbox *and* Dovecot's auth_socket_path. Is there a good solution for this which doesn't involve severe security implications? J. -- If all my friends had Playstations I would buy a Nintendo to prove my individuality. [Agree] [Disagree] <http://www.slowlydownward.com/NODATA/data_enter2.html>

6 7

[Dovecot] Another numptie question - making maildirs for shared mailboxes.
by Stroller 12 May '08

12 May '08

Hi there, Sorry to have so much trouble with my Dovecot setup & to ask so many questions, but I'm having some problems with shared folders. I've read the thread at <http://dovecot.org/pipermail/dovecot/2005-January/006079.html> and added the following lines to /etc/dovecot.conf: namespace private { separator = / prefix = inbox = yes } namespace public { separator = / prefix = General/ location = maildir:/tmp/dovecot-test/ inbox = no hidden = no } So I create a directory /tmp/dovecot-test/ and an empty file /tmp/dovecot-test/dovecot-shared with liberal (777 & 666 respectively) permissions and then try to subscribe to the "General" folder in Outlook - I get a "failed to subscribe to the folder. The server responded `the mailbox doesn't exist: General`" error message, but cur, new & tmp are created in /tmp/dovecot-test. These subfolders are owned by the user that attempted to subscribe & have 700 permissions; chmod'ing them 777 makes no difference. Since "default_mail_env = maildir:%h/.maildir" in dovecot.conf I make a directory /tmp/dovecot-test/.maildir also containing cur, new, tmp & dovecot-shared. When I select "IMAP folders" in Outlook I am still unable to subscribe to "General" but am now able to subscribe to "General/maildir". The subscribed folder is shown in Outlook underneath the users' Inbox as a sub-folder of general, screenshot at <http://linux.stroller.uk.eu.org/DovecotSharedFolders.png> - see how I can't subscribe to "General" alone, but it is "brought in" by subscribing to "General/maildir"? If I try to copy a test message into General then, after a delay of a minute or so, I get "Can't move the items - this IMAP command could not be sent to the server before the connect was terminated"; if I try to copy the test message into General/maildir then I immediately get "Can't copy the items. The server responded: `Internal error occurred. Refer to server log for more information [2005-12-20 12:05:40]". I've tried tinkering with other directory structures under /tmp/dovecot-test/ with no success - can anyone suggest what I'm doing wrong, please? It's quite important to me that the "General" folder is a fairly high-level one - only one additional folder icon should show up for the users' shared mail. At one point I managed to create a folder General/General, but that's undesirable - the users wouldn't understand the concept of having a folder showing in Outlook that they can't write to, or which is only used as a container for another folder. (In any case, I believe I was still unable to write to it). I was about to say, "I know I could resolve this with the use of a symlink-type shared folder, but that's undesirable because I'd have to manually add the symlink for each user that's added to the system", but writing this it occurs to me that I could resolve that by adding the symlink to the the etc/skel/. Nevertheless, I see the namespaces approach as more elegant, as it allows for different user groups having different shared folders. I'd appreciate any advice - I suspect I'm just getting the directory hierarchies wrong, or the syntax of dovecot.conf wrong AGAIN! If anyone with a working shared folders setup could provide a copy of the appropriate section of dovecot.conf & a listing of the appropriate directories then I'd be EXTREMELY grateful. Stroller.

7 12

[Dovecot] Dovecot, LDAP and something akin to Postfix' "table search order" from virtual deliver.
by Pawel Sawicki 01 Feb '08

01 Feb '08

Hi. /* The short version. */ Is there a way to mimic Postfix' "table search order" in Dovecot's LDAP configuration of pass_filter/user_filter? /* Here goes the detailed description :) */ Just to quote the http://www.postfix.org/virtual.8.html: --- TABLE SEARCH ORDER (...) The search order is as follows. The search stops upon the first successful lookup. (...) o The user(a)domain.tld address, without address exten- sion, is looked up next. o Finally, the recipient @domain is looked up. --- Basicaly this allows to have separate users assigned addresses like j.doe(a)example.com (first account), j.smith(a)example.com (second account) and at the same time to have a "catch all" anything(a)example.com mailbox (for the administrator e.g.). What do I need it for? To have dovecot-lda working. Why? I want Sieve :) The test data may look as follows: account #1: + mail: j.doe(a)example.com + mailAlternateAddress: j.c.doe(a)example.com account #2: + mail: j.smith(a)example.com + mailAlternateAddress: j.t.smith(a)example.com account #3 (administrator): + mail: bofh(a)example.com + mailAlternateAddress: @example.com (means - anything possible) The filter I used is (more or less - for both pass_filter and user_filter): (|(mail=%u)(mailAlternateAddress=%u)(mailAlternateAddress=@%d)) Just for testing purposes I've stripped off some irrelevant data (like checking if the account is active), since that made the filter about 140 characters long ;) The trouble is when there're both a "catch all" and a true account that are linked to different accounts. As an example, if I try to send mail to "j.doe(a)example.com" I receive: dovecot: auth(default): ldap(j.doe(a)example.com): Multiple replies found for user Which is of course true, since the filter returns "account #2" and "account #3". What I'd like to get? I'd like to set the filter to: (|(mail=%u)(mailAlternateAddress=%u)) Where "%u" is expanded to "j.doe(a)example.com" during the first run Then, if dovecot can't find any user that would match this literaly it should retry the search operation, but now with "@example.com" as the "%u". I've tried to fiddle with the filter itself, but I see any reasonable way of doing it. I don't even know if it is possible at all. Could anyone give me a hint? Or maybe it's already on someone's whish list? Some final info. * I've followed the examples on dovecot.org website: http://wiki.dovecot.org/LDA#head-dacb9b9a1f19c3ea86bb6f8caa3d64e3ddad9ef8 * Here's the dovecot's master.conf entry: dovecot unix - n n - - pipe flags=DRhu user=mail:mail argv=/usr/libexec/dovecot/deliver -d ${recipient} Best regards! Pawel Sawicki

2 2

[Dovecot] Too many open files
by Andrew Hutchings 10 Jan '08

10 Jan '08

Hi Guys, Just got this on one of my test72 dovecot servers: - Jun 10 12:40:28 svr21 dovecot: auth(default): net_connect_unix(/var/run/dovecot/auth-worker.18818) failed: Too many open files Jun 10 12:40:28 svr21 dovecot: pop3-login: Disconnected: method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx Jun 10 12:40:28 svr21 dovecot: pop3-login: Disconnected: method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx Jun 10 12:40:28 svr21 dovecot: child 18818 (auth) returned error 89 Jun 10 12:40:29 svr21 dovecot: Auth process died too early - shutting down Jun 10 12:40:29 svr21 dovecot: auth(default): mysql: Connected to localhost (maildb) Jun 10 12:40:29 svr21 dovecot: auth(default): net_connect_unix(/var/run/dovecot/auth-worker.13784) failed: No such file or directory Jun 10 12:40:29 svr21 dovecot: child 13784 (auth) returned error 89 - Is dovecot leaving files open? Or is it a large mailbox? Regards Andrew -- Andrew Hutchings (A-Wing) Linux Guru - Netserve Consultants Ltd. - http://www.domaincity.co.uk/ Admin - North Wales Linux User Group - http://www.nwlug.org.uk/ BOFH excuse 122: because Bill Gates is a Jehovah's witness and so nothing can work on St. Swithin's day.

5 5

[Dovecot] What would you tell the CIO in an "ABCs of Email" overview?
by Esther Schindler 07 Jan '08

07 Jan '08

One of the key points that came up when I researched and wrote the "Five Things CIOs Should Know about Fighting Spam" article (http:// www.cio.com/article/28830) was that they should know the basics of how email works. Otherwise, said plenty of techies, the CIO won't have the first idea of what the email admin is complaining about. So I'm going to do my part. I'm going to write an "ABCs of Email" article (to accompany the many other ABCs articles we have on CIO.com, at http://www.cio.com/article/40242 ). I'd like your input on the topics that should be included, keeping in mind the fact that the target reader is a CIO, IT manager, or someone who wants to understand the basics, *not* actively get involved in email management. You don't need to write an essay for me or inundate me with links (though hey, if you want to make my life easier I shall not complain). What I'm looking for, primarily, are the categories of information that I should cover. In other words, if your CIO had an email ephiphany and asked you to give a half-hour presentation, what would you include? This won't be an "Expert says..." article nor will it be "Geek on the street says...." I intend to compile and research the "least you need to know" for the not-necessarily-techie bosses out there. And hopefully the end result will be that you have one less dumb question to deal with in your life. So: any suggestions? (You can reply privately if you prefer.) Esther Schindler senior online editor, CIO.com her blog: http://advice.cio.com/taxonomy/term/34

2 3

[Dovecot] New full text search indexer
by Timo Sirainen 14 Nov '07

14 Nov '07

As described earlier (http://dovecot.org/list/dovecot/2006-December/018055.html) Dovecot nowadays has full text search indexing support in CVS HEAD. Currently there are two backends: Lucene and Squat. Lucene's problem is that standard IMAP SEARCH command can't be used with it without breaking IMAP RFC. So Lucene can be used only with non-standard X-BODY-FAST and X-TEXT-FAST search commands. Squat's code then is quite complex and it's pretty slow. It works with 4 character blocks, so it can answer: - Search string < 4 characters: Can't answer anything - Search string = 4 characters: Definite yes/no - Search string > 4 characters: Probable yes / definite no Since search strings are usually more than 4 characters, Dovecot uses the index only to filter out messages which definitely won't match and then does regular searching for the non-filtered messages. It works ok, but it could work better. The Squat indexes take maybe 30-50% of the mailbox's size. Most of the space goes to UID lists (list of messages matching each 4 character block). For example one UID list could be "2,4-10,20-30". This is already stored in a compressed format by storing the UID numbers relative to previous UID, which allows storing them usually with a single byte per UID, even if the UIDs are large. So, this morning I had an idea how this could maybe be improved with two changes: 1. Don't use 4 character blocks. Instead allow variable sized paths, and keep track of the UIDs for each node in the path. So for example adding "abc" to UID 1 will place the UID 1 to node "a", "ab" and "abc". 2. Use UIDs relative to parent's UIDs. For example if node "a" has UIDs 1,3,5 and "ab" is added for UID 5, the UID is changed to 2 (3th in 1,3,5 list - 1). This can help the compression a lot. The best case scenario for the above example is if "ab" contains also 1,3,5 UIDs. Then it can be written with a single 0-2 range. With these changes, the indexer can be configured to perform in different ways. Text can be indexed in two ways: full words, and substrings within those words (as required by IMAP). I think a useful combination of these would be to index all 4 character substrings and all full words up to 10 characters or so. This kind of an index can answer: - Search string <= 4 characters: Definite yes/no - For X-BODY-FAST / X-TEXT-FAST: - Search string 5-10 characters: Definite yes/no - Search string 11+ characters: Probable yes / definite no - For standard IMAP SEARCH: - Search string 5-10 characters: Definite yes if message contains the key as non-substring, otherwise same as below. This helps only if user does a search that matches a lot of messages. - Search string 11+ characters: Probable yes / definite no UTF-8 text is indexed as bytes, but the path depth can be counted as characters instead of as bytes. I've a test program finished, and unless there are some bugs left I think it shows that this can work pretty well: Squat-like 4 byte substrings (but can answer 1-3 char queries also): Message count: 7495 Index time: 4.19 CPU seconds (7.39MB/CPUs) Node memory: 3014072 bytes (2943 kB) in 90550 nodes UID memory: 8277212 bytes (8083 kB) in 352778 lists Total: 11291284 / 32474899 bytes = 34.77% Indexing the same mailbox file with squat-test gives: - Index time: 9.55 CPU seconds, 9.63 real seconds (3.24MB/CPUs) - 4145192 bytes in 102208 nodes (12.76%) - 7286058 bytes in 305501 uid_lists (22.44%) - 11431250 bytes total of 32474899 (35.20%) So the new indexer is over twice as fast and uses slightly less space. Indexing 4 char substrings and words up to 10 chars: Index time: 5.13 CPU seconds (6.04MB/CPUs) Node memory: 6002804 bytes (5862 kB) in 615501 nodes UID memory: 9044610 bytes (8832 kB) in 519488 lists Total: 15047414 / 32474899 bytes = 46.34% So it takes somewhat more space, but definitely less than having both Squat + Lucene. No substring indexing, words up to 10 chars (Lucene replacement): Index time: 1.39 CPU seconds (22.28MB/CPUs) Node memory: 3297663 bytes (3220 kB) in 548540 nodes UID memory: 1803735 bytes (1761 kB) in 213554 lists Total: 5101398 / 32474899 bytes = 15.71% Is there a need for Lucene anymore with this kind of a speed and disk space usage? :) The above tests are run with everything being indexed except space and control chars (ie. practically space + tab). If only alphanumeric characters are indexed, the total lines are (in same order as above): Total: 7048664 / 32474899 bytes = 21.70% (-13.07%) Total: 9610543 / 32474899 bytes = 29.59% (-16.75%) Total: 3963335 / 32474899 bytes = 12.20% (-3.51%) There are probably still a few optimizations that can be done to get the disk space usage even lower. I did spent quite a lot of time making it use less CPU though. Even tried a bit different coding style variations in the critical functions to get gcc to create faster code.. One interesting thing I noticed was that binary searching a character from a char array was slower than just sequentially going through the characters. I tried adding checks to do it only if the array was big, but it was always slower. Maybe something to do with CPU cache. If you want to try yourself, the test program is in http://dovecot.org/tmp/new-indexer.c

7 10