December 2006 - dovecot

[Dovecot] FreeBSD 6.1-RELEASE/kqueue high CPU load
by Taras Savchuk 11 Mar '07

11 Mar '07

Hi to ALL! I have dovecot-1.0r7 installed on FreeBSD 6.1, using kqueue and Maildir (<20 mailboxes, < 0,5 Gb size). Periodically CPU load of imap processes increasing up to 60-80%. Is it normall behavior or not? May be somebody had such kind of problems? -- С уважением, Савчук Тарас ООО "Элантек" : Аутсорсинг ИТ, WEB-разработка +7 (495) 589 68 81 +7 (926) 575 22 11 http://www.elantech.ru info(a)elantech.ru

8 32

[Dovecot] Disconnects with proxies / new facts
by Michael Kress 06 Mar '07

06 Mar '07

Hi, I just tried the below with the current version 1.0 RC15 and I still got the Problem: telnet gives me the following output: -----8<-----8<-----8<-----SNIP-----8<-----8<-----8<----- +OK dovecot ready.u-ERR invalid command u s-ERR invalid command s e-ERR invalid command e r-ERR invalid command r -ERR invalid command u-ERR invalid command u s-ERR invalid command s e-ERR invalid command e r-ERR invalid command r _-ERR invalid command _ 1-ERR invalid command 1 -ERR invalid command -----8<-----8<-----8<-----SNIP-----8<-----8<-----8<----- The above output results from the following entry: user user_1[ENTER] So now I know that this is no version issue. Can you tell me something on how I can find the solution for this problem? I don't think its a firewall issue, I disabled my Desktop Firewall. On the server I use shorewall, but of course, port 110 is open and it works disabling AntiVir on the client side. The error also occurs when I disable the Server Firewall as well. The error is reproducible on several PCs (I tried 3 PCs). Have you got any hint? Without solving the problem, I have to switch the pop3 server to another one, because this problem is really a crucial one.The only problem is: I loke dovecot. :) Thanks in advance! Regards, Michael -------- Original-Nachricht -------- Betreff: [Dovecot] Disconnects with proxies Datum: Wed, 06 Dec 2006 22:43:00 +0100 Von: Michael Kress <kress(a)hal.saar.de> An: dovecot(a)dovecot.org Hi, I know I shouldn't ask stuff like this when using dovecot-0.99.11-4.EL4 (CentOS-4.4), but why I'm asking this is whether it's worth to switch from rpm based upstream dovecot to a self compiled one or not and thus I think you wouldn't kill me presenting you this question. o:-) I've got a problem with dovecot regarding a pop3 proxy issue. When I configure my dovecot server to do just plain pop3 I can telnet from a windows box without any flaw, but as soon as the client uses some pop3 proxy (for example outlook uses avira antivir Mail-AntiVirus Protection), queries to that server fail. A log line looks like this: pop3-login: Dec 05 22:53:24 Info: Disconnected [::ffff:84.165.54.42] Moreover with the proxy activated, a telnet to port 110 (pop3) is not working properly, the cursor stays right behind "+OK dovecot ready." and as soon as I enter a character, the server rejects this character as an unknown command (Without pressing ENTER). As soon as I turn off the Antivir-Proxy the telnet to port 110 runs flawlessly and I can login. So I'd like to know is there anything I can do with this version or is it worth trying the newest version? Is this a known issue? Thanks in advance Michael -- Michael Kress, kress(a)hal.saar.de http://www.michael-kress.de / http://kress.net P E N G U I N S A R E C O O L -- Michael Kress, kress(a)hal.saar.de http://www.michael-kress.de / http://kress.net P E N G U I N S A R E C O O L

4 12

[Dovecot] master request not found
by Marshal Newrock 18 Feb '07

18 Feb '07

I am setting up dovecot 1.0-rc15 as a proxy. When I try to log in, I get errors such as the following: Dec 4 17:39:23 vega dovecot: Dovecot v1.0.rc15 starting up Dec 4 17:39:24 vega dovecot: auth-worker(default): mysql: Connected to localhost (ad_mail) Dec 4 17:39:31 vega dovecot: auth(default): client in: AUTH 1 PLAINservice=POP3 lip=xx.xx.xx.xx rip=xx.xx.xx.xx resp=<hidden> Dec 4 17:39:31 vega dovecot: auth(default): client out: CONT 1 Dec 4 17:39:31 vega dovecot: auth(default): client in: CONT<hidden> Dec 4 17:39:31 vega dovecot: auth-worker(default): sql(XXXX,xx.xx.xx.xx: query: SELECT NULL AS password, host, destuser, 'Y' AS proxy FROM exchange_proxy WHERE user='XXXX' Dec 4 17:39:31 vega dovecot: auth(default): client out: OK 1 user=XXXX host=XXXX destuser=newrock.marshal(a)acd.net proxy pass=<hidden> Dec 4 17:39:31 vega dovecot: auth(default): master in: REQUEST 1 171931 Dec 4 17:39:31 vega dovecot: auth(default): Master request 17193.1 not found Dec 4 17:39:31 vega dovecot: auth(default): master out: NOTFOUND 1 Dec 4 17:39:31 vega dovecot: pop3-login: Internal login failure: user=<XXXX>, method=PLAIN, rip=xx.xx.xx.xx, lip=xx.xx.xx.xx dovecot.conf contains: auth default { mechanisms = plain login passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb static { args = uid=0 gid=0 } user = nobody } In dovecot-sql.conf: password_query = SELECT NULL AS password, host, destuser, 'Y' AS proxy FROM exchange_proxy WHERE user='%u' I have verified that I can log into the remote server from the proxy via pop3. Unfortunately, there isn't currently imap access to the remote server. I am connecting to the proxy using pop3 (with "telnet localhost 110"). Any suggestions what the problem could be? -- Marshal Newrock, Ideal Solution LLC http://www.idealso.com

2 4

[Dovecot] Dovecot "open files" (again?)
by falz 17 Feb '07

17 Feb '07

I'm a new dovecot user, just began using it last week. I switched from courier because my research showed that Dovecot did indexes, and allows me to store them anywhere I chose. I did this, and it fixed some severe disk IO issues that my server had. (25k horde only webmail install using imap proxy). I'm using quotas, and by default had it set to 'dirsize'. my postfix is already creating 'maildirsize' files, so I opted to change dovecot to use this instead. However, when I do, I run into the "too many open files" issue that's been reported many times on the list. I've seen a lot of mention of too many open files, but I haven't seen it related to the quota type used. It's possible that simply my using 'dirsize' as the quota type slows things down enough to not have the issue? I don't really know. Anyhow. I doubled my max open files in FreeBSD's /boot/loader.conf to: kern.maxfiles="24656", but to no avail. When I have it set to maildirsize, the open files on the system grows uncontrollably. Under normal use, it's somewhere below 5k: sysctl -a | grep openfiles kern.openfiles: 4415 I looked into the patch mentioned here: - http://www.dovecot.org/list/dovecot/2006-November/017536.html - http://dovecot.org/list/dovecot-cvs/2006-November/006776.html Those patches already appear to be in rc15, which I'm already running (Via FreeBSD's ports). I also applied the patch mentioned here awhile back: - http://www.dovecot.org/list/dovecot/2005-June/007647.html and rebuilt, but it didn't help out either. There are also freebsd specific notes on forcing a limit on dovecot itsself using login.conf, but that wouldnt really fix anything, it would just cause dovecot to not interfere with other running daemons. - http://www.mail-archive.com/ports@openbsd.org/msg08646.html Any thoughts? TSS, you mention a few times in these notes that you're unable to reproduce this. Perhaps try with the maildir as quota? More details about my setup: - FreeBSD 4.8 (yea, I know) - Imapproxy (http://www.imapproxy.org/) - Postfix to create maildir++ maildirsize folders - MySQL backend The work of this is spread out between several servers, dovecot is running on the server that's the final SMTP delivery location, and has local disks with the maildirs. --falz

2 5

[Dovecot] Problem/workaround with X-Keywords and Apple's Mail.app
by Joey Coleman 17 Feb '07

17 Feb '07

Hello, I've just switched from UW-IMAP to Dovecot, and for the most part, things are good. However, I encountered the problem with the X-Keywords header not being written back into individual messages in an mbox file (originally described in [1], and Timo gave a clue as to maybe why in [2]). What was happening for me is that Apple's Mail.app would flag a message as spam, which happily sets a couple keywords in the message, but those weren't being written back (at least, I didn't see them in the file on the server). The "DO NOT DELETE..." message had a perfectly fine X-IMAP header, and up until I switched things were working great. The user visible symptom of this happened every time new spam arrived in my Junk folder [3] --- the new spam would arrive, be flagged as spam by Mail.app, and the earlier pieces of mail would promptly be unflagged! Anyway, Timo's clue in [2] prompted me to turn off both mbox_lazy_writes and mbox_dirty_syncs (the latter just on the off- chance... it's a low-volume server, so I can deal with the inefficiency). Lo and behold, Mail.app started acting sane again *and* the X-Keywords header appeared immediately. Oddly, during all of this, the Status and X-UID keywords would appear in the messages just fine; no clue as to why those and not X- Keywords. Version details are: - Dovecot 1.0rc13 [4] - FreeBSD 5.3-RELEASE - Apple Mail.app 2.1 (752/752.2) cheers, --joey [1] http://dovecot.org/list/dovecot/2006-March/011872.html [2] http://dovecot.org/list/dovecot/2006-June/014126.html [3] And it felt really strange while troubleshooting this as I found myself *hoping* for another piece of spam to arrive. Bleh. [4] Yes, two versions behind current, but in my defense, I did check the announcements and they didn't seem to mention the problem.

2 1

[Dovecot] again... cross compiling
by Martin Höfling 17 Feb '07

17 Feb '07

> Create a cache file with the right values, or call configure with those > values set in the environment. OK, i ran configure with -C one time without cross compiler env set, to obtain the variables another time with -C to use the config.cache . I could not find anything related to the check whether size_t is signed. I found a variable which i set that way: ignored_signed_size=${ignore_signed_size=1} What's wrong with that? Cheers Martin P.S.: Is there a way to obtain all variables in an easy way, or to obtain the variable that has to be set by hand? -- Frag' nicht - Du könntest eine Antwort erhalten. (Markus Mehring in d.a.arnooo)

3 2

[Dovecot] Proxy with MasterUser + Authentication issues
by Cassidy B. Larson 16 Feb '07

16 Feb '07

Per Timo's direction, I decided to give the MasterUser a try in connecting my proxy to my destination server. This might allow for the use of 'secure password' for my clients that like to check that box by default. A couple of problems I'm running into running RC15: #1: My Proxy's SQL password_query line: password_query = SELECT a.clearpasswd AS password, v.storeIP AS host, CONCAT(v.userID,'*someuser') AS destuser, 'Y' AS nologin, 'Y' AS nodelay, 'Y' AS proxy, 'somepass' AS pass FROM iwmailsystem.virtmailbox AS v NATURAL JOIN authenticate.users AS a WHERE v.userID = '%u' and v.imap_active = 1 I'm wanting to authenticate the user on the proxy before I send it off via the MasterUser to the destination server. However, I show this in the logs: Info: auth(default): client in: AUTH 1 PLAIN service=IMAP lip= x.y.z.a rip=a.b.c.d resp=AGJ1dGNoQGluZm93ZXN0LmNvbQBzZWNyZXQ= Info: auth-worker(default): sql(user(a)host.com,a.b.c.d): query: SELECT a.clearpass wd AS password, v.storeIP AS host, CONCAT(v.userID,'*someuser') AS destuser, 'Y' AS nologin, 'Y' AS nodelay, 'Y' AS p roxy, 'somepass' AS pass FROM iwmailsystem.virtmailbox AS v NATURAL JOIN authenticate.users AS a WHERE v.userID = 'user(a)host.com' and v.imap_active = 1 Error: auth(default): file auth-request.c: line 207 (auth_request_save_cache): assertion failed: (extra_fields == NULL || (strstr(extra_fields, "\tpass=") == NULL && strncmp(extra_fields, "pass=", 5) != 0)) Error: child 54122 (auth) killed with signal 6 The Wiki showed I should return a 'pass' column. However, that's choking the query up in auth-request.c. Could it be the "\tpass" from auth-request.c? If I change user(a)host.com's password to 'someass' and remove the 'pass' column from the password_query, it authenticates, and proxy's over just fine. Am I doing something wrong? I want to authenticate the user locally on the proxy and pass off the 'master password' to the proxy, but using 'pass' doesnt work. 'password' column does. Doesnt help me though for my situation :) Problem #2: I'm wanting to put the masteruser/pass in the database for quick retrieval and updating. So I setup dovecot.conf as: passdb sql { # Path for SQL configuration file, see doc/dovecot-sql.conf for example args = /usr/local/etc/dovecot-sql.conf master=yes #pass=yes } passdb passwd { } and dovecot-sql.conf with: password_query = SELECT '%u' AS user, 'master-password' AS password Trying to connect locally with a "telnet localhost 143" via: 1 login user(a)host.com*someuser "somepass" Give me this in the detail log file: Info: auth(default): client in: AUTH 1 PLAIN service=IMAP lip= a.b.c.d rip=w.x.y.z resp=AGJ1dGNoQGluZm93ZXN0LmNvbSphZG1pbkBpbmZvd2VzdC5jb20AbWFzdGVyLXBhc3N3b3Jk Error: auth-worker(default): BUG: PASSV had invalid passdb ID Warning: Killed with signal 15 I am able to successfully authenticate with the master user/pass if I store it in a passdb file. However, I'd prefer to have it in the database for obvious reasons. Any help to the above would be appreciated. thanks, Cassidy

2 6

[Dovecot] Changing the Seen Flags
by Amardeep Singh 15 Feb '07

15 Feb '07

Hi , Can we change the setting on dovecot so that the Seen mail flag S should be saved where everybody can see that and not in the dovecot.index file As we are facing problems with public folder read/unread mails. Any help is appreciated Thanks, Amardeep

3 5

[Dovecot] Shared folders plans for the future
by Timo Sirainen 11 Feb '07

11 Feb '07

So while driving for several hours in my car recently and having nothing else to do, I started thinking how they could be done. I think the "public folders" configuration will stay as it is now. Does it really even need anything else? The problematic case is how the users can share their mailboxes to other users. There are two problems related to it: 1) How to get (quickly) a list of another user's mailboxes that I have access to? 2) How to get quickly a list of all users who have mailboxes that I have access to? The 1) problem is easier to solve. Create a directory listing all the mailboxes that are shared to at least someone (have ACLs set). It could be configured like: namespace shared { # IMAP namespace prefix, %u expands to shared user prefix = user/%u/ # Directory containing symlinks to user's all shared mailboxes share_dir = ~/Maildir/shared # Where to store indexes for other users' accessed mailboxes index_dir = ~/Maildir/others-indexes } The ~/Maildir/shared directory could then contain symlinks to all the shared mailboxes. The directory list could probably be in Maildir++ layout even though the symlinks would point to mboxes. This requires adding code that can handle mixed maildir/mbox directories, but it'll come anyway.. The 2) problem is more difficult. There are 3 choices: a) Use a shared database file. lib-dict could be used for this, so the database could be in SQL or in a berkeley db file. The downside to this is that the file needs to be readable and writable to everyone, so if users have shell access the users could break the file. They dictionary would contain data in format "<destination_user>/<sharing_user>" = 1 and "<destination_group>/<sharing_user>" = 1, ie. the value would be ignored and only the existence of the keys would be important. Then iterating through "<my_username>/*" and "<all_my_groups>/*" would list the users who probably have mailboxes where I have access to. b) Use a directories with sticky bit set. Well, basically the same idea as in a), except filesystem would be used. Actually perhaps this could be made another lib-dict backend.. c) Brute force: Just assume everyone potentially have shared mailboxes. Get the list of users by readdir()ing through /home (configurable). I guess each of these could be made an option.. So something like: namespace shared { .. # a) shared_dict = db:/var/lib/dovecot/shared-mailboxes.db # b) shared_dict = dir:/var/lib/dovecot/shared-mailboxes # c) shared_dict = bruteforce:/home } I'm currently also implementing a "mailbox list index", which contains a list of all the user's mailboxes and some useful metadata for them, such as number of messages and so on. So that STATUS command wouldn't need to open the mailbox index files if they haven't changed. The mailbox list index updating is normally done when directory's mtime has changed. For list of shared users that would also work in 2b) and 2c) cases, but for 2a) I'd either have to always sync everything or alternatively add some sequence/<dest_user/group>=# value which is grown for each update, and the syncing is done only when the sequence is updated.

6 17

[Dovecot] duplicate emails
by Alex 07 Feb '07

07 Feb '07

i'm working with dovecot-1.0.rc15 I have receive a lot of duplicate emails with microsoft outlook, i don't have this problem with outlook express This is in pop3

10 36