- dovecot - dovecot.org

[Dovecot] Clients hang because imap-login discards post-login commands
by Mike Abbott 22 May '08

22 May '08

Some of my IMAP clients hang when connecting to Dovecot 1.1.rc5. I believe the problem is that imap-login reads eagerly rather than sparingly. If a client sends a login user password b select Inbox all at once, without waiting for the login reply before sending the select, imap-login eats the second command (reads it from the socket and does nothing with it; system call tracing confirms this) and the client hangs waiting for a reply to the select command. I can reproduce this simply using telnet to the dovecot on localhost, as long as I send both lines at once (I paste them in together). $ telnet localhost 143 * OK Dovecot ready. a login user password b select Inbox a OK Logged in. <dovecot never replies to select> This may also be true with pop3-login, I have not checked. It does happen (with IMAP) when authentication is delayed and Dovecot responds with "* OK Waiting for authentication process to respond.." but I believe that shares the same underlying cause as the simpler case above. I believe the login processes should read only as much as they need to satisfy the login request and leave subsequent commands in the socket for post-login processing. Or, the login processes should pass the remainder of their input buffers to the mail processes so no commands are dropped. This section of RFC 2060 explains why I think this: 5.5. Multiple Commands in Progress The client MAY send another command without waiting for the completion result response of a command, subject to ambiguity rules (see below) and flow control constraints on the underlying data stream. Similarly, a server MAY begin processing another command before processing the current command to completion, subject to ambiguity rules. [...] The exception is if an ambiguity would result because of a command that would affect the results of other commands. Clients MUST NOT send multiple commands without waiting if an ambiguity would result. If the server detects a possible ambiguity, it MUST execute commands to completion in the order given by the client. So it appears that both the client and Dovecot may be in violation, depending on how you define "ambiguity." The RFC does not explicitly define whether login causes an ambiguity, but I personally believe that login does "affect the results of other commands" (the login may fail, for instance). So the client should not issue commands before receiving the login reply, and Dovecot should not drop commands during login. I have reported the client issue to the developers of the IMAP client, as I am reporting the server issue to you. Fortunately for me, only one or the other needs to change to erase the symptom (hung clients), but fixing both would be best. Thanks for your attention.

3 2

[Dovecot] Dovecot or postfix Quota
by Adrián Ribao Martínez 22 May '08

22 May '08

Hello, I'm a bit confused with the dovecot quota support. What is it for? I thought postfix should be the one managing the quota. If I set up a quota in dovecot, does dovecot keep the messages size under certain limit? What happen when you reach the quota limit? What's the difference between postfix quotas and dovecot quota? I'm confused about this. Thank you! -- Adrián Ribao Martínez

3 4

[Dovecot] VMware and Time moved backwards
by Robert Henjes 22 May '08

22 May '08

Hi, I followed the discussions regarding the "time moved backward" problem and the use of ntp in such cases. At our department we are running two dovecot servers within an vmware server environment, and unfortunately the timedrift (with ntpd active) exceeds sometimes up to 30 minutes virtual drift within 10 minutes realtime (mostly into future). This is due to some overcorrections within the TSC algorithms of the vmware virtual machine. More information and some hints to workaround are documented here: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=di… Nevertheless we are currently working on evaluating a stable solution by doing some system measurements, which would be the best option to cope with this problem, since the XEN environment seems to have a similar issue. Running "ntpdate -u" as a cronjob is not an option due to two facts: a) Time is moving quickly and may cause "major problems" ;) b) It is not recommended by the vmware team itself as well We are currently considering two options: a) Using the "clock=pit" kernel option, which may cause the system to be to slow b) using vmware tools and use only the ntp synchronisation of the host (we have currently only little experience with this). Also vmware tools are somewhat critical in case of updates. So our intention was to use as less vmware specific things as possible within the virtual machine, so only our host itself depends on vmware specific software. The question is, which option would you prefer? Is there another solution beside the mentioned ones? According to Timo's postings, workarounds within the dovecot itself to cope with "time moved backward" problems are not planned yet to be impelented. So best regards, Robert

5 5

Re: [Dovecot] VMware and Time moved backwards
by Robert Henjes 21 May '08

21 May '08

Thank you for the prompt answers @Luuk Vosslamber >finding correct values for the following entrys in your vmware(host) >config file might solve some things: >host.cpukHz = 1596000 <== depends on processor speed ;-) >host.noTSC = TRUE >ptsc.noTSC = TRUE >hostinfo.noTSC = TRUE >tools.syntime = TRUE > >Above values go with the presumption that the host CPU is running at a >fixed frequency. The variable frequency seems to be the origin of this >problem.. We tried your parameters already, but they are not an option, unless you have the following a) machine smp disabled (enabled in our case) b) other machines on the host system have really low load c) vmware tools should not be missing on guest @Peter Hessler >never ever ever run ntp on virtual hardware. > >instead, run ntp on the host hardware, and tell the client to always obey the bios clock. I add "* * * * * /sbin/hwclock --localtime -->hctosys" >to my crontab for that. Thank you very much Peter, this hint opens up a bunch of new possibilities only mentioning "adjtimex" ... BUT: And this brings me back to, why I posted this issue to the dovecot mailinglist and not elsewhere: All these guest based solutions lead to a fixed time movement (we used previously ntpdate -u every 7 minutes, ntp itself synchronizes every 11 minutes with the hardware clock for calculating drifts), which happens about once per week, that the time movement backward is larger than 11 seconds between two dovecot imap operations, which leads to an immediate shutdown of the dovecot process. All other services on these machine can cope with this. So the question is how to get these adjustments smooth or close to realtime. Best regards, Robert Btw.: We are running Dovecot 1.0.13 from Debian etch-backports (which is an important information since debian seems to have its own rules ;) )

1 0

[Dovecot] Maildir locking by LDA of dovecot
by Mike Grozak 21 May '08

21 May '08

Hi all! First of all, I want ti thank the whole Dovecot community (developers for developing and fast responses, users for populating and responses too). Dovecot is a really good and fast IMAP server - it serves near 3000 in our installation. Now I have a problem and request community's help its resolving;) Dovecot version: 1.0.13, compiled with vpopmail support. I have an account spam(a)domain.com (:-), in which we put all our spam (we gather statistic later). There can be 10 and more delivery attempts in a second to maildir of this account during the spam attacks on our server. The queue of our server can grow up to 20k messages, and all of these messages are sent to spam(a)domain.com. I found a lot of such strings in dovecot.log file during the such DoS attack: dovecot.log:deliver(spam@domain.com): May 16 04:18:48 Info: msgid=<01c8b6da$a8bacf80$b7af2abe@sale>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:18:59 Info: msgid=<6611811842.20080516012444(a)poiusadfhnvxzmdiasdasgkhlsdkfg.com>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:20:06 Info: msgid=<01c8b6a8$7b552e00$a0a4034c@sale>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:20:14 Info: msgid=<360825920.32055246029506(a)mirgames.ru>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:20:50 Info: msgid=<01c8b6b1$0df9cb00$cc36dfc9@tek2>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:21:00 Info: msgid=<087901c7ca2d$ef272640$1e01a8c0(a)telefonia.InterCable.net>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:21:01 Info: msgid=<000701c8b6d2$01ff999f$c491aca7@snoueiij>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:22:32 Info: msgid=<000801c8b69b$060c0230$3cd8a4b7@cpbvlrx>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:22:45 Info: msgid=<000801c8b6a5$04ba6412$9910848d@aldldr>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:22:45 Info: msgid=<031401c772e2$adc5be40$1e01a8c0(a)mtu-net.ru>: save failed to INBOX: Timeout while waiting for lock dovecot.log:deliver(spam@domain.com): May 16 04:23:08 Info: msgid=<01c8b6db$0222fe80$b7af2abe@info>: save failed to INBOX: Timeout while waiting for lock My dovecot.conf: $ cat /etc/dovecot.conf | grep -v \# | grep -v ^\n protocols = imap pop3 log_path = /var/log/dovecot.log ssl_disable = yes login_user = dovecot login_process_per_connection = no login_max_connections = 1024 login_greeting = Server is ready. >>dotlock_use_excl = yes - can it be the reason? fsync_disable = yes first_valid_uid = 39 last_valid_uid = 39 first_valid_gid = 39 last_valid_gid = 39 max_mail_processes = 1024 maildir_copy_with_hardlinks = yes maildir_copy_preserve_filename = yes protocol imap { mail_plugins = quota imap_quota login_greeting_capability = yes imap_client_workarounds = outlook-idle } protocol pop3 { pop3_lock_session = no pop3_uidl_format = %08Xu%08Xv mail_plugins = quota } protocol lda { postmaster_address = postmaster(a)tversu.ru mail_plugins = quota cmusieve sendmail_path = /usr/lib/sendmail fsync_disable = no } auth_cache_size = 10240 auth_username_format = %Lu auth default { mechanisms = plain login cram-md5 socket listen { master { path = /var/run/dovecot/auth-master mode = 0666 } client { path = /var/spool/postfix/private/auth mode = 0666 user = postfix group = postfix } } passdb vpopmail { args = cache_key=%u dovecot } userdb vpopmail { } user = vpopmail ssl_require_client_cert = no } dict { } plugin { quota = maildir:ignore=Trash } What can I do for resolving this locks? May be, while looking into uidlist file, can dovecot LDA lock it? And other deliver processes can't open it and wait for him? --------------------------- WBR, Mike Grozak, TvSU IC

1 0

[Dovecot] Running sieve filters on already delivered mails
by Axel Gembe 20 May '08

20 May '08

I want to know what the best way to run sieve filters on already delivered mail is. Problem is that I have a rather large unfiltered mailbox and filtering it by hand would be a PITA. I don't mind coding something up to do this, but I just don't think that I'm the first one with this problem. Thanks - Axel

2 2

[Dovecot] Cannot compile current version of antispam
by Steffen Kaiser 20 May '08

20 May '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, with the current (2008-05-20) repository releases of Dovecot-1.0 and antispam the plugin no longer compiles: antispam-storage-1.0.c: In function âantispam_mail_update_keywordsâ: antispam-storage-1.0.c:285: error: âidxkwd__typeâ undeclared (first use in this function) antispam-storage-1.0.c:285: error: (Each undeclared identifier is reported only once antispam-storage-1.0.c:285: error: for each function it appears in.) antispam-storage-1.0.c:285: warning: type defaults to âintâ in declaration of âtype nameâ antispam-storage-1.0.c:285: warning: initialization from incompatible pointer type Version of antispam: (first entry of git log) commit c3a343290773d6e22a08946cb2ef80b1b8fe8a06 Author: Johannes Berg <johannes(a)sipsolutions.net> Date: Sat Apr 26 20:34:06 2008 +0200 add 1.1 storage. probably doesn't work Base version of Dovecot: changeset: 5551:71c02fdf1b59 branch: branch_1_0 tag: tip Bye, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIMtunVJMDrex4hCIRAjXgAJ9j0oNUa0wGaLMshw6F/kjQoQxvygCdHLE9 Vb7rUcHxSr1kcgN+MHCsI1U= =dBuy -----END PGP SIGNATURE-----

2 2

[Dovecot] New ManageSieve releases: v0.10.2 for Dovecot 1.1.rc4 and v9.3 for Dovecot 1.0.13
by Stephan Bosch 20 May '08

20 May '08

Hello Dovecot users, Recently, I was pointed towards a problem in the managesieve SASL implementation. As it turns out there was a bug in handling mechanisms that expect more than a single client response. That is why the PLAIN mechanism works and the (obsolete) LOGIN mechanism fails (along with probably many others). This problem was present in both v9.2 and v0.10.1 for Dovecot 1.0 and 1.1 respectively. This issue is fixed in the new releases: Change Log v9.3/v0.10.2 ------------------------ * Fixed bug that caused SASL mechanisms that require more than a single client response to fail. Reported by Steffen Kaiser and occured when he tried using the (obsolete) LOGIN mechanism. * Updated installation and configuration documentation to match the information provided in the wiki Installation ------------ The full installation procedure and other relevant information regarding ManageSieve is finally available on the Dovecot wiki at: http://wiki.dovecot.org/ManageSieve New releases: 1.0: (patch) http://www.rename-it.nl/dovecot/1.0/dovecot-1.0.13-MANAGESIEVE-v9.3.diff.gz http://www.rename-it.nl/dovecot/1.0/dovecot-1.0.13-MANAGESIEVE-v9.3.diff.gz… 1.1: (package + patch) http://www.rename-it.nl/dovecot/1.1/dovecot-1.1-managesieve-0.10.2.tar.gz http://www.rename-it.nl/dovecot/1.1/dovecot-1.1-managesieve-0.10.2.tar.gz.s… http://www.rename-it.nl/dovecot/1.1/dovecot-1.1.rc4-managesieve-0.10.2.diff… http://www.rename-it.nl/dovecot/1.1/dovecot-1.1.rc4-managesieve-0.10.2.diff… (my public key (id: 3DFBB4F4) can be found at wwwkeys.pgp.net) Have fun testing the Dovecot ManageSieve service. Don't hesitate to notify me when there are problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl IRC: Freenode, #dovecot, S[r]us

2 1

[Dovecot] Looking for suggestions: How to strip attachments from mails
by Patrick Nagel 20 May '08

20 May '08

Hi, I know this is not 100% on-topic, but I'll try anyway... I'm looking for a way to strip attachments from incoming mails on the server. The attachments should be stored on a file server (where users have access via SAMBA) and only a link to the file (UNC path) should remain in the mail body. This is to prevent the mail storage from filling up with a large amount of big mails where at the same time the 'payload' (attached file) is not conveniently accessible by the users. The current setup is postfix / dovecot deliver / dovecot (with sieve plugin). I'm considering piping the mails through something like mimeStrip.pl (http://freshmeat.net/projects/mimestrip.pl/) but therefore I'd probably need to bring procmail/maildrop into the game, right? Or are there other (better) ways to accomplish this? Patrick. -- STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/ Phone: +86 (21) 5427 7799 x 826 Fax: +86 (21) 6485 0071 PGP key: https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005

3 3

[Dovecot] RFC 3516 - IMAP4 Binary Content Extension
by Anton Yuzhaninov 20 May '08

20 May '08

Is Binary Content Extension (rfc3516) support planned? -- WBR, Anton Yuzhaninov

2 3