- dovecot - dovecot.org

[Dovecot] Large subjects increase memory-usage and enlarge index-files
by Peer Heinlein 08 Oct '12

08 Oct '12

Several times we already had the problems, that accounts with more the 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if vsize_limit of 750 MB is set. In this case, the lmtpd-process haven't been able to allocate more memory to read/write/update the index-files and crashed (and the index-files become corrupted at the end.) [Please -- don't discuss about the need of INBOXes with 1.7 million (unread) e-mails (don't discuss that with ME. Personally, I agree, that there's NO need for that...).] But: We also noticed accounts with ~ 300.000 e-Mails running out of memory in the same situations. This happends, if the subject is very large (subject or some other header attributes). And: We've been able to reproduce out-of-memory-Problems with just 13.000 e-mails with VERY long subjects (e.g.: network monitoring status informations), even with a vsize_limit of 750 MB (which is already very much). 13.000 e-mails isn't very much. And it's easy to inject several thousands of prepared e-mails. Having many mails for accounts with huge (and broken) index-files slows down the delivery rate VERY much and increases the need for memory and cpu resources and I/O very much. So: This could be used for a very easy to do denial-of-service attac against Dovecot-based mailservers. I don't have a clear solution for that, Dovecot needs the subject information in its index files. But it looks like, it isn't a good idea to put the whole subject into the index. Maybe it's better/necessary to use just the first 50-70 characters for that and to keep the rest away from the index? I think I would prefer that even if that means, that accessing those folders with "special" e-mails will become slower because Dovecot has to get those informations directly from the e-mail. This performance issue is just a problem for the user. But crashing lmtpd-processes and lowering the delivery rate is a *real* problem for the whole IMAP-cluster. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

4 4

[Dovecot] [OT] How do I convert maildir to bsmtp format?
by Marc Perkel 08 Oct '12

08 Oct '12

Here's what I'm trying to do. I have a spam filtering operation as a front end for other servers. I've created a virtual server for spam storage where the user will be able to log in using squirrelmail/dovecot to review and release their spam. The email is stored in maildir format. Piecing it together I can use squirrelmail to pipe the email into something so that if a use finds a false positive they can hit the "release" button and the message is sent on to the recipient. Squirrelmail sends the message as you would receive it as stored in maildir format. I need to take this format and translate it to send it on to the user. Wondering what is the best way to do that. Something that translated it into bsmtp format would be great. I'm sure thare must be something out there. I just haven't found it yet. Thanks in advance.

3 2

[Dovecot] Dovecot configuration and question about IP trusted
by Mik J 07 Oct '12

07 Oct '12

Hello list, I've just finished to install Dovecot and things seems to work so far. After some little efforts though. My version is 2.0.20 Question 1: I'm trying to tighten the security a little bit and added in dovecot.conf login_trusted_networks = 192.168.1.0/30 Then restarted Dovecot My client has the IP 192.168.1.20 and it's still able to retrieve emails. I expected it to be forbidden. Am I missing something ? # dovecot -n | grep trust login_trusted_networks = 192.168.1.0/30 Question 2: I feel that Dovecot is slow. I'm doing my test with my iphone as an imap client. Test 1: I retrieve a mail on a remote server provided by a hosting company, it takes 2 seconds Test 2: I retrieve a mail on my server which is on my LAN, the mail includes a few letters in the subject and a few letters in the body. The action takes about 8 seconds. It's quite subtule to measure so first I would like to know if Dovecot tries to do a dns reverse lookup or something like that. And it would explain the overhead. Thank you

2 4

[Dovecot] Getting rid of the subscription file
by Marc Perkel 06 Oct '12

06 Oct '12

I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? Thanks in advance.

4 5

[Dovecot] doveadm purge -A via doveadm-proxy director fails after some users
by Daniel Parthey 06 Oct '12

06 Oct '12

Hi, we have configured userdb and passdb in the director and try to iterate all users and pass the "purge" command via doveadm proxy to port 19000 on the correct director backend host. A single purge -u username(a)example.org via doveadm-proxy works correctly, but iterating over some users with -A fails. Note: users/domains have been anonymized in output: ------------------------------------------------------------------------ mail04:~# /usr/bin/doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A 2>&1 doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(user01(a)domain1.example.org): Debug: auth input: user=user01(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user02(a)domain1.example.org): Debug: auth input: user=user02(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user03(a)domain1.example.org): Debug: auth input: user=user03(a)domain1.example.org proxy host=10.129.3.192 proxy_refresh=86400 doveadm(user04(a)domain1.example.org): Debug: auth input: user=user04(a)domain1.example.org proxy host=10.129.3.192 proxy_refresh=86400 doveadm(user05(a)domain1.example.org): Debug: auth input: user=user05(a)domain1.example.org proxy host=10.129.3.190 proxy_refresh=86400 doveadm(user06(a)domain1.example.org): Debug: auth input: user=user06(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user07(a)domain1.example.org): Debug: auth input: user=user07(a)domain1.example.org proxy host=10.129.3.190 proxy_refresh=86400 doveadm(user08(a)domain1.example.org): Debug: auth input: user=user08(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user01(a)domain2.example.org): Debug: auth input: user=user01(a)domain2.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user09(a)domain1.example.org): Debug: auth input: user=user09(a)domain1.example.org proxy host=10.129.3.190 proxy_refresh=86400 10 / 94doveadm(user10(a)domain1.example.org): Debug: auth input: user=user10(a)domain1.example.org proxy host=10.129.3.190 proxy_refresh=86400 doveadm(user11(a)domain1.example.org): Debug: auth input: user=user11(a)domain1.example.org proxy host=10.129.3.191 proxy_refresh=86400 doveadm(user12(a)domain1.example.org): Debug: auth input: user=user12(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user13(a)domain1.example.org): Debug: auth input: user=user13(a)domain1.example.org proxy host=10.129.3.190 proxy_refresh=86400 doveadm(user14(a)domain1.example.org): Debug: auth input: user=user14(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user15(a)domain1.example.org): Debug: auth input: user=user15(a)domain1.example.org proxy host=10.129.3.191 proxy_refresh=86400 doveadm(user16(a)domain1.example.org): Debug: auth input: user=user16(a)domain1.example.org proxy host=10.129.3.191 proxy_refresh=86400 doveadm(user17(a)domain1.example.org): Debug: auth input: user=user17(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user18(a)domain1.example.org): Debug: auth input: user=user18(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user19(a)domain1.example.org): Debug: auth input: user=user19(a)domain1.example.org proxy host=10.129.3.192 proxy_refresh=86400 20 / 94doveadm(user20(a)domain1.example.org): Debug: auth input: user=user20(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user21(a)domain1.example.org): Debug: auth input: user=user21(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user22(a)domain1.example.org): Debug: auth input: user=user22(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user02(a)domain2.example.org): Debug: auth input: user=user02(a)domain2.example.org proxy host=10.129.3.190 proxy_refresh=86400 doveadm(user23(a)domain1.example.org): Debug: auth input: user=user23(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user24(a)domain1.example.org): Debug: auth input: user=user24(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user01(a)domain3.example.org): Debug: auth input: user=user01(a)domain3.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user25(a)domain1.example.org): Debug: auth input: user=user25(a)domain1.example.org proxy host=10.129.3.192 proxy_refresh=86400 doveadm(user26(a)domain1.example.org): Debug: auth input: user=user26(a)domain1.example.org proxy host=10.129.3.191 proxy_refresh=86400 doveadm(user27(a)domain1.example.org): Debug: auth input: user=user27(a)domain1.example.org proxy host=10.129.3.190 proxy_refresh=86400 30 / 94doveadm(user28(a)domain1.example.org): Debug: auth input: user=user28(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user29(a)domain1.example.org): Debug: auth input: user=user29(a)domain1.example.org proxy host=10.129.3.191 proxy_refresh=86400 doveadm(user30(a)domain1.example.org): Debug: auth input: user=user30(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user31(a)domain1.example.org): Debug: auth input: user=user31(a)domain1.example.org proxy host=10.129.3.193 proxy_refresh=86400 doveadm(user31(a)domain1.example.org): Error: doveadm server failure doveadm: Error: Failed to iterate through some users ------------------------------------------------------------------------ The user "user31(a)domain1.example.org" is proxied to the correct backend host according to director status, but the dovecot.log on the doveadm service backend host shows the following error: Jun 29 15:40:31 10.129.3.249 dovecot: doveadm(user31(a)domain1.example.org): Error: user user31(a)domain1.example.org: Error reading configuration: net_connect_unix(/var/run/dovecot/config) failed: Permission denied Jun 29 15:40:31 10.129.3.249 dovecot: doveadm(user31(a)domain1.example.org): Error: purge: User lookup failed: Internal error occurred. Refer to server log for more information. The wiki http://wiki2.dovecot.org/Services#doveadm states that the privileges are (temporarily) dropped to the mail user's privileges after userdb lookup. It seems that from the second purge on which is passed over a single doveadm connection, the user lookup fails. It also seems a bit strange, that the "-A" parameter can be observed in the doveadm tcp stream to the backend, since iteration should be already done in the director and the backend should purge only a single user: D username(a)example.org purge -A Is there a bug or have I misconfigured/overlooked something? Configs of mailbox backend and director are attached. Kind regards Daniel

2 8

[Dovecot] IPv6 & SSL
by Luigi Rosa 06 Oct '12

06 Oct '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I have a dual stack server with Dovecot 2.1.10 listening on v4 and v6 Dovecot has a Comodo SSL certificate issued via NameCheap that works as expected with IPv4 in 10-ssl.conf I have enabled these configuraction directives: ssl = yes ssl_cert = < /path/to/file.crt ssl_key = < /path/to/file.key ssl_parameters_regenerate = 202 hours If I connect to Dovecot using the IPv6 address of the server with Thunderbird 15.0.1 uising CRAM-MD5 averything is ok. If I enable SSL _and_ IPv6 on Thunderbird I get this error: Oct 5 20:05:04 mail dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=2001:470:1f09:203:fdbf:508e:4a29:56c5, lip=2001:470:1f09:203::badd:ecaf, TLS: SSL_read() failed: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, session=<ZcMRtlPLqgAgAQRwHwkCA/2/UI5KKVbF> Ciao, luigi - -- / +--[Luigi Rosa]-- \ I will tell you a great secret, Captain. Perhaps the greatest of all time. The molecules of your body are the same molecules that make up this station and the nebula outside, that burn inside the stars themselves. We are star stuff, we are the universe made manifest, trying to figure itself out. As we have both learned, sometimes the universe requires a change of perspective." --Delenn, "Distant Star", Babylon 5 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBvI50ACgkQ3kWu7Tfl6ZRBSACfRkp4FYpWaEZUQhIh0t6Vfs/I JbcAoKGZ769yogYS7faCXKvPTuhQiHA8 =jxCB -----END PGP SIGNATURE-----

5 8

[Dovecot] horde sync status ?
by Spyros Tsiolis 05 Oct '12

05 Oct '12

Hello all, I had a quick look at the horde site and noticed that horde is being advertised as, let's say, "smartphone friendly". Does anyone know if the newest horde version can "talk" to smart phones in regards to e-mails ? In other words, can a user owning a smartphone get his/her e-mails on it apart from the webpage ? Thank you, spyros ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis

5 5

[Dovecot] bug formatting results when using doveadm-server
by Angel L. Mateo 04 Oct '12

04 Oct '12

Hello, I have a infrastructure using director in front of backend servers. So I normally run doveadm commands in director servers, not the backends. The problem I have is when I try to concatenate commands like this: doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; done The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: e62e0d3834ed094e5c7900007efb8a67 66 e62e0d3834ed094e5c7900007efb8a67 71 e62e0d3834ed094e5c7900007efb8a67 74 e62e0d3834ed094e5c7900007efb8a67 75 e62e0d3834ed094e5c7900007efb8a67 77 e62e0d3834ed094e5c7900007efb8a67 78 e62e0d3834ed094e5c7900007efb8a67 so I can't postprocess correctly the output. I think it could be a bug in the communication between the doveadm client and server. PS: As the time of writing this email, I have realized that I could use fetch command directly. But, anyway, I want to report the problem/bug so it can be fixed.

2 6

[Dovecot] possible nfs issue
by Cor Bosman 04 Oct '12

04 Oct '12

Hi all, we've started receiving complaints from users that seemingly use more quota than they actually have. We noticed that these users have (in some cases many) .nfs files in their mailspool. Some of our admins checked their own dirs, and noticed them there as well. This could of course be unrelated to dovecot (kernel issue, netapp issue) but maybe somehow has an idea about if dovecot could cause this. This has been going on for at least a year, not really enough to notice before now. root@userimap1# find . -type f|grep -i .nfs ./cur/.nfs00000000003967ad003c0603 ./cur/.nfs000000000757b44b003be609 ./cur/.nfs00000000035e89bd003be60b ./cur/.nfs000000000796251c003be60c ./cur/.nfs000000000796251f003be60e ./cur/.nfs000000000262f9a1003be33a ./cur/.nfs00000000096513f3003be524 ./cur/.nfs0000000007962525003be60f ./cur/.nfs0000000003e7d8ab003be62b ./cur/.nfs00000000026f4fad003be50d ./cur/.nfs0000000000bdaeab003c0611 ./cur/.nfs0000000005da42c7003be525 ./cur/.nfs0000000003d74729003be526 ./cur/.nfs000000000229769e003be535 ./cur/.nfs000000000440969e003be516 With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. Anyone else with NFS mailspools seeing this? Cor

5 7

[Dovecot] LDA vs LMTP index files
by list＠airstreamcomm.net 03 Oct '12

03 Oct '12

In the docs it states that LDA " ...takes mail from anMTAand delivers it to a user's mailbox, while keeping Dovecot index files up to date." I am wondering if LMTP also interacts with the Dovecot index files and keeps them up to date?

3 3