- dovecot - dovecot.org

[Dovecot] Memory leak by getting the virtual size of a IMAP folder
by Dafan Zhai 11 Sep '12

11 Sep '12

Hi everyone, I am writing a dovecot statistic plugin, which calls the 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as the 2nd parameter. enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; struct mailbox_metadata metadata; mailbox_get_metadata(mailbox, metadata_items, &metadata); but Valgrind finds a memory leak when this function is called: ---------------------------------------------------snip-------------------------------------------------------------- ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 of 76 ==10304== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10304== by 0x416EE9F: block_alloc (mempool-alloconly.c:237) ==10304== by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) ==10304== by 0x409C07D: mail_search_build_init (mail-search-build.c:187) ==10304== by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) ==10304== by 0x4067102: maildir_mailbox_get_metadata (maildir-storage.c:486) ==10304== by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) ==10304== by 0x490492D: mailbox_status_update (statistic-plugin.c:310) ==10304== by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) ==10304== by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) ==10304== by 0x41A104F: notify_save_finish (notify-storage.c:143) ==10304== by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) ---------------------------------------------------snip-------------------------------------------------------------- I have looked into the source code, and found that from the 'mail_search_build_init' function call in lib-storage/index/index-status.c:200 a pool is created, but the pool is not freed in the mailbox_search_deinit function call in lib-storage/index/index-status.c:218. This may be the reason of the memory leak. But I do not know how to free the pool. It seems to me that doveadm is having the same memory leak because it is using 'mail_search_build_init' too. ---------------------------------------------------snip-------------------------------------------------------------- # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize INBOX ==10457== Memcheck, a memory error detector ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX ==10457== INBOX vsize=41643319 ==10457== ==10457== HEAP SUMMARY: ==10457== in use at exit: 4,356 bytes in 3 blocks ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated ==10457== ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 ==10457== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x41AF16C: pool_alloconly_create (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x40DC07D: mail_search_build_init (in /usr/lib/dovecot/libdovecot-storage.so.0.0.0) ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in /usr/bin/doveadm) ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) ==10457== by 0x805B863: main (in /usr/bin/doveadm) ==10457== ==10457== LEAK SUMMARY: ==10457== definitely lost: 4,096 bytes in 1 blocks ==10457== indirectly lost: 0 bytes in 0 blocks ==10457== possibly lost: 0 bytes in 0 blocks ==10457== still reachable: 260 bytes in 2 blocks ==10457== suppressed: 0 bytes in 0 blocks ==10457== Reachable blocks (those to which a pointer was found) are not shown. ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes ==10457== ==10457== For counts of detected and suppressed errors, rerun with: -v ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) ---------------------------------------------------snip-------------------------------------------------------------- Dovecot version: 2.1.9 OS: Linux 3.0.30-dist i686 maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX I have applied the following patches from the dovecot 2.1 branch fixing memory leaks: http://hg.dovecot.org/dovecot-2.1/rev/7bdbca7b0913 http://hg.dovecot.org/dovecot-2.1/rev/963482677c0b It would be nice if you could give me some hints how I can address this issue? Thanks in advance! Greetings, Dafan

2 2

[Dovecot] dovecot proxy ssl-parameter from database?
by Daniel Funke 11 Sep '12

11 Sep '12

Hi List, I want to migrate my IMAP-users from an old Courier-IMAP-Server to a new dovecot-Server. The old server supports IMAP without and with TLS. The new one only with TLS. To users should be migrated one by one with changing some client parameters. To leave the servername of the imap server I want to use the dovecot proxy during migration. the proxy uses a sql db to look for the old or new host for every user. So far all works fine. My question is the TLS or noTLS connection from the client to the proxy. How can I make the ssl-parameter user-dependant? clients to the old courier server should work without or with tls through the proxy and clients to the new dovecot server should only work WITH TLS between Client and proxy. (After migration of all clients the proxy should be switched off an all clients connect directly to the dovecot server.) I tried with http://wiki2.dovecot.org/UserDatabase/ExtraFields and returned a field "userdb_ssl" ("required" for host dovecot, "yes" for host courier). But it doesn't work. A migrated user can still login without tls to the proxy and the dovecot backend. Any ideas? Daniel Here is my doveconf -n # 2.1.9: /opt/dovecot/etc/dovecot/dovecot.conf doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503) # OS: Linux 2.6.32-279.2.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login default_process_limit = 500 disable_plaintext_auth = no listen = * managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /opt/dovecot/etc/dovecot/sql.conf driver = sql } protocols = imap pop3 service imap-login { inet_listener imaps { port = 0 } } service pop3-login { inet_listener pop3s { port = 0 } } ssl_cert = </etc/pki/tls/certs/server.crt ssl_key = </etc/pki/tls/private/server.key userdb { driver = prefetch } verbose_proctitle = yes

2 1

[Dovecot] Alerts when process limits are met
by David Anderson 11 Sep '12

11 Sep '12

Hi, I've not found the answer to this question anywhere - please forgive me if I overlooked. I'd like to be able to be automatically alerted if process limits are hit (e.g. max POP3 logins). Is there a way that I can configure a script to be run, in the same way that I can with quota warnings? I can of course use logwatch, but this alerts me the next day, and logwatch can be noisy and it's easy to overlook. My motivation: for some reason my POP3 listener was hanging, and I had to restart dovecot (this is 2.0.9). But I was not aware - the process was still there, it was accepting connections (but not doing anything useful with them) and was actually managing to log the fact that the process limit was being hit (as more and more connections came in). If I could have been alerted I could have restarted dovecot earlier. Many thanks, David -- WordShell - WordPress fast from the CLI - www.wordshell.net

3 2

[Dovecot] ACL MySQL dict: to_user==from_user entries
by Roland Rosenfeld 11 Sep '12

11 Sep '12

Hi! We run dovecot 2.1.6 with ACL support and MySQL dict for shared/shared-boxes/user/$to/$from. When a user does a SETACL (via IMAP) to himself he is added as $to and $from to the MySQL dict. Se for example this IMAP conversation: 5 GETACL INBOX * ACL "INBOX" "user@domain" lrwstipekxacd 5 OK Getacl completed. 6 SETACL INBOX user@domain lrwstipekxacd 6 OK Setacl complete. 7 GETACL INBOX * ACL "INBOX" "user@domain" akxeilprwtscd 7 OK Getacl completed. As you can see, user@domain sets the ACL of his INBOX to the same value it was set before (reading it from the server after this idempotent write changed the order but none of the flags). But after "SETACL INBOX user@domain lrwstipekxacd" I see a new entry in my imap_acl_shares table where to_user and from_user both contain the value "user@domain". As far as I can see, this entry does not make much sense in the database, because every user should be able so see its own folders. But adding this line may blow up the database especially according to the full table scan problem for acl dict rebuild mentioned in http://www.dovecot.org/list/dovecot/2012-August/067548.html. Does it make sense to run a daily cron job, that removes all entries with from_user=to_user or may this cause other inconsistencies? If you need to have a look at my complete setup, it's in http://www.dovecot.org/list/dovecot/2012-July/067130.html, so I don't have to clone this here. Tschoeeee Roland

2 1

[Dovecot] imap_idle_notify_interval
by Andreas Meyer 11 Sep '12

11 Sep '12

Hi! I want to change the imap_idle_notify_interval t0 29 minutes and have edited the 20-imap.conf with imap_idle_notify_interval = 29 mins and restarted the server. doveconf -n shows me this: .... protocol imap { imap_idle_notify_interval = 29 mins mail_max_userip_connections = 10 mail_plugins = " quota imap_quota autocreate" } .... doveconf |grep idle shows me the following output: default_idle_kill = 1 mins imap_idle_notify_interval = 2 mins mailbox_idle_check_interval = 30 secs idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 4294967295 secs imap_idle_notify_interval = 29 mins I see two line of imap_idle_notify_interval. What is the right one? Greetings Andreas

2 2

[Dovecot] Is it possible to save only some messages compressed to mdbox? (2.1.6)
by Jost Krieger 11 Sep '12

11 Sep '12

I'm trying to save some messages as compressed (via zlib plugin) and it doesn't seem to work. Here's a log line (I'm logging to stderr): d k 1345467448.705591500 1345467448.705596500 1345467448.974808500 1650 <XXXXXXXXXXXXXXXXXXXXXXXXXX(a)YYY.ruhr-uni-bochum.de> local.jost.krieger+hirn(a)ruhr-uni-bochum.de 10909 80 Extra: -o plugin/zlib_save_level=9 -o plugin/zlib_save=bz2 lda(kriegjcb): Info: msgid=<20120820125728.58CC9172B6C(a)c2-4-7.noc.ruhr-uni-bochum.de>: saved mail to Hirn-2012-08 did 0+0+1 This probably doesn't make much sense to anyone except the last line but 1. It is a somewhat complicated qmail/maildrop/dovecot setup, which works for the standard cases. The Extra gets interpolated into the lda command and this works because an incorrect option blows up the delivery. The mails arrive uncompressed. So here's the questions: 1. Is it possible to have compressed/uncompressed mails mixed in an mdbox? 2. Will it work via "-o" in lda delivery? I tried to find out in the source but found it hard to see which options are pulled in when. Thanks Jost Krieger -- | Helft Spam ausrotten! HTML in Mail ist unhöflich. | | Postmaster, JAPH, manchmal Wahrsager am RZ der RUB | | Wahre Worte sind nicht gefällig, gefällige Worte sind nicht wahr.| | Lao Tse, Tao Te King 81 |

2 2

Re: [Dovecot] ChgrpNoPerm issue
by Joseph Tam 11 Sep '12

11 Sep '12

Robert JR <roundcube222(a)alaadin.org> writes: > The problem is /var/mail (Owner is > useraccount and the group us mail) , and here comes the problem > Dovecot > keeps trying to chown the imap folder inside the homedirectory with > user:mail account and since > And this is the reason of error that appears > in my log file. Dovecot is trying to keep the permission of your index caches consistent with that of your mailbox it indexes. The INBOX index cache is kept in your user's mail directory (as per your default settings), although you can change that location. > The option mail_access_groups=mail solve the problem.. > but I read it is not secure.. > With my current configutation, users login > with imap to read mail , can they use > any the mail_access_groups=mail > and read other poeple mail ? Yes, that's the security problem. > does my configutation > is a shard mailbox > and could be unsecure.. Could not parse your question/comment. > why didnot have > this issue with uw-imapd? and why dovecot try to chown the .imap > folder > with the mail group ? uw-imapd was not as picky. The extra consideration for group ownership is so that shared access to mailbox files, and their associated index caches, remain consistent. For example, if you shared a mailbox among your colleagues in group "staff" and the mailbox has group=staff,perm=g+rw, then the index caches will inherit those permissions, and members of group staff can access mailbox and indices alike. [You later write ...] > Sep 9 11:22:30 dovecot: pop3(r): Error: > fchown(/home/r/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) > failed: Operation not permitted (egid=501(r), group based on > /var/mail/r) > > i know that chmod 0600 /var/mail , will solve the problem > and i will no longer receive the above errors again You also have to make sure that autocreated INBOXs (i.e. a brand new account) does not start out with anything other than 0600. You may have to use dovecot's LDA or twist your LDA's arm to create mailboxes that way. I guess you can also avoid these errors by using memory indices, but you forego the advantages of persistent indices. > But my question is > that incase i did not set chmod /var/mail 0600, can i ignore such > errors, is these errors harmful ? if this errors keep coming and i > ignored them would this cause mbx corruption .. please advise No, you can't ignore these errors. They will break IMAP access to those mailboxes (as you will find out). Joseph Tam <jtam.home(a)gmail.com>

1 0

[Dovecot] precise reading of plugin deleted to trash
by ml 10 Sep '12

10 Sep '12

hello timo and other dev after a precise reading of plugin deleted to trash it appears to me like things more varariables are declared NULL in the code. it appears to me as being possible to achieve actions even if messages with the same id appears several times in different INBOX when it is the exact order of things -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation

1 0

[Dovecot] Chown ChgrpNoPerm problem
by Robert JR 10 Sep '12

10 Sep '12

Hello, I just have a one simple question. my logs has the following entries Sep 9 02:48:10 dovecot: pop3(r): Error: fchown(/home/r/mail/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=501(r), group based on /var/mail/r) Sep 9 11:22:30 dovecot: pop3(r): Error: fchown(/home/r/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=501(r), group based on /var/mail/r) i know that chmod 0600 /var/mail , will solve the problem and i will no longer receive the above errors again But my question is that incase i did not set chmod /var/mail 0600, can i ignore such errors, is these errors harmful ? if this errors keep coming and i ignored them would this cause mbx corruption .. please advise My dovecot conf is disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mymail:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } Thanks. Robert

1 0

[Dovecot] ChgrpNoPerm issue
by Robert JR 09 Sep '12

09 Sep '12

I have dovecot version 2.0.9 I am confused about this article http://wiki2.dovecot.org/Errors/ChgrpNoPerm my mail_location = mbox:~/mymail:INBOX=/var/mail/%u So my inbox is at var/mail/ and my imap folder is /home/user/mymail The problem is /var/mail (Owner is useraccount and the group us mail) , and here comes the problem Dovecot keeps trying to chown the imap folder inside the homedirectory with user:mail account and since And this is the reason of error that appears in my log file. The option mail_access_groups=mail solve the problem.. but I read it is not secure.. With my current configutation, users login with imap to read mail , can they use any the mail_access_groups=mail and read other poeple mail ? does my configutation is a shard mailbox and could be unsecure.. my main quesion , for my current configuration , can i use mail_access_groups=mail safely or can some user use this to read other people mail ? attached my configuration. why didnot have this issue with uw-imapd? and why dovecot try to chown the .imap folder with the mail group ? dovecot -n output is # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.2.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mymail:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } [ Robert

1 0