- dovecot - dovecot.org

[Dovecot] mailbox_list_index, stops showing new mails in mailboxes
by Anton Lundin 22 Sep '13

22 Sep '13

Hello. I'm having some weird problem with (probably) mailbox_list_index, that it doesn't see new mail in mailboxes. I'm using 2.2.4 over imap and ssh/imap, and after a while dovecot stops noticing new mail in some folders. Its always the same 2-3 folders of about 30. I read something about list-index corruption in 2.2.2, and i thought it was that i was running into earlier, but this problem still occurs. This is dovecot 2.2.4, running on Solaris 10 x86_64 to homedirs on zfs, where procmail is used to deliver mail to mailboxes in mbox-format. I've tried to purge all indexes and let dovecot rebuild them, but the problem reoccurs after a couple of days. How can i help to debug this issue? Should i grab dovecot.list.index when its out-of-sync for debugging? //Anton -- Anton Lundin +46702-161604

2 1

[Dovecot] verbose_proctitle cuts dsync to proctitle to "dsyn"
by Sven Hartge 22 Sep '13

22 Sep '13

Hi Timo, little cosmetic bug report: using "verbose_proctitle = yes" shortens the proctitle of dsync to just "dsyn". Example: Note the whitespace instead of the 'c' in the proctitle: root@m-st-01:~# ps auwwwx | grep [d]syn virtmail 18141 13.0 0.1 49916 7952 pts/6 R+ 00:49 0:00 dsyn -v -u xxyyzz7 -R backup maildir:/staging/xxyyzz7/Maildir With "verbose_proctitle = no" the output looks like this: root@m-st-01:~# ps auwwwx | grep [d]syn virtmail 18347 78.6 1.6 127356 68884 pts/6 D+ 00:53 0:02 dsync -v -u aabbcc7 -R backup maildir:/staging/aabbcc7/Maildir Version of dovecot is 2.2.5 from the autobuilder (but rebuild locally): root@m-st-01:~# doveadm config -n # 2.2.5 (ec62ea3c940e): /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.1 [...] Grüße, Sven. -- Sigmentation fault. Core dumped.

2 1

[Dovecot] Problem with MySQL virtual users and dsync
by Jordan Cook 22 Sep '13

22 Sep '13

Hello, I am using dovecot 2.2.5 on FreeBSD and I am having trouble using dsync with SSH. Everything on the server seems to be working fine (I can login, receive and send emails etc). I am using SQL for both passdb, and userdb and this seems to work fine. The 'doveadm user' command returns all the correct information for the user. I am getting the following errors in my log file: Aug 27 21:34:17 server dovecot: replicator: Error: userdb lookup: Connecting timed out Aug 27 21:34:17 server dovecot: replicator: Error: userdb lookup: Disconnected unexpectedly Aug 27 21:34:20 server last message repeated 173381 times It did seem to do an initial sync, but after that nothing is being synchronised. Has anyone else solved this problem previously or is able to help? Regards, Jordan Cook -- Packet House Limited is a company registered in England and Wales. Registered number: 07602377. Registered office: 40 Preston Avenue, Alfreton, Derbyshire, DE55 7JY This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Packet House Limited. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.

2 1

[Dovecot] Error: file_dotlock_open() failed with file + Too many open files
by Kavish Karkera 22 Sep '13

22 Sep '13

Hi All, We are getting these below error on our IMAP server. is this because of open file limits.?? We have set it to 4096. [root@vish conf.d]# ulimit -n 4096 Aug 27 16:08:26 imap(kavish.karkera(a)example.com-24685): Error: file_dotlock_open() failed with file /data/quarantine/index//data/quarantine/mailstore/24685/.INBOX.kavish.karkera/dovecot.index.cache: Too many open files Aug 27 16:08:26 imap(kavish.karkera(a)example.com-24685): Error: open(/data/quarantine/index//data/quarantine/mailstore/24685/.INBOX.kavish.karkera/dovecot.index.cache.lock) failed: Too many open files Below is our dovecot conf file. ====================================================================== # 2.2.5: /usr/local/dovecot-2.2.5/etc/dovecot-imap/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 CentOS release 5.5 (Final) auth_socket_path = /usr/local/var/run/dovecot2.2-imap/auth-userdb base_dir = /usr/local/var/run/dovecot2.2-imap/ default_vsz_limit = 1 G first_valid_gid = 99 first_valid_uid = 99 hostname = log_path = /var/log/dovecot2.2-imap.log mail_location = maildir:~:INDEX=/data/quarantine/index/%h:CONTROL=/data/quarantine/index/%h mail_plugins = " virtual notify" namespace { inbox = no location = prefix = separator = . type = private } namespace { hidden = yes inbox = yes list = yes location = virtual:~/virtual:LAYOUT=maildir++:INDEX=~/virtual:CONTROL=~/virtual prefix = virtual. separator = . subscriptions = no type = private } namespace { hidden = yes inbox = no list = no location = maildir:~:INDEX=/data/quarantine/index/%h:CONTROL=/data/quarantine/index/%h:LAYOUT=maildir++ prefix = boxvirtual. separator = . subscriptions = no type = private } passdb { args = /usr/local/dovecot-2.2.5/etc/dovecot-imap/dovecot-sql.conf.ext driver = sql } protocols = imap service auth { unix_listener auth-userdb { group = nobody mode = 0666 user = nobody } } service imap-login { inet_listener imap { port = 20143 } vsz_limit = 1 G } service imap-postlogin { executable = script-login -d /usr/local/bin/dovecot.virtual.sh unix_listener imap-postlogin { mode = 0666 } } service imap { executable = /usr/local/dovecot-2.2.5/libexec/dovecot/imap imap-postlogin vsz_limit = 1 G } ssl = no userdb { args = /usr/local/dovecot-2.2.5/etc/dovecot-imap/dovecot-sql.conf.ext driver = sql } protocol imap { mail_max_userip_connections = 1000 mail_plugins = " virtual notify" } Regards, Kavish Karkera

2 1

[Dovecot] Logging pre-rewrite user ID for application-specific passwords
by Steven Murdoch 22 Sep '13

22 Sep '13

I have set up Dovecot v2.2.5 with application-specific passwords, along the lines of this blog post (http://www.dgsiegel.net/news/2013_05_21-application_specific_passwords_for_…) My users file looks like: user-foo:{BLF-CRYPT}$2...:42:42::/home/user::allow_nets=127.0.0.1/32 user=user user-bar:{BLF-CRYPT}$2...:42:42::/home/user::user=user user:{BLF-CRYPT}$2...:42:42::/home/user::nologin In this way, user-foo can log in with one password (but only from localhost), user-bar can log in from any host with a different password, but both actually being treated as the same user. When these users log in through, the log entries look like: Aug 26 23:03:01 hostname dovecot: imap-login: Login: user=<user>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=12345, TLS Whereas I would like to see the original user name (prior to the re-write), i.e. user-foo and user-bar so I can keep track of which user names are in use and from where. Is there some way to do this? I couldn't see any of the variables being suitable. There is some information in the debug logs, but I would prefer not to need to leave this on when I don't need them. Thanks in advance. Steven

2 1

[Dovecot] Dovecot never release preallocated space in mdbox
by Stéphane BERTHELOT 22 Sep '13

22 Sep '13

Hello everyone, I am currently evaluating dovecot for our new email production servers (20k+ mailboxes) and found out something strange. I'm using those settings on Dovecot 2.2.4 (x86_64 / Slackware / compiled from sources) mdbox_rotate_size = 128M mdbox_rotate_interval = 1d mdbox_preallocate_space = yes with virtual users and location like : mail_location = mdbox:~/mdbox I don't think the remaining config is relevant but ask me if you need some other parts. Using test accounts for 2 weeks now I've figured that the 128M preallocated space is never 'hole punched" (to use a similar term than "man fallocate" on Linux), even when rotating m.* files. From what I understand those files will never be appended again because of the mdbox_rotate_interval. Then doveadm purge creates new files so old ones would never grow again. Here is an example of a mdbox storage using ls -ls (which shows allocated VS used space) total 4065176 1884 -rw------- 1 mail mail 1926656 Jul 29 10:55 dovecot.map.index 4 -rw------- 1 mail mail 460 Jul 29 11:26 dovecot.map.index.log 48 -rw------- 1 mail mail 44304 Jul 29 10:55 dovecot.map.index.log.2 131072 -rw------- 1 mail mail 133165066 Jul 19 15:31 m.10 131072 -rw------- 1 mail mail 133507393 Jul 19 15:32 m.13 131072 -rw------- 1 mail mail 134155182 Jul 19 15:33 m.14 131072 -rw------- 1 mail mail 134213403 Jul 19 15:30 m.2 131072 -rw------- 1 mail mail 46464 Jul 21 04:30 m.21 131072 -rw------- 1 mail mail 134215030 Jul 19 15:30 m.3 131072 -rw------- 1 mail mail 25852 Jul 25 01:54 m.32 131072 -rw------- 1 mail mail 2360 Jul 26 00:05 m.34 131072 -rw------- 1 mail mail 169073 Jul 27 23:18 m.35 131072 -rw------- 1 mail mail 31624 Jul 27 01:55 m.36 131072 -rw------- 1 mail mail 134216982 Jul 28 04:30 m.37 131076 -rw------- 1 mail mail 134217804 Jul 28 04:30 m.38 131072 -rw------- 1 mail mail 134217341 Jul 28 04:30 m.39 131072 -rw------- 1 mail mail 134213719 Jul 19 15:30 m.4 131072 -rw------- 1 mail mail 29740970 Jul 28 04:30 m.40 131072 -rw------- 1 mail mail 129175917 Jul 28 04:30 m.41 131072 -rw------- 1 mail mail 133174937 Jul 28 04:30 m.42 131072 -rw------- 1 mail mail 633436 Jul 28 04:30 m.43 131072 -rw------- 1 mail mail 3154623 Jul 28 04:30 m.44 131072 -rw------- 1 mail mail 3676879 Jul 28 04:30 m.45 131072 -rw------- 1 mail mail 468158 Jul 28 04:30 m.46 131072 -rw------- 1 mail mail 26964 Jul 28 04:30 m.47 131072 -rw------- 1 mail mail 3574599 Jul 28 04:30 m.48 131072 -rw------- 1 mail mail 3789133 Jul 28 04:30 m.49 131072 -rw------- 1 mail mail 134215016 Jul 19 15:30 m.5 131072 -rw------- 1 mail mail 1280074 Jul 28 04:30 m.50 131076 -rw------- 1 mail mail 635459 Jul 28 22:47 m.51 131072 -rw------- 1 mail mail 1459418 Jul 29 10:55 m.52 131072 -rw------- 1 mail mail 132941013 Jul 29 11:26 m.53 131072 -rw------- 1 mail mail 134213475 Jul 19 15:30 m.7 131072 -rw------- 1 mail mail 132240074 Jul 19 15:31 m.9 There's a lot of "lost" space since preallocated space would only be reclaimed when *all* emails in m.X file have refcount=0 and after a doveadm purge call, if I read well the dovecot docs. On mailboxes patterns with low incoming mail (< 100kb / day) this would waste much space. Of course I can decrease rotate size a lot but it would then produce a lot of files and would certainly become similar performance-wise to sdbox/maildir/... There would certainly be smart to use something similar to "FALLOC_FL_PUNCH_HOLE" on rotation (when doing close() ?) so that when we're sure there won't be anymore data appended to file that the allocated space == used space. I will disable space preallocation for our next tests since it wastes much storage for us ; did you have any feedback on how much it may affect performance ? I found in this ML archives some messages about the implementation but didn't see anyone clearly stating how much better preallocation is. Thanks, best regards, Stephane Berthelot.

4 5

[Dovecot] Deleted mails in dsync backups
by Wolfgang.Friebel＠desy.de 22 Sep '13

22 Sep '13

Hello we are doing regular backups of the mdbox mail folders using dsync -u username backup mdbox:/somedir/BACKUPS/username (still with version 2.1.10, but 2.2.5 is in testing phase) We do observe that the directory size in the BACKUPS directory is growing with respect to the original mail folders. I believe this is due to deleted mails not being purged in the backup. We are doing regular doveadm purge -A cleanups, but this does (of course) not affect the backups and subsequent dsync backup runs seem not to clean up deleted mails in the backup either. Will the behaviour of the dsync backup command deal with deleted mails in the 2.2.5 release? If not, what could we do to get rid of the deleted mails in the backup (or is there another reason for the growth in size) We have e.g. after 9 month of running with 2.1.10 du -sk username 588520 du -sk /somedir/BACKUPS/username 863024 -- Wolfgang Friebel Deutsches Elektronen-Synchrotron DESY

2 1

[Dovecot] Rfc: Limit the size of an email IN the mailbox
by Ralf Hildebrandt 21 Sep '13

21 Sep '13

Currently we're in the process of migrating Dovecot mailboxes to Exchange. This is all working very well, except for the occasional user, who used his/her Drafts folder as a file storage: They started writing an email containing lots of image attachments (and thus exceeding our maximum_message_size by far). So the mail never gets sent, but stays in the Drafts folder. Once I try moving that mail to exchange, it won't accept the message since it's too big. I think it would be interesting to have a limit on the total size of an IMAP object, just like in SMTP. Thought? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

3 2

[Dovecot] INBOX protected foldername?
by Hajo Locke 21 Sep '13

21 Sep '13

Hello, have a little problem with folders with name INBOX created by Users. Some Clients use a Prefix for their Folders. In our old setup (dovecot 1.2) it was possible to create a Folder INBOX in mbox-space with some Subfolders. These Subfolders of INBOX showed up in Mailclient at toplevel. Now we updatet to 2.1.7. In higher dovecot versions INBOX seems to be a kind of protected foldername. dovecot is not offering this folder in folderlist. If i rename it, the folder is available again. There are only a few users, so i could fix this manually. But how to explain this? I did not find a setting etc. which could explain this behaviour. Our Config is pretty standard: mail_location is unchanged some dovecot versions we use: mail_location = mbox:~/mail:INBOX=/var/mail/%u most of conf is out of the box and also used in older versions. only new is namespace inbox, but should also be default: namespace inbox { inbox = yes } Is there an explaination for this behaviour? Thanks, Hajo

4 10

[Dovecot] . (dot) in maildir folder names
by Christoph Anton Mitterer 21 Sep '13

21 Sep '13

Hi. Uhm... yeah as the topic implies I want to have . (dots) in my folder names... Unfortunately dovecot's maildirmake hasn't a -f switch as the one from courier/maildrop, but that one in turn is buggy[0] and doesn't encode any characters at all. In principle, maildir++ should allow[1] having (encoded) dots in folder names,... but I don't quite understand how the encoding actually works, cause UTF-7 seems to be already the encoded (without padding and such)... but . is . in UTF-7 as well. Does dovecot support this? At least my mail clients (well I only tried Evolution so far which sucks in so many areas)... couldn't create it... but I guess it's rather a stupid client issue. Anyone knows how it would be encoded? Interestingly, and this might be actually an inssue in doveocot when I create maildirs with non-ASCII chars without encoding these, e.g. maildir/.München/ (instead of the correctly encoded (.M&APw-nchen) Dovecot seems to "export" these to the clients,... at least Evolution shows it in the list of subscribable folders (even correctly as "München")... but it doesn't allow me to actually subscribe to it. Cheers & thx, Chris. [0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=709555 [1] http://www.courier-mta.org/maildir.html#id351803

5 24