April 2010 - dovecot - dovecot.org

[Dovecot] Busy looping of 2.0Beta4 on NetBSD-current
by Scott Ellis 26 May '10

26 May '10

I built and install Dovecot 2.0Beta4 on my NetBSD/amd64-current system, and am seeing the user-owned imap processes busy-looping. In Dovecot 1.2.x (and 1.x and 0.x..) I force ionotify and ioloop to kqueue, so I suspected that was the problem. Turns out it's not: Using poll or none also causes the same problem. This did not happen in 1.2.8 (didn't try anything later than that). After a quick visit to gdb, I added some i_debug() statements in ioloop-kqueue.c (when I had that as the notify and loop method) which showed that the loop was in io_loop_handler_run() . I haven't done any similar debugging for the other ioloop methods, but a quick look at the 2.0 changelog for those files doesn't raise any immediate alarms. Any ideas what to look for here? Timo, any ideas? 1.2.x worked perfectly so it's something "new" in 2.0. ScottE

3 5

[Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
by Baptiste Malguy 26 May '10

26 May '10

Hello, I may have missed a point, but I could have found an issue between plugins quota and lazy_expunge. I have noticed that the "lazy_expunge"d mails are counted as part of the quota, while the documentation says it should not be. I have first tried to find out if this was a configuration mistake somewhere. Notably, you can notice that the expunged mails are in directory ~/expunged while the usual mail is in ~/Maildir : expunged mails are not in a subdirectory of ~/Maildir (this was just to make sure there were no side effect with this). I have deleted some expunged mails to be sure : at its next cycle, the file ~/Maildir/maildirsize contained expected value. So yes, I do know expunged mails in the expunged directory are counted (size, number of mails). So I've taken a look at the source code, adding some i_info(...) at different places of the quota and lazy_expunge plugins. >From the source code reading session, I understand that : 1. In src/lazy-expunge/lazy-expunge-plugin.c 1.1. lazy_expunge_mail_storage_init() sets the expunged mails namespace flag with NAMESPACE_FLAG_NOQUOTA 1.2. lazy_expunge_mail_storage_init() is called by lazy_expunge_mail_storage_created() 1.3. lazy_expunge_mail_storage_created() is part of the callback list hook_mail_storage_create 2. In src/plugins/quota/quota-storage.c: 2.1. The only place in the whole dovecot source code where NAMESPACE_FLAG_NOQUOTA is checked is in function quota_mailbox_list_created() 3. In src/plugins/quota/quota-plugin.c: 3.1 quota_plugin_init() add quota_mailbox_list_created() to the callback list hook_mailbox_list_created. >From my observations : - Callback functions in quota_mailbox_list_created are called _before_ callback functions in hook_mail_storage_create. - This observation led me to move the piece of code that sets NAMESPACE_FLAG_NOQUOTA from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created(). - But there, lazy_expunge_mail_storage_created() is called after quota_mail_storage_created() because names of the library files are lib10_quota_plugin.so and lib02_lazy_expunge_plugin.so. - There I have also learnt that callback functions of the same list are called in the reverse order compared to the library filenames. So see if my observations were right, I have renamed lib10_quota_plugin.so to lib01_quota_plugin.so, then it worked. But I doubt this is the right way to fix it. This also reverses the order other callbacks between the two plugins are called, which migh not be expected at all. I'm not event sure moving some code from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created() was right : I've improvized the reading the source code and do not understand the whole logic, so I suppose this could break something else. Timo, what do you propose ? Thanks for your great software. -- Baptiste MALGUY PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2

2 2

[Dovecot] multipul ssl certification
by WangSamuel 26 May '10

26 May '10

# 2.0.beta4: /etc/dovecot/dovecot.conf # OS: Linux 2.6.28.finerTEK i686 Gentoo Base System release 1.12.11.1 ----------------------------------------------------------- Multiple SSL certificates (v2.0+) local imap.example.org { ssl_cert = </etc/ssl/certs/imap.example.org.crt ssl_key = </etc/ssl/private/imap.example.org.key } local imap.example2.org { ssl_cert = </etc/ssl/certs/imap.example2.org.crt ssl_key = </etc/ssl/private/imap.example2.org.key } # ..etc..With v1.x you need to use multiple instances with multiple configurations. ----------------------------------------------------------------------------- which file should i put the "local imap.foo.bar {... } " in? i tried put it in ssl.conf, and comment the original ssl_cert and ssl_key lines. and start dovecot with -F parameter but doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: ssl enabled, but ssl_cert not set please help me thanks Sam _________________________________________________________________ 一张照片的自白――Windows Live照片的可爱视频介绍 http://windowslivesky.spaces.live.com/blog/cns!5892B6048E2498BD!889.entry

2 1

[Dovecot] Dovecot 1.2.11 core dump analyse
by Pierre 26 May '10

26 May '10

Hi list I recently experienced core dumps on one of my dovecot servers: Apr 30 13:46:49 <ip> dovecot: [ID 583609 mail.error] dovecot: child 4088 (imap) killed with signal 6 (core dumped) Apr 30 13:46:56 <ip> dovecot: [ID 583609 mail.error] IMAP(xx(a)xx.be): Raw backtrace: 0x80e42f9 -> 0x80e3df3 -> 0x80e9fbe -> 0x80e86d2 -> 0x80da020 -> 0x80e86d2 -> 0x806ff0c -> 0x806e80b -> 0x80671fa -> 0x806b4e1 -> 0x806c1b6 -> 0x806c171 -> 0x806c333 -> 0x806cc8a -> 0x80eb761 -> 0x80eb161 -> 0x8074261 -> 0x80652ec Apr 30 13:46:58 <ip> dovecot: [ID 583609 mail.error] IMAP(xx(a)xx.be): Raw backtrace: 0x80e42f9 -> 0x80e3df3 -> 0x80e9fbe -> 0x80e86d2 -> 0x80da020 -> 0x80e86d2 -> 0x806ff0c -> 0x806e80b -> 0x80671fa -> 0x806b4e1 -> 0x806c1b6 -> 0x806c171 -> 0x806c333 -> 0x806cc8a -> 0x80eb761 -> 0x80eb161 -> 0x8074261 -> 0x80652ec Apr 30 13:47:00 <ip> dovecot: [ID 583609 mail.error] dovecot: child 4279 (imap) killed with signal 6 (core dumped) Apr 30 13:47:00 <ip> dovecot: [ID 583609 mail.error] dovecot: child 4279 (imap) killed with signal 6 (core dumped) I went to xx(a)xxx.be 's maildir, and saw a big file named core: -rw-------+ 1 vpopmail vchkpw 3.9M 2010-04-30 13:47 core I'm on solaris 10 server, how to analyse this core dump to find what caused it ? Thanks in advance, Pierre

2 3

[Dovecot] performance of maildir on ocfs2
by mailinglists＠belfin.ch 26 May '10

26 May '10

Hi, I would like to run my imap service on a active-active cluster. I wonder how well OCFS2 performs in read and write with millions of smallest files involved. Has anybody got any experience? Thanks John

5 8

[Dovecot] order of variable modifiers
by Phil Howard 25 May '10

25 May '10

I'd like to have a variable substitute take the domain name part, lower case it, MD5 it, and take a 2-character substring of that. Would that be like %2MLd ? The documentation on the H hash modifier described reversing with R, but expresses it at %3RHu which seems to me like it would be reversing the hash result rather than the user string. I'm undecided about MD5 vs. hash, and could go either way. But I was thinking: mail_location = /data/mail/%2MLd/%Ld/%2MLn/%Ln

2 1

[Dovecot] Panic: file mail-index-sync-ext.c: line 212 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1)
by Sandro Tosi 25 May '10

25 May '10

Hello, we are using dovecot v1.2.10 on debian stable (lenny) systems with mailboxes on NFS, and we are facing this quite ugly crash: 2010-04-27 10:59:06 IMAP(<mailbox>): Panic: file mail-index-sync-ext.c: line 212 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) 2010-04-27 10:59:06 dovecot: Error: child 2429 (imap) killed with signal 6 (core dumps disabled) 2010-04-27 10:59:06 IMAP(<mailbox>): Error: Raw backtrace: imap [0x80f2ea1] -> imap [0x80f2f22] -> imap [0x80f2879] -> imap [0x80df6ad] -> imap(mail_index_sync_ext_intro+0x25b) [0x80e043b] -> imap(mail_index_sync_record+0x164) [0x80ce8e4] -> imap(mail_index_sync_map+0x2c3) [0x80cf6d3] -> imap(mail_index_map+0x465) [0x80c47f5] -> imap [0x80c1053] -> imap(mail_index_open+0x24f) [0x80c150f] -> imap(index_storage_mailbox_open+0x72) [0x80acbb2] -> imap(index_storage_mailbox_init+0x132) [0x80acf72] -> imap [0x80847f8] -> imap [0x8084a9d] -> imap(mailbox_open+0x69) [0x80b71b9] -> imap(cmd_select_full+0x1dd) [0x806503d] -> imap(cmd_select+0x19) [0x8065999] -> imap [0x8067c7c] -> imap [0x8067d19] -> imap(client_handle_input+0x2d) [0x8067e8d] -> imap(client_input+0x5f) [0x80687ef] -> imap(io_loop_handler_run+0xe0) [0x80fbc50] -> imap(io_loop_run+0x20) [0x80fb0c0] -> imap(main+0x5ea) [0x80712fa] -> /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5) [0xb7dbf455] -> imap [0x8060361] the only solution we have found is to delete the dovecot.index* files, and let dovecot regenerate them at the next access. could you please give it a look? Thanks in advance, Sandro

4 11

[Dovecot] double write-locking dovecot.index.log
by Mike Abbott 19 May '10

19 May '10

Dovecot's transactions often write-lock dovecot.index.log twice (without unlocking in between), and then unlock it twice. Is that intended, or is that a bug? If I read the man pages correctly, double write-locking is pointless but safe. But double unlocking could cause data corruption during the period between the first and second unlocks, because the locks don't nest (are not refcounted). The first unlock leaves the file totally unlocked. Could this be the source of some index corruption errors? For example: 1. Process A acquires a F_WRLCK on a file. 2. Through a different code path, process A again acquires a F_WRLCK on the same file. Since these locks don't nest this is essentially a no-op. 3. Process A modifies the file. 4. Process A unlocks the file to undo the lock in step 2. Now the file has no lock on it at all. 5a. Process B locks, modifies, and unlocks the file. 5b. Process A also modifies the file, thinking that it still has a write lock. 6. Process A unlocks the file to undo the lock in step 1, but this is another no-op. Data corruption is possible if steps 5a and 5b conflict, or if 5b uses stale data. Here is a code path in dovecot-1.2.11 which exhibits the above behavior. "Outer" write lock: maildir_transaction_save_commit_pre -> maildir_transaction_save_commit_pre_sync -> maildir_sync_index_begin -> mail_index_sync_begin -> mail_index_sync_begin_to -> mail_index_sync_begin_init -> mail_transaction_log_sync_lock -> mail_transaction_log_lock_head -> mail_transaction_log_file_lock -> mail_index_lock_fd -> file_wait_lock -> file_lock_do "Inner" write lock: maildir_transaction_save_commit_pre -> maildir_sync_index_finish -> mail_index_sync_commit -> mail_index_write -> mail_transaction_log_rotate -> mail_transaction_log_file_create -> mail_transaction_log_file_create2 -> mail_transaction_log_file_lock -> mail_index_lock_fd -> file_wait_lock -> file_lock_do The file being locked is the same, dovecot.index.log. In another case, maildir_sync_index_begin is reached this way (outer lock): mailbox_sync -> mailbox_sync_init -> maildir_storage_sync_init -> maildir_sync_context -> maildir_sync_index_begin -> mail_index_sync_begin -> etc. And maildir_sync_index_finish is reached this way (inner lock): mailbox_sync -> mailbox_sync_init -> maildir_storage_sync_init -> maildir_sync_context -> maildir_sync_index_finish -> etc. Could these lead to data corruption? Please let me know if you need any more information. Thanks.

1 1

[Dovecot] Tracing sieve execution
by Jerry 07 May '10

07 May '10

I have just recently begun learning how to use sieve. For the most part, it works fine. The problem is that I have created some rather complicated rules. Occasionally, an e-mail will get sorted incorrectly and I am not able to easily determine what rule caused it to occur. Is there any way or utility that I could use to actually see what rule or line in the sieve file caused the action? -- Jerry Dovecot.user(a)seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ I hate it when my foot falls asleep during the day cause that means it's going to be up all night. Steven Wright

2 1

[Dovecot] ACLs and public folders
by hyperbatus＠gmx.de 07 May '10

07 May '10

Dear list, I am using dovecot 1.0.15 on Debian Lenny. I have a public folder, I use ACL / vfile (without public ACL), and I use maildir / vmail. Now I have two questions: 1) http://wiki.dovecot.org/ACL states: "Mailboxes in public namespaces don't have owners, so by default no-one can access them." The same document, when explaining the meaning of the k flag in the ACLs, states: "Mailboxes can be created (or renamed) under this mailbox (there is no recursion, so creating a mailbox under this mailbox's child uses only the child's ACLs)" I have placed an ACL (dovecot-ACL) at the root of the public namespace (all rights to anyone). The public namespace was appearing correctly in my IMAP client. Then I have copied a large folder with hundreds of nested subfolders (nesting level > 3) from another account to the root of the public namespace. According to the documentation mentioned above, this should not have been possible (if I got the docs right): The first nesting level of folder should have been created, but not the messages in them and for sure not the deeper nested folders. There is no dovecot-acl within a single of the first level folders (I have verified this), so no one should have access to them. Nevertheless, in addition to the succesful copy, I can see every folder and every message in my IMAP client, I can delete and so on. Could some please explain if I got the docs wrong? 2) If there really is no ACL recursion, how are we supposed to copy large folder structures (perhaps thousands of folders, nested to 5 levels deep) to a public namespace? Do we need to create all folders by hand, then place the dovecot-acl in each folder, and then copy the messages to each folder by hand? Do we need to switch to global ACLs so that we can establish a master user for doing the act of filling the public space? Or is there a dovecot module which, immediately after creating a folder, looks into the parent folder of the new folder and copies the dovecot-acl from the parent folder to the new folder? Currently, I am very happy that dovecot didn't what it should do according to my understanding of the documentation (since I really needed to copy this folder structure to the public namespace), but on the other hand, I am a bit puzzled now not knowing if the documentation is wrong, my understanding of it is wrong or the source code is wrong :-) Thanks you very much for any help, Peter Here is the output of dovecot -n: # 1.0.15: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot-info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps listen: 192.168.20.23 ssl_cert_file: /etc/dovecot/imap-ssl.home.omeganet.de.crt ssl_key_file: /etc/dovecot/imap-ssl.home.omeganet.de.key ssl_parameters_regenerate: 24 ssl_cipher_list: ALL:!LOW login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_processes_count: 1 login_max_processes_count: 32 max_mail_processes: 32 mail_privileged_group: mail mail_location: maildir:~/Maildir mail_cache_fields: mail_never_cache_fields: mail_plugins: acl namespace: type: private inbox: yes namespace: type: public prefix: Archive. location: maildir:/home/vmail/archive:INDEX=~/Maildir/archive auth default: cache_size: 1 cache_ttl: 600 worker_max_count: 4 passdb: driver: passwd-file args: /etc/dovecot/passdb userdb: driver: static args: uid=vmail gid=vmail home=/home/vmail/%u socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: root plugin: acl: vfile -- GMX DSL: Internet, Telefon und Entertainment für nur 19,99 EUR/mtl.! http://portal.gmx.net/de/go/dsl02

2 2