May 2010 - dovecot - dovecot.org

[Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
by Baptiste Malguy 26 May '10

26 May '10

Hello, I may have missed a point, but I could have found an issue between plugins quota and lazy_expunge. I have noticed that the "lazy_expunge"d mails are counted as part of the quota, while the documentation says it should not be. I have first tried to find out if this was a configuration mistake somewhere. Notably, you can notice that the expunged mails are in directory ~/expunged while the usual mail is in ~/Maildir : expunged mails are not in a subdirectory of ~/Maildir (this was just to make sure there were no side effect with this). I have deleted some expunged mails to be sure : at its next cycle, the file ~/Maildir/maildirsize contained expected value. So yes, I do know expunged mails in the expunged directory are counted (size, number of mails). So I've taken a look at the source code, adding some i_info(...) at different places of the quota and lazy_expunge plugins. >From the source code reading session, I understand that : 1. In src/lazy-expunge/lazy-expunge-plugin.c 1.1. lazy_expunge_mail_storage_init() sets the expunged mails namespace flag with NAMESPACE_FLAG_NOQUOTA 1.2. lazy_expunge_mail_storage_init() is called by lazy_expunge_mail_storage_created() 1.3. lazy_expunge_mail_storage_created() is part of the callback list hook_mail_storage_create 2. In src/plugins/quota/quota-storage.c: 2.1. The only place in the whole dovecot source code where NAMESPACE_FLAG_NOQUOTA is checked is in function quota_mailbox_list_created() 3. In src/plugins/quota/quota-plugin.c: 3.1 quota_plugin_init() add quota_mailbox_list_created() to the callback list hook_mailbox_list_created. >From my observations : - Callback functions in quota_mailbox_list_created are called _before_ callback functions in hook_mail_storage_create. - This observation led me to move the piece of code that sets NAMESPACE_FLAG_NOQUOTA from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created(). - But there, lazy_expunge_mail_storage_created() is called after quota_mail_storage_created() because names of the library files are lib10_quota_plugin.so and lib02_lazy_expunge_plugin.so. - There I have also learnt that callback functions of the same list are called in the reverse order compared to the library filenames. So see if my observations were right, I have renamed lib10_quota_plugin.so to lib01_quota_plugin.so, then it worked. But I doubt this is the right way to fix it. This also reverses the order other callbacks between the two plugins are called, which migh not be expected at all. I'm not event sure moving some code from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created() was right : I've improvized the reading the source code and do not understand the whole logic, so I suppose this could break something else. Timo, what do you propose ? Thanks for your great software. -- Baptiste MALGUY PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2

2 2

[Dovecot] multipul ssl certification
by WangSamuel 26 May '10

26 May '10

# 2.0.beta4: /etc/dovecot/dovecot.conf # OS: Linux 2.6.28.finerTEK i686 Gentoo Base System release 1.12.11.1 ----------------------------------------------------------- Multiple SSL certificates (v2.0+) local imap.example.org { ssl_cert = </etc/ssl/certs/imap.example.org.crt ssl_key = </etc/ssl/private/imap.example.org.key } local imap.example2.org { ssl_cert = </etc/ssl/certs/imap.example2.org.crt ssl_key = </etc/ssl/private/imap.example2.org.key } # ..etc..With v1.x you need to use multiple instances with multiple configurations. ----------------------------------------------------------------------------- which file should i put the "local imap.foo.bar {... } " in? i tried put it in ssl.conf, and comment the original ssl_cert and ssl_key lines. and start dovecot with -F parameter but doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: ssl enabled, but ssl_cert not set please help me thanks Sam _________________________________________________________________ 一张照片的自白――Windows Live照片的可爱视频介绍 http://windowslivesky.spaces.live.com/blog/cns!5892B6048E2498BD!889.entry

2 1

[Dovecot] Dovecot 1.2.11 core dump analyse
by Pierre 26 May '10

26 May '10

Hi list I recently experienced core dumps on one of my dovecot servers: Apr 30 13:46:49 <ip> dovecot: [ID 583609 mail.error] dovecot: child 4088 (imap) killed with signal 6 (core dumped) Apr 30 13:46:56 <ip> dovecot: [ID 583609 mail.error] IMAP(xx(a)xx.be): Raw backtrace: 0x80e42f9 -> 0x80e3df3 -> 0x80e9fbe -> 0x80e86d2 -> 0x80da020 -> 0x80e86d2 -> 0x806ff0c -> 0x806e80b -> 0x80671fa -> 0x806b4e1 -> 0x806c1b6 -> 0x806c171 -> 0x806c333 -> 0x806cc8a -> 0x80eb761 -> 0x80eb161 -> 0x8074261 -> 0x80652ec Apr 30 13:46:58 <ip> dovecot: [ID 583609 mail.error] IMAP(xx(a)xx.be): Raw backtrace: 0x80e42f9 -> 0x80e3df3 -> 0x80e9fbe -> 0x80e86d2 -> 0x80da020 -> 0x80e86d2 -> 0x806ff0c -> 0x806e80b -> 0x80671fa -> 0x806b4e1 -> 0x806c1b6 -> 0x806c171 -> 0x806c333 -> 0x806cc8a -> 0x80eb761 -> 0x80eb161 -> 0x8074261 -> 0x80652ec Apr 30 13:47:00 <ip> dovecot: [ID 583609 mail.error] dovecot: child 4279 (imap) killed with signal 6 (core dumped) Apr 30 13:47:00 <ip> dovecot: [ID 583609 mail.error] dovecot: child 4279 (imap) killed with signal 6 (core dumped) I went to xx(a)xxx.be 's maildir, and saw a big file named core: -rw-------+ 1 vpopmail vchkpw 3.9M 2010-04-30 13:47 core I'm on solaris 10 server, how to analyse this core dump to find what caused it ? Thanks in advance, Pierre

2 3

[Dovecot] performance of maildir on ocfs2
by mailinglists＠belfin.ch 26 May '10

26 May '10

Hi, I would like to run my imap service on a active-active cluster. I wonder how well OCFS2 performs in read and write with millions of smallest files involved. Has anybody got any experience? Thanks John

5 8

[Dovecot] Zlib plugin dovecot 2.0beta4
by Alex Baule 26 May '10

26 May '10

Hi Timo... I made the plugin like you say. The header part is saved by dovecot, with o_stream_send , and the body with my function. This implementation, works fine. But i need to get / change some values to create the "link" from header to body. When a email is saved, have the S an W flags, that is the size of email. There is a way to increase this Flags ? I need to put one more line in the header, to have some link to body. To Increase the S Flag i increase the ostream.offset, but the W is based in the offset from istream, that's right ? There is a way to do this without rewrite the read function !? Exemple, when the email is moved from tmp to cur.

2 2

[Dovecot] proxy user wildcard
by Martin Ott 26 May '10

26 May '10

Hi, I'd like to try dovecot as a imapproxy in front of a webmailer. Authentication should be done by the main dovecot-imapserver. It works if I specify the user in passdb passwd-file: user2::0:0::::proxy=yes host=[ip-address] ssl=any-cert starttls=yes nopassword=y Is there some kind of catch-all function to pass all users to the main dovecot-imapserver without knowing the valid users? dovecot: 1.2.9 thx Martin

2 1

[Dovecot] Plugins and message file access
by Reinaldo Matukuma 26 May '10

26 May '10

Hello. I need to know how I can get the final message file name inside a plugin like the zlib 2.0 plugin. I saw another threads about zlib plugin, but don't see no one asking about this. My interest is to use the message's file name in a specific function that will add some lines on it and, after that, calculate a md5 hash of the modified file. As the size flags will not correspond to the effective message size/vsize, then I could change the flags using the filename too, right? This is my first approach. But... There are another way to do this directly on message been received by the plugin, like pre-allocating the size/vsize flags? I cannot simply to pass the extra text directly on the message being received because I need to do some extra validations over the full content of the message before add my data on the message. Thank's Everybody _________________________________________________________________ O seu navegador também te ajuda a ficar longe de vírus. Leia mais sobre segurança. http://www.microsoft.com/brasil/windows/internet-explorer/?WT.mc_id=1500

2 1

[Dovecot] 2.0.beta4 (f05909834219): Crashes with incorrect maildir permissions
by Thomas Leuxner 26 May '10

26 May '10

Hi, I know this is a self-inflicted problem due to wrong permissions set manually, but it should not crash I guess... Regards Thomas May 3 16:05:15 spectre dovecot: imap(alias(a)domain.tld): Error: opendir(/var/vmail/public/Mailing-Lists/Bind-Users/new) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +r perm: /var/vmail/public/Mailing-Lists/Bind-Users/new, euid is not dir owner) May 3 16:05:15 spectre dovecot: imap(alias(a)domain.tld): Error: stat(/var/vmail/public/Mailing-Lists/Bind-Users/tmp/ed136730ffd1de4b6d250000a998a2df) failed: Permission denied May 3 16:05:15 spectre dovecot: imap(alias(a)domain.tld): Panic: file mail-index.c: line 346 (mail_index_keywords_unref): assertion failed: (keywords->refcount > 0) May 3 16:05:15 spectre dovecot: imap(alias(a)domain.tld): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0x7f7e592e1772] -> /usr/lib/dovecot/libdovecot.so.0 [0x7f7e592e17da] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f7e592e1b83] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x7f7e5955d178] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_keywords_unref+0x10) [0x7f7e5958fcf0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x71) [0x7f7e5954dfa1] -> dovecot/imap [alias(a)domain.tld 188.138.0.199 UID](cmd_copy+0x209) [0x409b69] -> dovecot/imap [alias(a)domain.tld 188.138.0.199 UID] [0x41004c] -> dovecot/imap [alias(a)domain.tld 188.138.0.199 UID] [0x4100fd] -> dovecot/imap [alias(a)domain.tld 188.138.0.199 UID](client_handle_input+0x45) [0x410275] -> dovecot/imap [alias(a)domain.tld 188.138.0.199 UID](client_input+0x62) [0x410cc2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xcb) [0x7f7e592ec4ab] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x18) [0x7f7e592eb628] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f7e592db1a3] -> dovecot/imap [alias(a)domain.tld 188.138.0.199 UID](main+0x281) [0x418d61] -> /lib/libc.so.6(__libc_start_main+0xe6) [0x7f7e58b661a6] -> dovecot/imap [alias(a)domain.tld 188.138.0.199 UID] [0x408999] May 3 16:05:15 spectre dovecot: master: Error: service(imap): child 9756 killed with signal 6 (core dumped) #0 0x00007f7e58b79ed5 in raise () from /lib/libc.so.6 No symbol table info available. #1 0x00007f7e58b7b3f3 in abort () from /lib/libc.so.6 No symbol table info available. #2 0x00007f7e592e1785 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:164 backtrace = 0x10fc918 "/usr/lib/dovecot/libdovecot.so.0 [0x7f7e592e1772] -> /usr/lib/dovecot/libdovecot.so.0 [0x7f7e592e17da] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f7e592e1b83] -> /usr/lib/dovecot/libdovecot-st"... #3 0x00007f7e592e17da in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=<value optimized out>, args=<value optimized out>) at failures.c:592 No locals. #4 0x00007f7e592e1b83 in i_panic (format=<value optimized out>) at failures.c:230 args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fff547d8d40, reg_save_area = 0x7fff547d8c80}} #5 0x00007f7e5955d178 in mail_index_keywords_unref (_keywords=<value optimized out>) at mail-index.c:346 keywords = <value optimized out> __FUNCTION__ = "mail_index_keywords_unref" #6 0x00007f7e5958fcf0 in index_keywords_unref (keywords=0x115c620) at index-storage.c:614 No locals. #7 0x00007f7e5954dfa1 in mailbox_copy (_ctx=0x7fff547d8e20, mail=0x116d298) at mail-storage.c:1386 ctx = <value optimized out> box = (struct mailbox *) 0x1159e18 keywords = (struct mail_keywords *) 0x0 ret = -1 #8 0x0000000000409b69 in cmd_copy (cmd=0x112e8a8) at cmd-copy.c:66 client = (struct client *) 0x112e148 dest_ns = <value optimized out> dest_storage = <value optimized out> destbox = (struct mailbox *) 0x1159e18 t = (struct mailbox_transaction_context *) 0x115db60 search_args = (struct mail_search_args *) 0x116a6c8 messageset = 0x11301f8 "18430" mailbox = 0x1130200 "Public/Mailing-Lists/Bind-Users" storage_name = 0x1130207 "Mailing-Lists/Bind-Users" src_uidset = <value optimized out> status = MAILBOX_NAME_EXISTS_MAILBOX sync_flags = <value optimized out> imap_flags = <value optimized out> changes = {pool = 0x3000000020, uid_validity = 18022896, saved_uids = {arr = {buffer = 0x0, element_size = 9}, v = 0x0, v_modifiable = 0x0}, ignored_uid_changes = 1489302432, ignored_modseq_changes = 32638} copy_count = 1 msg = <value optimized out> ret = 1 __FUNCTION__ = "cmd_copy" #9 0x000000000041004c in client_command_input (cmd=0x112e8a8) at imap-client.c:646 client = (struct client *) 0x112e148 command = <value optimized out> __FUNCTION__ = "client_command_input" #10 0x00000000004100fd in client_command_input (cmd=0x112e8a8) at imap-client.c:696 client = (struct client *) 0x112e148 command = (struct command *) 0x6 __FUNCTION__ = "client_command_input" #11 0x0000000000410275 in client_handle_input (client=0x112e148) at imap-client.c:737 _data_stack_cur_id = 3 ret = 56 remove_io = <value optimized out> handled_commands = false __FUNCTION__ = "client_handle_input" #12 0x0000000000410cc2 in client_input (client=0x112e148) at imap-client.c:788 cmd = <value optimized out> output = (struct ostream *) 0x112e728 bytes = <value optimized out> #13 0x00007f7e592ec4ab in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:212 ctx = (struct ioloop_handler_context *) 0x11047d0 event = (const struct epoll_event *) 0x1104840 list = (struct io_list *) 0x112e800 io = (struct io_file *) 0x112e7c0 tv = {tv_sec = 2, tv_usec = 537890} t_id = 2 msecs = <value optimized out> ret = 1 call = <value optimized out> #14 0x00007f7e592eb628 in io_loop_run (ioloop=0x11045c0) at ioloop.c:350 No locals. #15 0x00007f7e592db1a3 in master_service_run (service=0x1104490, callback=0x261c) at master-service.c:480 No locals. #16 0x0000000000418d61 in main (argc=1, argv=0x1104370) at main.c:359 service_flags = <value optimized out> storage_service_flags = <value optimized out> postlogin_socket_path = 0x0 username = 0x0 c = <value optimized out> set_roots = {0x41bf40, 0x0}

2 1

[Dovecot] corrupted transaction log errors causing users mail to disappear temporarily
by Jordan Tardif 26 May '10

26 May '10

So Ive been looking into some problems with users reporting their mail goes missing for a second but then re-apears very soon after. I have tracked it down to happening at the same time these appear in the logs. Seems the transaction logs is getting corrupted somehow and then rebuilding itself (which works fine). The problem is while its re-builing the users think all of their mail is gone. May 3 09:35:52 mailmachine dovecot: IMAP(test(a)domain.com): Corrupted transaction log file usershomedir/Maildir/dovecot.index.log seq 3: Invalid transaction log size (32708 vs 32840): usershomedir/Maildir/dovecot.index.log (sync_offset=32708) We are using load balanced servers and data/index's are stored in their home dir's over nfs. All my config details and nfs options are below. We are currently balancing based on source address so most of the time users imap/pop processes should stick to one machine as much as possible. Does anyone know of a way to rid of this problem, or at least make it so dovecot just returns a server error to the imap/pop client instead of showing them a blank index? #nfs options defaults,intr,vers=3,bg,rsize=8192,wsize=8192,proto=udp 0 0 # 1.2.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.29.2-xenon-mail-ipv6-grsec x86_64 Debian 5.0.1 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap pop3 listen(default): *:143 listen(imap): *:143 listen(pop3): *:110 disable_plaintext_auth: no login_dir: /usr/var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_user: dhmail login_process_per_connection: no login_processes_count: 150 login_max_processes_count: 1000 login_max_connections: 3000 max_mail_processes: 2500 mail_max_userip_connections: 7 first_valid_uid: 100 mail_privileged_group: mail mail_location: maildir:~/Maildir mmap_disable: yes mail_nfs_index: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): UID%u-%v -- Jordan Tardif DreamHost

2 1

[Dovecot] Where should I define mail_uid for deliver LDA?
by Alex 26 May '10

26 May '10

Hello folks I'm trying to use Dovecot deliver LDA instead Postfix virtual LDA The last road block I have to resolve is to define mail_uid, becaus dovecot.err log has these error messages: May 04 14:57:19 deliver(user1(a)some_domain.com): Error: User user1(a)some_domain.com is missing UID (set mail_uid) May 04 14:57:19 deliver(user2(a)some_domain.com): Error: User user2(a)some_domain.com is missing UID (set mail_uid) May 04 14:57:25 deliver(user3(a)some_domain.com): Error: User user3(a)some_domain.com is missing UID (set mail_uid) Where exactly should I define mail_uid? Somewhere in Dovecot dovecot.conf file? I tried to set mail_uid in dovecot-sql.conf in user_query as follows: user_query = SELECT maildir as mail, 51 as userdb_uid, 51 as userdb_gid, 51 as mail_uid FROM dovecot_email_maps WHERE email = '%u' But this apparently doesn't provide that mail_uid as I expected. Could somebody please enlighten me on this issue. Thank you in advance, Alex

2 1