- dovecot - dovecot.org

[Dovecot] Panic 1.1.4
by Andre Hübner 14 Jan '09

14 Jan '09

Hi List, try 1.1.4 and have new Panics in Log. Hoped that 1.1.3 finishes this errors... Oct 8 21:56:28 dd10636 dovecot: IMAP(username): Loading modules from directory: /usr/lib/dovecot/imap Oct 8 21:56:28 dd10636 dovecot: IMAP(username): Module loaded: /usr/lib/dovecot/imap/lib20_mail_log_plugin.so Oct 8 21:56:28 dd10636 dovecot: IMAP(username): Effective uid=857, gid=628, home=/home/popuser/username Oct 8 21:56:28 dd10636 dovecot: IMAP(username): mbox: data=~/mail:INBOX=/var/spool/mail/username Oct 8 21:56:28 dd10636 dovecot: IMAP(username): fs: root=/home/popuser/username/mail, index=, control=, inbox=/var/spool/mail/username Oct 8 21:56:28 dd10636 dovecot: imap-login: Login: user=<username>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 8 21:56:28 dd10636 dovecot: Panic: IMAP(username): file istream-limit.c: line 84 (i_stream_limit_read): assertion failed: (ret != -1 || stream->istream.e of || stream->istream.stream_errno != 0) Oct 8 21:56:28 dd10636 dovecot: IMAP(username): Raw backtrace: imap [0x80c3a30] -> imap [0x80c3a8a] -> imap [0x80c338c] -> imap [0x80c9d04] -> imap(i_stream_ read+0x28) [0x80c7e28] -> imap(i_stream_read_data+0x1d) [0x80c7edd] -> imap(message_parse_header_next+0x7c) [0x80bbcdc] -> imap [0x80b9385] -> imap [0x80b9aa7 ] -> imap [0x80b9bb4] -> imap [0x80b9c95] -> imap(i_stream_seek+0x69) [0x80c7db9] -> imap [0x8071763] -> /usr/lib/dovecot/imap/lib20_mail_log_plugin.so [0xb7f a3aec] -> /usr/lib/dovecot/imap/lib20_mail_log_plugin.so [0xb7fa3d56] -> imap(cmd_copy+0x1d2) [0x805a142] -> imap [0x805edfc] -> imap [0x805eea5] -> imap [0x8 05f665] -> imap(client_input+0x5e) [0x805f87e] -> imap(io_loop_handler_run+0x100) [0x80cb590] -> imap(io_loop_run+0x28) [0x80ca6a8] -> imap(main+0x4ac) [0x806 73bc] -> /lib/libc.so.6(__libc_start_main+0xdc) [0xb7e8d87c] -> imap [0x8059191] Oct 8 21:56:28 dd10636 dovecot: child 9120 (imap) killed with signal 6 dovecot -n # 1.1.4: /etc/dovecot.conf protocols: imap imaps pop3 pop3s ssl_ca_file: /path/to/*hostname.com.bundle.crt ssl_cert_file: /path/to/*.hostname.com.crt ssl_key_file: /path/to/*.hostname.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_greeting: hostname.com mailserver ready. login_process_per_connection: no login_processes_count: 1 max_mail_processes: 100 verbose_proctitle: yes mail_location: mbox:~/mail:INBOX=/var/spool/mail/%u mail_cache_min_mail_count: 30 mail_debug: yes lock_method: flock mbox_read_locks: dotlock mbox_very_dirty_syncs: yes mail_drop_priv_before_exec: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): mail_log mail_plugins(imap): mail_log mail_plugins(pop3): mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle netscape-eoh tb-extra-mailbox-sep delay-newmail imap_client_workarounds(imap): outlook-idle netscape-eoh tb-extra-mailbox-sep delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): oe-ns-eoh outlook-no-nuls auth default: verbose: yes passdb: driver: shadow userdb: driver: passwd gdb /usr/libexec/dovecot/imap /home/popuser/username/core GNU gdb 6.4 Copyright 2005 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i586-suse-linux"...Using host libthread_db library "/lib/libthread_db.so.1". Core was generated by `imap [username 127.0.0.1]'. Program terminated with signal 6, Aborted. warning: Can't read pathname for load map: Eingabe-/Ausgabefehler. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/lib/dovecot/imap/lib20_mail_log_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_mail_log_plugin.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Reading symbols from /lib/libgcc_s.so.1...done. Loaded symbols for /lib/libgcc_s.so.1 #0 0xffffe410 in __kernel_vsyscall () (gdb) bt full #0 0xffffe410 in __kernel_vsyscall () No symbol table info available. #1 0xb7ea07d0 in raise () from /lib/libc.so.6 No symbol table info available. #2 0xb7ea1ea3 in abort () from /lib/libc.so.6 No symbol table info available. #3 0x080c3a3d in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:150 backtrace = 0x80f1ae8 "imap [0x80c3a30] -> imap [0x80c3a8a] -> imap [0x80c338c] -> imap [0x80c9d04] -> imap(i_stream_read+0x28) [0x80c7e28] -> imap(i_stream_read_data+0x1d) [0x80c7edd] -> imap(message_parse_header_next+0x7c"... #4 0x080c3a8a in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80d71ac "file %s: line %d (%s): assertion failed: (%s)", args=0xbf941da4 "Sj\016\bT") at failures.c:430 No locals. #5 0x080c338c in i_panic (format=0x80d71ac "file %s: line %d (%s): assertion failed: (%s)") at failures.c:197 args = 0xbf941da4 "Sj\016\bT" #6 0x080c9d04 in i_stream_limit_read (stream=0x8118cc8) at istream-limit.c:83 left = <value optimized out> ret = -1 pos = 65 __PRETTY_FUNCTION__ = "i_stream_limit_read" #7 0x080c7e28 in i_stream_read (stream=0x8118cf0) at istream.c:73 _stream = (struct istream_private *) 0x0 ret = <value optimized out> __PRETTY_FUNCTION__ = "i_stream_read" #8 0x080c7edd in i_stream_read_data (stream=0x8118cf0, data_r=0xbf941e68, size_r=0xbf941e64, threshold=65) at istream.c:299 ret = -1080811928 read_more = false __PRETTY_FUNCTION__ = "i_stream_read_data" #9 0x080bbcdc in message_parse_header_next (ctx=0x81118f0, hdr_r=0xbf941ec0) at message-header-parser.c:89 msg = ( const unsigned char *) 0x811292f "qan4iKSQ1jgszBF7yO1N1FgdflELgOSeaEzr+P+aEXdIbk/ooMFh+wew7C5rzd2W+AfY+P7e\n3el1/64bLf+hadyHUxkt3auVaT3Y6zcoxZaSmY1gEydp4JcyEVNapSNbsWFKYUTSJiju10VF\nkBLd+JYm8PvXkDSplAEe9MRASmX2Jzt/zG6b39O5byfU5IA6JiBj/5"... i = 6 size = 65 startpos = 64 colon_pos = 4294967295 parse_size = 64 value_pos = <value optimized out> ret = 6 continued = false continues = <value optimized out> crlf_newline = false #10 0x080b9385 in read_header (mstream=0x8118da8) at istream-header-filter.c:163 hdr = (struct message_header_line *) 0x0 highwater_offset = <value optimized out> pos = <value optimized out> ret = <value optimized out> matched = false hdr_ret = <value optimized out> __PRETTY_FUNCTION__ = "read_header" #11 0x080b9aa7 in i_stream_header_filter_read (stream=0x8118da8) at istream-header-filter.c:293 mstream = (struct header_filter_istream *) 0x0 ret = <value optimized out> pos = <value optimized out> #12 0x080b9bb4 in parse_header (mstream=0x8118da8) at istream-header-filter.c:333 pos = 73 #13 0x080b9c95 in i_stream_header_filter_seek (stream=0x8118da8, v_offset=1904, mark=false) at istream-header-filter.c:347 mstream = (struct header_filter_istream *) 0x0 #14 0x080c7db9 in i_stream_seek (stream=0x8118dd0, v_offset=1904) at istream.c:144 _stream = (struct istream_private *) 0x8118da8 ---Type <return> to continue, or q <return> to quit--- #15 0x08071763 in mbox_mail_get_physical_size (_mail=0x81142b8, size_r=0xbf942058) at mbox-mail.c:255 mail = <value optimized out> data = (struct index_mail_data *) 0x8114334 input = (struct istream *) 0x8118dd0 hdr_size = {physical_size = 1738, virtual_size = 1771, lines = 33} old_offset = 1904 body_offset = 151200709 body_size = <value optimized out> next_offset = 148393877 #16 0xb7fa3aec in mail_log_action (dest_trans=<value optimized out>, mail=0x81142b8, event=<value optimized out>, data=0x810f4f8 "Trash") at mail-log-plugin.c:305 lt = (struct mail_log_transaction_context *) 0x811c708 msgid = 0x8114bb4 "<20081008182128.D148E68E1BECD(a)relay1.senderdomain.net>" size = <value optimized out> str = (string_t *) 0x80f1858 pool = <value optimized out> #17 0xb7fa3d56 in mail_log_copy (t=0x810ff00, mail=0x81142b8, flags=MAIL_SEEN, keywords=0x0, dest_mail=0x0) at mail-log-plugin.c:400 _data_stack_cur_id = 4 name = <value optimized out> #18 0x0805a142 in cmd_copy (cmd=0x80fb9c8) at cmd-copy.c:66 client = (struct client *) 0x80fa548 storage = (struct mail_storage *) 0x80fb108 destbox = (struct mailbox *) 0x810f358 t = (struct mailbox_transaction_context *) 0x810ff00 search_arg = <value optimized out> messageset = 0x80ffac0 "737" mailbox = 0x80ffac8 "Trash" src_uidset = <value optimized out> msg = <value optimized out> sync_flags = <value optimized out> imap_flags = <value optimized out> copy_count = 1 uid_validity = <value optimized out> uid1 = <value optimized out> uid2 = <value optimized out> ret = 1 __PRETTY_FUNCTION__ = "cmd_copy" #19 0x0805edfc in client_command_input (cmd=0x80fb9c8) at client.c:580 client = (struct client *) 0x80fa548 command = <value optimized out> __PRETTY_FUNCTION__ = "client_command_input" #20 0x0805eea5 in client_command_input (cmd=0x80fb9c8) at client.c:629 client = (struct client *) 0x80fa548 command = (struct command *) 0x23a0 __PRETTY_FUNCTION__ = "client_command_input" #21 0x0805f665 in client_handle_input (client=0x80fa548) at client.c:670 _data_stack_cur_id = 3 ret = <value optimized out> remove_io = <value optimized out> handled_commands = false #22 0x0805f87e in client_input (client=0x80fa548) at client.c:725 cmd = <value optimized out> output = (struct ostream *) 0x80fb904 bytes = 23 __PRETTY_FUNCTION__ = "client_input" #23 0x080cb590 in io_loop_handler_run (ioloop=0x80f99b0) at ioloop-epoll.c:203 ctx = <value optimized out> event = (const struct epoll_event *) 0x80f9ae8 list = (struct io_list *) 0x80fa5b8 ---Type <return> to continue, or q <return> to quit--- io = (struct io_file *) 0x80fb960 tv = {tv_sec = 1799, tv_usec = 999127} t_id = 2 msecs = <value optimized out> ret = 1 i = 0 j = 0 call = <value optimized out> #24 0x080ca6a8 in io_loop_run (ioloop=0x80f99b0) at ioloop.c:320 No locals. #25 0x080673bc in main (argc=Cannot access memory at address 0x23a0 ) at main.c:293 No locals. Thanks Andre

2 2

[Dovecot] Maildir++ quota: When is it recalculated?
by Ulrich Zehl 14 Jan '09

14 Jan '09

We use Maildir++ quota, with the rules taken from our LDAP backend. We also have an old expunge script that's not quota-aware; it removes old messages from the maildir by simply deleting the files. Sometimes, a mailbox is over quota before the script runs, but well under quota after the old messages are deleted. This change does not seem to be picked up, however: When I try to deliver a new message (using Dovecot's deliver), the attempt fails with an "over quota" error. I tried finding out from the source code whether this is intended behavior, but unfortunately, I'm not enough of a programmer, so I have to ask here: Should I be able to delete mails from a maildir, and expect the quota plugin to notice the change? We use Dovecot 1.1.8, by the way. Ulrich

2 5

[Dovecot] PDF corruption issue
by Michael Dunne 14 Jan '09

14 Jan '09

Greetings, I have several users reporting that PDF attachments are coming in corrupted. They are using either thunderbird or outlook on windows xp. I have flushed the cache files for the users, but the problem still persists. Could anyone offer some troubleshooting tips? Thanks in advance for any assistance. Below is my version and config. Regards, Mike dovecot --version 1.1.7 dovecot -n # 1.1.7: /etc/dovecot.conf Warning: fd limit 1024 is lower than what Dovecot can use under full load (more than 1792). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: Linux 2.6.18-92.1.13.el5 x86_64 Red Hat Enterprise Linux Server release 5.2 (Tikanga) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot listen: [*] ssl_key_file: /etc/pki/dovecot/certs/dovecot.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_max_processes_count: 384 login_max_connections: 384 max_mail_processes: 1024 verbose_proctitle: yes first_valid_uid: 10 mail_privileged_group: mail mail_location: mbox:%h/mail:INBOX=/var/spool/mail/%u mail_debug: yes mail_full_filesystem_access: yes mmap_disable: yes mail_nfs_index: yes lock_method: dotlock mbox_read_locks: dotlock mbox_write_locks: dotlock mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 mail_log_max_lines_per_sec: 25 imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(pop3): namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~/mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~%u/mail/ hidden: yes subscriptions: yes auth default: mechanisms: plain login digest-md5 verbose: yes debug: yes passdb: driver: ldap args: /etc/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix

10 12

[Dovecot] postfix - dovecot - ldap
by Martin Sengstschmid 14 Jan '09

14 Jan '09

Hallo, I want to run postfix - dovecot - openldap working together The first problem is, that dovecot doesn't create the directories /usr/local/mail/abcshop.at/bamm/Maildir cur new tmp although I can contact the Dovecot-Server telnet localhost 143 OK .... 1 login bamm(a)abcshop.at secret OK Logged in /usr/local/mail/ is owned by virtual:virtual If I uncomment the line #mail_location = maildir:/usr/local/mail/%d/%n/Maildir dovecot creates the directories /usr/local/mail/abcshop.at/bamm/mail What is going wrong in my configuration files? Thanks for help! Martin dovecot-ldap.conf hosts = localhost dn = uid=dovecot,ou=services,dc=abcshop,dc=at dnpass = secret ldap_version = 3 base = ou=people,dc=abcshop,dc=at user_attrs = mailHomeDirectory=home,mailUidNumber=uid,mailGidNumber=gid,mailStorageDirectory=mail user_filter =(&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%u)) pass_attrs = uniqueIdentifier=user,userPassword=password pass_filter = (&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%u)) default_pass_scheme = PLAIN dovecot.conf: protocols = imap imaps disable_plaintext_auth = no ssl_cert_file = /usr/local/etc/dovecot/mail.example.com-cert.pem ssl_key_file = /usr/local/etc/dovecot/mail.example.com-key.pem log_path=/var/log/dovecot.log info_log_path=/var/log/dovecot.log mail_privileged_group = mail dotlock_use_excl = yes verbose_proctitle = yes first_valid_uid = 1000 first_valid_gid = 1000 mail_location = maildir:/usr/local/mail/%d/%n/Maildir maildir_copy_with_hardlinks = yes protocol imap { mail_plugins = quota imap_quota login_greeting_capability = yes imap_client_workarounds = delay-newmail netscape-eoh tb-extra-mailbox-sep } protocol pop3 { pop3_uidl_format = %08Xu%08Xv mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol managesieve { sieve=~/.dovecot.sieve sieve_storage=~/sieve } protocol lda { postmaster_address = postmaster(a)abcshop.at mail_plugins = quota sendmail_path = /usr/sbin/sendmail } auth_username_format = %Lu auth default { mechanisms = plain passdb ldap { args = /usr/local/etc/dovecot-ldap.conf } userdb ldap { args = /usr/local/etc/dovecot-ldap.conf } user = root socket listen { master { path = /var/run/dovecot/auth-master mode = 0600 user = virtual group = virtual } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } dict { } plugin { quota_rule = *:storage=1048576 quota = maildir }

2 4

[Dovecot] Maildir loses keywords in index beyond 26
by George Greer 14 Jan '09

14 Jan '09

(NOTE: I am not on the mailing list so please Cc: me.) dovecot-1.1.6-2.fc10.i386 (configuration not changed from stock) Fedora 10 AMD Athlon XP ext3 Maildir format doesn't store keywords in the index properly -- it goes up to 26 and then further keywords are dropped. I ran the tests with a simple script that feeds the lines from "commands" to Dovecot one at a time and waits for the "OK" before the next one. The IMAP command file is attached, as well as output of dovecot, idxview, and logview running against both mbox and maildir formats. Test mailbox created with: for i in `seq -f %02g 0 99`; do echo "Test message #$i" | mail -s "Test message #$i" sandbox@localhost; sleep 1; done For the maildir test I used the conversion script: Step 2: su sandbox -c 'perl /usr/share/doc/dovecot-1.1.6/UW-to-Dovecot-Migration/perfect_maildir.pl ~/Maildir < /var/spool/mail/sandbox' # 1.1.6: /etc/dovecot.conf # OS: Linux 2.6.25.6-55.fc9.i686 i686 Fedora release 10 (Cambridge) login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 auth default: passdb: driver: pam userdb: driver: passwd -- George Greer

2 1

[Dovecot] linuxconf migration story
by Thanos Chatziathanassiou 14 Jan '09

14 Jan '09

Hi, just subscribed after successfully (?) migrating from a somewhat old linuxconf/vimap config. dovecot-1.1.7, x86 A few issues: The index files seem to be created with the wrong permissions. I get numerous ``Dec 24 15:36:29 mail dovecot: POP3(user@domain): stat(/var/run/dovecot/imap-index/domain/user/.imap/INBOX) failed: Permission denied (euid=60107 egid=231 missing +x perm: /var/run/dovecot/imap-index/domain)'' errors. the directory /var/run/dovecot/imap-index is drwxrwxrwt 29 dovecot dovecot 4.0k Dec 24 15:57 imap-index but the directories inside it are created as drwx------ 15 60107 popusers 4.0k Dec 24 15:50 domain Questions: - Why is POP3 interested in the imap indices ? - IMAP clients do not complain all that much, as far as I can see, but what happens to POP3 ? - Is there something to configure to get rid of this ? Also, I have written a somewhat sloppy patch to determine the domain from the server's IP. That way it is not necessary to persuade all users to switch to logging in as ``user@domain'' (which I've tried with very limited success). Would this be the correct place to post the patch and discuss ? Best Regards, Thanos Chatziathanassiou

2 4

[Dovecot] Fwd: apply Sieve filter to existing mails
by Neil 14 Jan '09

14 Jan '09

Forgot reply-all... ---------- Forwarded message ---------- Date: Tue, Jan 13, 2009 at 1:39 AM Subject: Re: [Dovecot] apply Sieve filter to existing mails On Fri, Jan 9, 2009 at 11:01 AM, Jakob Grießmann <mail.an.jakob(a)googlemail.com> wrote: > Hello, > > is it possible to apply Sieve filter to existing mails, e.g. for > testing purposes whether my rules work as expected? > You can pipe mails to deliver again, and I would imagine it should run through the sieve script. And if it did, you could probably whip up a shell script to do that to all mails in a maildir... -Neil.

2 1

[Dovecot] unsubscribe dovecot.org
by Leo P CHENG 14 Jan '09

14 Jan '09

thanks

2 1

Re: [Dovecot] Can not Create Maildir using userdb
by Henry Yonathan 14 Jan '09

14 Jan '09

Dear Tim, Thank you for your clue. And I'm now able to do what I've been dreaming of :D Kind Regards, Henry Yonathan "All successful communication is Hypnosis" - Milton H. Erickson ----- Original Message ----- From: Timo Sirainen [mailto:tss@iki.fi] To: Dovecot Mailing List [mailto:dovecot@dovecot.org] Subject: Re: [Dovecot] Can not Create Maildir using userdb > On Tue, 2009-01-13 at 10:27 +0700, Henry Yonathan wrote: > > Thank you for your reply. > > So, you mean that both user_query and password_query should have the > > same amount of parameter and the mail parameter from user_query should > > have the exact parameter as userdb_mail from password_query? > > And also I've to turn on userdb prefetch {} > > password_query also needs to return the password. But other than that, > yes. ---- Belanja buku lewat internet, GRATIS ONGKOS KIRIM ke seluruh Indonesia. Kunjungi website http://www.bookoopedia.com. Hipokuku.com - Cara Baru Isi Pulsa Lewat Internet. Kunjungi website http://www.hipokuku.com untuk informasi selengkapnya.

1 0

[Dovecot] Apple patch 10
by Mike Abbott 14 Jan '09

14 Jan '09

Patch #10 allows the pop3 and imap mail processes to handle multiple clients. We know this weakens the security model but it greatly increases scalability especially when clients are idle. Here's how it works. When there are no mail processes, or none serving fewer than mail_max_connections clients, the master creates a new mail process pretty much like it does now, but with some new environment variables (PERSISTENT_MAIL_PROCESS and CONNECTION_ID, and ADVISE_SET[GU]ID instead of RESTRICT_SET[GU]ID) and a unix-domain socket connection back to the master. For future clients the master sends the client socket and a full dump of the environment via that connection. Persistent mail processes multiplex I/O for their connected clients and switch the whole environment and the effective user ID, group ID, and supplemental groups each time. The result is that most of dovecot's code and assumptions remain unchanged: the mail process executes imap/pop commands with the permissions of the user and with nearly the same environment so getenv() still works as expected. Some assumptions do change, for instance global state variables like last_partial and quota_set are forbidden, and must be stored per-client or per-user. Also it's no longer OK just to exit or panic on error, for that disconnects multiple users not just one. Some key entry points in the patch are create_mail_process() in mail- process.c and io_env_switch() in ioloop.c. Notes about this patch: - The base for this patch is dovecot-1.1.7 + Apple patch 9, not because this patch needs Open Directory but simply because it adds one line to the Apple attributions in COPYING. - This patch depends on Apple patch 7 (hash_table_create/destroy). - This patch introduces the following new config options for the pop3 and imap protocols: - mail_process_per_connection = yes - mail_max_connections = 20 The default value of mail_process_per_connection (yes) preserves the current secure model. Changing it to no allows multiplexing, with up to mail_max_connections simultaneous clients per process. See dovecot- example.conf for more info. - It restricts the mail_executable config option to minimize unpleasant surprises. When mail_process_per_connection = no the communication protocol between the master process and the mail processes changes as described above. A naive third-party mail_executable might break. See dovecot-example.conf for more info. - Some changes are intentionally awkward in order to minimize code deltas to simplify merges (of new dovecot releases into our source tree). These are so marked. Feel free to clean them up. - Changes tagged with "APPLE" are ours, including the whole new directory mail-common. All the untagged changes, including the entire contents of the new files mail-user.[ch], are copied straight from dovecot-1.2 (alpha3, I believe). We needed dovecot-1.2's support for multiple users in order to handle quotas correctly, and we needed it in 1.1. Specifically, we copied these changes from dovecot-1.2 (and a couple others too I think): 8082 http://hg.dovecot.org/dovecot-1.2/rev/db66611fd195 Added struct mail_user and fixed the code to support multiple users per process. 8084 http://hg.dovecot.org/dovecot-1.2/rev/f12f8c1da0bf Forgot to add mail-user.* files in previous struct mail_user commit. 8085 http://hg.dovecot.org/dovecot-1.2/rev/bf83aa9c3f4a Removed pool parameter from mail_namespaces_init*(). Use mail_user's pool intead. 8091 http://hg.dovecot.org/dovecot-1.2/rev/ceca59aaae89 quota-fs: compile fix for previous changes. 8096 http://hg.dovecot.org/dovecot-1.2/rev/f35a8a3dc06d Fixed FS quota compiling and Maildir++ quota with multiple users. 8109 http://hg.dovecot.org/dovecot-1.2/rev/e7929190cd32 fts-solr: Fixed compiling with recent struct mail_user changes. 8137 http://hg.dovecot.org/dovecot-1.2/rev/b2a258213ee0 Created mail_user_[try_]home_expand(). Used them for expanding mail directories. 8294 http://hg.dovecot.org/dovecot-1.2/rev/8aa69e3d27ef Trash plugin: Assign storage to all mailboxes at startup so errors are caught immediately. Also previous optimization change broke trash plugin when using multiple mail_users. This change fixes it to work again. If a change in the patch is not marked somehow with APPLE then it comes from dovecot-1.2. - Some other data structures also needed to be made per-client/user, for instance last_partial from imap-fetch-body.c, quota_set from quota- plugin.c, and process_primary_gid etc. from restrict-access.c. - Sending the master process SIGINFO (on platforms that support that signal) makes it print its view of all the clients connected to all the mail processes. Sending SIGINFO to an individual mail process makes it print its view. These are not necessarily always the same. For instance, on a config reload (SIGHUP), the master disconnects from the mail processes but leaves them running. The output is admittedly kinda geeky. - The logging tag for a persistent mail process is just "*" instead of the user name, since a process can serve multiple users. Some individual messages (such as connect/disconnect) identify the user. - The expire plugin uses global state variables which are incompatible with persistent mail processes. The patch detects trouble but does not fix it because we don't need it. We leave that as an exercise for you :). - The dict client interface is blocking which is unfortunate for persistent mail processes. The patch does not address this, since the delay is probably small and we don't use it anyway. - Some of the accounting is gnarly because of the need to honor mail_max_userip_connections in a world where a single user can have any number of active connections on any number of mail processes. Finally, here's the patch.

3 6