July 2014 - dovecot - dovecot.org

Dovecot2 Antispam MAILTRAIN backend with SpamAssassin
by Mohammad S. Babaei 15 Jul '14

15 Jul '14

Hi there, I've already asked this question on FreeBSD forums without getting any response. So, I thought it's a good idea to ask it again here. I've setup Dovecote2 (2.2.13) + Postfix + SpamAssassin on my FreeBSD 10 VPS and it works flawlessly. Though, I couldn't setup the Dovecot Antispam plugin to train SpamAssassin when I move mails to the Junk folder. I installed the plugin from FreeBSD ports and followed the official documentation [1] and the man page [2] without any success. This is my configuration: ######## # 2.2.13: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.0-RELEASE-p6 i386 ufs auth_mechanisms = plain login digest-md5 disable_plaintext_auth = no first_valid_gid = 6 first_valid_uid = 26 last_valid_gid = 6 last_valid_uid = 26 lmtp_save_to_detail_mailbox = yes mail_gid = mail mail_location = mdbox:/mail/%d/%u mail_privileged_group = mail mail_uid = mailnull managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Archive { auto = no special_use = Archive } mailbox Drafts { auto = subscribe special_use = Drafts } mailbox Flagged { auto = no special_use = Flagged } mailbox Junk { auto = subscribe special_use = Junk } mailbox Sent { auto = subscribe special_use = Sent } mailbox Trash { auto = subscribe special_use = Trash } mailbox virtual/All { auto = no special_use = All } prefix = separator = / } passdb { args = /mail/%d/dovecot_passwd driver = passwd-file } plugin { antispam_backend = MAILTRAIN antispam_mail_notspam = --ham antispam_mail_sendmail = /mail/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_spam = Junk antispam_trash = Trash quota_rule = *:storage=8GB sieve = /mail/%d/%u/dovecot.sieve sieve_dir = /mail/%d/%u sieve_global_dir = /mail/sieve sieve_global_path = /mail/sieve/globalfilter.sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-client { group = mail mode = 0660 user = postfix } unix_listener auth-master { group = mail mode = 0600 user = mailnull } user = root } service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 32 process_min_avail = 1 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 1024 } service pop3-login { inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_limit = 32 process_min_avail = 1 } ssl_cert = </etc/ssl/dovecot/cert.pem ssl_key = </etc/ssl/dovecot/key.pem userdb { args = /mail/%d/dovecot_passwd driver = passwd-file } verbose_proctitle = yes protocol lmtp { mail_plugins = " sieve" postmaster_address = postmaster(a)SomeDomain.com quota_full_tempfail = yes } protocol sieve { mail_max_userip_connections = 10 managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_line_length = 65536 } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = " quota imap_quota antispam" } protocol pop3 { mail_plugins = " quota" pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol lda { mail_plugins = " quota sieve" postmaster_address = postmaster(a)SomeDomain.com sendmail_path = /usr/local/sbin/sendmail } ######## And, this is the script /mail/sa-learn-pipe.sh: ######## echo /usr/local/bin/sa-learn $* /tmp/sendmail-msg-$$.txt echo "$$-start ($*)" >> /tmp/sa-learn-pipe.log #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/local/bin/sa-learn $* /tmp/sendmail-msg-$$.txt rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> /tmp/sa-learn-pipe.log exit 0 ######## I made it excecutable by all users using chmod a+x /mail/sa-learn-pipe.sh The problem is, there is no log in /var/log/maillog regarding the Antispam plugin, and the script won't get called when I move any message in/out to/from Junk folder. I'll appreciate any help. -- "When I give, I give myself." - Walt Whitman Links: ------ [1] http://wiki2.dovecot.org/Plugins/Antispam [2] http://hg.dovecot.org/dovecot-antispam-plugin/raw-file/tip/doc/dovecot-anti…

2 2

Permissions with mailman
by Niklas Lemcke - 林樂寬 15 Jul '14

15 Jul '14

Hi, I'm running a Mailserver, and recently also a Mailinglist. I'm sporting # dovecot --version 2.1.7 on a debian stable machine with postfix & mailman. Everything works fine, except that no archive is being created (here an excerpt from my log): Jul 2 15:29:39 blade01 postfix/qmgr[5217]: B30BCE1F52: from=<list(a)mail.myserver.com>, size=802, nrcpt=1 (queue active) Jul 2 15:29:39 blade01 dovecot: lda(list): Error: user list: Initialization failed: Initializing mail storage from mail_location setting failed: mkdir(/var/mail/vmail//list/mail) failed: Permission denied (euid=38(list) egid=38(list) missing +w perm: /var/mail/vmail/, we're not in group 5000(vmail), dir owned by 5000:5000 mode=0775) Jul 2 15:29:39 blade01 dovecot: lda(list): Fatal: Invalid user settings. Refer to server log for more information. Jul 2 15:29:39 blade01 postfix/local[2778]: B30BCE1F52: to=<list(a)mail.myserver.com>, orig_to=<list>, relay=local, delay=12578, delays=12578/0.16/0/0.17, dsn=4.3.0, status=deferred (temporary failure) Now since dovecot is running as user vmail (the other mail folders are all well managed and accessed by it, and all owned by vmail:vmail) I thought it might be saying that the user list is not in group vmail. Now: $ groups list list : list vmail and the permissions are 775, but it's still complaining about "missing +w perm". What is it referring to with "we're not in group 5000"? Thanks for any help! -- Niklas - 林樂寬

3 4

Fatal: master: service(imap): child 20258 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes })
by CJ Keist 15 Jul '14

15 Jul '14

All, Just move our mail servers over to a new mail server running postfix 2.11.1 and dovecot 2.2.13 and getting the subject line errors in my /var/adm/files. People are complaining of loosing their connections to the mail server. I've been able to google this error but haven't found fix for this yet. NOt sure where to put the drop-priv option in the config files either. Any suggestions? Var adm message: Jul 10 08:54:29 mail2 dovecot: [ID 583609 mail.crit] imap(chen): Fatal: master: service(imap): child 20258 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes }) Here is config output: root@mail2:/userM/mail-services/dovecot/sbin# ./dovecot -n # 2.2.13: /userM/mail-services/dovecot/etc/dovecot/dovecot.conf # OS: SunOS 5.11 i86pc auth_failure_delay = 5 secs auth_mechanisms = plain login cram-md5 auth_worker_max_count = 300 base_dir = /userM/mail-services/dovecot/var/run/dovecot/ disable_plaintext_auth = no hostname = mail2.engr.colostate.edu managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { driver = passwd } postmaster_address = cjay(a)engr.colostate.edu service auth { unix_listener /var/lib/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = postfix mode = 0666 user = postfix } user = root } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_cert = </userM/mail-services/dovecot/etc/ssl/dovecot.pem ssl_key = </userM/mail-services/dovecot/etc/ssl/privkey.pem userdb { args = blocking=yes driver = passwd } protocol imap { mail_max_userip_connections = 50 } protocol lda { mail_plugins = sieve } -- C. J. Keist Email: cj.keist(a)colostate.edu Systems Group Manager Solaris 10 OS (SAI) Engineering Network Services Phone: 970-491-0630 College of Engineering, CSU Fax: 970-491-5569 Ft. Collins, CO 80523-1301 All I want is a chance to prove 'Money can't buy happiness'

7 19

Changed IMAP MOVE behavior in Dovecot 2.2.x
by Iavor Stoev 15 Jul '14

15 Jul '14

Hello, After the migration to Dovecot 2.2.13 from Dovecot 2.1.17, I noticed a changed behavior when I move a message from one folder to another. I use Thunderbird with: Account Settings -> Server Settings -> When I delete a message: Just mark it as deleted. I use Thunderbird 24.6.0 with IMAP. With Dovecot 2.1.x when I move a message, the message appears in the new folder and is marked for deletion in the original folder (which gives me an easy option to undo the action if I made a mistake until I choose to Compact/Purge the folder). With Dovecot 2.2.x when I move the a message it is marked for deletion and is AUTO Compacted/Purged immediately. Is there a way to achieve the 2.1.x behavior with Dovecot 2.2.x? Thank you Iavor

3 3

Integrity check of mailboxes before dsync
by Murray Trainer 15 Jul '14

15 Jul '14

Hi All, I am migrating a large number of Dovecot mailboxes - some of them are quite old. I hit an issue yesterday with a corrupt file (index?) that was preventing the mailbox from being synced properly As soon as I removed the file the dsync worked. Is there a way to check the integrity of each mailbox / indexes before I dsync it? Thanks Murray

1 0

director / main instance
by Will Yardley 15 Jul '14

15 Jul '14

I have directors and backend servers running on the same systems (x3). To be able to run doveadm foo with a minimum of fuss (without having to list socket paths explicitly), should it be the director that's the "default"? If so, is it safe to symlink '/var/run/dovecot' to '/var/run/dovecot-director', or should I just make the director's base path /var/run/dovecot directly? in dovecot-director.conf, I have: service doveadm { inet_listener { port = 8889 } } local 192.168.x.0/24 { doveadm_password = Foo } doveadm_proxy_port = 8888 and in dovecot-main.conf, I have: service doveadm { inet_listener { port = 8888 } } local 192.168.x.0/24 { doveadm_password = Foo } protocol doveadm { auth_socket_path = director-userdb } Is this correct (and is there anything unneeded / redundant there)? # dovecot instance list path name last used running /var/run/dovecot-director director 2014-07-14 20:36:49 yes /var/run/dovecot-main main 2014-07-14 20:36:49 yes # dovecot director status mail server ip vhosts users 192.168.x.xx 100 2 192.168.x.xx 100 0 192.168.x.xx 100 0

1 0

Transition from one server to another.
by Anders Wegge Keller 14 Jul '14

14 Jul '14

A frind of mine and I are running a dedicated server, that among other things host mail for ourselves and friends and families. All in all about 15 different domains with 35-40 users. The machine in question is old, so we are doing a slow transition from the old server to the new one. So far, we've managed to move web hosts seamlessly. Due to the technical capabilities at some of the user base, it would be nice to get to a setup, where we can move individual users imaps from the old server to the new one, as we get the time to visit them. I have an idea how such a transition could go: 1. Upgrade the old dovecot 1.2.15 to 2.1.whateveritis from debian squeeze backports. 2. Set dsync up to replicate mails from the old server to the new server. I know that 2.2 is recommended, but with a limited amount of user, I'm willing to take a performance hit. 3. Migrate my parents &c to use the new server. 4. When all users have been moved on to using the new server, upgrade MX records for the domain to point at the new server. 5. When all MX records are updated, decalre success. Is this feasible, and what would the risks be. For instance, during step 4, mails are bound to arrive at bothe the old and new server for some time. Will this cause problems? Is there a simpler solution to the problem? -- /Wegge Leder efter redundant peering af dk.*,linux.debian.*

4 3

Re: Fatal: master: service(imap): child 20258 killed with, signal 6 (core not dumped - set service imap {, drop_priv_before_exec=yes }) (CJ Keist)
by Edgar Pettijohn 14 Jul '14

14 Jul '14

Have you tried: 10-master.conf service imap { drop_priv_before_exec=yes } as suggested by the error?

1 0

how many connection by pop3 ?
by ihab 14 Jul '14

14 Jul '14

how many connection by pop3 ? Thanks, Best regrades, -- View this message in context: http://dovecot.2317879.n4.nabble.com/how-many-connection-by-pop3-tp48876.ht… Sent from the Dovecot mailing list archive at Nabble.com.

4 3

OOM in Dovecot 2.2.13 imap
by Bernhard Schmidt 14 Jul '14

14 Jul '14

Hi, we run Dovecot 2.2.13 on Debian Wheezy with a couple thousand mailboxes. We have two users that repeatedly trigger an OOM condition with IMAP. Jun 23 12:53:21 lxmhs74 dovecot: imap(USER): Fatal: pool_system_realloc(268435456): Out of memory Jun 23 12:53:21 lxmhs74 dovecot: imap(USER): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) [0x7f11766cc15f] -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7f11766cc1be] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f1176685568] -> /usr/lib/dovecot/libdovecot.so.0(+0x81e80) [0x7f11766e1e80] -> /usr/lib/dovecot/libdovecot.so.0(+0x86cda) [0x7f11766e6cda] -> /usr/lib/dovecot/libdovecot.so.0(+0x86f96) [0x7f11766e6f96] -> /usr/lib/dovecot/libdovecot.so.0(+0x87b48) [0x7f11766e7b48] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_sendv+0xcd) [0x7f11766e60cd] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_send+0x1a) [0x7f11766e615a] -> /usr/lib/dovecot/modules/lib30_imap_zlib_plugin.so(+0x5849) [0x7f1175692849] -> /usr/lib/dovecot/modules/lib30_imap_zlib_plugin.so(+0x5982) [0x7f1175692982] -> /usr/lib/dovecot/modules/lib30_imap_zlib_plugin.so(+0x5b62) [0x7f1175692b62] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_flush+0x4d) [0x7f11766e5d6d] -> /usr/lib/dovecot/libdovecot.so.0(+0x85e2c) [0x7f11766e5e2c] -> dovecot/imap [USER IP UID fetch](client_output+0xe9) [0x7f1176e8d269] -> /usr/lib/dovecot/libdovecot.so.0(+0x879d5) [0x7f11766e79d5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) [0x7f11766dcfbe] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7f11766ddfb7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f11766dd049] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f11766dd0c8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f117668a7b3] -> dovecot/imap [USER IP UID fetch](main+0x2ae) [0x7f1176e8152e] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f11762f3ead] -> dovecot/imap [USER IP UID fetch](+0xd69d) [0x7f1176e8169d] Jun 23 12:53:21 lxmhs74 dovecot: imap(USER): Fatal: master: service(imap): child 33659 killed with signal 6 (core dumped) The gdb backtrace looks like this: gdb) bt full #0 0x00007f1176307475 in raise () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #1 0x00007f117630a6f0 in abort () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #2 0x00007f11766cc155 in default_fatal_finish (type=<optimized out>, status=status@entry=83) at failures.c:193 backtrace = 0x7f11789d4088 "/usr/lib/dovecot/libdovecot.so.0(+0x6c15f) [0x7f11766cc15f] -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7f11766cc1be] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f1176685568] -> /usr/lib/d"... #3 0x00007f11766cc1be in i_internal_fatal_handler (ctx=0x7fff5ffedfb0, format=<optimized out>, args=<optimized out>) at failures.c:657 status = 83 #4 0x00007f1176685568 in i_fatal_status (status=status@entry=83, format=format@entry=0x7f1176702ba8 "pool_system_realloc(%lu): Out of memory") at failures.c:295 ctx = {type = LOG_TYPE_FATAL, exit_status = 83, timestamp = 0x0} args = {{gp_offset = 24, fp_offset = 48, overflow_arg_area = 0x7fff5ffee0a0, reg_save_area = 0x7fff5ffedfe0}} #5 0x00007f11766e1e80 in pool_system_realloc (pool=<optimized out>, mem=0x0, old_size=134217728, new_size=268435456) at mempool-system.c:134 __FUNCTION__ = "pool_system_realloc" #6 0x00007f11766e6cda in o_stream_grow_buffer (fstream=fstream@entry=0x7f11789f41c0, bytes=<optimized out>) at ostream-file.c:440 size = 268435456 new_size = <optimized out> end_size = <optimized out> #7 0x00007f11766e6f96 in o_stream_add (fstream=fstream@entry=0x7f11789f41c0, data=0x7f11789fe549, size=12113) at ostream-file.c:501 unused = <optimized out> sent = <optimized out> i = <optimized out> #8 0x00007f11766e7b48 in o_stream_file_sendv (stream=0x7f11789f41c0, iov=<optimized out>, iov_count=<optimized out>) at ostream-file.c:588 fstream = 0x7f11789f41c0 size = <optimized out> total_size = <optimized out> added = <optimized out> optimal_size = <optimized out> i = <optimized out> ret = 0 __FUNCTION__ = "o_stream_file_sendv" #9 0x00007f11766e60cd in o_stream_sendv (stream=0x7f11789f4250, iov=iov@entry=0x7fff5ffee1a0, iov_count=iov_count@entry=1) at ostream.c:239 _stream = 0x7f11789f41c0 i = <optimized out> total_size = 12113 ret = <optimized out> __FUNCTION__ = "o_stream_sendv" #10 0x00007f11766e615a in o_stream_send (stream=<optimized out>, data=<optimized out>, size=size@entry=12113) at ostream.c:217 iov = {iov_base = 0x7f11789fe549, iov_len = 12113} #11 0x00007f1175692849 in o_stream_zlib_send_outbuf (zstream=0x7f11789f9340) at ostream-zlib.c:94 ret = <optimized out> size = 12113 #12 0x00007f1175692982 in o_stream_zlib_send_flush (zstream=zstream@entry=0x7f11789f9340, final=final@entry=true) at ostream-zlib.c:189 zs = 0x7f11789f9420 len = <optimized out> done = false ret = <optimized out> flush = <optimized out> __FUNCTION__ = "o_stream_zlib_send_flush" #13 0x00007f1175692b62 in o_stream_zlib_flush (stream=0x7f11789f9340) at ostream-zlib.c:235 zstream = 0x7f11789f9340 ret = <optimized out> #14 0x00007f11766e5d6d in o_stream_flush (stream=stream@entry=0x7f11789f93d0) at ostream.c:157 _stream = 0x7f11789f9340 ret = 1 __FUNCTION__ = "o_stream_flush" #15 0x00007f11766e5e2c in o_stream_default_cork (_stream=0x7f11789f9340, set=<optimized out>) at ostream.c:429 No locals. #16 o_stream_default_cork (_stream=0x7f11789f9340, set=<optimized out>) at ostream.c:422 No locals. #17 0x00007f1176e8d269 in client_output (client=0x7f11789f4470) at imap-client.c:1019 ret = 0 __FUNCTION__ = "client_output" #18 0x00007f11766e79d5 in stream_send_io (fstream=0x7f11789f41c0) at ostream-file.c:468 ostream = 0x7f11789f4250 ret = <optimized out> #19 0x00007f11766dcfbe in io_loop_call_io (io=0x7f1178a5bbe0) at ioloop.c:441 ioloop = 0x7f11789dc750 t_id = 2 __FUNCTION__ = "io_loop_call_io" #20 0x00007f11766ddfb7 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x7f11789dc750) at ioloop-epoll.c:220 ctx = 0x7f11789dd3e0 events = 0x0 event = 0x7f11789de250 list = 0x7f11789dee40 io = <optimized out> tv = {tv_sec = 297, tv_usec = 43980} events_count = <optimized out> msecs = <optimized out> ret = 1 i = <optimized out> call = <optimized out> __FUNCTION__ = "io_loop_handler_run_internal" #21 0x00007f11766dd049 in io_loop_handler_run (ioloop=ioloop@entry=0x7f11789dc750) at ioloop.c:488 No locals. #22 0x00007f11766dd0c8 in io_loop_run (ioloop=0x7f11789dc750) at ioloop.c:465 __FUNCTION__ = "io_loop_run" #23 0x00007f117668a7b3 in master_service_run (service=0x7f11789dc5e0, callback=callback@entry=0x7f1176e96a00 <client_connected>) at master-service.c:566 No locals. #24 0x00007f1176e8152e in main (argc=1, argv=0x7f11789dc390) at main.c:400 set_roots = {0x7f11770a4960, 0x0} login_set = {auth_socket_path = 0x7f11789d4048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x7f1176e96880 <login_client_connected>, failure_callback = 0x7f1176e96580 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out> I already deleted the index, but the issue persists. Bernhard

4 8