- dovecot - dovecot.org

antispam plugin pipe script seems not to be called when it "should be"
by Ben Johnson 23 Aug '16

23 Aug '16

Hello, (Full "doveconf -n" output is at the end of this message.) I'm attempting to recreate a "recipe" I had used successfully for quite some time (in Ubuntu 12.04 LTS), which allowed me to train SpamAssassin's Bayes database when someone dragged an email message from one sub-folder of the account to another. I later tried to re-implement it (in Ubuntu 14.04 LTS), and core-dumps resulted. I posted to this list about it several times, but nobody was ever able to help me solve the problem. I just tried the same (in Ubuntu 16.04 LTS), hoping that the source had evolved since then, and while I don't see a core-dump, the pipe script doesn't seem to be called. I've booby-trapped the pipe script with quite a bit of debugging output, which should be written to a file, but I don't see any indication that the pipe script is ever executed. When logged-in as an IMAP user, and I drag a message from the user's Inbox into the "Spam" folder, I do see the following activity in the syslog: Aug 18 22:12:15 example.com imap: antispam: mailbox_is_unsure(Junk): 0 Aug 18 22:12:15 example.com imap: antispam: mailbox_is_trash(INBOX): 0 Aug 18 22:12:15 example.com imap: antispam: mailbox_is_trash(Junk): 0 Aug 18 22:12:15 example.com imap: antispam: mail copy: from trash: 0, to trash: 0 Aug 18 22:12:15 example.com imap: antispam: mailbox_is_spam(INBOX): 0 Aug 18 22:12:15 example.com imap: antispam: mailbox_is_spam(Junk): 1 Aug 18 22:12:15 example.com imap: antispam: mailbox_is_unsure(INBOX): 0 Aug 18 22:12:15 example.com imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend program /bin/bash Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend program /bin/bash Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend program parameter 1 /usr/local/bin/sa-learn-pipe.sh Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend program parameter 2 --spam But the piped message is never delivered to the target mailbox. The mail log at /var/log/mail.log does not register any activity when I move messages (and maybe that's expected). Here are the contents of the /usr/local/bin/sa-learn-pipe.sh script: http://pastebin.com/7vm74jmk The permissions and mode on the script seem appropriate: $ stat /usr/local/bin/sa-learn-pipe.sh File: '/usr/local/bin/sa-learn-pipe.sh' Size: 1765 Blocks: 8 IO Block: 4096 regular file Device: fd01h/64769d Inode: 669345 Links: 1 Access: (0755/-rwxr-xr-x) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail) Access: 2016-08-18 19:09:47.307392919 -0400 Modify: 2016-08-18 19:09:31.231391749 -0400 Change: 2016-08-18 19:09:31.231391749 -0400 Birth: - What might I be doing incorrectly here? Please let me know if I can provide any other information. Thank you in advance, -Ben Full "doveconf -n" output: $ doveconf -n # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-34-generic x86_64 Ubuntu 16.04.1 LTS auth_mechanisms = plain login disable_plaintext_auth = no listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = " quota" mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_backend = pipe antispam_debug_target = syslog antispam_pipe_program = /bin/bash antispam_pipe_program_args = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_notspam_arg = --ham antispam_pipe_program_spam_arg = --spam antispam_pipe_tmpdir = /tmp antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_trash_pattern_ignorecase = trash;Deleted * antispam_verbose_debug = 1 quota = dict:user::file:/var/vmail/%d/%n/.quotausage quota_rule2 = Trash:storage=+100M quota_rule3 = Junk:ignore quota_rule4 = INBOX:storage=+100M quota_warning = storage=100%% quota-reached 100 %u %d quota_warning2 = storage=95%% quota-warning 95 %u %d quota_warning3 = storage=80%% quota-warning 80 %u %d quota_warning4 = -storage=100%% quota-below below %u %d sieve = /var/vmail/%d/%n/.sieve sieve_max_redirects = 25 } postmaster_address = postmaster(a)example.com protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-below { executable = script /usr/local/bin/quota-below.sh unix_listener quota-below { group = vmail mode = 0666 user = vmail } user = vmail } service quota-reached { executable = script /usr/local/bin/quota-reached.sh unix_listener quota-reached { group = vmail mode = 0666 user = vmail } user = vmail } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0666 user = vmail } user = vmail } ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key ssl_protocols = !SSLv2 !SSLv3 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { mail_plugins = quota imap_quota antispam } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve quota } protocol lmtp { mail_plugins = quota sieve postmaster_address = webmaster@localhost }

5 8

Director/Doveadm: Trying to iterate users, but userdbs don't support it
by Tom Sommer 23 Aug '16

23 Aug '16

Hi I'm doing: doveadm force-resync -S director1:1234 -A "*" The director gives this error: Jul 06 16:17:29 auth: Error: Trying to iterate users, but userdbs don't support it Jul 06 16:17:29 doveadm: Error: User listing returned failure Jul 06 16:17:29 doveadm: Error: Failed to iterate through some users The director has "iterate_query" set -- Tom

2 2

Re: Catch-all with LMTP and Postfix
by Aki Tuomi 23 Aug '16

23 Aug '16

On 22.08.2016 21:53, Michael Starks wrote: > On 08/22/2016 01:29 AM, Aki Tuomi wrote: >> It seems your postfix is misbehaving. Did you forget to run postmap? >> >> Aki Tuomi >> Dovecot oy > > Thanks for the response, Aki. I dug a little deeper and found this: > > lrwxrwxrwx. 1 root root 20 Jul 24 01:37 virtual_alias_maps -> > /etc/postfix/virtual > > I had been postmapping virtual_alias_maps, but not virtual. I guess > postmap doesn't follow links. When I postmapped virtual and did a > 'postfix reload' it started to work. Seems you accidentically replied to me only. Postfix uses the filename you provide as name for the db file, so running it against symlinks does follow the symlink, but it uses the symlink name as what it uses to create the .db file. You can test this by creating file and doing symlink for it and running postmap against the symlink. You'll see that it will create symlink-name.db file instead of file.db. Aki Tuomi Dovecot oy

2 1

Re: Change dovecot hostname
by Joseph Tam 23 Aug '16

23 Aug '16

"Scott W. Sander" writes: > Received: from mail.domain.test > by appserver4.domain.com (Dovecot) with LMTP id z7RGLzH4uldlPAAAxdv4Dw > for <user(a)domain.test>; Mon, 22 Aug 2016 09:03:45 -0400 > ------- > > I want the part that says "by appserver4.domain.com (Dovecot)" to say "by > mail.domain.test (Dovecot)". I don't want it to say the FQDN of the actual > host server that is running Dovecot. > > The server currently referenced as "mail.domain.test" in the headers is > postfix running on the same machine. Oh yeah, right, LMTP. I don't run LMTP myself, but I suspect the hostname is mapped from the IP of the LMTP listening interface (using /etc/hosts or DNS). Maybe you can change the IP of your listening interface to match mail.domain.test and firewall it off if that IP is public facing. >From reading the code, I can't quite grok how LMTP derives the host label, but as a last resort, you can patch at src/lmtp/commands.c in client_get_added_headers(). Joseph Tam <jtam.home(a)gmail.com>

1 0

specifying elliptic curve
by İhsan Doğan 23 Aug '16

23 Aug '16

Hi, I've noticed that Dovecot is using per default the elliptic curve sect571r1. Because not all clients might support sect571r1, I would like to set the elliptic curve manually. Is that possible? -Ihsan -- ihsan(a)dogan.ch http://blog.dogan.ch/

3 2

a question about certificates from letsencrypt
by Andreas Meyer 22 Aug '16

22 Aug '16

Hello! Certificates from letsencrypt are renewed every three months. Does that mean a MUA has to accept the renewed certificates manually everytime it is renewed? Sorry if this is OT! Greetings Andreas

6 6

Crash on opening mailbox
by Ladislav Laska 22 Aug '16

22 Aug '16

Hi! I'm currently experiencing a crash of dovecot 2.2.25 on Gentoo, upon opening mailbox. The crash only triggers on opening inbox, other mailboxes seem to work fine. It happens every time, and can be reproduced on request. I also briefly tried 2.2.19, and it behaves the same. The bug seems to be triggered by some mail, as I encountered it previously and fixed by removing some spam. I guess it might be malformed header of some kind. I can open the mailbox locally using mutt, but fail to do so remotely (using mutt or K-9 mail on android). Unforunately, I haven't been able to isolate the specific mail, as when I create a new mailbox, dovecot refuses to open it: [CANNOT] Mailbox isn't a valid mbox file (0.000 + 0.000 secs). I'm not really sure what's going on and will investigate further once I find some time to do so. I also experienced a similar problem on opening a specific mail, but it's much harder to reproduce and I currently don't have an example. The crash looks very similar, and only happens on first attempt to open a specific mail, it always works later on. I'm attaching a stack trace and dovecot -n. I'm also willing to send the binary and core dump, but not publicly, as I'm not eager to share whatever's inside with the whole world. Other than what I mentioned, I'm running pretty standard setup, with postfix on Gentoo x86_64, ext4 fs, quota and disk space is plentiful. The delivery agent is dovecot itself, in postfix/main.cf I have: mailbox_command = /usr/libexec/dovecot/deliver I'm using sieve script to file my mail, but the problem manifested even before when I was still using procmail to deliver my mail (I migrated because I had doubts about procmail and dovecot lock cooperation) I'll keep my mailbox corrupted for a few more days, and access the mail only locally, hoping to test a potential fix. Let me know if you need any more info/experimentation. Also, if you know a good way to find the mail that causes the trouble out of the core, let me know, I'd be glad to forward it for testing. Thanks and I hope we can find (and fix) the problem! -- S pozdravem Ladislav Láska <laska(a)kam.mff.cuni.cz> Katedra Aplikované Matematiky, MFF UK tel.: +420 739 464 167

2 8

Re: Change dovecot hostname
by Joseph Tam 22 Aug '16

22 Aug '16

"Scott W. Sander" writes: > I have noticed that the name of my private server running dovecot appears > in email headers rather than the public-friendly name of my server. Which headers are you taking about? If you're talking about Received: headers, that's usually inserted by your MTA, not dovecot. Joseph Tam <jtam.home(a)gmail.com>

3 2

LMTP doing passdb queries ?
by Peter Mogensen 22 Aug '16

22 Aug '16

Hi, I can see dovecot is doing a passdb query when handling the LMTP RCPT command. That's kinda unexpected for me. I would have thought it only did a userdb lookup. I have disabled lmtp_proxy to be sure it didn't do a passdb lookup to check the proxy field. Is this expected? Doesn't the LDA only do userdb lookups? /Peter

1 2

Catch-all with LMTP and Postfix
by Michael Starks 22 Aug '16

22 Aug '16

Hidy-ho, I'm having a difficult time getting catch-all working when using Dovecot LMTP. I would like *(a)example.com (everything) to go to virtual(a)example.com, where virtual is a valid virtual user. It seems that things are getting as far as LMTP, but then the mail gets bounced. To wit: Aug 21 08:02:50 hostname postfix/lmtp[4914]: 8DF8E9AFE6: to=<test(a)example.com>, relay=mail.example.com[private/dovecot-lmtp], delay=0.07, delays=0.04/0.01/0.01/0.01, dsn=5.1.1, status=bounced (host mail.example.com[private/dovecot-lmtp] said: 550 5.1.1 <test(a)example.com> User doesn't exist: test(a)example.com (in reply to RCPT TO command)) In the previous log example, test(a)example.com should have been forwarded (aliased) to the virtual mail user. Some config details: [root@hostname postfix]# cat /etc/postfix/virtual ######################################### #### Postmap this file after editing #### ######################################### # Person who should get root's mail. Don't receive mail as root! #root you # Basic system aliases -- these MUST be present MAILER-DAEMON postmaster postmaster root # General redirections for pseudo accounts bin root daemon root named root nobody root uucp root www root ftp-bugs root postfix root # Put your local aliases here. @example.com virtual(a)example.com # Well-known aliases manager root dumper root operator root abuse postmaster # trap decode to catch security attacks decode root [root@hostname postfix]# cat /etc/postfix/virtual_mailbox_domains ################################################ #### Do NOT postmap this file after editing #### ################################################ example.com [root@hostname postfix]# cat /etc/postfix/virtual_mailbox_maps ######################################### #### Postmap this file after editing #### ######################################### # From: http://wiki.dovecot.org/LDA/Postfix # Info: if you use the Dovecot LDA or LMTP it doesn't matter what you use behind the recipient address. Use "OK", the full name of the user or else. # I am using the Dovecot LMTP @example.com OK [root@hostname postfix]# dovecot --version 2.2.10 [root@hostname postfix]# dovecot -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.22.2.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_verbose = yes login_greeting = What's crackalackin? mail_debug = yes mail_home = /var/mail/%d/%n mail_location = maildir:~/mail maildir_very_dirty_syncs = yes mbox_write_locks = fcntl namespace inbox { hidden = no inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = scheme=SSHA512 /etc/dovecot/passdb driver = passwd-file } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = </etc/pki/tls/certs/hostname.com.crt ssl_cipher_list = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA256:EECDH:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA128-SHA:AES128-SHA ssl_dh_parameters_length = 2048 ssl_key = </etc/pki/tls/certs/example.com.key ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv3 !SSLv2 userdb { args = /etc/dovecot/userdb driver = passwd-file } [root@hostname postfix]# postconf -d mail_version mail_version = 2.10.1 [root@talyn postfix]# postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases body_checks = pcre:/etc/postfix/body_checks.pcre broken_sasl_auth_clients = no command_directory = /usr/sbin config_directory = /etc/postfix daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 disable_vrfy_command = yes header_checks = pcre:/etc/postfix/header_checks.pcre html_directory = no inet_protocols = all mail_owner = postfix mail_spool_directory = /var/spool/mail mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man message_size_limit = 20480000 mydestination = $myhostname, localhost.$mydomain, localhost myhostname = mail.example.com mynetworks_style = host newaliases_path = /usr/bin/newaliases.postfix notify_classes = resource, software, bounce queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix-2.10.1/README_FILES relay_domains = $mydestination sample_directory = /usr/share/doc/postfix-2.10.1/samples sendmail_path = /usr/sbin/sendmail.postfix setgid_group = postdrop smtp_header_checks = pcre:/etc/postfix/smtp_header_checks.pcre smtp_tls_note_starttls_offer = yes smtp_use_tls = yes smtpd_banner = $myhostname Microsoft ESMTP MAIL Service, Version: 6.0.3790.1830 ready. smtpd_data_restrictions = reject_unauth_pipelining, reject_multi_recipient_bounce permit smtpd_helo_required = yes smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, permit_sasl_authenticated, reject_invalid_hostname, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_destination, check_recipient_access hash:/etc/postfix/recipient_access.hash, check_recipient_access pcre:/etc/postfix/recipient_access.pcre, check_sender_access hash:/etc/postfix/sender_access.hash, check_sender_access pcre:/etc/postfix/sender_access.pcre, reject_rbl_client b.barracudacentral.org, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net permit smtpd_sasl_auth_enable = yes smtpd_sasl_local_domain = smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous smtpd_sasl_type = dovecot smtpd_tls_auth_only = yes smtpd_tls_cert_file = /etc/pki/tls/certs/example.com.crt smtpd_tls_key_file = /etc/pki/tls/certs/example.com.key smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_security_level = may smtpd_tls_session_cache_timeout = 3600s tls_random_source = dev:/dev/urandom unknown_local_recipient_reject_code = 550 virtual_alias_maps = hash:/etc/postfix/virtual virtual_mailbox_domains = /etc/postfix/virtual_mailbox_domains virtual_mailbox_maps = hash:/etc/postfix/virtual_mailbox_maps virtual_transport = lmtp:unix:private/dovecot-lmtp Thank you in advance for your assistance.

2 1