November 2015 - dovecot

Re: Connection between postfix and dovecot failing
by Sytse Kuijk 28 Nov '15

28 Nov '15

Well, I tried that. I can connect a telnet session on that port and that does not refuse the connection, it sits there waiting, but I do not know the protocol so I cannot check if it does what it is supposed to. I also checked who is listening on that port, here is the dialog: sh-3.2# sudo lsof -i -n -P | grep 10024 perl5.18 75844 _amavisd 5u IPv4 0x866f6119e39f833d 0t0 TCP 127.0.0.1:10024 (LISTEN) perl5.18 75844 _amavisd 6u IPv6 0x866f6119e03f253d 0t0 TCP [::1]:10024 (LISTEN) looks sort of correct to me… sytse -- ------------------------------------------ - Sytse Kuijk - ___ - R.S.R. n.v. - o|o - Gijzelaarsstraat 16 - \_____(.)_____/ - 2000 Antwerpen - - GSM: +32 468 20 00 00 - ___ - Phone: +32 3 216 4328 - o|o - E-mail: sytse(a)pobox.com - \_____(.)_____/ ------------------------------------------ iThink therefore iMac > On 28 nov. 2015, at 18:31, Edgar Pettifogging <edgar(a)pettijohn-web.com> wrote: > > I would bet that whatever is listening at 127.0.0.1 port 10024 didn't get restarted when the power came back on. Amavis perhaps or another proxy server of some sort. > > Sent from my iPhone > >> On Nov 28, 2015, at 11:14 AM, Sytse Kuijk <sytse(a)pobox.com> wrote: >> >> Hi all, >> >> Hope someone has been here and done that before. >> I am running a Mac mini with OSX server version 10.5, that is on El Capitan, but the problem I have started on Yosemite (the previous version) and did not change at all in the upgrade (I had hoped it would have solved itself). >> >> As you might know, OSX server uses postfix and dovecot as mailing engines, good choice. >> Since we suffered a short power outage a little over a week ago, the mail has failed in delivering messages to the local mailboxes. >> We can send mail out, check the mailboxes (they never change right now),etc. but incoming mail was first just not delivered and after a week warning are sent back to the senders, telling there is a temporary problem (hope that this message is right and I can solve the issue). >> >> What seems to go wrong is that postfix cannot connect to the local MTA. The error message that appears in several places is: >> >> recipient=ann.cools(a)surikata.be >> offset=769 >> dsn_orig_rcpt=rfc822;ann.cools(a)surikata.be >> status=4.4.1 >> action=delayed >> reason=delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused >> >> I find all sorts of explanations and cures, but none work up to now. >> I tried to find lock files that might be blocking, deleted some files ending in lck with zero size, with no result. >> I tried to switch off the spam/virus/junk features, no difference (some mail said that messages in the queue will not get through, but new ones would. Not so, not a single message up to now). >> I also get alerts in the Server application, warning that: >> >> An unassigned mail directory has been detected and is scheduled to be deleted. >> >> The unassigned mail directory located in: >> /Library/Server/Mail/Data/mail/702C16A8-6B2D-42D8-9362-659DD2949AFC >> is not associated with any user account and is scheduled to be deleted in 1 days. >> >> In the mean time no mail, and unfortunately, we need it badly. >> >> Any suggestions? >> >> Million thanks for your kind support. >> >> sytse >> -- >> ------------------------------------------ >> - Sytse Kuijk - ___ >> - R.S.R. n.v. - o|o >> - Gijzelaarsstraat 16 - \_____(.)_____/ >> - 2000 Antwerpen - >> - GSM: +32 468 20 00 00 - ___ >> - Phone: +32 3 216 4328 - o|o >> - E-mail: sytse(a)pobox.com <mailto:sytse@pobox.com> - \_____(.)_____/ >> ------------------------------------------ >> >> iThink therefore iMac

1 0

ldap paging
by Andre Helwig 27 Nov '15

27 Nov '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi List, in ldapsearch you are able to use paging in case of limited results on ldap server side with "-E pr=10000/noprompt" Is there any way to get the same effect for the iterate query? Thanks Andre -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJWWHvwAAoJEAoTNwRDnEhRVmMH/0+vGRx7urQ49bHbzc4asD7r XlbwS7whkDKrJpflK39PLPw5BJoptKhwLNOueotFv9Pq8EYQBkN/gX69BzxeoFk8 DZpiP/ogYQIG1lUijes4UkbttVpnviy9/mLBKjIfIkvGDu0q3zwuq5876q6t2Qxx kJkpLSizxAfnGLeD03txVPAc8wGzxhxkyEdosQ+ARriYI1Ce+dfK28z6hSAEXMA+ 4JZEvIsy6P+xepFCFYILFWeRO2YBzqHtgtjKu8KCiyBdUPG0Nii4z2YmN2AoCj7m rQ8l4CeTDw+lk0pMy26kCsmh+PK2/OXCUGjxgYJOR3mxn2HNSIfUXFVWs4ri464= =qWS+ -----END PGP SIGNATURE-----

1 0

Sieve Max Redirect
by Shakeel Shahzad 27 Nov '15

27 Nov '15

Hello, I have configured iRedMail ( 0.9.1) on Ubuntu (14.04 LTS). In my dovecot.conf file, I have set sieve_max_redirects as 500. In dovecot (2.2.9) configuration file, I see sieve_max_redirect as 100 but when I add email addresses more than 32, I see that redirect fails. Can you please help me to see this issue and fix it through? If you need anything from my side, please let me know. Regards... Shakeel

3 4

doveadm-backup verbosity?
by Andreas Fischer 27 Nov '15

27 Nov '15

Hi, according to the man page doveadm-backup, the verbosity switch "-v" should do the following: "Enables verbosity, including progress counter" However, i see no difference running the following command with or without the "-v" switch: >>doveadm -v backup -R -A remote:myhost The command itself works without problem, two mailboxes are correctly back-upped, but the output is only an empty line. Not sure if this is expected behavior. Regards # 2.2.19: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: Linux 4.2.5-1-ARCH x86_64 dsync_remote_cmd = ssh -p 19022 -l root %{host} doveadm dsync-server -u%u first_valid_uid = 99 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate editheader passdb { args = nopassword=y driver = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_extensions = +editheader } protocols = service anvil { process_min_avail = 0 unix_listener anvil-auth-penalty { mode = 064 } } ssl = no userdb { args = /etc/dovecot/users default_fields = uid=nobody gid=nobody home=/backup/tmp/mail/%d/%n mail=sdbox:~/box driver = passwd-file }

1 0

fts_solr does not quote lucene's special characters
by Michael Stilkerich 27 Nov '15

27 Nov '15

Hello, I am using Dovecot 2.2.19 with the solr backend for full text search and experience the following issue. When I issue a search that includes characters that are part of lucene's query syntax (e.g. the double quote character), dovecot does not escape the special characters and issues a request with invalid syntax to the solr server. For example, I search for foo"bar, then my IMAP client (roundcube) sends the following to dovecot: A0003 UID THREAD REFS US-ASCII ALL UNDELETED BODY "foo\"bar" Dovecot in turn uses the following request URI to query SOLR: "GET /solr/select?fl=uid,score&rows=113739&sort=uid+asc&q=body:%22foo%22bar%22&fq=%2Bbox:4d86808c56d44282b35cea8700dda537+%2Buser:%22myuser%22 HTTP/1.1" 400 994 Solr responds with error 400 and the following body: type Status report message null description The request sent by the client was syntactically incorrect. This is because the " character in foo"bar is not quoted in the solr request. When I adapt the request to quote the " character solr succeeds with the expected result: "GET /solr/select?fl=uid,score&rows=113739&sort=uid+asc&q=body:%22foo%5C%22bar%22&fq=%2Bbox:4d86808c56d44282b35cea8700dda537+%2Buser:%22myuser%22 HTTP/1.1" 200 535 A fix for this would be highly appreciated. Cheers, Michael

2 1

auth_request_get_var_expand test fails on big endian arches.
by Rafael dos Santos 27 Nov '15

27 Nov '15

Hi, when running the tests in a big endian machine, I get the following error: 0 / 1 tests failed test-auth-request-var-expand.c:99: Assert failed: strcmp(str_c(str), test_output) == 0 test-auth-request-var-expand.c:104: Assert failed: strcmp(str_c(str), test_output) == 0 auth request var expand short and long ............................... : FAILED test: random seed #1 was 1446803094 auth request var expand flags ........................................ : ok test-auth-request-var-expand.c:158: Assert failed: strcmp(str_c(str), test_output) == 0 auth request var expand long-only .................................... : FAILED auth request var expand usernames .................................... : ok auth request var expand funcs ........................................ : ok 2 / 5 tests failed Makefile:1257: recipe for target 'check-test' failed Looking into it I found that auth_request_get_var_expand_table ends up calling the net_ip2addr function. This function, in turn, uses ntohl to convert from network byte order to host byte order. In big endian machines, that means that the string output will be ------------- big endian --------------- +user@+domain1@+domain2 +user +domain1@+domain2 +service 7.91.205.21 73.150.2.210 54321 +password +mech secured 21 210 valid instead of ---------- expected output --------- +user@+domain1@+domain2 +user +domain1@+domain2 +service 21.205.91.7 210.2.150.73 54321 +password +mech secured 21 210 valid Let me know in case you need help testing on big endian arches. Att. -- Rafael Fonseca

2 3

IMAP COPY creates invalid index data with zlib and mail_log plugins enabled but zlib_save off
by Robert L Mathews 27 Nov '15

27 Nov '15

I've noticed that maildir IMAP COPY commands can generate invalid dovecot.index entries when all seven of the following are true: - The zlib plugin is enabled; - The zlib_save/zlib_save_level options are NOT enabled; - The source message being copied is compressed; - The mail_log plugin is logging "copy" events; - The mail_log_fields setting includes at least one message header; - The destination mailbox folder has an index file that is recording the logged headers; - The source mailbox folder does NOT have an index file recording the logged headers. In this situation, copying a message results in an index entry in the destination that has all the headers blank, like this: RECORD: seq=5, uid=182415, flags=0x08 (Seen) - ext 1 modseq : 3 (0300000000000000) - ext 2 cache : 2352 (30090000) - cache offset=2352 size=120, prev_offset = 0 - hdr.FROM: - hdr.MESSAGE-ID: - hdr.SUBJECT: - hdr.DATE: - hdr.BCC: - hdr.CC: - hdr.CONTENT-TYPE: - hdr.IN-REPLY-TO: - hdr.NEWSGROUPS: - hdr.PRIORITY: - hdr.REFERENCES: - hdr.REPLY-TO: - hdr.TO: - hdr.X-PRIORITY: For these copies, the mail_log plugin records blank header data in the log, too, like this (despite the message having a valid message-ID, from, and subject): Oct 29 21:11:42 wheezy dovecot: imap[31763]: user=<1(a)example.com> ip=192.168.0.4 session=RV/MmUojyADAqAAE copy from INBOX.Saved: box=INBOX.Trash, uid=182412, msgid=, size=1357, vsize=1392, from=, subject=, flags=(\Seen) I've found this occurs in at least Dovecot 2.1.7, 2.2.13, and 2.2.18 (all packaged Debian versions). Changing any of the seven things I mentioned above fixes it. I've tried to find the code that causes this, but not yet been successful. My theory is that when the mail_log plugin calls mail_get_first_header(), and there's no index header record to read it from, it looks at the real message on disk. Perhaps it somehow operates on a compressed copy of the message although it expects uncompressed plaintext, so it doesn't find any of the headers. The "missing" headers then get cached to the destination index file as a side-effect. The copied maildir message itself is fine, by the way. It's an identical, still-compressed copy of the original message. And deleting the destination "dovecot.index*" files that contain the bad data causes Dovecot to rebuild the index with the correct header data. It's unusual to use the zlib plugin without the zlib_save/zlib_save_level options enabled, of course. That's probably why (as far as I can tell) there have been no reports of this before. I noticed it only because I was testing a small amount of manual mail compression before deploying it for all mail. Here's the "doveconf -n" of a system in the "has the problem" state: # 2.2.18: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 3.2.0-4-686-pae i686 Debian stretch/sid auth_mechanisms = plain login auth_username_format = auth_verbose = yes auth_verbose_passwords = sha1 default_vsz_limit = 1 G disable_plaintext_auth = no lda_mailbox_autosubscribe = yes listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_log_format_elements = user=<%u> pid=[%{mail_pid}] method=%m rip=%r lip=%l %c mail_fsync = never mail_location = maildir:~/ mail_log_prefix = "%s[%{pid}]: user=<%u> ip=%{rip} session=%{session} " mail_max_userip_connections = 100 mail_plugins = mail_log notify zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate spamtest spamtestplus namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = separator = . type = private } passdb { args = /etc/dovecot/dovecot.users driver = passwd-file } plugin { fts = lucene fts_lucene = whitespace_chars=@. mail_log_events = copy delete undelete expunge mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = vsize size flags box uid box msgid size flags vsize from subject sieve = %h/mailbox.sieve sieve_after = %h/../../domain-after.sieve sieve_before = %h/../../domain-before.sieve sieve_dir = %h/sieve sieve_extensions = -reject +spamtest +spamtestplus sieve_global_dir = %h/../../sieve-global-include-scripts sieve_spamtest_max_value = 7 sieve_spamtest_status_header = X-Spam-Level sieve_spamtest_status_type = strlen } protocols = pop3 imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { process_min_avail = 8 service_count = 0 } service imap { process_limit = 2000 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service pop3-login { process_min_avail = 8 service_count = 0 } service pop3 { process_limit = 2000 } ssl = no userdb { args = /etc/dovecot/dovecot.users driver = passwd-file } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail imap_logout_format = %i bytes in, %o bytes out } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = ; %i bytes in, %o bytes out, %t top (%p bytes), %r retr (%b bytes), %d of %m deleted, mailbox size %s bytes pop3_uidl_format = UID%u-%v } protocol lda { mail_fsync = optimized mail_plugins = mail_log notify zlib sieve } protocol lmtp { mail_fsync = optimized mail_plugins = mail_log notify zlib sieve } -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/

2 2

Problems Converting from Cyrus to Dovecot (cyrus2dovecot)
by Forrest 27 Nov '15

27 Nov '15

I have been attempting to use the cyrus2dovecot script, to no avail. I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. inboxes=the "myaccount" that was copied over /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ --cyrus-sub /home/varimap/user/%h/%u.sub \ --dovecot-inbox /home/myaccount/dovecot/Maildir \ myaccount the log output complains of: cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. All directory paths are correct. Thanks.

6 9

logging imap searches ?
by mancyborg＠gmail.com 27 Nov '15

27 Nov '15

Hi thanks to this list support I'm using Dovecot's MailLog plugin http://wiki2.dovecot.org/Plugins/MailLog to log the following imap events: delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create and I'm very happy. Question: is it possible to also log imap searches https://tools.ietf.org/html/rfc3501#section-6.4.4 ? Thank you, Mike

1 0

imap logging ?
by mancyborg＠gmail.com 27 Nov '15

27 Nov '15

Hi I'm trying to log my users imap actions, like when creating a folder, moving an email or deleting an email. So I've enabled 'mail_debug' and I'm checking /var/log/dovecot/debug.log this is what happens when I delete an email: Nov 26 07:46:38 auth-worker(1555): Debug: sql(XXX,127.0.0.1): query: SELECT password FROM mailbox WHERE username = 'XXX' and active = 1 and restrictedAccess = 0 Nov 26 07:46:38 auth: Debug: client out: OK 1 user=XXX Nov 26 07:46:38 auth-worker(1555): Debug: sql(XXX,127.0.0.1): SELECT '/var/vmail/XXX/XXX' as home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota) AS quota_rule FROM mailbox WHERE username = 'XXX' Nov 26 07:46:38 auth: Debug: master out: USER 374472705 XXX home=/var/vmail/XXX/XXX uid=5000 gid=5000 quota_rule=*:storage=524288 Nov 26 07:46:38 imap(XXX): Debug: Effective uid=5000, gid=5000, home=/var/vmail/XXX/XXX Nov 26 07:46:38 imap(XXX): Debug: Quota root: name=User quota backend=maildir args= Nov 26 07:46:38 imap(XXX): Debug: Quota rule: root=User quota mailbox=* bytes=536870912 messages=0 Nov 26 07:46:38 imap(XXX): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 Nov 26 07:46:38 imap(XXX): Debug: maildir++: root=/var/vmail/XXX/XXX/Maildir, index=/var/vmail/XXX/XXX/Maildir/indexes, control=, inbox=/var/vmail/XXX/XXX/Maildir, alt= and when creating a folder, access an email or moving an email, the output is basically the same: I'm unable to find the actual IMAP command. So, question: is there a way to log IMAP commands to a file ? Thank you, Mike

2 2