December 2017 - dovecot

Errors and dovecot crashing...
by SH Development 06 Dec '17

06 Dec '17

Our production dovecot/postfix server has been stable for a number of years. In the last month or so, we are seeing increasing errors such as these: Dec 6 15:51:20 mail dovecot: imap(redacted(a)lilythicket.com): Warning: Transaction log file /home/vmail/lilythicket.com/diana/Maildir/dovecot.index.log was locked for 322 seconds Dec 6 15:50:54 mail dovecot: imap(redacted(a)theormans.com): Warning: Maildir /home/vmail/theormans.com/connieorman/Maildir/.Junk: Synchronization took 66 seconds (1 new msgs, 0 flag change attempts, 0 expunge attempts) Dec 6 15:51:43 mail dovecot: master: Error: service(pop3-login): Initial status notification not received in 30 seconds, killing the process Dec 6 15:51:43 mail dovecot: master: Error: service(pop3-login): command startup failed, throttling Dec 6 15:51:43 mail dovecot: master: Error: service(imap-login): child 5868 killed with signal 9 Dec 6 15:51:43 mail dovecot: master: Error: service(imap-login): command startup failed, throttling Dec 6 15:55:31 mail dovecot: imap-login: Fatal: Corrupted SSL ssl-parameters.dat in state_dir: Truncated file Dec 6 15:55:32 mail dovecot: pop3-login: Fatal: Error reading configuration: Timeout reading config from /var/run/dovecot/config And so forth. Seems to be all over the place. The server slows down to a crawl. Restarting dovecot or postfix has no effect on the problem. Only a server reboot solves it, temporarily. Sometimes for weeks, sometimes for hours. The hard drive SMART status reads okay. During this time, of course, users cannot connect to check their email. Thoughts on where to go to troubleshoot this and why it’s happening? Ethon

1 0

Re: Howto authenticate smartPhone via Active Directory
by Mark Foley 06 Dec '17

06 Dec '17

On Tue, 5 Dec 2017 16:42:15 +0100 mj <lists(a)merit.unu.edu> wrote: > Hi, > > Not much time to reply now. > > On 12/05/2017 05:21 AM, Mark Foley wrote: > > mj - thanks! That the first useful example I've received from any forum/list. I'm getting ready > > to try my config (have to do so after hours), but I have some probably simple-minded questions: > > Well, that looks as if you are testing/trying out on your production > machine. Why not setup a seperate (virtual?) test server to play with..? > Use the same os version, with the same dovecot version. > Or clone your production machine, so you can test as much as you like, > without time pressure, at any given time. I've been playing with this ldap authentication for a couple of years off and on. Time isn't a problem. The issue with setting up a test environment is that I really need the domain workstations and external smartphone attempting to connect when I make a change so I can follow what's going on in the Dovecot log and maillog. It's rather simple to test a change, then put things back. I'll likely not go the test platform route for now, but thanks for the input. > > Your example is not the complete dovecot-ldap.conf.ext file, right? Have you just given me > > differences in your config from the "original"? You've kept the hosts, base, ldap_version, > > scope, deref, debug_level, and auth_bind_userdn settings in your config, right? > Not the complete file, no. I just provided the essentials. > [deleted] Ok, here's what I've come up with for dovecot-ldap.conf.ext hosts = mail.hprs.local base = dc=mail, dc=hprs, dc=local ldap_version = 3 scope = subtree deref = never debug_level = -1 auth_bind = yes auth_bind_userdn = %n@dom dn = cn=Administrator,cn=users,dc=hprs,dc=local dnpass = ******* user_filter = (&(objectclass=person)(sAMAccountName=%n)(!(userAccountControl=514))) pass_filter = (&(objectclass=person)(sAMAccountName=%n)(!(userAccountControl=514))) I've enabled auth-ldap.conf.ext in 10-auth.conf. My doveconf is listed at bottom. Unfortuntately, this doesn't work. My remote devices are not even showing as trying to connect. For internal domain LAN users I get: Dec 06 01:08:10 imap-login: Info: Disconnected: Auth process broken (disconnected before auth was ready, waited 30 secs): user=<>, rip=192.168.0.52, lip=192.168.0.2, session=<3/ZyxaVfE8PAqAA0> I do see ldap listening on 389, imap[s] (Dovecot) listening on 143 and 993, these last two are opened externally through the firewall. > For the rest: my advise is that you *really* need to pay around with > this much more. Get yourself a test environment, and play and test. > > Plus: read some dovecot/ad howto's, and try things in your own environment. > > Quick google returns: > https://www.howtoforge.com/postfix-dovecot-authentication-against-active-di… > I know my level of sophistication on this must sound like I've glibly posted a question hoping someone will do the work for me without my having to do any thinking myself, but believe me, I've been reading and experimenting with this for a very long time. I've got internal AD authentication working with GSSAPI and I've got a rather complex checkpassword program able to do authentication, so I don't think I'm a complete moron, although this project makes me feel that way. Now, I just want smartphones to authenticate with their owners' domain credentials and get them out of /etc/passwd. I believe I've read all the Dovecot wikis on ldap plus things from many other sites. I've been to that howtoforge site before. It mostly deals with setting up Postfix, which I'm not using. The dovecot bits make more sense in light of your feedback. I've tried that ldapsearch example: ldapsearch -x -h ad.example.com -D 'vmail' -W -b 'cn=users,dc=example,dc=com' with the domain user I specified in my dovecot-ldap.conf.ext with my host and dc info and I get the error ldap_bind: Strong(er) authentication required (8) additional info: BindSimple: Transport encryption required. I've seen confusing postings on this error having to do with port 636 and LDAPS -- no idea what they're talking about. My user is the Samba/domain administrator and has a pretty complex password. None of the sites I've visited on this error indicate it has anything to do with the actual password's complexity. Perhaps I'm just thick-skulled with all this. If you or anyone can see something obviously wrong with my conf, or have any suggestion at all on a baby step I can take to incm me forward, please let me know. Thanks, --Mark doveconf -n: # 2.2.33.2 (d6601f4ec): /usr/local/etc/dovecot/dovecot.conf # OS: Linux 4.4.88 x86_64 Slackware 14.2 auth_debug = yes auth_debug_passwords = yes auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/dovecot/dovecot.keytab auth_mechanisms = plain login gssapi auth_use_winbind = yes auth_username_format = %n auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no info_log_path = /var/log/dovecot_info mail_location = maildir:~/Maildir passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { driver = shadow } protocols = imap ssl_cert = </etc/ssl/certs/OHPRS/GoDaddy/Apache/2016-08-10/54e789087d419b6e.crt ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } userdb { driver = passwd } verbose_ssl = yes

2 1

Re: Disabling index files?
by Stroller 06 Dec '17

06 Dec '17

> On 5 Dec 2017, at 15:32, Gregory Finch <gfinch(a)ldmltd.ca> wrote: >> >> How do I disable the index files, please? >> … >> > > from https://wiki2.dovecot.org/MailLocation#Index_files > > add :INDEX=MEMORY to your mail_location line, ie: > > mail_location = maildir:~/Maildir:INDEX=MEMORY Many thanks! That works perfectly. Stroller.

1 0

Do we really need Solr commit as cronjob?
by Gao 06 Dec '17

06 Dec '17

I am testing Solr FTS on dovecot. Read online that some suggested to run cronjob commit every minute, and optimize once a day. I am using Solr 7.1.0 and I see some configurations: In /etc/default/solr.in.sh: #SOLR_OPTS="$SOLR_OPTS -Dsolr.autoSoftCommit.maxTime=3000" #SOLR_OPTS="$SOLR_OPTS -Dsolr.autoCommit.maxTime=60000" Also in solrconfig.xml: <autoCommit> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> <openSearcher>false</openSearcher> </autoCommit> <autoSoftCommit> <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime> </autoSoftCommit> So my question is do I still need run cronjob for commit? Do I need uncomment these lines in the solr.in.sh? Does my solrconfig.xml overwrite the setting in solr.in.sh? Thanks for help.

2 1

Re: Howto authenticate smartPhone via Active Directory
by Aki Tuomi 05 Dec '17

05 Dec '17

with passdb ldap i guess. ---Aki TuomiDovecot oy -------- Original message --------From: Mark Foley <mfoley(a)ohprs.org> Date: 03/12/2017 21:18 (GMT+02:00) To: dovecot(a)dovecot.org Subject: Re: Howto authenticate smartPhone via Active Directory Yes, you are right. This link: https://www.redips.net/linux/android-email-postfix-auth/#section2 shows: passdb pam { } used for authenticating Android. Problem #1 is that Slackware does not ship with PAM and the AD/DC Samba4 does not use it. It is used on Slackware for a domain member, but I'm not sure I should try configuring PAM on the AD/DC. Is there some otherway I can get authentication using domain credentials besides pam? the phone can send user and password. --Mark -----Original Message----- > Date: Sun, 03 Dec 2017 15:22:56 +0200 > Subject: Re: Howto authenticate smartPhone via Active Directory > From: Aki Tuomi <aki.tuomi(a)dovecot.fi> > To: Mark Foley <mfoley(a)ohprs.org>, dovecot(a)dovecot.org > > Actually you are authenticating gssapi clients from ad and everyone else from shadow. maybe you need to configure pam module? > ---Aki TuomiDovecot oy > > -------- Original message -------- > From: Mark Foley <mfoley(a)ohprs.org> > Date: 03/12/2017 06:03 (GMT+02:00) > To: dovecot(a)dovecot.org > Subject: Howto authenticate smartPhone via Active Directory > I have a Samba4 Active Directory server. Dovecot authenticates AD Users with domain credentials > using GSSAPI (Thunderbird client). I believe I have Dovecot set to attempt authentication via > shadow first and. failing that, it does authenticate via GSSAPI. > > Smartphones connect to Dovecot via port 143 and SSL. They are not domain members so if the > shadow authentication fails, no other methods are tried and no connection is made. > > What can I do with my dovecot config to fix this? > > > doveconf -n > # 2.2.15: /usr/local/etc/dovecot/dovecot.conf > # OS: Linux 4.4.88 x86_64 Slackware 14.2 > auth_debug = yes > auth_debug_passwords = yes > auth_gssapi_hostname = $ALL > auth_krb5_keytab = /etc/dovecot/dovecot.keytab > auth_mechanisms = plain login gssapi > auth_use_winbind = yes > auth_username_format = %n > auth_verbose = yes > auth_verbose_passwords = plain > disable_plaintext_auth = no > info_log_path = /var/log/dovecot_info > mail_location = maildir:~/Maildir > passdb { > driver = shadow > } > protocols = imap > ssl_cert = </etc/ssl/certs/OHPRS/GoDaddy/Apache/2016-08-10/54e789087d419b6e.crt > ssl_key = </etc/ssl/certs/OHPRS/GoDaddy/mail.ohprs.org.key > userdb { > driver = passwd > } > verbose_ssl = yes > > Thanks, Mark

3 6

Disabling index files?
by Stroller 05 Dec '17

05 Dec '17

The wiki says: > Each mailbox has its own separate index files. **If the index files are disabled**, the same structures are still kept in the memory, except cache file is disabled completely (because the client probably won't fetch the same data twice within a connection). [1] I tend to grep my maildirs quite often, and use the output to cp emails to other folders, so it's annoying when the dovecot.index.cache files show up in the results. How do I disable the index files, please? My mail server is hardly loaded, so I don't think it'll be a problem to keep the caching in memory. I can find `mailbox_list_index = no` in /etc/dovecot/conf.d/10-mail.conf but this seems to be enabled by default, and it makes no difference when I enable it explicitly - deleted dovecot.index.cache are recreated when Dovecot is restarted. The setting `mbox_min_index_size` tends to suggests it's only for mbox files, not maildirs? Thanks in advance for any help you can offer, Stroller. [1] https://wiki.dovecot.org/IndexFiles

4 3

Re: Recommended tool for migrating IMAP servers
by Joseph Tam 05 Dec '17

05 Dec '17

"Davide Marchi" <danjde(a)msw.it> writes: >> UW-IMAP's mailutil, imapsync, YippieMove and Larch. Whatever you use, *don't* use UW-IMAP's mailutil unless you got lots of time to kill. It is dreadfully slow -- I used it to export some of my users' mailboxes to Gmail or other remote mail servers, and I could almost cut&paste the messages faster. Like Aki said, if you have the same mailbox format and FS access on both sides, rsync is much simpler. You can also try exporting the old mailboxes via NFS, and with some artful symlinks, march through your user mailboxes replacing the symlinks with the instantiated local copies and have almost zero downtime. Joseph Tam <jtam.home(a)gmail.com>

4 3

Can passdb be bypassed for non-plaintext authentication mechanisms
by Mark Foley 05 Dec '17

05 Dec '17

I am using Active directory authentication via gssapi for most users. In dovecot.conf I have: auth_mechanisms = plain login gssapi auth_use_winbind = yes I also have passdb { driver = shadow } userdb { driver = passwd } for those few users who are NOT AD users. Even though the AD users do not exist in /etc/passwd or /etc/shadow, Dovecot ALWAYS first looks them up in shadow, which ALWAYS fails. The https://wiki2.dovecot.org/PasswordDatabase wiki says, "these databases can't be used with non-plaintext authentication mechanisms." Is there a way to bypass checking passdb (and userdb?) for these mechanism? --Mark

2 1

Re: Recommended tool for migrating IMAP servers
by Davide Marchi 05 Dec '17

05 Dec '17

[..] > Hi, > > I vouch for imapsync. Have used it in the past with quite a big amount > of > emails. > > cheers. > > x0p Ah, thanks x0! > Also if you have fs access on both servers, and you are using maildir, > plain rsync works just as well. > > Aki no, I've not fs access on both servers! :-/ > If you want to preserve IMAP UID:s and possibly also POP3 UIDL:s then > dovecot internal > dsync is the only tool that can do it. > > With every other tool you will face end users needing to invalidate > their local caches and > redownloading all headers if not also all mail bodies. > > Sami > Can I use this tool even if I do not know the other remote server typology? Many thanks to all!! Davide -- firma cosmogoniA <https://www.cosmogonia.org/> n o p r o v a r e n o f a r e o n o n f a r e n o n c e p r o v a r e

5 5

Re: Dovecot lmtp doesn't log
by Tomislav Perisic 04 Dec '17

04 Dec '17

Does anyone have any idea regarding this? On 17 Nov 2017 11:36, "Tomislav Perisic" <tomiperisic(a)gmail.com> wrote: Hi, We have 2 servers, server A and server B. Server A has: Postfix dovecot-2.2.33.2-1.el6.x86_64 Server B has: dovecot-2.2.33.2-1.el6.x86_64 dovecot-pigeonhole-2.2.33.2-1.el6.x86_64 Server A receives email on postfix, dovecot then takes that email from postfix and proxies it to Server B Dovecot. Dovecot on Server B takes the proxied email and delivers it with lmtp to the user inboxes. The problem is that the dovecot on server B Doesn't log anything regarding the emails that are being delivered to the mailbox via lmtp. Dovecot on server A logs everything perfectly regarding the proxy, so my assumption is that there is an issue with Dovecot lmtp logging. We changed the logging from syslog directly to a file and we noticed the same problem, missing log entries. We also tried turning on verbose logging and it didn't help. Server B: Red Hat 6.7 x86_64 rpm -qa | grep dove dovecot-2.2.33.2-1.el6.x86_64 dovecot-pigeonhole-2.2.33.2-1.el6.x86_64 doveconf -n # 2.2.33.2: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 auth_cache_negative_ttl = 0 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes listen = * mail_debug = yes mail_gid = mail mail_location = maildir:~/Maildir mail_plugins = " quota zlib" mail_uid = vmail managesieve_notify_capability = mailto namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } mailbox Virus { auto = subscribe } plugin { quota = maildir:User quota sieve_extensions = +editheader sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M sieve_quota_max_scripts = 0 sieve_trace_debug = yes sieve_trace_level = matching sieve_vacation_dont_check_recipient = yes sieve_vacation_send_from_recipient = yes sieve_vacation_use_original_recipient = yes zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = mail mode = 0666 user = vmail } } service lmtp { inet_listener lmtp { port = xx } } service managesieve-login { inet_listener sieve { port = xx } service_count = 1 } protocol lmtp { mail_plugins = " quota zlib sieve mail_log notify" } protocol imap { mail_plugins = " quota zlib imap_quota imap_zlib" } protocol sieve { managesieve_implementation_string = dovecot managesieve_logout_format = bytes ( in=%i : out=%o ) managesieve_max_line_length = 65536 managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave }

3 6