October 2017 - dovecot

Re: My sub-folder with Outlook work-around to date - update 201710091046
by David.M.Clark 09 Oct '17

09 Oct '17

Hi Mike, I haven't seen your messages come through the feed so something is not working for your e-mail address there? No I am not using Sieve - the server has a third party handling spam etc so the Linux box is very basic SendMail, Dovecot and RoundCube setup (along with my own procmail based web interface for e-mail forwarding and away message responses). I struggle on. On 09/10/17 12:30, Mike wrote: > > Mr. Clark, > > I would post this question to the maillist, but for some unknown reason, > it will not post my messages. > > Are you currently using Sieve with your mbox setup? > > If so, would you be willing to share your configuration of that? > > If not, I guess I'll just convert to maildir like you are doing. > > I'd be willing to help in any way you feel needed. > > Thanks. > > Mike Barsalou > > Quoting "David.M.Clark" <david(a)davrom.com>: > >> Hi All, >> >> Will start using an "update" tag in my subject just to not cause >> confusion (mainly for yours truly). >> >> Pursuant to my victory in finding Maildir and mbox being used if >> either exist (and if both still seems to favour Maildir - a mistake my >> end: "rmdir -f mail" fixed it), I have found one issue with not >> declaring the mailbox location: it won't import your new e-mail from >> your Inbox (/var/spool/mail/user in my case). If I run the conversion >> script mb2md with a '-m', I get duplicate entries in Outlook's Inbox - >> I understand why, just a shame it doesn't sync with what 'isn't' >> already there. >> >> So yesterday and today I have done additional testing as the customer >> uses RoundCube and mobile devices - so attacking the same Linux server >> I have Thunderbird and a couple of Android e-mail clients. Outlook is >> the only stick in the mud with needing to manually subscribe to >> folders when I create them on other devices - again could be me as I >> don't use Outlook. (Thunderbird and other e-mail clients, all good >> with immediate auto-sync). >> >> I looked at the wiki to try and run both mail_location settings with >> the wiki suggested: >> >> ##namespace { >> ## separator = / >> ## prefix = "#mbox/" >> ## location = mbox:~/mail:INBOX=/var/mail/%u >> ## inbox = yes >> ## hidden = yes >> ## list = no >> ##} >> ##namespace { >> ## separator = / >> ## prefix = "Maildir/" >> ## location = maildir:~/Maildir >> ## list = yes >> ##} >> >> in 10-mail.conf but as you can see, I have commented it out now as it >> introduced some more layers of errors which I have to get back to - >> "Duplicate namespace prefix" was one issue. >> >> The goal is to run the server with the existing mbox users blissfully >> unaware they are being converted one-by-one so it looks like I 'can' >> run both formats but just need to get the configs right. Was hoping >> the mb2md would just sync new stuff so I could cron script the Inbox >> check for the users that are converted to Maildir until I can throw >> the switch and be done with the cut-over..... or I could be dangerous >> and remove and import each time.... but that could lead to more dramas >> at the user end. >> >> Looks like more R&D is needed so will persevere - annoying to get so >> close though, might see if I can just sync in the Inbox items that are >> 'new' - that means I am ready to roll on the customer site...... >> >> On 09/10/17 00:12, David.M.Clark wrote: >>> Wow excellent feedback Joseph, Philon and Aki, >>> >>> You have all pointed me in the right direction. Tonight I have tested >>> Dovecot with just the minor configs I do for Dovecot setup and I have >>> commented out the mbox/Maildir lines in 10-mail.conf from my testing: >>> >>> #mail_location = >>> #mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u >>> #mail_location = maildir:~/Maildir >>> >>> I had run up some test files (folders) under mbox using >>> "/u/home/user/mail_test" and then renamed this to 'mail' (so >>> /u/home/user/mail). >>> Ran some tests with sends/receives, created folders - all good. >>> Created mbox text folders. >>> >>> Then had my initial test 'Maildir_test' populated with test folders >>> which I renamed to /u/home/user/Maildir - and each time I restarted >>> Outlook it was obeying the folder 'format' based on whether I had >>> 'mail' or 'Maildir' in place. >>> >>> I did see in the posts and docs on-line somewhere that someone >>> mentioned the later implementations of Dovecot, without hard-setting >>> either mbox or Maildir in the 10-mail.conf file, Dovecot, based on >>> the presence of the particular directory in the user's $HOME >>> directory obeys the rules based on mbox/Maildir respectively. >>> >>> I haven't needed to touch the namespace rules as per: >>> >>> https://wiki2.dovecot.org/Namespaces >>> >>> so the only files that bear any changes from the installed config in >>> conf.d are: >>> >>> -rw-r--r-- 1 root root 14531 Oct 8 23:28 10-mail.conf >>> -rw-r--r-- 1 root root 2965 Oct 7 08:49 10-master.conf >>> -rw-r--r-- 1 root root 1728 Oct 7 08:52 10-ssl.conf >>> >>> This will make migration so much easier with converting one user at a >>> time and then getting them to start Outlook and re-sync things again. >>> >>> This will save a massive job from being just that and can be done in >>> batches of users to suit time constraints. >>> >>> I will post here once again to let you all know how I fare but just >>> proves to me over and over again the power of the Linux e-mail server >>> with Dovecot for IMAP and being able to keep with the older SendMail >>> MTA (been working with Sendmail since 1996 so....). >>> >>> Thank you all so much for taking time to respond. >>> >>> On 08/10/17 20:32, Aki Tuomi wrote: >>>> You can give mail location and home directory from userdb or passdb. >>>> >>>> For userdb, return mail=something:~/something and for home use >>>> home=/path/to/home >>>> >>>> For passdb, you can return userdb_mail and userdb_home >>>> >>>> Aki >>>> >>>>> On October 8, 2017 at 1:14 PM Philon <bytesplit(a)gmail.com> wrote: >>>>> >>>>> >>>>> Hi David, >>>>> >>>>> I somehow remember that there is a way to get the mail_location >>>>> from per-user-settings. I had this once when migrating from maildir >>>>> to mdbox. >>>>> >>>>> You might want to use Google and the mailing list archives for the >>>>> details but in summary it went like this: >>>>> >>>>> - set Dovecot to grab the mail_location from user-settings (I think >>>>> we used MySQL db field - or check docs for auto:) >>>>> - have login script which converts maildir from setup A to B and >>>>> update the user setting >>>>> - kick the user once again to apply the setting (I’m not sure about >>>>> this) >>>>> >>>>> This then automates the conversion process to happen on next user >>>>> login. >>>>> >>>>> Be warned though that today with all the idle phones, probably many >>>>> users will re-login after you restart the server. Perhaps the >>>>> scripting could handle a timer which randomizes this a little. >>>>> >>>>> >>>>> Philon >>>>> >>>>> >>>>>> Am 07.10.2017 um 02:22 schrieb David.M.Clark <david(a)davrom.com>: >>>>>> >>>>>> Hi All, >>>>>> >>>>>> Thanks to MJ for the suggestion as my only hope, and I have tested >>>>>> this with Outlook 2016 and Thunderbird today - seamless creation >>>>>> of folders. >>>>>> >>>>>> Now the mammoth task of converting customers with masses of mbox >>>>>> folders to MailDir format using the on-line perl script: >>>>>> mb2md-3.20.pl >>>>>> >>>>>> Works nicely and will be what gets us out of trouble. My only >>>>>> sadness is that Outlook then has to be 're-subscribed' to the >>>>>> folders again.... which makes sense or maybe I didn't give it long >>>>>> enough - its IMAP folder syncs are extremely slow compared to >>>>>> Thunderbird which I also tested in parallel. Thunderbird didn't >>>>>> need any re-sync, it just found the new subscription setup and was >>>>>> ready to go. But in all fairness, Outlook is still really only an >>>>>> Exchange compatible product for the most part. >>>>>> >>>>>> Would be great to run both mbox and MailDir at the same time so we >>>>>> can do the slow user-by-user process rather than having them all >>>>>> down at some point completely - might look at running Dovecot on a >>>>>> different port maybe with a different ".conf" but thinking this >>>>>> may not work anyway or cause issues.... back to my server tests to >>>>>> see :-) >>>>>> >>>>>> I use additional ports to 143 for external IMAP restrictions >>>>>> anyway so might be able to do this method somehow. >>>>>> >>>>>> Anyway, for those interested in where I got to in my testing so >>>>>> far... >>>>>> >>>>>> 1. Download perl script "mb2md-3.20.pl" from: >>>>>> >>>>>> http://batleth.sapienti-sat.org/projects/mb2md >>>>>> >>>>>> 2. Changed Dovecot /etc/dovecot/conf.d/10-mail.conf mbox entry to >>>>>> MailDir: >>>>>> >>>>>> #mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u >>>>>> mail_location = maildir:~/Maildir >>>>>> >>>>>> 3. Restarted Dovecot >>>>>> >>>>>> 4. su as user and in their home (/u/home/user on my setup): >>>>>> >>>>>> /u/packages/mb2md-3.20.pl -m >>>>>> >>>>>> then: >>>>>> >>>>>> mv mail mail_old >>>>>> >>>>>> then: >>>>>> >>>>>> /u/packages/mb2md-3.20.pl -s mail_old -R >>>>>> >>>>>> then: >>>>>> >>>>>> Login with Outlook and subscribe to IMAP folders again. >>>>>> >>>>>> Not a big fan of gazillions of little eml based text mail files in >>>>>> directories but hey, if it fixes our issues, maybe I need to stop >>>>>> wambulancing about it all and do as I am told :-) >>>>>> >>>>>> Hope this helps - will keep this e-mail updated on findings if >>>>>> anything else comes to light. >>> >> >> -- >> >> As always, I remain at your service. >> >> Kindest Regards, >> David.M.Clark (Director - Senior Linux/UNIX Consultant) >> =--------------------------------------------------------------------------= >> >> Davrom Consulting Pty Ltd Mobile: 0418763124 >> PO Box 1644, Sunnybank Hills, 4109 E-mail (Work): david(a)davrom.com >> ABN: 81 096 990 804 E-mail (Priv): >> dmc1961(a)dmc1961.id.au >> Website: http://davrom.com Skype: dmc1961 >> Podcast: http://ldup.com.au Google: dmc1961(a)gmail.com >> =--------------------------------------------------------------------------= >> >> Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, >> Networking/Internet, E-mail/Web Technologies >> =--------------------------------------------------------------------------= >> >> Please note: Any e-mail communication bearing this signature is for >> the exclusive purpose of the sender and is not for publication >> without the expressed permission of the sender or respective >> sender's organisation. >> =--------------------------------------------------------------------------= >> > > -- As always, I remain at your service. Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant) =--------------------------------------------------------------------------= Davrom Consulting Pty Ltd Mobile: 0418763124 PO Box 1644, Sunnybank Hills, 4109 E-mail (Work): david(a)davrom.com ABN: 81 096 990 804 E-mail (Priv): dmc1961(a)dmc1961.id.au Website: http://davrom.com Skype: dmc1961 Podcast: http://ldup.com.au Google: dmc1961(a)gmail.com =--------------------------------------------------------------------------= Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies =--------------------------------------------------------------------------= Please note: Any e-mail communication bearing this signature is for the exclusive purpose of the sender and is not for publication without the expressed permission of the sender or respective sender's organisation. =--------------------------------------------------------------------------=

1 0

My sub-folder with Outlook work-around to date
by David.M.Clark 09 Oct '17

09 Oct '17

Hi All, please be kind, this is my first e-mail to the list :-) I actively support CentOS based e-mail servers running Dovecot, Sendmail, Spammassassin and 3 x SOGo based setups. Dovecot is my goto IMAP server and have used it and modifications to it to net excellent results for years. Then we have people who insist on only using Outlook (and in some instances the MS Live Messenger thingy). Some shenanigans in recent years have arose with using sub-folders in the Outlook clients (2013 and 2016). Traditionally, placing a "/" after the name of a newly desired e-mail folder has netted the result of something like: "Rentals/" creating: /u/home/someuser/mail/Rentals/ Under which users then create actual e-mail folders under the "Rentals" Linux directory as such. With Outlook 2013 and 2016 this seems to have stopped working and so I implemented a work-around where the user creates a normal folder, example "Rentals++", and I have written a cron script that trawls the $HOMEs each minute and if it finds a folder with a "++" at the end, it creates the folder as a directory, so: /u/home/someuser/mail/Rentals++ becomes: /u/home/someuser/mail/Rentals/ and adds this new subscription to their .subscription folder. It also sends an e-mail to the user advising that the new folder is created and they can proceed to use the "Rentals" folder for adding sub-folders (as in real text based mail folders). The script was a quick work-around one weekend in a mad flurry to get things working and to date works but is not 'user-proof'. So I am now looking at developing an internal web interface to do the same thing and hoping with more controls, has no or far less margin for user error. Long times of Outlook folder refreshes don't help and users sometimes end up with issues that require my Linux command line help. I have been trawling e-mail forums for some time now and have not seen any other work-arounds (or perhaps I am living under a rock) but before I embark on this web interface adventure, I just wanted to make sure I had not missed some fundamental 'bit' that I should be observing. All servers are either CentOS 6.9 (or slightly less) and CentOS 7 with the latest updates and for things like Thunderbird and Roundcube and SOGo, work well. I need to experiment with the whole "/" for these but I am currently driven by the enforced Outlook chains. Any input from you guys on whether this is my best approach or 'hey mate, just do this', would be much appreciated. I am happy to share my travels script/web-wise if this is the only option to date. -- As always, I remain at your service. Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant) E-mail: david(a)davrom.com =--------------------------------------------------------------------------= Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies =--------------------------------------------------------------------------=

4 7

Released Pigeonhole v0.4.21.rc1 for Dovecot v2.2.33.rc1.
by Stephan Bosch 09 Oct '17

09 Oct '17

Hello Dovecot users, Here is the Pigeonhole hole release candidate that goes with the Dovecot release candidate. Nothing really special going on, just a few changes and fixes that accumulated over the last few months. Changelog v0.4.21: * redirect action: Always set the X-Sieve-Redirected-From header to sieve_user_email if configured. Before, it would use the envelope recipient instead if available, which makes no sense if the primary e-mail address is available. + vacation extension: Allow ignoring the envelope sender while composing the "To:" header for the reply. Normally, the "To:" header is composed from the address found in the "Sender", "Resent-From" or "From" headers that is equal to the envelope sender. If none is then found, the bare envelope sender is used. This change adds a new setting "sieve_vacation_to_header_ignore_envelope". With this setting enabled, the "To:" header is always composed from those headers in the source message. The new setting thus allows ignoring the envelope, which is useful e.g. when SRS is used. + vacation extension: Compose the "To:" header from the full sender address found in the first "Sender:", "From:" or "Resent-From:" header. Before, it would create a "To:" header without a phrase part. The new behavior is nicer, since the reply will be addressed to the sender by name if possible. - LDA Sieve plugin: Fixed sequential execution of LDAP-based scripts. A missing LDAP-based script could cause the script sequence to exit earlier. - sieve-filter: Removed the (now) duplicate utf8 to mutf7 mailbox name conversion. This caused problems with mailbox names containing UTF-8 characters. The Dovecot API was changed years ago, but apparently sieve-filter was never updated. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.21… http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.21… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release candidate and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

Permission denied error on private key...
by SH Development 08 Oct '17

08 Oct '17

I have a working dovecot/postfix/mysql server running and was trying to set up another one for replication purposes. If I copy my certificates from the working server, everything works fine. However, I purchased another updated certificate for the replication server, and I cannot get dovecot to start up. Keep getting: doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 31: ssl_key: Can't open file /etc/pki/dovecot/private/mailserver.crt: Permission denied Different permissions on the mailserver.crt have no effect on the error. Maybe something I did in creating the file? I’m a little fuzzy when it comes to how you’re supposed to create the whole thing. Here are the steps I took: openssl genrsa -out mailserverkey.crt 2048 I then took the CSR, submitted it to Comodo, a certificate was generated and emailed back to me. I placed the certs in the appropriately referenced places on the server… And that is where I’m at. I’ve spent quite a bit of time comparing my working server’s configuration files to the new server and cannot find any differences in the SSL configuration, OTHER than the certs themselves. Is dovecot looking for something I didn’t do correctly in the creation maybe? Jeff J. Starion Technologies, LLC 816-331-0030

2 4

Re: My sub-folder with Outlook work-around to date
by Joseph Tam 08 Oct '17

08 Oct '17

"David.M.Clark" <david(a)davrom.com> writes: > Would be great to run both mbox and MailDir at the same time so we can > do the slow user-by-user process rather than having them all down at > some point completely ... - might look at running Dovecot on a different I believe you can do this by using a userdb that can accommodate extra fields. In particular, supplying a mail field will overwrite the global mail_location. You can iterate through your users, convert them one at a time, then update the mail field as you go. At the end, update mail_location to your new default and remove the extra fields. https://wiki.dovecot.org/UserDatabase/ExtraFields Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: Core Dumps with dovecot-2.2 -git as of 2.2.devel (eaf276b33)
by Aki Tuomi 07 Oct '17

07 Oct '17

Thanks! We'll look into it. ---Aki TuomiDovecot oy -------- Original message --------From: Reuben Farrelly <reuben-dovecot(a)reub.net> Date: 07/10/2017 13:08 (GMT+02:00) To: dovecot(a)dovecot.org Subject: Core Dumps with dovecot-2.2 -git as of 2.2.devel (eaf276b33) This appears to be breaking dsync at the moment and is caused by something committed in the last week or so. This is with the current -git of dovecot-2.2 (eaf276b33). The crash happens all the time - the logs are full of this message. Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Warning: I/O leak: 0x7f8bb6581b4c (ostream-file.c:350, fd 14) Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Panic: file ioloop.c: line 126 (io_remove_full): assertion failed: (io->callback != NULL) Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xd48a6) [0x7f8bb654a8a6] -> /usr/lib64/dovecot/li bdovecot.so.0(+0xd5e26) [0x7f8bb654be26] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8bb654ac39] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf5cb0) [0x7f8bb6 56bcb0] -> /usr/lib64/dovecot/libdovecot.so.0(io_remove+0x1d) [0x7f8bb656be31] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10b5f8) [0x7f8bb65815f8] -> /usr/lib64/dovec ot/libdovecot.so.0(o_stream_cork+0x5a) [0x7f8bb657e289] -> /usr/lib64/dovecot/libssl_iostream_openssl.so(+0x6406) [0x7f8bb12d6406] -> /usr/lib64/dovecot/libssl_iost ream_openssl.so(openssl_iostream_bio_sync+0x18) [0x7f8bb12d6af6] -> /usr/lib64/dovecot/libssl_iostream_openssl.so(+0xa52e) [0x7f8bb12da52e] -> / usr/lib64/dovecot/libssl_iostream_openssl.so(+0xa6ef) [0x7f8bb12da6ef] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_sendv+0xc7) [0x7f8bb657e765] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_send+0x61) [0x7f8bb657e688] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10dc2c) [0x7f8bb6583c2c] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10ddd4) [0x7f8bb6583dd4] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_sendv+0xc7) [0x7f8bb657e765] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_nsendv+0x55) [0x7f8bb657e93f] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_nsend+0x61) [0x7f8bb657e8d3] -> dovecot/doveadm-server() [0x444a35] -> /usr/lib64/dovecot/libdovecot.so.0(i_warning+0xce) [0x7f8bb654afdc] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ca) [0x7f8bb656d8cd] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-server() [0x445dd6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0xd6) [0x7f8bb656d241] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x225) [0x7f8bb656fe58] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3e) [0x7f8bb656d422] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0xbe) [0x7f8bb656d345] Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Fatal: master: service(doveadm): child 27186 killed with signal 6 (core dumped) thunderstorm reuben # file core core: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'dovecot/doveadm-server', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/libexec/dovecot/doveadm-server', platform: 'x86_64' thunderstorm reuben # gdb /usr/libexec/dovecot/doveadm-server core GNU gdb (Gentoo 8.0.1 vanilla) 8.0.1 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://bugs.gentoo.org/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/dovecot/doveadm-server...done. [New LWP 27186] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/doveadm-server'. Program terminated with signal SIGABRT, Aborted. #0 0x00007f8bb60e7f70 in raise () from /lib64/libc.so.6 (gdb) bt full #0 0x00007f8bb60e7f70 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f8bb60e9aa7 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f8bb654a8f5 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201 backtrace = 0x228e4d8 "/usr/lib64/dovecot/libdovecot.so.0(+0xd48a6) [0x7f8bb654a8a6] -> /usr/lib64/dovecot/libdovecot.so.0(+0xd5e26) [0x7f8bb654be26] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8bb654ac39] -> /usr"... #3 0x00007f8bb654be26 in i_internal_fatal_handler (ctx=0x7ffef81daa50, format=0x7f8bb65ba898 "file %s: line %d (%s): assertion failed: (%s)", args=0x7ffef81daa70) at failures.c:670 status = 0 #4 0x00007f8bb654ac39 in i_panic (format=0x7f8bb65ba898 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7ffef81dab50, reg_save_area = 0x7ffef81daa90}} #5 0x00007f8bb656bcb0 in io_remove_full (_io=0x220a100, closed=false) at ioloop.c:126 io = 0x22694a0 __FUNCTION__ = "io_remove_full" #6 0x00007f8bb656be31 in io_remove (io=0x220a100) at ioloop.c:169 No locals. #7 0x00007f8bb65815f8 in o_stream_file_cork (stream=0x220a010, set=true) at ostream-file.c:342 fstream = 0x220a010 ret = 32766 __FUNCTION__ = "o_stream_file_cork" #8 0x00007f8bb657e289 in o_stream_cork (stream=0x220a0a0) at ostream.c:146 _stream = 0x220a010 #9 0x00007f8bb12d6406 in openssl_iostream_bio_output (ssl_io=0x220aa00) at iostream-openssl.c:325 bytes = 140238036407941 max_bytes = 18446742978492891135 sent = 35692560 buffer = "\027\003\003\000\001\000\000\000\020\240 \002\000\000\000\000\200\254\035\370\376\177\000\000\022\343W\266\213\177\000\000`\214\033\002\000\000\000\000\240\240 \002\000\000\000\000`\214\033\002\000\000\000\000\020\240 \002\000\000\000\000\320\314\035\370\376\177\000\000\236f-\261\213\177\000\000\025\071\260fK\247\000\000\000\252 \002\000\000\000\000`\214\033", '\000' <repeats 14 times>, "\315\035\370\376\177\000\000\236f-\261\213\177\000\000\225\305^\001?\236,,\000\252 \002\000\000\000\000=$h\000\272Å«\033\000\000\000\000\000\000\000\000\221\371Í¦f\237\035\224\335-\365\214l\333\347|\005\310\345\035/\200r\242\222}\313I\226G\313*\223\364\312\310\313j\263\260/\245"... bytes_sent = false ret = 32766 __FUNCTION__ = "openssl_iostream_bio_output" #10 0x00007f8bb12d6af6 in openssl_iostream_bio_sync (ssl_io=0x220aa00) at iostream-openssl.c:459 ret = false #11 0x00007f8bb12da52e in o_stream_ssl_flush_buffer (sstream=0x220aca0) at ostream-openssl.c:110 pos = 6 ret = 6 #12 0x00007f8bb12da6ef in o_stream_ssl_sendv (stream=0x220aca0, iov=0x7ffef81dcd70, iov_count=1) at ostream-openssl.c:155 sstream = 0x220aca0 bytes_sent = 6 #13 0x00007f8bb657e765 in o_stream_sendv (stream=0x220ad30, iov=0x7ffef81dcd70, iov_count=1) at ostream.c:260 _stream = 0x220aca0 i = 1 total_size = 6 ret = 35771392 __FUNCTION__ = "o_stream_sendv" #14 0x00007f8bb657e688 in o_stream_send (stream=0x220ad30, data=0x2308850, size=6) at ostream.c:239 iov = {iov_base = 0x2308850, iov_len = 6} #15 0x00007f8bb6583c2c in o_stream_multiplex_sendv (mstream=0x221d270) at ostream-multiplex.c:90 channel = 0x0 ret = 0 #16 0x00007f8bb6583dd4 in o_stream_multiplex_ochannel_sendv (stream=0x221d720, iov=0x7ffef81dcef0, iov_count=1) at ostream-multiplex.c:124 channel = 0x221d720 ret = 7354428521 total = 1 #17 0x00007f8bb657e765 in o_stream_sendv (stream=0x221d7b0, iov=0x7ffef81dcef0, iov_count=1) at ostream.c:260 _stream = 0x221d720 i = 1 total_size = 1 ret = 36233296 __FUNCTION__ = "o_stream_sendv" #18 0x00007f8bb657e93f in o_stream_nsendv (stream=0x221d7b0, iov=0x7ffef81dcef0, iov_count=1) at ostream.c:295 No locals. #19 0x00007f8bb657e8d3 in o_stream_nsend (stream=0x221d7b0, data=0x7ffef81dcf42, size=1) at ostream.c:287 iov = {iov_base = 0x7ffef81dcf42, iov_len = 1} #20 0x0000000000444a35 in doveadm_server_log_handler (ctx=0x7f8bb6805bd0 <failure_ctx_warning>, format=0x7f8bb65bac69 "Raw backtrace for leaks: %s", args=0x7ffef81dcfb0) at client-connection.c:64 c = 3 '\003' corked = false str = 0x228e050 ptr = 0x0 start = 0x228e088 "Raw backtrace for leaks: /usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ae) [0x7f8bb656d8b1] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-ser"... va = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x7ffef81dd090, reg_save_area = 0x7ffef81dcfd0}} _data_stack_cur_id = 4 #21 0x00007f8bb654afdc in i_warning (format=0x7f8bb65bac69 "Raw backtrace for leaks: %s") at failures.c:325 old_errno = 0 args = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7ffef81dd090, reg_save_area = 0x7ffef81dcfd0}} #22 0x00007f8bb656d8cd in io_loop_destroy (_ioloop=0x7ffef81dd138) at ioloop.c:766 backtrace = 0x228dc38 "/usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ae) [0x7f8bb656d8b1] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-server() [0x445dd6] -> /usr/"... ioloop = 0x221ddc0 to_idx = 0x221dfe0 item = 0x0 leaks = true __FUNCTION__ = "io_loop_destroy" #23 0x00000000004454d1 in doveadm_cmd_handle (conn=0x22059d8, cmd_name=0x21b8c20 "dsync-server", argc=3, argv=0x21b8c50, cctx=0x7ffef81dd1b0) at client-connection.c:337 ioloop = 0x0 prev_ioloop = 0x21bfc40 cmd = 0x0 mail_cmd = 0x21c65e8 mctx = 0x221d9f8 cmd_ver2 = 0x0 #24 0x0000000000445793 in client_handle_command (conn=0x22059d8, args=0x21b8c40) at client-connection.c:399 cctx = {cmd = 0x0, argc = 0, argv = 0x0, username = 0x21b8c19 "reuben", cli = false, tcp_server = true, local_ip = {family = 10, u = {ip6 = {__in6_u = { ---Type <return> to continue, or q <return> to quit--- Reuben

1 0

Core Dumps with dovecot-2.2 -git as of 2.2.devel (eaf276b33)
by Reuben Farrelly 07 Oct '17

07 Oct '17

This appears to be breaking dsync at the moment and is caused by something committed in the last week or so. This is with the current -git of dovecot-2.2 (eaf276b33). The crash happens all the time - the logs are full of this message. Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Warning: I/O leak: 0x7f8bb6581b4c (ostream-file.c:350, fd 14) Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Panic: file ioloop.c: line 126 (io_remove_full): assertion failed: (io->callback != NULL) Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xd48a6) [0x7f8bb654a8a6] -> /usr/lib64/dovecot/li bdovecot.so.0(+0xd5e26) [0x7f8bb654be26] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8bb654ac39] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf5cb0) [0x7f8bb6 56bcb0] -> /usr/lib64/dovecot/libdovecot.so.0(io_remove+0x1d) [0x7f8bb656be31] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10b5f8) [0x7f8bb65815f8] -> /usr/lib64/dovec ot/libdovecot.so.0(o_stream_cork+0x5a) [0x7f8bb657e289] -> /usr/lib64/dovecot/libssl_iostream_openssl.so(+0x6406) [0x7f8bb12d6406] -> /usr/lib64/dovecot/libssl_iost ream_openssl.so(openssl_iostream_bio_sync+0x18) [0x7f8bb12d6af6] -> /usr/lib64/dovecot/libssl_iostream_openssl.so(+0xa52e) [0x7f8bb12da52e] -> / usr/lib64/dovecot/libssl_iostream_openssl.so(+0xa6ef) [0x7f8bb12da6ef] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_sendv+0xc7) [0x7f8bb657e765] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_send+0x61) [0x7f8bb657e688] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10dc2c) [0x7f8bb6583c2c] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10ddd4) [0x7f8bb6583dd4] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_sendv+0xc7) [0x7f8bb657e765] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_nsendv+0x55) [0x7f8bb657e93f] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_nsend+0x61) [0x7f8bb657e8d3] -> dovecot/doveadm-server() [0x444a35] -> /usr/lib64/dovecot/libdovecot.so.0(i_warning+0xce) [0x7f8bb654afdc] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ca) [0x7f8bb656d8cd] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-server() [0x445dd6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0xd6) [0x7f8bb656d241] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x225) [0x7f8bb656fe58] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3e) [0x7f8bb656d422] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0xbe) [0x7f8bb656d345] Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Fatal: master: service(doveadm): child 27186 killed with signal 6 (core dumped) thunderstorm reuben # file core core: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'dovecot/doveadm-server', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/libexec/dovecot/doveadm-server', platform: 'x86_64' thunderstorm reuben # gdb /usr/libexec/dovecot/doveadm-server core GNU gdb (Gentoo 8.0.1 vanilla) 8.0.1 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://bugs.gentoo.org/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/dovecot/doveadm-server...done. [New LWP 27186] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/doveadm-server'. Program terminated with signal SIGABRT, Aborted. #0 0x00007f8bb60e7f70 in raise () from /lib64/libc.so.6 (gdb) bt full #0 0x00007f8bb60e7f70 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f8bb60e9aa7 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f8bb654a8f5 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201 backtrace = 0x228e4d8 "/usr/lib64/dovecot/libdovecot.so.0(+0xd48a6) [0x7f8bb654a8a6] -> /usr/lib64/dovecot/libdovecot.so.0(+0xd5e26) [0x7f8bb654be26] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8bb654ac39] -> /usr"... #3 0x00007f8bb654be26 in i_internal_fatal_handler (ctx=0x7ffef81daa50, format=0x7f8bb65ba898 "file %s: line %d (%s): assertion failed: (%s)", args=0x7ffef81daa70) at failures.c:670 status = 0 #4 0x00007f8bb654ac39 in i_panic (format=0x7f8bb65ba898 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7ffef81dab50, reg_save_area = 0x7ffef81daa90}} #5 0x00007f8bb656bcb0 in io_remove_full (_io=0x220a100, closed=false) at ioloop.c:126 io = 0x22694a0 __FUNCTION__ = "io_remove_full" #6 0x00007f8bb656be31 in io_remove (io=0x220a100) at ioloop.c:169 No locals. #7 0x00007f8bb65815f8 in o_stream_file_cork (stream=0x220a010, set=true) at ostream-file.c:342 fstream = 0x220a010 ret = 32766 __FUNCTION__ = "o_stream_file_cork" #8 0x00007f8bb657e289 in o_stream_cork (stream=0x220a0a0) at ostream.c:146 _stream = 0x220a010 #9 0x00007f8bb12d6406 in openssl_iostream_bio_output (ssl_io=0x220aa00) at iostream-openssl.c:325 bytes = 140238036407941 max_bytes = 18446742978492891135 sent = 35692560 buffer = "\027\003\003\000\001\000\000\000\020\240 \002\000\000\000\000\200\254\035\370\376\177\000\000\022\343W\266\213\177\000\000`\214\033\002\000\000\000\000\240\240 \002\000\000\000\000`\214\033\002\000\000\000\000\020\240 \002\000\000\000\000\320\314\035\370\376\177\000\000\236f-\261\213\177\000\000\025\071\260fK\247\000\000\000\252 \002\000\000\000\000`\214\033", '\000' <repeats 14 times>, "\315\035\370\376\177\000\000\236f-\261\213\177\000\000\225\305^\001?\236,,\000\252 \002\000\000\000\000=$h\000\272Å«\033\000\000\000\000\000\000\000\000\221\371Í¦f\237\035\224\335-\365\214l\333\347|\005\310\345\035/\200r\242\222}\313I\226G\313*\223\364\312\310\313j\263\260/\245"... bytes_sent = false ret = 32766 __FUNCTION__ = "openssl_iostream_bio_output" #10 0x00007f8bb12d6af6 in openssl_iostream_bio_sync (ssl_io=0x220aa00) at iostream-openssl.c:459 ret = false #11 0x00007f8bb12da52e in o_stream_ssl_flush_buffer (sstream=0x220aca0) at ostream-openssl.c:110 pos = 6 ret = 6 #12 0x00007f8bb12da6ef in o_stream_ssl_sendv (stream=0x220aca0, iov=0x7ffef81dcd70, iov_count=1) at ostream-openssl.c:155 sstream = 0x220aca0 bytes_sent = 6 #13 0x00007f8bb657e765 in o_stream_sendv (stream=0x220ad30, iov=0x7ffef81dcd70, iov_count=1) at ostream.c:260 _stream = 0x220aca0 i = 1 total_size = 6 ret = 35771392 __FUNCTION__ = "o_stream_sendv" #14 0x00007f8bb657e688 in o_stream_send (stream=0x220ad30, data=0x2308850, size=6) at ostream.c:239 iov = {iov_base = 0x2308850, iov_len = 6} #15 0x00007f8bb6583c2c in o_stream_multiplex_sendv (mstream=0x221d270) at ostream-multiplex.c:90 channel = 0x0 ret = 0 #16 0x00007f8bb6583dd4 in o_stream_multiplex_ochannel_sendv (stream=0x221d720, iov=0x7ffef81dcef0, iov_count=1) at ostream-multiplex.c:124 channel = 0x221d720 ret = 7354428521 total = 1 #17 0x00007f8bb657e765 in o_stream_sendv (stream=0x221d7b0, iov=0x7ffef81dcef0, iov_count=1) at ostream.c:260 _stream = 0x221d720 i = 1 total_size = 1 ret = 36233296 __FUNCTION__ = "o_stream_sendv" #18 0x00007f8bb657e93f in o_stream_nsendv (stream=0x221d7b0, iov=0x7ffef81dcef0, iov_count=1) at ostream.c:295 No locals. #19 0x00007f8bb657e8d3 in o_stream_nsend (stream=0x221d7b0, data=0x7ffef81dcf42, size=1) at ostream.c:287 iov = {iov_base = 0x7ffef81dcf42, iov_len = 1} #20 0x0000000000444a35 in doveadm_server_log_handler (ctx=0x7f8bb6805bd0 <failure_ctx_warning>, format=0x7f8bb65bac69 "Raw backtrace for leaks: %s", args=0x7ffef81dcfb0) at client-connection.c:64 c = 3 '\003' corked = false str = 0x228e050 ptr = 0x0 start = 0x228e088 "Raw backtrace for leaks: /usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ae) [0x7f8bb656d8b1] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-ser"... va = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x7ffef81dd090, reg_save_area = 0x7ffef81dcfd0}} _data_stack_cur_id = 4 #21 0x00007f8bb654afdc in i_warning (format=0x7f8bb65bac69 "Raw backtrace for leaks: %s") at failures.c:325 old_errno = 0 args = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7ffef81dd090, reg_save_area = 0x7ffef81dcfd0}} #22 0x00007f8bb656d8cd in io_loop_destroy (_ioloop=0x7ffef81dd138) at ioloop.c:766 backtrace = 0x228dc38 "/usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ae) [0x7f8bb656d8b1] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-server() [0x445dd6] -> /usr/"... ioloop = 0x221ddc0 to_idx = 0x221dfe0 item = 0x0 leaks = true __FUNCTION__ = "io_loop_destroy" #23 0x00000000004454d1 in doveadm_cmd_handle (conn=0x22059d8, cmd_name=0x21b8c20 "dsync-server", argc=3, argv=0x21b8c50, cctx=0x7ffef81dd1b0) at client-connection.c:337 ioloop = 0x0 prev_ioloop = 0x21bfc40 cmd = 0x0 mail_cmd = 0x21c65e8 mctx = 0x221d9f8 cmd_ver2 = 0x0 #24 0x0000000000445793 in client_handle_command (conn=0x22059d8, args=0x21b8c40) at client-connection.c:399 cctx = {cmd = 0x0, argc = 0, argv = 0x0, username = 0x21b8c19 "reuben", cli = false, tcp_server = true, local_ip = {family = 10, u = {ip6 = {__in6_u = { ---Type <return> to continue, or q <return> to quit--- Reuben

1 0

dovecot: master: Warning: Sent SIGKILL to 100 imap-login processes
by absolutely_free＠libero.it 07 Oct '17

07 Oct '17

Hi, I am using Dovecot 2.2.32 (dfbe293d4) I noticed lots of messages like: dovecot: master: Warning: Sent SIGKILL to 100 imap-login processes in /var/log/maillog I commented out "process_limit" service imap { # Most of the memory goes to mmap()ing files. You may need to increase this # limit if you have huge mailboxes. #vsz_limit = $default_vsz_limit # Max. number of IMAP processes (connections) process_limit = 1024 } but it seems that value doesn't change (still 100). Any suggestions? Thank you very much!

2 2

Re: moving from mysql to pgsql
by Joseph Tam 06 Oct '17

06 Oct '17

mws(a)alpenjodel.de writes: > 1) select digest('testing123','sha512'); > result: > ?\x4120117b3190ba5e24044732b0b09aa9ed50eb1567705abcbfa78431a4e0a96b1152ed7f4925966b1c82325e186a8100e692e6d2fcb6702572765820d25c7e9e > login fails I think 2 problems here: 1) the encoding should be some modified base64; and 2) this appears to compute the SHA512 hash, *not* the SHA512-Crypt hash, which is different. It involves adding a salt and doing many iterations of SHA512. A totally different algorithm. Perusing the PostgreSQL man pages, I think you need something like crypt(password,gen_salt('sha512')) I made the 'sha512' up -- I can't find PostgresSQL docs stating whether it supports this value. The docs I found support the older SHA1 ($5$) crypt hashes. If your version doesn't support creating SHA512 ($6$) salts, you can create your own by replacing gen_salt() with "$6$" + base64(long random value), and feed that to crypt(). Just as long as PostgreSQL uses the system crypt() and not its own implementation, it should produce a usable hash. Joseph Tam <jtam.home(a)gmail.com>

1 0

How to require client SSL certificate, except for local connections
by Evan Martin 06 Oct '17

06 Oct '17

Is there any way to make Dovecot 2.2.22 not require a client SSL certificate for a local IMAP connection, but require it for any remote IMAP connection? My server is configured to require client certificates: ssl = required ... auth_ssl_require_client_cert = yes I tried adding the following to create an exception for localhost: remote 127.0.0.1 { ssl = no auth_ssl_require_client_cert = no disable_plaintext_auth = no } But Dovecot fails to start with: doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 81: Auth settings not supported inside local/remote blocks: Is there any other way to do this? I don't need to override any other auth settings, just that one. I could probably use a Unix socket, if that would help. Thanks, Evan

1 0