September 2016 - dovecot

Capturing dovecot core dump and debugging with gdb: ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
by Ben Johnson 13 Sep '16

13 Sep '16

Hello, I'm experiencing a segfault, so I've followed the steps at http://www.dovecot.org/bugreport.html in an effort to capture the information necessary to submit a bug report. Full "doveconf -n" output is at the end of this message. Dovecot does seem to dump a core file, but when I attempt to parse it, I get this: # apt install dovecot-dbg lib64stdc++6-4.9-dbg # gdb /usr/lib/dovecot/deliver /var/vmail/tmp/core-deliver-6-5000-5000-29125-1473732949 # Generic gdb GNU disclaimers and such here... Reading symbols from /usr/lib/dovecot/deliver...Reading symbols from /usr/lib/debug/.build-id/49/c9c607c304a5c853c05a4ccb9e4b995f185ac4.debug...done. done. warning: core file may not match specified executable file. [New LWP 29125] warning: the debug information found in "/lib64/ld-2.23.so" does not match "/lib64/ld-linux-x86-64.so.2" (CRC mismatch). Core was generated by `/usr/lib/dovecot/deliver -d sa-training(a)example.com -m Training.HAM -p'. Program terminated with signal SIGABRT, Aborted. #0 0x00007f7712e77418 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) The obvious question is why the "core file may not match specified executable file". What am I doing wrong here? Any help would be greatly appreciated. Thanks in advance! --Ben # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-36-generic x86_64 Ubuntu 16.04.1 LTS auth_mechanisms = plain login disable_plaintext_auth = no listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = " quota" mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_backend = pipe antispam_debug_target = syslog antispam_pipe_program = /bin/bash antispam_pipe_program_args = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_notspam_arg = --ham antispam_pipe_program_spam_arg = --spam antispam_pipe_tmpdir = /tmp antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_trash_pattern_ignorecase = trash;Deleted * antispam_verbose_debug = 1 quota = dict:user::file:/var/vmail/%d/%n/.quotausage quota_rule2 = Trash:storage=+100M quota_rule3 = Junk:ignore quota_rule4 = INBOX:storage=+100M quota_warning = storage=100%% quota-reached 100 %u %d quota_warning2 = storage=95%% quota-warning 95 %u %d quota_warning3 = storage=80%% quota-warning 80 %u %d quota_warning4 = -storage=100%% quota-below below %u %d sieve = /var/vmail/%d/%n/.sieve sieve_max_redirects = 25 } postmaster_address = postmaster@example protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service config { unix_listener config { group = vmail mode = 0600 user = vmail } } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-below { executable = script /usr/local/bin/quota-below.sh unix_listener quota-below { group = vmail mode = 0666 user = vmail } user = vmail } service quota-reached { executable = script /usr/local/bin/quota-reached.sh unix_listener quota-reached { group = vmail mode = 0666 user = vmail } user = vmail } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0666 user = vmail } user = vmail } ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key ssl_protocols = !SSLv2 !SSLv3 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { mail_plugins = quota imap_quota antispam } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve quota } protocol lmtp { mail_plugins = quota sieve postmaster_address = webmaster@localhost }

1 0

Re: Use of obox2 and AWS S3
by Raymond Sellars 13 Sep '16

13 Sep '16

Thanks Karol My challenge with AWS EFS is waiting for it to be HIPAA/BAA certified. But good to know it's a satisfactory option. I like the aspect of not worrying about total storage space, in part why I was looking at obox and S3 for the elastic scale. ------------------------------ Message: 5 Date: Sat, 10 Sep 2016 22:27:52 +0100 From: Karol Augustin <karol(a)augustin.pl> To: dovecot(a)dovecot.org Subject: Re: Use of obox2 and AWS S3 Message-ID: <1fbb9ac24386c29a92a320d2d1a399bd(a)augustin.pl> Content-Type: text/plain; charset=US-ASCII On 2016-09-01 2:59, Raymond Sellars wrote: > Any one running a dovecot solution within AWS in general? Interested > in best practice for the storage layer. Although NFS is possible its > not really optimised for elastic cloud type hosting. > > Thanks > Raymond Hi Raymond, I am using dovecot on EC2 instance with EFS storage backend. As the initial performance is bad it scales pretty quickly with space used. So it handles maildir storage very well. The latency is a little high but with indexes there is no problem. It works as NFS storage with unlimited space (8EB) and you pay for what you use. So it turns out to be very cost effective as you do not have to worry about reserving space. For better performance you can keep indexes on ephermal storage or EBS, which both have much lower access times. As for 'elastic cloud type hosting' EFS store can be used by thousands of instances in multiple AWS AZs at the same time, so it looks like it is somehow optimized. Best, Karol -- Karol Augustin karol(a)augustin.pl http://karolaugustin.pl/ +353 85 775 5312

1 0

Sieve filtering SPAM and redirect
by Andre Luiz Paiz 12 Sep '16

12 Sep '16

Hi every body. I'm having difficulties to setup my filters correctly and I really need some help. This is my environment: Mail that is received on the system passes through a sieve_before script that checks the message header for SPAM tag and store it into the "Junk" folder. If no spam tag is found, mail goes to inbox. My horde webmail is integrated with sieve, so this allow users to create their own scripts to store mail in other folders or forward them to external accounts. And this are the problems I'm having: 1 - When users create a forward filter to redirect to external mail, mail that is stored inside the Junk folder are not forwarded; 2 - If I insert a "keep" parameter inside the sieve_before script to keep processing the rules, the mail is forwarded, but becomes duplicated and stored inside the inbox or other folders, if the user has more filters. 3 - If I move my sieve_before script to sieve_after, the same thing happened, one copy is stored on the user selected folder (inbox or filtered), including mail marked as SPAM. My needs are: I need to separate SPAM and HAM e-mail before users intervention (so I need sieve_before). But this SPAM e-mail has to be forwarded if a forward filter is created. Is it possible to use a sieve_after script that moves (instead of copy) the user mail marked as SPAM to the Junk folder even if it was stored in a different folder? Is that possible? Thanks a lot. Here is my config: sieve_after content: require "fileinto"; if header :contains "X-Spam-Flag" "YES" { fileinto "Junk"; } dovecot --version 2.2.10 # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.28.2.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_mechanisms = plain login auth_username_format = %Ln default_client_limit = 1024 default_process_limit = 1024 default_vsz_limit = 1 G mail_location = maildir:~/Maildir:INDEX=/var/indexes/dovecot/%u mail_plugins = " mail_log notify quota" mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir quota_exceeded_message = Cota de armazenamento de e-mails excedida. Quota exceeded for mail store. quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u quota_warning3 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/default.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags } protocols = imap pop3 lmtp sieve service anvil { client_limit = 14100 process_limit = 1 } service auth-worker { client_limit = 1 process_limit = 50 service_count = 1 } service auth { client_limit = 15100 process_limit = 1 } service dict { client_limit = 1 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 6000 process_min_avail = 10 service_count = 1 vsz_limit = 256 M } service imap { process_limit = 3000 vsz_limit = 1 G } service lmtp { client_limit = 1 inet_listener lmtp { port = 24 } unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 6000 process_min_avail = 10 service_count = 1 } service pop3 { process_limit = 3000 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = dovecot } user = root } ssl = required ssl_cert = ************** ssl_key = ************** userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " mail_log notify quota sieve quota" } protocol imap { mail_plugins = " mail_log notify quota imap_quota" } Atenciosamente André Luiz Paiz Administração de Redes Instituto de Química – Unicamp andre.paiz(a)iqm.unicamp.br Telefone: (19)3521-0197

4 8

Use of obox2 and AWS S3
by Raymond Sellars 11 Sep '16

11 Sep '16

Hi Has anyone used or is using dovecot obox and S3 storage? I see very little chatter or documentation on it. Interested in how mature it is and technical items such as high availability and region replication. I'm guessing that multiple updates to a S3 mailbox is mitigated via the standard dovecot pattern of Director pointing multiple user sessions to a single/common dovecot node. And for disaster recovery - that's out of scope of dovecot and more reliant on S3 replication? Any one running a dovecot solution within AWS in general? Interested in best practice for the storage layer. Although NFS is possible its not really optimised for elastic cloud type hosting. Thanks Raymond

3 2

nouvelles incroyables!
by bunkertor 09 Sep '16

09 Sep '16

Hey, Ces nouvelles vont vous surprendre, je vous le dis! Jeter juste un coup d’œil <http://shavopriji.countrysummersblog.com/xowmhs> Je t'embrasse, bunkertor

1 0

sieve - find in header
by Hajo Locke 09 Sep '16

09 Sep '16

Hello list, have a problem with sieve script. I want to perform an action if an arbitrary headerline contains a defined string. I tried with regex: if header :regex [".*"] ["searchstri.*"] { fileinto "junk"; } This is not working because regex is only used in 2nd argument ["searchstri.*"] In 1st argument [".*"] no regex is used, sieve expects just plain string. Something like this would work, if "x-myheader" contains "searchstring": if header :regex ["x-myheader"] ["searchstri.*"].... How to solve this case and find a string in arbitrary/unknown headerline? Thanks, Hajo

4 5

Re: Dovecot 2.2.25 fails on SSL
by Joseph Tam 09 Sep '16

09 Sep '16

Andreas M. Kirchwitz writes: >> You can also affect where shared libraries are loaded using the >> LD_LIBRARY_PATH environment variable. Try adding >> >> LD_LIBARY_PATH=/location/of/libdir; export LD_LIBARY_PATH >> >> to your service boot scripts. > > Thanks for the advice. It's fine for a temporary working around > problems (like this one, so you're absolutely right :-) > > However, no program should require that for regular use because > you never know exactly if somebody in the chain of executed code > removes certain environment variables. And also the opposite way, > if Dovecot runs external programs, those might not play well > with an existing LD_LIBARY_PATH and incompatible SSL libraries. Sure, I understand this, but it's handy in lots of cases where you need to loading from an alternate location. Not everyone has access to resource to recompile. > For every program I compile myself, I link it against my custom > OpenSSL library (always newest version; distributions usually tend > to stick with a specific version and only apply security fixes). OK, the origin of your problem becomes clearer. You can hardcode these paths into the executables by doing something like env CFLAGS='-I/my'ssl/include' \ LDFLAGS='-L/your/ssl/lib -Wl,-rpath,/my/ssl/lib' \ configure ... I use this myself (except the -Wl part since these libs are symlinked to my shared library path). I think "-R/my/ssl/lib" might also be synonymous with -Wl,... > Dovecot is the only package I know of where there are like a thousand > places to put additional libs in the Makefile.am files, but most of > them are totally ignored by configure. I don't have that problem -- I use configure to tell dovecot where to find my self-compiled openssl, and the resulting executables load from where I want. Joseph Tam <jtam.home(a)gmail.com>

1 0

doveadm quota recalc returns a Segmentation fault
by Alessio Cecchi 07 Sep '16

07 Sep '16

Hi, with the latest dovecot-ee version (2.2.25.4 (5cb7c97)) if I run “doveadm quota recalc -u alessio(a)skye.xn--it-02t dovecot return a “Segmentation fault” (but the quota is update fine). Since dovecot version 2.2.24.3 (55cdc32) works fine. This is my quota backends configuration: plugin { quota = maildir:UserQuota quota2 = dict:Quota Usage::noenforcing:proxy::sqlquota } dict { sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } I found that if I comment the line “quota2 = dict:…” doveadm quota recalc works fine. I have two quota backends because with dict I can store everybody's current quota usage in a SQL database for fast retrieving (like quota_clone plugin but I cannot switch to it immediately). The backtrace: # gdb --args doveadm quota recalc -u alessio(a)skye.it GNU gdb (GDB) Red Hat Enterprise Linux (7.2-90.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/bin/doveadm...Reading symbols from /usr/lib/debug/usr/bin/doveadm.debug...done. done. (gdb) run Starting program: /usr/bin/doveadm quota recalc -u alessio(a)skye.it [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Segmentation fault. 0x00007ffff73ed245 in io_loop_handler_run (ioloop=0x1006b1930) at ioloop.c:612 612 io_loop_handler_run_internal(ioloop); Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.5-7.el6_0.x86_64 cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64 dovecot-ee-pigeonhole-2.2.25.4-4.x86_64 glibc-2.12-1.192.el6.x86_64 nspr-4.11.0-1.el6.x86_64 nss-3.21.0-8.el6.x86_64 nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64 nss-util-3.21.0-2.el6.x86_64 openldap-2.4.40-12.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt full #0 0x00007ffff73ed245 in io_loop_handler_run (ioloop=0x1006b1930) at ioloop.c:612 No locals. #1 0x00007ffff73ed3e8 in io_loop_run (ioloop=0x6b1930) at ioloop.c:588 __FUNCTION__ = "io_loop_run" #2 0x00007ffff73ac45c in client_dict_wait (_dict=0x6b15f0) at dict-client.c:653 dict = 0x6b15f0 #3 0x00007ffff636f85f in dict_quota_deinit (_root=<value optimized out>) at quota-dict.c:106 root = 0x6b12f0 __FUNCTION__ = "dict_quota_deinit" #4 0x00007ffff636d326 in quota_root_deinit (root=0x6b12f0) at quota.c:279 pool = 0x6b1400 #5 0x00007ffff636d381 in quota_deinit (_quota=0x6aeae8) at quota.c:389 quota = 0x6ac0b0 roots = <value optimized out> i = <value optimized out> count = 2 #6 0x00007ffff637240d in quota_user_deinit (user=0x6ac7b0) at quota-storage.c:538 quser = 0x6aead8 quota_set = 0x6b07c0 #7 0x00007ffff76a9b52 in mail_user_unref (_user=<value optimized out>) at mail-user.c:174 user = 0x6ac7b0 __FUNCTION__ = "mail_user_unref" #8 0x00000000004216b4 in doveadm_mail_next_user (ctx=0x69bbe0, cctx=<value optimized out>, error_r=0x7fffffffe1c0) at doveadm-mail.c:403 input = {module = 0x0, service = 0x4569d9 "doveadm", username = 0x681288 "alessio(a)skye.it", session_id = 0x0, session_id_prefix = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = { family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0, debug = 0} error = <value optimized out> ip = <value optimized out> ret = <value optimized out> __FUNCTION__ = "doveadm_mail_next_user" #9 0x00000000004218dc in doveadm_mail_cmd_exec (ctx=0x69bbe0, cctx=0x7fffffffe400, wildcard_user=0x0) at doveadm-mail.c:592 ret = <value optimized out> error = <value optimized out> #10 0x0000000000422c7d in doveadm_cmd_ver2_to_mail_cmd_wrapper (cctx=0x7fffffffe400) at doveadm-mail.c:1053 mctx = 0x69bbe0 wildcard_user = <value optimized out> fieldstr = 0x456c52 "-u" i = <value optimized out> ---Type <return> to continue, or q <return> to quit--- mail_cmd = {alloc = 0x7ffff573fba0 <cmd_quota_recalc_alloc>, name = 0x7ffff573ffc5 "quota recalc", usage_args = 0x7ffff573ff7f ""} args_pos = <value optimized out> #11 0x00000000004303b6 in doveadm_cmd_run_ver2 (argc=3, argv=0x6883a0, cctx=0x7fffffffe400) at doveadm-cmd.c:523 param = <value optimized out> pargv = {arr = {buffer = 0x680a28, element_size = 104}, v = 0x680a28, v_modifiable = 0x680a28} pargc = <value optimized out> c = <value optimized out> li = 32767 pool = 0x680858 optbuf = 0x680870 __FUNCTION__ = "doveadm_cmd_run_ver2" #12 0x0000000000430467 in doveadm_cmd_try_run_ver2 (cmd_name=<value optimized out>, argc=3, argv=0x6883a0, cctx=0x7fffffffe400) at doveadm-cmd.c:446 cmd = <value optimized out> #13 0x0000000000432bdc in main (argc=4, argv=0x688398) at doveadm.c:379 cctx = {cmd = 0x69abf8, argc = 4, argv = 0x680a60, username = 0x681288 "alessio(a)skye.it", cli = true, tcp_server = false, local_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, conn = 0x0} cmd_name = <value optimized out> quick_init = false c = <value optimized out> (gdb) I hope can be fixed. Thanks -- Alessio Cecchi Postmaster AT http://www.qboxmail.it http://www.linkedin.com/in/alessice

2 2

Re: Dovecot 2.2.25 fails on SSL
by Joseph Tam 06 Sep '16

06 Sep '16

Aki Tuomi wrote: >>>> ldd /usr/local/Dovecot-2.2.25/lib/dovecot/libdcrypt_openssl.so >>>> linux-gate.so.1 => (0x00dca000) >>>> libcrypto.so.1.0.0 => not found >>>> ... > > Well, then it leaves only option of using /etc/ld.so.conf > so basically add your libssl location there. You can also affect where shared libraries are loaded using the LD_LIBRARY_PATH environment variable. Try adding LD_LIBARY_PATH=/location/of/libdir; export LD_LIBARY_PATH to your service boot scripts. Joseph Tam <jtam.home(a)gmail.com>

3 2

Re: Dovecot 2.2.25 fails on SSL
by Andreas M. Kirchwitz 06 Sep '16

06 Sep '16

Joseph Tam <jtam.home(a)gmail.com> wrote: > You can also affect where shared libraries are loaded using the > LD_LIBRARY_PATH environment variable. Try adding > > LD_LIBARY_PATH=/location/of/libdir; export LD_LIBARY_PATH > > to your service boot scripts. Thanks for the advice. It's fine for a temporary working around problems (like this one, so you're absolutely right :-) However, no program should require that for regular use because you never know exactly if somebody in the chain of executed code removes certain environment variables. And also the opposite way, if Dovecot runs external programs, those might not play well with an existing LD_LIBARY_PATH and incompatible SSL libraries. For every program I compile myself, I link it against my custom OpenSSL library (always newest version; distributions usually tend to stick with a specific version and only apply security fixes). For most packages (eg, Exim, Apache HTTP, BIND, INN, OpenSSH, Mutt, wget etc.) it pretty straightforward. Even if not, usually I can find a quick solution myself. Dovecot is the only package I know of where there are like a thousand places to put additional libs in the Makefile.am files, but most of them are totally ignored by configure. And even if I modify the Makefile files after configure, again most of the additional options get ignored. Never seen it that complicated. As Dovecot is my favorite IMAP daemon, there's no way I'm giving up. :-) I'm totally confident that the developers will find a good solution. And I'm happy to test every patch they come up with. Greetings, Andreas

1 0