- dovecot - dovecot.org

extra fields question
by joao＠7lan.net 21 Sep '20

21 Sep '20

Hi, I'm trying to configure the use of plugins per user in dovecot, more specifically, fts and fts_solr. It happen to have other necessary plugins per protocol, for example "imap_quota" which would be specifically for the imap protocol and which does not apply for the lmtp protocol. For this I need to specify that for the imap protocol I have: protocol imap { mail_plugins = quota imap_quota zlib imap_sieve mail_log notify } and for the lmtp protocol I have: protocol lmtp { mail_plugins = quota zlib fts fts_solr sieve } I did it according to the "extra fields" documentation and separated the name and key with /. My queries are correct but are not being applied to dovecot. Then in the case when the user has fts fts_solr he returns: procotol/imap/mail_plugins -> quota imap_quota zlib imap_sieve mail_log notify fts fts_solr procotol/lmtp/mail_plugins -> quota zlib sieve fts fts_solr but it does not work. Doing it for "mail_plugins" (without accessing config levels) extra fields works, but it also enable some plugins that are not compatible with some protocols. I thought about separating an instance of dovecot lmtp on the same server, but I would like a more direct configuration in the same instance. Does anyone have experience with this? Thanks in advance.

2 1

AW: doveadm search segfault Dovecot 2.2.22
by rudolf＠padaru.de 21 Sep '20

21 Sep '20

Hey, i am now a bit deeper into dovecot debugging with gdb and have a full bt now, with debugging infos. Is there a patch that resolve this Problem in dovecot 2.2.22-1ubuntu2.13 for ubuntu 16? Here the bt: (gdb) bt full #0 0x00007ffff73cf3f3 in array_count_i (array=0x555555810d38) at ../../src/lib/array.h:155 No locals. #1 message_part_finish (ctx=ctx@entry=0x555555810ce0) at message-parser.c:227 __FUNCTION__ = "message_part_finish" #2 0x00007ffff73d132c in message_parser_parse_next_block (ctx=0x555555810ce0, block_r=block_r@entry=0x7fffffffdc40) at message-parser.c:1244 ret = <optimized out> eof = false full = false __FUNCTION__ = "message_parser_parse_next_block" #3 0x00007ffff73d2218 in message_search_msg_real (error_r=0x7fffffffdca0, parts=<optimized out>, input=0x5555558159c0, ctx=0x55555580a870) at message-search.c:217 new_parts = 0x7fffffffdd50 ret = <optimized out> parser_ctx = 0x555555810ce0 raw_block = {part = 0x55555580cc50, hdr = 0x0, data = 0x0, size = 0} #4 message_search_msg (ctx=ctx@entry=0x55555580a870, input=0x5555558159c0, parts=<optimized out>, error_r=error_r@entry=0x7fffffffdca0) at message-search.c:244 _data_stack_cur_id = 6 error = <optimized out> ret = 1 #5 0x00007ffff7705c2f in search_body (arg=0x5555557ec948, ctx=0x7fffffffdd30) at index-search.c:649 msg_search_ctx = 0x55555580a870 error = 0x7fffffffdd40 "0ˀUUU" ret = <optimized out> __FUNCTION__ = "search_body" #6 0x00007ffff7690345 in mail_search_args_foreach (args=0x5555557ec948, args@entry=0x5555557ec8b0, callback=callback@entry=0x7ffff7705ba0 <search_body>, context=context@entry=0x7fffffffdd30) at mail-search.c:440 result = 1 #7 0x00007ffff7705870 in search_arg_match_text (ctx=0x55555580c0a0, args=0x5555557ec8b0) at index-search.c:779 hdr_parser_flags = MESSAGE_HEADER_PARSER_FLAG_CLEAN_ONELINE real_mail = 0x55555580c320 headers = <optimized out> hdr_ctx = {index_ctx = 0x55555580c0a0, imail = 0x55555580c320, args = 0x5555557ec8b0, decoded_block = {part = 0x0, hdr = 0x0, data = 0x0, size = 0}, decoded_block_set = false, hdr = 0x55555580a7e0, parse_headers = 0, custom_header = 0, threading = 0} body_ctx = {index_ctx = 0x55555580c0a0, input = 0x5555558159c0, part = 0x55555580cb30} have_headers = true have_body = true ret = <optimized out> imail = 0x55555580c320 input = 0x5555558159c0 headers_ctx = 0x0 failed = false #8 search_match_once (ctx=0x55555580c0a0) at index-search.c:1329 ctx = 0x55555580c0a0 ret = <optimized out> #9 0x00007ffff77078c1 in search_match_next (ctx=0x55555580c0a0) at index-search.c:1434 _data_stack_cur_id = 5 i = 2 n = 3 ret = <optimized out> #10 search_more_with_mail (mail=0x55555580c320, ctx=0x55555580c0a0) at index-search.c:1547 _data_stack_cur_id = 4 _ctx = 0x55555580c0a0 box = 0x5555557fc530 match = -1 ret = -1 imail = 0x55555580c320 cost1 = 3218534 #11 search_more_with_prefetching (mail_r=0x7fffffffde90, ctx=0x55555580c0a0) at index-search.c:1628 ---Type <return> to continue, or q <return> to quit--- _data_stack_cur_id = 3 count = <optimized out> ret = <optimized out> mail = 0x55555580c320 #12 search_more (ctx=ctx@entry=0x55555580c0a0, mail_r=mail_r@entry=0x7fffffffde90) at index-search.c:1700 imail = <optimized out> ret = <optimized out> #13 0x00007ffff7707fc4 in index_storage_search_next_nonblock (_ctx=0x55555580c0a0, mail_r=0x7fffffffdf58, tryagain_r=0x7fffffffdf07) at index-search.c:1728 mail = 0x5555557e4ce0 seq = 0 ret = <optimized out> #14 0x00007ffff7696ba2 in mailbox_search_next_nonblock (ctx=0x55555580c0a0, mail_r=0x7fffffffdf58, tryagain_r=<optimized out>) at mail-storage.c:1784 box = <optimized out> #15 0x00007ffff7696c0f in mailbox_search_next (ctx=0x55555580c0a0, mail_r=mail_r@entry=0x7fffffffdf58) at mail-storage.c:1769 tryagain = false #16 0x00005555555809d2 in doveadm_mail_iter_next (iter=<optimized out>, mail_r=mail_r@entry=0x7fffffffdf58) at doveadm-mail-iter.c:154 No locals. #17 0x00005555555839dd in cmd_search_box (info=<optimized out>, ctx=0x5555557e9800) at doveadm-mail-search.c:36 iter = 0x5555557fbb60 box = <optimized out> ret = 0 mail = 0x0 metadata = {guid = "\242\n\337\066\322|c_\330\004\000\000\303B(\247", virtual_size = 0, physical_size = 0, first_save_date = 0, cache_fields = 0x0, precache_fields = (unknown: 0), backend_ns_prefix = 0x0, backend_ns_type = (unknown: 0)} guid_str = 0x5555557da3f0 "a20adf36d27c635fd8040000c34228a7" #18 cmd_search_run (ctx=0x5555557e9800, user=<optimized out>) at doveadm-mail-search.c:61 _data_stack_cur_id = 2 iter = 0x5555557f3ed0 info = <optimized out> ret = <optimized out> #19 0x000055555557acef in doveadm_mail_next_user (ctx=ctx@entry=0x5555557e9800, cctx=cctx@entry=0x7fffffffe0f0, error_r=error_r@entry=0x7fffffffe0e8) at doveadm-mail.c:400 input = {module = 0x0, service = 0x5555555afc11 "doveadm", username = 0x5555557d8360 "m056e04b", session_id = 0x0, session_id_prefix = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0} error = 0x12830 <error: Cannot access memory at address 0x12830> ip = <optimized out> ret = <optimized out> __FUNCTION__ = "doveadm_mail_next_user" #20 0x000055555557b78d in doveadm_mail_single_user (ctx=ctx@entry=0x5555557e9800, cctx=cctx@entry=0x7fffffffe0f0, error_r=error_r@entry=0x7fffffffe0e8) at doveadm-mail.c:431 __FUNCTION__ = "doveadm_mail_single_user" #21 0x000055555557b876 in doveadm_mail_cmd_exec (ctx=ctx@entry=0x5555557e9800, wildcard_user=wildcard_user@entry=0x0) at doveadm-mail.c:594 cctx = {cmd = 0x0, argc = 0, argv = 0x0, username = 0x5555557d8360 "m056e04b", cli = false, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0} ret = <optimized out> error = 0x7ffff73dc07d <buffer_alloc+45> "\200K0\001H\211C\020H\211k H\211\003H\203\304\b[]\303\017\037D" #22 0x000055555557c5b6 in doveadm_cmd_ver2_to_mail_cmd_wrapper (cctx=0x7fffffffe320) at doveadm-mail.c:1010 mctx = 0x5555557e9800 wildcard_user = <optimized out> fieldstr = 0x5555557df41b "Hallo" i = <optimized out> mail_cmd = {alloc = 0x555555583890 <cmd_search_alloc>, name = 0x5555555b1e54 "search", usage_args = 0x5555555b1e88 "[-u <user>|-A] [-S <socket_path>] <search query>"} #23 0x000055555558a165 in doveadm_cmd_run_ver2 (argc=argc@entry=7, argv=argv@entry=0x5555557df3a0, cctx=cctx@entry=0x7fffffffe320) at doveadm-cmd.c:519 ---Type <return> to continue, or q <return> to quit---quit Quit (gdb) quit Von: rudolf(a)padaru.de <rudolf(a)padaru.de> Gesendet: Donnerstag, 17. September 2020 17:28 An: 'Dovecot Mailing List' <dovecot(a)dovecot.org> Betreff: doveadm search segfault Dovecot 2.2.22 Good Evening, i have a serius problem by usung the search function via imap or via doveadm. If i run this command: doveadm search -u <user> mailbox INBOX text 'Hello' I receive this line in my maillog: dovecot: imap(<user>): Fatal: master: service(imap): child 7472 killed with signal 11 (core dumped) in my /var/log/apport.log i can see this: ERROR: apport (pid 17785) Thu Sep 17 17:19:01 2020: called for pid 17777, signal 11, core limit 0, dump mode 1 ERROR: apport (pid 17785) Thu Sep 17 17:19:01 2020: executable: /usr/bin/doveadm (command line "doveadm search -u <user> mailbox INBOX text Hello") ERROR: apport (pid 17785) Thu Sep 17 17:19:01 2020: is_closing_session(): no DBUS_SESSION_BUS_ADDRESS in environment ERROR: apport (pid 17785) Thu Sep 17 17:19:01 2020: this executable already crashed 2 times, ignoring A ‘gdb $(cat ExecutablePath) CoreDump‘ from the unpackes dumpe show this: GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1 Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/bin/doveadm...(no debugging symbols found)...done. [New LWP 1441] Core was generated by `doveadm search -u <user> mailbox INBOX text Hello'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007fb4da1f83f3 in ?? () from /usr/lib/dovecot/libdovecot.so.0 A doveconf -n show this output: # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.15.0-117-generic x86_64 Ubuntu 16.04.7 LTS auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@üÜöÖäÄ auth_worker_max_count = 60 disable_plaintext_auth = no imap_capability = +XLIST imap_logout_format = in=%i out=%o del=%{deleted} fetched_mails=%{fetch_hdr_count} lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lock_method = flock login_greeting = <myServer> mailserver ready. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_cache_min_mail_count = 30 mail_location = auto: mail_max_userip_connections = 30 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader imapflags notify vnd.dovecot.pipe vnd.dovecot.filter vnd.dovecot.execute mbox_read_locks = dotlock mbox_very_dirty_syncs = yes mdbox_rotate_size = 250 M namespace inbox { inbox = yes location = mailbox Archiv { auto = no special_use = } mailbox Drafts { auto = no special_use = } mailbox Junk { auto = no special_use = } mailbox Sent { auto = no special_use = } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = no special_use = } prefix = } passdb { args = /etc/dovecot/conf.d/zDovecot-sql.conf.ext driver = sql } plugin { sieve = ~/sieve/.dovecot.sieve sieve_after = ~/sieve_after/.dovecot_after.sieve sieve_before = ~/sieve_before/.dovecot_before.sieve sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/local/bin/ sieve_execute_input_eol = lf sieve_extensions = +notify +imapflags +vnd.dovecot.pipe +vnd.dovecot.filter +vnd.dovecot.execute +editheader -reject sieve_filter_bin_dir = /usr/local/bin/ sieve_filter_input_eol = lf sieve_global_dir = /etc/dovecot/sieve/ sieve_max_redirects = 25 sieve_pipe_bin_dir = /usr/local/bin/ sieve_pipe_input_eol = lf sieve_plugins = sieve_extprograms sieve_redirect_envelope_from = recipient zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 sieve service auth-worker { idle_kill = 1 mins } service auth { idle_kill = 1 mins unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { executable = /usr/lib/dovecot/imap-login process_min_avail = 4 service_count = 0 user = dovecot } service imap { process_limit = 1024 } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 0 user = dovecot } service managesieve { process_limit = 1024 user = dovecot } service pop3-login { executable = /usr/lib/dovecot/pop3-login process_min_avail = 4 service_count = 0 user = dovecot } service pop3 { executable = /usr/lib/dovecot/pop3 process_limit = 1024 } shutdown_clients = no ssl_ca = </etc/apache2/zertifikate/*.<myServer>.pem ssl_cert = </etc/apache2/zertifikate/*.<myServer>.pem ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:!DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_dh_parameters_length = 2048 ssl_key = </etc/apache2/zertifikate/*.<myServer>.pem ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv3 userdb { args = /etc/dovecot/conf.d/zDovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lda { mail_plugins = " sieve zlib" rejection_reason = Your message to <%t> was automatically discarded:%n%r } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = " mail_log notify zlib" ssl = yes } protocol pop3 { mail_plugins = zlib pop3_client_workarounds = oe-ns-eoh outlook-no-nuls pop3_no_flag_updates = no pop3_uidl_format = %08Xu%08Xv ssl = yes } What surprises me is that this problem only occurs with a text search in the inbox, I search in other mailboxes or only in the header this problem does not occur. Can anyone give me a tip on how I can solve this problem without upgrading to 2.33? Mit freundlichen Grüßen Yours sincerely <https://padaru.de/> Pascal Rudolf | Geschäftsführer PADARU-IT | Bielebohstraße 10 | 02736 Beiersdorf @: <mailto:rudolf@padaru.de> rudolf(a)padaru.de | www: <http://padaru.de/> padaru.de mobil: +49 (0) 17 772 595 32

2 1

migrations from 2.2.36
by Maciej Milaszewski 21 Sep '20

21 Sep '20

Hi I started migrations dovecot cluster from 2.2.36.4 (debian) to 2.3.8 (cetos8) .I read https://wiki2.dovecot.org/Upgrading/2.3 In 10-director.conf I must disable director_doveadm_port like: #director_doveadm_port = 2424 and add: ..... service doveadm { inet_listener { port = 2424 } } Is this correct ? the rest of the contents of the file service director { unix_listener login/director { mode = 0666 } fifo_listener login/proxy-notify { mode = 0666 } unix_listener director-userdb { mode = 0600 } inet_listener { port = 9090 } } service imap-login { executable = imap-login director } service pop3-login { executable = pop3-login director } # Enable director for LMTP proxying: protocol lmtp { auth_socket_path = director-userdb } service managesieve-login { executable = managesieve-login director } protocol doveadm { auth_socket_path=director-userdb }

1 0

auth_passdb_request_finished request fires twice on IMAP login
by Eirik Rye 21 Sep '20

21 Sep '20

Hi, Dovecot v2.3.11.3 (502c39af9) auth_passdb_request_finished appears to fire twice for an IMAP login using a Redis passdb. auth_request_finished only fires once. In the debug log below, the line "Finished passdb lookup" appears to be logged twice (once by "auth" and once by "auth-worker"). However, the actual passdb lookup against Redis is only performed once. This is confirmed by monitoring queries made against Redis. Configuration: -------------- metric auth_passdb_request_finished { event_name = auth_passdb_request_finished } auth_debug = yes passdb { auth_verbose = yes override_fields = proxy=y name = redis driver = dict args = /etc/dovecot/dovecot-redis.conf.ext } -------------- With the above configuration, I am seeing the auth_passdb_request_finished metric being incremented twice for every successful login: -------------- / # doveadm stats dump metric_name field count sum min max avg median stddev %95 auth_passdb_request_finished duration 0 0 0 0 0.00 0 0.00 0 / # curl imap://localhost -u foo:bar -XNOOP / # doveadm stats dump metric_name field count sum min max avg median stddev %95 auth_passdb_request_finished duration 2 20327 6240 14087 10163.50 10163 3923.50 14087 -------------- This is the (anonymized) debug output emitted for the above commands: -------------- Sep 19 15:03:32 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=msC166uvOL0K9HPG lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=48440 resp=<hidden> Sep 19 15:03:32 auth: Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Performing passdb lookup Sep 19 15:03:32 auth-worker(76): Debug: conn unix:auth-worker (pid=30,uid=90): auth-worker<3>: Handling PASSV request Sep 19 15:03:32 auth-worker(76): Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Performing passdb lookup Sep 19 15:03:32 auth-worker(76): Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Lookup: shared/users/foo = {"user":"foo","password":"{SHA256-CRYPT}[...]","host":"other_host","uid":1000,"home":"\/mail\/foo"} Sep 19 15:03:32 auth-worker(76): Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Finished passdb lookup Sep 19 15:03:32 auth-worker(76): Debug: conn unix:auth-worker (pid=30,uid=90): auth-worker<3>: Finished Sep 19 15:03:32 auth: Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Finished passdb lookup Sep 19 15:03:32 auth: Debug: auth(foo,127.0.0.1,<msC166uvOL0K9HPG>): Auth request finished Sep 19 15:03:32 auth: Debug: client passdb out: OK 1 user=foo proxy host=other_host uid=1000 home=/mail/foo hostip=127.0.0.2 pass=<hidden> -------------- Best regards, Eirik

2 1

lmtp multiple mailboxes
by Janis 20 Sep '20

20 Sep '20

Hello, What would be the best way to implement "deliver mail to multiple maildirs" through LMTP in Dovecot? Like in Postfix virtual_maps. Thank you! Best wishes, Janis

2 1

auth_passdb_request_finished request fires twice on IMAP login
by Eirik Rye 19 Sep '20

19 Sep '20

Hi, Dovecot v2.3.11.3 (502c39af9) auth_passdb_request_finished appears to fire twice for an IMAP login using a Redis passdb. auth_request_finished only fires once. In the debug log below, the line "Finished passdb lookup" appears to be logged twice (once by "auth" and once by "auth-worker"). However, the actual passdb lookup against Redis is only performed once. This is confirmed by monitoring queries made against Redis. Configuration: -------------- metric auth_passdb_request_finished { event_name = auth_passdb_request_finished } auth_debug = yes passdb { auth_verbose = yes override_fields = proxy=y name = redis driver = dict args = /etc/dovecot/dovecot-redis.conf.ext } -------------- With the above configuration, I am seeing the auth_passdb_request_finished metric being incremented twice for every successful login: -------------- / # doveadm stats dump metric_name field count sum min max avg median stddev %95 auth_passdb_request_finished duration 0 0 0 0 0.00 0 0.00 0 / # curl imap://localhost -u foo:bar -XNOOP / # doveadm stats dump metric_name field count sum min max avg median stddev %95 auth_passdb_request_finished duration 2 20327 6240 14087 10163.50 10163 3923.50 14087 -------------- This is the (anonymized) debug output emitted for the above commands: -------------- Sep 19 15:03:32 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=msC166uvOL0K9HPG lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=48440 resp=<hidden> Sep 19 15:03:32 auth: Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Performing passdb lookup Sep 19 15:03:32 auth-worker(76): Debug: conn unix:auth-worker (pid=30,uid=90): auth-worker<3>: Handling PASSV request Sep 19 15:03:32 auth-worker(76): Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Performing passdb lookup Sep 19 15:03:32 auth-worker(76): Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Lookup: shared/users/foo = {"user":"foo","password":"{SHA256-CRYPT}[...]","host":"other_host","uid":1000,"home":"\/mail\/foo"} Sep 19 15:03:32 auth-worker(76): Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Finished passdb lookup Sep 19 15:03:32 auth-worker(76): Debug: conn unix:auth-worker (pid=30,uid=90): auth-worker<3>: Finished Sep 19 15:03:32 auth: Debug: redis(foo,127.0.0.1,<msC166uvOL0K9HPG>): Finished passdb lookup Sep 19 15:03:32 auth: Debug: auth(foo,127.0.0.1,<msC166uvOL0K9HPG>): Auth request finished Sep 19 15:03:32 auth: Debug: client passdb out: OK 1 user=foo proxy host=other_host uid=1000 home=/mail/foo hostip=127.0.0.2 pass=<hidden> -------------- Best regards, Eirik -- Eirik

1 0

Dovecot core-2.3.11.3 compile error on macOS with src/lib/ioloop-notify-kqueue.c
by Steven Smith 19 Sep '20

19 Sep '20

I’ve hit this compile error on macOS 10.15 with the file src/lib/ioloop-notify-kqueue.c: > error: implicit declaration of function 'i_gettimeofday' is invalid in C99 The fix is to #include "time-util.h" > --- ./src/lib/ioloop-notify-kqueue.c 2020-09-18 19:56:38.000000000 -0400 > +++ ./src/lib/ioloop-notify-kqueue.c 2020-09-18 20:54:36.000000000 -0400 > @@ -11,6 +11,7 @@ > > #include "ioloop-private.h" > #include "llist.h" > +#include "time-util.h" > #include <unistd.h> > #include <fcntl.h> > #include <sys/types.h>

1 0

How to use dovecot only as POP3 server / prevent it from creating .imap directories?
by Binarus 18 Sep '20

18 Sep '20

Dear all, on our mail server, I let sendmail deliver incoming messages to a file in mbox format called "inbox" in each user's home directory. Now I would like to use dovecot as POP3 server so that users can let their MUA download their messages via POP3. I do not want to use any IMAP functionality. The POP3 part works with the configuration shown below, but nevertheless there is a problem which is a show stopper: Although I think I have turned off IMAP (e.g. no IMAP listener), dovecot still creates .imap directories and does so in a very weird way. It creates one .imap subdirectory in every subdirectory of each user's home directory, and for every file in each user's home directory creates an additional subdirectory with the same name in its .imap directories. It does this recursively, and 1000 files in a user's home directory would mean 1000 additional subdirectories. Of course, I can't live with that. I know that the dovecot documentation discourages using the home directory as mail location, but in my case this should not be a problem at all if IMAP is not used. I have played around with dovecot for several days, but could not solve that problem. So how can I turn off IMAP completely and prevent dovecot from creating any .imap directories or IMAP related files? This happens with dovecot 2.3.4.1 (f79e8e7e4) on Debian buster (with all updates applied). My configuration (doveconf -n) is (the configuration is complete - I don't use SQL or LDAP): # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-9-amd64 x86_64 Debian 10.4 # Hostname: host.example.com listen = aaa.bbb.ccc.ddd mail_location = mbox:~:INBOX=~/inbox mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } pop3_lock_session = yes protocols = " pop3" service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 0 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_cert = </etc/dovecot/tls/xxxx.crt ssl_cipher_list = DHE-RSA-AES256-GCM-SHA384 ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { driver = passwd } Thank you very much in advance for any idea! Regards, Binarus

4 8

pam dovecot not working with authentication from roundcube
by Ranbir 18 Sep '20

18 Sep '20

Hi Everyone, I made some small changes in my dovecot setup to switch it from looking up users and passwords from a mix of ldap (i.e. freeipa) and password files. One of the changes was to switch from using one id for all authentication to using individual ids) It's working fine with Evolution. I have one account authenticating with GSSAPI, which is my userid for logging into my desktop and for email. The other account in Evolution is logging in using PLAIN and is only used for email (it's shell is set to /sbin/nologin). The problem is with roundcube: I can login with the second, email only account, but my personal ID always errors out. I never use the domain with either one. auth worker: PASSV: pam_sss(dovecot:auth): authentication failure; logname= uid=97 euid=97 tty=dovecot ruser=ranbir rhost=1.2.3.4 user=ranbir auth worker: PASSV: pam_sss(dovecot:auth): received for user ranbir: 17 (Failure setting user credentials) It doesn't matter what user or group I use for unix_listener. If I use 0777 for the mode, I still see the failure and dovecot goes to try the name against the passwd-file, where it obviously fails. This is the pam error: auth-worker(4474): pam(ranbir,1.2.3.4,<oS10hHmv7qkKyAkP>): pam_authenticate() failed: Authentication failure (password mismatch?) What have I misconfigured? Here's the "service auth" section: service auth { chroot = client_limit = 0 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 inet_listener { address = 9.8.7.5 haproxy = no port = 17900 reuse_port = no ssl = no } privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = $default_internal_user } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0777 user = $default_internal_user } unix_listener login/login { group = mode = 0666 user = } unix_listener token-login/tokenlogin { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } Any guidance is appreciated. -- Ranbir

1 2

doveadm search segfault Dovecot 2.2.22
by rudolf＠padaru.de 17 Sep '20

17 Sep '20

Good Evening, i have a serius problem by usung the search function via imap or via doveadm. If i run this command: doveadm search -u <user> mailbox INBOX text 'Hello' I receive this line in my maillog: dovecot: imap(<user>): Fatal: master: service(imap): child 7472 killed with signal 11 (core dumped) in my /var/log/apport.log i can see this: ERROR: apport (pid 17785) Thu Sep 17 17:19:01 2020: called for pid 17777, signal 11, core limit 0, dump mode 1 ERROR: apport (pid 17785) Thu Sep 17 17:19:01 2020: executable: /usr/bin/doveadm (command line "doveadm search -u <user> mailbox INBOX text Hello") ERROR: apport (pid 17785) Thu Sep 17 17:19:01 2020: is_closing_session(): no DBUS_SESSION_BUS_ADDRESS in environment ERROR: apport (pid 17785) Thu Sep 17 17:19:01 2020: this executable already crashed 2 times, ignoring A gdb $(cat ExecutablePath) CoreDump from the unpackes dumpe show this: GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1 Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/bin/doveadm...(no debugging symbols found)...done. [New LWP 1441] Core was generated by `doveadm search -u <user> mailbox INBOX text Hello'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007fb4da1f83f3 in ?? () from /usr/lib/dovecot/libdovecot.so.0 A doveconf -n show this output: # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.15.0-117-generic x86_64 Ubuntu 16.04.7 LTS auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@üÜöÖäÄ auth_worker_max_count = 60 disable_plaintext_auth = no imap_capability = +XLIST imap_logout_format = in=%i out=%o del=%{deleted} fetched_mails=%{fetch_hdr_count} lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lock_method = flock login_greeting = <myServer> mailserver ready. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_cache_min_mail_count = 30 mail_location = auto: mail_max_userip_connections = 30 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader imapflags notify vnd.dovecot.pipe vnd.dovecot.filter vnd.dovecot.execute mbox_read_locks = dotlock mbox_very_dirty_syncs = yes mdbox_rotate_size = 250 M namespace inbox { inbox = yes location = mailbox Archiv { auto = no special_use = } mailbox Drafts { auto = no special_use = } mailbox Junk { auto = no special_use = } mailbox Sent { auto = no special_use = } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = no special_use = } prefix = } passdb { args = /etc/dovecot/conf.d/zDovecot-sql.conf.ext driver = sql } plugin { sieve = ~/sieve/.dovecot.sieve sieve_after = ~/sieve_after/.dovecot_after.sieve sieve_before = ~/sieve_before/.dovecot_before.sieve sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/local/bin/ sieve_execute_input_eol = lf sieve_extensions = +notify +imapflags +vnd.dovecot.pipe +vnd.dovecot.filter +vnd.dovecot.execute +editheader -reject sieve_filter_bin_dir = /usr/local/bin/ sieve_filter_input_eol = lf sieve_global_dir = /etc/dovecot/sieve/ sieve_max_redirects = 25 sieve_pipe_bin_dir = /usr/local/bin/ sieve_pipe_input_eol = lf sieve_plugins = sieve_extprograms sieve_redirect_envelope_from = recipient zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 sieve service auth-worker { idle_kill = 1 mins } service auth { idle_kill = 1 mins unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { executable = /usr/lib/dovecot/imap-login process_min_avail = 4 service_count = 0 user = dovecot } service imap { process_limit = 1024 } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 0 user = dovecot } service managesieve { process_limit = 1024 user = dovecot } service pop3-login { executable = /usr/lib/dovecot/pop3-login process_min_avail = 4 service_count = 0 user = dovecot } service pop3 { executable = /usr/lib/dovecot/pop3 process_limit = 1024 } shutdown_clients = no ssl_ca = </etc/apache2/zertifikate/*.<myServer>.pem ssl_cert = </etc/apache2/zertifikate/*.<myServer>.pem ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-G CM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AE S128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA25 6:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE- ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES1 28-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE -DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES12 8-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:!DES-CBC3-SHA:!aNU LL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA- DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_dh_parameters_length = 2048 ssl_key = </etc/apache2/zertifikate/*.<myServer>.pem ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv3 userdb { args = /etc/dovecot/conf.d/zDovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lda { mail_plugins = " sieve zlib" rejection_reason = Your message to <%t> was automatically discarded:%n%r } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = " mail_log notify zlib" ssl = yes } protocol pop3 { mail_plugins = zlib pop3_client_workarounds = oe-ns-eoh outlook-no-nuls pop3_no_flag_updates = no pop3_uidl_format = %08Xu%08Xv ssl = yes } What surprises me is that this problem only occurs with a text search in the inbox, I search in other mailboxes or only in the header this problem does not occur. Can anyone give me a tip on how I can solve this problem without upgrading to 2.33? Mit freundlichen Grüßen Yours sincerely <https://padaru.de/> Pascal Rudolf | Geschäftsführer PADARU-IT | Bielebohstraße 10 | 02736 Beiersdorf @: <mailto:rudolf@padaru.de> rudolf(a)padaru.de | www: <http://padaru.de/> padaru.de mobil: +49 (0) 17 772 595 32

1 0

another problem with 2.3.36.4 after update os
by Maciej Milaszewski 17 Sep '20

17 Sep '20

Hi A few days ago I upgraded debian8 to debian9 dovecot is from source # 2.2.36.4 (baf9232c1): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24.2 (aaba65b7) # OS: Linux 4.9.0-13-amd64 x86_64 Debian 9.13 Today I get some times in logs: "dovecot-uidlist line 112: Invalid data" and I dont know why This is claster dovecot: dovecot1 - debian8 dovecot2 - debian8 dovecot3 - debian8 dovecot4 - debian9 dovecot5 - debian8 director - debian8 storage is mont via nfs I upgrade os debian8->debian9 to one node like documentations

1 1

Cannot load key: Invalid dovecot key version
by Mrinal Sharma 16 Sep '20

16 Sep '20

I am trying to use a newly added Local Validation functionality in dovecot version 2.3.11. I am running dovecot inside a Docker container With base image " debian:buster-slim". When I try to login through below command , a crash is seen. Algorithm Used is RS256 and certificate is self-signed. ''' a1 login admin eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1pbiIsIm5hbWUiOiJKb2huIERvZSIsImV4cCI6MTcwMDIyNDg3MiwiaWF0IjoxNTE2MjM5MDIyfQ.gCXOiUDKV5Bjxp_Vkd-I76A86qxxsCCm5YTtvoLhS5CwOV5rBQozcvnkAOkaIEWWMQqFqEIdA6V9RBztl0XlMe5tyil-2NFaXiKsbv6aYmNnBgX678MZlXRi8UFrIK1ymc4ya7Xi11hV5tuTQHAqi6MPEOdB9zTrFBa9CPCKU25G458cd8SMy7t6i_pb-CrBKCA0KKLFF-SJyz4gSu4ce9FFBTx8oYS1lDaQvf6drNw-_N6FfA1k4HVgEK9bU-SEZkeqE0t5nsFA-7V1-BgYvNaR5-tpzOjymeJP7dZ7rJzUqchVz-Z9V3hONGJEuromcEzgve_r9NSBlUTZlcT8Eg '''' dovecot_1 | Sep 16 03:29:36 auth-worker(25): Info: pam(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): pam_authenticate() failed: Authenticat ion failure (Password mismatch?) dovecot_1 | Sep 16 03:29:36 auth-worker(25): Debug: pam(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): Finished passdb lookup dovecot_1 | Sep 16 03:29:36 auth-worker(25): Debug: conn unix:auth-worker (pid=24,uid=101): auth-worker<1>: Finished dovecot_1 | Sep 16 03:29:36 auth: Debug: pam(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): Finished passdb lookup dovecot_1 | Sep 16 03:29:36 auth: Debug: oauth2(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): Performing passdb lookup dovecot_1 | Sep 16 03:29:36 auth: Debug: oauth2(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): Attempting to locally validate token dovecot_1 | Sep 16 03:29:36 auth: Debug: oauth2(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): callback(result: password_mismatch, error: Ca nnot load key: Invalid dovecot key version) dovecot_1 | Sep 16 03:29:36 auth: Info: oauth2(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): oauth2 failed: Cannot load key: Invalid doveco t key version dovecot_1 | Sep 16 03:29:36 auth: Debug: oauth2(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): Finished passdb lookup dovecot_1 | Sep 16 03:29:36 auth: Debug: auth(admin,172.18.0.1,<Q5Bc4GWv9tqsEgAB>): Auth request finished dovecot --version 2.3.11.3 (502c39af9) dovecot -n # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # OS: Linux 4.19.76-linuxkit x86_64 Debian 10.5 fuse.grpcfuse # Hostname: 45e39b46f6ab auth_debug = yes auth_mechanisms = plain oauthbearer xoauth2 auth_verbose = yes disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it log_path = /dev/stdout mail_debug = yes mail_location = maildir:/data/imap_store/%n mail_plugins = " quota" namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Greeting { auto = create } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = no special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-oauth2.plain.conf.ext driver = oauth2 mechanisms = plain login } plugin { quota = maildir:User quota quota_rule = *:storage=5MB quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO } protocols = imap lmtp service doveadm { inet_listener http { port = 80 } } service lmtp { inet_listener lmtp { address = * port = 24 } process_min_avail = 5 } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota postmaster_address = postmaster@domainname } protocol lda { mail_plugins = " quota notify push_notification" } protocol imap { imap_metadata = yes mail_plugins = " quota imap_quota quota" } Any help would be appreciated. Thanks, Mrinal

2 2

dovecot 2.2.36.4 problem with ulimit
by Maciej Milaszewski 16 Sep '20

16 Sep '20

Hi I update os from debian8 to debian9 # 2.2.36.4 (baf9232c1): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24.2 (aaba65b7) # OS: Linux 4.9.0-13-amd64 x86_64 Debian 9.13 All works fine but sometimes I get: Sep 16 09:17:00 dovecot4 dovecot: master: Error: service(pop3): fork() failed: Resource temporarily unavailable (ulimit -u 257577 reached?) Sep 16 09:17:00 dovecot4 dovecot: master: Error: service(imap): fork() failed: Resource temporarily unavailable (ulimit -u 257577 reached?) Sep 16 09:17:00 dovecot4 dovecot: master: Error: service(doveadm): fork() failed: Resource temporarily unavailable (ulimit -u 257577 reached?) Sep 16 09:17:02 dovecot4 dovecot: master: Error: service(doveadm): fork() failed: Resource temporarily unavailable (ulimit -u 257577 reached?) Sep 16 09:17:02 dovecot4 dovecot: master: Error: service(pop3): fork() failed: Resource temporarily unavailable (ulimit -u 257577 reached?) Sep 16 09:17:02 dovecot4 dovecot: master: Error: service(imap): fork() failed: Resource temporarily unavailable (ulimit -u 257577 reached?) Sep 16 09:17:04 dovecot4 dovecot: master: Error: service(imap): fork() failed: Resource temporarily unavailable (ulimit -u 257577 reached?) Other dovecot is debian8 and problem not exists - any idea ?

2 3

Imaptest using the 'DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA'
by Marc Roos 16 Sep '20

16 Sep '20

I just noticed that the imaptest program is using this default first message 'DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA' of the mbox file to test with. I think this should be execluded, as one does not expect this behaviour.

1 0

correct usage for Pigeonhole sieve message *redirect* to external server?
by PGNet Dev 16 Sep '20

16 Sep '20

i've installed rpm -qa | grep dovecot dovecot-2.3.10.1-1.fc32.x86_64 dovecot-pigeonhole-2.3.10.1-1.fc32.x86_64 i'm setting up imapsieve spam reporting; following, https://doc.dovecot.org/configuration_manual/spam_reporting/ https://wiki.dovecot.org/HowTo/AntispamWithSieve https://wiki.dovecot.org/Pigeonhole/Sieve/Examples my config currrently includes protocol imap { mail_plugins = $mail_plugins acl imap_acl quota imap_quota imap_sieve mail_log notify mail_max_userip_connections = 20 imap_client_workarounds = tb-extra-mailbox-sep } protocol sieve { mail_plugins = $mail_plugins mail_max_userip_connections = 10 } plugin { # From elsewhere to Spam folder imapsieve_mailbox1_name = Spam imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_before = file:/etc/dovecot/scripts/report-spam.sieve # From Spam folder to elsewhere imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/etc/dovecot/scripts/report-ham.sieve sieve_plugins = sieve_imapsieve sieve_pipe_bin_dir = /etc/dovecot/scripts } service submission-login { inet_listener submission { address = 10.0.1.70, 127.0.0.1 port = 60465 ssl = yes } } examples @ wiki include report-spam.sieve require "vnd.dovecot.report"; report "abuse" "User added this message to the Spam folder." "spam-report(a)example.com"; instead of a 'report', i want to *redirect* the _unmodified_ Spam message to an _external_ email that's a feed for a bayes spam learner. to that end i modified /etc/dovecot/scripts/report-spam.sieve require ["vnd.dovecot.report", "vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"]; if environment "imap.mailbox" "Trash" { stop; } if environment "imap.mailbox" "Spam" { stop; } redirect "spam.learn(a)othermx.example.net"; with that^, when I drag a spam message into my user account's Spam folder, I see in logs only 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: Mailbox Junk: Mailbox opened because: UID move 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: acl vfile: file /home/vmail/conf.d/example.com/acls/Junk not found 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: acl vfile: file /data/vmail/example.com/myuser/Maildir/Junk/dovecot-acl not found 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: imapsieve: mailbox Junk: MOVE event 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: sieve: Pigeonhole version 0.5.10 (bf8ef1c2) initializing 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.10 (bf8ef1c2) loaded 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam' from=`*' causes=(COPY) => before=`file:/etc/dovecot/scripts/report-spam.sieve' after=(none) 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' from=`Spam' causes=(COPY) => before=`file:/etc/dovecot/scripts/report-ham.sieve' after=(none) ==> /var/log/dovecot/dovecot-info.log <== 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Info: copy from INBOX: box=Junk, uid=4, msgid=<0e9fc21a-c046-46bd-a4ae-e7a1b6424ca7.maildroid@localhost>, size=5647 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Info: expunge: box=INBOX, uid=50, msgid=<0e9fc21a-c046-46bd-a4ae-e7a1b6424ca7.maildroid@localhost>, size=5647 but no redirect occurs; nothing's sent. what additional/different config is required to get the sieve processor to *send* that email? in sieve redirect, _is_ the dovecot submission port implicitly used for sending/redirect? or does the report-spam script need additional capability to exec the send?

1 1

Auth Panic hmac.c while Local Validation
by Mrinal Sharma 16 Sep '20

16 Sep '20

I am trying to use a newly added Local Validation functionality in dovecot version 2.3.11. I am running dovecot inside a Docker container With base image " debian:buster-slim". When I try to login through below command , a crash is seen. ''' a1 login admin eyJhbGci44444zUxMiIsInR5cCI6IkpXVCJ9.eyJ1c2fyX25hbWUiOiJhZG1pbiIsInNjb3BlIjpbIm9wZW5pZCJdLCJleffiOjE3MDAxODAwNTksInN1YiI6ImFkbWluIiwiaWF0IjoxNjAwMTc5NzU5LCJhdXRob3JpdGllcyI6WyJST0xFX0FETUlOIiwiUk9MRV9VU0VSIl0sImp0aSI6ImRyOUV0MVVJWkdJZkZ0emFVZW5VRzRzcmQtQSIsImNsaWVudF9pZCI6IndlYl9hcHAifQ.T9BTZYW52p0VG9gxmTb-cf5GXF5jTOjdkqMaUMAGX-tffffft7YfdPptphGKE8FO9opxcnL--Bjy9ip-XYuWqA '''' Crash: dovecot_1 | Sep 15 16:11:50 auth: Panic: file hmac.c: line 26 (hmac_init): assertion failed: (meth->context_size <= HMAC_MAX_CONTEXT_SIZE) dovecot_1 | Sep 15 16:11:50 auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7ff77b36beed] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7ff77b36c00e] -> /usr/lib/dovecot/libdovecot.so.0(+0xf465b) [0x7ff77b37565b] -> /usr/lib/dovecot/libdovecot.so.0(+0xf46f1) [0x7ff77b3756f1] -> /usr/lib/dovecot/libdovecot.so.0(+0x514a6) [0x7ff77b2d24a6] -> /usr/lib/dovecot/libdovecot.so.0(+0x52415) [0x7ff77b2d3415] -> /usr/lib/dovecot/libdovecot.so.0(+0x59c09) [0x7ff77b2dac09] -> /usr/lib/dovecot/libdovecot.so.0(oauth2_try_parse_jwt+0x7bb) [0x7ff77b2db60b] -> dovecot/auth(+0x3bf71) [0x55576d3cdf71] -> dovecot/auth(db_oauth2_lookup+0x350) [0x55576d3cf030] -> dovecot/auth(auth_request_default_verify_plain_continue+0x2d6) [0x55576d3b3386] -> dovecot/auth(auth_request_verify_plain_callback_finish+0x5c) [0x55576d3b214c] -> dovecot/auth(auth_request_verify_plain_callback+0x51) [0x55576d3b2281] -> dovecot/auth(+0x30171) [0x55576d3c2171] -> dovecot/auth(+0x27ebb) [0x55576d3b9ebb] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7ff77b38b989] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7ff77b38ced1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7ff77b38ba2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7ff77b38bba0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ff77b3059c3] -> dovecot/auth(main+0x3e1) [0x55576d3a6fa1] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7ff77b03809b] -> dovecot/auth(_start+0x2a) [0x55576d3a713a] dovecot_1 | Sep 15 16:11:50 auth-worker(25): Debug: conn unix:auth-worker (pid=24,uid=101): Disconnected: Connection closed (fd=-1) dovecot_1 | Sep 15 16:11:50 imap-login: Debug: Ignoring unknown passdb extra field: temp dovecot --version 2.3.11.3 (502c39af9) dovecot -n # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # OS: Linux 4.19.76-linuxkit x86_64 Debian 10.5 fuse.grpcfuse # Hostname: 45e39b46f6ab auth_debug = yes auth_mechanisms = plain oauthbearer xoauth2 auth_verbose = yes disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it log_path = /dev/stdout mail_debug = yes mail_location = maildir:/data/imap_store/%n mail_plugins = " quota" namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Greeting { auto = create } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = no special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-oauth2.plain.conf.ext driver = oauth2 mechanisms = plain login } plugin { quota = maildir:User quota quota_rule = *:storage=5MB quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO } protocols = imap lmtp service doveadm { inet_listener http { port = 80 } } service lmtp { inet_listener lmtp { address = * port = 24 } process_min_avail = 5 } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota postmaster_address = postmaster@domainname } protocol lda { mail_plugins = " quota notify push_notification" } protocol imap { imap_metadata = yes mail_plugins = " quota imap_quota quota" } Any help would be appreciated. Thanks, Mrinal

2 2

Shared mailboxes setups and dictionaries
by Matej Tyc 16 Sep '20

16 Sep '20

Hello, I am relatively new to the world of MTAs and MDAs, and I try to set up shared mailboxes. So far I have somehow succeeded - I have defined a shared namespace and I have managed to create per-mailbox ACL files thanks to the doveadm command. However, I have been following these resources and there were bits that have puzzled me: When learning about how ACL work in e.g. https://doc.dovecot.org/settings/plugin/acl/ - when one wishes to use the Global ACL file, how does one link it to a particular user's mailboxes? Examples that are listed in the documentation are far too generic. For example does "* user=foo lrw" imply that all mailboxes of all accounts are shared to the user foo? The doveadm command works only if dovecot is set up with per-mailbox ACL files, so I can't use it to reverse-engineer the correct syntax. An interesting aspect to ACLs are dictionaries. I understood it as some kind of cache - if there is no dictionary or it is empty, then shared mailboxes don't work. Conversely, dictionary itself is not enough, one needs actual ACLs set up correctly. Is this a correct understanding? Next what https://wiki.dovecot.org/SharedMailboxes/Shared and https://wiki.dovecot.org/Dictionary describe is a possibility to reference LDAP data to define an ACL dictionary. Do I understand it correctly that if a LDAP database is the single source of truth, then I don't have to worry about updating dictionaries as long as LDAP itself is up-to-date, but I have to keep ACLs and LDAP in sync manually (or using an application)? Thanks, Matej

2 3

Btrfs RAID-10 performance
by Miloslav Hůla 15 Sep '20

15 Sep '20

Hello, I sent this into the Linux Kernel Btrfs mailing list and I got reply: "RAID-1 would be preferable" (https://lore.kernel.org/linux-btrfs/7b364356-7041-7d18-bd77-f60e0e2e2112@le…) May I ask you for the comments as from people around the Dovecot? We are using btrfs RAID-10 (/data, 4.7TB) on a physical Supermicro server with Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz and 125GB of RAM. We run 'btrfs scrub start -B -d /data' every Sunday as a cron task. It takes about 50 minutes to finish. # uname -a Linux imap 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64 GNU/Linux RAID is a composition of 16 harddrives. Harddrives are connected via AVAGO MegaRAID SAS 9361-8i as a RAID-0 devices. All harddrives are SAS 2.5" 15k drives. Server serves as a IMAP with Dovecot 2.2.27-3+deb9u6, 4104 accounts, Mailbox format, LMTP delivery. We run 'rsync' to remote NAS daily. It takes about 6.5 hours to finish, 12'265'387 files last night. Last half year, we encoutered into performace troubles. Server load grows up to 30 in rush hours, due to IO waits. We tried to attach next harddrives (the 838G ones in a list below) and increase a free space by rebalace. I think, it helped a little bit, not not so rapidly. Is this a reasonable setup and use case for btrfs RAID-10? If so, are there some recommendations to achieve better performance? Thank you. With kind regards Milo # megaclisas-status -- Controller information -- -- ID | H/W Model | RAM | Temp | BBU | Firmware c0 | AVAGO MegaRAID SAS 9361-8i | 1024MB | 72C | Good | FW: 24.16.0-0082 -- Array information -- -- ID | Type | Size | Strpsz | Flags | DskCache | Status | OS Path | CacheCade |InProgress c0u0 | RAID-0 | 838G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdq | None |None c0u1 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sda | None |None c0u2 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdb | None |None c0u3 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdc | None |None c0u4 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdd | None |None c0u5 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sde | None |None c0u6 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdf | None |None c0u7 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdg | None |None c0u8 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdh | None |None c0u9 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdi | None |None c0u10 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdj | None |None c0u11 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdk | None |None c0u12 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdl | None |None c0u13 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdm | None |None c0u14 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdn | None |None c0u15 | RAID-0 | 838G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdr | None |None -- Disk information -- -- ID | Type | Drive Model | Size | Status | Speed | Temp | Slot ID | LSI ID c0u0p0 | HDD | SEAGATE ST900MP0006 N003WAG0Q3S3 | 837.8 Gb | Online, Spun Up | 12.0Gb/s | 53C | [8:14] | 32 c0u1p0 | HDD | HGST HUC156060CSS200 A3800XV250TJ | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 38C | [8:0] | 12 c0u2p0 | HDD | HGST HUC156060CSS200 A3800XV3XT4J | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 43C | [8:1] | 11 c0u3p0 | HDD | HGST HUC156060CSS200 ADB05ZG4XLZU | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 46C | [8:2] | 25 c0u4p0 | HDD | HGST HUC156060CSS200 A3800XV3DWRL | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 48C | [8:3] | 14 c0u5p0 | HDD | HGST HUC156060CSS200 A3800XV3XZTL | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 52C | [8:4] | 18 c0u6p0 | HDD | HGST HUC156060CSS200 A3800XV3VSKJ | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 55C | [8:5] | 15 c0u7p0 | HDD | SEAGATE ST600MP0006 N003WAF1LWKE | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 56C | [8:6] | 28 c0u8p0 | HDD | HGST HUC156060CSS200 A3800XV3XTDJ | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 55C | [8:7] | 20 c0u9p0 | HDD | HGST HUC156060CSS200 A3800XV3T8XL | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 57C | [8:8] | 19 c0u10p0 | HDD | HGST HUC156060CSS200 A7030XHL0ZYP | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 61C | [8:9] | 23 c0u11p0 | HDD | HGST HUC156060CSS200 ADB05ZG4VR3P | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 60C | [8:10] | 24 c0u12p0 | HDD | SEAGATE ST600MP0006 N003WAF195KA | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 60C | [8:11] | 29 c0u13p0 | HDD | SEAGATE ST600MP0006 N003WAF1LTZW | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 56C | [8:12] | 26 c0u14p0 | HDD | SEAGATE ST600MP0006 N003WAF1LWH6 | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 55C | [8:13] | 27 c0u15p0 | HDD | SEAGATE ST900MP0006 N003WAG0Q414 | 837.8 Gb | Online, Spun Up | 12.0Gb/s | 47C | [8:15] | 33 # btrfs --version btrfs-progs v4.7.3 # btrfs fi show Label: 'DATA' uuid: 5b285a46-e55d-4191-924f-0884fa06edd8 Total devices 16 FS bytes used 3.49TiB devid 1 size 558.41GiB used 448.66GiB path /dev/sda devid 2 size 558.41GiB used 448.66GiB path /dev/sdb devid 4 size 558.41GiB used 448.66GiB path /dev/sdd devid 5 size 558.41GiB used 448.66GiB path /dev/sde devid 7 size 558.41GiB used 448.66GiB path /dev/sdg devid 8 size 558.41GiB used 448.66GiB path /dev/sdh devid 9 size 558.41GiB used 448.66GiB path /dev/sdf devid 10 size 558.41GiB used 448.66GiB path /dev/sdi devid 11 size 558.41GiB used 448.66GiB path /dev/sdj devid 13 size 558.41GiB used 448.66GiB path /dev/sdk devid 14 size 558.41GiB used 448.66GiB path /dev/sdc devid 15 size 558.41GiB used 448.66GiB path /dev/sdl devid 16 size 558.41GiB used 448.66GiB path /dev/sdm devid 17 size 558.41GiB used 448.66GiB path /dev/sdn devid 18 size 837.84GiB used 448.66GiB path /dev/sdr devid 19 size 837.84GiB used 448.66GiB path /dev/sdq # btrfs fi df /data/ Data, RAID10: total=3.48TiB, used=3.47TiB System, RAID10: total=256.00MiB, used=320.00KiB Metadata, RAID10: total=21.00GiB, used=18.17GiB GlobalReserve, single: total=512.00MiB, used=0.00B I do not attach whole dmesg log. It is almost empty, without errors. Only lines about BTRFS are about relocations, like: BTRFS info (device sda): relocating block group 29435663220736 flags 65 BTRFS info (device sda): found 54460 extents BTRFS info (device sda): found 54459 extents

6 18

[BUG] GSSAPI auth doesn't work on 2.3.11
by Carlos Mogas da Silva 15 Sep '20

15 Sep '20

Hi list! I have a CentOS 7 server running dovecot with GSSAPI auth enabled. Version 2.3.10 was working fine but after the upgrade to 2.3.11 it stopped working. On thing I see on the logs is that the user field comes up empty when compared to the 2.3.10. Example: 2.3.10: Sep 14 13:39:20 mail.r3pek.org dovecot[10189]: imap-login: Login: user=<r3pek(a)r3pek.org>, method=GSSAPI, rip=xxxx, lip=2a01:4f8:1c17:61c0::1, mpid=10195, TLS, session=<PaDSUkWvAOkgAQgY3LZuALKwbzMw3BW/> 2.3.11: Sep 14 12:19:24 mail.r3pek.org dovecot[9264]: imap-login: Disconnected (auth failed, 1 attempts in 6 secs): user=<>, method=GSSAPI, rip=2001:818:dcb6:6e00:b2b0:6f33:30dc:15bf, lip=2a01:4f8:1c17:61c0::1, TLS, session=<pRubNESvMuQgAQgY3LZuALKwbzMw3BW/> Reverting back to 2.3.10 confirmed it's a 2.3.11 thing. Is there any change necessary after the upgrade to 2.3.11 or is this really a bug?

2 2

Is it possible to only replicate indexes and not the mail data
by Marc Roos 14 Sep '20

14 Sep '20

Is it possible to only replicate indexes and not the mail data? (Because the data is already on distributed storage)

2 2

logging LMTP activity
by Janis 11 Sep '20

11 Sep '20

Hello, I've been trying to set up Postfix + Dovecot LMTP setup. Basic idea taken from https://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP, but since Postfix is on different host inet listener is used instead of unix socket. Message delivery works. Messages are delivered to correct mailboxes, if they exist. But the problem is with logging on Dovecot side; i can not get any lmtp logs at all. I kind of expected that i could get logs about: 1) Postfix has connected and delivering mail to mail(a)address.com, 2) If mailbox does not exists, would expect to see in logs the reason for rejection - mailbox does not exist 3) any other useful info would be nice as well. How do you get LMTP to log something? My Dovecot version is 2.3.11.3. Basic IMAP logging is working. I see when users have connected, etc. I tried as suggested at https://doc.dovecot.org/configuration_manual/protocols/lmtp_server/#logging If the given logging example block is used in configuration, i get no logs in file /var/log/dovecot-lmtp.log. I thought maybe file is not created automatically, so i tried to create it myself as well. Does not work. chmod 0777 does not either, chown to dovecot:dovecot neither. In a way i am curious about "executable = lmtp -L" part. What is -L flag? Are there any more flags? Where could i learn about them? I have read all the docs that come with the buld and mention something of lmtp, it's basically wiki, i guess... but i could not find any helpful info. I found that file lmtp is at /usr/local/libexec/dovecot/lmtp. When i execute it with -L flag it works. Other flags don't. The problem is that there are no -h or --help flags. Then i tried raw logging as described here: https://doc.dovecot.org/admin_manual/debugging/debugging_rawlog/#debugging-… As i understand, only 4th possibility is applicable to LMTP, am i right? 2nd kind of seems maybe somehow could be usable, but i don't see how... It is not clearly stated in wiki where "lmtp_rawlog_dir" should be added, but i add it within "protocol lmtp { .. }" block, similar as imap. Does not work for me - no logs while messages arrive. The catchy part in wiki is that for imap special arguments are used "-R rawlog" in service imap-login, postlogin. But i don't see any lmtp-login or such service available. So should it work only with "lmtp_rawlog_dir" option? What could i be doing wrong? Does anyone have working lmtp configuration with logging available? Thank you very much! I am sorry that my message is so long, but i tried even more to get it working... :) Any help or idea of how to better debug the situation or search for solution is kindly appreciated.

2 2

Re: Doveadm error since 2.3.11.3 when run as unprivileged user
by Olav Seyfarth 11 Sep '20

11 Sep '20

On Thu, Aug 13, 2020, Arjen de Korte wrote: > include optional config file and suppress errors if it doesn't exist I followed this workaround, which is nicely summarized at https://github.com/postfixadmin/postfixadmin/issues/381#issuecomment-678269… Yet, not all errors are suppressed this way: $ doveadm pw doveadm(olav): Error: net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied $ ls -l /run/dovecot/stats-writer srw-rw---- 1 root dovecot 0 11. Sep 00:27 stats-writer Olav

2 2

Re: Btrfs RAID-10 performance
by Robert Nowotny 11 Sep '20

11 Sep '20

"Miloslav" == Miloslav Hůla <miloslav.hula(a)gmail.com> <miloslav.hula(a)gmail.com> writes: Miloslav> Dne 09.09.2020 v 17:52 John Stoffel napsal(a): Miloslav> There is a one PCIe RAID controller in a chasis. AVAGO Miloslav> MegaRAID SAS 9361-8i. And 16x SAS 15k drives conneced to Miloslav> it. Because the controller does not support pass-through for Miloslav> the drives, we use 16x RAID-0 on controller. So, we get Miloslav> /dev/sda ... /dev/sdp (roughly) in OS. And over that we have Miloslav> single btrfs RAID-10, composed of 16 devices, mounted as Miloslav> /data. I will bet that this is one of your bottlenecks as well. Get a secord or third controller and split your disks across them evenly. Miloslav> That's plan for a next step. Miloslav> We run 'rsync' to remote NAS daily. It takes about 6.5 hours to finish, Miloslav> 12'265'387 files last night. That's.... sucky. So basically you're hitting the drives hard with random IOPs and you're probably running out of performance. How much space are you using on the filesystem? Miloslav> It's not so sucky how it seems. rsync runs during the Miloslav> night. And even reading is high, server load stays low. We Miloslav> have problems with writes. Ok. So putting in an SSD pair to cache things should help. And why not use brtfs send to ship off snapshots instead of using rsync? I'm sure that would be an improvement... Miloslav> We run backup to external NAS (NetApp) for a disaster Miloslav> recovery scenario. Moreover NAS is spreaded across multiple Miloslav> locations. Then we create NAS snapshot, tens days Miloslav> backward. All snapshots easily available via NFS mount. And Miloslav> NAS capacity is cheaper. So why not run the backend storage on the Netapp, and just keep the indexes and such local to the system? I've run Netapps for many years and they work really well. And then you'd get automatic backups using schedule snapshots. Keep the index files local on disk/SSDs and put the maildirs out to NFSv3 volume(s) on the Netapp(s). Should do wonders. And you'll stop needing to do rsync at night. Miloslav> It's the option we have in minds. As you wrote, NetApp is very solid. Miloslav> The main reason for local storage is, that IMAP server is completely Miloslav> isolated from network. But maybe one day will use it. It's not completely isolated, it can rsync data to another host that has access to the Netapp. **grin** Miloslav> Unfortunately, to quickly fix the problem and make server Miloslav> usable again, we already added SSD and moved indexes on Miloslav> it. So we have no measurements in old state. That's ok, if it's better, then its better. How is the load now? Looking at the output of 'iostat -x 30' might be a good thing. Miloslav> Situation is better, but I guess, problem still exists. I Miloslav> takes some time to load be growing. We will see. Hmm... how did you setup the new indexes volume? Did you just use btrfs again? Did you mirror your SSDs as well? Do the indexes fill the SSD, or is there 20-30% free space? When an SSD gets fragmented, it's performance can drop quite a bit. Did you put the SSDs onto a seperate controller? Probably not. So now you've just increased the load on the single controller, when you really should be spreading it out more to improve things. Another possible hack would be to move some stuff to a RAM disk, assuming your server is on a UPS/Generator incase of power loss. But that's an unsafe hack. Also, do you have quotas turned on? That's a performance hit for sure. Miloslav> Thank you for the fio tip. Definetly I'll try that. It's a good way to test and measure how the system will react. Unfortunately, you will need to do your testing outside of normal work hours so as to not impact your users too much. Good luck! Please post some numbers if you get them. If you see only a few disks are 75% or more busy, then **maybe** you have a bad disk in the system, and moving off that disk or replacing it might help. Again, hard to know. Rebalancing btrfs might also help, especially now that you've moved the indexes off that volume. Robert: Fio is an acronym for *Flexible IO Tester* and describes a tool for measuring IO performance. With Fio, devices such as hard drives or SSDs can be tested for their speed by executing a *user-defined **workload* and collecting performance data. Therefore it might be difficult to really simulate the load with that, because You need to define the workload Yourself. But, at least, You might use it to get an idea of maximum transfer rates, random I/O etc. *iotop* shows the *current I/O transfer **rates* for the currently running processes / threads. It uses the I/O usage information of the Linux kernel, so it might be a good tool for You. *htop *might also be Your friend. and of course, as someone else pointed out : > Looking at the output of 'iostat -x 30' might be a good thing. my default setup for such diagnosis is to run iotop, htop, ntop to get an idea whats going on and measure ... for the index-SSD (on a different Channel ? NvMe ?) I strongly suggest to compare EXT4 vs BTRFS - some Benchmarks suggest that on fast SSD´s EXT4 is much (factor 4-5) faster then btrfs. You might also try ZSTD transparent compression on the remaining (non-SSD) btrfs disks - this can boost the transfer rate (since the data is compressed) if the Server CPU´s have enough juice. But from my guts feeling, its not the transfer rate - its the random I/O (in that case the situation will get worse with compression), but You really need to measure that. In general I dont understand that (old fashioned) configuration - using a bunch of 2TB or even 4TB TLC SSD in Raid10 is nowadays really affordable. You can keep the old disks for (local) rsync before You rsync from there to the NAS ... Robert

1 0

Bug in sieve script compilation
by Иннокентий Сенновский 10 Sep '20

10 Sep '20

Hi, this is Rumata888 from hackerone. This is the other bug I found in sievec. If you try to compile the following script, it will result in a panic. The reason is: export expects its arguments to be either strings or lists of strings. When we issue an export command with a number immediately followed by an export command with a string argument, on parsing the second command sievec tries to merge the commands by merging the arguments. When it encounters an argument of unexpected type, it panics. Once again, I would advise unlinking invalid commands from ast, so that parsing of following commands was not dependent on previous irregularities.

1 0

dbox alternate storage and archived namespace
by Daniel Miller 10 Sep '20

10 Sep '20

This may (and probably does) come under the heading of "really dumb ideas", but... Before I develop this further I need to ask - is it possible to have a "primary" mail_location using single-dbox with an alternate storage using multi-dbox? This is *not* the same as different storages for different namespaces (already have that). -- Daniel

1 0

Re: zlib errors after upgrading to 2.3.11.3
by Robert Nowotny 09 Sep '20

09 Sep '20

On 8. Sep 2020, at 12.35, Robert Nowotny <rnowotny1966(a)gmail.com> wrote: Dear Aki, I switched to "gz" now, since "zstd" also gave some errors on writing to files. What kind of errors? Probably this? : Panic: file ostream.c: line 287 (o_stream_sendv_int): assertion failed: (!stream->blocking) can be - unfortunately I do not have the log files anymore... Sep 3 08:33:25 lxc-imap dovecot: imap(mpaul)<48684><2/9E5mKuAezAqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Also this way you can see if the broken mail is actually xz or zstd or zlib. It would be nice to know if there are any zstd or zlib compressed mails that have problems. We did a lot of stress testing with zstd and also with xz, but haven't been able to reproduce any problems. It's also strange that it says taht the error is "Broken pipe" - that doesn't indicate that the mail is corrupted but that there is something more strange going on. So perhaps you don't actually have any mails written as corrupted, but Dovecot is just somehow having trouble reading the mails. I managed to reproduce this. The files aren't corrupted, it's just that reading is failing. The attached patch should fix the xz code and should make your files readable again. Yes, it works ! I switched now to "zstd" compression and currently re-compressing all user imap folders (one by one) thank You so much ! Robert

2 1

Leaked Events
by bobby 08 Sep '20

08 Sep '20

I am going through my maillog file, and noticed these events: Sep 8 15:56:17 mail dovecot[294462]: lmtp(391257): Warning: Event 0x55db5bc41200 leaked (parent=0x55db5bc3b830): mail-storage-service.c:1325 Sep 8 15:56:17 mail dovecot[294462]: lmtp(391257): Warning: Event 0x55db5bc3b830 leaked (parent=0x55db5bc34450): smtp-server-recipient.c:38 Sep 8 15:56:17 mail dovecot[294462]: lmtp(391257): Warning: Event 0x55db5bc34450 leaked (parent=0x55db5bc34130): connection.c:523 Sep 8 15:56:17 mail dovecot[294462]: lmtp(391257): Warning: Event 0x55db5bc34130 leaked (parent=0x55db5bc20ce0): smtp-server-connection.c:773 Sep 8 15:56:17 mail dovecot[294462]: lmtp(391257): Warning: Event 0x55db5bc20ce0 leaked (parent=(nil)): lmtp-client.c:164 Is this something to be concerned about?

2 1

zlib errors after upgrading to 2.3.11.3
by Robert Nowotny 08 Sep '20

08 Sep '20

Dear Aki, I switched to "gz" now, since "zstd" also gave some errors on writing to files. I dont know if "xz" compression or "zstd" shreddered my MDBOX Files, but I lost 4 days of mail. (a couple of thousand mails). After restoring the backup (what was made after switching to version 2.3.11.3) I still have some broken mdfiles, but not too many. Interestingly always in the "Sent" Mailbox on a number of Users. I just can not go back to 20.08.2020 before I updated to 2.3.11.3 - too many emails would have been lost. So - the current status is 2.3.11.3, with "gz" compression. I force-synced and re-indexed all the mdbox files (250 GB), but still have some broken - please how can I fix those ? You stated : (In theory you could leave the existing mails xz-compressed, but best would be to re-compress everything via dsync so old mails can be read when we eventually remove xz support.) what is the optimal way to do that, expecially not loosing the index (?) for outlook/thunderbird, for third party tools that rely on some index/hash (dunno how exactly that works) ? Sep 3 08:33:25 lxc-imap dovecot: imap(mpaul)<48684><2/9E5mKuAezAqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<48089><YRio1GKuEdHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<49228><p/0g9WKuVdHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<49229><GMAm9WKuVtHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:27 lxc-imap dovecot: imap(mpaul)<49230><YYot9WKuV9HAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) ... Sep 3 09:04:44 lxc-imap dovecot: imap(cpotzinger)<49040></X+P72KuUNHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:04:51 lxc-imap dovecot: imap(cpotzinger)<54382><eYT1XWOux9HAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:04:59 lxc-imap dovecot: imap(cpotzinger)<54396><nKllXmOuyNHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:05:06 lxc-imap dovecot: imap(cpotzinger)<54409><m2TOXmOuydHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:05:12 lxc-imap dovecot: imap(cpotzinger)<54422><IP86X2OuytHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) ... Sep 3 09:08:58 lxc-imap dovecot: imap(cpotzinger)<54867><BtGtbGOu49HAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:19:15 lxc-imap dovecot: imap(mpaul)<56030></kHNkWOuAO3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:41 lxc-imap dovecot: imap(cpotzinger)<57274><c4rCtWOuKuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:47 lxc-imap dovecot: imap(cpotzinger)<57320><K3Uqt2OuLOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:54 lxc-imap dovecot: imap(cpotzinger)<57333><e4aMt2OuLezAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:01 lxc-imap dovecot: imap(cpotzinger)<57346><JEfzt2OuLuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:08 lxc-imap dovecot: imap(cpotzinger)<57360><mH1YuGOuMOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:14 lxc-imap dovecot: imap(cpotzinger)<57382><IzLBuGOuMezAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:21 lxc-imap dovecot: imap(cpotzinger)<57397><q10kuWOuM+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:27 lxc-imap dovecot: imap(cpotzinger)<57410><CSuJuWOuNOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:22 lxc-imap dovecot: imap(mpaul)<57525><TfoOvWOuWO3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:49 lxc-imap dovecot: imap(cpotzinger)<57423><Be/ruWOuNuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:55 lxc-imap dovecot: imap(cpotzinger)<57584><yvXOvmOuQuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:02 lxc-imap dovecot: imap(cpotzinger)<57597><e/8tv2OuQ+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:08 lxc-imap dovecot: imap(cpotzinger)<57611><BLSOv2OuROzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:19 lxc-imap dovecot: imap(cpotzinger)<57633><Ky0twGOuR+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:26 lxc-imap dovecot: imap(cpotzinger)<57646><ljSRwGOuS+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:33 lxc-imap dovecot: imap(cpotzinger)<57659><rn75wGOuT+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:39 lxc-imap dovecot: imap(cpotzinger)<57672><GP9jwWOuUuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:33:45 lxc-imap dovecot: imap(mpaul)<57795><2CEwxWOubu3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:29 lxc-imap dovecot: imap(mpaul)<58347><CKyX1mOuie3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:32 lxc-imap dovecot: imap(mpaul)<58352><osHU1mOuiu3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:44 lxc-imap dovecot: imap(cpotzinger)<54527><5ZpiYmOuBOfAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:51 lxc-imap dovecot: imap(cpotzinger)<58378><kAuN12OumuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:57 lxc-imap dovecot: imap(cpotzinger)<58392><xJLx12Oum+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) ... Am 26.08.2020 um 09:42 schrieb Robert Nowotny: I had 4 broken mdbox files for 4 different users. After deleting them, dovecot re-indexed, now everything is ok again. I can try to update tonight again to 2.3.11.3 and see if some broken files will appear again tomorrow. Am 25.08.2020 um 14:51 schrieb Aki Tuomi: On 25/08/2020 14:35 Robert Nowotny <rnowotny(a)rotek.at> <rnowotny(a)rotek.at> wrote: I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3 Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63870><jZk...>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[]).... many of those entries.... after installing2.3.10.1 those files are still broken - but new files are working again. anything I can tweak or do ? Hi! Is it possible your mdbox file is corrupted? Can you share doveconf -n? Aki

2 3

Re: Re: Outlook problems on Apple systems(Mac os)
by hfh＠cndns.com 08 Sep '20

08 Sep '20

Outlook version is 16.40 hfh(a)cndns.com From: Jerry Date: 2020-09-08 13:22 To: Dovecot Mailing List Subject: Re: Outlook problems on Apple systems(Mac os) On Tue, 8 Sep 2020 09:57:59 +0800, hfh(a)cndns.com <hfh(a)cndns.com> stated: >Dovecot is on version 2.2.36. I tested it. Outlook on the Apple system >successfully logged in with the imap protocol, and then sent the LIST >command and outlook did not send any further. Dovecot returns the >results as follows: > >A003 OK LIST completed (0.001 + 0.000 secs). > > >There is no problem with version 2.1.17 using dovecot before. The >information returned by dovecot in version 2.1.17 is as follows: > >A003 OK LIST completed. > > >I suspect that the feedback of the following information causes >Outlook to not recognize it normally: > >(0.001 + 0.000 secs) What version of Outlook? -- Jerry

2 1

Outlook problems on Apple systems(Mac os)
by hfh＠cndns.com 08 Sep '20

08 Sep '20

Dovecot is on version 2.2.36. I tested it. Outlook on the Apple system successfully logged in with the imap protocol, and then sent the LIST command and outlook did not send any further. Dovecot returns the results as follows: A003 OK LIST completed (0.001 + 0.000 secs). There is no problem with version 2.1.17 using dovecot before. The information returned by dovecot in version 2.1.17 is as follows: A003 OK LIST completed. I suspect that the feedback of the following information causes Outlook to not recognize it normally: (0.001 + 0.000 secs) hfh(a)cndns.com

2 1

Search in Mailbox via doveadm REST-API
by rudolf＠padaru.de 07 Sep '20

07 Sep '20

Hey, i need some help with the http-api from doveadm, i try to search a string in the mail-text from all mails in my inbox. My curl-post looks like these: curl -v -X POST -u <user>:<password> -H "Content-Type: application/json" -d '[["search",{"query":["mailbox","INBOX","TEXT","8887"],"user":"<mailUser>"}, "tag1"]]' http://<server>:8080/doveadm/v1 | python -mjson.tool > out.json I receive only empty results, do have anyone an idea? Mit freundlichen Grüßen Yours sincerely <https://padaru.de/> Pascal Rudolf | Geschäftsführer PADARU-IT | Bielebohstraße 10 | 02736 Beiersdorf @: rudolf(a)padaru.de | www: <http://padaru.de/> padaru.de mobil: +49 (0) 17 772 595 32

1 0

LMTP-Process stays in RCPT TO state
by Pascal Christen 07 Sep '20

07 Sep '20

Hi We tried to update our Dovecot Director-Ring to 2.3 (latest version) and had one problem/bug. When we try to deliver an email from Exim to a receiving Dovecot (2.3 latest) via the LMTP-Proxy the LMTP process stays open forever and never closes. So after a few hours, we hit the LMPT-Process limit. The problem only exists if the receiving Dovecot reports "Quota exceeded (mailbox for user is full)" back to the Exim. If this is the case the state on the LMPT process stays at "lmtp: [_IP_ RCPT TO] (lmtp) and never closes. The TCP-Connection to the Exim is then in CLOSE_WAIT state on the Proxy. On the receiving Dovecot we set the "lmtp_rcpt_check_quota=yes" option. For me, it sounds similar to this bug: https://dovecot.org/list/dovecot/2018-January/110697.html but without the SEGFAULT which makes it harder to debug it. May you guys know why it doesn't change the state to IDLE or READY after reporting the quota issue or see something fishy in the commit above. If not I try to get a trace of the hanging process. Greetings Pascal

2 6

Backup questions - errors
by Mark Patruck 07 Sep '20

07 Sep '20

Hi, although i've search the archives, there are questions left regarding backup strategies and errors that occur. I'm using dovecot 2.3.11.3 with mdbox (mailbox size 50MB - 30GB). Every night, i ran the following command: doveadm -o mail_fsync=never -o plugin/quota= -o plugin/zlib_save=gz \ backup -u account mdbox:/nfs/storage/account/mdbox ### incremental backup Thanks to zlib compression, i save a lot of storage space, nonetheless i'd go with an incremental backup, but it seems to only merge changes, comparing to rsyncs' --link-dest option where you can put all "changes of the day" in an extra folder. I'd like to prevent loosing mail that gets deleted by accident/on purpose. How can i achieve this with dsync and w/o doing a full backup every day? ### backup errors After a few days w/o any errors, doveadm prints the following lines (only for the biggest, 30GB mailbox) two days in a row. As dovecot has self- healing capabilities, i think these errors/warnings can be ignored and should go away...? Error: Transaction log ....mdbox/dovecot.list.index.log: duplicate transaction log sequence (38) Error: Index ....mdbox/dovecot.list.index: Lost log for seq=38 offset=63312: Missing middle file seq=38 (between 38..4294967295, we have seqs 37,39): .log.2 contains file_seq=37 (initial_mapped=1, reason=Index mapped) Warning: fscking index file ....mdbox/dovecot.list.index Error: Fixed index file ....mdbox/dovecot.list.index: log_file_seq 38 -> 39 Error: Mailbox list index was marked as fsck'd ....mdbox/dovecot.list.index Error: Mailbox list index was marked as fsck'd ....mdbox/dovecot.list.index Error: Mailbox list index was marked as fsck'd ....mdbox/dovecot.list.index Thanks, -Mark -- Mark Patruck ( mark at wrapped.cx ) GPG key 0xF2865E51 / 187F F6D3 EE04 1DCE 1C74 F644 0D3C F66F F286 5E51 https://www.wrapped.cx

1 1

sieve_max_script_size is ignored
by Zdeněk Zámečník 06 Sep '20

06 Sep '20

I am in troubles with compiling sieve scripts larger than 1MB. I see in logs following errors: Aug 19 13:10:26 mail dovecot: lmtp(z.z(a)xxx.xxx)<22117><uNBGHKIIPV9lVgAA5ldI4A>: Error: sieve: autoreply: line 16818: quoted string started at line 3 is too long (longer than 1048576 bytes) Aug 19 13:10:26 mail dovecot: lmtp(z.z(a)xxx.xxx)<22117><uNBGHKIIPV9lVgAA5ldI4A>: Error: sieve: autoreply: parse failed Aug 19 13:10:26 mail dovecot: lmtp(z.z(a)xxx.xxx)<22117><uNBGHKIIPV9lVgAA5ldI4A>: Error: sieve: Failed to compile script `/var/vmail/xxx.xxx/z.z/sieve/autoreply.sieve' I tried to set sieve_max_script_size = 2M or 0 (unlimited). I also tried to run "sievec -o sieve_max_script_size=2M /var/vmail/xxx.xxx/z.z/sieve/autoreply.sieve" but the result is still same. Below is my config. Can you please advice where can be the problem? # 2.3.10.1 (a3d0e1171): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.10 (67bf5bd7) # OS: Linux 5.4.44-2-pve x86_64 Debian 10.4 # Hostname: mail.xxx.xxx auth_cache_size = 2 M auth_cache_ttl = 5 mins auth_master_user_separator = * auth_mechanisms = plain login auth_policy_check_before_auth = no auth_policy_hash_nonce = # hidden, use -P to show it auth_policy_report_after_auth = no auth_policy_server_timeout_msecs = 1500 auth_policy_server_url = http://127.0.0.1:8090/ dict { acl = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imap_client_workarounds = delay-newmail imap_id_log = * imap_id_retain = yes login_trusted_networks = 127.0.0.0/8 mail_gid = 2000 mail_home = /var/vmail/%d/%n mail_location = mdbox:~/mdbox:ALT=/var/vmail-archive/%d/%n/mdbox mail_max_userip_connections = 60 mail_plugins = acl zlib fts quota mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader imapsieve vnd.dovecot.imapsieve mdbox_rotate_interval = 1 days mdbox_rotate_size = 16 M namespace { inbox = yes location = mailbox "Deleted Items" { auto = no special_use = \Trash } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox "Junk E-mail" { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { special_use = \Trash } mailbox virtual/All { auto = no special_use = \All } prefix = separator = / type = private } namespace { inbox = no list = children location = mdbox:/var/vmail/%%d/%%n/mdbox:ALT=/var/vmail-archive/%%d/%%n/mdbox prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /etc/dovecot/masters.db driver = passwd-file master = yes pass = yes } plugin { acl = vfile acl_shared_dict = proxy::acl imapsieve_mailbox1_before = file:/var/vmail/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/vmail/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * imapsieve_mailbox3_before = file:/var/vmail/sieve/report-spam.sieve imapsieve_mailbox3_causes = COPY imapsieve_mailbox3_name = Junk imapsieve_mailbox4_before = file:/var/vmail/sieve/report-ham.sieve imapsieve_mailbox4_causes = COPY imapsieve_mailbox4_from = Junk imapsieve_mailbox4_name = * mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_new3 = Deleted Items mailbox_alias_old = Sent mailbox_alias_old2 = Sent mailbox_alias_old3 = Trash quota = dict:User quota::proxy::quota quota_grace = 10%% quota_rule2 = Trash:ignore quota_rule3 = Junk:ignore quota_warning = storage=80%% quota-warning 90 %u quota_warning2 = storage=85%% quota-warning 95 %u quota_warning3 = storage=95%% quota-warning 105 %u sieve = /var/vmail/%d/%n/sieve/.sieve sieve_after = /var/vmail/%d/%n/sieve/autoreply.sieve sieve_before = /var/vmail/sieve/global.sieve sieve_dir = /var/vmail/%d/%n/sieve sieve_extensions = +editheader +vacation-seconds sieve_global_dir = /var/vmail/sieve/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_max_redirects = 20 sieve_max_script_size = 2M sieve_pipe_bin_dir = /usr/lib/dovecot sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_send_from_recipient = yes } protocols = imap sieve lmtp pop3 service auth-worker { unix_listener auth-worker { user = vmail } user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = vmail } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { process_min_avail = 10 service_count = 0 vsz_limit = 512 M } service imap { executable = imap process_limit = 3500 unix_listener imap-master { user = $default_internal_user } vsz_limit = 2 G } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 400 } service pop3-login { process_min_avail = 3 service_count = 0 vsz_limit = 320 M } service pop3 { process_limit = 200 vsz_limit = 320 M } service quota-warning { executable = script /etc/dovecot/quota_warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_cert = </etc/ssl/private/multi.z-technics.cz.dovecot.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it stats_writer_socket_path = userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_compile_errors = 5 } protocol imap { mail_plugins = quota imap_quota fts mailbox_alias imap_acl acl imap_zlib imap_sieve } protocol lda { mail_fsync = optimized } protocol lmtp { mail_fsync = optimized mail_plugins = quota sieve acl }

2 4

imapc master user & shared mailboxes
by Toma 04 Sep '20

04 Sep '20

Hello, I am trying to configure dovecot like this: https://wiki.dovecot.org/SharedMailboxes/ClusterSetup imapc_host = 127.0.0.1 imapc_master_user = %u #imapc_user = imapc_password = XXXXXXXX namespace { type = shared separator = / prefix = shared/%%u/ location = imapc:~/shared/%%u } plugin { acl_shared_dict = proxy::acl } However, contrary to the article claims, imapc master user is set to the destination username. This makes it impossible to check the ACLs against the logged user. If user.b shares mailbox with user.a, accessing /shared/user.b as user.a a will log in as "user.b*user.b". Can we tell dovecot to use the currently authenticated user for imapc_master_user and the remote user for imapc_user, so it logs in as "user.b*user.a"? I'm using dovecot 2.3.11.3 (502c39af9). Thanks! Toma

1 0

syslog logging + 2.3.10 + centos 7
by Matt Bryant 04 Sep '20

04 Sep '20

All, Running dovecot 2.3.10 on a CentOS 7.x and logging via syslog seems to stop when i restart rsyslog process. Reload doesnt get it working again only a dovecot restart. Tested this with dovecot 2.2.36 and this did not exhibit the same behaviour. systemctl restart dovecot rsyslog ----- this gets a working state systemctl restart rsyslog ------------- this restarts syslog other processes like exim continue to log but dovecot stops systemctl reload dovecot -------------- this has no impact logging still does not work systemctl restart dovecot ------------- this fixes the issue. Are there any changes to the log process between 2.2 and 2.3 that might explain this ? rgds Matt

1 1

How to backup maildir
by Robert Moskowitz 04 Sep '20

04 Sep '20

I have been thinking, and reading, on how to back up my mailserver. I have not found any approach that seems ready to use. I have run years without any backup, but would really like to have something in place. I figure I can attach a USB drive and backup to that, then from there rsync to something elsewhere. Further if that USB drive is a full mailserver image, I actually have a 'hot backup' where I only have to put the backup drive into a system and boot up at the last backup. But this means properly copying all of /home/vmail and probably /home/sieve plus the /var/lib/mysql Are there good tools that nicely does this? Or do I choose a time late at night (only I am sometimes in non-US timezones) to shut down all services and just use rsync? And stopping services itself is thought provoking. What if Dovecot, amavis, mysql, or whatelse is in the middle of writing out a mail file what happens to that file and restart. Just scary stuff and, in part, why I have never tackled this in the past. thanks for all feedback

4 6

Deleted mail reappearing
by John S 04 Sep '20

04 Sep '20

Recently I upgraded dovecot quite a few major versions. Since then some mailboxes are experiencing deleted mail reappearing. It seems to be happening roughly once a day, but not for every mail client or mailbox, I have at least two clients with this issue: Thunderbird and Outlook. Storage is local under two partitions, one as alternative storage. # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-29-generic x86_64 Ubuntu 20.04 LTS ext4 auth_mechanisms = plain login digest-md5 base_dir = /var/run/dovecot/ default_client_limit = 4000 default_vsz_limit = 512 M dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = mdbox:/home/vmail/%d/%n mail_max_userip_connections = 240 mail_plugins = quota mail_privileged_group = mail mdbox_rotate_size = 64 M namespace { inbox = yes location = prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user_quota::proxy:/var/run/dovecot/dict:quota quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /etc/dovecot/sieve/ } protocols = imap pop3 service anvil { client_limit = 2615 } service auth { client_limit = 5536 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = root mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { client_limit = 2115 executable = /usr/lib/dovecot/imap-login inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 2000 } service imap { executable = /usr/lib/dovecot/imap process_limit = 2000 } service pop3-login { client_limit = 1024 executable = /usr/lib/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_limit = 512 } service pop3 { executable = /usr/lib/dovecot/pop3 } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert = < /etc/dovecot/ssl/crt.pem ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes syslog_facility = local0 userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { imap_max_line_length = 64 k mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota sieve postmaster_address = postmaster@localhost sendmail_path = /usr/lib/sendmail } dovecot-sql.conf: driver = mysql connect = host=127.0.0.1 dbname=mail user=mail password=#### password_query = SELECT IF('%m' = 'DIGEST-MD5', users.password_digest_md5, NULL) as password, IF('%m' = 'DIGEST-MD5', NULL, 'Y') as nopassword, username FROM users LEFT JOIN domains ON users.domain = domains.domain WHERE users.username = '%u' AND users.domain = '%d' AND users.active = 'yes' AND domains.state = '1' AND ((users.password = encrypt('%w', LEFT(users.password, 11)) AND ('%m' = 'PLAIN' OR '%m' = 'LOGIN')) OR ('%m' = 'DIGEST-MD5')) user_query = SELECT maildir as home, uid, gid, CONCAT(CONCAT(type, ':', maildir), ':ALT=/home/altmail/%d/%n') AS mail FROM users LEFT JOIN domains ON users.domain = domains.domain WHERE users.username = '%u' AND domains.state = '1' iterate_query = SELECT SUBSTRING_INDEX(email, '@', 1) AS username, domain FROM users;

3 3

Sieve: deleteheader now working with duplicate filter for implicit keep
by Alec Moskvin 04 Sep '20

04 Sep '20

Hello, I have a rule to always delete a header. If the message gets fileinto'd, the header is gone, but if it's delivered into the INBOX through implicit keep, the header does not get deleted. Interestingly, if I remove the "if duplicate" filter, the header does get deleted as expected. I'm using the latest Dovecot 2.3.11.3 / Pigeonhole 0.5.11. Any idea why this is happening? Thanks, Alec === Sieve filter require ["fileinto", "editheader", "duplicate", "mailbox"]; deleteheader "X-Bad-Header"; if duplicate { fileinto :create "Trash"; stop; } if header :matches "subject" "*filter*" { fileinto :create "Filtered"; } === Example (note ${RANDOM} to bypass duplicate filter) $ /usr/libexec/dovecot/deliver << -end- To: me(a)example.com From: someone(a)example.com Subject: Test Message Message-Id: <${RANDOM}(a)example.net> Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC) X-Bad-Header: DELETE ME Hello world -end-

2 4

ltmp delivery with ldap and prefetch
by Marc Roos 04 Sep '20

04 Sep '20

I am able to authorize and connect via imap. But I am not able to deliver a message via lmtp I am getting this error message. passdb didn't return userdb entries, trying the next userdb [1] dovecot-ldap.conf.ext pass_attrs = uid=user,userPassword=password,host=host,homeDirectory=userdb_home,uidNu mber=userdb_uid,gidNumber=userdb_gid user_attrs = uid=%{ldap:uidNumber},gid=%{ldap:gidNumber},home=%{ldap:homeDirectory} [2] pass / user db's passdb { args = /etc/dovecot/dovecot-ldap.conf.ext auth_verbose = default default_fields = deny = no driver = ldap master = no mechanisms = name = override_fields = pass = no result_failure = continue result_internalfail = continue result_success = return-ok skip = never username_filter = } userdb { args = /etc/dovecot/special-userdb auth_verbose = default default_fields = driver = passwd-file name = override_fields = result_failure = continue result_internalfail = continue result_success = return-ok skip = never } userdb { args = auth_verbose = default default_fields = driver = prefetch name = override_fields = result_failure = continue result_internalfail = continue result_success = return-ok skip = never }

1 0

Moving Spam to Junk Folder
by bobby 03 Sep '20

03 Sep '20

I am following this tutorial: https://www.linuxbabe.com/redhat/spamassassin-centos-rhel-block-email-spam. I followed the steps in "Move Spam into the Junk Folder". When I send an email from a blacklisted e-mail address, I get a bounce e-mail from my e-mail server. Here is what is in my spamass-milter file: EXTRA_FLAGS="-m -r 8 -R NO_SPAM -i 127.0.0.1 -g sa-milt -- --max-size=5120000" I would prefer it to go into my Junk folder. How can I make this happen?

3 3

Fwd: zlib errors after upgrading to 2.3.11.3
by Robert Nowotny 03 Sep '20

03 Sep '20

Dear Aki, I switched to "gz" now, since "zstd" also gave some errors on writing to files. I dont know if "xz" compression or "zstd" shreddered my MDBOX Files, but I lost 4 days of mail. (a couple of thousand mails). After restoring the backup (what was made after switching to version 2.3.11.3) I still have some broken mdfiles, but not too many. Interestingly always in the "Sent" Mailbox on a number of Users. I just can not go back to 20.08.2020 before I updated to 2.3.11.3 - too many emails would have been lost. So - the current status is 2.3.11.3, with "gz" compression. I force-synced and re-indexed all the mdbox files (250 GB), but still have some broken - please how can I fix those ? You stated : (In theory you could leave the existing mails xz-compressed, but best would be to re-compress everything via dsync so old mails can be read when we eventually remove xz support.) what is the optimal way to do that, expecially not loosing the index (?) for outlook/thunderbird, for third party tools that rely on some index/hash (dunno how exactly that works) ? Sep 3 08:33:25 lxc-imap dovecot: imap(mpaul)<48684><2/9E5mKuAezAqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<48089><YRio1GKuEdHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<49228><p/0g9WKuVdHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<49229><GMAm9WKuVtHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:27 lxc-imap dovecot: imap(mpaul)<49230><YYot9WKuV9HAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) ... Sep 3 09:04:44 lxc-imap dovecot: imap(cpotzinger)<49040></X+P72KuUNHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:04:51 lxc-imap dovecot: imap(cpotzinger)<54382><eYT1XWOux9HAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:04:59 lxc-imap dovecot: imap(cpotzinger)<54396><nKllXmOuyNHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:05:06 lxc-imap dovecot: imap(cpotzinger)<54409><m2TOXmOuydHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:05:12 lxc-imap dovecot: imap(cpotzinger)<54422><IP86X2OuytHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) ... Sep 3 09:08:58 lxc-imap dovecot: imap(cpotzinger)<54867><BtGtbGOu49HAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:19:15 lxc-imap dovecot: imap(mpaul)<56030></kHNkWOuAO3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:41 lxc-imap dovecot: imap(cpotzinger)<57274><c4rCtWOuKuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:47 lxc-imap dovecot: imap(cpotzinger)<57320><K3Uqt2OuLOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:54 lxc-imap dovecot: imap(cpotzinger)<57333><e4aMt2OuLezAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:01 lxc-imap dovecot: imap(cpotzinger)<57346><JEfzt2OuLuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:08 lxc-imap dovecot: imap(cpotzinger)<57360><mH1YuGOuMOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:14 lxc-imap dovecot: imap(cpotzinger)<57382><IzLBuGOuMezAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:21 lxc-imap dovecot: imap(cpotzinger)<57397><q10kuWOuM+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:27 lxc-imap dovecot: imap(cpotzinger)<57410><CSuJuWOuNOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:22 lxc-imap dovecot: imap(mpaul)<57525><TfoOvWOuWO3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:49 lxc-imap dovecot: imap(cpotzinger)<57423><Be/ruWOuNuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:55 lxc-imap dovecot: imap(cpotzinger)<57584><yvXOvmOuQuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:02 lxc-imap dovecot: imap(cpotzinger)<57597><e/8tv2OuQ+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:08 lxc-imap dovecot: imap(cpotzinger)<57611><BLSOv2OuROzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:19 lxc-imap dovecot: imap(cpotzinger)<57633><Ky0twGOuR+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:26 lxc-imap dovecot: imap(cpotzinger)<57646><ljSRwGOuS+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:33 lxc-imap dovecot: imap(cpotzinger)<57659><rn75wGOuT+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:39 lxc-imap dovecot: imap(cpotzinger)<57672><GP9jwWOuUuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:33:45 lxc-imap dovecot: imap(mpaul)<57795><2CEwxWOubu3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:29 lxc-imap dovecot: imap(mpaul)<58347><CKyX1mOuie3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:32 lxc-imap dovecot: imap(mpaul)<58352><osHU1mOuiu3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:44 lxc-imap dovecot: imap(cpotzinger)<54527><5ZpiYmOuBOfAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:51 lxc-imap dovecot: imap(cpotzinger)<58378><kAuN12OumuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:57 lxc-imap dovecot: imap(cpotzinger)<58392><xJLx12Oum+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) ... Am 26.08.2020 um 09:42 schrieb Robert Nowotny: > I had 4 broken mdbox files for 4 different users. > After deleting them, dovecot re-indexed, now everything is ok again. > > I can try to update tonight again to 2.3.11.3 and see if some broken > files will appear again tomorrow. > > Am 25.08.2020 um 14:51 schrieb Aki Tuomi: >>> On 25/08/2020 14:35 Robert Nowotny<rnowotny(a)rotek.at> wrote: >>> >>> >>> I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3 >>> >>> >>> Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63870><jZk...>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[]).... >>> >>> many of those entries.... >>> >>> after installing2.3.10.1 those files are still broken - but new files are working again. >>> >>> anything I can tweak or do ? >>> >>> >> Hi! >> >> Is it possible your mdbox file is corrupted? Can you share doveconf -n? >> >> Aki >

1 0

Submission proxy warning connecting to Exim
by Alec Moskvin 03 Sep '20

03 Sep '20

Hello, When connecting to Exim 4.94, dovecot submission logs this warning: dovecot[25235]: submission(alec)<25347><QNydV8+t+oMKAgAF>: Warning: smtp-client: conn localhost:25 ([::1]:25) [1]: Received invalid EHLO response line: Unexpected character in EHLO keyword Looks like it doesn't like the underscore in "X_PIPE_CONNECT", since it's neither "isalnum" nor a dash [1]: 1598480923.401085 220 myhostname ESMTP Exim 4.94 Wed, 26 Aug 2020 18:28:43 -0400 1598480923.401906 250-myhostname Hello localhost [::1] 1598480923.401906 250-SIZE 52428800 1598480923.401906 250-8BITMIME 1598480923.401906 250-PIPELINING 1598480923.401906 250-X_PIPE_CONNECT 1598480923.401906 250-CHUNKING 1598480923.401906 250-STARTTLS 1598480923.401906 250 HELP 1598480923.409242 250 OK 1598480923.434743 250 Accepted 1598480923.578335 250- 511 byte chunk, total 511 1598480923.578335 250 OK id=1kB3uN-0006ar-HU Adding the line "pipelining_connect_advertise_hosts =" to Exim's config fixes the issue by disabling X_PIPE_CONNECT. Is this a bug in Exim or is Dovecot being too strict? Thanks, Alec [1] https://github.com/dovecot/core/blob/2.3.11.3/src/lib-smtp/smtp-syntax.c#L2…

3 2

Sumbission crashes when relaying over TLS
by Alec Moskvin 03 Sep '20

03 Sep '20

Hello, I'm trying to set up the submission proxy, but if I set submission_relay_ssl = starttls, it crashes. Without it, it works. Please find the details below. Thanks, Alec dovecot[256855]: submission-login: Login: user=<alec>, method=PLAIN, rip=::1, lip=::1, mpid=257033, secured, session=<kMd1B9uthLUAAAAAAAAAAAAAAAAAAAAB> dovecot[256855]: submission(alec)<257033><kMd1B9uthLUAAAAAAAAAAAAAAAAAAAAB>: Panic: file smtp-client-connection.c: line 290 (smtp_client_connection_fail): assertion failed: (conn->to_connect == NULL) dovecot[256855]: submission(alec)<257033><kMd1B9uthLUAAAAAAAAAAAAAAAAAAAAB>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7fbbfb2bcaf2] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7fbbfb2bcc0e] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf5f09) [0x7fbbfb2c5f09] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf5fa1) [0x7fbbfb2c5fa1] -> /usr/lib64/dovecot/libdovecot.so.0(+0x51eec) [0x7fbbfb221eec] -> /usr/lib64/dovecot/libdovecot.so.0(+0x44d5e) [0x7fbbfb214d5e] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6c431) [0x7fbbfb23c431] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_client_command_input_reply+0x201) [0x7fbbfb235261] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6ccfd) [0x7fbbfb23ccfd] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x68) [0x7fbbfb2dc048] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) [0x7fbbfb2dd662] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x51) [0x7fbbfb2dc0f1] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fbbfb2dc2b0] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fbbfb255ee3] -> dovecot/submission(main+0x4df) [0x561d486b3daf] -> /lib64/libc.so.6(__libc_start_main+0xea) [0x7fbbfb054b8a] -> dovecot/submission(_start+0x2a) [0x561d486b3e3a] dovecot[256855]: submission(alec)<257033><kMd1B9uthLUAAAAAAAAAAAAAAAAAAAAB>: Fatal: master: service(submission): child 257033 killed with signal 6 (core dumped) (gdb) bt #0 0x00007fbbfb069a31 in raise () from /lib64/libc.so.6 #1 0x00007fbbfb0532b6 in abort () from /lib64/libc.so.6 #2 0x00007fbbfb22223c in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at failures.c:459 #3 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at failures.c:471 #4 0x00007fbbfb2c5fa1 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at failures.c:848 #5 0x00007fbbfb221eec in i_panic (format=format@entry=0x7fbbfb30fd08 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:523 #6 0x00007fbbfb214d5e in smtp_client_connection_fail (status=9002, error=<optimized out>, conn=0x561d491296a8) at smtp-client-connection.c:290 #7 smtp_client_connection_fail (conn=0x561d491296a8, status=<optimized out>, error=<optimized out>) at smtp-client-connection.c:282 #8 0x00007fbbfb23c431 in smtp_client_connection_starttls_cb (reply=0x561d4912d048, conn=0x561d491296a8) at smtp-client-connection.c:788 #9 0x00007fbbfb235261 in smtp_client_command_input_reply (cmd=<optimized out>, reply=reply@entry=0x561d4912d048) at smtp-client-command.c:971 #10 0x00007fbbfb23ccfd in smtp_client_connection_input_reply (reply=0x561d4912d048, conn=0x561d491296a8) at smtp-client-connection.c:1045 #11 smtp_client_connection_input (_conn=<optimized out>) at smtp-client-connection.c:1116 #12 0x00007fbbfb2dc048 in io_loop_call_io (io=0x561d49109cb0) at ioloop.c:713 #13 0x00007fbbfb2dd662 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x561d490fd050) at ioloop-epoll.c:222 #14 0x00007fbbfb2dc0f1 in io_loop_handler_run (ioloop=0x561d490fd050) at ioloop.c:765 #15 0x00007fbbfb2dc2b0 in io_loop_run (ioloop=0x561d490fd050) at ioloop.c:738 #16 0x00007fbbfb255ee3 in master_service_run (service=0x561d490fceb0, callback=<optimized out>) at master-service.c:851 #17 0x0000561d486b3daf in main (argc=<optimized out>, argv=<optimized out>) at main.c:396 $ openssl version OpenSSL 1.1.1g 21 Apr 2020 # doveconf -n # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # OS: Linux 5.8.4 x86_64 Gentoo Base System release 2.7 # Hostname: haswell listen = localhost mail_location = maildir:~/.maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } protocols = submission ssl_cert = </etc/ssl/dovecot/server.pem ssl_client_ca_file = </etc/dovecot/ca.crt ssl_key = # hidden, use -P to show it submission_relay_host = localhost submission_relay_port = 2525 submission_relay_rawlog_dir = %h/rawlog submission_relay_ssl = starttls userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol submission { rawlog_dir = %h/rawlog }

2 2

submission message quota
by Gerry 03 Sep '20

03 Sep '20

Hello, I am trying to come up with a way to have individual quotas per user for the submission service. Similar to what I could achieve with Postfix and policyd. More specifically, the quota I am most interested in, is limiting the number of messages a single account can send within a given timeframe. Ideally, I'd also like to limit the number of total recipients within a given timeframe, to mitigate the loophole of adding multiple recipients to a single message. Example: account Y is allowed to send 500 messages per 60 minutes, with a maximum of 2000 recipients overall. What would be the best path to take? Thanks! Gerry

4 5

Metric label values truncated when using OpenMetrics endpoint
by Daan van Gorkum 02 Sep '20

02 Sep '20

Hi, Recently we upgraded to Dovecot 2.3.11.3 and configured an example metric like this: metric imap_command { event_name = imap_command_finished group_by = cmd_name tagged_reply_state user remote_ip } And enabled the OpenMetrics listener like this: service stats { inet_listener http { port = 5000 } } While the result is great, I noticed that some metrics are being truncated (three dots at the end of the label value): dovecot_imap_command_duration_usecs_sum{cmd_name="FETCH",tagged_reply_state="OK",user="xxxxxxxxxxx.inbox@xxxxxxxxxxxxx…",remote_ip="XXXX:XXXX:XXXX:XXXX:cc6b:cc7a:d…"} 2346 1598055662473 I couldn't find this information in the documentation so I'm wondering if this is a bug. Thank you for your time. Regards, Daan

2 5

systemctl unit should depend on remote-fs
by Tom Sommer 02 Sep '20

02 Sep '20

The shipped systemctl unit file should have "remote-fs.target" added in "After="? -- Tom

1 0

Mail replication does not work
by Pavel Just 02 Sep '20

02 Sep '20

Hi, I have installed two Debian 10.5 (SSH server) systems and I would like replicate mails between them. But the replication is very unstable. Before start of dovecot Source mailbox: 278 messages, Destination mailbox: 0 messages Another mailbox: 16887 messages, D.. 0 messages After start of dovecot Source mailbox: 278 messages, Destination mailbox: 303 messages Another mailbox: 16887 messages, Dest..: 31904 messages In many cases replication hangs. Error message is in the end of the message. If replication hangs, after restart a lot of mail are copied from destination back to source and mailbox grows without new messages. Postfix was stopped during replication. Please, what is wrong in my configuration (except security :-) ? -- Pavel # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.11 (6c69c917) # OS: Linux 4.19.0-10-amd64 x86_64 Debian 10.5 # Hostname: mail.simac.cz disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_uid = 1001 last_valid_uid = 32000 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " notify replication" mail_privileged_group = mail mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { mail_replica = tcp:192.168.240.2:12345 sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = " imap sieve pop3" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = root } unix_listener replication-notify { mode = 0666 user = root } } service doveadm { inet_listener { port = 12345 ssl = no } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 user = root } } service stats { unix_listener stats-reader { mode = 0666 user = root } unix_listener stats-writer { mode = 0666 user = root } } ssl = no userdb { driver = passwd } protocol lmtp { mail_plugins = " notify replication sieve" } protocol lda { mail_plugins = " notify replication sieve" } ____________________________________ Error message ________________ Sep 2 14:08:00 backup dovecot: doveadm: Panic: file mbox-lock.c: line 796 (mbox_lock): assertion failed: (lock_type == F_RDLCK || mbox->mbox_lock_type != F_RDLCK) Sep 2 14:08:00 backup dovecot: doveadm: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7f300ef00eed] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f300ef0100e] -> /usr/lib/dovecot/libdovecot.so.0(+0xf465b) [0x7f300ef0a65b] -> /usr/lib/dovecot/libdovecot.so.0(+0xf46f1) [0x7f300ef0a6f1] -> /usr/lib/dovecot/libdovecot.so.0(+0x514a6) [0x7f300ee674a6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x438e5) [0x7f300f0108e5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mbox_save_begin+0x6fd) [0x7f300f05fa7d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x92) [0x7f300f0299e2] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x89) [0x7f300f01c4e9] -> /usr/lib/dovecot/modules/lib15_notify_plugin.so(+0x246f) [0x7f300e71846f] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5cb3c) [0x7f300f029b3c] -> dovecot/doveadm-server(+0x4f805) [0x556ee519a805] -> dovecot/doveadm-server(+0x501f7) [0x556ee519b1f7] -> dovecot/doveadm-server(dsync_mailbox_import_changes_finish+0x164) [0x556ee519dba4] -> dovecot/doveadm-server(dsync_brain_sync_mails+0x735) [0x556ee5198e85] -> dovecot/doveadm-server(dsync_brain_run+0x218) [0x556ee5194858] -> dovecot/doveadm-server(+0x49eae) [0x556ee5194eae] -> dovecot/doveadm-server(+0x5aa6f) [0x556ee51a5a6f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f300ef20989] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7f300ef21ed1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f300ef20a2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f300ef20ba0] -> dovecot/doveadm-server(+0x2e20e) [0x556ee517920e] -> dovecot/doveadm-server(+0x2fa56) [0x556ee517aa56] -> dovecot/doveadm-server(+0x45cad) [0x556ee5190cad] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f300ef20989] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7f300ef21ed1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f300ef20a2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f300ef20ba0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f300ee9a9c3] Sep 2 14:08:00 backup dovecot: doveadm: Fatal: master: service(doveadm): child 1977 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps)

2 1

Incremental purge?
by Jesus Cea 02 Sep '20

02 Sep '20

I am using mdbox storage format. As far as I know, “doveadm purge” does a complete purge. The time and disk activity would be proportional to the number mdbox files on the storage. The purge time could be huge. I wonder if there is some way of doing a kind of incremental purge, stopping/restarting purging, etc. Purging by file range would be nice too. Advice? Thanks Enviado desde mi iPhone

5 7

Dovecot Proxy
by Thoralf Rickert-Wendt 02 Sep '20

02 Sep '20

Hello everyone, it's my first post here on this mailing list and I hope, I make it right. I posted a question on https://serverfault.com/questions/1031441/dovecot-as-proxy-with-submission and nobody was able to answer it. So I decided to push that question here (I'm talking about any new dovecot version and I've tested it with 2.3.4.1 (f79e8e7e4)). I try to run a dovecot proxy in front of a big number of mail servers (serving SMTP-in, submission, IMAP, POP3, Sieve). I need that proxy, because I run out of IPv4 addresses. Of course I use IPv6 too, but many customers still have problems with there providers and they really don't want to share their mails on a "shared-mailserver". I planed to use Dovecot for IMAPS, POP3S, SMTP-submission(465) and postfix for the rest. If I find a solution for sieve, I would try that too, but that is very optional. With the documentation https://wiki1.dovecot.org/HowTo/ImapProxy (which is really old and should be updated) and some other ascii docs (from an Apple mirror somewhere deep in the web) I was able to build a IMAP/POP3 proxy that forwards requests from outside to a specific backend using SSL (993,995). That works - I think.You can find the config on the serverfault page. In general - all known domains in backend are using SSL and the passdb forwards all requests to the backend via SSL. So - I understand:|| |password_query = SELECT NULL AS password, NULL AS destuser, host, 'Y' AS nologin, 'Y' AS nodelay, 'Y' AS nopassword, 'Y' AS proxy, 'any-cert' AS `ssl` FROM proxy_domain WHERE domain = '%d' | But that is only 50% of the show. The rest ist submission (and maybe sieve). Practically the submission implementation in dovecot works too. But because dovecot by default only opens port 587 (starttls), my passdb setting has a problem. When I try to use that port Dovecot tries to use SSL on the backend/587 too - but that is wrong (it should either use 465 or should try to use starttls). So, I have the following options. - find a way to configure dovecot-proxy to listen on 465 with SSL for submission service and hope that it uses the same port - but I didn't find any documentation for that and need help - find a way to configure dovecot-proxy/passdb to return starttls=y when dovecot-submission is used (use a different passdb) - but I didn't find any documentation for that and I'm not sure, if this worls on service/protocol level - find a way to configure the passdb answer based on the used port/protocol. But I only know the parameter %u, %d and %p. - so it would be nice to find a way to also select the protocol (if already developed) - find a way to make a patch in dovecot (which isn't easy for me, because I don't really know the code) Has somebody an idea, how I can configure the dovecot-proxy in that way. bye Thoralf

3 4