- dovecot - dovecot.org

Question about login_log_format_elements in a proxy environment
by Adi Pircalabu 08 Dec '20

08 Dec '20

Hi, I have a Dovecot proxy setup with several proxy machines (currently running 2.3.11.3) in front of the real Dovecot servers (2.3.10.1) storing the mailboxes. "doveconf -a | egrep lip" returns: login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c session=<%{session}> In the real server maillog I'm expecting to have "lip" replaced with the IP address of the proxy. It works as expected for imap-login processes, however for pop3-login processes I still see the real server IP instead of the proxy IP. Ideas? Regards, -- Adi Pircalabu

2 8

Fts solr add fields message-id, date, in-reply-to
by michael1970 08 Dec '20

08 Dec '20

Hi, how can I add fields message-id, date, in-reply-to to fts solr index like hdr,body,subject or to? Thanks, Michael -- Sent from: http://dovecot.2317879.n4.nabble.com/

3 6

dovecot/imap killed with signal 6 after SEARCH in specific mailbox
by Jiri 07 Dec '20

07 Dec '20

Hi All, on my older production server (Deb9 dovecot version 2.2.27-3+deb9u6) a client reported that searching text in one mailbox failed while other mailboxes do search ok. He is using Rouddcube webmail and is searching the whole messages including the body. Errorlog shows panic and assertion failed logs. So I'v moved the 'problematic' mailbox to my latest production server (Deb10 2.3.4.1-5+deb10u4 = 2.3.4.1 (f79e8e7e4)) where I have more RAM to eliminate that and the same error appeared. There is nothing special about the mailbox, roughly 600MiB and 2200 messges. Other much larger mailboxes on the same server do search fine. After turning on imap logging in Roundcube mail, the critical command was: A0004 UID SEARCH RETURN (ALL) TEXT whatever I increased default_process_limit and default_client_limit, I run: doveadm index -u ttt(a)mail2.swpraha.cz -q INBOX but didn't help. I used the gdb method to reproduce it and to get the whole info including backtrace: vmail@s6mail:~$ gdb --args /usr/lib/dovecot/imap -u ttt(a)mail2.swpraha.cz (gdb) r Starting program: /usr/lib/dovecot/imap -u ttt(a)mail2.swpraha.cz process 4458 is executing new program: /usr/bin/doveconf process 4458 is executing new program: /usr/lib/dovecot/imap * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY SPECIAL-USE QUOTA] Logged in as ttt(a)mail2.swpraha.cz a select INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent Junk $Forwarded NonJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent Junk $Forwarded NonJunk \*)] Flags permitted. * 682 EXISTS * 0 RECENT * OK [UNSEEN 639] First unseen. * OK [UIDVALIDITY 1147771637] UIDs valid * OK [UIDNEXT 104487] Predicted next UID * OK [HIGHESTMODSEQ 53108] Highest a OK [READ-WRITE] Select completed (0.002 + 0.000 + 0.001 secs). A0004 UID SEARCH RETURN (ALL) TEXT whatewer imap(ttt(a)mail2.swpraha.cz)<4458><Auy4NnMnzl9qEQAAevFY8w>: Panic: file message-parser.c: line 174 (message_part_finish): assertion failed: (ctx->nested_parts_count > 0) imap(ttt(a)mail2.swpraha.cz)<4458><Auy4NnMnzl9qEQAAevFY8w>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb62b) [0x7ffff7dbb62b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb661) [0x7ffff7dbb661] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a149) [0x7ffff7d2a149] -> /usr/lib/dovecot/libdovecot.so.0(+0x474ac) [0x7ffff7d274ac] -> /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_next_block+0x104) [0x7ffff7da3844] -> /usr/lib/dovecot/libdovecot.so.0(message_search_msg+0xa8) [0x7ffff7da5df8] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xcf89e) [0x7ffff7f3e89e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_foreach+0x45) [0x7ffff7ec0445] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xd0774) [0x7ffff7f3f774] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xd1a68) [0x7ffff7f40a68] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x10d) [0x7ffff7f4118d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x28) [0x7ffff7ec9e58] -> /usr/lib/dovecot/imap [ttt(a)mail2.swpraha.cz UID SEARCH](+0x2691f) [0x55555557a91f] -> /usr/lib/dovecot/imap [ttt(a)mail2.swpraha.cz UID SEARCH](command_exec+0x70) [0x555555573d80] -> /usr/lib/dovecot/imap [ttt(a)mail2.swpraha.cz UID SEARCH](+0x25ed2) [0x555555579ed2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0x111) [0x7ffff7dd18f1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd0) [0x7ffff7dd3070] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7ffff7dd1b7c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7ffff7dd1ce0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ffff7d520d3] -> /usr/lib/dovecot/imap [ttt(a)mail2.swpraha.cz UID SEARCH](main+0x325) [0x555555564bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7ffff7b3c09b] -> /usr/lib/dovecot/imap [ttt(a)mail2.swpraha.cz UID SEARCH](_start+0x2a) [0x555555564d8a] Program received signal SIGABRT, Aborted. __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt full #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 set = {__val = {0, 9355454118400172544, 1928, 1862, 1863, 93824992506008, 65, 140737352109185, 140737351192526, 140737351951989, 257, 145, 206158430224, 140737488344944, 140737488344736, 140737351756174}} pid = <optimized out> tid = <optimized out> ret = <optimized out> #1 0x00007ffff7b3a535 in __GI_abort () at abort.c:79 save_stage = 1 act = {__sigaction_handler = {sa_handler = 0x9ef, sa_sigaction = 0x9ef}, sa_mask = {__val = {30216, 93824992300426, 0, 513, 513, 140737351192526, 140737351744763, 140737351192526, 9355454118400172544, 140737351192526, 140737351951234, 140737488344944, 93824992503880, 140737351192526, 140737351951609, 140737351192526}}, sa_flags = -136625910, sa_restorer = 0x5} sigs = {__val = {32, 0 <repeats 15 times>}} #2 0x00007ffff7d2a481 in ?? () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007ffff7dbb661 in default_fatal_handler () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00007ffff7d2a149 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00007ffff7d274ac in ?? () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x00007ffff7da3844 in message_parser_parse_next_block () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00007ffff7da5df8 in message_search_msg () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x00007ffff7f3e89e in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #9 0x00007ffff7ec0445 in mail_search_args_foreach () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #10 0x00007ffff7f3f774 in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x00007ffff7f40a68 in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x00007ffff7f4118d in index_storage_search_next_nonblock () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #13 0x00007ffff7ec9e58 in mailbox_search_next_nonblock () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #14 0x000055555557a91f in ?? () No symbol table info available. #15 0x0000555555573d80 in command_exec () No symbol table info available. #16 0x0000555555579ed2 in ?? () No symbol table info available. #17 0x00007ffff7dd18f1 in io_loop_handle_timeouts () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #18 0x00007ffff7dd3070 in io_loop_handler_run_internal () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x00007ffff7dd1b7c in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x00007ffff7dd1ce0 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #21 0x00007ffff7d520d3 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #22 0x0000555555564bf5 in main () No symbol table info available. (gdb) ========================== dovecot -n: # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 5.4.65-1-pve x86_64 Debian 10.6 # Hostname: s6mail.ehosting.cz default_client_limit = 10000 default_process_limit = 1000 deliver_log_format = msgid=%m, from=%f, Subj=%s: %$ dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 150 first_valid_uid = 150 last_valid_gid = 150 last_valid_uid = 150 log_timestamp = "%Y-%m-%d %H:%M:%S " mail_plugins = quota trash expire mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext spamtest spamtestplus namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = pipe antispam_crm_binary = /bin/false antispam_dspam_args = --deliver=;--user;%u;--source=error;--signature=%%s antispam_dspam_binary = /usr/bin/dspam antispam_dspam_notspam = --class=innocent antispam_dspam_result_header = X-DSPAM-Result antispam_dspam_spam = --class=spam antispam_pipe_program = /usr/local/bin/postfix/reclasify-from-dovecot antispam_pipe_program_args = %u;%d antispam_pipe_program_notspam_arg = --ham antispam_pipe_program_spam_arg = --spam antispam_pipe_tmpdir = /tmp antispam_signature = X-CRM114-CacheID antispam_signature_missing = error antispam_spam = Junk;Spam antispam_spool2dir_notspam = /tmp/spamspool/%%020lu-%u-%%05luh antispam_spool2dir_spam = /tmp/spamspool/%%020lu-%u-%%05lus antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_verbose_debug = 0 expire = Trash expire2 = Trash/* expire3 = Spam expire_dict = proxy::expire quota = dict:User quota::proxy::quota quota_exceeded_message = Uzivatel %u ma zaplnenou schranku. quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/dovecot/sieve_after sieve_before = /etc/dovecot/sieve_before sieve_extensions = +spamtest +spamtestplus sieve_spamtest_max_value = 15 sieve_spamtest_status_header = X-Spam-Level sieve_spamtest_status_type = strlen trash = /etc/dovecot/dovecot-trash.conf.ext } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = UID%u-%v postmaster_address = sysadmin(a)ehosting.cz protocols = " imap lmtp sieve pop3" service auth-worker { user = $default_internal_user } service dict { unix_listener dict { user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } service stats { unix_listener stats-writer { user = vmail } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it syslog_facility = local0 userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = quota trash expire sieve quota trash expire } protocol lda { mail_plugins = quota trash expire sieve } protocol imap { mail_plugins = quota trash expire antispam imap_quota } ========================== The dovecot.err shows the two mentioned error lines (Panic:, Error:) plus: Dec 7 13:12:21 s6mail dovecot: imap(ttt(a)mail2.swpraha.cz)<13466><wezwvN61RK0AAAAAAAAAAAAAAAAAAAAB>: Fatal: master: service(imap): child 13466 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) Please let me know if you need any other details. Thanks for any suggestion --Jiri Kacena

2 2

logging in to postfixadmin failed probably because of a dovecot error
by peacecop kalmer: 07 Dec '20

07 Dec '20

I can’t log in to postfixadmin anymore. It says that the username or password is incorrect. It can’t be incorrect as I’ve stored them in Google and “Chrome”. I have two admin accounts and none of them can’t be used for logging in. So I changed the main password with “setup.php” and copied that hash into the configuration file. Then, I deleted one of the admin accounts directly in the database from the table “admin”. Finally, I tried to recreate that deleted admin account using “setup.php”. I get the following error message: “can’t encrypt password with dovecotpw, see error log for details”. I had a look into the error log file of postfixadmin and found these complaints: [Wed Dec 02 02:33:37.551048 2020] [authz_core:debug] [pid 30540] mod_authz_core.c(809): [client 192.168.1.65:54948] AH01626: authorization result of Require all granted: granted, referer:https://postfixadmin.tennis24.ee/setup.php [Wed Dec 02 02:33:37.551070 2020] [authz_core:debug] [pid 30540] mod_authz_core.c(809): [client 192.168.1.65:54948] AH01626: authorization result of : granted, referer:https://postfixadmin.tennis24.ee/setup.php [Wed Dec 02 02:33:37.551280 2020] [php7:warn] [pid 30540] [client 192.168.1.65:54948] PHP Warning: Cannot assign an empty string to a string offset in /usr/share/postfixadmin/variables.inc.php on line 31, referer:https://postfixadmin.tennis24.ee/setup.php [Wed Dec 02 02:33:37.568128 2020] [php7:notice] [pid 30540] [client 192.168.1.65:54948] dovecotpw password encryption failed., referer:https://postfixadmin.tennis24.ee/setup.php [Wed Dec 02 02:33:37.568155 2020] [php7:notice] [pid 30540] [client 192.168.1.65:54948] STDERR output: doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-ssl.conf line 13: ssl_cert: Can't open file /etc/letsencrypt/live/test.tennis24.ee/fullchain.pem: Permission denied\n, referer:https://postfixadmin.tennis24.ee/setup.php [Wed Dec 02 02:33:37.568897 2020] [deflate:debug] [pid 30540] mod_deflate.c(854): [client 192.168.1.65:54948] AH01384: Zlib: Compressed 1320 to 732 : URL /setup.php, referer:https://postfixadmin.tennis24.ee/setup.php Is the issue connected to the permission problem? If so why is the permission denied? I can't even receive any mails to the address kalmer(a)test.tennis24.ee anymore. If I send out it works fine but no mail arrives. -- *peacecop kalmer:* *Instituudi tee 3-27 * *76902 * *Harku * *Harku* *Harjumaa* Estonia *+372 652 4228 * *+372 5620 4556 * *__________________________________________________________________ *

2 1

http pipelining on auth_policy_server
by Adrian M 07 Dec '20

07 Dec '20

Hi, I'm curious to know if dovecot can be configured to do pipelining for auth_policy in order to use less connection to the server and keep them for longer. I'm working on making a small policy server and the lack of persistent connections is a cause of overhead. Thank you !

2 2

[patch] enhancement for adding limits to solr and tika parsing
by John Fawcett 07 Dec '20

07 Dec '20

Hi As mentioned in a previous thread, the solr + tika combination has caused me some issues due to attachment size. While tika seems to be able to parse large attachments, the resulting volume of text can overwhelm the solr server. One solution would be to throw resources at the problem, but in my case such large attachments don't contain anything worthwile indexing. Additionally I don't want people to be able to randomly crash my solr server by sending large compressed attachments that expand into huge volumes for solr. It's also a safety feature to have sane limits on what can be indexed. Attached is a first attempt to address the problem. I did not find a way to easily get actual attachment sizes, so I used an already available information - the overall message size. It may not be ideal but at least introduces limits where none existed. I have introduced two new parameters for the plugin section, for example: plugin { fts_max_size = 2M fts_max_size_tika = 1M } They can be used separately or together. Both sizes refer to the overall message size. The meaning is: fts_max_size - do not parse message bodies if the message size exceeds this value. A value of 0 indicates no limit. If the message body is not parsed, attachments are also not parsed. fts_max_size_tika - do not parse message attachments with tika if the message size exceeds this value. A value of 0 indicates no limit. If using both settings it makes sense to have fts_max_size > fts_max_size_tika, since with a smaller fts_max_size bodies are not indexed including attachments and the fts_max_size_tika will have no effect. The difference (ft_max_size - fts_max_size_tika) places an upper bound on the size of the non attachment body text that will be indexed. However, any attachments over the fts_size will automatically consume this limit and no body text will be indexed for those messages. I've only updated the tika parser not the script parser though the script parser potentially could benefit from this approach. The attached patch also includes the rolled up patch for using basic auth with the tika server and the previous posted patch (not mine) which solves an assert when using solr and tika together. John

1 0

Re: Re: Re: Dovecot+Solr+Tika segmentation fault
by Manuel Josupeit-Walter 07 Dec '20

07 Dec '20

>>> A fix was posted on this list which seems to avoid segfaults when >>> using solr+tika. I reposted it not long ago in response to another >>> question. Before exploring the other segfaults it would be worthwhile >>> getting that fix. If other segfaults continue then I suggest posting >>> the logging from Dovecot >> >> https://github.com/dovecot/core/commit/043d111cea78533cf22d3bfe0872c8376d84… >> (DOV-4090) >> >> Scheduled for 2.3.13. > > I applied the fix and will observe it for the next couple of days. Unfortunately, the issue still persists even with this patch. The log message is slightly different, though, regarding disconnection: Dec 03 17:10:49 server.name tika[988]: INFO tika/ (application/pdf) Dec 03 17:10:49 server.name tika[988]: INFO tika/ (image/png) Dec 03 17:10:49 server.name kernel: indexer-worker[15805]: segfault at 0 ip 00007f140483db64 sp 00007ffc04fa7960 error 4 in libdovecot.so.0.0.0[7f14047f3000+db000] Dec 03 17:10:49 server.name kernel: Code: 89 f5 48 8d b8 f0 01 00 00 48 8b 52 08 4c 8b 62 48 e8 90 89 fb ff 48 8b 44 24 08 be 00 20 00 00 4c 89 ef 48 8b 80 10 01 00 00 <4c> 8b 38 e8 64 b2 fb ff 48 8b 44 24 0 Dec 03 17:10:49 server.name dovecot[988]: Dec 03 16:10:49 indexer: Error: Indexer worker disconnected, discarding 1 requests for username_stripped Dec 03 17:10:49 server.name dovecot[988]: Dec 03 16:10:49 indexer-worker(username_stripped)<1952><Ed3ek5G1BMQgARa4EL8DAKGMINRlZjjS:c+gACAkOyV+gBwAAhFs1yA>: Fatal: master: service(indexer-worker): child 1952 killed It seems that there is something else going wrong in this version. Best, Manuel

3 3

quota-status for outgoing mail (Dovecot/Postfix)
by Peter Folta 06 Dec '20

06 Dec '20

[Dovecot version 2.3.11.3 (502c39af9)] Hi all, I’m using Dovecot’s quota-status service to let Postfix check the recipient’s quota before accepting emails following the guide here: https://blog.sys4.de/postfix-dovecot-mailbox-quota-en.html <https://blog.sys4.de/postfix-dovecot-mailbox-quota-en.html>. This works well for inbound emails. I would like to set up something similar for outbound email, i.e. if a user is over quota, they should not be allowed to send new emails. At the moment, the setup is in a bit of an inconsistent state: Users that have exceeded their quota are still able to send outbound emails but saving them in their Sent folder will fail leaving them without a copy. I have tried using Dovecot’s quota-status in Postfix’s smtpd_send_restrictions but didn’t get this to work. A quick look at the Dovecot source code seems to suggest that the quota-status plugin is only checking the recipient address: https://github.com/dovecot/core/blob/master/src/plugins/quota/quota-status.… <https://github.com/dovecot/core/blob/master/src/plugins/quota/quota-status.…> and https://github.com/dovecot/core/blob/master/src/plugins/quota/quota-status.… <https://github.com/dovecot/core/blob/master/src/plugins/quota/quota-status.…>. I suspect it’s not possible to use the quota-status service to control outbound email but thought I’d ask to confirm. I’m wondering if there’s a different way to achieve this? Thanks Peter

3 2

Files in tmp directory
by David Morsberger 05 Dec '20

05 Dec '20

I had to restore many thousands of dovecot emails from a crashed Mac Server to a new Ubuntu server. After completing I am noticing a few things that may be okay or may need to be fixed. 1) a couple of mailboxes have a lot (one has 102462) emails in the tmp directory. The files do not have the standard Mairdir++ filename format. An example: 1581262582.M397740P72267.mini.mmpcrofton.com <http://1581262582.m397740p72267.mini.mmpcrofton.com/> (missing the stuff at the end, ,S=,W=:2,S) 2) I see files in the cur directory that do not have the Maildir++ format. An example: 1601287326.V810Ie80087M240104.mmp-mail.mmpcrofton.com:2,S. (do not have the ,S=,W= values and the second value, V810Ie80087M240104, is longer that the other file names) How do I get the emails in (1) out of tmp? Do I need to convert the emails in (2)? David

2 3

Combine director and HAProxy for loadbalancig and failover
by Steven Varco 05 Dec '20

05 Dec '20

Hi All I’m trying to achive an active/active cluster with dovecot-director and HAProxy, as the director does not do health checks (loadblancing only) and I want both, loadblancing AND failover, where the latter is far more important to me (loadblancing I would just use as an addon and for curiosity, it is not really need in my setup). However, I’m not sure if this combination can be used that way, as I found almost no documentation on this. What I found is using either directory OR haproxy with dovecot, but not both. So I guess, that the description here is intended to use without the director: https://wiki2.dovecot.org/HAProxy This older list post is basically exaclty, what I’m trying to achive: https://dovecot.org/pipermail/dovecot/2015-July/101487.html Clients --> Load Balancer(HAProxy) --> Dovecot Proxy(DP) --> Dovecot Director(DD) --> MS1 / MS2 As far as I understood, this would require to (statically) set a host= entry for each client which would give another single point of failure? However reading the documentation: https://doc.dovecot.org/configuration_manual/haproxy/ assumes that this setup IS actually possible. Unfortunately it does not describe how dovecot must be setup at this point. So I assume since HAProxy is listening on port 1143 (for IMAP) and dovecot (with directror enabled) is listening on port 143, which is the main entry point for clients. So in this setup dovecot-director should first pass the mail traffic to HAProxy on port 1143 and then HAProxy passes it to the actual imap servers? So how would the dovecot setup on the director servers look like and how on the dovecot imap mail servers? My setup consist of each two machines: loadblancer (with HAProxy and dovecot-director) and dovecot imap servers. Last but not least, I found documentation for „dovemon“, which, if I got that correctly, should replace the external tool „poolmon“: https://doc.dovecot.org/configuration_manual/dovemon/ - However, I could not find out where the configuration YAML file (/etc/dovecot/dovemon.config.yml) should get included in the main dovecot configuration. By default this file would not be included at all and therefore would have no effect… thanks. Steven My setup: LOADBALANCERS: Currently one only running on: 10.0.2.26 haproxy: Exactly as: https://doc.dovecot.org/configuration_manual/haproxy/ where the backend servers line have been replaced with the mail servers. dovecot -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 3.10.0-1127.19.1.el7.x86_64 x86_64 CentOS Linux release 7.8.2003 (Core) # Hostname: lb01.example.com auth_verbose = yes director_mail_servers = 10.0.2.30 10.0.2.29 director_servers = 10.0.2.26 disable_plaintext_auth = no first_valid_uid = 1000 haproxy_trusted_networks = 10.0.2.0/24 mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = proxy=y ssl=any-cert nopassword=y driver = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap pop3 lmtp sieve service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { port = 143 } inet_listener imap_haproxy { haproxy = yes port = 10143 } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } ssl_cert = </etc/letsencrypt/live/mail-lb.tech-island.xyz/fullchain.pem ssl_key = # hidden, use -P to show it verbose_proctitle = yes protocol lmtp { auth_socket_path = director-userdb mail_plugins = " sieve" } protocol lda { mail_plugins = " sieve" } MAIL SERVERS: mx01: 10.0.2.30 mx02: 10.0.2.29 dovecot -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 3.10.0-1127.19.1.el7.x86_64 x86_64 CentOS Linux release 7.8.2003 (Core) # Hostname: mx01.example.com auth_mechanisms = plain login auth_verbose = yes dict { sqlquota = mysql:/etc/dovecot/dict-sqlquota.conf.ext } doveadm_password = # hidden, use -P to show it doveadm_port = 14310 first_valid_uid = 1000 login_trusted_networks = 10.0.2.0/24 mail_location = maildir:~/Maildir mail_plugins = quota notify replication managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_replica = tcp:10.0.2.29 quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.tech-island.xyz/over_quota_help for instructions on how to fix this. quota_rule2 = INBOX.Trash:storage=+100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full / Mailbox ist voll quota_status_success = DUNNO quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = -storage=90%% quota-warning below %u sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = postmaster(a)tech-island.xyz protocols = imap pop3 lmtp sieve replication_dsync_parameters = -d -l 30 -U service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { user = vmail } } service dict { unix_listener dict { user = vmail } } service doveadm { inet_listener { port = 14310 } } service managesieve-login { inet_listener sieve { port = 4190 } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 14340 } } service quota-warning { executable = script /usr/local/libexec/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_cert = </etc/ssl/acme/certs/mail.tech-island.xyz.chain.crt ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = quota notify replication sieve } protocol lda { mail_plugins = quota notify replication sieve } protocol imap { mail_max_userip_connections = 20 mail_plugins = quota notify replication imap_quota } -- https://steven.varco.ch/

1 0