Replicator: Panic: data stack: Out of memory
Reproduce of error (Replicator: Panic: data stack: Out of memory):
When replay on any mail I will got OOM.
I try add in service replicate vsz_limit = 0 but that not help (after proces dovecot/replicator eat all ram I got OOM)
Log error:
Jun 03 09:38:59 Warning: imap(ivan@myserv.local)<2533034><h7EQN4bgaJ4FvZdE>: replication(ivan@myserv.local): Sync failure: Timeout in 2 secs Jun 03 09:39:03 Panic: replicator: data stack: Out of memory when allocating 4294967336 bytes Jun 03 09:39:03 Error: replicator: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7feef08f0582] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7feef08f069e] -> /usr/lib/dovecot/libdovecot.so.0(+0x1022fb) [0x7feef08fd2fb] -> /usr/lib/dovecot/libdovecot.so.0(+0x102391) [0x7feef08fd391] -> /usr/lib/dovecot/libdovecot.so.0(+0x55589) [0x7feef0850589] -> /usr/lib/dovecot/libdovecot.so.0(+0x54d52) [0x7feef084fd52] -> /usr/lib/dovecot/libdovecot.so.0(+0xfb808) [0x7feef08f6808] -> /usr/lib/dovecot/libdovecot.so.0(+0x122938) [0x7feef091d938] -> /usr/lib/dovecot/libdovecot.so.0(+0xf76bf) [0x7feef08f26bf] -> /usr/lib/dovecot/libdovecot.so.0(+0xf777e) [0x7feef08f277e] -> /usr/lib/dovecot/libdovecot.so.0(buffer_append+0x61) [0x7feef08f2a21] -> dovecot/replicator(replicator_queue_push+0x15a) [0x55953ece9b7a] -> dovecot/replicator(+0x62b6) [0x55953ece92b6] -> dovecot/replicator(+0x594e) [0x55953ece894e] -> dovecot/replicator(+0x5b2c) [0x55953ece8b2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7feef0913529] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) [0x7feef0914c12] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) [0x7feef09135d0] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7feef0913790] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7feef0886353] -> dovecot/replicator(main+0x18d) [0x55953ece7cbd] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7feef0651d0a] -> dovecot/replicator(_start+0x2a) [0x55953ece7d7a] Jun 03 09:39:03 Fatal: replicator: master: service(replicator): child 2532886 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps)
My configuration:
# 2.3.19 (b3ad6004dc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 5.10.0-13-amd64 x86_64 Debian 11.3 ext4 # Hostname: mail.myserv.local auth_mechanisms = plain login auth_verbose = yes debug_log_path = /var/log/dovecot.debug default_client_limit = 5000 default_process_limit = 1000 default_vsz_limit = 2 G deliver_log_format = msgid=%m, subject=%s, from=%f, size=%p(%w), %$ dict { quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } doveadm_password = # hidden, use -P to show it hostname = myserv.local lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot.log mail_home = /var/mail/vhosts/%d/mail/%n mail_location = maildir:/var/mail/vhosts/%d/mail/%n:INDEX=/var/mail/vhosts/%d/indexes/%n mail_plugins = " quota fts fts_solr notify replication" mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = Sent autocreate2 = Drafts autocreate3 = Junk autocreate4 = Trash autosubscribe = Sent autosubscribe2 = Drafts autosubscribe3 = Junk autosubscribe4 = Trash expire = Trash expire2 = Trash/* expire3 = Spam fts = solr fts_autoindex = yes fts_solr = url=http://localhost:8983/solr/dovecot/ imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * mail_replica = tcp:imap.myserv2.local:12345 quota = dict:Mailbox::proxy::quota quota_rule = *:storage=1G quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u replication_sync_timeout = 2 sieve = file:/var/mail/vhosts/%d/sieve/%n;active=/var/mail/vhosts/%d/sieve/%n/.dovecot.sieve sieve_after = /etc/dovecot/sieve/after.d/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_quota_max_scripts = 50 } postmaster_address = postmaster@%d protocols = " imap lmtp sieve pop3" service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service dict { unix_listener dict { user = vmail } } service doveadm { inet_listener { port = 12345 ssl = no } } service imap-login { process_min_avail = 4 service_count = 1 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-warning { executable = script /var/lib/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_cert =
Ok a little more help :
vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram.
I use :
service replicator { client_limit = 0 drop_priv_before_exec = no idle_kill = 4294967295s process_limit = 1 process_min_avail = 0 service_count = 0 unix_listener replicator-doveadm { mode = 0600 user = vmail } vsz_limit = 8192M }
next :
replication_max_conns = 100
think of this as a socket that gets opened everytime you connect to the other server
100 means it will handle 100 seperate replication user requests
at a time (which is way more then sufficent)
if the connections overflows it just means the replication que will carry on when resources opens up thus not killing the server resources.
If a que overflows because too much other stuff is going on it will just reque and go again.
also note :
service aggregator { process_limit = 1000 #vsz_limit = 1g fifo_listener replication-notify-fifo { user = vmail group = vmail mode = 0666 }
}
Happy Friday !!! Thanks - paul
Paul Kudla
Scom.ca Internet Services http://www.scom.ca 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3
Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca
On 6/3/2022 5:13 AM, Ivan Jurišić wrote:
Reproduce of error (Replicator: Panic: data stack: Out of memory):
When replay on any mail I will got OOM.
I try add in service replicate vsz_limit = 0 but that not help (after proces dovecot/replicator eat all ram I got OOM)
Log error:
Jun 03 09:38:59 Warning: imap(ivan@myserv.local)<2533034><h7EQN4bgaJ4FvZdE>: replication(ivan@myserv.local): Sync failure: Timeout in 2 secs Jun 03 09:39:03 Panic: replicator: data stack: Out of memory when allocating 4294967336 bytes Jun 03 09:39:03 Error: replicator: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7feef08f0582] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7feef08f069e] -> /usr/lib/dovecot/libdovecot.so.0(+0x1022fb) [0x7feef08fd2fb] -> /usr/lib/dovecot/libdovecot.so.0(+0x102391) [0x7feef08fd391] -> /usr/lib/dovecot/libdovecot.so.0(+0x55589) [0x7feef0850589] -> /usr/lib/dovecot/libdovecot.so.0(+0x54d52) [0x7feef084fd52] -> /usr/lib/dovecot/libdovecot.so.0(+0xfb808) [0x7feef08f6808] -> /usr/lib/dovecot/libdovecot.so.0(+0x122938) [0x7feef091d938] -> /usr/lib/dovecot/libdovecot.so.0(+0xf76bf) [0x7feef08f26bf] -> /usr/lib/dovecot/libdovecot.so.0(+0xf777e) [0x7feef08f277e] -> /usr/lib/dovecot/libdovecot.so.0(buffer_append+0x61) [0x7feef08f2a21] -> dovecot/replicator(replicator_queue_push+0x15a) [0x55953ece9b7a] -> dovecot/replicator(+0x62b6) [0x55953ece92b6] -> dovecot/replicator(+0x594e) [0x55953ece894e] -> dovecot/replicator(+0x5b2c) [0x55953ece8b2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7feef0913529] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) [0x7feef0914c12] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) [0x7feef09135d0] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7feef0913790] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7feef0886353] -> dovecot/replicator(main+0x18d) [0x55953ece7cbd] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7feef0651d0a] -> dovecot/replicator(_start+0x2a) [0x55953ece7d7a] Jun 03 09:39:03 Fatal: replicator: master: service(replicator): child 2532886 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps)
My configuration:
# 2.3.19 (b3ad6004dc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 5.10.0-13-amd64 x86_64 Debian 11.3 ext4 # Hostname: mail.myserv.local auth_mechanisms = plain login auth_verbose = yes debug_log_path = /var/log/dovecot.debug default_client_limit = 5000 default_process_limit = 1000 default_vsz_limit = 2 G deliver_log_format = msgid=%m, subject=%s, from=%f, size=%p(%w), %$ dict { quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } doveadm_password = # hidden, use -P to show it hostname = myserv.local lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot.log mail_home = /var/mail/vhosts/%d/mail/%n mail_location = maildir:/var/mail/vhosts/%d/mail/%n:INDEX=/var/mail/vhosts/%d/indexes/%n mail_plugins = " quota fts fts_solr notify replication" mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = Sent autocreate2 = Drafts autocreate3 = Junk autocreate4 = Trash autosubscribe = Sent autosubscribe2 = Drafts autosubscribe3 = Junk autosubscribe4 = Trash expire = Trash expire2 = Trash/* expire3 = Spam fts = solr fts_autoindex = yes fts_solr = url=http://localhost:8983/solr/dovecot/ imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * mail_replica = tcp:imap.myserv2.local:12345 quota = dict:Mailbox::proxy::quota quota_rule = *:storage=1G quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u replication_sync_timeout = 2 sieve = file:/var/mail/vhosts/%d/sieve/%n;active=/var/mail/vhosts/%d/sieve/%n/.dovecot.sieve
sieve_after = /etc/dovecot/sieve/after.d/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_quota_max_scripts = 50 } postmaster_address = postmaster@%d protocols = " imap lmtp sieve pop3" service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service dict { unix_listener dict { user = vmail } } service doveadm { inet_listener { port = 12345 ssl = no } } service imap-login { process_min_avail = 4 service_count = 1 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-warning { executable = script /var/lib/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_cert =
On 2022-06-04 02:46, Ivan Jurišić wrote:
Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram.
I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator proces got crash.
Maybe there is another service process causing OOM? e.g. check clamd, antivirus DBs tend to be quite big and in updating for sometime becomes double the size due to reloading.
Also, somtimes httpd service when using event worker, and its not tuned properly, it will cause the OOM crash to other service along itself.
Good luck.
Zakaria.
I think it is because of the domain, obviously you should only be running a website.
-----Original Message----- From: dovecot dovecot-bounces@dovecot.org On Behalf Of hi@zakaria.website Sent: Saturday, 4 June 2022 11:15 To: Dovecot dovecot@dovecot.org Subject: Re: Replicator: Panic: data stack: Out of memory
On 2022-06-04 02:46, Ivan Jurišić wrote:
Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram.
I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator proces got crash.
Maybe there is another service process causing OOM? e.g. check clamd, antivirus DBs tend to be quite big and in updating for sometime becomes double the size due to reloading.
Also, somtimes httpd service when using event worker, and its not tuned properly, it will cause the OOM crash to other service along itself.
just an fyi domain is registered and appears to be active so there should not be any issues with the domain
.website is an actual domain (like .com, .ca etc)
however i did note
ZAKARIA.WEBSITE. 14400 IN MX 10 ZAKARIA.WEBSITE.
usually the mx record points to an actual sub domain like mail. or whatever
if you are running everything on one server then this is ok
its just usually better to seperate the mx record incase you want to goto a different server down the road.
Domain Name: ZAKARIA.WEBSITE Registry Domain ID: D198561373-CNIC Registrar WHOIS Server: whois.ionos.com Registrar URL: https://ionos.com Updated Date: 2021-11-02T01:42:25.0Z Creation Date: 2020-08-29T09:28:59.0Z Registry Expiry Date: 2022-08-29T23:59:59.0Z Registrar: IONOS SE Registrar IANA ID: 83 Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited Registrant Organization: 1&1 Internet Limited Registrant State/Province: GLS Registrant Country: GB Registrant Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Admin Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Tech Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Name Server: NS1.ZAKARIA.WEBSITE Name Server: NS2.ZAKARIA.WEBSITE
## nslookup ZAKARIA.WEBSITE Server: 10.220.0.2 Address: 10.220.0.2#53
Non-authoritative answer: Name: ZAKARIA.WEBSITE Address: 213.171.210.111 Name: ZAKARIA.WEBSITE Address: 2a00:da00:1800:834c::1
## dig mx ZAKARIA.WEBSITE
; <<>> DiG 9.14.3 <<>> mx ZAKARIA.WEBSITE ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32110 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3
;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: ba2f2ec47dfcc90f4581111d629b4d2855567ad8dfa57bf8 (good) ;; QUESTION SECTION: ;ZAKARIA.WEBSITE. IN MX
;; ANSWER SECTION: ZAKARIA.WEBSITE. 14400 IN MX 10 ZAKARIA.WEBSITE.
;; ADDITIONAL SECTION: zakaria.website. 14372 IN A 213.171.210.111 zakaria.website. 14372 IN AAAA 2a00:da00:1800:834c::1
;; Query time: 87 msec ;; SERVER: 10.220.0.2#53(10.220.0.2) ;; WHEN: Sat Jun 04 08:16:40 EDT 2022 ;; MSG SIZE rcvd: 147
Happy Saturday !!! Thanks - paul
Paul Kudla
Scom.ca Internet Services http://www.scom.ca 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3
Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca
On 6/4/2022 6:07 AM, Marc wrote:
I think it is because of the domain, obviously you should only be running a website.
-----Original Message----- From: dovecot dovecot-bounces@dovecot.org On Behalf Of hi@zakaria.website Sent: Saturday, 4 June 2022 11:15 To: Dovecot dovecot@dovecot.org Subject: Re: Replicator: Panic: data stack: Out of memory
On 2022-06-04 02:46, Ivan Jurišić wrote:
Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram.
I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator proces got crash.
Maybe there is another service process causing OOM? e.g. check clamd, antivirus DBs tend to be quite big and in updating for sometime becomes double the size due to reloading.
Also, somtimes httpd service when using event worker, and its not tuned properly, it will cause the OOM crash to other service along itself.
:) You should not be responding to my post, as it was sort of off topic and only focused on the text '.website'
just an fyi domain is registered and appears to be active so there should not be any issues with the domain
.website is an actual domain (like .com, .ca etc)
however i did note
ZAKARIA.WEBSITE. 14400 IN MX 10 ZAKARIA.WEBSITE.
usually the mx record points to an actual sub domain like mail. or whatever
if you are running everything on one server then this is ok
its just usually better to seperate the mx record incase you want to goto a different server down the road.
Domain Name: ZAKARIA.WEBSITE Registry Domain ID: D198561373-CNIC Registrar WHOIS Server: whois.ionos.com Registrar URL: https://ionos.com Updated Date: 2021-11-02T01:42:25.0Z Creation Date: 2020-08-29T09:28:59.0Z Registry Expiry Date: 2022-08-29T23:59:59.0Z Registrar: IONOS SE Registrar IANA ID: 83 Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited Registrant Organization: 1&1 Internet Limited Registrant State/Province: GLS Registrant Country: GB Registrant Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Admin Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Tech Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Name Server: NS1.ZAKARIA.WEBSITE Name Server: NS2.ZAKARIA.WEBSITE
## nslookup ZAKARIA.WEBSITE Server: 10.220.0.2 Address: 10.220.0.2#53
Non-authoritative answer: Name: ZAKARIA.WEBSITE Address: 213.171.210.111 Name: ZAKARIA.WEBSITE Address: 2a00:da00:1800:834c::1
## dig mx ZAKARIA.WEBSITE
; <<>> DiG 9.14.3 <<>> mx ZAKARIA.WEBSITE ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32110 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3
;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: ba2f2ec47dfcc90f4581111d629b4d2855567ad8dfa57bf8 (good) ;; QUESTION SECTION: ;ZAKARIA.WEBSITE. IN MX
;; ANSWER SECTION: ZAKARIA.WEBSITE. 14400 IN MX 10 ZAKARIA.WEBSITE.
;; ADDITIONAL SECTION: zakaria.website. 14372 IN A 213.171.210.111 zakaria.website. 14372 IN AAAA 2a00:da00:1800:834c::1
;; Query time: 87 msec ;; SERVER: 10.220.0.2#53(10.220.0.2) ;; WHEN: Sat Jun 04 08:16:40 EDT 2022 ;; MSG SIZE rcvd: 147
Happy Saturday !!! Thanks - paul
Paul Kudla
Scom.ca Internet Services http://www.scom.ca 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3
Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca
On 6/4/2022 6:07 AM, Marc wrote:
I think it is because of the domain, obviously you should only be
running a website.
-----Original Message----- From: dovecot dovecot-bounces@dovecot.org On Behalf Of hi@zakaria.website Sent: Saturday, 4 June 2022 11:15 To: Dovecot dovecot@dovecot.org Subject: Re: Replicator: Panic: data stack: Out of memory
On 2022-06-04 02:46, Ivan Jurišić wrote:
Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram.
I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator
proces
got crash.
Maybe there is another service process causing OOM? e.g. check clamd, antivirus DBs tend to be quite big and in updating for sometime becomes double the size due to reloading.
Also, somtimes httpd service when using event worker, and its not tuned properly, it will cause the OOM crash to other service along itself.
On 2022-06-04 12:20, Paul Kudla (SCOM.CA Internet Services Inc.) wrote:
just an fyi domain is registered and appears to be active so there should not be any issues with the domain
.website is an actual domain (like .com, .ca etc)
however i did note
ZAKARIA.WEBSITE. 14400 IN MX 10 ZAKARIA.WEBSITE.
usually the mx record points to an actual sub domain like mail. or whatever
if you are running everything on one server then this is ok
its just usually better to seperate the mx record incase you want to goto a different server down the road.
Domain Name: ZAKARIA.WEBSITE Registry Domain ID: D198561373-CNIC Registrar WHOIS Server: whois.ionos.com Registrar URL: https://ionos.com Updated Date: 2021-11-02T01:42:25.0Z Creation Date: 2020-08-29T09:28:59.0Z Registry Expiry Date: 2022-08-29T23:59:59.0Z Registrar: IONOS SE Registrar IANA ID: 83 Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited Registrant Organization: 1&1 Internet Limited Registrant State/Province: GLS Registrant Country: GB Registrant Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Admin Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Tech Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Name Server: NS1.ZAKARIA.WEBSITE Name Server: NS2.ZAKARIA.WEBSITE
## nslookup ZAKARIA.WEBSITE Server: 10.220.0.2 Address: 10.220.0.2#53
Non-authoritative answer: Name: ZAKARIA.WEBSITE Address: 213.171.210.111 Name: ZAKARIA.WEBSITE Address: 2a00:da00:1800:834c::1
## dig mx ZAKARIA.WEBSITE
; <<>> DiG 9.14.3 <<>> mx ZAKARIA.WEBSITE ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32110 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3
;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: ba2f2ec47dfcc90f4581111d629b4d2855567ad8dfa57bf8 (good) ;; QUESTION SECTION: ;ZAKARIA.WEBSITE. IN MX
;; ANSWER SECTION: ZAKARIA.WEBSITE. 14400 IN MX 10 ZAKARIA.WEBSITE.
;; ADDITIONAL SECTION: zakaria.website. 14372 IN A 213.171.210.111 zakaria.website. 14372 IN AAAA 2a00:da00:1800:834c::1
;; Query time: 87 msec ;; SERVER: 10.220.0.2#53(10.220.0.2) ;; WHEN: Sat Jun 04 08:16:40 EDT 2022 ;; MSG SIZE rcvd: 147
Happy Saturday !!! Thanks - paul
Paul Kudla
Scom.ca Internet Services http://www.scom.ca 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3
Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca
On 6/4/2022 6:07 AM, Marc wrote:
I think it is because of the domain, obviously you should only be running a website.
-----Original Message----- From: dovecot dovecot-bounces@dovecot.org On Behalf Of hi@zakaria.website Sent: Saturday, 4 June 2022 11:15 To: Dovecot dovecot@dovecot.org Subject: Re: Replicator: Panic: data stack: Out of memory
On 2022-06-04 02:46, Ivan Jurišić wrote:
Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram.
I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator proces got crash.
Maybe there is another service process causing OOM? e.g. check clamd, antivirus DBs tend to be quite big and in updating for sometime becomes double the size due to reloading.
Also, somtimes httpd service when using event worker, and its not tuned properly, it will cause the OOM crash to other service along itself.
Hi Paul,
I couldnt get the context, and not sure if you've addressed me with confusion, since I didnt post the OOM issue, but its Ivan, anyhow thanks for the heads up.
I think the MX record its ok. I ran validation on internet.nl, as well as staging.hardenize.net and many other mail server validating services and none of them in terms of meeting standards, has picked anything wrong on the MX records, therefore I kept it as I thought was right. If I by any chance had the mail services running on another server, then indeed as you stated, invitably, I will have to point the MX record to it's domain and using unique domain or subdomain will turn as a requirement.
Also, another thought for Ivan about OOM, it seems that the memory leak sometimes can happen legitimately, if you have low resoucess, make sure your server didnt run out of resources including CPU cores, storage as well as RAM.
To check storage you can run: df -h.
Zakaria.
actually suggestion below is a good idea
run
ps -axww (or top)
to list active processes this will give you some hints
top is better for overall memory
i also have a perl script that will show actual memory useage, free etc
utilities like this are handy to have
also i found i had to set in dovecot.conf
default_process_limit = 16384
also are you running debug ?
auth_debug = no auth_debug_passwords = no
mail_debug = no
ie set debug to = yes?
might give more detail if this is really a dovecot issue.
other background processes can eat memory
I run mailscanner for example and someone every one in a while tries to crash it!
it recovers but lord knows
mem outputs :
# mem
SYSTEM MEMORY SUMMARY: mem_used: 16GB [ 12%] Logically used memory mem_avail: + 111GB [ 87%] Logically available memory
mem_total: = 128GB [100%] Logically total memory
SYSTEM MEMORY INFORMATION: mem_wire: 13GB [ 10%] Wired: disabled for paging out mem_active: + 0GB [ 0%] Active: recently referenced mem_inactive:+ 71GB [ 57%] Inactive: recently not referenced mem_cache: + 0GB [ 0%] Cached: almost avail. for allocation mem_free: + 40GB [ 32%] Free: fully available for allocation mem_gap_vm: + 0GB [ 0%] Memory gap: UNKNOWN
mem_all: = 124GB [100%] Total real memory managed mem_gap_sys: + 3GB Memory gap: Kernel?!
mem_phys: = 127GB Total real memory available mem_gap_hw: + 0GB Memory gap: Segment Mappings?!
mem_hw: = 128GB Total real memory installed
# cat /programs/common/mem #!/usr/local/bin/perl ## ## freebsd-memory -- List Total System Memory Usage ## Copyright (c) 2003-2004 Ralf S. Engelschall rse@engelschall.com ## ## Redistribution and use in source and binary forms, with or without ## modification, are permitted provided that the following conditions ## are met: ## 1. Redistributions of source code must retain the above copyright ## notice, this list of conditions and the following disclaimer. ## 2. Redistributions in binary form must reproduce the above copyright ## notice, this list of conditions and the following disclaimer in the ## documentation and/or other materials provided with the distribution. ## ## THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND ## ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE ## IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ## ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE ## FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ## DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS ## OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) ## HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT ## LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY ## OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF ## SUCH DAMAGE. ##
# query the system through the generic sysctl(8) interface
# (this does not require special priviledges)
my $sysctl = {};
my $sysctl_output = /sbin/sysctl -a
;
foreach my $line (split(/\n/, $sysctl_output)) {
if ($line =~ m/^([^:]+):\s+(.+)\s*$/s) {
$sysctl->{$1} = $2;
}
}
# round the physical memory size to the next power of two which is # reasonable for memory cards. We do this by first determining the # guessed memory card size under the assumption that usual computer # hardware has an average of a maximally eight memory cards installed # and those are usually of equal size. sub mem_rounded { my ($mem_size) = @_; my $chip_size = 1; my $chip_guess = ($mem_size / 8) - 1; while ($chip_guess != 0) { $chip_guess >>= 1; $chip_size <<= 1; } my $mem_round = (int($mem_size / $chip_size) + 1) * $chip_size; return $mem_round; }
# determine the individual known information # NOTICE: forget hw.usermem, it is just (hw.physmem - vm.stats.vm.v_wire_count). # NOTICE: forget vm.stats.misc.zero_page_count, it is just the subset of # vm.stats.vm.v_free_count which is already pre-zeroed. my $mem_hw = &mem_rounded($sysctl->{"hw.physmem"}); my $mem_phys = $sysctl->{"hw.physmem"}; my $mem_all = $sysctl->{"vm.stats.vm.v_page_count"} * $sysctl->{"hw.pagesize"}; my $mem_wire = $sysctl->{"vm.stats.vm.v_wire_count"} * $sysctl->{"hw.pagesize"}; my $mem_active = $sysctl->{"vm.stats.vm.v_active_count"} * $sysctl->{"hw.pagesize"}; my $mem_inactive = $sysctl->{"vm.stats.vm.v_inactive_count"} * $sysctl->{"hw.pagesize"}; my $mem_cache = $sysctl->{"vm.stats.vm.v_cache_count"} * $sysctl->{"hw.pagesize"}; my $mem_free = $sysctl->{"vm.stats.vm.v_free_count"} * $sysctl->{"hw.pagesize"};
# determine the individual unknown information my $mem_gap_vm = $mem_all - ($mem_wire + $mem_active + $mem_inactive
- $mem_cache + $mem_free); my $mem_gap_sys = $mem_phys - $mem_all; my $mem_gap_hw = $mem_hw - $mem_phys;
determine logical summary information
my $mem_total = $mem_hw; my $mem_avail = $mem_inactive + $mem_cache + $mem_free; my $mem_used = $mem_total - $mem_avail;
# information annotations my $info = { "mem_wire" => 'Wired: disabled for paging out', "mem_active" => 'Active: recently referenced', "mem_inactive" => 'Inactive: recently not referenced', "mem_cache" => 'Cached: almost avail. for allocation', "mem_free" => 'Free: fully available for allocation', "mem_gap_vm" => 'Memory gap: UNKNOWN', "mem_all" => 'Total real memory managed', "mem_gap_sys" => 'Memory gap: Kernel?!', "mem_phys" => 'Total real memory available', "mem_gap_hw" => 'Memory gap: Segment Mappings?!', "mem_hw" => 'Total real memory installed', "mem_used" => 'Logically used memory', "mem_avail" => 'Logically available memory', "mem_total" => 'Logically total memory', };
# print system results printf("\n");
printf("SYSTEM MEMORY SUMMARY:\n"); printf("mem_used: %7dGB [%3d%%] %s\n", $mem_used / (1024*1024*1024), ($mem_used / $mem_total) * 100, $info->{"mem_used"}); printf("mem_avail: + %7dGB [%3d%%] %s\n", $mem_avail / (1024*1024*1024), ($mem_avail / $mem_total) * 100, $info->{"mem_avail"}); printf("-------------- ------------ ----------- ------\n"); printf("mem_total: = %7dGB [100%%] %s\n", $mem_total / (1024*1024*1024), $info->{"mem_total"});
printf("\n");
printf("SYSTEM MEMORY INFORMATION:\n"); printf("mem_wire: %7dGB [%3d%%] %s\n", $mem_wire / (1024*1024*1024), ($mem_wire / $mem_all) * 100, $info->{"mem_wire"}); printf("mem_active: + %7dGB [%3d%%] %s\n", $mem_active / (1024*1024*1024), ($mem_active / $mem_all) * 100, $info->{"mem_active"}); printf("mem_inactive:+ %7dGB [%3d%%] %s\n", $mem_inactive / (1024*1024*1024), ($mem_inactive / $mem_all) * 100, $info->{"mem_inactive"}); printf("mem_cache: + %7dGB [%3d%%] %s\n", $mem_cache / (1024*1024*1024), ($mem_cache / $mem_all) * 100, $info->{"mem_cache"}); printf("mem_free: + %7dGB [%3d%%] %s\n", $mem_free / (1024*1024*1024), ($mem_free / $mem_all) * 100, $info->{"mem_free"}); printf("mem_gap_vm: + %7dGB [%3d%%] %s\n", $mem_gap_vm / (1024*1024*1024), ($mem_gap_vm / $mem_all) * 100, $info->{"mem_gap_vm"}); printf("-------------- ------------ ----------- ------\n"); printf("mem_all: = %7dGB [100%%] %s\n", $mem_all / (1024*1024*1024), $info->{"mem_all"}); printf("mem_gap_sys: + %7dGB %s\n", $mem_gap_sys / (1024*1024*1024), $info->{"mem_gap_sys"}); printf("-------------- ------------ -----------\n"); printf("mem_phys: = %7dGB %s\n", $mem_phys / (1024*1024*1024), $info->{"mem_phys"}); printf("mem_gap_hw: + %7dGB %s\n", $mem_gap_hw / (1024*1024*1024), $info->{"mem_gap_hw"}); printf("-------------- ------------ -----------\n"); printf("mem_hw: = %7dGB %s\n", $mem_hw / (1024*1024*1024), $info->{"mem_hw"});
# print logical results
top will display something like this ?
last pid: 85373; load averages: 0.71, 0.48, 0.38
up 72+04:51:41 08:22:11
207 processes: 1 running, 206 sleeping CPU: 1.5% user, 0.0% nice, 0.4% system, 0.0% interrupt, 98.0% idle Mem: 336M Active, 71G Inact, 139M Laundry, 13G Wired, 770M Buf, 40G Free ARC: 4319M Total, 1346M MFU, 501M MRU, 2368K Anon, 55M Header, 2414M Other 383M Compressed, 1469M Uncompressed, 3.83:1 Ratio Swap: 16G Total, 16G Free
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 1986 pgsql 1 26 0 195M 46M select 14 426:27 11.47% postgres 83810 pgsql 1 27 0 200M 171M select 13 3:15 9.19% postgres 1882 root 128 20 0 11M 3732K rpcsvc 15 29.7H 2.26% nfsd 1987 pgsql 1 20 0 195M 47M select 5 33:21 1.84% postgres 1985 root 34 21 0 141M 88M sigwai 14 72:22 1.32% named 1937 root 1 20 0 27M 15M select 15 491:36 0.90% python3.8 99555 root 1 20 0 28M 18M select 10 634:23 0.88% python3.8 1939 root 1 20 0 27M 15M select 1 939:47 0.87% python3.8 1988 pgsql 1 20 0 195M 47M select 7 6:58 0.28% postgres 1989 pgsql 1 20 0 195M 47M select 8 2:14 0.17% postgres 1964 pgsql 1 20 0 194M 164M select 9 10:02 0.08% postgres 85373 root 1 20 0 14M 3644K CPU0 0 0:00 0.07% top 3150 pgsql 1 20 0 195M 42M select 6 39:21 0.06% postgres
ps -axw or ps -axww or freebsd
# ps -axww PID TT STAT TIME COMMAND 0 - DLs 3788:48.94 [kernel] 1 - ILs 0:05.38 /sbin/init -- 2 - DL 0:00.00 [crypto] 3 - DL 0:00.00 [crypto returns 0] 4 - DL 0:00.00 [crypto returns 1] 5 - DL 0:00.00 [crypto returns 2] 6 - DL 0:00.00 [crypto returns 3] 7 - DL 0:00.00 [crypto returns 4] 8 - DL 0:00.00 [crypto returns 5] 9 - DL 0:00.00 [crypto returns 6] 10 - DL 0:00.00 [audit] 11 - RNL 1629112:33.34 [idle] 12 - WL 180:00.70 [intr] 13 - DL 123:57.70 [geom] 14 - DL 0:00.00 [crypto returns 7] 15 - DL 0:00.00 [crypto returns 8] 16 - DL 0:00.00 [crypto returns 9] 17 - DL 0:00.00 [crypto returns 10] 18 - DL 0:00.00 [crypto returns 11] 19 - DL 0:00.00 [crypto returns 12] 20 - DL 0:00.00 [crypto returns 13] 21 - DL 0:00.00 [crypto returns 14] 22 - DL 0:00.00 [crypto returns 15] 23 - DL 0:00.00 [sequencer 00] 24 - DL 0:00.00 [cam] 25 - DL 5:42.32 [usb] 26 - DL 0:00.47 [soaiod1] 27 - DL 0:00.47 [soaiod2] 28 - DL 0:00.46 [soaiod3] 29 - DL 0:00.47 [soaiod4] 30 - DL 1714:58.15 [zfskern] 31 - DL 0:00.00 [sctp_iterator] 32 - DL 12:50.77 [pf purge] 33 - DL 2:16.82 [rand_harvestq] 34 - DL 29:00.62 [pagedaemon] 35 - DL 0:00.00 [vmdaemon] 36 - DL 5:25.68 [bufdaemon] 37 - DL 1:44.98 [vnlru] 38 - DL 2040:33.82 [syncer] 1657 - Is 0:01.21 /sbin/devd 1863 - Ss 0:03.44 /usr/sbin/rpcbind 1878 - Is 0:00.08 /usr/sbin/mountd -r -S 1880 - Is 0:00.27 nfsd: master (nfsd) 1882 - S 1780:23.16 nfsd: server (nfsd) 1907 - Ss 10:01.06 /usr/sbin/syslogd -s 1909 - Is 0:00.55 /usr/sbin/inetd -wW -C 50 -s 500 1911 - Is 0:00.25 /usr/sbin/sshd 1955 - Is 24:50.70 /usr/local/sbin/clamd 1964 - Ss 10:02.28 postmaster: checkpointer (postgres) 1965 - Ss 1:38.52 postmaster: background writer (postgres) 1966 - Ss 3:48.60 postmaster: walwriter (postgres) 1967 - Ss 2:03.84 postmaster: autovacuum launcher (postgres) 1968 - Ss 12:41.60 postmaster: stats collector (postgres) 1969 - Is 0:01.82 postmaster: logical replication launcher (postgres) 1974 - Ss 37:19.26 postmaster: walsender pgsql 10.221.0.16(30421) (postgres) 1976 - Ss 39:37.29 postmaster: walsender pgsql 10.221.0.10(64872) (postgres) 1985 - Is 72:21.96 /usr/local/sbin/named -d 0 -4 1986 - Ss 426:29.15 postmaster: pgsql scom_billing 10.221.0.18(52852) (postgres) 1987 - Ss 33:21.50 postmaster: pgsql scom_billing 10.221.0.18(60830) (postgres) 1988 - Ss 6:57.70 postmaster: pgsql scom_billing 10.221.0.18(34255) (postgres) 1989 - Ss 2:13.52 postmaster: pgsql scom_billing 10.221.0.18(17265) (postgres) 2073 - Ss 10:12.46 /usr/local/libexec/postfix/master -w 2076 - I 0:07.82 qmgr -l -t fifo -u 2166 - Is 1:53.61 /usr/local/libexec/postfix/master -w 2168 - I 0:55.23 qmgr -l -t fifo -u 2238 - Is 1:49.77 /usr/local/libexec/postfix/master -w 2240 - I 1:01.17 qmgr -l -t fifo -u 2253 - I 0:39.34 tlsmgr -l -t unix -u 2397 - Is 0:05.58 MailScanner: starting child (perl) 2513 - Is 0:20.43 /usr/sbin/cron -s 3150 - Rs 39:21.01 postmaster: walsender pgsql 10.221.0.6(10000) (postgres) 3175 - Is 0:00.35 postmaster: pgsql scom_billing 10.221.0.6(10017) (postgres) 3176 - Is 0:10.80 postmaster: pgsql scom_billing 10.221.0.6(10018) (postgres) 3177 - Ss 1:10.22 postmaster: pgsql scom_billing 10.221.0.6(10019) (postgres)
Happy Saturday !!! Thanks - paul
Paul Kudla
Scom.ca Internet Services http://www.scom.ca 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3
Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca
On 6/4/2022 5:15 AM, dovecot-bounces@dovecot.org wrote:
On 2022-06-04 02:46, Ivan Jurišić wrote:
Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram.
I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator proces got crash.
Maybe there is another service process causing OOM? e.g. check clamd, antivirus DBs tend to be quite big and in updating for sometime becomes double the size due to reloading.
Also, somtimes httpd service when using event worker, and its not tuned properly, it will cause the OOM crash to other service along itself.
Good luck.
Zakaria.
Dana 04.06.2022 15:34, Paul Kudla (SCOM.CA Internet Services Inc.) je napisao(la): ok thanks for the info from here you need to turn on full debugging and then filter the log by "replicat"
Now replication work when set vsz_limit in service aggregator and remove parametar replication_dsync_parameters and replication_full_sync_interval from my 90-replicator.conf. Now my configuration work for replication on another mail server.
Config file for replication /etc/dovecot/conf.d/90-replicator.conf
service aggregator { vsz_limit = 256M fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } }
service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } }
service doveadm { inet_listener { port = 12345 ssl = no } }
replication_max_conns = 100 #replication_dsync_parameters = -d -N -l 30 -U #replication_full_sync_interval = 1 days
doveadm_port = 12345 doveadm_password = Jados82!
plugin { mail_replica = tcp:imap.myserv2.local:12345 }
Sry, I check my logs and this configuration of replication is not help.
Jun 06 12:05:40 doveadm(ivan@myserv.local)<2864367><O9xlAHTRnWLvtCsAeFl1HA>: Panic: data stack: Out of memory when allocating 134217768 bytes Jun 06 12:05:40 doveadm(ivan@myserv.local)<2864367><O9xlAHTRnWLvtCsAeFl1HA>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f722cdb6582] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f722cdb669e] -> /usr/lib/dovecot/libdovecot.so.0(+0x1022fb) [0x7f722cdc32fb] -> /usr/lib/dovecot/libdovecot.so.0(+0x102391) [0x7f722cdc3391] -> /usr/lib/dovecot/libdovecot.so.0(+0x55589) [0x7f722cd16589] -> /usr/lib/dovecot/libdovecot.so.0(+0x54d52) [0x7f722cd15d52] -> /usr/lib/dovecot/libdovecot.so.0(+0xfb808) [0x7f722cdbc808] -> /usr/lib/dovecot/libdovecot.so.0(t_strdup_until+0x21) [0x7f722cdf95c1] -> dovecot/doveadm-server(dsync_mailbox_tree_find_delete+0x1a5) [0x5622f20c6e05] -> dovecot/doveadm-server(+0x4fe05) [0x5622f20bbe05] -> dovecot/doveadm-server(dsync_brain_recv_mailbox_tree_deletes+0xd8) [0x5622f20bcd68] -> dovecot/doveadm-server(dsync_brain_run+0x1ce) [0x5622f20b938e] -> dovecot/doveadm-server(+0x4da85) [0x5622f20b9a85] -> dovecot/doveadm-server(+0x5f3cf) [0x5622f20cb3cf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f722cdd9529] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) [0x7f722cddac12] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) [0x7f722cdd95d0] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f722cdd9790] -> dovecot/doveadm-server(+0x31195) [0x5622f209d195] -> dovecot/doveadm-server(+0x33a05) [0x5622f209fa05] -> dovecot/doveadm-server(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x2ca) [0x5622f20a0aba] -> dovecot/doveadm-server(doveadm_cmd_run_ver2+0x501) [0x5622f20b1161] -> dovecot/doveadm-server(+0x495aa) [0x5622f20b55aa] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f722cdd9529] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) [0x7f722cddac12] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) [0x7f722cdd95d0] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f722cdd9790] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f722cd4c353] -> dovecot/doveadm-server(main+0x102) [0x5622f208ffc2] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f722c9ded0a]
Dana 06.06.2022 10:03, Ivan Jurišić je napisao(la):
Dana 04.06.2022 15:34, Paul Kudla (SCOM.CA Internet Services Inc.) je napisao(la): ok thanks for the info from here you need to turn on full debugging and then filter the log by "replicat"
Now replication work when set vsz_limit in service aggregator and remove parametar replication_dsync_parameters and replication_full_sync_interval from my 90-replicator.conf. Now my configuration work for replication on another mail server.
Config file for replication /etc/dovecot/conf.d/90-replicator.conf
service aggregator { vsz_limit = 256M fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } }
service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } }
service doveadm { inet_listener { port = 12345 ssl = no } }
replication_max_conns = 100 #replication_dsync_parameters = -d -N -l 30 -U #replication_full_sync_interval = 1 days
doveadm_port = 12345 doveadm_password = Jados82!
plugin { mail_replica = tcp:imap.myserv2.local:12345 }
did you get this figured out / working
getting posts out of order - just wanted to make sure your ok?
Happy Monday !!! Thanks - paul
Paul Kudla
Scom.ca Internet Services http://www.scom.ca 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3
Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca
On 6/6/2022 4:03 AM, Ivan Jurišić wrote:
Dana 04.06.2022 15:34, Paul Kudla (SCOM.CA Internet Services Inc.) je napisao(la): ok thanks for the info from here you need to turn on full debugging and then filter the log by "replicat"
Now replication work when set vsz_limit in service aggregator and remove parametar replication_dsync_parameters and replication_full_sync_interval from my 90-replicator.conf. Now my configuration work for replication on another mail server.
Config file for replication /etc/dovecot/conf.d/90-replicator.conf
service aggregator { vsz_limit = 256M fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } }
service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } }
service doveadm { inet_listener { port = 12345 ssl = no } }
replication_max_conns = 100 #replication_dsync_parameters = -d -N -l 30 -U #replication_full_sync_interval = 1 days
doveadm_port = 12345 doveadm_password = Jados82!
plugin { mail_replica = tcp:imap.myserv2.local:12345 }
Dana 06.06.2022 14:14, Paul Kudla (SCOM.CA Internet Services Inc.) je napisao(la):
did you get this figured out / working getting posts out of order - just wanted to make sure your ok?
Replication not work, now have problem with "data stack", error is:
Jun 06 15:49:32
doveadm(ivan@myserv.local)<79253>
I have lots free memory and not sure what is problem?
# free -m
total used free shared buff/cache
available
Mem: 7956 2685 4966 26 305
5151
Swap: 7629 0 7629
ok thanks for the info
from here you need to turn on full debugging and then filter the log by
"replicat"
It is starting to look like you have a bad email somewhere causing the crash
I have yet to see this however anything is possible, when you replicate anything it needs to update indexes & control files on the other side and vice versa
that being said its a good place to start looking?
At least see in the debug mail logs what it is trying to replicate and where it fails.
Im running a large email server and in 2.3.18 there was a bug that any account over 300+ folders would fail to replicate.
This appears to be fixed in 2.3.19
other then that TCP replication etc seems to work the best
if you look back on the mailing lists i have uploaded a sample config of my systems which seems to work ok.
Happy Saturday !!! Thanks - paul
Paul Kudla
Scom.ca Internet Services http://www.scom.ca 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3
Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca
On 6/3/2022 10:46 PM, Ivan Jurišić wrote:
Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram.
I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator proces got crash.
-- This message has been scanned for viruses and dangerous content by *MailScanner* http://www.mailscanner.info/, and is believed to be clean.
participants (4)
-
hi@zakaria.website
-
Ivan Jurišić
-
Marc
-
Paul Kudla (SCOM.CA Internet Services Inc.)