Very slow mail download/notification with dovecot 2.3.7 and Thunderbird
Hi!
I know that there are many results if I search for "dovecot thunderbird very slow" on Google but none of them helped me with my problem. :( (I already posted this on February 2020, but forgot to answer. O.o)
I am running Ubuntu 20.04 (4GB RAM, 40GB HDD with EXT4) server with the following software:
Dovecot 2.3.7.2 ---- All accounts are using Maildir ---- Sieve filters
Postfix 3.4.13 ---- with opendkim, spamassassin, razor, pyzor, ....
And as IMAP client I am using Thunderbird 78.6 on Windows 10. Sieve is used to sort some incoming mails into my 228 folders (56 INBOX. folders and its subfolders (output of "find .INBOX.* -maxdepth 4 -type d -print |wc -l")). The total size of my Maildir directory is 5.5GB.
My problem is that it always takes very very long to synchronize the INBOX and its subfolders. Even if there are only a few new messages to be downloaded. Sometimes Thunderbird outputs something like "downloading message 1 of 19" and it takes at least 60 seconds to download all of them. None of these mails is larger than 5kB. And sometimes it seems that the transfer is suspended for a long time and it takes >5 Minutes to synchronize only a few mails. :(
And it seems that some folders are never updated in TB. In example I sort all messages from this list into a "INBOX.Mailinglists.ML-Dovecot" subfolder by using a simple sieve rule. But even after 10 minutes TB thinks that there are no new messages. As soon as I click on the ML-Dovecot folder I see in the /var/log/dovecot-info.log file that TB logs on to the server and then displays hundreds, of new messages. I checked twice but I have subscribed to all 228 folders.
How can I find the bottleneck? Is there a test tool available that simulates a simple mail client and can analyse an IMAP server?
This is the output of "dovecot -n":
=============================== # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-54-generic x86_64 Ubuntu 20.04.1 LTS # Hostname: mail.XXXXXXX.de auth_mechanisms = plain login debug_log_path = /var/log/dovecot-debug.log default_client_limit = 15653 default_process_limit = 5200 first_valid_uid = 150 info_log_path = /var/log/dovecot-info.log last_valid_uid = 150 lda_original_recipient_header = X-Original-To log_path = /var/log/dovecot-error.log login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_gid = mail mail_home = /var/vmail/%d/%n mail_location = maildir:~/Maildir mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Archive { special_use = \Archive } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_default = /var/vmail/global.sieve sieve_dir = ~/sieve } postmaster_address = thorsten@MY_DOMAIN.TLD protocols = " imap sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = mail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 50 process_min_avail = 2 service_count = 20 } service imap { process_limit = 1024 } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 0 } ssl_cert = was automatically rejected:%n%r } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep imap_idle_notify_interval = 10 mins mail_max_userip_connections = 100 mail_plugins = } protocol sieve { info_log_path = /var/log/dovecot-sieve.log mail_max_userip_connections = 100 managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_line_length = 65536
Output of "free -h":
free -h total used free shared buff/cache available Mem: 3.8Gi 1.7Gi 1.1Gi 10Mi 958Mi 1.7Gi Swap: 1.0Gi 375Mi 648Mi
Thank you very much! Thorsten
On 27/12/2020 15:11, ml_dovecot@thorsten-reichelt.de wrote:
And it seems that some folders are never updated in TB. In example I sort all messages from this list into a "INBOX.Mailinglists.ML-Dovecot" subfolder by using a simple sieve rule. But even after 10 minutes TB thinks that there are no new messages. As soon as I click on the ML-Dovecot folder I see in the /var/log/dovecot-info.log file that TB logs on to the server and then displays hundreds, of new messages. I checked twice but I have subscribed to all 228 folders.
I can't help you with your performance problem, but for Thunderbird to check all folders, you need to set "mail.check_all_imap_folders_for_new" to 'true' in the Thunderbird config editor.
Regards, Juri
Am 27.12.2020 um 16:11 schrieb Juri Haberland:
I can't help you with your performance problem, but for Thunderbird to check all folders, you need to set "mail.check_all_imap_folders_for_new" to 'true' in the Thunderbird config editor.
I think the setting is: mail.server.default.check_all_folders_for_new;true
Gruß Matthias
--
"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." -- Rich Cook
On 28/12/2020 09:44, Matthias Fechner wrote:
Am 27.12.2020 um 16:11 schrieb Juri Haberland:
I can't help you with your performance problem, but for Thunderbird to check all folders, you need to set "mail.check_all_imap_folders_for_new" to 'true' in the Thunderbird config editor.
I think the setting is: mail.server.default.check_all_folders_for_new;true
Both settings exist, but you are right, my setting is deprecated:
From http://kb.mozillazine.org/Checking_for_new_messages_in_other_folders_%28Thun...:
Thunderbird used to support setting mail.check_all_imap_folders_for_new to true to make it check every remote folder for new mail. The downside was there is no way to exclude a specific folder (such as a junk mail folder). However, that setting was replaced in version 5.0 with server-specific ones. Set mail.server.default.check_all_folders_for_new to true instead to make it effective for all accounts using the Config Editor.
Regards, Juri
On 28/12/2020 13:15 Juri Haberland juri@koschikode.com wrote:
On 28/12/2020 09:44, Matthias Fechner wrote:
Am 27.12.2020 um 16:11 schrieb Juri Haberland:
I can't help you with your performance problem, but for Thunderbird to check all folders, you need to set "mail.check_all_imap_folders_for_new" to 'true' in the Thunderbird config editor.
I think the setting is: mail.server.default.check_all_folders_for_new;true
Both settings exist, but you are right, my setting is deprecated:
From http://kb.mozillazine.org/Checking_for_new_messages_in_other_folders_%28Thun...:
Thunderbird used to support setting mail.check_all_imap_folders_for_new to true to make it check every remote folder for new mail. The downside was there is no way to exclude a specific folder (such as a junk mail folder). However, that setting was replaced in version 5.0 with server-specific ones. Set mail.server.default.check_all_folders_for_new to true instead to make it effective for all accounts using the Config Editor.
Regards, Juri
Try adding mail_debug=yes
to your dovecot config and check logs
Aki
Hi!
Thank you very much! This parameter (check_all_folders_for_new) was actually the solution to my problem that not all new mails were reported.
Regards Thorsten
On 28/12/2020 09:44, Matthias Fechner wrote:
Am 27.12.2020 um 16:11 schrieb Juri Haberland:
I can't help you with your performance problem, but for Thunderbird to check all folders, you need to set "mail.check_all_imap_folders_for_new" to 'true' in the Thunderbird config editor. I think the setting is: mail.server.default.check_all_folders_for_new;true Both settings exist, but you are right, my setting is deprecated:
participants (5)
-
Aki Tuomi
-
Juri Haberland
-
Matthias Fechner
-
ml_dovecot@thorsten-reichelt.de
-
Thorsten Reichelt