Dovecot/K9/Thunderbird IMAP interaction issues: replicated messages, constant reloading (TB) & missing new messages (K-9)
David Gessel
gessel at blackrosetech.com
Tue Jul 15 12:08:31 UTC 2014
-------- Original Message --------
Subject: Re: Dovecot/K9/Thunderbird IMAP interaction issues: replicated
messages, constant reloading (TB) & missing new messages (K-9)
From: Alex JOST <jost+lists at dimejo.at>
To: dovecot at dovecot.org
Date: Tue Jul 15 2014 13:30:31 GMT+0300 (Arabic Standard Time)
> Am 15.07.2014 03:12, schrieb David Gessel:
>> * In thunderbird, I get constant downloading of messages. Every time I
>> switch to the inbox from another folder, the messages start
>> redownloading. This sometimes, but not always, seems to create
>> duplicate messages, but I'm now fairly sure they are only created in the
>> client and not on the server. Searching through various forums, it
>> appears this has been a problem for Thunderbird for some time, but it
>> wasn't for me on Courier. As posts about similar errors seem to have
>> petered out over the last year without any clear guidance on how to fix
>> it that I could find, I expect there's a simple fix I just haven't found
>> yet, either in TB configuration or in Dovecot. Any hints? I tried
>> Thunderfix to remove the .msf files and will be redownloading my mail
>> over the thin pipe out to Iraq for the next 6 months (DOH!). One
>> artifact is that when I restart thunderbird, my inbox is reported as
>> having about 130 messages in it, then thunderbird starts redownloading
>> headers and eventually gets back up to the 1297 currently listed both in
>> the TB pane and by ls | wc -l in /cur. Is it possible there is a time
>> out that needs to be extended? In TB, I've set (among other less
>> apparently relevant options)
>> mailnews.tcptimeout;1000
>> mail.db.idle_limit;30000000
>
> How old is your Thunderbird profile. Have you tried creating a new one
> and see if the problem persists?
It is a new profile created with the transition from Cyrus to Dovecot
about a week ago, but I used IMAPSync to move all my mail from the
ancient Cyrus server to the shiny new Dovecot server. I then deleted
the old profile.
>
> Thunderfix should not be needed by the way. Just right click on the
> folder => Properties => Repair
I did see that, but first thought there was something magic about
"thunderfix" then decided to use it anyway because I have annual
sent/recieved archives from 2000 (and one for pre-2000) so that would
have been a lotta right-clicking.
>
>
>> server # dovecot -n
>> # 2.2.13: /usr/local/etc/dovecot/dovecot.conf
>> # OS: FreeBSD 9.2-RELEASE amd64
>> auth_default_realm = mydomain.com
>> auth_mechanisms = plain login
>> auth_realms = mydomain.com mydomain2.org
>> disable_plaintext_auth = no
>> first_valid_gid = 5000
>> first_valid_uid = 5000
>> hostname = mydomain.com
>> imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags
>> imap_idle_notify_interval = 29 mins
>> last_valid_gid = 5000
>> last_valid_uid = 5000
>> lda_mailbox_autocreate = yes
>> listen = *
>> mail_gid = 5000
>> mail_location = maildir:~/:CONTROL=/var/no-quota/%u
>> mail_plugins = quota mail_log notify
>> mail_temp_dir = /var/tmp
>> mail_uid = 5000
>> maildir_broken_filename_sizes = yes
>> managesieve_notify_capability = mailto
>> managesieve_sieve_capability = fileinto reject envelope
>> encoded-character vacation subaddress comparator-i;ascii-numeric
>> relational regex imap4flags copy include variables body enotify
>> environment mailbox date ihave duplicate spamtest spamtestplus editheader
>> mbox_lazy_writes = no
>> namespace inbox {
>> inbox = yes
>> location =
>> mailbox Drafts {
>> special_use = \Drafts
>> }
>> mailbox Junk {
>> special_use = \Junk
>> }
>> mailbox Sent {
>> special_use = \Sent
>> }
>> mailbox "Sent Messages" {
>> special_use = \Sent
>> }
>> mailbox Trash {
>> special_use = \Trash
>> }
>> prefix =
>> }
>> passdb {
>> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
>> driver = sql
>> }
>> plugin {
>> expire = Trash
>> mail_log_events = delete undelete expunge copy mailbox_delete
>> mailbox_rename
>> mail_log_fields = uid box msgid size
>> quota = maildir:User quota
>> quota_exceeded_message = Storage quota for this account has been
>> exceeded, please try again later.
>> quota_rule = *:storage=50G
>> quota_rule2 = Trash:storage=+30%%
>> quota_rule3 = Sent:storage=+30%%
>> quota_warning = storage=90%% quota-warning 90 %u
>> quota_warning2 = storage=75%% quota-warning 75 %u
>> sieve = ~/.dovecot.sieve
>> sieve_before = /usr/local/etc/dovecot/sieve/
>> sieve_dir = ~/sieve
>> sieve_extensions = +spamtest +spamtestplus +relational
>> +comparator-i;ascii-numeric +editheader
>> }
>> postmaster_address = gessel at mydomain.com
>> protocols = imap lmtp sieve sieve
>> sendmail_path = /usr/local/sbin/sendmail
>> service auth {
>> unix_listener /var/spool/postfix/private/auth {
>> group = postfix
>> mode = 0666
>> user = postfix
>> }
>> unix_listener auth-userdb {
>> group = vmail
>> mode = 0600
>> user = vmail
>> }
>> }
>> service managesieve-login {
>> inet_listener sieve {
>> port = 4190
>> }
>> service_count = 1
>> vsz_limit = 128 M
>> }
>> service quota-warning {
>> executable = script /usr/local/bin/quota-warning.sh
>> unix_listener quota-warning {
>> user = vmail
>> }
>> user = vmail
>> }
>> ssl = required
>> ssl_cert = </usr/local/etc/ca/server-cert.pem
>> ssl_cipher_list =
>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES256-SHA:!SSLv2
>>
>> ssl_key = </usr/local/etc/ca/server-unencrypted-key.pem
>> ssl_prefer_server_ciphers = yes
>> userdb {
>> driver = prefetch
>> }
>> userdb {
>> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
>> driver = sql
>> }
>> protocol lda {
>> mail_plugins = quota mail_log notify sieve
>> }
>> protocol imap {
>> mail_max_userip_connections = 20
>> mail_plugins = quota mail_log notify imap_quota imap_zlib
>> }
>> protocol lmtp {
>> mail_plugins = quota mail_log notify sieve
>> }
>>
>
> I don't know if it makes a difference for you as you are not using
> "LAYOUT=fs" but try to disable those imap_client_workarounds.
>
Good catch, thank you. (I used LAYOUT=fs at first, but found behavior a
little anomalous. I forgot to disable the workarounds).
This may have stopped the problem of redownloading the entire mailbox
(on restart I got downloading 1 of 8 instead of 1 of 170... 30 of 1330
in the status pane) . My other mailboxes still need to sync and it
might take a few days to verify that things are stable. I did get 81
immediate "Warning: Maildir /mail/mydomain.com/gessel//.Bulkmail:
Expunged message reappeared, giving a new UID" warnings on connect, but
if this is the problem, it might stop after all the directories are
resynced.
I'll update with further data as it comes in.
I am also somewhat suspect of QRSYNC from this post:
http://dovecot.org/pipermail/dovecot/2011-August/130687.html - though
this is from a long time ago and it seems a little surprising that bugs
on this seemingly useful capability would persist. In any event (if
useful information), my capability enumeration reads:
# telnet localhost 143
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
a login user pass
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN
NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH
ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY
MOVE COMPRESS=DEFLATE QUOTA] Logged in
I did try setting mail.server.default.use_condstore;false in
Thunderbird, but that didn't change the behavior of reloading the entire
inbox on startup so I've changed that back as that wasn't the desired
result.
More information about the dovecot
mailing list