Unseen field reported by imap status command returns wrong count for shared mailboxes on dovecot cluster

Wed Oct 19 06:17:58 UTC 2022

Hi!

This has now been fixed with https://github.com/dovecot/core/commit/41cc1bd9538b68fa874f9e7a3cd02cbc2cd33f09.patch

Aki

> On 11/10/2022 10:21 EEST Aki Tuomi <aki.tuomi at open-xchange.com> wrote:
> 
>  
> Hi!
> 
> This seems to be a bug in imapc client, we'll look into this. Thank you for reporting this issue. It's currently tracked as DOV-5579.
> 
> Aki
> 
> > On 07/10/2022 15:38 EEST Nikolaos Pyrgiotis <npyrgiotis at modulus.gr> wrote:
> > 
> > 
> > Hello, 
> > 
> > I want to make a correction on my first post. We are using version 2.3.19.1 for all our dovecot servers. Is there any update regarding this issue? I would be happy to provide more details if necessary.
> > 
> > Best Regards,
> > Nikos Pyrgiotis
> > 
> > On 8/9/22 17:47, Nikolaos Pyrgiotis wrote:
> > 
> > > Hello,
> > > We recently migrated our mail server to a dovecot cluster of5nodes, a dovecot proxy,2directors and2dovecot backends.
> > > All dovecot nodes run version2.19.1. We use a glusterfs mounted volume on the backendsforthe mail storage.
> > > We noticed that when issuing the IMAP command to checkforUNSEEN messagesformails on the shared namespace instead of seeing the value of the unseen messages of the user that the mailbox has been shared to,
> > > the value of the unseen messages of the owner of the mailbox is returned.
> > > This behavior causes thunderbird when getting new messagesforshared mailboxes, to show briefly all messages of the mailbox as unseenforthe user before showing the correct unseen countforthe shared mailbox.
> > > The mail location of the shared mailboxes is defined with imapc storage type as describedinthe documentation.
> > > Is this a dovecot bug? Can we configure a different dovecot configuration setting so that imap status command reports the correct unseen field count when mail location is an imapc storage location?
> > > 
> > > An examples is given below when running doveadm command from one the 2 dovecot directors:
> > > When issuing the status command we see that the unseen count is2:
> > > root at doved0-rmt0-cn1:/etc/cron.d# doveadm mailbox status -u npyrgiotis all shared.sysadmins doveadm(npyrgiotis): Info: remote(10.101.0.71:8080): doveadm(npyrgiotis at domie02.com)<19078><9l8IJTb+GWOGSgAAEU9A+w>: imapc(10.101.0.75:143): Connected to 10.101.0.75:143 (local 10.101.0.71:33476) shared.sysadmins messages=2 recent=0 uidnext=3 uidvalidity=1662640492 unseen=2 highestmodseq=3 vsize=3950 guid=c92f64f79f0d1ed01e6d5b314f04886c firstsaved=1662643853
> > > shared.sysadmins messages=2recent=0uidnext=3uidvalidity=1662640492unseen=2highestmodseq=3vsize=3950guid=c92f64f79f0d1ed01e6d5b314f04886c firstsaved=1662643853
> > > 
> > > But when fetching the emails of the mailboxes we can see that the \Seen flag is set for both emails
> > > 
> > > root at doved0-rmt0-cn1:/etc/cron.d# doveadm fetch -u npyrgiotis flags mailbox shared.sysadmins ALL doveadm(npyrgiotis): Info: remote(10.101.0.71:8080): doveadm(npyrgiotis at domie02.com)<19074><e7rWDgj+GWOCSgAAEU9A+w>: imapc(10.101.0.75:143): Connected to 10.101.0.75:143 (local 10.101.0.71:38750) flags: \Seen flags: \Seen 
> > > Below i post the dovecot configuration of one of the two dovecot backends:
> > > #2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf
> > > # Pigeonhole version0.5.19 (4eae2f79)
> > > # OS: Linux5.10.0-17-amd64 x86_64 Debian11.4
> > > # Hostname: doveb0-rmt0-cn1
> > > auth_cache_negative_ttl =5mins
> > > auth_cache_size =50M
> > > auth_debug = yes
> > > auth_default_realm = example.com
> > > auth_master_user_separator = *
> > > auth_mechanisms = plain login
> > > auth_verbose = yes
> > > auth_worker_max_count =16
> > > disable_plaintext_auth = no
> > > first_valid_uid =499
> > > hostname = smtp.example.com
> > > imapc_features = fetch-bodystructure fetch-headers rfc822.size search modseq acl delay-login
> > > imapc_host =10.101.0.75
> > > imapc_password = # hidden, use -P to show it
> > > imapc_sasl_mechanisms = plain login
> > > imapc_ssl = starttls
> > > imapc_ssl_verify = no
> > > last_valid_uid =499
> > > lda_mailbox_autocreate = yes
> > > lda_mailbox_autosubscribe = yes
> > > login_greeting = You have successfully loggedinto example.com IMAP server
> > > login_trusted_networks =10.101.0.7310.101.0.74
> > > mail_always_cache_fields = flags hdr.* date.received date.sent
> > > mail_cache_fields = flags date.received guid size.physical size.virtual imap.bodystructure body.snippet
> > > mail_debug = yes
> > > mail_fsync = always
> > > mail_plugins =" notify mail_log zlib acl"
> > > mail_privileged_group = mail
> > > managesieve_notify_capability = mailto
> > > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext
> > > mdbox_rotate_size =200M
> > > mmap_disable = yes
> > > namespace example {
> > > list = children
> > > location = imapc:~/shared/%%n:INDEXPVT=~/shared-pvt/%%n
> > > prefix = shared.%%n.
> > > separator = .
> > > subscriptions = no
> > > type = shared
> > > }
> > > namespace inbox {
> > > inbox = yes
> > > location =
> > > mailbox Drafts {
> > > special_use = \Drafts
> > > }
> > > mailbox Junk {
> > > special_use = \Junk
> > > }
> > > mailbox Sent {
> > > special_use = \Sent
> > > }
> > > mailbox"Sent Messages"{
> > > special_use = \Sent
> > > }
> > > mailbox Spam.Virus {
> > > auto = subscribe
> > > }
> > > mailbox Spam {
> > > auto = subscribe
> > > }
> > > mailbox Trash {
> > > special_use = \Trash
> > > }
> > > prefix =
> > > separator = .
> > > subscriptions = yes
> > > type = private
> > > }
> > > passdb {
> > > args = password=#hidden_use-P_to_show#
> > > driver = static
> > > master = yes
> > > override_fields = allow_real_nets=127.0.0.0/8,local,10.101.0.73,10.101.0.74 userdb_acl_user=%{auth_user} userdb_namespace/example/disabled=yes
> > > result_success = continue
> > > }
> > > passdb {
> > > args = password=#hidden_use-P_to_show#
> > > driver = static
> > > override_fields = allow_real_nets=127.0.0.0/8,local,10.101.0.73,10.101.0.74
> > > result_success = continue
> > > }
> > > passdb {
> > > args = /etc/dovecot/dovecot-ldap-users.conf.ext
> > > driver = ldap
> > > }
> > > passdb {
> > > args = /etc/dovecot/dovecot-ldap-groups.conf.ext
> > > driver = ldap
> > > }
> > > plugin {
> > > acl = vfile:/etc/dovecot/global-acls/%d/master-dovecot-acl:cache_secs=300
> > > acl_defaults_from_inbox = yes
> > > acl_globals_only = no
> > > acl_shared_dict = file:/var/dovecot-acl/%d/shared-mailboxes.db
> > > imap_compress_deflate_level =6
> > > mail_log_events = flag_change
> > > mail_log_fields = uid box flags
> > > sieve = file:~/sieve;active=~/sieve/%n.sieve
> > > sieve_before = /var/vmail/global-sieve/global.sieve
> > > sieve_env_email_address = %u
> > > sieve_env_mailbox_prefix = %{userdb:mailbox_prefix}
> > > sieve_global = /var/vmail/sieve
> > > sieve_global_extensions = +editheader +vnd.dovecot.environment
> > > zlib_save = zstd
> > > zlib_save_level =5
> > > }
> > > protocols =" imap lmtp sieve submission"
> > > service auth-worker {
> > > user = dovecot
> > > }
> > > service auth {
> > > unix_listener auth-userdb {
> > > group = mail
> > > mode =0666
> > > user = vmail
> > > }
> > > }
> > > service doveadm {
> > > inet_listener dovadm-dom-mgmt {
> > > address =10.100.1.71
> > > port =8080
> > > ssl = no
> > > }
> > > inet_listener dovadm-dom {
> > > address =10.101.0.71
> > > port =8080
> > > ssl = no
> > > }
> > > vsz_limit =256M
> > > }
> > > service imap-login {
> > > inet_listener imap-dom {
> > > address =10.101.0.71
> > > port =143
> > > ssl = no
> > > }
> > > inet_listener imap {
> > > port =0
> > > }
> > > inet_listener imaps-dom {
> > > address =10.101.0.71
> > > port =993
> > > ssl = yes
> > > }
> > > inet_listener imaps {
> > > port =0
> > > }
> > > process_limit =400
> > > process_min_avail =2
> > > service_count =20
> > > vsz_limit =512M
> > > }
> > > service imap {
> > > process_limit =2048
> > > process_min_avail =0
> > > service_count =1
> > > vsz_limit =4G
> > > }
> > > service lmtp {
> > > inet_listener lmtp-dom {
> > > address =10.101.0.71
> > > port =24
> > > ssl = no
> > > }
> > > process_limit =0
> > > process_min_avail =0
> > > service_count =0
> > > unix_listener lmtp {
> > > mode =0666
> > > }
> > > vsz_limit =256M
> > > }
> > > service managesieve-login {
> > > inet_listener sieve-dom {
> > > address =10.101.0.71
> > > port =4190
> > > ssl = no
> > > }
> > > inet_listener sieve {
> > > port =0
> > > }
> > > process_limit =0
> > > process_min_avail =0
> > > service_count =0
> > > vsz_limit =64M
> > > }
> > > service managesieve {
> > > process_limit =1024
> > > process_min_avail =0
> > > service_count =1
> > > vsz_limit =256M
> > > }
> > > service submission-login {
> > > inet_listener submission-dom {
> > > address =10.101.0.71
> > > port =587
> > > ssl = no
> > > }
> > > inet_listener submission {
> > > port =0
> > > }
> > > inet_listener submissions-dom {
> > > address =10.101.0.71
> > > port =465
> > > ssl = yes
> > > }
> > > process_limit =250
> > > process_min_avail =2
> > > service_count =1
> > > vsz_limit =256M
> > > }
> > > service submission {
> > > process_limit =1024
> > > process_min_avail =0
> > > service_count =1
> > > vsz_limit =256M
> > > }
> > > ssl = required
> > > ssl_cert = </etc/ssl/certs/doveb0-rmt0-cn1.mail.rmt.example.com.pem
> > > ssl_client_ca_dir = /etc/ssl/certs
> > > ssl_key = # hidden, use -P to show it
> > > submission_max_mail_size =30000k
> > > submission_relay_host =10.101.0.7
> > > submission_relay_port =587
> > > submission_relay_ssl = starttls
> > > submission_relay_ssl_verify = no
> > > submission_relay_trusted = yes
> > > userdb {
> > > driver = prefetch
> > > }
> > > userdb {
> > > args = /etc/dovecot/dovecot-ldap-users.conf.ext
> > > driver = ldap
> > > }
> > > userdb {
> > > args = /etc/dovecot/dovecot-ldap-groups.conf.ext
> > > driver = ldap
> > > }
> > > protocol lmtp {
> > > mail_plugins =" notify mail_log zlib acl sieve acl"
> > > }
> > > protocol lda {
> > > mail_plugins =" notify mail_log zlib acl sieve acl"
> > > }
> > > protocol imap {
> > > mail_max_userip_connections =100
> > > mail_plugins =" notify mail_log zlib acl"
> > > }
> > > protocol sieve {
> > > mail_max_userip_connections =40
> > > }
> > > protocol submission {
> > > mail_max_userip_connections =70
> > > }
> > > local10.101.0.71 {
> > > doveadm_allowed_commands =
> > > doveadm_password = # hidden, use -P to show it
> > > }
> > > local10.100.1.71 {
> > > doveadm_allowed_commands =
> > > doveadm_password = # hidden, use -P to show it
> > > }
> > > Thank you,
> > > Nikolaos Pyrgiotis