August 2019 - dovecot

flags not synced correctly with dovecot sync (dsync)
by Dan Christensen 31 Aug '19

31 Aug '19

I'm running dovecot 2.3.4.1 from https://repo.dovecot.org/ on Ubuntu 18.04 on three machines that I'll call server, laptop1 and laptop2. Both laptop1 and laptop2 run dovecot sync against server to keep local copies of my imap folders. Even when I initially had only two machines, laptop1 and server, I occasionally noticed that flags were lost, usually custom flags used by Gnus, but I couldn't reliably reproduce the problem. Now that I have two laptops syncing against the server, the problem has gotten worse and I figured out a way to reproduce it: - on server: create new IMAP folder test, and put two read messages in it - on laptop1: doveadm sync -u user -l 10 -m test -f user@server - on laptop2: doveadm sync -u user -l 10 -m test -f user@server At this point, all three machines show the two messages M1 and M2 as being read. - on laptop1: mark message M1 unread - on laptop2: mark message M2 unread - on laptop1: doveadm sync -u user -l 10 -m test -f user@server Both laptop1 and server have M1 unread, M2 read, as expected. - on laptop2: doveadm sync -u user -l 10 -m test -f user@server Now laptop2 and server have M1 *read*, M2 unread. - on laptop1: doveadm sync -u user -l 10 -m test -f user@server Now laptop1 and the server have both M1 and M2 *read*. - on laptop2: doveadm sync -u user -l 10 -m test -f user@server Now laptop2 has both read as well. The two lines that say "*read*" are wrong in my opinion. dsync propagated a read mark to an unread message, even though that message was marked unread more recently than it was marked read. I usually use stateful sync, and get many related problems. I just did a test in which M1 and M2 started out read, and I started with empty files named dstate.test on laptop1 and laptop2. Then I did the above procedure, using the command doveadm sync -u user -l 10 -m test -s "`cat dstate.test`" user@server > dstate.test At the end, laptop2 and server had both messages unread (which is good), but laptop1 had only M1 unread, and repeated runs of the sync command did not correct this. So the stateful sync failed to detect a change. Are these bugs in dovecot? Is there more information that I can provide? The output of doveconf -n on one machine is below, and the others are almost identical. Thanks for any help! Dan # 2.3.4.1 (3c0b8769e): /etc/dovecot/dovecot.conf # OS: Linux 4.15.0-45-generic x86_64 Ubuntu 18.04.1 LTS # Hostname: laptop2 auth_mechanisms = plain login listen = 127.0.0.1 mail_index_log2_max_age = 10 days mail_index_log_rotate_min_age = 1 days mail_index_log_rotate_min_size = 300 k mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } protocols = imap service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 943 ssl = yes } } service imap { process_limit = 25 } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { postmaster_address = [elided] } protocol imap { mail_max_userip_connections = 20 }

3 8

Sieve Header question.
by Larry Rosenman 31 Aug '19

31 Aug '19

I'm trying to make my github processing better, but I'm missing something. I have the following: if address :all :contains "from" ["github.com"] { addflag "github"; addflag "MyFlags" "github"; set "mailbox" "GitHub"; if address :matches :user "to" "*" { set "GHUser" "${1}"; addflag "${GHUser}"; addflag "MyFlags" "${GHuser}"; } if header :matches "List-ID" "(.*/.*) <(.*)>" { set "mailbox" "github-lists/${1}"; } fileinto :flags "${MyFlags}" :create "${mailbox}"; stop; } I'm trying to match the apache-incubator-superset part of: List-ID: apache/incubator-superset <incubator-superset.apache.github.com> What am I missing? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

1 1

problems witch quota working
by Bartłomiej Solarz-Niesłuchowski 30 Aug '19

30 Aug '19

Good evening! Dear list I have problem with quota-fs (rquota) support with dovecot. I use dovecot version: dovecot --version 2.3.7.1 (0152c8b10) build options: dovecot --build-options Build options: ioloop=epoll notify=inotify openssl io_block_size=8192 SQL driver plugins: mysql postgresql sqlite Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap(plugin) passwd prefetch passwd-file sql the privatedevices is in service set as no: systemctl show -p PrivateDevices dovecot PrivateDevices=no but quota (rquota on nfsv4) does not work: doveadm quota get -u solarz \doveadm(solarz): Error: Failed to get quota resource STORAGE: quota-fs: quotactl(Q_GETQUOTA, oceanic:/home) failed: No such file or directory Quota name Type Value Limit % User quota STORAGE error error error quota from os works: quota solarz Disk quotas for user solarz (uid 1761): Filesystem blocks quota limit grace files quota limit grace /dev/mapper/vg01-lvol00 1588264 40480000 40480000 12383 0 0 oceanic:/home 76318340 86284030 87416830 561911 1190000 1650000 configuration: # OS: Linux 5.2.9-200.fc30.x86_64 x86_64 Fedora release 30 (Thirty) # Hostname: dervish.wsisiz.edu.pl auth_cache_size = 8 k mail_access_groups = mail mail_fsync = always mail_location = maildir:~/Maildir:INBOX=/var/spool/mail/%u mail_max_userip_connections = 500 mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = acl quota zlib trash mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = cache_key=#hidden_use-P_to_show# max_requests=256 driver = pam } passdb { args = cache_key=#hidden_use-P_to_show# max_requests=256 driver = pam } plugin { quota = fs:User quota quota_grace = 10%% quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = postmaster(a)wit.edu.pl protocols = imap pop3 service anvil { client_limit = 1155 } service auth { client_limit = 5248 user = root } service imap-login { client_limit = 1024 inet_listener imap { port = 0 } inet_listener imaps { address = [::], * port = 993 } process_limit = 1024 service_count = 0 } service imap { process_limit = 2048 vsz_limit = 512 M } service pop3-login { client_limit = 1024 inet_listener pop3 { port = 0 } inet_listener pop3s { address = [::], * port = 995 } process_limit = 128 service_count = 0 } service pop3 { process_limit = 2048 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = dovenull } ssl = required ssl_cert = </etc/pki/tls/certs/ipop3d.pem ssl_cipher_list = PROFILE=SYSTEM ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { driver = passwd } protocol imap { imap_client_workarounds = " tb-extra-mailbox-sep" mail_max_userip_connections = 200 mail_plugins = acl quota zlib trash imap_quota ssl_cert = </etc/pki/tls/certs/imapd.pem ssl_key = # hidden, use -P to show it } protocol pop3 { mail_max_userip_connections = 200 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv ssl_cert = </etc/pki/tls/certs/ipop3d.pem ssl_key = # hidden, use -P to show it } Thank for any clues. Best Regards -- Bartłomiej Solarz-Niesłuchowski, Administrator WSISiZ e-mail: Bartlomiej.Solarz-Niesluchowski(a)wit.edu.pl tel. 223486547, fax 223486501 JID: solarz(a)jabber.wit.edu.pl 01-447 Warszawa, ul. Newelska 6, pokój 421, pon.-pt. 8-16 Motto - Jak sobie pościelisz tak sie wyśpisz

1 0

doveadm backup mdbox - initial copy slow
by Gerald Galster 30 Aug '19

30 Aug '19

Hello, when calling doveadm backup like in the following example, it seems to flush every email to disk maxing out harddrives (iops) and resulting in very poor performance: Calling doveadm -o plugin/quota= backup -u "statusmails(a)domain.com" "mdbox:/backup/vmail/domain.com/statusmails" finished in 212 secs [changes:1800 copy:1800 delete:0 expunge:0] The source mdbox holds 1800 small emails and uses about 20 MB disk space only. I've tried -f (full sync) and -l 30 (locking the mailbox) which did not get faster, nor did doveadm sync -1. When using doveadm import all mails are copied in less than 3 seconds: time doveadm -o mail=mdbox:/backup/vmail/domain.com/statusmails import mdbox:/var/vmail/domain.com/statusmails "" all real 0m2.605s Are there any other options I could try to speed up doveadm backup, avoiding the flush after each email? This is doveadm from dovecot 2.2 (>= 2.2.33). Does anyone get better results with 2.3 tree? Best regards Gerald

2 2

User / Pass SQL queries
by Michael Hallager 29 Aug '19

29 Aug '19

Hi all, Is there any reason Dovecot shows the 'user' variable (ie: user(a)domain.tld) being obtained in the password query and not the (more logical) user query? Thanks, Michael

2 1

permission issue after yum update
by Voytek Eymont 29 Aug '19

29 Aug '19

I've just updated my dovecot on centos, after 'yum update' dovecot failed to work, with manual restart it worked fine, just giving a warning: # dovecot Warning: Corrected permissions for login directory /var/run/dovecot/token-login dovecot --version 2.3.7.2 (3c910f64b) all's good, just reporting in case this is relevant to rpm maintainers thanks, V

1 0

download Dovecot
by Miquel Demur 29 Aug '19

29 Aug '19

I have downloaded in MacMini MacOS X applications directory the Dovecot.tzr.gz file from Dovecot v2.3.7.2 in https://www.dovecot.org/download I have clicked on dovecot-2.3.7.2 tar.gz and it creates a dovecot-2.3.7.2 directory containung 35 files. https://wiki2.dovecot.org/FrontPage?action=show&redirect=PageD%27Accueil#Do… https://wiki2.dovecot.org/FrontPage?action=show&redirect=PageD%27Accueil#Do… I have to compile to install in Terminal Application? ./configure make sudo make install in https://wiki2.dovecot.org/CompilingSource#Compiling_Dovecot_From_Sources without reference for dovecot.

2 1

Pigeonhole release v0.4.24.2
by Aki Tuomi 29 Aug '19

29 Aug '19

Hi! We are pleased to release Pigeonhole release v0.4.24.2 Tarball is available at https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.2… https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.2… Changes ------- * CVE-2019-11500: ManageSieve protocol parser does not properly handle NUL byte when scanning data in quoted strings, leading to out of bounds heap memory writes. Found by Nick Roessler and Rafi Rubin. --- Aki Tuomi Open-Xchange oy

2 2

Feature wishlist: Allow to hide client IP/host in submission service
by Sebastian Krause 28 Aug '19

28 Aug '19

Hi, In many mail setups a required feature (for privacy reasons) is to hide the host and IP of clients (in the "Received" header) that use the authenticated submission over port 587. In Postfix that's possible (https://serverfault.com/q/413533/86332) but not very nice to configure especially if you only want want to strip the Received header for port 587 submissions, but not on port 25. As far as I can see this configuration is not possible at all in the Dovecot submission server because the function which adds the Received header with the client's IP address (smtp_server_transaction_write_trace_record) is always called in submission-commands.c. It would be very useful if the submission server could anonymize the client with a single configuration option, then all the Postfix configuration mess (and using SASL) could be skipped by simply using the Dovecot submission server instead. The anonymization would work by replacing the client's EHLO host with "submission" and the IP address with 127.0.0.1. In full the Received header would look something like this where the first line is always the same: Received: from submission (unknown [127.0.0.1]) by mail.example.com with ESMTPSA id 8bV9D+51Yl1FOwAA1ctoJQ (envelope-from <john(a)example.com>) for <jane(a)example.com>; Sun, 25 Aug 2019 13:50:06 +0200 Sebastian

4 6

Pigeonhole release v0.5.7.2
by Aki Tuomi 28 Aug '19

28 Aug '19

Hi! We are pleased to release Pigeonhole release v0.5.7.2 Tarball is available at https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.2.… https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.2.… Binary packages are available at https://repo.dovecot.org/ Changes ------- * CVE-2019-11500: ManageSieve protocol parser does not properly handle NUL byte when scanning data in quoted strings, leading to out of bounds heap memory writes. Found by Nick Roessler and Rafi Rubin. --- Aki Tuomi Open-Xchange oy

1 0