- dovecot - dovecot.org

Imaptest using the 'DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA'
by Marc Roos 16 Sep '20

16 Sep '20

I just noticed that the imaptest program is using this default first message 'DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA' of the mbox file to test with. I think this should be execluded, as one does not expect this behaviour.

1 0

correct usage for Pigeonhole sieve message *redirect* to external server?
by PGNet Dev 16 Sep '20

16 Sep '20

i've installed rpm -qa | grep dovecot dovecot-2.3.10.1-1.fc32.x86_64 dovecot-pigeonhole-2.3.10.1-1.fc32.x86_64 i'm setting up imapsieve spam reporting; following, https://doc.dovecot.org/configuration_manual/spam_reporting/ https://wiki.dovecot.org/HowTo/AntispamWithSieve https://wiki.dovecot.org/Pigeonhole/Sieve/Examples my config currrently includes protocol imap { mail_plugins = $mail_plugins acl imap_acl quota imap_quota imap_sieve mail_log notify mail_max_userip_connections = 20 imap_client_workarounds = tb-extra-mailbox-sep } protocol sieve { mail_plugins = $mail_plugins mail_max_userip_connections = 10 } plugin { # From elsewhere to Spam folder imapsieve_mailbox1_name = Spam imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_before = file:/etc/dovecot/scripts/report-spam.sieve # From Spam folder to elsewhere imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/etc/dovecot/scripts/report-ham.sieve sieve_plugins = sieve_imapsieve sieve_pipe_bin_dir = /etc/dovecot/scripts } service submission-login { inet_listener submission { address = 10.0.1.70, 127.0.0.1 port = 60465 ssl = yes } } examples @ wiki include report-spam.sieve require "vnd.dovecot.report"; report "abuse" "User added this message to the Spam folder." "spam-report(a)example.com"; instead of a 'report', i want to *redirect* the _unmodified_ Spam message to an _external_ email that's a feed for a bayes spam learner. to that end i modified /etc/dovecot/scripts/report-spam.sieve require ["vnd.dovecot.report", "vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"]; if environment "imap.mailbox" "Trash" { stop; } if environment "imap.mailbox" "Spam" { stop; } redirect "spam.learn(a)othermx.example.net"; with that^, when I drag a spam message into my user account's Spam folder, I see in logs only 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: Mailbox Junk: Mailbox opened because: UID move 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: acl vfile: file /home/vmail/conf.d/example.com/acls/Junk not found 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: acl vfile: file /data/vmail/example.com/myuser/Maildir/Junk/dovecot-acl not found 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: imapsieve: mailbox Junk: MOVE event 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: sieve: Pigeonhole version 0.5.10 (bf8ef1c2) initializing 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.10 (bf8ef1c2) loaded 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam' from=`*' causes=(COPY) => before=`file:/etc/dovecot/scripts/report-spam.sieve' after=(none) 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' from=`Spam' causes=(COPY) => before=`file:/etc/dovecot/scripts/report-ham.sieve' after=(none) ==> /var/log/dovecot/dovecot-info.log <== 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Info: copy from INBOX: box=Junk, uid=4, msgid=<0e9fc21a-c046-46bd-a4ae-e7a1b6424ca7.maildroid@localhost>, size=5647 2020-09-15 08:52:16 imap(myuser(a)example.com)<bCdiIOuhPImsDgsd>: Info: expunge: box=INBOX, uid=50, msgid=<0e9fc21a-c046-46bd-a4ae-e7a1b6424ca7.maildroid@localhost>, size=5647 but no redirect occurs; nothing's sent. what additional/different config is required to get the sieve processor to *send* that email? in sieve redirect, _is_ the dovecot submission port implicitly used for sending/redirect? or does the report-spam script need additional capability to exec the send?

1 1

Auth Panic hmac.c while Local Validation
by Mrinal Sharma 16 Sep '20

16 Sep '20

I am trying to use a newly added Local Validation functionality in dovecot version 2.3.11. I am running dovecot inside a Docker container With base image " debian:buster-slim". When I try to login through below command , a crash is seen. ''' a1 login admin eyJhbGci44444zUxMiIsInR5cCI6IkpXVCJ9.eyJ1c2fyX25hbWUiOiJhZG1pbiIsInNjb3BlIjpbIm9wZW5pZCJdLCJleffiOjE3MDAxODAwNTksInN1YiI6ImFkbWluIiwiaWF0IjoxNjAwMTc5NzU5LCJhdXRob3JpdGllcyI6WyJST0xFX0FETUlOIiwiUk9MRV9VU0VSIl0sImp0aSI6ImRyOUV0MVVJWkdJZkZ0emFVZW5VRzRzcmQtQSIsImNsaWVudF9pZCI6IndlYl9hcHAifQ.T9BTZYW52p0VG9gxmTb-cf5GXF5jTOjdkqMaUMAGX-tffffft7YfdPptphGKE8FO9opxcnL--Bjy9ip-XYuWqA '''' Crash: dovecot_1 | Sep 15 16:11:50 auth: Panic: file hmac.c: line 26 (hmac_init): assertion failed: (meth->context_size <= HMAC_MAX_CONTEXT_SIZE) dovecot_1 | Sep 15 16:11:50 auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7ff77b36beed] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7ff77b36c00e] -> /usr/lib/dovecot/libdovecot.so.0(+0xf465b) [0x7ff77b37565b] -> /usr/lib/dovecot/libdovecot.so.0(+0xf46f1) [0x7ff77b3756f1] -> /usr/lib/dovecot/libdovecot.so.0(+0x514a6) [0x7ff77b2d24a6] -> /usr/lib/dovecot/libdovecot.so.0(+0x52415) [0x7ff77b2d3415] -> /usr/lib/dovecot/libdovecot.so.0(+0x59c09) [0x7ff77b2dac09] -> /usr/lib/dovecot/libdovecot.so.0(oauth2_try_parse_jwt+0x7bb) [0x7ff77b2db60b] -> dovecot/auth(+0x3bf71) [0x55576d3cdf71] -> dovecot/auth(db_oauth2_lookup+0x350) [0x55576d3cf030] -> dovecot/auth(auth_request_default_verify_plain_continue+0x2d6) [0x55576d3b3386] -> dovecot/auth(auth_request_verify_plain_callback_finish+0x5c) [0x55576d3b214c] -> dovecot/auth(auth_request_verify_plain_callback+0x51) [0x55576d3b2281] -> dovecot/auth(+0x30171) [0x55576d3c2171] -> dovecot/auth(+0x27ebb) [0x55576d3b9ebb] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7ff77b38b989] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7ff77b38ced1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7ff77b38ba2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7ff77b38bba0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ff77b3059c3] -> dovecot/auth(main+0x3e1) [0x55576d3a6fa1] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7ff77b03809b] -> dovecot/auth(_start+0x2a) [0x55576d3a713a] dovecot_1 | Sep 15 16:11:50 auth-worker(25): Debug: conn unix:auth-worker (pid=24,uid=101): Disconnected: Connection closed (fd=-1) dovecot_1 | Sep 15 16:11:50 imap-login: Debug: Ignoring unknown passdb extra field: temp dovecot --version 2.3.11.3 (502c39af9) dovecot -n # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # OS: Linux 4.19.76-linuxkit x86_64 Debian 10.5 fuse.grpcfuse # Hostname: 45e39b46f6ab auth_debug = yes auth_mechanisms = plain oauthbearer xoauth2 auth_verbose = yes disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it log_path = /dev/stdout mail_debug = yes mail_location = maildir:/data/imap_store/%n mail_plugins = " quota" namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Greeting { auto = create } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = no special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-oauth2.plain.conf.ext driver = oauth2 mechanisms = plain login } plugin { quota = maildir:User quota quota_rule = *:storage=5MB quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO } protocols = imap lmtp service doveadm { inet_listener http { port = 80 } } service lmtp { inet_listener lmtp { address = * port = 24 } process_min_avail = 5 } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota postmaster_address = postmaster@domainname } protocol lda { mail_plugins = " quota notify push_notification" } protocol imap { imap_metadata = yes mail_plugins = " quota imap_quota quota" } Any help would be appreciated. Thanks, Mrinal

2 2

Shared mailboxes setups and dictionaries
by Matej Tyc 16 Sep '20

16 Sep '20

Hello, I am relatively new to the world of MTAs and MDAs, and I try to set up shared mailboxes. So far I have somehow succeeded - I have defined a shared namespace and I have managed to create per-mailbox ACL files thanks to the doveadm command. However, I have been following these resources and there were bits that have puzzled me: When learning about how ACL work in e.g. https://doc.dovecot.org/settings/plugin/acl/ - when one wishes to use the Global ACL file, how does one link it to a particular user's mailboxes? Examples that are listed in the documentation are far too generic. For example does "* user=foo lrw" imply that all mailboxes of all accounts are shared to the user foo? The doveadm command works only if dovecot is set up with per-mailbox ACL files, so I can't use it to reverse-engineer the correct syntax. An interesting aspect to ACLs are dictionaries. I understood it as some kind of cache - if there is no dictionary or it is empty, then shared mailboxes don't work. Conversely, dictionary itself is not enough, one needs actual ACLs set up correctly. Is this a correct understanding? Next what https://wiki.dovecot.org/SharedMailboxes/Shared and https://wiki.dovecot.org/Dictionary describe is a possibility to reference LDAP data to define an ACL dictionary. Do I understand it correctly that if a LDAP database is the single source of truth, then I don't have to worry about updating dictionaries as long as LDAP itself is up-to-date, but I have to keep ACLs and LDAP in sync manually (or using an application)? Thanks, Matej

2 3

Btrfs RAID-10 performance
by Miloslav Hůla 15 Sep '20

15 Sep '20

Hello, I sent this into the Linux Kernel Btrfs mailing list and I got reply: "RAID-1 would be preferable" (https://lore.kernel.org/linux-btrfs/7b364356-7041-7d18-bd77-f60e0e2e2112@le…) May I ask you for the comments as from people around the Dovecot? We are using btrfs RAID-10 (/data, 4.7TB) on a physical Supermicro server with Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz and 125GB of RAM. We run 'btrfs scrub start -B -d /data' every Sunday as a cron task. It takes about 50 minutes to finish. # uname -a Linux imap 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64 GNU/Linux RAID is a composition of 16 harddrives. Harddrives are connected via AVAGO MegaRAID SAS 9361-8i as a RAID-0 devices. All harddrives are SAS 2.5" 15k drives. Server serves as a IMAP with Dovecot 2.2.27-3+deb9u6, 4104 accounts, Mailbox format, LMTP delivery. We run 'rsync' to remote NAS daily. It takes about 6.5 hours to finish, 12'265'387 files last night. Last half year, we encoutered into performace troubles. Server load grows up to 30 in rush hours, due to IO waits. We tried to attach next harddrives (the 838G ones in a list below) and increase a free space by rebalace. I think, it helped a little bit, not not so rapidly. Is this a reasonable setup and use case for btrfs RAID-10? If so, are there some recommendations to achieve better performance? Thank you. With kind regards Milo # megaclisas-status -- Controller information -- -- ID | H/W Model | RAM | Temp | BBU | Firmware c0 | AVAGO MegaRAID SAS 9361-8i | 1024MB | 72C | Good | FW: 24.16.0-0082 -- Array information -- -- ID | Type | Size | Strpsz | Flags | DskCache | Status | OS Path | CacheCade |InProgress c0u0 | RAID-0 | 838G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdq | None |None c0u1 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sda | None |None c0u2 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdb | None |None c0u3 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdc | None |None c0u4 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdd | None |None c0u5 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sde | None |None c0u6 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdf | None |None c0u7 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdg | None |None c0u8 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdh | None |None c0u9 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdi | None |None c0u10 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdj | None |None c0u11 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdk | None |None c0u12 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdl | None |None c0u13 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdm | None |None c0u14 | RAID-0 | 558G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdn | None |None c0u15 | RAID-0 | 838G | 256 KB | RA,WB | Enabled | Optimal | /dev/sdr | None |None -- Disk information -- -- ID | Type | Drive Model | Size | Status | Speed | Temp | Slot ID | LSI ID c0u0p0 | HDD | SEAGATE ST900MP0006 N003WAG0Q3S3 | 837.8 Gb | Online, Spun Up | 12.0Gb/s | 53C | [8:14] | 32 c0u1p0 | HDD | HGST HUC156060CSS200 A3800XV250TJ | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 38C | [8:0] | 12 c0u2p0 | HDD | HGST HUC156060CSS200 A3800XV3XT4J | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 43C | [8:1] | 11 c0u3p0 | HDD | HGST HUC156060CSS200 ADB05ZG4XLZU | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 46C | [8:2] | 25 c0u4p0 | HDD | HGST HUC156060CSS200 A3800XV3DWRL | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 48C | [8:3] | 14 c0u5p0 | HDD | HGST HUC156060CSS200 A3800XV3XZTL | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 52C | [8:4] | 18 c0u6p0 | HDD | HGST HUC156060CSS200 A3800XV3VSKJ | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 55C | [8:5] | 15 c0u7p0 | HDD | SEAGATE ST600MP0006 N003WAF1LWKE | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 56C | [8:6] | 28 c0u8p0 | HDD | HGST HUC156060CSS200 A3800XV3XTDJ | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 55C | [8:7] | 20 c0u9p0 | HDD | HGST HUC156060CSS200 A3800XV3T8XL | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 57C | [8:8] | 19 c0u10p0 | HDD | HGST HUC156060CSS200 A7030XHL0ZYP | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 61C | [8:9] | 23 c0u11p0 | HDD | HGST HUC156060CSS200 ADB05ZG4VR3P | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 60C | [8:10] | 24 c0u12p0 | HDD | SEAGATE ST600MP0006 N003WAF195KA | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 60C | [8:11] | 29 c0u13p0 | HDD | SEAGATE ST600MP0006 N003WAF1LTZW | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 56C | [8:12] | 26 c0u14p0 | HDD | SEAGATE ST600MP0006 N003WAF1LWH6 | 558.4 Gb | Online, Spun Up | 12.0Gb/s | 55C | [8:13] | 27 c0u15p0 | HDD | SEAGATE ST900MP0006 N003WAG0Q414 | 837.8 Gb | Online, Spun Up | 12.0Gb/s | 47C | [8:15] | 33 # btrfs --version btrfs-progs v4.7.3 # btrfs fi show Label: 'DATA' uuid: 5b285a46-e55d-4191-924f-0884fa06edd8 Total devices 16 FS bytes used 3.49TiB devid 1 size 558.41GiB used 448.66GiB path /dev/sda devid 2 size 558.41GiB used 448.66GiB path /dev/sdb devid 4 size 558.41GiB used 448.66GiB path /dev/sdd devid 5 size 558.41GiB used 448.66GiB path /dev/sde devid 7 size 558.41GiB used 448.66GiB path /dev/sdg devid 8 size 558.41GiB used 448.66GiB path /dev/sdh devid 9 size 558.41GiB used 448.66GiB path /dev/sdf devid 10 size 558.41GiB used 448.66GiB path /dev/sdi devid 11 size 558.41GiB used 448.66GiB path /dev/sdj devid 13 size 558.41GiB used 448.66GiB path /dev/sdk devid 14 size 558.41GiB used 448.66GiB path /dev/sdc devid 15 size 558.41GiB used 448.66GiB path /dev/sdl devid 16 size 558.41GiB used 448.66GiB path /dev/sdm devid 17 size 558.41GiB used 448.66GiB path /dev/sdn devid 18 size 837.84GiB used 448.66GiB path /dev/sdr devid 19 size 837.84GiB used 448.66GiB path /dev/sdq # btrfs fi df /data/ Data, RAID10: total=3.48TiB, used=3.47TiB System, RAID10: total=256.00MiB, used=320.00KiB Metadata, RAID10: total=21.00GiB, used=18.17GiB GlobalReserve, single: total=512.00MiB, used=0.00B I do not attach whole dmesg log. It is almost empty, without errors. Only lines about BTRFS are about relocations, like: BTRFS info (device sda): relocating block group 29435663220736 flags 65 BTRFS info (device sda): found 54460 extents BTRFS info (device sda): found 54459 extents

6 18

[BUG] GSSAPI auth doesn't work on 2.3.11
by Carlos Mogas da Silva 15 Sep '20

15 Sep '20

Hi list! I have a CentOS 7 server running dovecot with GSSAPI auth enabled. Version 2.3.10 was working fine but after the upgrade to 2.3.11 it stopped working. On thing I see on the logs is that the user field comes up empty when compared to the 2.3.10. Example: 2.3.10: Sep 14 13:39:20 mail.r3pek.org dovecot[10189]: imap-login: Login: user=<r3pek(a)r3pek.org>, method=GSSAPI, rip=xxxx, lip=2a01:4f8:1c17:61c0::1, mpid=10195, TLS, session=<PaDSUkWvAOkgAQgY3LZuALKwbzMw3BW/> 2.3.11: Sep 14 12:19:24 mail.r3pek.org dovecot[9264]: imap-login: Disconnected (auth failed, 1 attempts in 6 secs): user=<>, method=GSSAPI, rip=2001:818:dcb6:6e00:b2b0:6f33:30dc:15bf, lip=2a01:4f8:1c17:61c0::1, TLS, session=<pRubNESvMuQgAQgY3LZuALKwbzMw3BW/> Reverting back to 2.3.10 confirmed it's a 2.3.11 thing. Is there any change necessary after the upgrade to 2.3.11 or is this really a bug?

2 2

Is it possible to only replicate indexes and not the mail data
by Marc Roos 14 Sep '20

14 Sep '20

Is it possible to only replicate indexes and not the mail data? (Because the data is already on distributed storage)

2 2

logging LMTP activity
by Janis 11 Sep '20

11 Sep '20

Hello, I've been trying to set up Postfix + Dovecot LMTP setup. Basic idea taken from https://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP, but since Postfix is on different host inet listener is used instead of unix socket. Message delivery works. Messages are delivered to correct mailboxes, if they exist. But the problem is with logging on Dovecot side; i can not get any lmtp logs at all. I kind of expected that i could get logs about: 1) Postfix has connected and delivering mail to mail(a)address.com, 2) If mailbox does not exists, would expect to see in logs the reason for rejection - mailbox does not exist 3) any other useful info would be nice as well. How do you get LMTP to log something? My Dovecot version is 2.3.11.3. Basic IMAP logging is working. I see when users have connected, etc. I tried as suggested at https://doc.dovecot.org/configuration_manual/protocols/lmtp_server/#logging If the given logging example block is used in configuration, i get no logs in file /var/log/dovecot-lmtp.log. I thought maybe file is not created automatically, so i tried to create it myself as well. Does not work. chmod 0777 does not either, chown to dovecot:dovecot neither. In a way i am curious about "executable = lmtp -L" part. What is -L flag? Are there any more flags? Where could i learn about them? I have read all the docs that come with the buld and mention something of lmtp, it's basically wiki, i guess... but i could not find any helpful info. I found that file lmtp is at /usr/local/libexec/dovecot/lmtp. When i execute it with -L flag it works. Other flags don't. The problem is that there are no -h or --help flags. Then i tried raw logging as described here: https://doc.dovecot.org/admin_manual/debugging/debugging_rawlog/#debugging-… As i understand, only 4th possibility is applicable to LMTP, am i right? 2nd kind of seems maybe somehow could be usable, but i don't see how... It is not clearly stated in wiki where "lmtp_rawlog_dir" should be added, but i add it within "protocol lmtp { .. }" block, similar as imap. Does not work for me - no logs while messages arrive. The catchy part in wiki is that for imap special arguments are used "-R rawlog" in service imap-login, postlogin. But i don't see any lmtp-login or such service available. So should it work only with "lmtp_rawlog_dir" option? What could i be doing wrong? Does anyone have working lmtp configuration with logging available? Thank you very much! I am sorry that my message is so long, but i tried even more to get it working... :) Any help or idea of how to better debug the situation or search for solution is kindly appreciated.

2 2

Re: Doveadm error since 2.3.11.3 when run as unprivileged user
by Olav Seyfarth 11 Sep '20

11 Sep '20

On Thu, Aug 13, 2020, Arjen de Korte wrote: > include optional config file and suppress errors if it doesn't exist I followed this workaround, which is nicely summarized at https://github.com/postfixadmin/postfixadmin/issues/381#issuecomment-678269… Yet, not all errors are suppressed this way: $ doveadm pw doveadm(olav): Error: net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied $ ls -l /run/dovecot/stats-writer srw-rw---- 1 root dovecot 0 11. Sep 00:27 stats-writer Olav

2 2

Re: Btrfs RAID-10 performance
by Robert Nowotny 11 Sep '20

11 Sep '20

"Miloslav" == Miloslav Hůla <miloslav.hula(a)gmail.com> <miloslav.hula(a)gmail.com> writes: Miloslav> Dne 09.09.2020 v 17:52 John Stoffel napsal(a): Miloslav> There is a one PCIe RAID controller in a chasis. AVAGO Miloslav> MegaRAID SAS 9361-8i. And 16x SAS 15k drives conneced to Miloslav> it. Because the controller does not support pass-through for Miloslav> the drives, we use 16x RAID-0 on controller. So, we get Miloslav> /dev/sda ... /dev/sdp (roughly) in OS. And over that we have Miloslav> single btrfs RAID-10, composed of 16 devices, mounted as Miloslav> /data. I will bet that this is one of your bottlenecks as well. Get a secord or third controller and split your disks across them evenly. Miloslav> That's plan for a next step. Miloslav> We run 'rsync' to remote NAS daily. It takes about 6.5 hours to finish, Miloslav> 12'265'387 files last night. That's.... sucky. So basically you're hitting the drives hard with random IOPs and you're probably running out of performance. How much space are you using on the filesystem? Miloslav> It's not so sucky how it seems. rsync runs during the Miloslav> night. And even reading is high, server load stays low. We Miloslav> have problems with writes. Ok. So putting in an SSD pair to cache things should help. And why not use brtfs send to ship off snapshots instead of using rsync? I'm sure that would be an improvement... Miloslav> We run backup to external NAS (NetApp) for a disaster Miloslav> recovery scenario. Moreover NAS is spreaded across multiple Miloslav> locations. Then we create NAS snapshot, tens days Miloslav> backward. All snapshots easily available via NFS mount. And Miloslav> NAS capacity is cheaper. So why not run the backend storage on the Netapp, and just keep the indexes and such local to the system? I've run Netapps for many years and they work really well. And then you'd get automatic backups using schedule snapshots. Keep the index files local on disk/SSDs and put the maildirs out to NFSv3 volume(s) on the Netapp(s). Should do wonders. And you'll stop needing to do rsync at night. Miloslav> It's the option we have in minds. As you wrote, NetApp is very solid. Miloslav> The main reason for local storage is, that IMAP server is completely Miloslav> isolated from network. But maybe one day will use it. It's not completely isolated, it can rsync data to another host that has access to the Netapp. **grin** Miloslav> Unfortunately, to quickly fix the problem and make server Miloslav> usable again, we already added SSD and moved indexes on Miloslav> it. So we have no measurements in old state. That's ok, if it's better, then its better. How is the load now? Looking at the output of 'iostat -x 30' might be a good thing. Miloslav> Situation is better, but I guess, problem still exists. I Miloslav> takes some time to load be growing. We will see. Hmm... how did you setup the new indexes volume? Did you just use btrfs again? Did you mirror your SSDs as well? Do the indexes fill the SSD, or is there 20-30% free space? When an SSD gets fragmented, it's performance can drop quite a bit. Did you put the SSDs onto a seperate controller? Probably not. So now you've just increased the load on the single controller, when you really should be spreading it out more to improve things. Another possible hack would be to move some stuff to a RAM disk, assuming your server is on a UPS/Generator incase of power loss. But that's an unsafe hack. Also, do you have quotas turned on? That's a performance hit for sure. Miloslav> Thank you for the fio tip. Definetly I'll try that. It's a good way to test and measure how the system will react. Unfortunately, you will need to do your testing outside of normal work hours so as to not impact your users too much. Good luck! Please post some numbers if you get them. If you see only a few disks are 75% or more busy, then **maybe** you have a bad disk in the system, and moving off that disk or replacing it might help. Again, hard to know. Rebalancing btrfs might also help, especially now that you've moved the indexes off that volume. Robert: Fio is an acronym for *Flexible IO Tester* and describes a tool for measuring IO performance. With Fio, devices such as hard drives or SSDs can be tested for their speed by executing a *user-defined **workload* and collecting performance data. Therefore it might be difficult to really simulate the load with that, because You need to define the workload Yourself. But, at least, You might use it to get an idea of maximum transfer rates, random I/O etc. *iotop* shows the *current I/O transfer **rates* for the currently running processes / threads. It uses the I/O usage information of the Linux kernel, so it might be a good tool for You. *htop *might also be Your friend. and of course, as someone else pointed out : > Looking at the output of 'iostat -x 30' might be a good thing. my default setup for such diagnosis is to run iotop, htop, ntop to get an idea whats going on and measure ... for the index-SSD (on a different Channel ? NvMe ?) I strongly suggest to compare EXT4 vs BTRFS - some Benchmarks suggest that on fast SSD´s EXT4 is much (factor 4-5) faster then btrfs. You might also try ZSTD transparent compression on the remaining (non-SSD) btrfs disks - this can boost the transfer rate (since the data is compressed) if the Server CPU´s have enough juice. But from my guts feeling, its not the transfer rate - its the random I/O (in that case the situation will get worse with compression), but You really need to measure that. In general I dont understand that (old fashioned) configuration - using a bunch of 2TB or even 4TB TLC SSD in Raid10 is nowadays really affordable. You can keep the old disks for (local) rsync before You rsync from there to the NAS ... Robert

1 0