- dovecot - dovecot.org

FW: imapsieve rules not matching at all?
by dovecot＠steve.wattlink.net 20 Mar '21

20 Mar '21

Greetings! I feel like this has been beaten to death, but my searches on the web (and about 10 hours spent over the last two days) haven't revealed what's going on. Basically, it's the usual "I'd like to auto-learn spam/ham based on moves to/from a folder" problem. But in my debugging, I don't see any evidence that the static rules are matching, so the scripts aren't running, which makes me think I'm missing something obvious. Below I list doveconf -n output, and some log snippets. In my experiments, I've tried removing the report-spam.sieve and report-ham.sieve scripts, and absolutely nothing changed, which is why I think the rules aren't being tried. The requisite doveconf -n output: - - - -8< - - - - # 2.3.13 (89f716dc2): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: FreeBSD 12.2-RELEASE-p1 amd64 # Hostname: mhv3 lock_method = dotlock mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.debug vnd.dovecot.environment imapsieve vnd.dovecot.imapsieve vnd.dovecot.pipe vnd.dovecot.execute namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { imapsieve_mailbox1_before = file:/usr/local/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY APPEND imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/usr/local/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * imapsieve_url = sieve://127.0.0.1:4190 sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /usr/local/etc/dovecot/sieve/common.sieve sieve_default = /usr/local/etc/dovecot/sieve/default.sieve sieve_default_name = SystemDefault sieve_execute_bin_dir = /usr/local/etc/dovecot/sieve/bin sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.execute +vnd.dovecot.debug sieve_filter_bin_dir = /usr/local/etc/dovecot/sieve/bin sieve_global = /usr/local/etc/dovecot/sieve sieve_pipe_bin_dir = /usr/local/etc/dovecot/sieve/bin sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_addresses = no sieve_trace_debug = yes sieve_trace_dir = /var/log/dovecot/trace sieve_trace_level = matching } protocols = imap lmtp sieve service imap-login { inet_listener imap { address = 127.0.0.1 } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/ssl/services/dovecot.cert ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol lda { mail_plugins = " sieve" } protocol imap { imap_metadata = yes mail_max_userip_connections = 30 mail_plugins = " imap_sieve" } - - - ->8 - - - - I see that the static rule ought to have matched, but it didn't? - - - - 8<- - - - Mar 19 16:21:47 mhv3 dovecot[47532]: imap(steve)<47539><CdLa+uu9iNkYgsde>: Debug: sieve: Pigeonhole version 0.5.13 (cdd19fe3) initializing Mar 19 16:21:47 mhv3 dovecot[47532]: imap(steve)<47539><CdLa+uu9iNkYgsde>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.13 (cdd19fe3) loaded Mar 19 16:21:47 mhv3 dovecot[47532]: imap(steve)<47539><CdLa+uu9iNkYgsde>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.13 (cdd19fe3) loaded Mar 19 16:21:47 mhv3 dovecot[47532]: imap(steve)<47539><CdLa+uu9iNkYgsde>: Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam' from=`*' causes=(COPY APPEND) => before=`file:/usr/local/etc/dovecot/sieve/report-spam.sieve' after=(none) Mar 19 16:21:47 mhv3 dovecot[47532]: imap(steve)<47539><CdLa+uu9iNkYgsde>: Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' from=`Spam' causes=(COPY) => before=`file:/usr/local/etc/dovecot/sieve/report-ham.sieve' after=(none) Mar 19 16:21:47 mhv3 dovecot[47532]: imap(steve)<47539><CdLa+uu9iNkYgsde>: Debug: Mailbox Spam: Mailbox opened because: SELECT Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Loading modules from directory: /usr/local/lib/dovecot Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Module loaded: /usr/local/lib/dovecot/lib95_imap_sieve_plugin.so Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Effective uid=1001, gid=1000, home=/home/steve Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: maildir++: root=/home/steve/Maildir, index=, indexpvt=, control=, inbox=/home/steve/Maildir, alt= Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Mailbox INBOX: Mailbox opened because: SELECT Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: imapsieve: mailbox INBOX: APPEND event Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: imapsieve: mailbox INBOX: Mailbox attribute /shared/imapsieve/script not found Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: imapsieve: mailbox INBOX: Server attribute /shared/imapsieve/script not found Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: sieve: Pigeonhole version 0.5.13 (cdd19fe3) initializing Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.13 (cdd19fe3) loaded Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.13 (cdd19fe3) loaded Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam' from=`*' causes=(COPY APPEND) => before=`file:/usr/local/etc/dovecot/sieve/report-spam.sieve' after=(none) Mar 19 16:21:48 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' from=`Spam' causes=(COPY) => before=`file:/usr/local/etc/dovecot/sieve/report-ham.sieve' after=(none) Mar 19 16:21:49 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Mailbox INBOX: Mailbox opened because: SELECT Mar 19 16:21:54 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Mailbox INBOX: UID 77131: Opened mail because: prefetch Mar 19 16:21:54 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Mailbox INBOX: UID 77131: Opened mail because: access Mar 19 16:21:54 mhv3 dovecot[47532]: imap(steve)<47541><ROnt+uu9idkYgsde>: Debug: Mailbox INBOX: UID 77131: Opened mail because: full mail - - - ->8 - - - - Thanks for clues, -- Steve Watt KD6GGD PP-ASEL-IA factories.words.yappy Don't let your schooling get in the way of your education.

6 10

Error mailbox: Sync failed
by Elise 19 Mar '21

19 Mar '21

Dear team, Lately I get these messages randomly in my logfile: imap(elise)<68480><3asBXUG9A9hSsH9H>: Error: Mailbox INBOX: Sync failed for mbox: UID inserted in the middle of mailbox (186165 > 186164, seq=4, idx_msgs=4): 1 Time(s) I use IMAP for Outlook 2019 and Thunderbird clients simoultanious icw Dovecot 2.3.13 Can you tell me how to solve this issue? thanks. /elise

1 1

bug: some table header(?) output goes to stderr instead of stdout
by Marc 19 Mar '21

19 Mar '21

[@ sbin]# doveadm -f table -o mail_location=mdbox_deleted:/home/popusers/testtest/mdbox:INDEX=/home/popindex/testtest/index fetch -u testtest 'guid' mailbox INBOX 2> /dev/null 3c967f33b8aea671f3551db1ea8e33e9 6fa01ccc103a7009c7b940657dbcd72c ba955a6d6218950f42e5b0ee0a33a916 [@ sbin]# doveadm -f table -o mail_location=mdbox_deleted:/home/popusers/testtest/mdbox:INDEX=/home/popindex/testtest/index fetch -u testtest 'guid' mailbox INBOX guid 3c967f33b8aea671f3551db1ea8e33e9 6fa01ccc103a7009c7b940657dbcd72c ba955a6d6218950f42e5b0ee0a33a916

3 4

bug(?): trailing tab/whitespace in output
by Marc 19 Mar '21

19 Mar '21

doveadm -v -f table search -u testtest mailbox Archive/2011 guid 1de6d8632073c49f272bed211b71a8b6 returns this (echo ".$guid.") .1de6d8632073c49f272bed211b71a8b6 .

1 0

feature request: doveadm purge taking guid argument
by Marc 18 Mar '21

18 Mar '21

Hi Timo, What about giving doveadm purge an argument to delete a specific message guid? Is that a lot of work?

1 0

Some questions about code in lib-storage/index/maildir
by songliny 18 Mar '21

18 Mar '21

Hi, Recently, I have read some code in dovecot 2.3.13. And I have 2 questions about some lines of code in lib-storage/index/maildir Firstly, in the function, maildir_scan_dir, from lib-storage/index/maildir/maildir-sync.c around line 575 (https://github.com/dovecot/core/blob/6986a435e31dfe5bcd14985534b76b3c735f78…) I would like to know that after successfully stat the cur directory, does it assign the max value between st.st_mtime and start_time intensionally to new_check_time? Or maybe should it assign the value to the cur_check_time. if (stat(ctx->cur_dir, &st) == 0) { ctx->mbox->maildir_hdr.new_check_time = I_MAX(st.st_mtime, start_time); ctx->mbox->maildir_hdr.cur_mtime = st.st_mtime; ctx->mbox->maildir_hdr.cur_mtime_nsecs = ST_MTIME_NSEC(st); } Secondly, in the function, maildir_uidlist_sync_next_uid, from lib-storage/index/maildir/maildir-uidlist.c (https://github.com/dovecot/core/blob/6986a435e31dfe5bcd14985534b76b3c735f78… rc/lib-storage/index/maildir/maildir-uidlist.c#L1760) Is it ok to remove MAILDIR_UIDLIST_REC_FLAG_NEW_DIR flag from the record for the record that is not found in ctx->files just like the behavior (https://github.com/dovecot/core/blob/6986a435e31dfe5bcd14985534b76b3c735f78… the function, maildir_uidlist_sync_next_partial (https://github.com/dovecot/core/blob/6986a435e31dfe5bcd14985534b76b3c735f78…) from the same file? I think when it gets to the function, maildir_uidlist_sync_next_uid, the mail file have already been moved to cur directory. And maybe it is ok to remove MAILDIR_UIDLIST_REC_FLAG_NEW_DIR flag from the record. That is, is it ok to add this line rec->flags &= ENUM_NEGATE(MAILDIR_UIDLIST_REC_FLAG_NEW_DIR); just before rec->flags = (rec->flags | flags) & ENUM_NEGATE(MAILDIR_UIDLIST_REC_FLAG_NONSYNCED); in maildir_uidlist_sync_next_uid? Thank you in advance to he lp me get more understanding to these questions, Sony

1 0

dovecot director and keepalived
by Steven Varco 18 Mar '21

18 Mar '21

Hi All I’m trying to establish a dovecot HA setup with two loadbalancers, running keepalived for sharing a virtual public IP. On the same machines I’m running a dovecot director which proxies the requests to two underlying mail servers (on seperate machines). Now I’m hitting the issue with the way director determines his „Self IP“ by trying to bind to all configured director_servers IPs, taking the first one possible. However this approach only works, when the sysctl setting is: net.ipv4.ip_nonlocal_bind=0 On the other side keepalived needs net.ipv4.ip_nonlocal_bind=1 in order to bind the VIP. The last topic on that is dating back to 2016 (https://dovecot.org/pipermail/dovecot/2016-August/105191.html) with references to 2012 (https://www.dovecot.org/list/dovecot/2012-November/087033.html) and no solution posted so far. After five more years :D, I’m asking myself if we finally have a solution for that, or if my approach of achieving clustered director servers is potentially wrong? Other possible solutions I could think about: - Configure each director as „independent“ by setting only one IP in director_servers. => With this aporach you would loose the user to mailserver mapping, although only in a a case of a failover on the loadbalancer, which might can be neglected (or are there any other fallbacks?) - Only have director running on the currently active loadblancer node and stopped on the passive loadblancer node (would possibly have the same effects as above). - Putting director on seperated intermediate machines and proxing the requests through haproxy on the keepalived servers (keepalived -> haproxy -> director -> IMAP => Besides the disadvantage of having another bunch of servers in the chain, also some special configuration on the directory servers might be neccessary to assure director works neatly with haproxy. So 2021, what is the „correct“ (best practive) way of having a reduntant HA setup for dovecot? This means a MUA connects to one public IP and gets connected to (preferably the same) IMAP Server, no matter which machine in the whole chain might be down? PS: Using just multiple A records on the mail domain name (round-robin), while working perfectly for SMTP is not accepatbl for IMAP IMHO, as in case of a failure every second request from the client (MUA) would fail and most MUAs are not automatially reconnecting again in that case. thanks, Steven -- https://steven.varco.ch/

7 10

best practice purging only moved messages
by Marc 18 Mar '21

18 Mar '21

If I move a message from the 'inbox' namespace to my 'archive' namespace. The message is expunged at the 'inbox' namespace and added to the 'archive' namespace. What is the best way to only purge messages that have been moved. (I do not want to purge emails that have been deleted yet)

1 0

How can I migrate mailboxes after changing hierarchy separator?
by Popkornium18 17 Mar '21

17 Mar '21

Hi, I had a common problem (folders can't contain dots) and the obvious solution seems to be changing the hierarchy separator and using the listescape plugin. My question is: Do I need to take any extra steps after changing the separator to migrate my existing mailboxes? If so, which? I tried just simply renaming the folders and doing a `doveadm force-resync` but that did not work as intended. Afterwards I wasn't able to subscribe to any of my folders, so I guess that isn't the way to go. Kind regards Simon

1 0

inconsistency in reporting from mdbox_deleted
by Marc 17 Mar '21

17 Mar '21

This reports all folders, ok [@ testtest]# doveadm -o mail_location=mdbox_deleted:/home/mail-archive/testtest/Archive:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' '*' 2011 messages=0 vsize=0 2012 messages=0 vsize=0 2013 messages=0 vsize=0 2014 messages=0 vsize=0 2015 messages=0 vsize=0 2016 messages=0 vsize=0 2017 messages=0 vsize=0 2018 messages=0 vsize=0 2019 messages=0 vsize=0 doveadm(testtest): Error: Mailbox Archive: Failed to lookup mailbox status: Mailbox doesn't exist: Archive INBOX messages=15 vsize=12066832 This gets folder INBOX, ok [@ testtest]# doveadm -o mail_location=mdbox_deleted:/home/mail-archive/testtest/Archive:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' 'INBOX' INBOX messages=15 vsize=12066832 This reports error that a folder does not exist, ok [@ testtest]# doveadm -o mail_location=mdbox_deleted:/home/mail-archive/testtest/Archive:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' 'asdfasdfasdfasdf' doveadm(testtest): Error: Mailbox asdfasdfasdfasdf: Failed to lookup mailbox status: Mailbox doesn't exist: asdfasdfasdfasdf This query for Archive/2012 reports same as for 2012(?), by design, I guess ok [@ testtest]# doveadm -o mail_location=mdbox_deleted:/home/mail-archive/testtest/Archive:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' 'Archive/2012' Archive/2012 messages=0 vsize=0 This reports 0?, This should either report the folder does not exist, or it should report the same as getting the INBOX, like previous result [@ testtest]# doveadm -o mail_location=mdbox_deleted:/home/mail-archive/testtest/Archive:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' 'Archive/INBOX' Archive/INBOX messages=0 vsize=0

1 0