August 2021 - dovecot

Get Director vhostCount
by William Edwards 11 Aug '21

11 Aug '21

Hi, I would like to monitor each mail server's amount of available vhosts, so I can take action when a certain threshold is reached (e.g. adding mail servers). I'm using Dovecot 2.3.4.1 with Director. My first idea was to use the Doveadm HTTP API, but the 'directorStatus' command doesn't return the 'vhostCount' attribute (although the 'directorAdd' and 'directorUpdate' commands take it as input). My second idea was to directly connect to the 'director-admin' socket and look at the output of 'HOST-LIST'. I can find all the information I need in doveadm-connection.c, but I'm not getting the impression that we're meant to interact with the socket directly. I can't find protocol documentation, and the error message "doveadm not compatible with this server (mixed old and new binaries?)" when passing faulty input gives me the impression that the socket is only meant to be used by Dovecot directly. My third idea is to parse 'doveadm director status', but in that case I'd like to know if that output is likely to change in the future. New major versions aside, of course. Or, if anyone else has a better idea, which could very well be the case, I'd love to hear how people monitor this in the real world. TIA. -- With kind regards, William Edwards

1 1

JWT local validation
by Tomas Habarta 11 Aug '21

11 Aug '21

Hello, I have a working setup with Roundcube using OAuth2 -- introspection works without any problem, unfortunately local validation does not as tokens are missing "typ" header (seems that one is indeed optional per RFC7519 and therefore not present in the implementation in place). Is there any parameter to assert the token type or any other workaround to make local validation work as it currently fails with: oauth2 failed: Local validation failed: Cannot find 'typ' field. dovecot v2.3.15 Roundcube 1.5beta CentOS 8 Thanks, regards Tomas

2 3

Test for implicit keep within a sieve script
by Eric Durand 11 Aug '21

11 Aug '21

Hi folks, Is there a good way to test for an implicit keep in a sieve script ? At the end of my sieve script, if a message still has an implicit keep, it will end up in my inbox, and I would like to push a notification. Right now I am doing this with an ad-hoc variable that is essentially emulating the implicit keep, is there a better way to do this ? Thanks! Eric

2 1

pigeonhole-0.5.16 compilation error?
by Sylvain Robitaille 10 Aug '21

10 Aug '21

Hello Dovecot folks ... In response to the recent announcement of Dovecot-2.3.16 and pigeonhole-0.5.16, we decided to update our installation (full disclosure: this is really an update from dovecot-1.2.x in progress; we're working on a development system where we already had dovecot-2.3.x installed and running). The update of dovecot itself (compiled from source), to version 2.3.16 was quite uneventful, and I can report that it's working. We're seeing some behaviour with indexing that will need attention, but I suspect that that's a matter of configuration, and is completely unrelated to what I'm reporting. More disclosure: we've never used nor compiled pigeonhole on our systems before. We've been using procmail for decades, but are now looking into potentially switching to sieve, and as I understand it, that means installing pigeonhole. I downloaded dovecot-2.3-pigeonhole-0.5.16 at the same time that I downloaded dovecot-2.3.16, a few days ago. My build-time configure and make for pigeonhole is relatively simple, but fails to build: ./configure --prefix=/local/pkg/pigeonhole-0.5.16/root \ --sbindir=/local/pkg/pigeonhole-0.5.16/root/bin \ --localstatedir=/var \ --mandir=/local/pkg/pigeonhole-0.5.16/root/man \ --with-dovecot=/local/pkg/dovecot-CURRENT/root/lib/dovecot make -j2 "/local/pkg/dovecot-CURRENT/root/lib/dovecot" is correct for us, if I understand the installation instructions correctly: -rw-r--r-- 1 root root 1324 Aug 3 14:03 /local/pkg/dovecot-CURRENT/root/lib/dovecot/dovecot-config The make fails, while compiling sieve-extprograms-common.c: sieve-extprograms-common.c: In function ‘sieve_extprogram_run’: sieve-extprograms-common.c:639:7: error: ‘PROGRAM_CLIENT_EXIT_STATUS_INTERNAL_FAILURE’ undeclared (first use in this function) case PROGRAM_CLIENT_EXIT_STATUS_INTERNAL_FAILURE: ^ sieve-extprograms-common.c:639:7: note: each undeclared identifier is reported only once for each function it appears in sieve-extprograms-common.c:641:7: error: ‘PROGRAM_CLIENT_EXIT_STATUS_FAILURE’ undeclared (first use in this function) case PROGRAM_CLIENT_EXIT_STATUS_FAILURE: ^ sieve-extprograms-common.c:643:7: error: ‘PROGRAM_CLIENT_EXIT_STATUS_SUCCESS’ undeclared (first use in this function) case PROGRAM_CLIENT_EXIT_STATUS_SUCCESS: ^ I decided to check against dovecot-2.3-pigeonhole-0.5.15, figuring that problems compiling that version likely would have been reported already, and sure enough I find the following difference in this file: --- ./dovecot-2.3-pigeonhole-0.5.15/src/plugins/sieve-extprograms/sieve-extprograms-common.c 2021-06-14 09:41:29.000000000 -0400 +++ ./dovecot-2.3-pigeonhole-0.5.16/src/plugins/sieve-extprograms/sieve-extprograms-common.c 2021-08-06 05:26:46.000000000 -0400 @@ -635,6 +635,14 @@ int sieve_extprogram_set_input_mail int sieve_extprogram_run(struct sieve_extprogram *sprog) { - return program_client_run(sprog->program_client); + switch (program_client_run(sprog->program_client)) { + case PROGRAM_CLIENT_EXIT_STATUS_INTERNAL_FAILURE: + return -1; + case PROGRAM_CLIENT_EXIT_STATUS_FAILURE: + return 0; + case PROGRAM_CLIENT_EXIT_STATUS_SUCCESS: + return 1; + } + i_unreached(); } ... and no mention of these PROGRAM_CLIENT_EXIT_STATUS_* constants anywhere else in the code, in either version. Have I stumbled onto an omission in the packaged sources, or is there some other package missing on my system, or is there perhaps even some other explanation for what I'm seeing? I've checked the mailing list archive and found no mention of anyone else having the same (or similar) problem, so I do think it's possible that I'm just missing something ... Thanks in advance for any insight folks might be able to give me ... -- ---------------------------------------------------------------------- Sylvain Robitaille syl(a)encs.concordia.ca Systems analyst / AITS Concordia University Faculty of Engineering and Computer Science Montreal, Quebec, Canada ----------------------------------------------------------------------

1 1

GSSAPI mail home mapping problem
by Lucas Castro 10 Aug '21

10 Aug '21

Hello, I'm trying to map authenticated kerberos users to mail_location, The problem when I set mail_home = /var/mail/virtual/domain1.zw.loca/%n works fine. But if mail_home is set as /var/mail/virtual/%d/%n I get Apr 12 19:53:18 postfix10 dovecot: imap-login: Login: user=<user0>, method=GSSAPI, rip=172.16.0.44, lip=10.16.0.220, mpid=2428, session=<iPA4Yc6/lJCsEAAs> Apr 12 19:53:18 postfix10 dovecot: imap(user0(a)domain1.zw.local =login_user=user0)<2428><iPA4Yc6/lJCsEAAs>: Debug: Added userdb setting: plugin/=yes Apr 12 19:53:18 postfix10 dovecot: imap(user0(a)domain1.zw.local =login_user=user0)<2428><iPA4Yc6/lJCsEAAs>: Debug: Effective uid=5000, gid=5000, home=/var/mail/virtual/domain1.zw.local =login_user=user0/user0 Right here, I can't figure out why login_user=user0/user0 Apr 12 19:53:18 postfix10 dovecot: imap(user0(a)domain1.zw.local =login_user=user0)<2428><iPA4Yc6/lJCsEAAs>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/mail Now login_user=user0 Apr 12 19:53:18 postfix10 dovecot: imap(user0(a)domain1.zw.local =login_user=user0)<2428><iPA4Yc6/lJCsEAAs>: Debug: maildir++: root=/var/mail/virtual/domain1.zw.local =login_user=user0/user0/mail, index=, indexpvt=, control=, inbox=/var/mail/virtual/domain1.zw.local =login_user=user0/user0/mail, alt= then login_user=user0/user0/mail Apr 12 19:53:18 postfix10 dovecot: imap(user0(a)domain1.zw.local =login_user=user0)<2428><iPA4Yc6/lJCsEAAs>: Debug: Mailbox INBOX: Mailbox opened because: SELECT doveconf -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 5.10.0-7-amd64 x86_64 Debian 10.6 # Hostname: postfix10.zw.local auth_debug = yes auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/dovecot/imap.keytab auth_mechanisms = gssapi auth_verbose = yes disable_plaintext_auth = no import_environment = TZ KRB5CCNAME=/etc/dovecot/imap.ticket KRB5_KTNAME=/etc/dovecot/imap.keytab mail_debug = yes mail_gid = 5000 mail_home = /var/mail/virtual/%d/%n mail_location = maildir:~/mail mail_privileged_group = mail mail_uid = 5000 namespace inbox { disabled = no inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap lmtp pop3" service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = no ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } userdb { driver = prefetch } protocol lmtp { postmaster_address = postmaster(a)zw.local } egrep -v "^$|^#" /etc/dovecot/dovecot-ldap.conf.ext uris = ldap://ldap10.zw.local auth_bind = yes sasl_bind = yes sasl_mech = gssapi sasl_realm = zw.local debug_level = -1 ldap_version = 3 base = dc=zw,dc=local user_attrs = \ =user=%{ldap:mail} \ =login_user=%{ldap:uid} user_filter = (uid=%n) pass_attrs = \ =user=%{ldap:uid},\ =k5principals=%{ldap:krbPrincipalName} pass_filter = (&(objectClass=krbPrincipalAux)(uid=%n)) -- Lucas Castro

1 1

Replication stalled by failed attemts to read attachments (mail_attachment_dir)
by Ralf Becker 09 Aug '21

09 Aug '21

I'm migrating an older Dovecot 2.2.19 installation with dbox and mail_attachment_dir to 2.3.15 with replication to a second server. The storage from the old server was rsync'ed to a new server running Dovecot 2.3.15 in a container using similar configuration with dbox and mail_attachments. That servers alone seems to run fine. As a next step I tried to establish replication to the second empty server, which is configured to use mdbox and no mail_attachment_dir, as the rest of my Dovecot servers. Looking at "doveadm replicator status '*'" some of the mailboxes replicate correct, but roughly half of them fail :( Looking a the Dovecot log with mail_debug=yes shows the replication fails on the source (server with dbox and mail_attachment_dir) typically with one of two errors: a) attachment has different size: Aug 05 15:41:58 doveadm: Debug: Mailbox INBOX: UID 30: Looked up field date.received from mail cache Aug 05 15:41:58 doveadm: Error: dsync(cdaf50f7580e): read(attachments-connector(/var/dovecot/imap/<domain>/<user>/mailboxes/INBOX/dbox-Mails/u.30)) failed: read(/var/dovecot/imap/attachments/64/4e/644ecfbc09223b8ccafc6f36aa86603cde91e76716b9b5ed0cf875bc0e183b6b4581d671f8321647be99f26d2d22088e999714529e189b25a6e7d5b428fbf5d5-1a6b701e68c9c050b10a0000838cbfe1-f86a701e68c9c050b10a0000838cbfe1-30[base64:18 b/l]) failed: Stream is larger than expected (262658 > 262657, eof=1) (last sent=mail, last recv=mail_request (EOL)) showing on the other log as: Aug 05 15:49:18 doveadm: Debug: Mailbox INBOX: UID 29: Looked up field guid from mail cache Aug 05 15:49:19 doveadm: Error: dsync(142878d8d6bf): read() failed: read(10.44.99.1) failed: dot-input stream ends without '.' line (last sent=mail_request (EOL), last recv=mail) b) attachment is not found at all: Aug 05 15:43:58 doveadm(<user>@<domain> 1441): Debug: fs-sis: fs-sis: fs-posix: open(/var/dovecot/imap/attachments/e2/0a/e20ab165f3a2e7164520341cddb06cd053dc1a87fdd5c6360b45c20a2c9d33576c8ea2daa42062f9bb0fd94f9c5f9919ad86fb11e29effe3bdf4e853fa7c0e1d-20a0b63613e6c050620b0000838cbfe1-c8d5221c39a40b61250900007dc04144-3) failed: No such file or directory Aug 05 15:43:58 doveadm(<user>@<domain> 1441): Error: dsync(cdaf50f7580e): read(attachments-connector(/var/dovecot/imap/<domain>/<user>/mailboxes/INBOX/dbox-Mails/u.3)) failed: read(/var/dovecot/imap/attachments/e2/0a/e20ab165f3a2e7164520341cddb06cd053dc1a87fdd5c6360b45c20a2c9d33576c8ea2daa42062f9bb0fd94f9c5f9919ad86fb11e29effe3bdf4e853fa7c0e1d-20a0b63613e6c050620b0000838cbfe1-c8d5221c39a40b61250900007dc04144-3[base64:19 b/l]) failed: open(/var/dovecot/imap/attachments/e2/0a/e20ab165f3a2e7164520341cddb06cd053dc1a87fdd5c6360b45c20a2c9d33576c8ea2daa42062f9bb0fd94f9c5f9919ad86fb11e29effe3bdf4e853fa7c0e1d-20a0b63613e6c050620b0000838cbfe1-c8d5221c39a40b61250900007dc04144-3) failed: No such file or directory (last sent=mail, last recv=mail_request (EOL)) That caused the replication to fail and not newer mails to be replicated for the given user and mailbox, which can be verified eg. with "doveadm mailbox status -u <user>@<domain> all inbox". The concerned mails are pretty old / the attachment itself can not be recovered, and nothing can be done about them. But how can I repair the mailbox, so all unaffected mails / attachments get replicated? I already tried multiple "doveadm force-resync -u <user>@<domain> INBOX" in case a) with no visible effect / still same error. Both Dovecot configurations are attached, in case they matter ... Ralf -- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Geschäftsführer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0

2 4

Pigeonhole Sieve Config Warning
by dovecot＠ptld.com 09 Aug '21

09 Aug '21

In the sieve plugin settings i have plugin { ... sieve_user_email = %u } Variables pages says %u = full username (e.g. user@domain) This config seems to work in practice, the email is from: the users email address. However on starting dovecot im getting a warning twice: dovecot[35893]: managesieve: Warning: sieve: Invalid address value for setting `sieve_user_email': Missing domain dovecot[35893]: master: Dovecot v2.3.8 (9df20d2db) starting up for imap, lmtp, sieve dovecot[35897]: managesieve: Warning: sieve: Invalid address value for setting `sieve_user_email': Missing domain So is there really an issue? Or just an improper warning?

2 3

v2.3.16 released
by Timo Sirainen 09 Aug '21

09 Aug '21

Hi, One interesting thing in this release is the support for configuring OAUTH2 openid-configuration element. It would be nice if IMAP clients started supporting this feature to enable OAUTH2 for all IMAP servers, not just Gmail and a few others. This would allow all kinds of new authentication methods for IMAP and improve the authentication security in general. https://dovecot.org/releases/2.3/dovecot-2.3.16.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.16.tar.gz.sig Binary packages in https://repo.dovecot.org/ Docker images in https://hub.docker.com/r/dovecot/dovecot <https://hub.docker.com/r/dovecot/dovecot> * Any unexpected exit() will now result in a core dump. This can especially help notice problems when a Lua script causes exit(0). * auth-worker process is now restarted when the number of auth requests reaches service auth-worker { service_count }. The default is still unlimited. + Event improvements: Added data_stack_grow event and http-client category. See https://doc.dovecot.org/admin_manual/list_of_events/ + oauth2: Support RFC 7628 openid-configuration element. This allows clients to support OAUTH2 for any server, not just a few hardcoded servers like they do now. See openid_configuration_url setting in dovecot-oauth2.conf.ext. + mysql: Single statements are no longer enclosed with BEGIN/COMMIT. + dovecot-sysreport --core supports multiple core files now and does not require specifying the binary path. + imapc: When imap_acl plugin is loaded and imapc_features=acl is used, IMAP ACL commands are proxied to the remote server. See https://doc.dovecot.org/configuration_manual/mail_location/imapc/ + dict-sql now supports the "UPSERT" syntax for SQLite and PostgreSQL. + imap: If IMAP client disconnects during a COPY command, the copying is aborted, and changes are reverted. This may help to avoid many email duplicates if client disconnects during COPY and retries it after reconnecting. - master process was using 100% CPU if service attempted to create more processes due to process_min_avail, but process_limit was already reached. v2.3.15 regression. - Using attachment detection flags wrongly logged unnecessary "Failed to add attachment keywords" errors. v2.3.13 regression. - IMAP QRESYNC: Expunging UID 1 mail resulted in broken VANISHED response, which could have confused IMAP clients. v2.3.13 regression. - imap: STORE didn't send untagged replies for \Seen changes for (shared) mailboxes using INDEXPVT. v2.3.10 regression. - rawlog_dir setting would not log input that was pipelined after authentication command. - Fixed potential infinite looping with autoexpunging. - Log event exporter: Truncate long fields to 1000 bytes - LAYOUT=index: ACL inheritance didn't work when creating mailboxes - Event filters: Unquoted '?' wildcard caused a crash at startup - fs-metawrap: Fix to handling zero sized files - imap-hibernate: Fixed potential crash at deinit. - acl: dovecot-acl-list files were written for acl_ignore_namespaces - program-client (used by Sieve extprograms, director_flush_socket) may have missed status response from UNIX and network sockets, resulting in unexpected failures.

7 16

Problem migrating POP accounts into dovecot
by Jesús Ángel del Pozo Domínguez 09 Aug '21

09 Aug '21

Hello, I am trying to migrate mailboxes from an old pop/imap server to a new one using Dovecot 2.3.15. The old server uses tpop3d and courier as POP and IMAP servers, respectively. Whereas I can migrate IMAP mailboxes without many problems, I am facing difficulties while trying to do the same with POP ones. The problem is that dsync opens simultaneous connections to the remote POP server, so the remote mailbox gets locked and dsync can not copy the messages. This is the command I am using to migrate POP mailboxes: # doveadm -Dv -c dovecot-migration.conf -o pop3c_password="*******" backup -R -u test2(a)testemail.domain.com pop3c: And these are some interesting lines from the output of the above command: Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Looking up IP address Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Connecting to 10.0.0.104:110 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Authenticating as 'test2(a)testemail.domain.com' (with USER+PASS) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Mailbox INBOX: Adding field pop3.uidl to cache for the first time (uid=1) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Mailbox INBOX: UID 1: Opened mail because: virtual size (Cache file is unusable) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: quota: quota_over_flag check: STORAGE ret=3value=3 limit=104857600 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: quota: quota_over_flag check: MESSAGE ret=4value=1 limit=0 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: quota: quota_over_flag=0((null)) vs currently overquota=0 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: brain M: Locking done locally in /var/mail/testemail.domain.com/test2/.dovecot-sync.lock (local hostname=tremedal.domain.com, remote hostname=tremedal.domain.com) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: brain S: Namespace has location pop3c: Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Looking up IP address Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Connecting to 10.0.0.104:110 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Authenticating as 'test2(a)testemail.domain.com' (with USER+PASS) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Mailbox INBOX: Adding field pop3.uidl to cache for the first time (uid=1) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: brain S: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1628081654 uid_next=2 subs=no last_change=0 last_subs=0 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: brain M: Remote mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1628081654 uid_next=2 subs=no last_change=0 last_subs=0 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: brain M: Mailbox INBOX: local=00000000000000000000000000000000/0/0, remote=c92f64f79f0d1ed01e6d5b314f04886c/0/1: mailbox not selectable yet Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Namespace : /var/mail/testemail.domain.com/test2/Mailbox/inbox doesn't exist yet, using default permissions Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Namespace : Using permissions from /var/mail/testemail.domain.com/test2/Mailbox: mode=0700 gid=default Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Namespace : /var/mail/testemail.domain.com/test2/Mailbox/inbox doesn't exist yet, using default permissions Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Namespace : Using permissions from /var/mail/testemail.domain.com/test2/Mailbox: mode=0700 gid=default Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: brain S: Mailbox INBOX: local=c92f64f79f0d1ed01e6d5b314f04886c/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Looking up IP address Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Connecting to 10.0.0.104:110 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Authenticating as 'test2(a)testemail.domain.com' (with USER+PASS) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Mailbox INBOX: Adding field pop3.uidl to cache for the first time (uid=1) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: Pigeonhole version 0.5.15 (e6a84e31) initializing Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: file storage: Storage path `/var/mail/testemail.domain.com/test2/sieve' not found Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: file storage: Using active Sieve script path: /var/mail/testemail.domain.com/test2/.dovecot.sieve Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: file storage: Using script storage path: /var/mail/testemail.domain.com/test2/sieve Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: file storage: Using permissions from defaults: mode=0700 gid=-1 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: file storage: Created storage directory /var/mail/testemail.domain.com/test2/sieve/tmp Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: sieve: file storage: sync: Synchronization active Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: brain M: Import INBOX: Last common UID=0. Delayed expunges= Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: brain M: Import INBOX: Import change type=save GUID=1a82a18794d50d5de2b3f0faa8a6e27c UID=1 hdr_hash= result=Mail's UID is above local UIDNEXT - No more local mails found Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Mailbox INBOX: UID 1: Opened mail because: mail stream Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Mailbox INBOX: Adding field size.virtual to cache for the first time (uid=1) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Mailbox INBOX: Adding field size.physical to cache for the first time (uid=1) Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: Mailbox POP3-MIGRATION-NS/INBOX: Mailbox opened because: pop3_migration Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Looking up IP address Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Connecting to 10.0.0.104:110 Aug 04 14:54:14 dsync(test2(a)testemail.domain.com): Debug: pop3c(10.0.0.104): Authenticating as 'test2(a)testemail.domain.com' (with USER+PASS) Aug 04 14:54:22 dsync(test2(a)testemail.domain.com): Error: pop3_migration: Couldn't sync mailbox POP3-MIGRATION-NS/INBOX: Unable to open mailbox; it may be locked by another concurrent session. In the remote server I get these lines from the tpop3d log: Aug 4 14:54:14 lmail1 tpop3d[1887]: authcontext_new_user_pass: began session for `test2(a)testemail.domain.com' with mysql; uid 30018, gid 8 Aug 4 14:54:14 lmail1 tpop3d[1887]: fork_child: [6]test2(a)testemail.domain.com(10.0.0.105): successfully authenticated with mysql Aug 4 14:54:14 lmail1 tpop3d[2270]: mailspool_new_from_file: indexed mailspool /var/spool/mail/testemail.domain.com/test2 (2831 bytes) in 0.000s Aug 4 14:54:14 lmail1 tpop3d[2270]: connections_post_select: connection_read: client [6]test2(a)testemail.domain.com(10.0.0.105): closed connection Aug 4 14:54:14 lmail1 tpop3d[2270]: authcontext_delete: finished session for `test2(a)testemail.domain.com' with mysql Aug 4 14:54:14 lmail1 tpop3d[1887]: authcontext_new_user_pass: began session for `test2(a)testemail.domain.com' with mysql; uid 30018, gid 8 Aug 4 14:54:14 lmail1 tpop3d[1887]: fork_child: [5]test2(a)testemail.domain.com(10.0.0.105): successfully authenticated with mysql Aug 4 14:54:14 lmail1 tpop3d[2272]: mailspool_new_from_file: indexed mailspool /var/spool/mail/testemail.domain.com/test2 (2831 bytes) in 0.000s Aug 4 14:54:14 lmail1 tpop3d[2272]: connections_post_select: connection_read: client [5]test2(a)testemail.domain.com(10.0.0.105): closed connection Aug 4 14:54:14 lmail1 tpop3d[2272]: authcontext_delete: finished session for `test2(a)testemail.domain.com' with mysql Aug 4 14:54:14 lmail1 tpop3d[1887]: authcontext_new_user_pass: began session for `test2(a)testemail.domain.com' with mysql; uid 30018, gid 8 Aug 4 14:54:14 lmail1 tpop3d[1887]: fork_child: [5]test2(a)testemail.domain.com(10.0.0.105): successfully authenticated with mysql Aug 4 14:54:14 lmail1 tpop3d[2273]: mailspool_new_from_file: indexed mailspool /var/spool/mail/testemail.domain.com/test2 (2831 bytes) in 0.000s Aug 4 14:54:14 lmail1 tpop3d[1887]: authcontext_new_user_pass: began session for `test2(a)testemail.domain.com' with mysql; uid 30018, gid 8 Aug 4 14:54:14 lmail1 tpop3d[1887]: fork_child: [5]test2(a)testemail.domain.com(10.0.0.105): successfully authenticated with mysql Aug 4 14:54:22 lmail1 tpop3d[2274]: mailspool_new_from_file: failed to lock /var/spool/mail/testemail.domain.com/test2: Resource temporarily unavailable Aug 4 14:54:22 lmail1 tpop3d[2274]: dotfile_unlock(/var/spool/mail/testemail.domain.com/test2): lockfile does not have our PID Aug 4 14:54:22 lmail1 tpop3d[2274]: authcontext_delete: finished session for `test2(a)testemail.domain.com' with mysql Aug 4 14:54:22 lmail1 tpop3d[2273]: connections_post_select: connection_read: client [5]test2(a)testemail.domain.com(10.0.0.105): closed connection Aug 4 14:54:22 lmail1 tpop3d[2273]: authcontext_delete: finished session for `test2(a)testemail.domain.com' with mysql As you can see, in the first five lines the first connection from dsync is opened and closed. Then, the same sequence is repeated again (second connection from dsync). However, on opening the third connection, it is not closed before the fourth one is opened. On authenticating a user, the tpop3d server locks the mailbox to prevent other clients wirting to it. That is why the fourth connection fails as it can not get a lock for the mailbox: Aug 4 14:54:22 lmail1 tpop3d[2274]: mailspool_new_from_file: failed to lock /var/spool/mail/testemail.domain.com/test2: Resource temporarily unavailable Aug 4 14:54:22 lmail1 tpop3d[2274]: dotfile_unlock(/var/spool/mail/testemail.domain.com/test2): lockfile does not have our PID Could you please tell me how can I avoid this problem? I wonder why dsync opens so many connections to the remote server, and why it does not close the previous connections before opening new ones. Regards,

2 2

dsync and sieve script
by Andrea Gabellini 09 Aug '21

09 Aug '21

Hello, I'm testing the migration from and old dovecot server 2.2.24 to a new one with 2.3.15 using this command: doveadm -c /etc/dovecot/dovecot-migration.conf -o mail_fsync=never backup -x "shared" -x "shared/*" -x "dovecot/sieve" -R -T 60 -u andrea.gabellini.sc(a)telecomitalia.sm imapc: This command replicate email messages but not the sieve scripts. The esclusion of "dovecot/sieve" is mandatory because the old server is misconfigured and doesn't have a separate home and maildir location. Is dsync able to replicate the sieve scripts or I'm doing something wrong? Thanks, Andrea -- __________________________ The less a man knows the easier it is to convince him he knows it all. __________________________ TIM San Marino S.p.A. Andrea Gabellini Engineering R&D TIM San Marino S.p.A. - https://www.telecomitalia.sm Via Ventotto Luglio, 212 - Piano -2 47893 - Borgo Maggiore - Republic of San Marino Tel: (+378) 0549 886237 Fax: (+378) 0549 886188 -- Informativa Privacy Questa email ha per destinatari dei contatti presenti negli archivi di TIM San Marino S.p.A.. Tutte le informazioni vengono trattate e tutelate nel rispetto della normativa vigente sulla protezione dei dati personali (Reg. EU 2016/679). Per richiedere informazioni e/o variazioni e/o la cancellazione dei vostri dati presenti nei nostri archivi potete inviare una email a privacy(a)telecomitalia.sm. Avviso di Riservatezza Il contenuto di questa e-mail e degli eventuali allegati e' strettamente confidenziale e destinato alla/e persona/e a cui e' indirizzato. Se avete ricevuto per errore questa e-mail, vi preghiamo di segnalarcelo immediatamente e di cancellarla dal vostro computer. E' fatto divieto di copiare e divulgare il contenuto di questa e-mail. Ogni utilizzo abusivo delle informazioni qui contenute da parte di persone terze o comunque non indicate nella presente e-mail potra' essere perseguito ai sensi di legge.

2 1