- dovecot - dovecot.org

doveadm sync backup from old to new server
by Gregory Sloop 15 May '20

15 May '20

So I've done quite a lot of searching on the list and on the web - and perhaps my google-fu is really bad - but I can't find any real recipes on how to sync mail from the old server to the new. As an FYI - the old server is a CPanel/WHM setup on a VPS. The new is mailcow - which uses docker. However, I don't think either of these platforms is what's causing the issue - but I'm certainly not sure of that. --- I've tried several things - but have lost track of all the things I've tried. This seemed like the best of all the things I've tried. This particular mailbox/user+domain is setup on both servers. doveadm backup -D -u mc-user(a)abc.net ssh root(a)abc.net -p2200 doveadm dsync-server -u cp-user(a)abc.net mc-user(a)abc.net is the MC/NEW mailbox/domain cp-user(a)abc.net is the CPanel/OLD user/domain account The SSH server of the remote system is running on port 2200. However when I try this, I get: WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set. Defaulting to a blank string. dsync-local(user(a)abc.net)<v44NFyRWvF5efgAAb6OASA>: Error: read(remote) failed: EOF (version not received) doveadm(user(a)abc.net): Fatal: execvp(ssh) failed: No such file or directory The process appears to hang, and a Ctrl+C stops it. I'd love to get pointed at a reasonable recipe on how to make this work. I don't really get/understand the docs much at all. [And either everyone else understands it just fine, and never thinks to write a document on how to do it - or, and I think this is a lot more likely - they're using something like imapsync to do it. I found numerous places where others were, essentially, "Dovecot's tool is way too complicated and I can't get it to work right, so I used imapsync." I suppose I should probably just do that too, but it does seem a shame to do that when the dovecot tool is almost certainly the best tool for the job, but I can't figure out how to use it.] If someone can help me grok what's going on, I'm glad to write it up for the list and or a blog entry so it's more accessible. TIA -Greg

3 14

Can't connect to Managesieve Server - what´s wrong?
by Anton Blau 15 May '20

15 May '20

Hi, please excuse my bad english language. But I am not a native speaker. I have installed dovecot including sieve and managesieve on a ubuntu 18.04 lts. When I try to access the filter with roundcube (from another vm) I get the error message "Cannot connect to Managesieve-Server". If I try a "telnet myserver.dynvpn.de 4190" from the roundcube-vm I get Trying 188.194.95.63... Connected to myserver.dynvpn.de. Escape character is '^]'. "IMPLEMENTATION" "Dovecot (Ubuntu) Pigeonhole" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparat or-i;ascii-numeric relational regex imap4flags copy include variables body enoti fy environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve" "NOTIFY" "mailto" "SASL" "" "STARTTLS" "VERSION" "1.0" OK "Dovecot (Ubuntu) ready." So I think that dovecot managesieve is running. In /var/log/syslog I get with sieve: ... May 13 23:56:04 firstmail dovecot: imap(debian(a)firstmail.locallan): Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so ... When I try with telnet I get: ... May 14 00:00:30 firstmail dovecot: managesieve-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges May 14 00:00:30 firstmail dovecot: managesieve-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges ... What can I do to get connect from Roundcube to dovecot-managemsieve? Thank you! Tony ---> 20-imap.conf protocol imap { mail_plugins = $mail_plugins imap_sieve } ---> 20-lmtp.conf protocol lmtp { postmaster_address = administrator@locallan # required mail_plugins = $mail_plugins sieve } ---> 20-managesieve.conf protocols = $protocols sieve service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { # Max. number of ManageSieve processes (connections) process_limit = 1024 } protocol sieve { } ---> 90-sieve.conf plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_before = /var/vmail/sieve/global/spam-global.sieve sieve = file:/var/vmail/sieve/%u;active=/var/vmail/sieve/%u/active-script.sieve sieve_default = /var/vmail/sieve/default/default.sieve sieve_global = /var/vmail/sieve/global/ imapsieve_mailbox1_name = Spam imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_before = file:/var/vmail/sieve/global/report-spam.sieve imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/var/vmail/sieve/global/report-ham.sieve sieve_pipe_bin_dir = /usr/bin sieve_global_extensions = +vnd.dovecot.pipe } ---> 90-sieve-extprograms.conf plugin { }

3 6

[Dovecot v2.3.9.3] HTTP API Endpoint for mailbox cryptokey operations
by Thibault HUTTIN-PASSERON 14 May '20

14 May '20

Hello everyone, I successfully set up the mail_crypt plugin using folder keys, and require user's key to be encrypted with a password using mail_crypt_require_encrypted_user_key = yes. As I'm trying to streamline the process of creating a user, and want to develop an application in PHP to help me in that process, I'm very interested in the doveadm HTTP API. Although the mail_crypt plugin is not referenced in the API, I discovered that it could be accessed all the same, provided some camelCase changes. So far I've been able to generate the following json for the mailbox cryptokey generate command: curl -X POST -H "Authorization: X-Dovecot-API <base64-api-token>" -H "Content-Type: application/json" -d '[["mailboxCryptokeyGenerate",{"user":"<username>","userKeyOnly":true},"tag1"]]' http://localhost:8081/doveadm/v1 which is able to give me the same output as the related command: doveadm mailbox cryptokey generate -u <username> -U. This only works if a keypair has already been generated for the user, which is kind of counter-productive in my use case. I would like to know if there was a way to adapt the following command into a json API call : doveadm -o plugin/mail_crypt_private_password=<password> mailbox cryptokey generate -u <username> -U Especially the -o plugin/mail_crypt_private_password=<password> part, since it seems I've figured out the other part partly tinkering partly looking through the source code. Thanks everyone for your input. Thibault PASSERON PS : excerpts from my configuration : doveconf -n =========== # 2.3.9.3 (9f41b88fa): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.9 (db4e9a2f) # OS: Linux 5.6.7-300.fc32.x86_64 x86_64 Fedora release 32 (Thirty Two) # Hostname: <hostname> auth_mechanisms = plain login dict { sqldict = mysql:/etc/dovecot/dovecot-sql-dict.conf.ext } doveadm_api_key = # hidden, use -P to show it mail_attribute_dict = proxy:dict:sqldict mail_plugins = quota mail_crypt passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_crypt_curve = secp521r1 mail_crypt_require_encrypted_user_key = yes mail_crypt_save_version = 2 } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service doveadm { inet_listener http { port = 8081 } unix_listener doveadm-server { } } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota mail_crypt sieve } dovecot-sql.conf.ext ==================== driver = mysql connect = "host=/var/lib/mysql/mysql.sock user=<user> password=<pass> dbname=<db>" #SQL Query used to fetch the user from the DB user_query = SELECT email as user, \ CONCAT('*:bytes=',quota) AS quota_rule, \ '/var/spool/mail/virtual_mailboxes/%d/%n' as home, \ 5000 AS uid, \ 5000 AS gid, \ crypt AS mail_crypt_save_version \ FROM virtual_users WHERE email='%u'; #SQL Query used to fetch the user's password from the DB password_query = SELECT password, \ crypt AS userdb_mail_crypt_save_version, \ '%w' AS userdb_mail_crypt_private_password \ FROM virtual_users WHERE email='%u'; #SQL Query to iterate over all users iterate_query = SELECT email AS user FROM virtual_users; dovecot-sql-dict.conf.ext ========================= #driver = mysql connect = "host=/var/lib/mysql/mysql.sock user=<user> password=<pass> dbname=<db>" # User's private keys map { pattern = priv/$mailboxid/vendor/vendor.dovecot/pvt/crypt/privkeys/$digest table = virtual_users_cryptokeys_priv_dict username_field = email value_field = key_contents fields { mailbox_id = $mailboxid key_digest = $digest } } map { pattern = priv/$mailboxid/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/pvt/crypt/privkeys/$digest table = virtual_users_cryptokeys_priv_server_dict username_field = email value_field = key_contents fields { mailbox_id = $mailboxid key_digest = $digest } } #User's active private key map { pattern = priv/$mailboxid/vendor/vendor.dovecot/pvt/crypt/active table = virtual_users_cryptokeys_priv_active_dict username_field = email value_field = key_digest fields { mailbox_id = $mailboxid } } map { pattern = priv/$mailboxid/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/pvt/crypt/active table = virtual_users_cryptokeys_priv_active_server_dict username_field = email value_field = key_digest fields { mailbox_id = $mailboxid } } #Public keys map { pattern = shared/$mailboxid/vendor/vendor.dovecot/pvt/crypt/pubkeys/$digest table = virtual_users_cryptokeys_shared_dict value_field = key_contents fields { mailbox_id = $mailboxid key_digest = $digest } } map { pattern = shared/$mailboxid/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/pvt/crypt/pubkeys/$digest table = virtual_users_cryptokeys_shared_server_dict value_field = key_contents fields { mailbox_id = $mailboxid key_digest = $digest } } #Active public keys map { pattern = shared/$mailboxid/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/pvt/crypt/active table = virtual_users_cryptokeys_shared_active_server_dict value_field = key_digest fields { mailbox_id = $mailboxid } } map { pattern = shared/$mailboxid/vendor/vendor.dovecot/pvt/crypt/active table = virtual_users_cryptokeys_shared_active_dict value_field = key_digest fields { mailbox_id = $mailboxid } }

1 0

Support for MULTISEARCH
by Daniel Miller 14 May '20

14 May '20

Does Dovecot presently support the MULTISEARCH command, or are there plans to do so? --- Daniel

6 12

Re: doveadm sync backup from old to new server
by Aki Tuomi 14 May '20

14 May '20

> On 14/05/2020 09:02 Adi Pircalabu <adi(a)ddns.com.au> wrote: > > > On 14-05-2020 15:16, Aki Tuomi wrote: > >> On 14/05/2020 04:09 Gregory Sloop <gregs(a)sloop.net> wrote: > >> > >> > >> So I've done quite a lot of searching on the list and on the web - and > >> perhaps my google-fu is really bad - but I can't find any real recipes > >> on how to sync mail from the old server to the new. > >> > >> As an FYI - the old server is a CPanel/WHM setup on a VPS. > >> The new is mailcow - which uses docker. > >> > >> However, I don't think either of these platforms is what's causing > >> the issue - but I'm certainly not sure of that. > >> > >> --- > >> I've tried several things - but have lost track of all the things > >> I've tried. > >> This seemed like the best of all the things I've tried. > >> > >> This particular mailbox/user+domain is setup on both servers. > >> > >> doveadm backup -D -u mc-user(a)abc.net ssh root(a)abc.net -p2200 doveadm > >> dsync-server -u cp-user(a)abc.net > >> > >> mc-user(a)abc.net is the MC/NEW mailbox/domain > >> cp-user(a)abc.net is the CPanel/OLD user/domain account > >> The SSH server of the remote system is running on port 2200. > >> > >> However when I try this, I get: > >> WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set. Defaulting to > >> a blank string. > >> dsync-local(user(a)abc.net)<v44NFyRWvF5efgAAb6OASA>: Error: > >> read(remote) failed: EOF (version not received) > >> doveadm(user(a)abc.net): Fatal: execvp(ssh) failed: No such file or > >> directory > >> > >> The process appears to hang, and a Ctrl+C stops it. > >> > >> I'd love to get pointed at a reasonable recipe on how to make this > >> work. > >> > >> I don't really get/understand the docs much at all. > >> [And either everyone else understands it just fine, and never thinks > >> to write a document on how to do it - or, and I think this is a lot > >> more likely - they're using something like imapsync to do it. I found > >> numerous places where others were, essentially, "Dovecot's tool is way > >> too complicated and I can't get it to work right, so I used imapsync." > >> I suppose I should probably just do that too, but it does seem a shame > >> to do that when the dovecot tool is almost certainly the best tool for > >> the job, but I can't figure out how to use it.] > >> > >> If someone can help me grok what's going on, I'm glad to write it up > >> for the list and or a blog entry so it's more accessible. > >> > >> TIA > >> -Greg > >> > > > > Just thought I'd point out that if you want to use ssh, it should be > > installed on both ends: > > > > "doveadm(user(a)abc.net): Fatal: execvp(ssh) failed: No such file or > > directory" > > > > If you don't want to, or can't install ssh, you can also use doveadm > > server: > > > > service doveadm { > > inet_listener { > > port = 12534 > > } > > } > > > > doveadm_port = 12534 > > doveadm_password = secretpass > > > > Then you can use tcp:otherhost:12534 as target/source. > > Hi Aki, > Does this service provide encryption in transit? > > -- > Adi Pircalabu You can use tcps if you enable SSL for doveadm listener. Add ssl=yes after port. Please keep replies on list. Aki

1 0

last_login_dict: Failed to write value: dict-server returned failure: sql dict: commit failed
by Nicolas DEFFAYET 12 May '20

12 May '20

Hello, I get last_login_dict: Failed to write value: dict-server returned failure: sql dict: commit failed randomly in the logs: --- 1 mailserver dovecot: imap(test-email-bb(a)exemple.com) <NtiT9H2SFIVcuGID> [xxx.xxx.xxx.xxx]: Error: last_login_dict: Failed to write value: dict-server returned failure: sql dict: commit failed: INSERT, UPDATE command denied to user 'dovecot'@'sqlserver' for table 'dovecot_last_login' (reply took 0.003 secs (0.000 in dict wait, 0.003 in other ioloops, 0.000 in locks, async-id reply 0.002 secs ago, started on dict-server 0.003 secs ago, took 0.003 secs)) --- The goal is to keep the unique couple email and ip_address in the table of SQL database and update only timestamp if email and ip_address exist on same row. I have enabled auth_debug and mail_debug but it don't show last_login_dict SQL queries in the log (auth SQL queries are shown). So I have enabled SQL debug on SQL server. In the SQL server log: --- 190918 13:49:59 3 Connect dovecot@sqlserver as anonymous on mail 4 Connect dovecot@sqlserver as anonymous on mail 4 Query BEGIN 4 Query INSERT INTO dovecot_last_login (timestamp,email,ip_address) VALUES (1568814599,'test-email-cc@exemple. com','abcd:aabb:cc:ddd::e') ON DUPLICATE KEY UPDATE timestamp=1568814599 5 Connect dovecot@sqlserver as anonymous on mail 4 Query COMMIT --- In the Dovecot mail server log: --- 1 mailserver dovecot: imap(test-email-bb(a)exemple.com) <NtiT9H2SFIVcuGID> [xxx.xxx.xxx.xxx]: Error: last_login_dict: Failed to write value: dict-server returned failure: sql dict: commit failed: INSERT, UPDATE command denied to user 'dovecot'@'sqlserver' for table 'dovecot_last_login' (reply took 0.003 secs (0.000 in dict wait, 0.003 in other ioloops, 0.000 in locks, async-id reply 0.002 secs ago, started on dict-server 0.003 secs ago, took 0.003 secs)) --- In the SQL server log: --- 17 Query BEGIN 17 Query INSERT INTO dovecot_last_login (timestamp,email,ip_address) VALUES (1568816073,'test-email-aa@exemple. com','abcd:aabb:cc:ddd::e') ON DUPLICATE KEY UPDATE timestamp=1568816073 17 Query COMMIT 19 Query BEGIN 19 Query INSERT INTO dovecot_last_login (timestamp,email,ip_address) VALUES (1568816073,'test-email-cc@exemple. com','abcd:aabb:cc:ddd::e') ON DUPLICATE KEY UPDATE timestamp=1568816073 19 Query COMMIT 20 Query BEGIN 20 Query INSERT INTO dovecot_last_login (timestamp,email,ip_address) VALUES (1568816073,'test-email-bb@exemple. com','abcd:aabb:cc:ddd::e') ON DUPLICATE KEY UPDATE timestamp=1568816073 20 Query COMMIT --- Database contain: MariaDB [mail]> SELECT * FROM mail.dovecot_last_login WHERE ip_address='abcd:aabb:cc:ddd::e'; +----------------------------+---------------------+------------------- --+------------+ | email | ip_address | datetime | timestamp | +----------------------------+---------------------+------------------- --+------------+ | test-email-cc(a)exemple.com | abcd:aabb:cc:ddd::e | 2019-09-18 14:14:33 | 1568816073 | | test-email-aa(a)exemple.com | abcd:aabb:cc:ddd::e | 2019-09-18 14:14:33 | 1568816073 | | test-email-bb(a)exemple.com | abcd:aabb:cc:ddd::e | 2019-09-18 14:14:33 | 1568816073 | +----------------------------+---------------------+------------------- --+------------+ 3 rows in set (0.00 sec) MariaDB [mail]> The SQL request executed directly on SQL server works: MariaDB [mail]> INSERT INTO dovecot_last_login (timestamp,email,ip_address) VALUES (1568815883,'test-email-aa@exemple. com','abcd:aabb:cc:ddd::e') ON DUPLICATE KEY UPDATE timestamp=1568815883; MariaDB [mail]> INSERT INTO dovecot_last_login (timestamp,email,ip_address) VALUES (1568815883,'test-email-bb@exemple. com','abcd:aabb:cc:ddd::e') ON DUPLICATE KEY UPDATE timestamp=1568815883; Did you have any tips how fix this issue ? Many thanks Best Regards, -- Nicolas DEFFAYET

1 0

Compress by zlib only selected Maildir folders
by filbar＠centrum.cz 12 May '20

12 May '20

Hello, is it possible to compress by zlib only selected folders? My e-mail accounts have archive folder and this folder contains old comunications with my customers and this I use very rare, so I want to compress them by zlib to save space. Is this possible? Thanks, Filip Bartmann

2 1

Deleting messages from filesystem with sdbox mail format
by Gionatan Danti 11 May '20

11 May '20

Dear list, I recently had to delete some messages from an sdbox mailbox, for which I used the canonical "doveadm expunge" command without issues. However I got curious and, on a test mailbox, I deleted an unimportant message directly from the filesystem (rm u.650). After removing the file, the webmail continued to show it in the mail list - but no text was retrieved when I selected it. I supposed this was due to dovecot.index.cache and, indeed, deleting the cache got rid of the message in the webmail list. While I understand that it is not recommended (deleting and regenerating the index cache is an expensive operation), I wonder if removing a file from the filesystem, and regenerating the index cache file, is a safe operation when using an sdbox mailstore. Specifically, I would like to know if, after a manual filesystem deletion, the main index file (dovecot.index) is automatically updated by dovecot (reflecting the deletion) or if lingering references are kept in it. Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it [1] email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

2 6

Re: [EXT] Support for MULTISEARCH
by Sami Ketola 11 May '20

11 May '20

> On 10. May 2020, at 11.22, Peter <pch(a)myzel.net> wrote: > > Am 10.05.20 um 09:54 schrieb Sami Ketola: >>> On 10. May 2020, at 1.51, Peter <pch(a)myzel.net> wrote: >>> Am 10.05.20 um 00:22 schrieb Daniel Miller: >>>> Thank you - I'm aware of the virtual folder option and do use it. My interest is for a Windows client, EM Client, which I otherwise really enjoy. Unfortunately, they've implemented server-side searching only via MULTISEARCH - for reasons passing my understanding. So I was hoping to hear Dovecot either already had support or there were plans to implement it. >>> >>> Virtual folder does not scale. Thank you for naming a client that does multisearch! >> Virtual folder scales just fine. What makes you think it does not? >> We have customers that have users with thousand folders and millions of emails and still virtual folder scales. > > Hello Sami, I was thinking, that the client then will be overwhelmed with a mailbox of such a size? > > Or does the folder start empty and only gets populated upon search with the resulting mails? virtual folder with contents of all actual folders is usually hidden from imap clients and is used only for searching. Sami

1 0

Directory hashing
by lists＠mlserv.org 11 May '20

11 May '20

Hi, I struggle with directory hashing. I want something like this: /srv/mail/c/cf37a8dff5e360927ba10ab2 The final folder is simpel, as it is: %{sha256;truncate=96:user} But how do I get a first level from sha256? Unfortunately, the truncate option aligns only full 8bit and does not divide into low and high nibbles. How can I express this for sha256? in MD5 this would be %1Mu Many thanks in advance Christian -- Rößner-Network-Solutions Karl-Bröger-Str. 10, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

3 4