Best mail encryption solution for per-user

[Aki Tuomi]

On 23.05.2018 10:15, mail at sjemm.net wrote:
> May 23, 2018 8:31 AM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>> On 23.05.2018 09:13, mail at sjemm.net wrote:
>>
>>> May 20, 2018 8:01 PM, mail at sjemm.net wrote:
>>>> May 20, 2018 2:47 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>>> On 19 May 2018 at 16:40 mail at sjemm.net wrote:
>>>
>>> May 18, 2018 10:01 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>>> On 18 May 2018 at 21:44 mail at sjemm.net wrote:
>>>
>>> May 18, 2018 4:43 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>>> On 18 May 2018 at 17:38 mail at sjemm.net wrote:
>>>
>>> May 18, 2018 4:05 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>>> On 18 May 2018 at 16:43 mail at sjemm.net wrote:
>>>
>>> Hi Tai74 and Aki,
>>> I followed your conversation with interest on how to setup per user encryption in dovecot.
>>> I have setup my dovecot with the following in a conf file:
>>>
>>> ==============
>>>
>>> mail_attribute_dict = file:%h/Maildir/dovecot-attributes
>>> mail_plugins = $mail_plugins mail_crypt
>>> plugin {
>>>
>>> mail_crypt_curve = secp521r1
>>>
>>> mail_crypt_save_version = 2
>>>
>>> }
>>>
>>> ==============
>>>
>>> This works nice, all emails are being encrypted and every user/folder has keys.
>>> But as I understood from your conversation these keys are not protected. And I want them to be
>>> protected by the users password used by imap.
>>>
>>> Those passwords are stored in a mysql DB file. ( I used a guide from workaround [dot] org to set up
>>> the DB and postfix/dovecot)
>>>
>>> but how would i set it so, that the users password from the DB is used to encrypt the keys?
>>>
>>> should i use mail_crypt_private_password = ?
>>> how do i point it to the mysql db then?
>>> im unsure about this
>>>
>>> Do you have any hints on this?
>>>
>>> Kind regards,
>>> Zjemm
>>>
>>> The passwords in your MySQL database are, hopefully, not in plaintext. If you want to secure your
>>> user's keys using user's login password, you must have a TOOL that manages this.
>>>
>>> You can use mail_crypt_private_password = %w in (mysql) passdb fields to provide the user's login
>>> password as private password. You might want to run it thru some hash, so %{sha1:password} might be
>>> a good option.
>>>
>>> You can change the key password using 'doveadm mailbox cryptokey', this needs to be done every time
>>> user changes his password.
>>>
>>> Also note that if you go down this road, and the user forgets his password, you will not be able to
>>> recover the emails without backup copy of the private key.
>>>
>>> Aki
>>>
>>> Hi Aki
>>>
>>> I used the following command:
>>> dovecot pw -s SHA256-CRYPT
>>>
>>> the output on the chosen password looks like: {SHA256-CRYPT}$5$Rokc06a7In4SF3bO$OQpGQWqg........
>>>
>>> This output is used to store in the password fields in the database. So no plain text passwords no
>>> :)
>>>
>>> You can use mail_crypt_private_password = %w in (mysql) passdb fields to provide the user's login
>>> password as private password.
>>>
>>> can you explain this a bit more for me?
>>>
>>> for now i have in the 10-auth.conf file the following:
>>> ==============
>>> passdb {
>>> driver = sql
>>>
>>> # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
>>> args = /etc/dovecot/dovecot-sql.conf.ext
>>> }
>>>
>>> and:
>>>
>>> userdb {
>>> driver = static
>>> args = uid=vmail gid=vmail home=/var/vmail/%d/%n
>>> }
>>> ==============
>>>
>>> then i have in dovecot-sql.conf.ext
>>> ==============
>>> driver = mysql
>>> connect = host=x.x.x.x dbname=mailserver user=mailuser password=mailpasswordexample
>>> default_pass_scheme = SHA256-CRYPT
>>> password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';
>>> ==============
>>> Where do i need to set : mail_crypt_private_password = %w ?
>>>
>>> password as private password. You might want to run it thru some hash, so %{sha1:password} might be
>>> a good option.
>>>
>>> the passwords are allready hashed in the DB using: dovecot pw -s SHA256-CRYPT to genereate the has.
>>> so this step isnt nesesary anymore am i right?
>>>
>>> Thank you for your quick response, very helpfull
>>>
>>> Zjemm
>>>
>>> You misunderstood a bit. The idea is to use the *plaintext* password as the password for the
>>> private key. Otherwise anyone could just decrypt it by looking at your database where the hashed
>>> password is..
>>>
>>> So:
>>>
>>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
>>> virtual_users WHERE email='%u'
>>>
>>> Aki
>>>
>>> Hi Aki,
>>>
>>> Thank you very much for your help, i realy appreciate that.
>>>
>>> Ok so if i understand it correctly i'll have to use:
>>>
>>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
>>> virtual_users WHERE email='%u'
>>>
>>> in my dovecot-sql.conf.ext file
>>>
>>> This query selects the user, the password, and %w
>>>
>>> if i run a little query myself:
>>> MariaDB [mailserver]> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password
>>> FROM virtual_users;
>>> +------------------+----------------------------------+------------------------------------+
>>> | user | password | userdb_mail_crypt_private_password |
>>> +------------------+----------------------------------+------------------------------------+
>>> | john at example.org | {SHA256-CRYPT}$5$M/GWzmtjsLroRWI | %w |
>>> +------------------+----------------------------------+------------------------------------+
>>>
>>> %w is a dovecot variable, and stands for the plaintext password, but the password is not stored as
>>> plaintext in the DB, %w get filled with the actual plaintext password by dovecot upon the user that
>>> is typing in the password when authenticating.
>>>
>>> is this correct?
>>>
>>> yes.
>>>
>>> so then i have the username the hashed password en the plaintext password as a result of the query.
>>>
>>> yes
>>>
>>> now userdb_mail_crypt_private_password = the plaintext password
>>> do i need to reference it somewhere? or is userdb_mail_crypt_private_password autmatically used by
>>> the dovecot mail_crypt plugin to encrypt the keys? or should it be mail_crypt_private_password?
>>>
>>> It gets injected into the mail process as 'mail_crypt_private_password', as if it was set in plugin
>>> {} section.
>>>
>>> if i have this setup working i'm going to write a blog post on this topic to share this knowledge
>>>
>>> Thanks again and have a great weekend.
>>>
>>> Zjemm
>>>
>>> Aki
>>>
>>> Hi Aki,
>>>
>>> Cool i'm testing it right now.
>>> I have set up a new mailserver (life is great with lxc containers :) )
>>>
>>> postfix and dovecot are working like normal
>>>
>>> next i enable mail_crypt
>>>
>>> i did create a file: /etc/dovecot/conf.d/10-mailcrypt.conf
>>> ==========================
>>> mail_attribute_dict = file:%h/Maildir/dovecot-attributes
>>>
>>> mail_plugins = $mail_plugins mail_crypt
>>>
>>> plugin {
>>> mail_crypt_curve = secp521r1
>>> mail_crypt_save_version = 2
>>> }
>>> ==========================
>>>
>>> and then i changed the file: /etc/dovecot/dovecot-sql.conf.ext
>>>
>>> so the query is now the new query:
>>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
>>> virtual_users WHERE email='%u';
>>>
>>> then i restarted dovecot and postfix and send a test email to the one and only testuser that is in
>>> there.
>>>
>>> when i open the mailbox with the tool mutt, i can see the new email, and when openening the email
>>> the mutt client drops the connection.
>>>
>>> in the log i can see:
>>>
>>> May 19 13:34:48 mailserver1.example.local dovecot[600]: imap-login: Login: user=<john at example.org>,
>>> method=PLAIN, rip=::1, lip=::1, mpid=713, TLS, session=<E3PnIY9sNM4AAAAAAAAAAAAAAAAAAAAB>
>>> May 19 13:34:49 mailserver1.example.local dovecot[600]: imap(john at example.org): Error: read()
>>> failed:
>>> read(/var/vmail/example.org/john/Maildir/cur/1526736378.M161472P641.mailserver1.example.local,S=559,
>>> =571:2,) failed: Private key not available: Cannot decrypt key
>>> bfc5bb25b1bf64290eea6dc14b516c6a0a25b64551b6e4f0f8677ba7274887cb: error:03070068:bignum
>>> routines:BN_mpi2bn:encoding error (FETCH BODY[] for mailbox INBOX UID 8)
>>>
>>> i think i missed a step, but witch one?
>>>
>>> the userpassword hasnt been changed (that would be the next step in the testing process)
>>>
>>> should i've use doveadm first to encrypt the key with that userpassword? i thought it would do that
>>> on the fly, because the initial keys where only just created when enabling the mail_crypt plugin
>>>
>>> please let me know your thougts
>>> Zjemm
>>> I noticed you replied directly to me, and not to the list, too... fixed that for you.
>>>
>>> mail_crypt_private_password is used when key is created, but if you have created it before using
>>> password, you'll need to encrypt it before turning the setting on.
>>>
>>> Aki
>>>> Hi, Ok i'm sorry about that, thank you for fixing that. i'm a bit new to mailing lists :)
>>>>
>>>> are you a dovecot devoloper? or a enthusiastic user?
>>>>
>>>> anyway, i managed to get it working now with all your help :)
>>>>
>>>> to start fresh i did a:
>>>> rm -rf /var/vmail/example.org
>>>>
>>>> so all email is gone, and i assume all dovecot keys are gone that way.
>>>>
>>>> Then i tried to set a password first:
>>>>
>>>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
>>>> result: Changed password for 0 key(s)
>>>>
>>>> (summersun = the password for this user)
>>>>
>>>> but then it says there are 0 keys, so then i send the test user an email and try to open the
>>>> mailbox, i'll get:
>>>>
>>>> imap(john at example.org): Error: read() failed:
>>>> read(/var/vmail/example.org/john/Maildir/cur/1526827992.M149712P883.mailserver1.example.local,S=549,
>>>> =561:2,) failed: Private key not available: Cannot decrypt key
>>>> 71849013a70b0c631c06112077e7c2fe39b0b2737b4933b219793841209d4e7f: error:03070068:bignum
>>>> routines:BN_mpi2bn:encoding error
>>>>
>>>> but hey, now the key's are created, so i tried to set the password again:
>>>>
>>>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
>>>> result: dcrypt_key_load_private(787701bd677dd69a26842547a3926cbaa625b0b5a91751f06678c3e9708343d9)
>>>> failed: password missing
>>>> =========
>>>>
>>>> Lets start over again, i did a new:
>>>> rm -rf /var/vmail/example.org
>>>>
>>>> i disabled the mail_crypt_private_password in the sql query and send an email to the test user.
>>>> now again new keys are generated.
>>>>
>>>> then i did set the password via doveadm:
>>>>
>>>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
>>>> result: Changed password for 1 key(s)
>>>>
>>>> i enabled the mail_crypt_private_password again in the sql query part and now it works :)
>>>>
>>>> YESSS!!!
>>>>
>>>> if i now change the password for the user, i cannot read the email anymore. that is what should
>>>> happen.
>>>> so now i changed the password for this user also with doveadm:
>>>> - doveadm mailbox cryptokey password -u john at example.org -n qwerty -o summersun
>>>>
>>>> and now i'm able to read the emails again. so it all works now as expected.
>>>>
>>>> i have now only one thing remaining.
>>>>
>>>> when i want to add a new user, i have to somehow:
>>>> - create the new user in the mysql DB
>>>> - start with dovecot not using mail_crypt_private_password (so disabled)
>>>> - send an initial email to let the users mailbox be created and generate the keys
>>>> - set the users password with doveadm
>>>> - restart dovecot again with mail_crypt_private_password (so enabled)
>>>>
>>>> but this would be disruptive to existing users, and i would have to script the whole thing as a
>>>> "add-new-user" script to make it usable i guess.
>>>>
>>>> do you know of a more easy way of acomplishing this?
>>>> could i use doveadm for all steps, and get around the mail_crypt_private_password disable part?
>>>>
>>>> or could i add a column in the mysql db called userdb_mail_crypt_private_password and fill that
>>>> column with %w for users that allready exist, and empty for new users.
>>>> when being empty, it might not set the mail_crypt_private_password for new users
>>>>
>>>> or is this the wrong way of thinking?
>>>>
>>>> let me know your thoughts on this.
>>>>
>>>> And last but not least, i really want to thank you for all your help. It is a really nice
>>>> experience.
>>>> Bless you!
>>>>
>>>> Zjemm
>>> Hi Aki,
>>>
>>> You did fix the posting on the mail archive, i can see our conversation on there, all but the last
>>> email i send to you.
>>> Maybe you did not received the last email?
>>>
>>> would you be willing to take a look at my last question in the email See below:
>>> Thank you very much.
>>>
>>> ================================
>>> i have now only one thing remaining.
>>>
>>> when i want to add a new user, i have to somehow:
>>> - create the new user in the mysql DB
>>> - start with dovecot not using mail_crypt_private_password (so disabled)
>>> - send an initial email to let the users mailbox be created and generate the keys
>>> - set the users password with doveadm
>>> - restart dovecot again with mail_crypt_private_password (so enabled)
>>>
>>> but this would be disruptive to existing users, and i would have to script the whole thing as a
>>> "add-new-user" script to make it usable i guess.
>>>
>>> do you know of a more easy way of acomplishing this?
>>> could i use doveadm for all steps, and get around the mail_crypt_private_password disable part?
>>>
>>> or could i add a column in the mysql db called userdb_mail_crypt_private_password and fill that
>>> column with %w for users that allready exist, and empty for new users.
>>> when being empty, it might not set the mail_crypt_private_password for new users
>>>
>>> or is this the wrong way of thinking?
>>>
>>> let me know your thoughts on this.
>>>
>>> And last but not least, i really want to thank you for all your help. It is a really nice
>>> experience.
>>> Bless you!
>>>
>>> Zjemm
>> If you have mail_crypt_private_password set *when* keys are generated,
>> dovecot should use that password to encrypt the user key.
>>
>> Aki
>
> Hi Aki,
>
> well, if there is a new email user setup in the mysql DB, and no email has been send to that new user. at that point there are no keys.
>
> So trying to set a password on those key's wont work. see:
> root at mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey password -u john2 at example.org -n qwerty
> result: Changed password for 0 key(s)
>
> the keys need to be generated first. And that only happens when sending a mail to that new user. Also, at the point of creating the keys, the mail_crypt_private_password should be disabled otherwise the keys are generated and also encrypted with no user password, and then im unable to set a new pasword for that key
>
>
> How can i set the password before the keys are generated?
> or can i manually generate key's and then set the password?
>
> so at this point i have to do the following:

You can use doveadm mailbox cryptokey generate -u user -U

to generate a user key.

Aki

>>> - create the new user in the mysql DB
>>> - start with dovecot not using mail_crypt_private_password (so disabled)
>>> - send an initial email to let the users mailbox be created and generate the keys
>>> - set the users password with doveadm
>>> - restart dovecot again with mail_crypt_private_password (so enabled)
> can i somehow enable the mail_crypt_private_password per user?
> because this mysql query always returns: %w 
> and thus allways sets: userdb_mail_crypt_private_password
> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM virtual_users;