Best mail encryption solution for per-user

Wed May 23 09:31:04 EEST 2018

On 23.05.2018 09:13, mail at sjemm.net wrote:
> May 20, 2018 8:01 PM, mail at sjemm.net wrote:
>> May 20, 2018 2:47 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>>
>>>> On 19 May 2018 at 16:40 mail at sjemm.net wrote:
>>>>
>>>> May 18, 2018 10:01 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>>>> On 18 May 2018 at 21:44 mail at sjemm.net wrote:
>>>>
>>>> May 18, 2018 4:43 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>>>> On 18 May 2018 at 17:38 mail at sjemm.net wrote:
>>>>
>>>> May 18, 2018 4:05 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
>>>> On 18 May 2018 at 16:43 mail at sjemm.net wrote:
>>>>
>>>> Hi Tai74 and Aki,
>>>> I followed your conversation with interest on how to setup per user encryption in dovecot.
>>>> I have setup my dovecot with the following in a conf file:
>>>>
>>>> ==============
>>>>
>>>> mail_attribute_dict = file:%h/Maildir/dovecot-attributes
>>>> mail_plugins = $mail_plugins mail_crypt
>>>> plugin {
>>>>
>>>> mail_crypt_curve = secp521r1
>>>>
>>>> mail_crypt_save_version = 2
>>>>
>>>> }
>>>>
>>>> ==============
>>>>
>>>> This works nice, all emails are being encrypted and every user/folder has keys.
>>>> But as I understood from your conversation these keys are not protected. And I want them to be
>>>> protected by the users password used by imap.
>>>>
>>>> Those passwords are stored in a mysql DB file. ( I used a guide from workaround [dot] org to set up
>>>> the DB and postfix/dovecot)
>>>>
>>>> but how would i set it so, that the users password from the DB is used to encrypt the keys?
>>>>
>>>> should i use mail_crypt_private_password = ?
>>>> how do i point it to the mysql db then?
>>>> im unsure about this
>>>>
>>>> Do you have any hints on this?
>>>>
>>>> Kind regards,
>>>> Zjemm
>>>>
>>>> The passwords in your MySQL database are, hopefully, not in plaintext. If you want to secure your
>>>> user's keys using user's login password, you must have a TOOL that manages this.
>>>>
>>>> You can use mail_crypt_private_password = %w in (mysql) passdb fields to provide the user's login
>>>> password as private password. You might want to run it thru some hash, so %{sha1:password} might be
>>>> a good option.
>>>>
>>>> You can change the key password using 'doveadm mailbox cryptokey', this needs to be done every time
>>>> user changes his password.
>>>>
>>>> Also note that if you go down this road, and the user forgets his password, you will not be able to
>>>> recover the emails without backup copy of the private key.
>>>>
>>>> Aki
>>>>
>>>> Hi Aki
>>>>
>>>> I used the following command:
>>>> dovecot pw -s SHA256-CRYPT
>>>>
>>>> the output on the chosen password looks like: {SHA256-CRYPT}$5$Rokc06a7In4SF3bO$OQpGQWqg........
>>>>
>>>> This output is used to store in the password fields in the database. So no plain text passwords no
>>>> :)
>>>>
>>>> You can use mail_crypt_private_password = %w in (mysql) passdb fields to provide the user's login
>>>> password as private password.
>>>>
>>>> can you explain this a bit more for me?
>>>>
>>>> for now i have in the 10-auth.conf file the following:
>>>> ==============
>>>> passdb {
>>>> driver = sql
>>>>
>>>> # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
>>>> args = /etc/dovecot/dovecot-sql.conf.ext
>>>> }
>>>>
>>>> and:
>>>>
>>>> userdb {
>>>> driver = static
>>>> args = uid=vmail gid=vmail home=/var/vmail/%d/%n
>>>> }
>>>> ==============
>>>>
>>>> then i have in dovecot-sql.conf.ext
>>>> ==============
>>>> driver = mysql
>>>> connect = host=x.x.x.x dbname=mailserver user=mailuser password=mailpasswordexample
>>>> default_pass_scheme = SHA256-CRYPT
>>>> password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';
>>>> ==============
>>>> Where do i need to set : mail_crypt_private_password = %w ?
>>>>
>>>> password as private password. You might want to run it thru some hash, so %{sha1:password} might be
>>>> a good option.
>>>>
>>>> the passwords are allready hashed in the DB using: dovecot pw -s SHA256-CRYPT to genereate the has.
>>>> so this step isnt nesesary anymore am i right?
>>>>
>>>> Thank you for your quick response, very helpfull
>>>>
>>>> Zjemm
>>>>
>>>> You misunderstood a bit. The idea is to use the *plaintext* password as the password for the
>>>> private key. Otherwise anyone could just decrypt it by looking at your database where the hashed
>>>> password is..
>>>>
>>>> So:
>>>>
>>>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
>>>> virtual_users WHERE email='%u'
>>>>
>>>> Aki
>>>>
>>>> Hi Aki,
>>>>
>>>> Thank you very much for your help, i realy appreciate that.
>>>>
>>>> Ok so if i understand it correctly i'll have to use:
>>>>
>>>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
>>>> virtual_users WHERE email='%u'
>>>>
>>>> in my dovecot-sql.conf.ext file
>>>>
>>>> This query selects the user, the password, and %w
>>>>
>>>> if i run a little query myself:
>>>> MariaDB [mailserver]> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password
>>>> FROM virtual_users;
>>>> +------------------+----------------------------------+------------------------------------+
>>>> | user | password | userdb_mail_crypt_private_password |
>>>> +------------------+----------------------------------+------------------------------------+
>>>> | john at example.org | {SHA256-CRYPT}$5$M/GWzmtjsLroRWI | %w |
>>>> +------------------+----------------------------------+------------------------------------+
>>>>
>>>> %w is a dovecot variable, and stands for the plaintext password, but the password is not stored as
>>>> plaintext in the DB, %w get filled with the actual plaintext password by dovecot upon the user that
>>>> is typing in the password when authenticating.
>>>>
>>>> is this correct?
>>>>
>>>> yes.
>>>>
>>>> so then i have the username the hashed password en the plaintext password as a result of the query.
>>>>
>>>> yes
>>>>
>>>> now userdb_mail_crypt_private_password = the plaintext password
>>>> do i need to reference it somewhere? or is userdb_mail_crypt_private_password autmatically used by
>>>> the dovecot mail_crypt plugin to encrypt the keys? or should it be mail_crypt_private_password?
>>>>
>>>> It gets injected into the mail process as 'mail_crypt_private_password', as if it was set in plugin
>>>> {} section.
>>>>
>>>> if i have this setup working i'm going to write a blog post on this topic to share this knowledge
>>>>
>>>> Thanks again and have a great weekend.
>>>>
>>>> Zjemm
>>>>
>>>> Aki
>>>>
>>>> Hi Aki,
>>>>
>>>> Cool i'm testing it right now.
>>>> I have set up a new mailserver (life is great with lxc containers :) )
>>>>
>>>> postfix and dovecot are working like normal
>>>>
>>>> next i enable mail_crypt
>>>>
>>>> i did create a file: /etc/dovecot/conf.d/10-mailcrypt.conf
>>>> ==========================
>>>> mail_attribute_dict = file:%h/Maildir/dovecot-attributes
>>>>
>>>> mail_plugins = $mail_plugins mail_crypt
>>>>
>>>> plugin {
>>>> mail_crypt_curve = secp521r1
>>>> mail_crypt_save_version = 2
>>>> }
>>>> ==========================
>>>>
>>>> and then i changed the file: /etc/dovecot/dovecot-sql.conf.ext
>>>>
>>>> so the query is now the new query:
>>>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
>>>> virtual_users WHERE email='%u';
>>>>
>>>> then i restarted dovecot and postfix and send a test email to the one and only testuser that is in
>>>> there.
>>>>
>>>> when i open the mailbox with the tool mutt, i can see the new email, and when openening the email
>>>> the mutt client drops the connection.
>>>>
>>>> in the log i can see:
>>>>
>>>> May 19 13:34:48 mailserver1.example.local dovecot[600]: imap-login: Login: user=<john at example.org>,
>>>> method=PLAIN, rip=::1, lip=::1, mpid=713, TLS, session=<E3PnIY9sNM4AAAAAAAAAAAAAAAAAAAAB>
>>>> May 19 13:34:49 mailserver1.example.local dovecot[600]: imap(john at example.org): Error: read()
>>>> failed:
>>>> read(/var/vmail/example.org/john/Maildir/cur/1526736378.M161472P641.mailserver1.example.local,S=559,
>>>> =571:2,) failed: Private key not available: Cannot decrypt key
>>>> bfc5bb25b1bf64290eea6dc14b516c6a0a25b64551b6e4f0f8677ba7274887cb: error:03070068:bignum
>>>> routines:BN_mpi2bn:encoding error (FETCH BODY[] for mailbox INBOX UID 8)
>>>>
>>>> i think i missed a step, but witch one?
>>>>
>>>> the userpassword hasnt been changed (that would be the next step in the testing process)
>>>>
>>>> should i've use doveadm first to encrypt the key with that userpassword? i thought it would do that
>>>> on the fly, because the initial keys where only just created when enabling the mail_crypt plugin
>>>>
>>>> please let me know your thougts
>>>> Zjemm
>>> I noticed you replied directly to me, and not to the list, too... fixed that for you.
>>>
>>> mail_crypt_private_password is used when key is created, but if you have created it before using
>>> password, you'll need to encrypt it before turning the setting on.
>>>
>>> Aki
>> Hi, Ok i'm sorry about that, thank you for fixing that. i'm a bit new to mailing lists :)
>>
>> are you a dovecot devoloper? or a enthusiastic user?
>>
>> anyway, i managed to get it working now with all your help :)
>>
>> to start fresh i did a:
>> rm -rf /var/vmail/example.org
>>
>> so all email is gone, and i assume all dovecot keys are gone that way.
>>
>> Then i tried to set a password first:
>>
>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
>> result: Changed password for 0 key(s)
>>
>> (summersun = the password for this user)
>>
>> but then it says there are 0 keys, so then i send the test user an email and try to open the
>> mailbox, i'll get:
>>
>> imap(john at example.org): Error: read() failed:
>> read(/var/vmail/example.org/john/Maildir/cur/1526827992.M149712P883.mailserver1.example.local,S=549,
>> =561:2,) failed: Private key not available: Cannot decrypt key
>> 71849013a70b0c631c06112077e7c2fe39b0b2737b4933b219793841209d4e7f: error:03070068:bignum
>> routines:BN_mpi2bn:encoding error
>>
>> but hey, now the key's are created, so i tried to set the password again:
>>
>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
>> result: dcrypt_key_load_private(787701bd677dd69a26842547a3926cbaa625b0b5a91751f06678c3e9708343d9)
>> failed: password missing
>> =========
>>
>> Lets start over again, i did a new:
>> rm -rf /var/vmail/example.org
>>
>> i disabled the mail_crypt_private_password in the sql query and send an email to the test user.
>> now again new keys are generated.
>>
>> then i did set the password via doveadm:
>>
>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
>> result: Changed password for 1 key(s)
>>
>> i enabled the mail_crypt_private_password again in the sql query part and now it works :)
>>
>> YESSS!!!
>>
>> if i now change the password for the user, i cannot read the email anymore. that is what should
>> happen.
>> so now i changed the password for this user also with doveadm:
>> - doveadm mailbox cryptokey password -u john at example.org -n qwerty -o summersun
>>
>> and now i'm able to read the emails again. so it all works now as expected.
>>
>> i have now only one thing remaining.
>>
>> when i want to add a new user, i have to somehow:
>> - create the new user in the mysql DB
>> - start with dovecot not using mail_crypt_private_password (so disabled)
>> - send an initial email to let the users mailbox be created and generate the keys
>> - set the users password with doveadm
>> - restart dovecot again with mail_crypt_private_password (so enabled)
>>
>> but this would be disruptive to existing users, and i would have to script the whole thing as a
>> "add-new-user" script to make it usable i guess.
>>
>> do you know of a more easy way of acomplishing this?
>> could i use doveadm for all steps, and get around the mail_crypt_private_password disable part?
>>
>> or could i add a column in the mysql db called userdb_mail_crypt_private_password and fill that
>> column with %w for users that allready exist, and empty for new users.
>> when being empty, it might not set the mail_crypt_private_password for new users
>>
>> or is this the wrong way of thinking?
>>
>> let me know your thoughts on this.
>>
>> And last but not least, i really want to thank you for all your help. It is a really nice
>> experience.
>> Bless you!
>>
>> Zjemm
> Hi Aki,
>
> You did fix the posting on the mail archive, i can see our conversation on there, all but the last email i send to you.
> Maybe you did not received the last email?
>
> would you be willing to take a look at my last question in the email See below:
> Thank you very much.
>
> ================================
> i have now only one thing remaining.
>
> when i want to add a new user, i have to somehow:
> - create the new user in the mysql DB
> - start with dovecot not using mail_crypt_private_password (so disabled)
> - send an initial email to let the users mailbox be created and generate the keys
> - set the users password with doveadm
> - restart dovecot again with mail_crypt_private_password (so enabled)
>
> but this would be disruptive to existing users, and i would have to script the whole thing as a "add-new-user" script to make it usable i guess.
>
> do you know of a more easy way of acomplishing this?
> could i use doveadm for all steps, and get around the mail_crypt_private_password disable part?
>
> or could i add a column in the mysql db called userdb_mail_crypt_private_password and fill that column with %w for users that allready exist, and empty for new users.
> when being empty, it might not set the mail_crypt_private_password for new users
>
> or is this the wrong way of thinking?
>
> let me know your thoughts on this.
>
> And last but not least, i really want to thank you for all your help. It is a really nice experience.
> Bless you!
>
> Zjemm

If you have mail_crypt_private_password set *when* keys are generated,
dovecot should use that password to encrypt the user key.

Aki