Best mail encryption solution for per-user
Aki Tuomi
aki.tuomi at dovecot.fi
Sun May 27 09:48:52 EEST 2018
> On 26 May 2018 at 10:36 mail at sjemm.net wrote:
>
>
> May 23, 2018 10:10 AM, mail at sjemm.net wrote:
> > May 23, 2018 9:46 AM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
> >
> >> On 23.05.2018 10:15, mail at sjemm.net wrote:
> >>
> >>> May 23, 2018 8:31 AM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
> >>
> >> On 23.05.2018 09:13, mail at sjemm.net wrote:
> >>> May 20, 2018 8:01 PM, mail at sjemm.net wrote:
> >>> May 20, 2018 2:47 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
> >>> On 19 May 2018 at 16:40 mail at sjemm.net wrote:
> >>>
> >>> May 18, 2018 10:01 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
> >>> On 18 May 2018 at 21:44 mail at sjemm.net wrote:
> >>>
> >>> May 18, 2018 4:43 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
> >>> On 18 May 2018 at 17:38 mail at sjemm.net wrote:
> >>>
> >>> May 18, 2018 4:05 PM, "Aki Tuomi" <aki.tuomi at dovecot.fi> wrote:
> >>> On 18 May 2018 at 16:43 mail at sjemm.net wrote:
> >>>
> >>> Hi Tai74 and Aki,
> >>> I followed your conversation with interest on how to setup per user encryption in dovecot.
> >>> I have setup my dovecot with the following in a conf file:
> >>>
> >>> ==============
> >>>
> >>> mail_attribute_dict = file:%h/Maildir/dovecot-attributes
> >>> mail_plugins = $mail_plugins mail_crypt
> >>> plugin {
> >>>
> >>> mail_crypt_curve = secp521r1
> >>>
> >>> mail_crypt_save_version = 2
> >>>
> >>> }
> >>>
> >>> ==============
> >>>
> >>> This works nice, all emails are being encrypted and every user/folder has keys.
> >>> But as I understood from your conversation these keys are not protected. And I want them to be
> >>> protected by the users password used by imap.
> >>>
> >>> Those passwords are stored in a mysql DB file. ( I used a guide from workaround [dot] org to set up
> >>> the DB and postfix/dovecot)
> >>>
> >>> but how would i set it so, that the users password from the DB is used to encrypt the keys?
> >>>
> >>> should i use mail_crypt_private_password = ?
> >>> how do i point it to the mysql db then?
> >>> im unsure about this
> >>>
> >>> Do you have any hints on this?
> >>>
> >>> Kind regards,
> >>> Zjemm
> >>>
> >>> The passwords in your MySQL database are, hopefully, not in plaintext. If you want to secure your
> >>> user's keys using user's login password, you must have a TOOL that manages this.
> >>>
> >>> You can use mail_crypt_private_password = %w in (mysql) passdb fields to provide the user's login
> >>> password as private password. You might want to run it thru some hash, so %{sha1:password} might be
> >>> a good option.
> >>>
> >>> You can change the key password using 'doveadm mailbox cryptokey', this needs to be done every time
> >>> user changes his password.
> >>>
> >>> Also note that if you go down this road, and the user forgets his password, you will not be able to
> >>> recover the emails without backup copy of the private key.
> >>>
> >>> Aki
> >>>
> >>> Hi Aki
> >>>
> >>> I used the following command:
> >>> dovecot pw -s SHA256-CRYPT
> >>>
> >>> the output on the chosen password looks like: {SHA256-CRYPT}$5$Rokc06a7In4SF3bO$OQpGQWqg........
> >>>
> >>> This output is used to store in the password fields in the database. So no plain text passwords no
> >>> :)
> >>>
> >>> You can use mail_crypt_private_password = %w in (mysql) passdb fields to provide the user's login
> >>> password as private password.
> >>>
> >>> can you explain this a bit more for me?
> >>>
> >>> for now i have in the 10-auth.conf file the following:
> >>> ==============
> >>> passdb {
> >>> driver = sql
> >>>
> >>> # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
> >>> args = /etc/dovecot/dovecot-sql.conf.ext
> >>> }
> >>>
> >>> and:
> >>>
> >>> userdb {
> >>> driver = static
> >>> args = uid=vmail gid=vmail home=/var/vmail/%d/%n
> >>> }
> >>> ==============
> >>>
> >>> then i have in dovecot-sql.conf.ext
> >>> ==============
> >>> driver = mysql
> >>> connect = host=x.x.x.x dbname=mailserver user=mailuser password=mailpasswordexample
> >>> default_pass_scheme = SHA256-CRYPT
> >>> password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';
> >>> ==============
> >>> Where do i need to set : mail_crypt_private_password = %w ?
> >>>
> >>> password as private password. You might want to run it thru some hash, so %{sha1:password} might be
> >>> a good option.
> >>>
> >>> the passwords are allready hashed in the DB using: dovecot pw -s SHA256-CRYPT to genereate the has.
> >>> so this step isnt nesesary anymore am i right?
> >>>
> >>> Thank you for your quick response, very helpfull
> >>>
> >>> Zjemm
> >>>
> >>> You misunderstood a bit. The idea is to use the *plaintext* password as the password for the
> >>> private key. Otherwise anyone could just decrypt it by looking at your database where the hashed
> >>> password is..
> >>>
> >>> So:
> >>>
> >>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
> >>> virtual_users WHERE email='%u'
> >>>
> >>> Aki
> >>>
> >>> Hi Aki,
> >>>
> >>> Thank you very much for your help, i realy appreciate that.
> >>>
> >>> Ok so if i understand it correctly i'll have to use:
> >>>
> >>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
> >>> virtual_users WHERE email='%u'
> >>>
> >>> in my dovecot-sql.conf.ext file
> >>>
> >>> This query selects the user, the password, and %w
> >>>
> >>> if i run a little query myself:
> >>> MariaDB [mailserver]> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password
> >>> FROM virtual_users;
> >>> +------------------+----------------------------------+------------------------------------+
> >>> | user | password | userdb_mail_crypt_private_password |
> >>> +------------------+----------------------------------+------------------------------------+
> >>> | john at example.org | {SHA256-CRYPT}$5$M/GWzmtjsLroRWI | %w |
> >>> +------------------+----------------------------------+------------------------------------+
> >>>
> >>> %w is a dovecot variable, and stands for the plaintext password, but the password is not stored as
> >>> plaintext in the DB, %w get filled with the actual plaintext password by dovecot upon the user that
> >>> is typing in the password when authenticating.
> >>>
> >>> is this correct?
> >>>
> >>> yes.
> >>>
> >>> so then i have the username the hashed password en the plaintext password as a result of the query.
> >>>
> >>> yes
> >>>
> >>> now userdb_mail_crypt_private_password = the plaintext password
> >>> do i need to reference it somewhere? or is userdb_mail_crypt_private_password autmatically used by
> >>> the dovecot mail_crypt plugin to encrypt the keys? or should it be mail_crypt_private_password?
> >>>
> >>> It gets injected into the mail process as 'mail_crypt_private_password', as if it was set in plugin
> >>> {} section.
> >>>
> >>> if i have this setup working i'm going to write a blog post on this topic to share this knowledge
> >>>
> >>> Thanks again and have a great weekend.
> >>>
> >>> Zjemm
> >>>
> >>> Aki
> >>>
> >>> Hi Aki,
> >>>
> >>> Cool i'm testing it right now.
> >>> I have set up a new mailserver (life is great with lxc containers :) )
> >>>
> >>> postfix and dovecot are working like normal
> >>>
> >>> next i enable mail_crypt
> >>>
> >>> i did create a file: /etc/dovecot/conf.d/10-mailcrypt.conf
> >>> ==========================
> >>> mail_attribute_dict = file:%h/Maildir/dovecot-attributes
> >>>
> >>> mail_plugins = $mail_plugins mail_crypt
> >>>
> >>> plugin {
> >>> mail_crypt_curve = secp521r1
> >>> mail_crypt_save_version = 2
> >>> }
> >>> ==========================
> >>>
> >>> and then i changed the file: /etc/dovecot/dovecot-sql.conf.ext
> >>>
> >>> so the query is now the new query:
> >>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM
> >>> virtual_users WHERE email='%u';
> >>>
> >>> then i restarted dovecot and postfix and send a test email to the one and only testuser that is in
> >>> there.
> >>>
> >>> when i open the mailbox with the tool mutt, i can see the new email, and when openening the email
> >>> the mutt client drops the connection.
> >>>
> >>> in the log i can see:
> >>>
> >>> May 19 13:34:48 mailserver1.example.local dovecot[600]: imap-login: Login: user=<john at example.org>,
> >>> method=PLAIN, rip=::1, lip=::1, mpid=713, TLS, session=<E3PnIY9sNM4AAAAAAAAAAAAAAAAAAAAB>
> >>> May 19 13:34:49 mailserver1.example.local dovecot[600]: imap(john at example.org): Error: read()
> >>> failed:
> >>> read(/var/vmail/example.org/john/Maildir/cur/1526736378.M161472P641.mailserver1.example.local,S=559,
> >>> =571:2,) failed: Private key not available: Cannot decrypt key
> >>> bfc5bb25b1bf64290eea6dc14b516c6a0a25b64551b6e4f0f8677ba7274887cb: error:03070068:bignum
> >>> routines:BN_mpi2bn:encoding error (FETCH BODY[] for mailbox INBOX UID 8)
> >>>
> >>> i think i missed a step, but witch one?
> >>>
> >>> the userpassword hasnt been changed (that would be the next step in the testing process)
> >>>
> >>> should i've use doveadm first to encrypt the key with that userpassword? i thought it would do that
> >>> on the fly, because the initial keys where only just created when enabling the mail_crypt plugin
> >>>
> >>> please let me know your thougts
> >>> Zjemm
> >>> I noticed you replied directly to me, and not to the list, too... fixed that for you.
> >>>
> >>> mail_crypt_private_password is used when key is created, but if you have created it before using
> >>> password, you'll need to encrypt it before turning the setting on.
> >>>
> >>> Aki
> >>> Hi, Ok i'm sorry about that, thank you for fixing that. i'm a bit new to mailing lists :)
> >>>
> >>> are you a dovecot devoloper? or a enthusiastic user?
> >>>
> >>> anyway, i managed to get it working now with all your help :)
> >>>
> >>> to start fresh i did a:
> >>> rm -rf /var/vmail/example.org
> >>>
> >>> so all email is gone, and i assume all dovecot keys are gone that way.
> >>>
> >>> Then i tried to set a password first:
> >>>
> >>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
> >>> result: Changed password for 0 key(s)
> >>>
> >>> (summersun = the password for this user)
> >>>
> >>> but then it says there are 0 keys, so then i send the test user an email and try to open the
> >>> mailbox, i'll get:
> >>>
> >>> imap(john at example.org): Error: read() failed:
> >>> read(/var/vmail/example.org/john/Maildir/cur/1526827992.M149712P883.mailserver1.example.local,S=549,
> >>> =561:2,) failed: Private key not available: Cannot decrypt key
> >>> 71849013a70b0c631c06112077e7c2fe39b0b2737b4933b219793841209d4e7f: error:03070068:bignum
> >>> routines:BN_mpi2bn:encoding error
> >>>
> >>> but hey, now the key's are created, so i tried to set the password again:
> >>>
> >>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
> >>> result: dcrypt_key_load_private(787701bd677dd69a26842547a3926cbaa625b0b5a91751f06678c3e9708343d9)
> >>> failed: password missing
> >>> =========
> >>>
> >>> Lets start over again, i did a new:
> >>> rm -rf /var/vmail/example.org
> >>>
> >>> i disabled the mail_crypt_private_password in the sql query and send an email to the test user.
> >>> now again new keys are generated.
> >>>
> >>> then i did set the password via doveadm:
> >>>
> >>> root at mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john at example.org -n summersun
> >>> result: Changed password for 1 key(s)
> >>>
> >>> i enabled the mail_crypt_private_password again in the sql query part and now it works :)
> >>>
> >>> YESSS!!!
> >>>
> >>> if i now change the password for the user, i cannot read the email anymore. that is what should
> >>> happen.
> >>> so now i changed the password for this user also with doveadm:
> >>> - doveadm mailbox cryptokey password -u john at example.org -n qwerty -o summersun
> >>>
> >>> and now i'm able to read the emails again. so it all works now as expected.
> >>>
> >>> i have now only one thing remaining.
> >>>
> >>> when i want to add a new user, i have to somehow:
> >>> - create the new user in the mysql DB
> >>> - start with dovecot not using mail_crypt_private_password (so disabled)
> >>> - send an initial email to let the users mailbox be created and generate the keys
> >>> - set the users password with doveadm
> >>> - restart dovecot again with mail_crypt_private_password (so enabled)
> >>>
> >>> but this would be disruptive to existing users, and i would have to script the whole thing as a
> >>> "add-new-user" script to make it usable i guess.
> >>>
> >>> do you know of a more easy way of acomplishing this?
> >>> could i use doveadm for all steps, and get around the mail_crypt_private_password disable part?
> >>>
> >>> or could i add a column in the mysql db called userdb_mail_crypt_private_password and fill that
> >>> column with %w for users that allready exist, and empty for new users.
> >>> when being empty, it might not set the mail_crypt_private_password for new users
> >>>
> >>> or is this the wrong way of thinking?
> >>>
> >>> let me know your thoughts on this.
> >>>
> >>> And last but not least, i really want to thank you for all your help. It is a really nice
> >>> experience.
> >>> Bless you!
> >>>
> >>> Zjemm
> >>> Hi Aki,
> >>>
> >>> You did fix the posting on the mail archive, i can see our conversation on there, all but the last
> >>> email i send to you.
> >>> Maybe you did not received the last email?
> >>>
> >>> would you be willing to take a look at my last question in the email See below:
> >>> Thank you very much.
> >>>
> >>> ================================
> >>> i have now only one thing remaining.
> >>>
> >>> when i want to add a new user, i have to somehow:
> >>> - create the new user in the mysql DB
> >>> - start with dovecot not using mail_crypt_private_password (so disabled)
> >>> - send an initial email to let the users mailbox be created and generate the keys
> >>> - set the users password with doveadm
> >>> - restart dovecot again with mail_crypt_private_password (so enabled)
> >>>
> >>> but this would be disruptive to existing users, and i would have to script the whole thing as a
> >>> "add-new-user" script to make it usable i guess.
> >>>
> >>> do you know of a more easy way of acomplishing this?
> >>> could i use doveadm for all steps, and get around the mail_crypt_private_password disable part?
> >>>
> >>> or could i add a column in the mysql db called userdb_mail_crypt_private_password and fill that
> >>> column with %w for users that allready exist, and empty for new users.
> >>> when being empty, it might not set the mail_crypt_private_password for new users
> >>>
> >>> or is this the wrong way of thinking?
> >>>
> >>> let me know your thoughts on this.
> >>>
> >>> And last but not least, i really want to thank you for all your help. It is a really nice
> >>> experience.
> >>> Bless you!
> >>>
> >>> Zjemm
> >>
> >> If you have mail_crypt_private_password set *when* keys are generated,
> >> dovecot should use that password to encrypt the user key.
> >>
> >> Aki
> >>> Hi Aki,
> >>>
> >>> well, if there is a new email user setup in the mysql DB, and no email has been send to that new
> >>> user. at that point there are no keys.
> >>>
> >>> So trying to set a password on those key's wont work. see:
> >>> root at mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey password -u john2 at example.org -n
> >>> qwerty
> >>> result: Changed password for 0 key(s)
> >>>
> >>> the keys need to be generated first. And that only happens when sending a mail to that new user.
> >>> Also, at the point of creating the keys, the mail_crypt_private_password should be disabled
> >>> otherwise the keys are generated and also encrypted with no user password, and then im unable to
> >>> set a new pasword for that key
> >>>
> >>> How can i set the password before the keys are generated?
> >>> or can i manually generate key's and then set the password?
> >>>
> >>> so at this point i have to do the following:
> >>
> >> You can use doveadm mailbox cryptokey generate -u user -U
> >>
> >> to generate a user key.
> >>
> >> Aki
> >>
> >>> - create the new user in the mysql DB
> >>> - start with dovecot not using mail_crypt_private_password (so disabled)
> >>> - send an initial email to let the users mailbox be created and generate the keys
> >>> - set the users password with doveadm
> >>> - restart dovecot again with mail_crypt_private_password (so enabled)
> >>> can i somehow enable the mail_crypt_private_password per user?
> >>> because this mysql query always returns: %w
> >>> and thus allways sets: userdb_mail_crypt_private_password
> >>> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM virtual_users;
> >
> > Hi Aki,
> >
> > You are correct
> >
> > doveadm mailbox cryptokey generate -u user -U generates the keys, but i have to first remove:
> > '%w' AS userdb_mail_crypt_private_password
> > from dovecot configuration and restart dovecot.
> >
> > if i don't do that, the keys are generated, but also somehow encrypted using: no
> > password/empty/something else (i don't know)
> >
> > here you can see that happening:
> >
> > root at mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey generate -u john2 at example.org -U
> > Folder Public ID
> > ✓ <userkey> 53004acf9886f887ca081c83d7392a5aa3fac4b5a20ce013db91a341fb4644c4
> > root at mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey password -u john2 at example.org -n
> > qwerty
> > result: dcrypt_key_load_private(53004acf9886f887ca081c83d7392a5aa3fac4b5a20ce013db91a341fb4644c4)
> > failed: password missing
> >
> > it expects an old password wich is never set in my opinion.
> >
> > so the last part of this setup would be to figure out a way to disable '%w' AS
> > userdb_mail_crypt_private_password for new users without manually removing that part from the query
> > and restarting dovecot. As that would mean connection interruption while adding new users
> >
> > if we could add new users to the DB and somehow not give '%w' AS userdb_mail_crypt_private_password
> > for that new user in the query, we can add the password afterward, and then enable '%w' AS
> > userdb_mail_crypt_private_password in the query for this user
> >
> > could we add a column in the user db lets say called: encrypted (yes/no)
> >
> > and then change the existing query: SELECT email as user, password, '%w' AS
> > userdb_mail_crypt_private_password FROM virtual_users;
> >
> > to firrt check if encryption is set to 'yes', if so retrun:
> > SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM virtual_users;
> >
> > if not return:
> > SELECT email as user, password FROM virtual_users;
> >
> > but then in one mysql query so dovecot can work with it
> >
> > ?
>
>
> Hi Aki,
>
> would you be willing to respond to my last question i had send to you last week?
> its this one:
> ==================================
> > doveadm mailbox cryptokey generate -u user -U generates the keys, but i have to first remove:
> > '%w' AS userdb_mail_crypt_private_password
> > from dovecot configuration and restart dovecot.
> >
> > if i don't do that, the keys are generated, but also somehow encrypted using: no
> > password/empty/something else (i don't know)
> >
> > here you can see that happening:
> >
> > root at mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey generate -u john2 at example.org -U
> > Folder Public ID
> > ✓ <userkey> 53004acf9886f887ca081c83d7392a5aa3fac4b5a20ce013db91a341fb4644c4
> > root at mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey password -u john2 at example.org -n
> > qwerty
> > result: dcrypt_key_load_private(53004acf9886f887ca081c83d7392a5aa3fac4b5a20ce013db91a341fb4644c4)
> > failed: password missing
> >
> > it expects an old password wich is never set in my opinion.
>
> could this be a bug? or am i doing it wrong? if needed i could send in a bug report?
>
> or might we need to enable the userdb_mail_crypt_private_password "per user" ?
>
> >
> > so the last part of this setup would be to figure out a way to disable '%w' AS
> > userdb_mail_crypt_private_password for new users without manually removing that part from the query
> > and restarting dovecot. As that would mean connection interruption while adding new users
> >
> > if we could add new users to the DB and somehow not give '%w' AS userdb_mail_crypt_private_password
> > for that new user in the query, we can add the password afterward, and then enable '%w' AS
> > userdb_mail_crypt_private_password in the query for this user
> >
> > could we add a column in the user db lets say called: encrypted (yes/no)
> >
> > and then change the existing query: SELECT email as user, password, '%w' AS
> > userdb_mail_crypt_private_password FROM virtual_users;
> >
> > to firrt check if encryption is set to 'yes', if so retrun:
> > SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM virtual_users;
> >
> > if not return:
> > SELECT email as user, password FROM virtual_users;
> >
> > but then in one mysql query so dovecot can work with it
>
> Please let me know your thougts, after this last thing im done :)
> also let me know if i can support this project in some way, im willing to contribute
>
> Zjemm
You can select it as userdb_mail_crypt_save_version=0 to disable encryption (it won't disable decryption). You can use value '2' for enabling encryption. DO NOT USE 1.
Also I tried the SQL based thing, and if you had EMPTY mail_crypt_private_password, try these to change the password. If the one that works is the second one, let me know =)
doveadm mailbox cryptokey password -u testuser -nnewpass -o ""
doveadm mailbox cryptokey password -u testuser -nnewpass -o "yes"
Aki
More information about the dovecot
mailing list