PBKDF2 password hashing as in ASP.NET Core
Hello,
I'm setting up a new server and, again, seek for a decently secure (from a security specialist's POV) way to store and verify user passwords in a database. Additionally now, GDPR requires me to use a solid state-of-the-art solution.
My OS is Ubuntu 20.04, Dovecot version 2.3.7, database backend with PostgreSQL 12.
Obviously, storing the plaintext password is a terrible idea. SHA-based methods aren't suitable either. bcrypt has been recommended often [1]. PBKDF2 was preferred over bcrypt even more [2]. I'm managing all database contents with an ASP.NET Core application that implements the management user frontend. It's a bit hard to find bcrypt support for .NET (there are a few NuGet packages of unknown quality [3]).
.NET does however implement, use and recommend PBKDF2 for its own user management. If this is by far the best way to go, I'm already covered on that side. Now the problem is, once again*, how I can use this in applications to make them as secure.
I need a solution for Dovecot and Exim. Exim seems to be able to ask Dovecot (IMAP) for user authentication, so I might try that and only need to solve the problem in Dovecot alone.
Dovecot documentation says that PBKDF2 is somewhat possible [4]. It requires the hash in the format "$1$salt$rounds$hash". I guess that "salt", "rounds" and "hash" are the parameters here. But what is their format?
The .NET implementation [5] describes its format as "{ 0x01, prf (UInt32), iter count (UInt32), salt length (UInt32), salt, subkey }" with big-edian integers. The result is base64-encoded. Prf is an enum value, describing the HMAC SHA-256. Subkey is the hash value part.
I might find a way to convert .NET's format into what Dovecot understands. The hmac is SHA-256, the rounds is 10000. But I wasn't able to get it working.
My test password is: simplepassword The .NET hash: AQAAAAEAACcQAAAAEG0Anzb9vKOqsAKxLyhXedCTJoHrP381hKiKBHuPHhMdkjqW8Bks8RFOQZLssJ2grQ== The converted hash for Dovecot: $1$bQCfNv28o6qwArEvKFd50A==$10000$kyaB6z9/NYSoigR7jx4THZI6lvAZLPERTkGS7LCdoK0=
I've also found the source code in Dovecot that should verify the hash [6]. It gives some more hints about the expected format that are sadly missing from the documentation, making it almost useless. I also tried with the "{PBKDF2}" prefix, with the base64 padding "=" removed and with the hash part converted from base64 to hex. Nothing works. The source mentions "SHA1" somewhere. Is that all it can accept? No up-to-date SHA-256?
So what have I done wrong here? Why can't I authenticate? The Dovecot log isn't helpful, it doesn't even mention the user name I tried to log in with from Thunderbird, most of the time (it's unpredictable).
What is the correct usage of Dovecot's PBKDF2 feature? Is it functional at all? There's a test case for it [7] but that's not helpful to me.
If Dovecot's PBKDF2 support is not functional or not compatible with ASP.NET Core's parameters, what options do I have? Can I build my own authentication service that Dovecot can communicate with, to fill the gap of missing crypto support?
Yves
*) During my research today, I find myself finding my own questions from 6 years ago when I did this the last time. The situation hasn't changed much since then. Secure password hashing is still impossible or complicated in server applications like Exim or Dovecot.
[1] https://codahale.com/how-to-safely-store-a-password/ (linked from the Dovecot documentation) [2] https://www.reddit.com/r/node/comments/4u1jcn/is_bcrypt_the_best_possible_pa... [3] https://nugetmusthaves.com/Tag/bcrypt [4] https://doc.dovecot.org/configuration_manual/authentication/password_schemes... [5] https://github.com/dotnet/AspNetCore/blob/master/src/Identity/Extensions.Cor... [6] https://github.com/dovecot/core/blob/81b5b188c478ec36bea8bda8fcad1e5f32ac612... [7] https://github.com/dovecot/core/blob/ff5305136ae747867b6f6af9a1737188ae7b3b5...
On 29/08/2020 23:49 Yves Goergen <nospam.list@unclassified.de> wrote:
Hello,
I'm setting up a new server and, again, seek for a decently secure (from a security specialist's POV) way to store and verify user passwords in a database. Additionally now, GDPR requires me to use a solid state-of-the-art solution.
My OS is Ubuntu 20.04, Dovecot version 2.3.7, database backend with PostgreSQL 12.
Obviously, storing the plaintext password is a terrible idea. SHA-based methods aren't suitable either. bcrypt has been recommended often [1]. PBKDF2 was preferred over bcrypt even more [2]. I'm managing all database contents with an ASP.NET Core application that implements the management user frontend. It's a bit hard to find bcrypt support for .NET (there are a few NuGet packages of unknown quality [3]).
.NET does however implement, use and recommend PBKDF2 for its own user management. If this is by far the best way to go, I'm already covered on that side. Now the problem is, once again*, how I can use this in applications to make them as secure.
I need a solution for Dovecot and Exim. Exim seems to be able to ask Dovecot (IMAP) for user authentication, so I might try that and only need to solve the problem in Dovecot alone.
Dovecot documentation says that PBKDF2 is somewhat possible [4]. It requires the hash in the format "$1$salt$rounds$hash". I guess that "salt", "rounds" and "hash" are the parameters here. But what is their format?
The .NET implementation [5] describes its format as "{ 0x01, prf (UInt32), iter count (UInt32), salt length (UInt32), salt, subkey }" with big-edian integers. The result is base64-encoded. Prf is an enum value, describing the HMAC SHA-256. Subkey is the hash value part.
I might find a way to convert .NET's format into what Dovecot understands. The hmac is SHA-256, the rounds is 10000. But I wasn't able to get it working.
My test password is: simplepassword The .NET hash: AQAAAAEAACcQAAAAEG0Anzb9vKOqsAKxLyhXedCTJoHrP381hKiKBHuPHhMdkjqW8Bks8RFOQZLssJ2grQ== The converted hash for Dovecot: $1$bQCfNv28o6qwArEvKFd50A==$10000$kyaB6z9/NYSoigR7jx4THZI6lvAZLPERTkGS7LCdoK0=
I've also found the source code in Dovecot that should verify the hash [6]. It gives some more hints about the expected format that are sadly missing from the documentation, making it almost useless. I also tried with the "{PBKDF2}" prefix, with the base64 padding "=" removed and with the hash part converted from base64 to hex. Nothing works. The source mentions "SHA1" somewhere. Is that all it can accept? No up-to-date SHA-256?
So what have I done wrong here? Why can't I authenticate? The Dovecot log isn't helpful, it doesn't even mention the user name I tried to log in with from Thunderbird, most of the time (it's unpredictable).
What is the correct usage of Dovecot's PBKDF2 feature? Is it functional at all? There's a test case for it [7] but that's not helpful to me.
If Dovecot's PBKDF2 support is not functional or not compatible with ASP.NET Core's parameters, what options do I have? Can I build my own authentication service that Dovecot can communicate with, to fill the gap of missing crypto support?
Yves
Hi!
The PBKDF2 algorithm is standard and should be compatible with ASP.NET Core.
The salt parameter is 16 symbols from the salt character set
./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
followed by number of rounds
hash is hex encoded 160-bit value which comes out of the PBKDF2 function with SHA1.
Dovecot does not currently have support for PBKDF2-SHA256, only PBKDF2-SHA1. You could use CRYPT-SHA512 instead which is probably just as good?
Aki
Thank you for your reply.
It's not that simple, though. Just because some core algorithms are standardised and should be compatible doesn't mean their use in different implementations leads to interoperable data. The key point here seems to be that Dovecot just supports SHA-1 with PBKDF2, not SHA-256. So I'm out of luck here. The different formats are no longer relevant then.
CRYPT-SHA512 is not anywhere near as secure as PBKDF2.
But I've read and learned a lot about secure password hashing in the past 24 hours. My initial point that PBKDF2 is the state of the art has been disproved already. This order seems to be the case [1]:
MD5/SHA1 << SHA2 << PBKDF2 < bcrypt < scrypt < Argon2
So I've changed my plans and try to go for Argon2 now. I found support for .NET Core [2] and Python [3].
My original question is kind of obsolete now because I also found another requirement: password rehashing. I'm migrating from an old database that has CRYPT-SHA512 hashes and want to upgrade them to Argon2. This affects multiple services (IMAP, SMTP, FTP, Management UI) so I think I'll better make a central authentication service that has all the passwords and crypto in one place and handles requests from those service daemons.
I'm currently investigating how to build such a service and integrate it into the services. Maybe a Unix socket is a good communication channel. Dovecot should be able to query it with a custom Lua script. Haven't looked into the other services yet. That auth service could be built with Python for isolation from other services, high availability and relatively low memory footprint. It connects to the database, reads and updates the hashes and does all the crypto for its clients.
Any suggestions about how to do that?
Yves
[1] https://cryptobook.nakov.com/mac-and-key-derivation/argon2 [2] https://github.com/tabrath/libsodium-core [3] https://passlib.readthedocs.io/en/stable/lib/passlib.hash.argon2.html
-------- Ursprüngliche Nachricht -------- Von: Aki Tuomi <aki.tuomi@open-xchange.com> Gesendet: Sonntag, 30. August 2020, 16:33 MESZ Betreff: PBKDF2 password hashing as in ASP.NET Core
Hi!
The PBKDF2 algorithm is standard and should be compatible with ASP.NET Core.
The salt parameter is 16 symbols from the salt character set
./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
followed by number of rounds
hash is hex encoded 160-bit value which comes out of the PBKDF2 function with SHA1.
Dovecot does not currently have support for PBKDF2-SHA256, only PBKDF2-SHA1. You could use CRYPT-SHA512 instead which is probably just as good?
Aki
In case you are interested,
https://wiki.dovecot.org/HowTo/ConvertPasswordSchemes
By the way, I am bit sceptical that CRYPT-SHA512 is less secure than PBKDF2.
CRYPT-SHA512 is not "just" SHA512(salt||password), it does at least 1000 rounds of hashing in similar way as PBKDF2 does. So, what is your reasoning for claiming that PBKDF2 is much secure than CRYPT-SHA512?
Also, if you look at hashcat cracking speeds, you'll see that the speed of cracking is slower for CRYPT-SHA512 than for PBKDF2-SHA512. See https://github.com/siseci/hashcat-benchmark-comparison/blob/master/1x%20GTX%...
Aki
On 30/08/2020 19:54 Yves Goergen <nospam.list@unclassified.de> wrote:
Thank you for your reply.
It's not that simple, though. Just because some core algorithms are standardised and should be compatible doesn't mean their use in different implementations leads to interoperable data. The key point here seems to be that Dovecot just supports SHA-1 with PBKDF2, not SHA-256. So I'm out of luck here. The different formats are no longer relevant then.
CRYPT-SHA512 is not anywhere near as secure as PBKDF2.
But I've read and learned a lot about secure password hashing in the past 24 hours. My initial point that PBKDF2 is the state of the art has been disproved already. This order seems to be the case [1]:
MD5/SHA1 << SHA2 << PBKDF2 < bcrypt < scrypt < Argon2
So I've changed my plans and try to go for Argon2 now. I found support for .NET Core [2] and Python [3].
My original question is kind of obsolete now because I also found another requirement: password rehashing. I'm migrating from an old database that has CRYPT-SHA512 hashes and want to upgrade them to Argon2. This affects multiple services (IMAP, SMTP, FTP, Management UI) so I think I'll better make a central authentication service that has all the passwords and crypto in one place and handles requests from those service daemons.
I'm currently investigating how to build such a service and integrate it into the services. Maybe a Unix socket is a good communication channel. Dovecot should be able to query it with a custom Lua script. Haven't looked into the other services yet. That auth service could be built with Python for isolation from other services, high availability and relatively low memory footprint. It connects to the database, reads and updates the hashes and does all the crypto for its clients.
Any suggestions about how to do that?
Yves
[1] https://cryptobook.nakov.com/mac-and-key-derivation/argon2 [2] https://github.com/tabrath/libsodium-core [3] https://passlib.readthedocs.io/en/stable/lib/passlib.hash.argon2.html
-------- Ursprüngliche Nachricht -------- Von: Aki Tuomi <aki.tuomi@open-xchange.com> Gesendet: Sonntag, 30. August 2020, 16:33 MESZ Betreff: PBKDF2 password hashing as in ASP.NET Core
Hi!
The PBKDF2 algorithm is standard and should be compatible with ASP.NET Core.
The salt parameter is 16 symbols from the salt character set
./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
followed by number of rounds
hash is hex encoded 160-bit value which comes out of the PBKDF2 function with SHA1.
Dovecot does not currently have support for PBKDF2-SHA256, only PBKDF2-SHA1. You could use CRYPT-SHA512 instead which is probably just as good?
Aki
I'm not an expert at this, I just read other sources that make suggestions. I have no strong comparison of SHA2 and PBKDF2. So my "<<" may be exaggerated. The most important points today seem to be resilience against GPU and ASIC attacks. Those devices have only little memory to work with. SHA2 is said to be designed to use very little memory which only leaves the CPU load as defence. This can be compensated by massive parallelisation, as in GPUs.
Neither SHA2 nor PBKDF2 should be resilient against GPU attacks. This is where the other algorithms are better because they introduce bigger memory requirements.
I'll probably look into PAM authentication and see if I can get my own module into there somehow. This should be the most versatile method to provide secure authentication for all services. At least Dovecot, Exim and ProFTPd support PAM.
-------- Ursprüngliche Nachricht -------- Von: Aki Tuomi <aki.tuomi@open-xchange.com> Gesendet: Sonntag, 30. August 2020, 19:27 MESZ Betreff: PBKDF2 password hashing as in ASP.NET Core
In case you are interested,
https://wiki.dovecot.org/HowTo/ConvertPasswordSchemes
By the way, I am bit sceptical that CRYPT-SHA512 is less secure than PBKDF2.
CRYPT-SHA512 is not "just" SHA512(salt||password), it does at least 1000 rounds of hashing in similar way as PBKDF2 does. So, what is your reasoning for claiming that PBKDF2 is much secure than CRYPT-SHA512?
Also, if you look at hashcat cracking speeds, you'll see that the speed of cracking is slower for CRYPT-SHA512 than for PBKDF2-SHA512. See https://github.com/siseci/hashcat-benchmark-comparison/blob/master/1x%20GTX%...
Aki
participants (2)
-
Aki Tuomi
-
Yves Goergen