[SOLVED] Thunderbird client certification validation fails with same profile but changed IMAP server software (Re: It works for two SMTP servers and cyrus-imap, why not Dovecot?)

[Daniel Dickinson]

In hopes that searching may turn up the solution for others:

The reason client certificate validation was failing in Thunderbird when
it had previously succeeded with other servers (both IMAP and SMTP) is
precisely that: the client and profile where the same ones used to
connect to the server who's hostname hadn't changed, and email addresses
and usernames were the same, and Thunderbird can't handle that.

Keeping your existing Thunderbird profile when switching the IMAP server
software running on the hosting server doesn't work even if you change
all the configs that should matter (rename old accounts out of the way,
replace the server they are supposed to be contacting with blank, and so
on) and create new accounts within the same profile.

The workaround for this bug in Thunderbird is to move the old profile
out of the way and start fresh.

The apparent cause of failure (invalid cert messages and other messages
that make it look like it's a configuration or certificate/CA file
issue) actually has nothing to do with the actual issue and is a major
red herring.

Sorry for the noise, back to regular scheduled program...

Regards,

Daniel

On 2015-02-12 6:29 AM, Daniel Curran-Dickinson wrote:
> Ok, the patch doesn't actually fix the bug.  It appeared to do so in
> that after running the server with the patch applied client certificate
> validation succeeded, however, it appears this bug is actually
> intermittent as, even with the patched package, the server is now
> complaining that the client has not provided a valid SSL certificate.
> 
> This is definitely not true as the certificates, and in general
> verification of the same client-side certificates work, even with the
> same Thunderbird client, with postfix, exim, and cyrus-imapd.
> 
> In short dovecot has some bug that causes verification of certificates
> presented by the client to fail, however the bug is not easy to debug as
> sometimes config changes work, but later, running the same config,
> things fail again.
> 
> There appears to be some sort of caching even across client and server
> restarts that is coming into play and confusing the issue.
> 
> Anyone know of SSL caching issues with Window 8.1, particular
> Thunderbird on that platform?
> 
> Also why is this bug only affecting dovecot?  There is some strangeness
> going on here, and, from web searching for the same issue, it appears
> others have run into the same issue and had no success in resolving it,
> despite also doing everything according to documentation.
> 
> With dovecot 2.2.9 from Ubuntu (i.e. not patched version):
> The relevant config bits from dovecot -n are:
> 
> auth_mechanisms = login plain digest-md5 cram-md5
> auth_ssl_require_client_cert = yes
> ssl = required
> ssl_ca = </path/to/ca.pem
> ssl_cert = </path/to/cert.pem
> ssl_key = </path/to/key.pem
> ssl_protocols = !SSLv2 !SSLv3
> ssl_require_crl = no (yes or no makes no difference)
> ssl_verify_client_cert = yes
> 
> Oddly this doesn't appear in dovecot -n, though set
> 
> disable_plaintext_auth = yes
> 
> Client fails whether configured for encrypted passwords or not
> Client fails whether using ssl = yes (imaps) on 993 or STARTTLS on 143
> 
> And yes I have followed the correct ordering of the CA followed by CRL
> (and tried without CRL as well).
> 
> In addition I have tried CA + intermediate and CA + crl + intermediate +
> crl for startssl.com certificates as well as the above test with
> self-signed CA (root CA + crl or just root CA).
> 
> I have also just confirmed (by connecting and sending mail on port 587
> with STARTTLS and SSL required, with client certificate validation
> required by postfix) that using the same certificate, same client, same
> mail server, that SMTP AUTH + verification of client certificates succeeds.
> 
> Since I've had the same client using the same certificates work with all
> three of postfix (SMTP), exim (SMTP), and cyrus-imapd (IMAP), there is
> not doubt that there is an issue with dovecot's handling of this scenario.
> 
> I know I'm harping on this 'it works elsewhere' theme, but this bug has
> been ignored for ages because of the assumption that user is doing
> something wrong, and that is simply not the case, or if it is, dovecot
> is very, very bad at indicating what the actual problem is.
> 
> Line-ending are *nix line endings.
> 
> Anything else that you want to suggest along with how to make dovecot
> report what the actual problem is if there is some other magic formulae
> to invoke?
> 
> If I've missed something it's by no means obvious or explained clearly
> anywhere.
> 
> Regards,
> 
> Daniel
> 
> On 2015-02-11 2:33 AM, Daniel Dickinson wrote:
>> Hi all,
>>
>> As I reported earlier (with a typo in the work [BUG]) client
>> certification validation *does not* work even if you do everything
>> exactly according to all documentation and attempts at helpful advice.
>>
>> I have seen this issue with both startssl.com and self-signed
>> certificates, and based on what I've seen from searching the web, this
>> is a problem that has gotten little attention because most people don't
>> bother, but are more than willing to give out useless advice on how to
>> make it work.
>>
>> Furthermore the issue does NOT occur with the cyrus-imap mail server, so
>> it is definitely a server-side issue.
>>
>> The actual issue is that the code for calling OpenSSL that constructs
>> the client certificate validation is in fact WRONG.
>>
>> I don't have a perfect patch as I was mostly interested in getting it
>> working for my needs and didn't bother with constructing the list of CA
>> names to send to the client, preferring to let OpenSSL handle all that
>> sort of thing.
>>
>> What it comes down to is that the code, which probably worked at one
>> point, was not correctly updated at some point and since then client
>> side certificate validation has been BROKEN.
>>
>> I have patched against 2.2.9, however I have seen this problem in the
>> versions in both Debian Wheezy and Debian Jessie as well.
>>
>> As you will see from the patch (which is an attachment as people tend to
>> complain that patches get mangled when you inline them, and even if I
>> have a good client I've gotten heck because the receiver didn't.
>>
>> Regards,
>>
>> Daniel
>>
>