Oops, now I finally understand why Mail.app kept asking for my password for each mail I sent: it helpfully decided to start signing mails with the only client cert I had, without asking me.. Forget about those signatures in the last two mails :)