Conclusion: The crypto errors I have been complaining about occur if the Private Key in the RSACryptoServiceProvider changes on disk while HTTPS handshakes are taking place. This happened in Fiddler because it was trying to use the same KeyContainerName for all of the private keys for all of the certificates.

When the LSASS.EXE process tries to complete the HTTPS handshake (this is always out-of-process to your application), it finds that the private key is unable to decrypt the data sent by the client, which was encrypted using the original private key. Lesson learned: While it's called a KeyContainer, it can only hold one exchange key at a time. If you want multiple keys (e.g. for multiple certificates) you need multiple containers.