Insecure OPs easily bring problems to consumers trusting them.
I think that there should be a kind of world wide reliability evaluation system for OPs.
Suppose every OPs has its score on the list of OPs and they should make an effort to keep its score high.
RPs are able to exploit it to know which OP is trustworthy.

But obviously, the list is centralized.
This is the SPoF and is a quite big drawback.
It should be kept decentralized as well as OpenID itself.