I believe I have discovered where the problem is originating from. The background for the theme is automatically applied to the latex image *source* attribute. I.e., when I change the theme colors, the latex image link becomes, e.g.
What needs to happen (and I don't know where this should happen) is that the bg and fg parts need to be taken from the right places. Right now it appears fg comes from the link hover color... an awkward choice... and the bg from the background color. I want it to always be white bg and black fg.
From googling around, it appears this is a commonly voiced bug by fellow math bloggers. It would be nice if WordPress fixed it once and for all.