Oh my. You have worse problems than that, much worse, but they're easily fixed.
It comes from introducing code errors into your blog, usually from copy-pasting from Word or some other non-web-compatible software.
Here's how to fix it and prevent it from ever happening again:
1) instead of Word, use Windows Live Writer. It allows you to add tags and categories and format and preview your posts even when you're offline.
2) go to Dashboard->Settings->Writing and select the tickbox for "Wordpress should correct invalidly nested XHTML" and Save Changes.
3) Go to Dashboard->Posts and select the latest 20 posts. Using the Bulk Editor, set them all to Draft status. Save Changes. Then set them back to Public and save that.
This will force the setting you changed in #2 to take effect on all of those posts, and that should fix the issue. That setting will prevent it from ever happening again.
Sounds like a lot, but it's really only a few clicks.