It’s in your dashboard. Dashboard > Manage > Categories, and you can delete or edit any categories you’ve added. You’ll have two default categories you can’t delete, though; just change the name to something else instead.
My dashboard only has categories there, not tags. Is there any way to delete the tags? I deleted a post, but the tags it had are still listed in the tag cloud, even though no other posts use those tags.
According to this FAQ, which goes into detail about the differences between tags and categories, to delete a tag all you have to do is open the post and delete the tags from it. Perhaps the tag cloud takes a little while to refresh itself? Or perhaps you need to clear cookies, temp files, cache, etc from your browser so that it sees the page fresh.