Let me be blunt and inform you that I click the button on thousands every day, not the paltry number you report, and I don't consider that to be a big deal at all.
The vast and overwhelming majority of all so-called information on the internet is spam and much of it is pornography? Over 80% of all so-called comments submitted to free hosted WordPress.com blogs every day is spam.
Over 80% of all so-called comments submitted to .wordpress.com blogs every day are spam. Akismet is sequestering the spam you point to. On the spam that Akismet catches it takes only seconds to scan the pages, remove any false positives, and click "empty spam".
You don't even have to click "empty spam" if you don't want to. Akismet will maintain comments it has caught for 15 days from the time they were received, and then it will delete the comment automatically.
If you get spam that slips by Akismet then mark it as spam (do not delete it) and over time Akismet will learn it's spam.
For more information please take the time to read "5 things every blogger should know about spam" on this page Akismet: How it works http://akismet.com/how/
If you have et up the blog for anonymous commenting that does not require an email address and/or website URL then do be sure to set up full comment moderation ie. "An administrator must always approve the comment" because anonymous commenting is bound to increase the amount of spam your blog will receive. http://en.support.wordpress.com/settings/discussion-settings/#comment-moderation