Google can't retrieve robots.txt ???
Set up a blog, set the settings to be searchable and indexed by search engines… got a meta code from google and entered it in settings, it validated ok but now has warning up that google can’t retrieve robots.txt and therefore won’t crawl the blog.
Is this common? i’ve seen a few similar posts in forums similar and all answers seem to suggest clicking making it searchable in settings.. but i’ve done that
(yes i know it’s now set to private as i’m now looking for alternatives).
The blog I need help with is ukworkingman.wordpress.com.
You have published a single post. You need to post frequently. It takes weeks for search engines to index content in a new blog and/or to re-index content under a new URL. What attracts search engines is unique content the cannot in posts that cannot be found anywhere else on the internet. Start now publishing original content posts frequently two or even three times weekly. For more information read > http://onecoolsitebloggingtips.com/2010/01/21/omg-i-cant-find-my-blog-on-google/
To verify blog ownership of a WordPress.com blog with the major search engines you must use this process > http://en.support.wordpress.com/webmaster-tools/
Note: Even if you do not verify the blog the content will be indexed by search engines so don’t feel panicky about this please.
Note also that WordPress.com automatically supplies sitemaps for our blogs to search engines – we do nothing. http://en.support.wordpress.com/sitemaps/#xml-sitemaps-for-search-engines
thanks for the response but i didnt actually ask about getting indexed… the question was about googlebot being blocked by robots.txt.
The is only one post as the rest have been exported to a self hosted installation now and has already been indexed within 12hrs
the one post has been left while i try to figure out this problem.
Specifically which URLs are being reported to you by Google as blocked by robot text?
the whole blog
I’ll tag this thread for Staff assistance. Please subscribe to the thread so you are notified when they respond and please be patient while waiting.
blog deleted will try again with a different blog.
The topic ‘Google can't retrieve robots.txt ???’ is closed to new replies.