Robots.txt not working correctly
Shows your robots.txt as allowing search engines in – the syntax as I remember is a bit odd – if Google was locked out of an area there would be a directory showing
Thank you for your reply.
I did check ismyblogworking and did see my sites was working properly, However, I was unsure as to why It keeps showing up in my webmaster account as disallowed.
If I’m in my webmaster account and I go to Site Configuration>Crawler Access and run a test, it gives the following error:
Disallowed:Allowed by line 2: Disallow:
Detected as a directory; specific files may have different restrictions
Should I trust ismyblogworking over my webmaster account?
Your robots.txt file is now updated and looks way different than it did a while ago – forgot the time lag factor – it can take a while for the file to be updated.
Wow, thats pretty amazing the robots.txt file changed within the 30 minutes I posted this lol. Thank you for the help, although I ran another test on webmaster and it still says its disallowed. As always, these things seem to work themselves out.
You be welcome & good luck
The topic ‘Robots.txt not working correctly’ is closed to new replies.