Need help? Check out our Support site, then


WordPress.com Forums » Support

Robots.txt not working correctly

  1. foleysfolly
    Member

    Hello,

    I have had an issue with my blog. I recently opened a Google Webmaster account. Everything seemed to be going smoothly. On my wordpresss.com dashboard I went to Tools>Available Tools and verified my site. I then went to Settings>Privacy and clicked 'allow search engines to index this site'.

    However, my google webmaster account keeps saying "URL restricted by robots.txt" I have searched dozens of sites and have not found a solution. I have also given it a couple of days to maybe 'work itself out'.

    It still says 'Disallow' at http://foleysfolly.com/robots.txt

    Anyone have a solution?

    The blog I need help with is foleysfolly.com.

  2. auxclass
    Member

    http://ismyblogworking.com/foleysfolly.com

    Shows your robots.txt as allowing search engines in - the syntax as I remember is a bit odd - if Google was locked out of an area there would be a directory showing

  3. foleysfolly
    Member

    Thank you for your reply.

    I did check ismyblogworking and did see my sites was working properly, However, I was unsure as to why It keeps showing up in my webmaster account as disallowed.

    If I'm in my webmaster account and I go to Site Configuration>Crawler Access and run a test, it gives the following error:

    Disallowed:Allowed by line 2: Disallow:
    Detected as a directory; specific files may have different restrictions

    Should I trust ismyblogworking over my webmaster account?

  4. auxclass
    Member

    Your robots.txt file is now updated and looks way different than it did a while ago - forgot the time lag factor - it can take a while for the file to be updated.

  5. foleysfolly
    Member

    Wow, thats pretty amazing the robots.txt file changed within the 30 minutes I posted this lol. Thank you for the help, although I ran another test on webmaster and it still says its disallowed. As always, these things seem to work themselves out.

    Thanks again.

  6. auxclass
    Member

    You be welcome & good luck

Topic Closed

This topic has been closed to new replies.

About this Topic

Tags