Blog blocked by robots.txt Need Help

  • Author
    Posts
  • #886725

    I updated my blog to domain .net and blog is being blocked by robots.txt , why?

    The blog I need help with is whitelineninteriors.net.

    #887033

    Hi,

    What’s your blog URL?

    Where are you seeing that robots.txt is being blocked? Can you reply with the error message that you’re seeing?

    You can configure your blog privacy settings in Settings -> Privacy from your blog dashboard, and you’ll want it to be set to “Allow search engines to index this site”.

    WordPress.com automatically sends notifications to Google with every post and page update. The search engines can take 4-6 weeks or more to add any new sites. Here’s more information about WordPress.com and Search Engines.

    #887052

    Hi Bryan
    Here’s my URL
    http://whitelineninteriors.net

    Google analytics – (Crawls error) and Blocked URL says that it hasn’t been able to crawls because of the robots.txt

    Also the server connectivity blocked and robots.txt fetch inaccesable
    I have the privacy settings to allow search engines to index site.

    thanks for your help.

    #887053

    motre
    Member

    When I fetch http://whitelineninteriors.net/robots.txt myself, I see a normal looking robots.txt file that is only blocking things that should be blocked. I also accessed http://whitelineninteriors.wordpress.com/robots.txt and it is the same except that it also blocks the /admin/ URLs (no reason for Google to crawl any page like that which it has no access to anyway).

    What do you mean by “the server connectivity blocked”? If you are blocked from the server, then you would not be able to access anything in your page. Can you explain what you see happening with each specific URL you try?

    #887166

    Would’nt be the following instructions blocking the crawl on keywords ?
    “User-agent: *
    Disallow: /related-tags.php”

    I have the same problem here on http://2050ruedessources.net/robots.txt
    No trafic on keywords :(
    Can’t manage to fix it !

    #887167

    cadolphin
    Member

    I’m also interested in how to fix this issue. I have the same problem: http://www.earth4energysolarpowerreview.com/robots.txt

    I’ve double checked all the privacy settings to make sure that “Allow search engines to index this site” has been checked. I’ve also checked all the plug-ins to see if any are blocking access.

    I can’t find a robots.txt file in the root directory to make a change.

    Help would be most appreciated.

    #887168

    auxclass
    Member

    @cadolphin

    The site you are asking about does not seem to be hosted on WordPress.COM so you need to make friends over at WordPress.ORG the keepers of the software you are using.

    https://en.forums.wordpress.com/topic/7-things-to-know-before-posting-in-wordpresscom-forums?replies=1

    This site is for support of sites hosted on WordPress.COM. You should address your questions to WordPress.ORG the keepers of the software you are using: http://wordpress.org/support/

    For more on the difference: http://support.wordpress.com/com-vs-org/

    #887169

    cadolphin
    Member

    Sorry. I didn’t realize that they were different. My apologies.

    #887170

    auxclass
    Member

    It’s OK – good luck – some days 30% of the traffic here is WordPress.ORG installs.

The topic ‘Blog blocked by robots.txt Need Help’ is closed to new replies.