toy robots

Meta Robots NOINDEX

/in /by

NOINDEX can be even more damaging than a misconfigured robots.txt file.

Most commonly, the NOINDEX is set up when a website is in its development phase. Since so many web development projects are running behind schedule and pushed to live at the last hour, this is where the mistake can happen. A mistakenly configured robots.txt won’t pull your pages out of Google’s index if they’re already there, but a NOINDEX directive will remove all pages with this configuration.

Most developers will make sure this NOINDEX is removed from your site, but if you’re having ranking problems, you should check this immediately.

What to do:

  • Manually do a spot-check by viewing the source code of your page, and looking for one of these:

<META NAME=”ROBOTS” CONTENT=”NOINDEX, FOLLOW”>

<META NAME=”ROBOTS” CONTENT=”INDEX, NOFOLLOW”>

<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>

90% of the time you’ll want it to be either “INDEX, FOLLOW” or nothing at all. If you see one of the above, you need to take action.

What to do next:

If your web site is constantly being updated and improved by your development team, set a reminder to check this weekly or after every new site upgrade.