Have you experienced your blog being crawled by search engines but not indexed? You can check this out by using Google Webmaster Tool or other Analytics software. I had always no problem with installing new blog and got it indexed but unfortunately not this blog …

Blog is favored by search engines and it can almost indexed immediately in most search engines if there is good content. I remembered I had helped one of my clients to install a new blog with only one post and surprisingly, it was indexed the next day. That was the fastest record I ever had, I had tried installing other blogs for my other clients using wordpress and similarly, never encountered any indexing problem. However, I’m just clueless why I have no luck with this blog – ehomepreneur.com, it is already 4 months old but still not being indexed in any search engines. This kind of worries me as I’ve never experienced this before so for the last few months I was trying desperately all kinds of methods to get it indexed.

… Check my robot.txt
The first thing I checked was my robot.txt as I afraid I might accidentally block the search engine spiders without realizing it. I went through it line by line and there was nothing wrong. The most important line is the content and it is not blocked, it is stated as “Allow: /wp-content” so robot.txt is not the problem.

Result: No Change        Next Action: Go Next

… Set up Sitemap.xml using Google Webmaster Tool
Even though Google stated that submitting a sitemap.xml will not guarantee indexing but it definitely help to inform Googlebot about the structure of the blog and its updated pages easily. Hence setting up a sitemap.xml in Google Webmaster Tool is a must, at least by doing so, I know those pages which could be ignored or not discovered will be crawled again. I was able to get my sitemap verified but unfortunately there was still no pages indexed after checking back 2 weeks later.

Result: Failed          Next Action: Good Try!

… Submit Blog to Search Engines
I tried submitting the blog to major search engines like Google, Yahoo and Bing and waited patiently for them to crawl my blog. However, after 2 months, it was still the same … nothing happened.

Result: Failed          Next Action: Try harder!

… Back Link from High Ranking Sites
It was said that having back link from other high ranking blog will help to boost one’s indexing. One of the fastest ways to do this is to leave comments on some high-profile broggers’ blog. When I checked back using “link:” operator in Google few hours later, it was successfully listed. I was delighted and thought it worked this time but I was wrong, none of the pages was indexed even after a month.

Result: Failed          Next Action : Not hard enough

Then one day ….
 
I was trying to install a new plugin and suddenly the word “Privacy” at the WordPress Settings caught my eyes, I realized I’ve not been checking this since I installed this blog. And guess what? The blog visibility was set to “I would like to block search engines, but allow normal visitors”. I almost fainted when I saw this. Immediately I knew this was the culprit, it had caused me fear and sweat for the last few months wondering why my blog was not indexed and this was it! I always thought that the default setting for the blog is “I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archivers” as I had set up numerous blogs for my clients without any problems. I wish someone can tell me why? At that moment, I wanted very badly to bang my head against the wall for such a silly oversight. Two weeks later, I checked again on Google, Yahoo and yes, all the pages were listed now. This double confirmed that it was my silly mistakes.
 
So if you ever wonder why your blog is crawled and not indexed? Please do check your wordpress setting first. I hope you can learn from my mistake so you won’t have to make it and waste unnecessary time.

Related posts:

1. How to Verify Technorati Claim Code?

RSS & Email Subscription

Hi, if this is your first time here, you are welcome to subscribe to our RSS Feed or Feedburner Email. Click on the icons below to start now, once subscribed, you will not miss out any latest post.

       

Thank you for susbcribing and stay in touch.