WP Remix
“Finding Solutions in the World of Search Engine Marketing”

How to Break Up With a Search Engine Robot

Posted by Justin Smith at 5 November, 2008, 12:03 am

People in the search marketing industry are so obsessed with search engines.  All they can seem to talk about is how to please the search engines, get on their good sides, and kiss up to Matt Cutts in some way shape or form.  But so many people never talk about the different reasons you might want to break off a relationship.

I thought it would be fun to take a look at a few ways you can break up with a search engine and manage your relationships.

Breaking the Heart of Your Favorite Search Robot

Time to call it quits.  Maybe you have a private member website, want privacy, or simply don’t want your site to be in the search engine results.

To cut off all communication and break the heart of that poor search engine robot, insert the following into the header section of your pages:

“Let’s Just be Friends”

“It’s nothing personal, I still really like you, but we make better friends than lovers.”  Maybe you don’t want the search engines to index your pages (maybe you’re trying to combat duplicate content?), but you still want them to follow your links and find other pages of your site.

Use this in the header section of your pages:

The Messy Break Up

The search robot found out you were cheating and decided to pull your rankings.  There’s nothing worse than a messy break up.  Better make up as soon as possible.  Leave behind your unfaithful ways and beg for forgiveness.

“Let’s Agree To See Other People”

Sometimes it’s ok to see other people for a while.  Maybe you have a reason to want to include Googlebot, but exclude Yahoo Slurp from your website.  Or maybe you want to exclude a specialized bot like Linkscape, image search, etc.

The best way to allow and disallow certain robots is through robots.txt.  Here is a link to a very thourough tutorial on the subject of creating and using a robots.txt file.

Don’t Be a Stalker

Let’s face it, you know you are in love.  You’ve become obsessed with this relationship.  All your friends tell you that you need to break it off.  This confined intimacy is killing you, and you don’t even know it.  You’ve become so dependent upon approval that your entire livelihood is wrapped up in it.  You sit around at home wondering why she doesn’t call (err, crawl)  as often as she used to.

Don’t get too obsessed with the search engines.  You want a steady long lasting relationship, not a one night stand.

  • Don’t spam the engines
  • Don’t become over obsessed and forget that SEO is about marketing
  • Focus on metrics that matter like traffic, conversions, and business  (not just rankings)

===

Yes, this was more of a fun post, and I didn’t include alot of “details” about how to disallow certain robots, proper use of robots.txt, excluding certain directories, using webmaster tools, etc…  so if you have any questions, please comment and I’ll answer any specific questions you have…

Related posts:

Category : "How-To" SEO | Offbeat

TwitterCounter for @justinnerdLinkedInFacebookTwitter


Comments

Angie Weeks November 5, 2008

Too funny Justin! We also enjoy the “looking for love” websites, the ones wandering around without meta tags… searching for the Mr. Perfect mindreader robot :)

Always good info - thanks!

Bart Smith November 5, 2008

What a great blog….and I thought I had to call them….

Bart Smith

J Boyer Madison NJ November 5, 2008

Yes, way to funny. But here is an idea, what is the best bait for the search engine lover bot to use while still writing for normal humans?

Jodi Suguitan December 6, 2008

Great post! Got a laugh out of it, but I don’t think I’ll be breaking up anytime soon. Wish I could figure out how to make the robot monogamous!

Leave a comment

Need Help With SEO?

No problem. We have the solutions that can help your website improve ranking, traffic & sales. Give us an opportunity to create a free custom proposal for your business. » Get a Proposal »

Subscribe to Receive Future Articles

Via RSS Feed Reader

Contact Us

4833 Front St. Unit B#204
Castle Rock, CO 80104

Ph:720-670-0742 Fax:303-663-1560

Contact Form