Orange County Internet Marketing Advice by Darby Works, Inc.

Internet Marketing News

Outlook 2008: Online Marketing
Brandweek Magazine - New York,NY,USA
Search engine marketing (aka buying keywords) continues to be the dominant force in online advertising. A full 40% of online ad budgets are expected to be ...

It's clicking
Business Line - Chennai,India
Multiple conversion tracking tools are available - such as Webtrends, Hitwise, Click tracks, Core Metrics and, of course, Google Analytics - that track the ...

Google's Paid Links Double Standard: An Analogy
Traffick - USA
Matt Cutts is never going to be able to say: "you know what Ted? what the hell. we'll treat you the same as Techcrunch." This is for a whole lot of reasons. ...

Online Videos – How Old Media, New Technology Is Driving Search!
4Hoteliers - Wanchai,Hong Kong
No Cost Per Click/Action: Unlike PPC/ Paid Search, currently there is no cost per click charge for video views on Video Search Engines. ...

Pew/Internet Study Finds Most Americans Get Their Answers From the ...
Search Engine Land - Redding,CT,USA
Jupiter Research rearched similar conclusions, finding that 71% of online consumers use search engines to find health-related information. ...

Celebrities and gadgets lead Web searches
Buffalo News - NY, United States
That’s based on new data from the largest Internet search engines showing what their users looked for when they went online this year. ...

It is the sales stupid!
MediaShift Idea Lab - San Francisco,CA,USA
Now, at least for the time being, I'm declining Google (adsense) and any pay per click routine. Why? Many reasons but one of them is that Google's publisher ...

Robots.txt 101

Sorry it's been so long since I posted. Frankly, I've been a little burned out and needed a break. But I am back now and good to go...at least I think!

Anyway, here's a quick rundown of an important file to have on your web server right on the root. It's called robots.txt. Uploading the file to the root means that it is not located in any subfolder. After you upload this file to your server, you should be able to access the file by typing this into your web browser:  http://www.mydomain.com/robots.txt

Robots.txt helps the visiting search engine spiders navigate to the pages and directories on your website. You can also set your robots.txt to tell the spiders to block certain directories on your site. One last item is that you can add the location of your site map so the spiders can find it.

Here is a sample robots.txt:

User-agent: *
Disallow: /cgi-bin/
Disallow: /private/
Sitemap: http://www.mydomain.com/sitemap.xml

The "User-agent: *" means this bit of code applies to all search engine robots.

The "Disallow:  tells the robots that they should not visit any pages within those directories.If you are okay with the robots visiting all areas of your site then simply don't put anything after it like this:  Disallow:

The line of code about the sitemap tells the spider where your sitemap is located. A sitemap is basically a list of all individual webpages on your site saved in a format called XML. I can't get into all the details here, but I will say that it is VERY important you create a sitemap! There a number of free tools online that will help you do this. Google also has some sitemap resources as well. I'll give more sitemap explanation in another post.

Remember that robots.txt can't force the spiders to do anything. If that were the case anyone could easily manipulate search engine results. However, having a robots.txt helps guide the spiders to the areas they need to go.

You can also use robots.txt to block visiting spiders, but again, some robots can ignore your robots.txt. Don't try to use your robots.txt for website security!

To create the file, open up notepad. Type in the code you want. Then save the file. Be sure to save it all lower case - robots.txt. Then upload the file to your webserver root.

To learn more advanced aspects of robots.txt, you can visit http://www.robotstxt.org.