John Nasta » optimization

Excluding Your Web Site From Search Engines

John Nasta — Sat, 18 Apr 2009 00:03:12 +0000

People often write about how to increase your search engine ranking by using noindex and nofollow meta tags to avoid indexing of links & pages that lead to duplicate content, and robots.txt files to invite the search engines in, but what if you don’t want your site indexed at all? This is useful for private organizations and anyone who does not want their site indexed for any reason.

Noindex is fairly self explanatory. Simply put, a noindex tag tells the search engines not to index this page.

Nofollow is a similar tag. It tells the search engines not to follow any of the links on the page.

That’s just the beginning…

By adding this code to <head> section of all of your pages:

<meta name='robots' content='noindex,nofollow' />

you can tell the search engines not to do either of the above.

You can also create a plain text file called robots.txt containing this code:

User-agent: * Disallow: /

Upload it to the root directory of your site. This will tell all “user agents” (a.k.a. search engines) to disallow indexing of all pages in the root directory as well as any directory within it.

So, what happens if you found this article too late and your site has already been indexed. How can you get it de-listed?

There are two answers…

The first is to upload the robots.txt file as described above. The next time your site is crawled, it will be removed from the index.

If you want your site removed from Google, and you can’t wait until the next time it’s crawled, sumbit a Google Urgent Removal Request. Google says that this process will take 3 to 5 business days.

Is there more to the story? Of course there is!

You can also use this tag in the <head> section of your pages:

<meta name="robots" content="noodp,noydir,noarchive" />

noodp – Prevents all search engines from using the DMOZ description for your site in the search results.

noydir – Prevents Yahoo! from using the Yahoo! directory description for your site in the search results.

noarchive – Prevents sites like archive.org and Google from putting copies of your pages into their archive or cache.

What about WordPress sites? You may have already noticed that you can click Settings > Privacy and choose the “I would like to block search engines, but allow normal visitors” option. This will add noindex and nofollow tags to all of your site’s pages EXCEPT the Login and Forgot Password pages. A nice way to add these tags to those pages is to use the Robots Meta plugin by Joost de Valk. This plugin actually does more than just that. Read the instructions for a better understanding of the plugin’s features.

So, now your site is not being indexed, but people can still link to it from other sites or manually type in the URL. What can you do about that?

WordPress users have the option of marking their pages and posts Private, which means that only those subscribers whose Role allows them to see those pages and posts can access them. There’s also the User Access Manager plugin that allows you to assign which subscribers can read selected pages and/or posts. You can even assign people who are not subscribed and/or logged in by IP address or IP range.

Static sites can be password protected by putting a .htaccess file in the folder. If you don’t know how to create a .htaccess file, most commercial web hosts will do this for you via your Plesk or Control Panel interface.

Who would have thought that there’s as much involved in keeping people out of your site as there is in getting them to find it and come in?

This article copyright © John Nasta 2009 – All Rights Reserved

Share This!

SEO Google Search Ranking Optimization Tips – Titles and Title Tags Here is a Google SEO optimization technique that anyone can employ on just about any...
SEO – Avoiding Duplicates with Canonical URL Tags You may have seen my article about the Ultimate Noindex Nofollow tool for WordPress, but...
Changing The URL Of Your WordPress Site WordPress web site owners often come to realize that they want to change the URL...

WordPress SEO – Permalinks Options and Structure

John Nasta — Mon, 06 Apr 2009 21:06:11 +0000

A great way to increase your blog’s search engine friendliness is to customize your permalink structure. This is extremely simple to do and ideally should be done as soon as you set up your blog, before people start linking to it. However, the good news is that the old “ugly” links will still work (and they are shorter, which can be better for emailing).

In your WP Admin, click Settings > Permalinks

Click the “Custom Structure” radio button

Copy & paste this code into the box to the right to show the year, category name, and post name in the Permalink (this is probably the most popular link structure):

/%year%/%category%/%postname%/

Note that it is a good idea to START your permalink structure with a numeric value to avoid database problems, and END it with the post name (or I.D.) to make sure that it points to a unique post. The WordPress documentation on this topic can be found here.

The full list of “tokens” that can be used is:

%year% – The year of the post, four digits, for example 2004

%monthnum% – Month of the year, for example 05

%day% – Day of the month, for example 28

%hour% – Hour of the day, for example 15

%minute% – Minute of the hour, for example 43

%second% – Second of the minute, for example 33

%postname% – A sanitized version of the title of the post (post slug field on Edit Post/Page panel). So “This Is A Great Post!” becomes this-is-a-great-post in the permalink URL

%post_id% – The unique ID # of the post, for example 423

%category% – A sanitized version of the category name (category slug field on New/Edit Category panel). Nested sub-categories appear as nested directories in the URI.

%tag% – A sanitized version of the tag name (tag slug field on New/Edit Tag panel).

%author% – A sanitized version of the author name.

In the category base box, enter a simple word like categories or topics. This will alter your category links to include the word you entered as well as the category name.

Likewise, enter a simple word like tags into the tag base box.

Click “Save Changes”

You’re done!

A couple of nice side effects of doing this are that your page links will also be modified to include the page name, and when editing your posts you will now have a button that will allow you to enter any words you like into your permalink for that post. Note that if you change a post’s title, the permalink will continue to show the old title unless you edit it as well.

Amazing, and amazingly simple!

This article copyright © John Nasta 2009 – All Rights Reserved

Share This!

WordPress SEO Plugin – The Ultimate Noindex Nofollow Tool Now that my blog has been up a few weeks I feel like I finally...
SEO – Avoiding Duplicates with Canonical URL Tags You may have seen my article about the Ultimate Noindex Nofollow tool for WordPress, but...
Changing The URL Of Your WordPress Site WordPress web site owners often come to realize that they want to change the URL...

SEO Google Search Ranking Optimization Tips – Titles and Title Tags

John Nasta — Fri, 27 Feb 2009 20:51:51 +0000

Here is a Google SEO optimization technique that anyone can employ on just about any web page. In a nutshell, write better titles and title tags.

Static web pages generally have a <title> tag in the <head> section of the document, and in many dynamically generated sites (blogs, ecommerce, etc.) the page’s Title tag is the same as the on-page title of the page, post, product, etc.

Search engines read up to 255 characters of the page title, but they weigh the relevance of those words based on the assumption that the most important word in the title is the very first word, and the importance of each subsequent word drops accordingly.

That’s why the first word of the title of this article is SEO. That is the single term that I thought people who might click on this article might be searching for. It is followed accordingly with what I felt were important keywords, and written in the exact order somebody might type those words (exact matches score higher). It’s no accident that the title of this article reads exactly the way it does. I want people to find it!

The search engines will also rank your site more favorably if your meta tags (keywords, description, title, etc.) are different for every page and tailored to the content of the page. Very common mistakes include giving every page the same title, no title, or having the first handful of words of every title the same.

Also, bear in mind that if you use a plural or longer form of a word, it will come up in search results for shorter forms of the word (e.g. use Lawyers instead of Law, etc.).

So, if you are a lawyer named John Smith who specializes in divorces and custody matters, you might want to have a page with a title like “Divorces Child Custody Family Lawyers Attorneys John Smith” rather than “John Smith Attorney at Law specializes in civil matters including divorce and child custody” because you want the most important buzz words, especially those that differentiate that page from the other pages on your site, at the front of your title.

Keywords and Descriptions also come into play and I will discuss those separately, but as you can see SEO begins when you write the first piece of content for your web site.

This article copyright © John Nasta 2009 – All Rights Reserved

Share This!

SEO – Avoiding Duplicates with Canonical URL Tags You may have seen my article about the Ultimate Noindex Nofollow tool for WordPress, but...
Excluding Your Web Site From Search Engines People often write about how to increase your search engine ranking by using noindex and...
WordPress SEO – Permalinks Options and Structure A great way to increase your blog’s search engine friendliness is to customize your permalink...

SEO – Avoiding Duplicates with Canonical URL Tags

John Nasta — Fri, 27 Feb 2009 19:04:26 +0000

You may have seen my article about the Ultimate Noindex Nofollow tool for WordPress, but avoiding duplicate content can help the search engine ranking of any web site, including static sites.

Recently, Yahoo!, Live, and Google all agreed to support Canonical URL Tags. You put a line of code into the head section of your document like the one below:

<link rel="canonical" href="http://www.johnnasta.com/blog" />

This tells the search engines that this page URL should be regarded as a copy of johnnasta.com/blog and that any link or content metrics should be applied to that URL.

When your site contains multiple URLs to the same content (for example blogs and ecommerce platforms can contain links to the same articles or products via the home page, category pages, search results, etc. each creating its own path to the same article or product), your search ranking suffers. The search engines don’t know which ones to index or rank, and your metrics get spread out over all the streams instead of always pointing to the ultimate URL of the individual piece of content.

Examples of links that might point to the same content are:

http://johnnasta.com/blog/?category=1

and

http://johnnasta.com/blog

This happens because this article currently appears on both my home page as well as the page for the category that the post is assigned to, but the “real” URL to this post that I actually want Google to index is (currently):

http://johnnasta.com/blog/?p=296

(yeah, I know I haven’t dressed up my permalinks yet – one thing at a time)

Makes sense?

Share This!

SEO Google Search Ranking Optimization Tips – Titles and Title Tags Here is a Google SEO optimization technique that anyone can employ on just about any...
Excluding Your Web Site From Search Engines People often write about how to increase your search engine ranking by using noindex and...
WordPress SEO – Permalinks Options and Structure A great way to increase your blog’s search engine friendliness is to customize your permalink...

WordPress SEO Plugin – The Ultimate Noindex Nofollow Tool

John Nasta — Wed, 25 Feb 2009 21:54:49 +0000

Now that my blog has been up a few weeks I feel like I finally have some time to start thinking about SEO (Search Engine Optimization).

Today I’m checking out The Ultimate Noindex Nofollow Tool plugin by Jonathan Kemp.

This plugin allows you to add “noindex” and “nofollow” tags to your WordPress blog pages.

Right now some of you are thinking “Great! Why?”.

The reason this is desirable is because by default, WordPress creates a lot of duplicate content. For example, you can read the same post on the home page as you can by clicking the link for the category that the post is in, or the one for the month that it was posted, or by finding it with the on-site search, etc. This actually hurts your site’s ranking, so it behooves you to install this plugin or something very much like it. What I like about this plugin is that it lets you check off the common pages that you would like the tags to be placed in, and also lets you type in the page IDs of any others that you’d like to add. What more could you want?

BTW, this is also why you should assign your posts to only one category. Assigning them to more than one is another way of creating duplicate content, and thus hurting your site’s ranking.

Since most bloggers rely on the idea of building up traffic and statitics in order to gain clients or sell advertising, blogging without this plugin is like… well, let’s just say that you get what you deserve if you don’t use one.

Share This!

SEO – Avoiding Duplicates with Canonical URL Tags You may have seen my article about the Ultimate Noindex Nofollow tool for WordPress, but...
Excluding Your Web Site From Search Engines People often write about how to increase your search engine ranking by using noindex and...
WordPress SEO – Permalinks Options and Structure A great way to increase your blog’s search engine friendliness is to customize your permalink...