Robots Meta Tag

Robots Meta Tag For SEO

Sometimes you want to only exclude a single page, but you don’t want to actually mention that page in a robots.txt.

The best way, of course, is to implement real file-level security on the file. But failing that you have 2 other options:

1. Create a folder and put the files you wish to exclude in the folder. then exclude that folder in the robots.txt file. This will automatically exclude everything under it, but you will not need to actually name the files.

2. Use the robots metatag on the page(s) in question. What will happen is that the search engine will look at you robots.txt, see that the page is not mentioned, and go to it. But then it will see that robots are actually excluded from the page in the robots metatag and it will then obey it.

I’ve gone into the robots.txt file in detail elsewhere, so let’s talk about the robots metatag.

Robots Meta Tag Code Example

<meta name=”robots” content=”noindex,nofollow”>

-or-

<meta name=”robots” content=”none”>

This example will tell all robots not index this page, and to not follow links from this page.

Attributes and Directives

There are other options, of course:

content = all | none | directives

all = “ALL”
none = “NONE”
directives = directive [“,” directives]

directive = index | follow

index = “INDEX” | “NOINDEX”
follow = “FOLLOW” | “NOFOLLOW”
This results in the following choices:

  • <meta name=”robots” content=”index,follow”>
  • <meta name=”robots” content=”noindex,follow”>
  • <meta name=”robots” content=”index,nofollow”>
  • <meta name=”robots” content=”noindex,nofollow”>

Plus the two attributes:

  • <meta name=”robots” content=”all”>
  • <meta name=”robots” content=”none”>

ALL is the equivalent of “index, follow”

NONE is the equivalent of “noindex, nofollow”

You can find more information here: http://www.robotstxt.org/wc/meta-user.html. These attributes are NOT CASE SENSITIVE – you can use upper or lower case.

Important Note: Robots will treat the absence of a specific disallowing tag (“noindex” and/or “nofollow”) as permission. Therefore if you do not use “noindex” for example, the robots will assume you mean “index”.

For this reason, it’s usually a waste of time to use the “index,follow” or “all” directives, since they are assumed already as a default. These are only useful when you wish to specify, for example, that all robots are to noindex,nofollow, but a specific robot (ie googlebot) is allowed to. In this case, you would put in something like this:

<meta name=”robots” content=”noindex,nofollow”>
<meta name=”googlebot” content=”index,follow”>

If you did not specifically tell Googlebot to index and follow, it would in this case assume that you did not want it to. This is the only scenario where the index and follow (and ALL) directives are used. SEO’s who misuse metatags are in danger of losing both respect from their peers, and savvy clients who are checking their basic SEO abilities before hiring.

It won’t hurt you to use the ALL or index,follow robots meta directives by itself, but it makes you look bad. You should know how to your your own tools.

Engine Specific Commands

All 4 major search engines (Google, Yahoo, MSN, Ask) also support the NOARCHIVE attribute.

Google

NOARCHIVE: Google uses this to prevent archiving (caching) of a page. See http://www.google.com/bot.html

Although Google will follow this if it applies to all robots, since only Google uses it, the best method of using this is to specify  “googlebot”, rather than “robots” in general.

Example:

<meta name=”googlebot” content=”noarchive”>

You can also combine it:

<meta name=”googlebot” content=”nofollow,noarchive”>

-or-

<meta name=”googlebot” content=”noindex,nofollow,noarchive”>

etc..

NOSNIPPET: A snippet is a text excerpt that appears below a page’s title in Googles search results and describes the content of the page.

To prevent Google from displaying snippets for your page, place this tag in the <HEAD> section of your page:

<META NAME=”GOOGLEBOT” CONTENT=”NOSNIPPET”>

Note: removing snippets also removes cached pages (acts as NOARCHIVE,NOSNIPPET)

Yahoo

Yahoo also obeys the NOARCHIVE attribute.

Ask/Teoma

Ask obeys the NOARCHIVE attribute.

MSN Search

MSN Search obeys the NOARCHIVE attribute, and if you use the NOCACHE attribute, it acts as the NOARCHIVE attribute.

Free Robots Metatag Generator

My free meta tag generator includes a simple section for the generation of a robots metatag for your use.

Conclusion

The robots metatag is a simple method of telling robots what you want and don’t want indexed or followed on your site. Like the robots.txt file, it is NOT secure, nor is it a requirement. Rogue robots can ignore it if they wish to. Further, a human is under no restrictions at all.

The robots metatag has the advantage of being very granular – it allows you to restrict the spidering behavior of a single page. That same granularity is also a disadvantage – it’s a real pain to put one on a lot of pages, and to keep track of what pages you used it on. It’s best used for small numbers of specific pages.

Rule of thumb: If you want to restrict robots from entire websites and directories, use the robots.txt file. If you want to restrict robots from a single page, use the robots metatag. If you are looking to restrict the spidering of a single link, you would use the link “nofollow” attribute.

Granularity Best Method
Websites or Directories robots.txt
Single Pages robots metatag
Single Links nofollow attribute
Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2004 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

gTLD

TLD (Top Level Domain)

This is page three of a series, and it lists and discusses Top Level Domains (TLD). Page one of this series lists and discusses Language Codes and Attributes. Page two discusses ccTLD (country code Top Level Domain) issues.

IANA

The top level domain (TLD) system is run by IANA (Internet Assigned Numbers Authority) which also assigns IP addresses and ranges, ports, and other related attributes.

ICANN

For those of you who were wondering, ICANN is the private contractor with an MOU (Memorandum of Understanding) with the US government that currently runs IANA, so sometimes people use the organizations interchangeably. They are not, however. As a matter of fact there are a lot of people who are not happy with ICANN at all.

One of my clients is Industry Canada, and I discovered a very well written description of ICANN in their website I’d like to share:Consultation on ICANN reform.

Registering a gTLD

There are many registrars for gTLD’s. I personally use BulkRegister, but many people like GoDaddy.

Top Level Domain List

Generic TLD (gTLD)

Although each of these domains originally had a purpose, anyone can register these for any reason now. This lack of control and direction is the force behind the restricted and new sponsored TLD.

gTLD

Intended Target

Operated by

.com Commercial VeriSign Global Registry Services
.net Technical / Internet VeriSign Global Registry Services
.org Catch-all for anything non-commercial and non-government, including nonprofit and nongovernmental organizations, philanthropies, charities, religious organizations, educational and cultural institutions, arts organizations, sports clubs, and others. Public Interest Registry
.info Information Resources Afilias Limited

Restricted / Reserved TLD (rTLD)

You have to be a member of the restricted class in order to register these domains, and each operator has rules for how stringent the qualifications are.

rTLD

Restricted To

Operated by

.biz Business NeuLevel, Inc.
.name Individuals Global Name Registry
.pro Credentialed Professionals and Related Entities RegistryPro
.edu US Accredited Post-Secondary Institutions Educause
.gov US Government US General Services Administration
.mil US Military US DoD Network Information Center
.int Organizations Established by International Treaty IANA .int Domain Registry

Sponsored TLD (sTLD)

Like rTLD’s, You have to be a member of the “reserved for” class in order to register these domains, and each sponsor has rules for how stringent the qualifications are.

These domains are quite new and as a result many have not been indexed by search engines yet. I have verified that search engines have no problems finding and indexing these, however.

sTLD

Reserved For

Sponsored by

.aero Air Transport Industry S.I.T.A.
.coop Cooperative Associations Dot Cooperation LLC
.jobs Human Resource Managers Employ Media LLC
.mobi Mobile Products and Services mTLD Top Level Domain, Ltd
.museum Museums Museum Domain Management Association
.travel Travel Industry Tralliance Corporation

 

Third Party Option – New.Net

There is another option I’ve been asked about – www.new.net offers a bunch of descriptive TLD’s (I’ll call them nTLD for consistency in this article). The difference is that new.net domains are NOT authorized by IANA and require the user to install a plugin for their browser (or web hosts to specifically add new.net to their DNS servers).

Although I like the idea of new domains, in this case, the lack of authorization and the resulting requirement for a plugin make this option undesirable, especially from a SEO perspective. Witness this quote from their FAQ:

Currently, in order to submit your New.net domain URL into search engines you will need to use the “long” URL. For example, if you own www.pie.shop, you would need to submit it as www.pie.shop.new.net. We are actively pursuing search engine partnerships so that domains can be submitted with the “short” URL.

The problem is that they have been saying that last line for several years now and nothing has happened. I personally doubt it ever will. Note that .travel is now a sponsored TLD that IS authorized (by a completely different organization) this means there is now a conflict – something IANA has a mandate to prevent.

This leads me to 2 opinions: 1) IANA does not care in the slightest about new.net, and 2) professional network administrators who care about the integrity of their networks will not allow such a conflict, and the IANA version will win.

As a matter of fact, this letter from ICANN to new.net rejecting their position is pretty clear.

Here is a list of some of the nTLDs:  .agent .church .game .law .ltd .scifi .tech .arts .club .golf .llc .med .shop .travel
.auction .family .inc .llp .mp3 .soc .video .chat .free .kids .love .school .sport .xxx

One last thought – several anti-spyware software flag the new.net plugins are potentially dangerous, and suggest that they be removed. Many consumers follow these suggestions. Do a search on new.net in most search engines and you’ll find most of the ads showing up in relation to the search offer to “remove the new.net spyware”.

Given the above, I do not recommend the use of new.net domains at this time, and probably will not in the future.

 


Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2006 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

ccTLD

Country Code and ccTLD (country code Top Level Domain)

This is page two of a series, and it lists and discusses country codes and ccTLD’s. Page one of this series lists and discusses Language Codes and Attributes. Page three discusses gTLD (generic Top Level Domain) issues.

IANA

The top level domain (TLD) system is run by IANA (Internet Assigned Numbers Authority) which also assigns IP addresses and ranges, ports, and other related attributes.

ICANN

For those of you who were wondering, ICANN is the private contractor with an MOU (Memorandum of Understanding) with the US government that currently runs IANA, so sometimes people use the organizations interchangeably. They are not, however. As a matter of fact there are a lot of people who are not happy with ICANN.

One of my clients is Industry Canada, and I discovered a very well written description of ICANN in their website I’d like to share:Consultation on ICANN reform.

Registering a ccTLD

There are many registrars for ccTLD’s. I personally use CanReg for .ca domains, and BulkRegister for the rest.

Country Code Domain (ccTLD) List

Sorted By Country

Sorted By Domain

Afghanistan .af Ascension Island .ac
Aland Islands .ax Andorra .ad
Albania .al United Arab Emirates .ae
Algeria .dz Afghanistan .af
American Samoa .as Antigua and Barbuda .ag
Andorra .ad Anguilla .ai
Angola .ao Albania .al
Anguilla .ai Armenia .am
Antarctica .aq Netherlands Antilles .an
Antigua and Barbuda .ag Angola .ao
Argentina .ar Antarctica .aq
Armenia .am Argentina .ar
Aruba .aw American Samoa .as
Ascension Island .ac Austria .at
Australia .au Australia .au
Austria .at Aruba .aw
Azerbaijan .az Aland Islands .ax
Bahamas .bs Azerbaijan .az
Bahrain .bh Bosnia and Herzegovina .ba
Bangladesh .bd Barbados .bb
Barbados .bb Bangladesh .bd
Belarus .by Belgium .be
Belgium .be Burkina Faso .bf
Belize .bz Bulgaria .bg
Benin .bj Bahrain .bh
Bermuda .bm Burundi .bi
Bhutan .bt Benin .bj
Bolivia .bo Bermuda .bm
Bosnia and Herzegovina .ba Brunei Darussalam .bn
Botswana .bw Bolivia .bo
Bouvet Island .bv Brazil .br
Brazil .br Bahamas .bs
British Indian Ocean Territory .io Bhutan .bt
Brunei Darussalam .bn Bouvet Island .bv
Bulgaria .bg Botswana .bw
Burkina Faso .bf Belarus .by
Burundi .bi Belize .bz
Cambodia .kh Canada .ca
Cameroon .cm Cocos (Keeling) Islands .cc
Canada .ca Congo, The Democratic Republic of the .cd
Cape Verde .cv Central African Republic .cf
Cayman Islands .ky Congo, Republic of .cg
Central African Republic .cf Switzerland .ch
Chad .td Cote d’Ivoire .ci
Chile .cl Cook Islands .ck
China .cn Chile .cl
Christmas Island .cx Cameroon .cm
Cocos (Keeling) Islands .cc China .cn
Colombia .co Colombia .co
Comoros .km Costa Rica .cr
Congo, Republic of .cg Serbia and Montenegro .cs
Congo, The Democratic Republic of the .cd Cuba .cu
Cook Islands .ck Cape Verde .cv
Costa Rica .cr Christmas Island .cx
Cote d’Ivoire .ci Cyprus .cy
Croatia/Hrvatska .hr Czech Republic .cz
Cuba .cu Germany .de
Cyprus .cy Djibouti .dj
Czech Republic .cz Denmark .dk
Denmark .dk Dominica .dm
Djibouti .dj Dominican Republic .do
Dominica .dm Algeria .dz
Dominican Republic .do Ecuador .ec
East Timor .tp Estonia .ee
Ecuador .ec Egypt .eg
Egypt .eg Western Sahara .eh
El Salvador .sv Eritrea .er
Equatorial Guinea .gq Spain .es
Eritrea .er Ethiopia .et
Estonia .ee European Union .eu
Ethiopia .et Finland .fi
European Union .eu Fiji .fj
Falkland Islands (Malvinas) .fk Falkland Islands (Malvinas) .fk
Faroe Islands .fo Micronesia, Federal State of .fm
Fiji .fj Faroe Islands .fo
Finland .fi France .fr
France .fr Gabon .ga
French Guiana .gf United Kingdom .gb
French Polynesia .pf Grenada .gd
French Southern Territories .tf Georgia .ge
Gabon .ga French Guiana .gf
Gambia .gm Guernsey .gg
Georgia .ge Ghana .gh
Germany .de Gibraltar .gi
Ghana .gh Greenland .gl
Gibraltar .gi Gambia .gm
Greece .gr Guinea .gn
Greenland .gl Guadeloupe .gp
Grenada .gd Equatorial Guinea .gq
Guadeloupe .gp Greece .gr
Guam .gu South Georgia and the South Sandwich Islands .gs
Guatemala .gt Guatemala .gt
Guernsey .gg Guam .gu
Guinea .gn Guinea-Bissau .gw
Guinea-Bissau .gw Guyana .gy
Guyana .gy Hong Kong .hk
Haiti .ht Heard and McDonald Islands .hm
Heard and McDonald Islands .hm Honduras .hn
Holy See (Vatican City State) .va Croatia/Hrvatska .hr
Honduras .hn Haiti .ht
Hong Kong .hk Hungary .hu
Hungary .hu Indonesia .id
Iceland .is Ireland .ie
India .in Israel .il
Indonesia .id Isle of Man .im
Iran, Islamic Republic of .ir India .in
Iraq .iq British Indian Ocean Territory .io
Ireland .ie Iraq .iq
Isle of Man .im Iran, Islamic Republic of .ir
Israel .il Iceland .is
Italy .it Italy .it
Jamaica .jm Jersey .je
Japan .jp Jamaica .jm
Jersey .je Jordan .jo
Jordan .jo Japan .jp
Kazakhstan .kz Kenya .ke
Kenya .ke Kyrgyzstan .kg
Kiribati .ki Cambodia .kh
Korea, Democratic People’s Republic .kp Kiribati .ki
Korea, Republic of .kr Comoros .km
Kuwait .kw Saint Kitts and Nevis .kn
Kyrgyzstan .kg Korea, Democratic People’s Republic .kp
Lao People’s Democratic Republic .la Korea, Republic of .kr
Latvia .lv Kuwait .kw
Lebanon .lb Cayman Islands .ky
Lesotho .ls Kazakhstan .kz
Liberia .lr Lao People’s Democratic Republic .la
Libyan Arab Jamahiriya .ly Lebanon .lb
Liechtenstein .li Saint Lucia .lc
Lithuania .lt Liechtenstein .li
Luxembourg .lu Sri Lanka .lk
Macau .mo Liberia .lr
Macedonia, The Former Yugoslav Republic of .mk Lesotho .ls
Madagascar .mg Lithuania .lt
Malawi .mw Luxembourg .lu
Malaysia .my Latvia .lv
Maldives .mv Libyan Arab Jamahiriya .ly
Mali .ml Morocco .ma
Malta .mt Monaco .mc
Marshall Islands .mh Moldova, Republic of .md
Martinique .mq Madagascar .mg
Mauritania .mr Marshall Islands .mh
Mauritius .mu Macedonia, The Former Yugoslav Republic of .mk
Mayotte .yt Mali .ml
Mexico .mx Myanmar .mm
Micronesia, Federal State of .fm Mongolia .mn
Moldova, Republic of .md Macau .mo
Monaco .mc Northern Mariana Islands .mp
Mongolia .mn Martinique .mq
Montserrat .ms Mauritania .mr
Morocco .ma Montserrat .ms
Mozambique .mz Malta .mt
Myanmar .mm Mauritius .mu
Namibia .na Maldives .mv
Nauru .nr Malawi .mw
Nepal .np Mexico .mx
Netherlands .nl Malaysia .my
Netherlands Antilles .an Mozambique .mz
New Caledonia .na Namibia .na
New Zealand .nz New Caledonia .na
Nicaragua .ni Niger .ne
Niger .ne Norfolk Island .nf
Nigeria .ng Nigeria .ng
Niue .nu Nicaragua .ni
Norfolk Island .nf Netherlands .nl
Northern Mariana Islands .mp Norway .no
Norway .no Nepal .np
Oman .om Nauru .nr
Pakistan .pk Niue .nu
Palau .pw New Zealand .nz
Palestinian Territories .ps Oman .om
Panama .pa Panama .pa
Papua New Guinea .pg Peru .pe
Paraguay .py French Polynesia .pf
Peru .pe Papua New Guinea .pg
Philippines .ph Philippines .ph
Pitcairn Island .pn Pakistan .pk
Poland .pl Poland .pl
Portugal .pt Saint Pierre and Miquelon .pm
Puerto Rico .pr Pitcairn Island .pn
Qatar .qa Puerto Rico .pr
Reunion Island .re Palestinian Territories .ps
Romania .ro Portugal .pt
Russian Federation .ru Palau .pw
Rwanda .rw Paraguay .py
Saint Helena .sh Qatar .qa
Saint Kitts and Nevis .kn Reunion Island .re
Saint Lucia .lc Romania .ro
Saint Pierre and Miquelon .pm Russian Federation .ru
Saint Vincent and the Grenadines .vc Rwanda .rw
San Marino .sm Saudi Arabia .sa
Sao Tome and Principe .st Solomon Islands .sb
Saudi Arabia .sa Seychelles .sc
Senegal .sn Sudan .sd
Serbia and Montenegro .cs Sweden .se
Seychelles .sc Singapore .sg
Sierra Leone .sl Saint Helena .sh
Singapore .sg Slovenia .si
Slovak Republic .sk Svalbard and Jan Mayen Islands .sj
Slovenia .si Slovak Republic .sk
Solomon Islands .sb Sierra Leone .sl
Somalia .so San Marino .sm
South Africa .za Senegal .sn
South Georgia and the South Sandwich Islands .gs Somalia .so
Spain .es Suriname .sr
Sri Lanka .lk Sao Tome and Principe .st
Sudan .sd El Salvador .sv
Suriname .sr Syrian Arab Republic .sy
Svalbard and Jan Mayen Islands .sj Swaziland .sz
Swaziland .sz Turks and Caicos Islands .tc
Sweden .se Chad .td
Switzerland .ch French Southern Territories .tf
Syrian Arab Republic .sy Togo .tg
Taiwan .tw Thailand .th
Tajikistan .tj Tajikistan .tj
Tanzania .tz Tokelau .tk
Thailand .th Timor-Leste .tl
Timor-Leste .tl Turkmenistan .tm
Togo .tg Tunisia .tn
Tokelau .tk Tonga .to
Tonga .to East Timor .tp
Trinidad and Tobago .tt Turkey .tr
Tunisia .tn Trinidad and Tobago .tt
Turkey .tr Tuvalu .tv
Turkmenistan .tm Taiwan .tw
Turks and Caicos Islands .tc Tanzania .tz
Tuvalu .tv Ukraine .ua
Uganda .ug Uganda .ug
Ukraine .ua United Kingdom .uk
United Arab Emirates .ae United States Minor Outlying Islands .um
United Kingdom .gb United States .us
United Kingdom .uk Uruguay .uy
United States .us Uzbekistan .uz
United States Minor Outlying Islands .um Holy See (Vatican City State) .va
Uruguay .uy Saint Vincent and the Grenadines .vc
Uzbekistan .uz Venezuela .ve
Vanuatu .vu Virgin Islands, British .vg
Venezuela .ve Virgin Islands, U.S. .vi
Vietnam .vn Vietnam .vn
Virgin Islands, British .vg Vanuatu .vu
Virgin Islands, U.S. .vi Wallis and Futuna Islands .wf
Wallis and Futuna Islands .wf Western Samoa .ws
Western Sahara .eh Yemen .ye
Western Samoa .ws Mayotte .yt
Yemen .ye Yugoslavia .yu
Yugoslavia .yu South Africa .za
Zambia .zm Zambia .zm
Zimbabwe .zw Zimbabwe .zw

 

Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2006 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

Meta Language Codes

Meta Tag ISO Language and Country Codes

This page lists and discusses Language Codes and Attributes. Page two of this series lists and discusses Country Codes and ccTLD issues. Page three discusses gTLD (generic Top Level Domain) issues.

The lang  attribute is used to define the base language to be used for displaying text and characters on a Web site. This allows an internationalization of HTML for a very large number of languages. If the metatag generator does not have the one you want, feel free to use one of these by typing it into the code (see below).The languages are designated by a two letter code, such as “en” for English or “es” for Spanish. One or more hyphenated values can be tacked on to the initial two letter code to specify regional or ethnic variations, such as “en-us” for U.S. English. You can get a list of Country Codes here

Note that html texts that contain foreign languages that use special characters will have to be saved as a Unicode file, rather than as an ANSI file, in order for the characters to be properly displayed. Please be aware that some browsers may not be capable of correctly displaying a Unicode file.

HTML tags are always considered English. Therefore, the DocType declaration at the top of web pages will always say EN, even though the rest of the page is in a different language:

<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.1//EN” “xhtml11.dtd”>
-or
<!DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4.01 Transitional//EN“>

Don’t get carried away and change this. It has to be EN for the DOCTYPE.

Got that? Good. Now, on to the more interesting part of the story…

DocType

This:

<meta http-equiv=”Content-Language” content=”en”>

Defines the encoding of the whole page as English as far as the language set goes – in short, it tells the user agent (browser) to expect English language characters. It’s purely a technical declaration. There can be only one language declared in this tag.

Other Tags

The lang attribute applies to all HTML tags except applet, base, basefront, br, frame, frameset, iframe, param and  script.

I find it most useful in the HTML, DIV and SPAN tags.

This:

HTML:

<html lang=”en”>

-or-

XHTML:

<html xmlns=”http://www.w3.org/1999/xhtml” xml:lang=”en” />

Tells the visitor/search engine what audience the text is intended for.

It’s perfectly possible for the meta http-equiv and html lang tags to be out of sync – for example, I declare the technical language of my Canadian websites to be English, but will have French and English on the same page at times, both with properly declared language tags (the language can be declared in spans, divs and other tags, not just the HTML one.

From a search engine standpoint, the most useful declaration of a language is via the HTML, Div, or Span language declaration, which describes the intended audience, rather than the meta-equiv, which only addresses the software that renders the language (i.e. the browser).

Naturally, both should exist and be proper, but the SE would only look at the language declaration of the content intended for the visitor, not the technical declaration of the document intended for the browser, as far as returning results to a searcher is concerned.

Reference: http://www.w3.org/International/questions/qa-http-and-lang

Multiple Languages

A document can declare more than one language or language group for a visitor, but only one for a user-agent (browser).

This means that the meta http-equiv=”Content-Language” tag can only reference one language – i.e. “en”

The Lang= or xml:lang= tags can reference several languages – i.e. “en, zh, pt, fr” or even local variants: “en-us” (US English).

One method of having more than one language on a page is by using the DIV or SPAN language attributes:

<!DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4.01 Transitional//EN”>
<html lang=”en”>
<DIV Lang=”en”>
Let us drink a carbonated beverage and sit on this chesterfield, ok?
</DIV>
<DIV Lang=”en-us”>
Let’s drink a soda on the sofa, hey?
</DIV>
<DIV Lang=”en-ca”>
Let’s down a pop on the couch, eh?
<DIV Lang=”fr”>
Buvons une boisson carbonatée et reposons-nous sur ce Chesterfield.
</DIV>

This lets you tell the browser what to expect, but lets a search engine or screen reader how to deal with certain blocks of text.

Case

Language and country codes are NOT case sensitive.

 

ISO 639: Two-letter Language Codes

Sorted by Language

Sorted by Language Code

Abkhazian AB AA Afar
Afar AA AB Abkhazian
Afrikaans AF AF Afrikaans
Albanian SQ AM Amharic
Amharic AM AR Arabic
Arabic AR AS Assamese
Armenian HY AY Aymara
Assamese AS AZ Azerbaijani
Aymara AY BA Bashkir
Azerbaijani AZ BE Byelorussian
Bashkir BA BG Bulgarian
Basque EU BH Bihari
Bengali, Bangla BN BI Bislama
Bhutani DZ BN Bengali, Bangla
Bihari BH BO Tibetan
Bislama BI BR Breton
Breton BR CA Catalan
Bulgarian BG CO Corsican
Burmese MY CS Czech
Byelorussian BE CY Welsh
Cambodian KM DA Danish
Catalan CA DE German
Chinese ZH DZ Bhutani
Corsican CO EL Greek
Croatian HR EN English, American
Czech CS EO Esperanto
Danish DA ES Spanish
Dutch NL ET Estonian
English, American EN EU Basque
Esperanto EO FA Persian
Estonian ET FI Finnish
Faeroese FO FJ Fiji
Fiji FJ FO Faeroese
Finnish FI FR French
French FR FY Frisian
Frisian FY GA Irish
Gaelic (Scots Gaelic) GD GD Gaelic (“Scots Gaelic”)
Galician GL GL Galician
Georgian KA GN Guarani
German DE GU Gujarati
Greek EL HA Hausa
Greenlandic KL HI Hindi
Guarani GN HR Croatian
Gujarati GU HU Hungarian
Hausa HA HY Armenian
Hebrew IW IA Interlingua
Hindi HI IE Interlingue
Hungarian HU IK Inupiak
Icelandic IS IN Indonesian
Indonesian IN IS Icelandic
Interlingua IA IT Italian
Interlingue IE IW Hebrew
Inupiak IK JA Japanese
Irish GA JI Yiddish
Italian IT JW Javanese
Japanese JA KA Georgian
Javanese JW KK Kazakh
Kannada KN KL Greenlandic
Kashmiri KS KM Cambodian
Kazakh KK KN Kannada
Kinyarwanda RW KO Korean
Kirghiz KY KS Kashmiri
Kirundi RN KU Kurdish
Korean KO KY Kirghiz
Kurdish KU LA Latin
Laothian LO LN Lingala
Latin LA LO Laothian
Latvian, Lettish LV LT Lithuanian
Lingala LN LV Latvian, Lettish
Lithuanian LT MG Malagasy
Macedonian MK MI Maori
Malagasy MG MK Macedonian
Malay MS ML Malayalam
Malayalam ML MN Mongolian
Maltese MT MO Moldavian
Maori MI MR Marathi
Marathi MR MS Malay
Moldavian MO MT Maltese
Mongolian MN MY Burmese
Nauru NA NA Nauru
Nepali NE NE Nepali
Norwegian NO NL Dutch
Occitan OC NO Norwegian
Oriya OR OC Occitan
Oromo, Afan OM OM Oromo, Afan
Pashto, Pushto PS OR Oriya
Persian FA PA Punjabi
Polish PL PL Polish
Portuguese PT PS Pashto, Pushto
Punjabi PA PT Portuguese
Quechua QU QU Quechua
Rhaeto-Romance RM RM Rhaeto-Romance
Romanian RO RN Kirundi
Russian RU RO Romanian
Samoan SM RU Russian
Sangro SG RW Kinyarwanda
Sanskrit SA SA Sanskrit
Serbian SR SD Sindhi
Serbo-Croatian SH SG Sangro
Sesotho ST SH Serbo-Croatian
Setswana TN SI Singhalese
Shona SN SK Slovak
Sindhi SD SL Slovenian
Singhalese SI SM Samoan
Siswati SS SN Shona
Slovak SK SO Somali
Slovenian SL SQ Albanian
Somali SO SR Serbian
Spanish ES SS Siswati
Sudanese SU ST Sesotho
Swahili SW SU Sudanese
Swedish SV SV Swedish
Tagalog TL SW Swahili
Tajik TG TA Tamil
Tamil TA TE Tegulu
Tatar TT TG Tajik
Tegulu TE TH Thai
Thai TH TI Tigrinya
Tibetan BO TK Turkmen
Tigrinya TI TL Tagalog
Tonga TO TN Setswana
Tsonga TS TO Tonga
Turkish TR TR Turkish
Turkmen TK TS Tsonga
Twi TW TT Tatar
Ukrainian UK TW Twi
Urdu UR UK Ukrainian
Uzbek UZ UR Urdu
Vietnamese VI UZ Uzbek
Volapuk VO VI Vietnamese
Welsh CY VO Volapuk
Wolof WO WO Wolof
Xhosa XH XH Xhosa
Yiddish JI YO Yoruba
Yoruba YO ZH Chinese
Zulu ZU ZU Zulu

 


Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2006 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

Bogon

Ian McAnerin and Mike Churchill
2005

Bogons Ate My Web Site

An unforeseen risk of changing ISPs

The Scenario

We were recently contacted by one of our customers wanting to know why their site was dropping in their search engine rankings. The site’s presence in Yahoo! was slowing eroding – every day a few more indexed pages would disappear. The site had not undergone any content changes, but it had recently changed ISPs. What could be causing the problem? It was only through the teamwork of networking staff at both Yahoo! and RackSpace that the issue was identified, isolated, and corrected. Curiously, neither was to blame, but both were instrumental in solving the problem.

There were a number of unusual aspects to the case, including the fact that only rankings on Yahoo seemed to be affected. More interestingly, Yahoo’s spiders were reaching the site according to the logs, but when we checked with Yahoo they were apparently reporting an error.

So we began a detailed analysis of what was going on. The analysis was made more difficult by the fact that running a traceroute from Yahoo to the website showed that the ping made it into RackSpace’s network before disappearing. RackSpace, doing the same trace in reverse, noted that the ping disappeared once Yahoo’s network was reached, thus prompting a scenario where each initially blamed the other based on the evidence presented.

We had no reason to believe that either party was misleading us, so we began a careful troubleshooting checklist. The site itself was clean and fast, used no risky tactics and was very spiderable. The robots.txt and other robot behavior control codes freely allowed spiders into the site, and there were no errors in the DNS or other related server technologies. Additionally, the site had been spidered by various search engines in the past, and was ranking well in several categories. We knew it wasn’t the site, and began to check the network.

First, we checked the IP address of the website. There was no indication of it being blacklisted for spam purposes.

Next, we checked for the website being blocked by Yahoo or the Yahoo spiders being blocked by RackSpace. Both parties assured us that this was not the case. Yahoo was able to confirm that the spiders were unable to access the site, however.

The only odd factor in the new hosting location was that the IP block was only recently allocated – previously it was part of a reserved block not in use anywhere. It was this oddity that allowed us to identify the culprit: a monster lurking in the internet known as a bogon.

After a few phone calls and some additional tests, we verified that a bogon had indeed eaten our website and Yahoo’s spiders, and could very well be lurking out there on the internet waiting for others, as well. Once we confirmed this to Yahoo they were able to deal with the matter and the client is now enjoying his previous rankings and traffic.

The Bogon

So, you are probably asking, what is a bogon?

Let’s back up a step and outline an issue that networks deal with all the time – spammers and hackers. Naturally these people do not wish to be traced or identified, so they often send fake return IP addresses to make it difficult to track them. It’s like sending a nasty letter to someone and using a fake return address.

There are billions of potential IP addresses in the world, including a lot of IP addresses that are not being used. They are either being held in reserve for future needs, or are allocated for testing or other purposes. It’s a fair assumption that anything coming to you with an address you know is fake is probably up to no good.

Network administrators have discovered that if they block all of these known unused addresses from moving through their networks, they can block upwards of 60% of spam and hackers. This makes blocking known unused addresses a very easy way to make your network run better and make your users happy.

These unused IP addresses are collectively known as a bogon, a contraction of “bogus logon”, or a logon from a place you know no one can actually logon from.

Smart network administrators block bogons at the router or firewall level, never letting them get into the system in the first place. But what happens when a previously unallocated IP address is released into public use?

In a perfect world, all network administrators everywhere would either manually or automatically update their bogon filters to allow the new IP’s to flow through their network. But it’s not a perfect world, and RackSpace reports that a large number of administrators either do not update their bogon filters automatically or are running systems that need to be manually updated and have not done so yet.

This means that if you are given an IP address that was a bogon until recently, you may find your website blocked from various ISP’s and networks.

The issue with our client was not that Yahoo was blocking the IP, but Yahoo’s ISP was. Since the main router for RackSpace was a known safe IP address, what would happen was that the trace from Yahoo would leave Yahoo, go through their ISP, enter RackSpace’s network and then get directed to the new IP, all the while reporting back at each step that it had successfully arrived.

Of course, the report back from the new IP was being blocked by Yahoo’s ISP, which made it look like the trace ping entered RackSpace but never arrived at the new IP, since the return packet was being blocked. This explains why Yahoo’s spiders were showing up in the logs, but were not reporting their visit back to Yahoo – they were visiting and reporting, but the reports were being intercepted by the bogon filter and discarded.

The lucky thing is that in this case there was only one intervening network between Yahoo and RackSpace, allowing us to narrow in quickly. Imagine the potential confusion if there had been several ISP’s. Worse, what if the spiders took a different path to the website sometimes? You would get an intermittent block to the spiders and visitors, which would be very hard to track down unless you knew exactly what you were looking for.

Yahoo’s ISP has since fixed this problem, but there are a great many ISP’s and networks out there that have not. There may be websites that are not accessible to some search engines and visitors, with the website owner not having any clue as to the reason why.

The Fix

So, how do you check to see if you are affected by a bogon filter? First, check your IP address. If it’s a recently allocated (within the last year) IP then there is a possibility that it may be affected by bogon filters put in place by various networks. Another symptom is that a traceroute will show a response at each step of the way until the new IP is reached, and it is consistently blocked by the same network node.

At this time, we recommend that you avoid newly released IP addresses unless you are certain that they are not being affected by bogon filters.

If you are a network administrator who uses bogon filters, we strongly recommend you use one of the many freely available automatic bogon updating scripts and services.

Bogon filters can be a powerful ally in fighting hackers and spammers, but it’s important to realize that a website or mail server can be inadvertently blocked by out-of-date bogon lists as a false positive. Without the help and support of both Yahoo and RackSpace, this would have been a very difficult issue to troubleshoot.

Useful Links:

http://www.cymru.com/Bogons/
http://www.completewhois.com/bogons/
http://www.antionline.com/jargon/bogon.php

 


© Ian McAnerin and Mike Churchill 2005

Ian McAnerin, founder of McAnerin Networks Inc, is a moderator for the High Rankings and the Search Engine Watch forums, and has been vetted by the standards watchdog SEOConsultants.com and SEOPros directories. He is the president of the Search Marketing Association of North America, with published articles in many international newsletters and web site news reports. He has a special interest in SEO legal issues due to his legal background.

Mike Churchill is the Tech Guru at KeyRelevance.com. He has been an Internet Junkie since 1992, cutting his Internet teeth on FTP, Archie, and Mosaic 1.0. He is the former Chief Technical Officer and co-founder of NetMechanic.com and is an avid developer of Internet applications.


First Published by Mike Grehan. Search engine marketing consultant, speaker and author. http://www.search-engine-book.co.uk

Associate Editor: Christine Churchill. KeyRelevance.com

e-marketing-news is published selectively on a when it’s ready basis. ©2005 Net Writer Publishing.

< http://www.e-marketing-news.co.uk >
Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2004 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

301 Redirect IIS

IIS Redirects – 301 , 302

Microsoft’s Internet Information Server ( MS IIS ) is very different from Apache, and you need to handle redirects on it differently.

Although you can use the popular FrontPage web development software with Apache + FP extensions, most people who use FrontPage prefer to use IIS. Also, Microsoft’s .NET and ASP development platforms usually work best on MS IIS (no surprise there).

Definitions for Terms used in this Article

Detailed Information

Overview

What you are trying to accomplish here is to have one resource (either a page or an entire site) redirect a visitor to a completely different page or site, and while doing so tell the visitor’s browser that the redirect is either permanent (301) or temporary (302).

Therefore you need to do three things:

  1. Have 2 resources – one source page or website, and one destination page or website.
  2. When an attempt to access the source resource is made, IIS transfers the visitor to the destination instead.
  3. During the transfer, IIS reports to the visitor that a redirect is happening and it’s either temporary or permanent.

The good news is that IIS has always supported this, and you can use it’s control panel to make the redirect.

Administrator Mode – Redirecting a Domain

If you can log into the Windows 2000 (or higher) server and access the desktop, then choose:

Start > Programs > Administrative Tools > Internet Services Manager

Now choose the server running the site you want to forward. Remember you need 2 sites – one to forward FROM and one to forward TO. These can be on the same, or separate servers.

Right click on the site you want to redirect FROM and choose Properties > Home Directory

You will see the following:


The default is the first choice, “A directory located on this computer”. Change it to “A redirection to a URL” and type in the new URL.

If you want it to be a 301, then be sure to check ON the choice for “A permanent redirection for this resource”. If you want it to be a 302, leave the choice checked OFF.

Administrator Mode – Redirecting an Individual Page

If you can log into the Windows 2000 (or higher) server and access the desktop, then choose:

Start > Programs > Administrative Tools > Internet Services Manager

Now choose the server running the site you want to forward. Choose the site with the webpage you want to forward in it, then right click on the it and choose “Properties”.

You will see the following:


The default is the first choice, “The designated file”. Change it to “A redirection to a URL” and type in the new URL.

If you want it to be a 301, then be sure to check ON the choice for “A permanent redirection for this resource”. If you want it to be a 302, leave the choice checked OFF.

If you don’t control the IIS server, ask the admin to do the above. Done.

Passing on Variables or a Query String During IIS Redirects

Let’s say that you want to pass on some variables, for example, you wanted to redirect an ASP site that accepted arguments for some pages and pass those same arguments on to the same pages at the new site.

In this case, in the “Redirect to:” box, enter the domain you wish to move to (no trailing slash), plus $S$Q .

For example:

http://www.newdomain.com$S$Q

Next, check the options that state the client will be sent to “The exact URL entered above”, as well as “A permanent redirection for this resource” (if you want it to be a 301). Done.

What does this $S$Q do? These are tags that IIS will automatically replace – $S will be replaced with the subdirectory location (such as /shopping/cart.aspx) and $Q will be replaced with the querystring (such as ?id=Blue).

Server Variable Function Example
$P Passes parameters that were passed to the URL to the new URL. If the request contains parameters such as http://www.oldsite.com/cart.asp?id=Blue , then $P would represent all the values after the question mark in the URL, example $P would equal id=Blue (no question mark).
$Q Passes the parameters including the question mark. This is the same as $P but includes the question mark or query string. So $P would equal ?id=Blue
$S Passes the matching suffix of the URL to the new URL. If the request is for http://www.oldsite.com/shopping/cart.asp, then $S represents /cart.asp. If the request was for http://www.oldsite.com/shopping then the value of $S would be /shopping
$V Removes the server name from the original request. If the request is for http://www.oldsite.com/shopping/cart.asp then $V would contain everything after the server name, eg: /shopping/cart.asp.
* Wildcard symbol used for replacement. Let’s say you want to redirect all requests for html pages to a single asp page – you could do so in the following way: *;*.htm;page.asp

This works for both Site Redirects and Individual Page Redirects.

Common Scenarios

I Just Want To Switch Domains!

This is actually pretty straightforward. You can tweak things to get exactly what you need with the variables above, but the following will work for the most common setups:

http://www.newdomain.com$V$Q

Do NOT add a trailing slash “/” at the end of the domain name!

Make sure you check:

  • The exact URL entered above
  • A permanent redirection for this resource

Done! This will send everything after the old domain name to the new domain name, including variables. You need to do is make sure that the new website is set up exactly the same as the old one and that all you are doing is changing domain names.

Non-WWW to WWW Redirect for IIS

When you set up a site in IIS, usually you set up the site with a domain name and add the domain name with both versions, www and non-www, to the host headers for the account. This creates a potential duplication issue, though.

The better way to do this is to actually create 2 accounts – one with the www version and one without. Then you put the website in your preferred version and a single page in the other. Normally, you would then treat this as if you were switching domains (above), with the 2 domains being the www and the non-www versions.

Another method, for those that don’t have direct access to the control panel (common with many ISP’s/ hosts), is to put your entire website in the www account and then a single page in the non-www account. Then you would put the following code in the default page in the account with the single page (i.e. the one you are redirecting):

< %@ Language=VBScript %>
< %
Response.Status=”301 Moved Permanently”
Response.AddHeader “Location”, http://www.example.com
%>

What about Parameters?

If you do a non-www to www redirect for MS Internet Information Services (IIS) and have parameters that need to be passed on, then do the following (Thanks to Chris Hirst for testing this):

  1. Do the above non-WWW to WWW redirect
  2. Make double sure that you do NOT have a trailing slash at the end of the domain (http://www.newdomain.com$V$Q, NOT http://www.newdomain.com/$V$Q)
  3. As with all these redirects, make sure that you check “The exact URL entered above”

Don’t Have Administrator or Desktop Access?

If possible, ask the Administrator for the site to perform the above steps. If you can’t, you need to use ASP or .NET scripting on-page to do redirects.

Conclusion

IIS is a powerful and easy to use web hosting platform. If you have access to the Admin panel you can accomplish basic tasks (like simple redirection) easily and quickly. If you don’t have access (and don’t want to switch hosts) then you will have to use redirection scripting to accomplish this. You’ll learn more about that in the Redirects Using On-Page Scripting and Headers section.

Definitions for Terms used in this Article

Next: Redirects Using On-Page Scripting and Headers


Main Article

Detailed Technical Information

Specific Scenarios and How To Deal With Them

Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2004 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

301 Redirect Apache

Apache Redirects for SEOs

Definitions and Glossary

Overview

What you are trying to accomplish here is to have one resource (either a page or an entire site) redirect a visitor to a completely different page or site, and while doing so tell the visitor’s browser that the redirect is either permanent (301) or temporary (302).

Therefore you need to do three things:

  1. Have 2 resources – one source page or website, and one destination page or website.
  2. When an attempt to  access the source resource is made, the webserver transfers the visitor to the destination instead.
  3. During the transfer, the webserver reports to the visitor that a redirect is happening and it’s either temporary or permanent.

The ability to control the “status” argument in the redirect directive (which sets whether it’s a 301 or 302) within Apache is only available in version 1.2 and above. You are best off using version 2 or above for maximum stability, security and usefulness.

Mod_Rewrite and the Apache Redirect

If you have the mod_rewrite extension installed (it comes with most Apache installs as a default) you can use it to dynamically change URL’s using arguments on the fly – this is NOT a 301 redirect, but rather it’s related behavior. For example, if you wanted to redirect .htmfiles from an old server to their equivalent .php files on a new one using a 301 redirect, you would use a combination of mod_rewrite and the redirect directive to do the redirection + URL change.

You could do it on a file by file basis by making a really long list of possible redirects in the .htaccess file by hand without mod_rewrite, but that would be a real pain on a server with a lot of files, or a completely dynamic system. Therefore these 2 functions are often used together.

Syntax for a 301 Redirect

The syntax for the redirect directive is:

Redirect /yourdirectory http://www.newdomain.com/newdirectory

If the client requests http://myserver/service/foo.txt, it will be told to access http://www.yourdomain.com/service/foo.txt instead.

Note: Redirect directives take precedence over Alias and ScriptAlias directives, irrespective of their ordering in the configuration file. Also, URL-path must be a fully qualified URL, not a relative path, even when used with .htaccess files or inside of <Directory> sections.

If you use the redirect without the status argument, it will return a status code of 302 by default. This default behaviour has given me problems over the years as an SEO, so it’s important to remember to use it, like this:

Redirect permanent /one http://www.newdomain.com/two

or

Redirect 301 /two http://www.newdomain.com/other

Both of which will return the 301 status code. If you wanted to return a 302 you could either not specify anything, or use “302” or “temp” as the status argument above.

You can also use 2 other directives – RedirectPermanent URL-path URL (returns a 301 and works the same as Redirect permanent /URL PathURL) and RedirectTemp URL-path URL (same, but for a 302 status).

For more global changes, you would use redirectMatch, with the same syntax:

RedirectMatch 301 ^(.*)$ http://www.newdomain.com

or

RedirectMatch permanent ^(.*)$ http://www.newdomain.com

These arguments will match any file requested at the old account, change the domain, and redirect it to the file of the same name at the new account.

You would use these directives in either the .htaccess file or the httpd file. It’s most common to do it in the .htaccess file because it’s the easiest and doesn’t require a restart, but the httpd method has less overhead and works fine, as well.

Simple Domain 301 Redirect Checklist

This assumes you just have a new domain (with no working pages under it) and want it to redirect properly to your main domain.

1. Ensure that you have 2 accounts – the old site and the new site (they do not have to be on different IP’s or different machines).

2. Your main (proper or canonical) site should be pointed at the new site using DNS. All your other domains should be pointed at the old site using DNS. Parking them there is fine at this point.

3. Find the .htaccess file at the root of your old account. Yes, it starts with a “.” We will be working with this file. The new site does not need any changes made to it – the old site does all the redirection work.

4. Download the .htaccess file and open it in a text only editor.

5a. Add this code:

Redirect 301 / http://www.newdomain.com/

6. Then upload the file to your root folder and test your new redirect. Make you you also check it using a HTTP Header viewer just to be sure it shows as a 301.

Control Panel Method

cPanel redirect

  • Log into your cPanel, and look for “Redirects” under Site Management
  • Put in the current directory into the first box
  • Put the new directory in the second box
  • Choose the type (temporary or permanent) temporary=302 and permanent=301
  • Click “Add” and you’re done

You can only do 302 redirects (or frame forwarding – bad!) using the Plesk control panel – use .htaccess for 301’s instead.

If you use Ensim, the only way to redirect is by using the .htaccess file (no control panel option at this time).

Basic Old Website to New Website Redirection

This is used when you have an existing website (with pages) and want to move it to a new domain, while keeping all your page names and the links to them.

1. Ensure that you have 2 websites – the old site and the new site, and that they are on different accounts (they do not have to be on different IP’s or different machines).

2. Your main (proper or canonical) site should be pointed at the new site using DNS. All your old domains should be pointed at the old site using DNS.

3. Find the .htaccess file at the root of your old account. Yes, it starts with a “.”  We will be working with this file. The new site does not need any changes made to it – the old site does all the redirection work.

4. Download the .htaccess file and open it in a text only editor.

5a. If you have mod_rewrite installed, add this code:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} !^newdomain\.com
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=301,L]

5b. If you don’t have mod_rewrite installed, you really should. If you can’t install it, then you can use this code instead:

RedirectMatch 301 ^(.*)$ http://www.newdomain.com

6. Then upload the file to your root folder and test your new redirect. Make you you also check it using a HTTP Header viewer just to be sure it shows as a 301.


FrontPage on Apache

After you’ve done the basic Apache 301 redirection described in this article, you will also need to change the .htaccess files in:  

_vti_bin
_vti_bin /_vti_adm
_vti_bin/ _vti_aut

Replace “Options None” to “Options +FollowSymLinks”

Those folders are part of your FrontPage extensions on the server, so you will have to use FTP to get to them, since FrontPage hides these folders by default to prevent them from accidentally being messed with by novice users.

More Complicated Redirects

You can’t use a control panel in Apache currently for these – .htaccess only.

Redirecting everything to a single page

This is common when you are totally changing the new website from the old and you just want all your links and requests form the old site to be directed to a spot on your new site (usually the home page). You actually need to do it on a page by page basis.

Redirect 301 /oldfile1.htm http://www.newdomain.com
Redirect 301 /oldfile2.htm http://www.newdomain.com
Redirect 301 /oldfile3.htm http://www.newdomain.com

Redirection while changing the filename

This example will redirect all the files on the old account that end in html to the same file on the new account, but with a php extension. You can also use this technique within the same account if you want to change all your extensions but don’t want to lose your incoming links to the old pages. This is common when people switch to from static htm files to dynamic ones while keeping the same domain name, for example.

Just change the “html” and “php” parts of the below example to your specific situation, if needed.

RedirectMatch 301 (.*)\.html$ http://www.newdomain.com$1.php

Redirection while changing the filename, but keeping the GET arguments

Sometimes, you will want to change to a different CMS, but keep your database the same, or you want to switch everything but you like the arguments and don’t want to change them.

RedirectMatch 301 /oldcart.php(.*) http://www.newdomain.com/newcart.php$1

This will result in “http://www.olddomain.com/oldcart.php?Cat_ID=Blue” being redirected to “http://www.newdomain.com/newcart.php?Cat_ID=Blue”

URL Forwarding

Many ISP’s and domain registrars offer a service called URL Forwarding, or something similar. This term has no proper meaning and can mean pretty much anything that the registrar’s marketing team or tech’s decide it should mean or do. Most commonly, it’s a 302 redirect. Sometimes it’s a CNAME, park or even a frame. Always check to see what type of “forwarding” is actually done.

If they don’t know, consider switching to a company with more technical skill, or asking to see a demonstration URL and then checking it with an Http Header viewer.

CNAME Record

There is one other type of redirect, and that is by using the CNAME function, which is actually an alias for one domain to another. A common example is when someone uses CNAME to point www.mcanerin.com to just plain mcanerin.com. This apparently works just fine, and it technically tells the visitor that mcanerin.com is the main (canonical) domain and that www version is an alias.

In Apache, the header response for this is a 301. So yes, a CNAME can be used as a “poor man’s” 301 redirect. There are 2 serious problems, though. First, you can’t CNAME the “origin point” – the actual domain name itself – only sub domains, like www, ftp, mail, etc.

The second problem is that because it’s really easy to mess up CNAMES by accidentally setting up an infinite loop, they are not really recommended unless you know exactly what you are doing. For example, using a CNAME record will often break your email unless you are really careful and know exactly what you are doing. In general, avoid using CNAME.

The only time it’s really useful is when you want to alias a sub domain under the current domain to an external domain.

Conclusion

This type of redirection works great if you are using Apache and have access to your .htaccess file. If you don’t (and don’t want to switch hosts) then you will have to use redirection scripting to accomplish this. You’ll learn more about that in the Redirects Using On-Page Scripting and Headers section.

Next: IIS Redirects – 301, 302

Definitions and Glossary


Main Article

Detailed Technical Information

Specific Scenarios and How To Deal With Them

Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2007 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

301 Redirect Parking

Domain Parking for SEOs

Definition

Domain Parking
A term used to describe for temporarily placing a record in a nameserver (DNS) for later use.

This is normally done because you usually can’t register a domain unless you provide a nameserver address, but many people buy domains before there is a website ready. Usually the domain is pointed at a temporary web page until the “real” site is ready.

Domain Parking Technical Details

The act of parking is physically the same as assigning a domain name to an IP address. Technically, you simply add an “A Record” to a DNS server and point it somewhere. Physically, it’s exactly the same coding as a normal domain – what makes it a “park” is the contents of the destination – which is not a separate and unique site intended for the domain, but either a temporary holding page, an unconfigured server, or some other website altogether.

It will often look something like this:

www.mcanerin.com.     IN     A     192.168.5.100

  • The “www.mcanerin.com” is the domain name.
  • The “IN” is the Address Class, and in this case stands for “INternet”.
  • The “A” tells the DNS that this is an Address (as opposed to a mail server, alias, or whatever)
  • The “192.168.5.100” is the IP (Internet Protocol) of the server.

Altogether, this line tells anyone who asks that the domain name www.mcanerin.com is an internet address that you can find at the IP address of 192.168.5.100.

Now, what would happen if I made another line in this DNS so it looked like:

www.mcanerin.com    IN     A     192.168.5.100

mcanerin.com        IN     A     192.168.5.100

This would now point BOTH of these names at the same address. Otherwise, if someone typed in “mcanerin.com” without the “www” the DNS server would not know what to do and the user would get a dead page. Most web hosts automatically add both versions to your DNS because this is a common problem.

But a search engine would consider this to be 2 separate websites, at least at first. Why? Because although it’s common for people to be lazy and not type in the “www”, you can actually set things up so that these point to 2 separate servers. In the old days, servers were not very powerful, so it was common to have separate servers do separate jobs. You could have a firewall called mcanerin.com, a webserver called www.mcanerin.com, an ftp server called ftp.mcanerin.com, a mail server called mail.mcanerin.com, and so on. Each of these would be separate servers with their own IP addresses.

So a search engine (unlike many of today’s casual internet users) can’t just assume that mcanerin.com and www.mcanerin.com are the same site – not everyone has a powerful server. Many companies are running legacy systems set up back in the old days, and even today some people in third world countries are getting by with servers that are old and slow.

At first, the search engine will treat the two domains as two separate sites, but will eventually (once it’s fully indexed both for some time) figure out that they are the same, and then merge the data for the two, which can take up to a year. In the meantime, any links that you have going to mcanerin.com would not be credited to www.mcanerin.com. For a while when I first launched my website, mcanerin.com had a PR of 2 and 9 backlinks on Google, but www.mcanerin.com had a PR of 4 with 46 backlinks showing. Eventually (it took a long time), the data merged and I ended up with a PR of 5 with 55 backlinks showing. This obviously has SEO issues, since usually you want to get credit for all your backlinks as soon as possible.

So eventually (once they figure it out) the search engine is treating both addresses as belonging to the same website. This is domain parking in a nutshell from an SEO perspective. You are pointing 2 (or more) domains at the same website.

Now, lets add another domain to the mix:

www.mcanerin.com    IN     A     192.168.5.100

mcanerin.com        IN     A     192.168.5.100

mcanerin.ca         IN     A     192.168.5.100

You will notice that this is the Canadian ccTLD (country code Top Level Domain). If we add this to the mix you will find that the search engines will now also associate the .ca domain to the website, and therefore treat it as a Canadian website, which is good because the Canadian versions of the major search engines give preference (and a ranking bonus) to Canadian sites. The same applies to other areas, such as the UK, China, Australia and so forth.

In general, unless you are attempting to attach a ccTLD to an existing .com/net/org site for the purposes of geolocation, I do notrecommend parking for SEO purposes, for 2 reasons:

  1. The effect of your links (and PageRank) being split between the two sites
  2. The fact that the second site will be considered a duplicate, and possibly trip an anti-spam filter if you have enough parks

The only time I recommend parking is for geolocation purposes. The above two reasons eventually disappear when the search engine figures things out, so it’s a temporary problem, usually. Yahoo is very well known for not liking multiple domains, though, so I like to be very careful with parks.

How to  Park a Domain

Telling your host to do a park is easy – just tell them to point the domain at your IP address. If you use a shared IP, then they will also need to tell the webserver to respond to the domain, as well – but if they use shared IP’s, they will probably know that. Using a typical domain control panel, you simply go to the “Add new Domain” menu (or something saying basically the same thing) type in your new domain, and then make sure it’s pointing at your current websites IP address.

You’ll know it’s working when you type in the new domain and it goes to your current website, but the new domain stays in the address bar.

Checklist

  1. Register your new domain (or acquire an existing one)
  2. During the registration process, you will be asked for at least 2 nameservers (DNS) – either type in the ones that are being used by your current domain or ones under your control.
  3. Now that the domain is pointing at the nameservers, you need to point the nameservers in turn to your target webserver. you do this by going to the nameserver controls, which are not usually near the domain name registration area – it’s a totally different area, and usually controlled by your web host or ISP. Tell them what your domain name is (make sure you spell it correctly for them!) and have them point it to your website’s IP address, or do it yourself if you know how.
  4. If you have a dedicated IP address, skip to step 6.
  5. If it’s a shared IP, then go to your webserver and set your website to respond to the new parked domain by adding the domain to the Host Headers for the site.
  6. Once enough time has gone by for the DNS servers to be updated (anywhere from a few minutes to 3 days, depending on where you live and your ISP), test the park by typing in the parked domain. You should go to the website, and the parked domain should stay in your browsers address bar.
  7. Done!

Absolute and Relative Link Issues with Domain Parking

One clear indication of a park is that the domain name does not change in the address bar to the second site. It stays with whatever you typed in. Now, what happens if you click on a link in the website?

There are 2 possible ways to link within websites – relative or absolute. A relative link just links to the other page in relation to the current page. A relative link to this page, in code, looks like this:

<a href=”301-redirect-parking.htm”>

Look closely at this – do you see www.mcanerin.com (this website) showing up anywhere in this code? No.This is just a link to a page, and it’s only in relation to the other pages in this directory. This is what we mean by a relative link. In general, good website design uses relative links. That way, if you change a domain name you don’t have to go back and change every single link in the website.

If you park a domain on a site with relative links, the domain name in the address bar will remain whatever you typed in and only the path and page names will change.

Now look at what this link would look like as an absolute link:

<a href=”www.mcanerin.com/articles/301-redirect-parking.htm”>

You see that the entire web address is spelled out.

Now, what would happen if you went to a page using a parked domain and then clicked on an absolute link? Why, the link in the address bar would change to the new address, of course! Just as if it were a new webpage on a new site.

Therefore, in practice, if you use relative links your parked domain stays with the visitor for the entire session, but if you use absolute links the domain changes to the hard-coded one as soon as the user clicks on any link. This behavior is very important to understand, since it can affect your branding and geolocation.

Conclusion

I hope you have a better understanding of domain parking now. It’s actually a pretty easy concept, but since many people use the term interchangeably with other terms, there is a lot of confusion out there. Don’t focus on what your ISP is calling it, find out what it’s doing and how it works, and you will be fine. If you check a website using an Http Viewer, it will go straight to the website in one hop (no redirects) and report a 200 (OK) as a result, and the domain name in the browser address bar will not change.

Next: Apache Domain Redirects – 301 vs. 302


Main Article

Detailed Technical Information

Specific Scenarios and How To Deal With Them

Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2004 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

301 Redirect for SEO

301, Parking and Other Redirects for SEOs ( FAQ )

Ian McAnerin, Oct 2005

Parking VS Redirect

Let’s talk about redirects and parking. You’d think this would be a fairly straightforward topic, but it can be a very complicated issue, and is certainly responsible for a lot of questions I field every week. Hopefully this will clear a few things up.

One of the major issues is that SEOs tend to refer to “parking” in a different context than domain registrars, causing a lot of confusion.

Worse, a “redirect”, even among SEOs, can refer to a spam tactic, a helpful forwarding click to another domain, a sever level redirect or even other scenarios. Google specifically forbids “sneaky redirects” and yet encourages the use of 301 redirection for dealing with multiple domain names, for example.

No wonder everyone is confused – even the experts use the terms wrong! Hopefully I can help straighten things out. This series of articles was originally intended only for my clients, but due to popular demand I’m making it available to the public. I’ll start with a simple analogy and go into more details in the other sections.

Finding Fred

Imagine that there are 2 houses on a block. You want to talk to Fred. Lets say that Fred actually owns both of these houses, but you are not certain which house Fred is actually in.

Lets say that Fred is in the second house, and you go to the first one. What happens?

Scenario 1: You go to the first house, ring the doorbell, and ask for Fred. Although Fred is not there at that time, when you ask for him, he gets a page, walks over and meets you at the door. As far as you know, Fred lives in this house, since you knocked on the door, and Fred answered it. Fred would also answer if you went to the second house.

This is parking. In technical terms, you have a single IP address and both domains point to it, and the webserver will serve up the same website when asked for either domain. It’s the easiest way to do it, but now a visitor (including a search engine) has no idea what domain is your “real” one, and can therefore think you you have 2 different identical sites. One way to tell if a site is parked is that when you type in the domain, that domain stays in the address bar. It does not change to the new domain.

Scenario 2: You go to the first house, ring the doorbell, and ask for Fred. Since Fred is not there, the person who answers the door confirms to you that this is Freds place, but that he is next door right now. You now know that although you can find out where Fred is by asking at this house, you also know that Fred is actually next door. You then have to physically walk next door to talk to Fred. He doesn’t come to you.

This is a 301 redirect. In technical terms, you usually have 2 different IP addresses (or at least 2 different accounts) and you put your main address on the first one, and all the other addresses on the other second one. You then tell the webserver to redirect any requests for the secondary sites to the first one. What this tells a search engine is that the secondary sites are no longer active and the proper site is the main one. When you type in a secondary domain, the address will change when you arrive at the final site.

URL Cloaking

Some domain hosts offer a service called “URL cloaking” or some name similar to it. This allows you to “park” your domain name at another website (usually one with a long, ugly URL) but still show your preferred domain name in the address bar. Sounds good, right? Wrong.

How this is accomplished is they create a one page frame and point your domain at it. Then they load the other website with the ugly (or otherwise unwanted) URL into the frame, thus creating the illusion that you are using your preferred domain name and hiding the other one. Frames are not very search engine friendly. Not only is the probable result being your blank frames page being the only thing the search engine will see, but your titles don’t change (they remain the main frame page) which is not very helpful when trying to get your site to rank. Bottom line, “url cloaking” is not search engine friendly, and isn’t really even a redirect – you are just loading pages from one site into another and making people think it’s all one site. Not recommended if you are interested in showing up well on searches.

The Bottom Line and the Exception to the Rule

Yahoo seems to plain dislike the concept of multiple domain names. Always routinely check your domain redirection and how it is being handled by Yahoo, just to be certain.

Under normal circumstances, as an SEO you want a 301 redirect. It tells the search engine that all of the PR and links going to the secondary domains should be redirected to the main site. Otherwise, there will be a time frame where the search engine may think there are actually 2 sites and will only assign PR and link weight to the domains they are attached to, rather than the final destination site.

So, normally you want to do a 301 redirect as an SEO, since it tells the search engine that all the other URLs have been moved permanently to the new location.

However, if you do that with a ccTLD (country code top level domain) like .au, .ca, or .co.uk what will happen is that you will be telling the search engine that the old domain (including the ccTLD) is no longer valid and that the new one is the proper one to use. If the new one is not a ccTLD, then you will lose your localization. Basically, the 301 tells the search engine to throw away the old domains and use the new one instead, and to assign any links to the old domains to the new domain instead.

But if you park or 302 the ccTLD, then the search engine will be told that BOTH domains are valid and therefore assign the appropriate localization (geolocation) to the site.

Got it? Good. Now, there is one last thing to keep aware of. Search engine spiders don’t execute scripts. So if your redirect is script based (ie a javascript redirect), then the search engine will not execute it and will therefore not arrive at the new page.

To confuse the issue, the meta-tag “refresh” is followed by some search engines (it’s not a script, it’s a tag), and can be used, as well. The problem is that the search engines are notorious for not handling these very well. Sometimes they work, and sometimes they screw your rankings up horribly. Therefore, although they do sometimes work, it’s not recommended. Basically, don’t trust any redirection or forwarding that is accomplished by using a code or tag on a web page – only trust it at the server level, unless you have no other options.

Next: Domain Parking and SEO


Main Article

Detailed Technical Information

Specific Scenarios and How To Deal With Them

Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2004 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.

301 Redirect 404 Error

404 Error Pages and Redirects for SEOs

Introduction

A 404 error means “not found”. This is usually the page you get when you make a mistake spelling page name in a site, or if the page is deleted or moved. The problem is that the standard 404 page is ugly and unhelpful.

Many people have figured out that if you use a custom 404 page you can present a much more helpful page to you visitors. Others have taken it a step further and made that custom page a redirect to the home page, so that any links (and PR) pointing to pages that have been deleted (or misspelled) will be passed on to the website.

Sounds great, right? Well, there is a problem (there is almost always a problem with things that sound too good to be true…). The problem is that if you use a redirect to pass PR from an error page to a normal page, the redirecting page will usually return a “200 OK” or 302 Redirect code, rather than a proper 404. This messes up search engines and can result in a whole bunch of indexed URL’s all looking to the search engine like duplicates of your home page (there is no redirect code, it’s a pure 200 OK).

This usually isn’t necessary, but can be useful if you are trying to remove all traces of a page you no longer want associated with your site (i.e. one you were sued over, for example). It says that the page is missing on purpose, and is not an accident or temporary problem.

In this case a URL removal Request to Google followed by a 410 on the page location itself should do it. You can also use robots.txt and robots metatag as backup.

This is bad for your site. Additionally, there are a LOT of indexed “error” pages in search engines (especially Yahoo) that should not be there.

The proper behaviour for an error page is to return a 404 error code. The best result for your visitors is an error page that is either helpful by itself or redirects to a helpful page. The best result from an SEO viewpoint is for any link popularity for broken links be passed on to the page of your choice.

Naturally, the best result overall would be something that accomplishes all of the above. Unfortunately, this is not directly possible. As soon as the search engine is sent the error code, it treats it as a dead page and will eventually remove it.

PR and link weight are only passed on if a page is not a 404. But your site logs will not report errors if it responds as a 200, and your site will not verify (for example, if you use Google Sitemaps) if you don’t have a valid 404 page.

There are 4 possible scenarios with custom pages:

  • 404 – Responds with an error, but shows a custom page to help your visitors
  • 200 – If a page is missing, it’s replaced with the custom error page
  • 302 – If the page is missing, it’s replaced with a temporary redirect to a custom error page
  • 301 – Redirects errors to either a custom error page, or some other page in the site (i.e. sitemap, homepage or best guess)

Each has benefits and drawbacks. You have to choose – “Red or Blue”:

Custom Error Page Types

404 Not Found Response 200 OK (or 302, or 301) Response
Properly Defines the result – a missing page. Tricks the search engine into thinking all is well.
Validates. Does not validate, but won’t break your site.
Shows up in logs so you can fix it. Does not show up as an error – harder to find.
Does not pass on PR or link weight. Passes on PR to final page.
No duplication issues. Can result in a duplication penalty.

Custom Error Page Link Issue

One thing I’d like to make sure everyone is aware of – a custom error page can be called anywhere in your site. This means that if you put any links on that page to help the visitor find their way, you cannot make them relative – since you don’t know where they are relative to.

You must make them either absolute (recommended) or set the base HREF using this code in the header of the page and make sure all your links are relative to it:

<base href=”http://www.yoursite.com”>

Nifty Misuse of the Error Page

Sometimes you won’t have access to the .htaccess of a site, but do have access to a custom error page. Let’s say you have a dynamic site on this site but due to security issues (i.e. PHP “safe” mode) you can’t write pages dynamically to disk, and therefore, without .htaccess to do it on the fly or php permissions to write static pages, you can’t have a CMS with “SEO friendly” URLS. Or can you?

Normally, I’d suggest switching hosts in this case. Really. But let’s say you want to stay with them.

You can write a script on your custom error page to parse what the requested URL is: ie yourdomain.com/content/blue.htm into a database query that is actually yourdomain.com?content=blue and then put the results of that query into the error page thus “faking” .htaccess.

In reality, you are using the .htaccess, but just not in the way it was intended. Naturally, this technique is not standard and your mileage may vary depending on the server setup. It also results in a 200 OK. Make sure that you program in error capturing so if someone legitimately types in the wrong URL that it results in a 404.

Naturally, this also works with IIS and an ASP error page, as well.

Server Issues

Apache and IIS handle custom error pages differently. Usually, I’ve noticed that the custom error pages on IIS are more likely to be wrong than the ones on Apache, but they can both have issues.

First things first – you need to issue a 404 error code at the server level in order for it to work consistently. Attempting to write it in at the page level will not work. If the page is dynamic and the error is written at the server level before the page is served, that will usually work. Once you are at the ISAPI level, it’s too late to send an error code.

Important!

Normally, it’s a good idea to define pages in a server as absolute URLS or files. Not for custom error pages. The path MUST BE RELATIVE TO ROOT or it will return a 200 OK. This applies to all servers I’m aware of, including both Apache and IIS.

Of course, if you are trying to get a 200 OK status in an attempt to pass on PR, then you would use the full URL, not the relative one.

Apache Custom 404

This one is easy. Just go to your.htaccess file (or control panel) and type in the following:

ErrorDocument 404 /404.php

Change the name “404.php” to whatever the name for your custom error page is.

You might be tempted to type in:

ErrorDocument 404 http://www.mysite.com/404.php     *WRONG! Results in 200 OK*

But it won’t work. Usually it will result in a 200 OK response. Once again, the path must be relative to root or it won’t respond with a 404 error code properly.

IIS Custom 404

Dynamic Error Page

If you are running IIS and you are using .asp or aspx custom error pages, then you can put:

Response.status = “404 Not Found”

In code, this usually looks like:

<%
dim pageRequested
with request
pageRequested = _
mid(.queryString, instr(.queryString,”;”) + 1)
end with
response.status = “404 Not Found”
%>

Put this at the very top of the page. Then, create the rest of the page to do and say what you want it to.

Static Error Page

There is nothing special you need to do to static error pages, just make sure you connect to them using the full file name (i.e. c:/www/404.htm).

Setting the Custom Error Page in IIS

Go to IIS Administration and choose the web that you want to set the custom error page for (each web may have it’s own). Right click and go to the “Custom Errors” tab.

Custom 404 settings in IIS

For dynamic error pages, make sure that you are pointing to the custom page using the URL not the File choice. If you use File, it will not pre-process the page, and will simply treat it as static. Remember to use the RELATIVE PATH FROM HOME or it will return a 200 OK instead of the 404 Not Found.

If you are using a static page (i.e. .htm) then you use the File choice to connect to it, using the physical drive location (i.e. c:/www/404.htm)

You can then test it using a Header Viewer: If it comes back in the Header section as:

HTTP/1.1·200·OK(CR)(LF)

Then it’s not working, but if it comes back and shows:

HTTP/1.1·404·Object·Not·Found(CR)(LF)

Then it is.

The Metarefresh Problem

In some cases, people will create a custom error page that displays the error, then “helpfully” uses a metarefresh to forward the visitor to the site map, home page or some other page.

The problem with this is that each search engine treats metarefreshes differently. Yahoo, for example, treats a metarefresh of 0 as a 301, and anything larger as a 302. Most of the time, this works great. But in the case of a 404 error page with a metarefresh- what is it being treated as?

The other search engines vary widely in how they handle these. I believe that since the URL is sent to the search engine by the server as 404;http://www.mysite.com it would normally attempt to treat the page as a 404, and not look at the metarefresh, but I’m not certain if that’s the case, since the metarefresh overrides the initial 200 OK for other pages in order to create the effect of a 302. There is no standardized method of dealing with this from a search engine perspective.

Bottom line, don’t use a metarefresh on an error page. If the page used to exist but is somewhere else, then that’s a legitimate redirect, not an error.

I recommend avoiding metarefreshes on 404 error pages if you are hoping for 404 behaviour (i.e. avoiding duplication issues).

You could use a javascript refresh/forward with no issues, however, since search engines do not execute those.

Holy Grail: Best Practice for Capturing PR and Still Validating

In general, you want a custom error page to respond with a 404 Not Found. However, if you have a lot of broken incoming links to non-existent pages, you may be tempted to capture the PR for them by setting up a custom error page  that does not respond with a 404 or full fledged redirect.

The problem is that this can result in a duplication error, and will mess up validation of your site. There is another option.

What you can do is set up a custom 404 Error page that returns a proper 404 code, then watch your error logs. If you see visits to a bad page, either create a copy of that bad page and 301 it to your home or some other page, or use .htaccess to 301 those specific page calls.

This way, genuine on-the-fly misspellings are sent to an error page, but existing broken links to your site are redirected using a 301 and therefore the PR is passed on to your site. Win both ways 🙂

Conclusion

It’s very common for people to use redirects while attempting to deal with error pages and broken links. Hopefully this has provided some guidance on how to deal with this properly.

End of Redirects for SEO’s Series


Main Article

Detailed Technical Information

Specific Scenarios and How To Deal With Them

Unless otherwise noted, all articles written by Ian McAnerin, BASc, LLB. Copyright © 2002-2004 All Rights Reserved. Permission must be specifically granted in writing for use or reprinting anywhere but on this site, but we do allow it and don’t charge for it, other than a backlink. Contact Us for more information.