14,974
edits
Robots.txt: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
== Robots.txt for golbal website or specific folders == | |||
* [http://www.robotstxt.org/wc/norobots.html A Standard for Robot Exclusion] | * [http://www.robotstxt.org/wc/norobots.html A Standard for Robot Exclusion] | ||
* [http://www.searchengineworld.com/cgi-bin/robotcheck.cgi Robots.txt Validator] | * [http://www.searchengineworld.com/cgi-bin/robotcheck.cgi Robots.txt Validator] | ||
| Line 4: | Line 5: | ||
* [http://www.google.com.tw/intl/zh-TW/webmasters/remove.html#outdated 給網站管理員的 Google 資訊] | * [http://www.google.com.tw/intl/zh-TW/webmasters/remove.html#outdated 給網站管理員的 Google 資訊] | ||
: [http://www.google.com/webmasters/remove.html Google Information for Webmasters] (last visited: February 10, 2006) | : [http://www.google.com/webmasters/remove.html Google Information for Webmasters] (last visited: February 10, 2006) | ||
== meta tag for each webpage == | |||
<meta name="robots" content="noindex,nofollow" /> | |||
related: | |||
* [http://www.robotstxt.org/wc/meta-user.html HTML Author's Guide to the Robots META tag] | |||
* [[Add meta tag to movabletype]] | |||
[[Category:Web_Dev]] | [[Category:Web_Dev]] | ||