Robots.txt: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
== Robots.txt for golbal website or specific folders == | == Robots.txt for golbal website or specific folders == | ||
manual | |||
* [http://www.robotstxt.org/wc/norobots.html A Standard for Robot Exclusion] | * [http://www.robotstxt.org/wc/norobots.html A Standard for Robot Exclusion] | ||
* [http://www.searchengineworld.com/cgi-bin/robotcheck.cgi Robots.txt Validator] | * [http://www.searchengineworld.com/cgi-bin/robotcheck.cgi Robots.txt Validator] | ||
| Line 5: | Line 6: | ||
* [http://www.google.com.tw/intl/zh-TW/webmasters/remove.html#outdated 給網站管理員的 Google 資訊] | * [http://www.google.com.tw/intl/zh-TW/webmasters/remove.html#outdated 給網站管理員的 Google 資訊] | ||
: [http://www.google.com/webmasters/remove.html Google Information for Webmasters] (last visited: February 10, 2006) | : [http://www.google.com/webmasters/remove.html Google Information for Webmasters] (last visited: February 10, 2006) | ||
robots.txt validator | |||
* [http://tool.motoricerca.info/robots-checker.phtml New Robots.txt Syntax Checker: a validator for robots.txt files] | |||
== meta tag for each webpage == | == meta tag for each webpage == | ||
Revision as of 20:20, 11 May 2007
Robots.txt for golbal website or specific folders
manual
- A Standard for Robot Exclusion
- Robots.txt Validator
- 網站管理員說明中心 - 我不想將要攔截的每個檔案都列出。我可以使用* 和 $ 模式媒合嗎？(EN)
- 給網站管理員的 Google 資訊
- Google Information for Webmasters (last visited: February 10, 2006)
robots.txt validator
meta tag for each webpage
<meta name="robots" content="noindex,nofollow" />
related: