Robots.txt: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
== Robots.txt for golbal website or specific folders == | |||
* [http://www.robotstxt.org/wc/norobots.html A Standard for Robot Exclusion] | * [http://www.robotstxt.org/wc/norobots.html A Standard for Robot Exclusion] | ||
* [http://www.searchengineworld.com/cgi-bin/robotcheck.cgi Robots.txt Validator] | * [http://www.searchengineworld.com/cgi-bin/robotcheck.cgi Robots.txt Validator] | ||
| Line 4: | Line 5: | ||
* [http://www.google.com.tw/intl/zh-TW/webmasters/remove.html#outdated 給網站管理員的 Google 資訊] | * [http://www.google.com.tw/intl/zh-TW/webmasters/remove.html#outdated 給網站管理員的 Google 資訊] | ||
: [http://www.google.com/webmasters/remove.html Google Information for Webmasters] (last visited: February 10, 2006) | : [http://www.google.com/webmasters/remove.html Google Information for Webmasters] (last visited: February 10, 2006) | ||
== meta tag for each webpage == | |||
<meta name="robots" content="noindex,nofollow" /> | |||
related: | |||
* [http://www.robotstxt.org/wc/meta-user.html HTML Author's Guide to the Robots META tag] | |||
* [[Add meta tag to movabletype]] | |||
[[Category:Web_Dev]] | [[Category:Web_Dev]] | ||
Revision as of 11:19, 6 May 2007
Robots.txt for golbal website or specific folders
- A Standard for Robot Exclusion
- Robots.txt Validator
- 網站管理員說明中心 - 我不想將要攔截的每個檔案都列出。我可以使用* 和 $ 模式媒合嗎？(EN)
- 給網站管理員的 Google 資訊
- Google Information for Webmasters (last visited: February 10, 2006)
meta tag for each webpage
<meta name="robots" content="noindex,nofollow" />
related: