Robots.txt: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| Line 19: | Line 19: | ||
* [http://www.robotstxt.org/wc/meta-user.html HTML Author's Guide to the Robots META tag] | * [http://www.robotstxt.org/wc/meta-user.html HTML Author's Guide to the Robots META tag] | ||
* [[Add meta tag to movabletype]] | * [[Add meta tag to movabletype]] | ||
* [http://www.robbin.cc/vb/showthread.php?t=485 如何用.htaccess擋搜尋器 - WordPress歡樂正體中文交流所] | |||
[[Category:Web_Dev]] | [[Category:Web_Dev]] | ||
Revision as of 21:16, 13 June 2007
Robots.txt for golbal website or specific folders
manual
- A Standard for Robot Exclusion
- Robots.txt Validator
- 網站管理員說明中心 - 我不想將要攔截的每個檔案都列出。我可以使用* 和 $ 模式媒合嗎？(EN)
- 給網站管理員的 Google 資訊
- Google Information for Webmasters (last visited: February 10, 2006)
robots.txt validator
meta tag for each webpage
<meta name="robots" content="noindex,nofollow" />
related: