15,030
edits
User agent: Difference between revisions
Jump to navigation
Jump to search
m
→Search Engine Spiders (Bots)
| Line 13: | Line 13: | ||
| [http://search.msn.com/msnbot.htm MSNBot] || MSN | | [http://search.msn.com/msnbot.htm MSNBot] || MSN | ||
|- | |- | ||
| Slurp || [http://www.inktomi.com/slurp.html Inktomi] | | Slurp || [http://www.inktomi.com/slurp.html Inktomi] | ||
|- | |- | ||
| Fast-Webcrawler || [http://www.fastsearch.com/us/products/fast_web_search/crawler_faq AllTheWeb] | | Fast-Webcrawler || [http://www.fastsearch.com/us/products/fast_web_search/crawler_faq AllTheWeb] | ||
| Line 29: | Line 29: | ||
* Robotstxt.org: [http://www.robotstxt.org/wc/active.html The Web Robots Database] | * Robotstxt.org: [http://www.robotstxt.org/wc/active.html The Web Robots Database] | ||
* WebmasterWorld Forms: [http://www.webmasterworld.com/category1.htm The Search Engine World] | * WebmasterWorld Forms: [http://www.webmasterworld.com/category1.htm The Search Engine World] | ||
* [http://www.hkwebs.net/catalog/teach/setting/robots.php 設定 robots.txt] | |||
== Internet Browsers == | == Internet Browsers == | ||