Web scrape troubleshooting: Difference between revisions
Jump to navigation
Jump to search
m
→Before start to web scrpae
| Line 46: | Line 46: | ||
</div> | </div> | ||
== Before start to web | == Before start to writing the scirpt of web scraping (crawler) == | ||
* Are | * Are the website offer datasets or files? e.g. open data | ||
* Are | * Are the website offer [https://en.wikipedia.org/wiki/Application_programming_interface API] (Application programming interface)? | ||
'''How to find the unofficial (3rd-party) web crawler?''' Search keyword strategy suggested as follows: | '''How to find the unofficial (3rd-party) web crawler?''' Search keyword strategy suggested as follows: | ||
* ''target website'' + crawler | * ''target website'' + crawler | ||
* ''target website'' + bot | * ''target website'' + bot | ||
* ''target website'' + download | * ''target website'' + download / downloader | ||
== Further reading == | == Further reading == | ||