15,068
edits
Skill tree of web scraping: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| (One intermediate revision by the same user not shown) | |||
| Line 19: | Line 19: | ||
** submit the from after logged the account ★★★☆☆ | ** submit the from after logged the account ★★★☆☆ | ||
* Detection of abnormal data | * Detection of abnormal data | ||
** [https://en.wikipedia.org/wiki/List_of_HTTP_status_codes HTTP status codes] ★★☆☆☆ | ** [https://en.wikipedia.org/wiki/List_of_HTTP_status_codes HTTP status codes] ([[Troubleshooting of HTTP errors]]) ★★☆☆☆ | ||
** Data is wrong even the server throw HTTP 200 status code ★★★☆☆ | ** Data is wrong even the server throw HTTP 200 status code ★★★☆☆ | ||
* Etiquette of web scraping | * Etiquette of web scraping | ||
| Line 31: | Line 31: | ||
* Character encoding ★☆☆☆☆ | * Character encoding ★☆☆☆☆ | ||
* [[Data cleaning]] e.g. unprintable characters ★★☆☆☆ | * [[Data cleaning]] e.g. unprintable characters ★★☆☆☆ | ||
* [ | * [[Regular expression]] ★★★☆☆ | ||
* Selection of database engine ★★★★☆ | * Selection of database engine ★★★★☆ | ||