Skill tree of web scraping: Difference between revisions
mNo edit summary |
No edit summary Tag: wikieditor |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 19: | Line 19: | ||
** submit the from after logged the account ★★★☆☆ | ** submit the from after logged the account ★★★☆☆ | ||
* Detection of abnormal data | * Detection of abnormal data | ||
** [https://en.wikipedia.org/wiki/List_of_HTTP_status_codes HTTP status codes] ★★☆☆☆ | ** [https://en.wikipedia.org/wiki/List_of_HTTP_status_codes HTTP status codes] ([[Troubleshooting of HTTP errors]]) ★★☆☆☆ | ||
** Data is wrong even the server throw HTTP 200 status code ★★★☆☆ | ** Data is wrong even the server throw HTTP 200 status code ★★★☆☆ | ||
* Etiquette of web scraping | * Etiquette of web scraping | ||
| Line 30: | Line 30: | ||
Data transforming | Data transforming | ||
* Character encoding ★☆☆☆☆ | * Character encoding ★☆☆☆☆ | ||
* Data cleaning e.g. unprintable characters ★★☆☆☆ | * [[Data cleaning]] e.g. unprintable characters ★★☆☆☆ | ||
* [ | * [[Regular expression]] ★★★☆☆ | ||
* Selection of database engine ★★★★☆ | * Selection of database engine ★★★★☆ | ||
| Line 38: | Line 38: | ||
[[Category:Data Science]] | [[Category:Data Science]] | ||
[[Category:Data collecting]] | [[Category:Data collecting]] | ||
[[Category:web scraping]] | |||