How to extract content from websites: Difference between revisions
Jump to navigation
Jump to search
| Line 2: | Line 2: | ||
== Methods == | == Methods == | ||
[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib] | |||
* Demo: | |||
* Requirement: Node.js | |||
* License: Apache License, Version 2.0 {{Gd}} | |||
* Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image | Docker Hub] | |||
[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites] | [https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites] | ||
| Line 7: | Line 12: | ||
* Requirement: PHP | * Requirement: PHP | ||
* License: GNU Affero General Public License v3.0 {{Gd}} | * License: GNU Affero General Public License v3.0 {{Gd}} | ||
* Container | |||
[https://github.com/postlight/mercury-parser postlight/mercury-parser: 📜 Extract meaningful content from the chaos of a web page] | [https://github.com/postlight/mercury-parser postlight/mercury-parser: 📜 Extract meaningful content from the chaos of a web page] | ||
| Line 13: | Line 18: | ||
* Requirement: Node.js | * Requirement: Node.js | ||
* License: Apache License, Version 2.0 or MIT license {{Gd}} | * License: Apache License, Version 2.0 or MIT license {{Gd}} | ||
* Container | |||
[https://github.com/luin/readability luin/readability: 📚 Turn any web page into a clean view] | [https://github.com/luin/readability luin/readability: 📚 Turn any web page into a clean view] | ||
| Line 19: | Line 24: | ||
* Requirement: Node.js | * Requirement: Node.js | ||
* License: Apache License 2.0 | * License: Apache License 2.0 | ||
* Container | |||
[https://github.com/adbar/trafilatura adbar/trafilatura: Web scraping library and command-line tool for text discovery and extraction (main content, metadata, comments)] + [https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib] | [https://github.com/adbar/trafilatura adbar/trafilatura: Web scraping library and command-line tool for text discovery and extraction (main content, metadata, comments)] + [https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib] | ||
| Line 24: | Line 30: | ||
* Requirement: | * Requirement: | ||
* License: | * License: | ||
* Container | |||
''$'' [https://www.diffbot.com/products/extract/ Diffbot | Extract Content From Websites Automatically] two weeks free trial | ''$'' [https://www.diffbot.com/products/extract/ Diffbot | Extract Content From Websites Automatically] two weeks free trial | ||
| Line 29: | Line 36: | ||
* Requirement: | * Requirement: | ||
* License: | * License: | ||
* Container | |||
[https://totheweb.com/learning_center/tools-convert-html-text-to-plain-text-for-content-review/ Free Tool: Convert Your Webpage to Plain Text » ToTheWeb] | [https://totheweb.com/learning_center/tools-convert-html-text-to-plain-text-for-content-review/ Free Tool: Convert Your Webpage to Plain Text » ToTheWeb] | ||
| Line 35: | Line 42: | ||
* Requirement: | * Requirement: | ||
* License: GPL + Apache License Version 2.0 | * License: GPL + Apache License Version 2.0 | ||
* Container | |||
== Related pages == | == Related pages == | ||
Revision as of 10:27, 18 January 2023
How to extract article content from websites
Methods
mozilla/readability: A standalone version of the readability lib
- Demo:
- Requirement: Node.js
- License: Apache License, Version 2.0
- Container: phpdockerio/readability-js-server - Docker Image | Docker Hub
timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites
- Demo: Full-Text RSS - FiveFilters.org
- Requirement: PHP
- License: GNU Affero General Public License v3.0
- Container
postlight/mercury-parser: 📜 Extract meaningful content from the chaos of a web page
- Demo:
- Requirement: Node.js
- License: Apache License, Version 2.0 or MIT license
- Container
luin/readability: 📚 Turn any web page into a clean view
- Demo:
- Requirement: Node.js
- License: Apache License 2.0
- Container
adbar/trafilatura: Web scraping library and command-line tool for text discovery and extraction (main content, metadata, comments) + mozilla/readability: A standalone version of the readability lib
- Demo:
- Requirement:
- License:
- Container
$ Diffbot | Extract Content From Websites Automatically two weeks free trial
- Demo:
- Requirement:
- License:
- Container
Free Tool: Convert Your Webpage to Plain Text » ToTheWeb
- Demo:
- Requirement:
- License: GPL + Apache License Version 2.0
- Container