How to extract content from websites: Difference between revisions
| Line 2: | Line 2: | ||
== Methods for article extraction == | == Methods for article extraction == | ||
[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]<ref>[https://videoinu.com/blog/firefox-reader-view-heuristics/ How does Firefox's Reader View work?]</ref> | {{Gd}} [https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]<ref>[https://videoinu.com/blog/firefox-reader-view-heuristics/ How does Firefox's Reader View work?]</ref> | ||
* Demo: | * Demo: | ||
* Pricing / Free Limit: free | * Pricing / Free Limit: free | ||
| Line 8: | Line 8: | ||
* License: Apache License, Version 2.0 {{Gd}} | * License: Apache License, Version 2.0 {{Gd}} | ||
* Source code of client: Available on GitHub & Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image | Docker Hub] | * Source code of client: Available on GitHub & Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image | Docker Hub] | ||
{{Gd}} [https://github.com/postlight/parser postlight/parser: 📜 Extract meaningful content from the chaos of a web page] (Replacement of [https://github.com/postlight/mercury-parser postlight/mercury-parser]) | |||
* Demo: | |||
* Requirement: Node.js | |||
* License: Apache License, Version 2.0 or MIT license {{Gd}} | |||
* Container | |||
[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites] | [https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites] | ||
| Line 13: | Line 19: | ||
* Requirement: PHP | * Requirement: PHP | ||
* License: GNU Affero General Public License v3.0 {{Gd}} | * License: GNU Affero General Public License v3.0 {{Gd}} | ||
* Container | * Container | ||
Revision as of 16:07, 18 January 2024
How to extract article content from websites
Methods for article extraction
mozilla/readability: A standalone version of the readability lib[1]
- Demo:
- Pricing / Free Limit: free
- Requirement: Node.js
- License: Apache License, Version 2.0
- Source code of client: Available on GitHub & Container: phpdockerio/readability-js-server - Docker Image | Docker Hub
postlight/parser: 📜 Extract meaningful content from the chaos of a web page (Replacement of postlight/mercury-parser)
- Demo:
- Requirement: Node.js
- License: Apache License, Version 2.0 or MIT license
- Container
timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites
- Demo: Full-Text RSS - FiveFilters.org
- Requirement: PHP
- License: GNU Affero General Public License v3.0
- Container
luin/readability: 📚 Turn any web page into a clean view
- Demo:
- Requirement: Node.js
- License: Apache License 2.0
- Container
adbar/trafilatura: Web scraping library and command-line tool for text discovery and extraction (main content, metadata, comments) + mozilla/readability: A standalone version of the readability lib
- Demo:
- Requirement:
- License:
- Container
$ Diffbot | Extract Content From Websites Automatically two weeks free trial
- Demo:
- Pricing / Free Limit: 14 days free trial
- Requirement:
- License:
- Container
$ Full-Text RSS - FiveFilters.org
- Demo: Full-Text RSS Feeds | from fivefilters.org
- Requirement: PHP
- License:
- Source code of client: Diffbot Libraries - Diffbot
Free Tool: Convert Your Webpage to Plain Text » ToTheWeb
- Demo:
- Requirement:
- License: GPL + Apache License Version 2.0
- Container
crscheid/php-article-extractor: A PHP library to extract article text from web pages
- Pricing / Free Limit: free
- Source code of client: Available on GitHub
- License:
Pocket Developer Program: Pocket API: Article View not available 
- Pricing / Free Limit:
- Source code of client: n/a
- License: