How to extract content from websites: Difference between revisions

From LemonWiki共筆
Jump to navigation Jump to search
Line 2: Line 2:


== Methods for article extraction ==
== Methods for article extraction ==
[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]
[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]<ref>[https://videoinu.com/blog/firefox-reader-view-heuristics/ How does Firefox's Reader View work?]</ref>
* Demo:
* Demo:
* Pricing / Free Limit: free
* Requirement: Node.js  
* Requirement: Node.js  
* License: Apache License, Version 2.0 {{Gd}}
* License: Apache License, Version 2.0 {{Gd}}
* Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image | Docker Hub]
* Source code of client: Available on GitHub & Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image | Docker Hub]


[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites]
[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites]
Line 34: Line 35:
''$'' [https://www.diffbot.com/products/extract/ Diffbot | Extract Content From Websites Automatically] two weeks free trial
''$'' [https://www.diffbot.com/products/extract/ Diffbot | Extract Content From Websites Automatically] two weeks free trial
* Demo:  
* Demo:  
* Pricing / Free Limit: 14 days free trial
* Requirement:  
* Requirement:  
* License:
* License:
Line 42: Line 44:
* Requirement: PHP
* Requirement: PHP
* License:  
* License:  
* Container
* Source code of client: [https://www.diffbot.com/dev/docs/libraries/ Diffbot Libraries - Diffbot]


[https://totheweb.com/learning_center/tools-convert-html-text-to-plain-text-for-content-review/ Free Tool: Convert Your Webpage to Plain Text » ToTheWeb]
[https://totheweb.com/learning_center/tools-convert-html-text-to-plain-text-for-content-review/ Free Tool: Convert Your Webpage to Plain Text » ToTheWeb]
Line 49: Line 51:
* License: GPL + Apache License Version 2.0 {{Gd}}
* License: GPL + Apache License Version 2.0 {{Gd}}
* Container
* Container
[https://github.com/crscheid/php-article-extractor crscheid/php-article-extractor: A PHP library to extract article text from web pages]
* Pricing / Free Limit: free
* Source code of client: Available on GitHub
* License:
[https://getpocket.com/developer/docs/v3/article-view Pocket Developer Program: Pocket API: Article View] not available {{exclaim}}
* Pricing / Free Limit:
* Source code of client: n/a
* License:
== References ==
<references />
[[Category:Tool]]


== Related pages ==
== Related pages ==

Revision as of 18:02, 16 January 2024

How to extract article content from websites

Methods for article extraction

mozilla/readability: A standalone version of the readability lib[1]

timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites

postlight/mercury-parser: 📜 Extract meaningful content from the chaos of a web page

  • Demo:
  • Requirement: Node.js
  • License: Apache License, Version 2.0 or MIT license Good.gif
  • Container

luin/readability: 📚 Turn any web page into a clean view

  • Demo:
  • Requirement: Node.js
  • License: Apache License 2.0 Good.gif
  • Container

adbar/trafilatura: Web scraping library and command-line tool for text discovery and extraction (main content, metadata, comments) + mozilla/readability: A standalone version of the readability lib

  • Demo:
  • Requirement:
  • License:
  • Container

$ Diffbot | Extract Content From Websites Automatically two weeks free trial

  • Demo:
  • Pricing / Free Limit: 14 days free trial
  • Requirement:
  • License:
  • Container

$ Full-Text RSS - FiveFilters.org

Free Tool: Convert Your Webpage to Plain Text » ToTheWeb

  • Demo:
  • Requirement:
  • License: GPL + Apache License Version 2.0 Good.gif
  • Container

crscheid/php-article-extractor: A PHP library to extract article text from web pages

  • Pricing / Free Limit: free
  • Source code of client: Available on GitHub
  • License:

Pocket Developer Program: Pocket API: Article View not available Icon_exclaim.gif

  • Pricing / Free Limit:
  • Source code of client: n/a
  • License:

References

  1. ↑ How does Firefox's Reader View work?

Related pages

Retrieved from "https://wiki.planetoid.info/index.php?title=How_to_extract_content_from_websites&oldid=24307"