How to extract content from websites - Revision history

Planetoid at 03:30, 15 July 2025

2025-07-15T03:30:51Z

← Older revision		Revision as of 11:30, 15 July 2025
Line 83:		Line 83:
	* [[Named entity recognition tools]]		* [[Named entity recognition tools]]
	* ''$'' [https://www.producthunt.com/products/diffbot/alternatives Best Diffbot Alternatives - 2024 \| Product Hunt]		* ''$'' [https://www.producthunt.com/products/diffbot/alternatives Best Diffbot Alternatives - 2024 \| Product Hunt]

			{{Template:Data factory flow}}

	[[Category:Tool]]		[[Category:Tool]]

Unknown user: /* Related pages */

2024-01-22T11:26:52Z

← Older revision		Revision as of 19:26, 22 January 2024
Line 82:		Line 82:
	== Related pages ==		== Related pages ==
	* [[Named entity recognition tools]]		* [[Named entity recognition tools]]
			* ''$'' [https://www.producthunt.com/products/diffbot/alternatives Best Diffbot Alternatives - 2024 \| Product Hunt]

	[[Category:Tool]]		[[Category:Tool]]

Unknown user: /* Commercial solution for article extraction */

2024-01-22T11:25:18Z

Commercial solution for article extraction

← Older revision		Revision as of 19:25, 22 January 2024
Line 52:		Line 52:
	* Requirement:		* Requirement:
	* License:		* License:
	* ~~Container~~		* Source code of client: [https://www.diffbot.com/dev/docs/libraries/ Diffbot Libraries - Diffbot]

	''$'' [https://www.fivefilters.org/full-text-rss/ Full-Text RSS - FiveFilters.org]		''$'' [https://www.fivefilters.org/full-text-rss/ Full-Text RSS - FiveFilters.org]
Line 58:		Line 58:
	* Requirement: PHP		* Requirement: PHP
	* License:		* License:
	* Source code of client: ~~[https://www.diffbot.com/dev/docs/libraries/ Diffbot Libraries - Diffbot]~~		* Source code of client:

	[https://getpocket.com/developer/docs/v3/article-view Pocket Developer Program: Pocket API: Article View] not available {{exclaim}}		[https://getpocket.com/developer/docs/v3/article-view Pocket Developer Program: Pocket API: Article View] not available {{exclaim}}
Line 65:		Line 65:
	* License:		* License:

			''$'' [https://newsblur.com/ NewsBlur] > [https://newsblur.com/api The NewsBlur API] > GET /rss_feeds/original_story
			* Pricing:
			* License:
			* Source code of client:

			''$'' [https://feedbin.com/ Feedbin] > API > [https://github.com/feedbin/feedbin-api/blob/master/content/extract-full-content.md Extracting Full Content]
			* Pricing:
			* License:
			* Source code of client:

	== References ==		== References ==

Unknown user at 04:25, 19 January 2024

2024-01-19T04:25:03Z

@@ Line 2: / Line 2: @@
 == Methods for article extraction ==
 {{Gd}} [https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]<ref>[https://videoinu.com/blog/firefox-reader-view-heuristics/ How does Firefox's Reader View work?]</ref>
 * Demo:
@@ Line 32: / Line 33: @@
 * License:
 * Container
 ''$'' [https://www.diffbot.com/products/extract/ Diffbot | Extract Content From Websites Automatically] two weeks free trial
@@ Line 45: / Line 59: @@
 * License:
 * Source code of client: [https://www.diffbot.com/dev/docs/libraries/ Diffbot Libraries - Diffbot]
 [https://getpocket.com/developer/docs/v3/article-view Pocket Developer Program: Pocket API: Article View] not available {{exclaim}}
@@ Line 61: / Line 64: @@
 * Source code of client: n/a
 * License:
 == References ==

Unknown user: /* Methods for article extraction */

2024-01-18T08:07:06Z

Methods for article extraction

← Older revision		Revision as of 16:07, 18 January 2024
Line 2:		Line 2:

	== Methods for article extraction ==		== Methods for article extraction ==
	[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]<ref>[https://videoinu.com/blog/firefox-reader-view-heuristics/ How does Firefox's Reader View work?]</ref>		{{Gd}} [https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]<ref>[https://videoinu.com/blog/firefox-reader-view-heuristics/ How does Firefox's Reader View work?]</ref>
	* Demo:		* Demo:
	* Pricing / Free Limit: free		* Pricing / Free Limit: free
Line 8:		Line 8:
	* License: Apache License, Version 2.0 {{Gd}}		* License: Apache License, Version 2.0 {{Gd}}
	* Source code of client: Available on GitHub & Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image \| Docker Hub]		* Source code of client: Available on GitHub & Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image \| Docker Hub]

			{{Gd}} [https://github.com/postlight/parser postlight/parser: 📜 Extract meaningful content from the chaos of a web page] (Replacement of [https://github.com/postlight/mercury-parser postlight/mercury-parser])
			* Demo:
			* Requirement: Node.js
			* License: Apache License, Version 2.0 or MIT license {{Gd}}
			* Container

	[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites]		[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites]
Line 13:		Line 19:
	* Requirement: PHP		* Requirement: PHP
	* License: GNU Affero General Public License v3.0 {{Gd}}		* License: GNU Affero General Public License v3.0 {{Gd}}
	* Container

	~~[https://github.com/postlight/mercury-parser postlight/mercury-parser: 📜 Extract meaningful content from the chaos of a web page]~~
	* Demo:
	* Requirement: Node.js
	* License: Apache License, Version 2.0 or MIT license {{Gd}}
	* Container		* Container

Unknown user: /* Methods for article extraction */

2024-01-16T10:02:09Z

Methods for article extraction

← Older revision		Revision as of 18:02, 16 January 2024
Line 2:		Line 2:

	== Methods for article extraction ==		== Methods for article extraction ==
	[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]		[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]<ref>[https://videoinu.com/blog/firefox-reader-view-heuristics/ How does Firefox's Reader View work?]</ref>
	* Demo:		* Demo:
			* Pricing / Free Limit: free
	* Requirement: Node.js		* Requirement: Node.js
	* License: Apache License, Version 2.0 {{Gd}}		* License: Apache License, Version 2.0 {{Gd}}
	* Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image \| Docker Hub]		* Source code of client: Available on GitHub & Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image \| Docker Hub]

	[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites]		[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites]
Line 34:		Line 35:
	''$'' [https://www.diffbot.com/products/extract/ Diffbot \| Extract Content From Websites Automatically] two weeks free trial		''$'' [https://www.diffbot.com/products/extract/ Diffbot \| Extract Content From Websites Automatically] two weeks free trial
	* Demo:		* Demo:
			* Pricing / Free Limit: 14 days free trial
	* Requirement:		* Requirement:
	* License:		* License:
Line 42:		Line 44:
	* Requirement: PHP		* Requirement: PHP
	* License:		* License:
	* ~~Container~~		* Source code of client: [https://www.diffbot.com/dev/docs/libraries/ Diffbot Libraries - Diffbot]

	[https://totheweb.com/learning_center/tools-convert-html-text-to-plain-text-for-content-review/ Free Tool: Convert Your Webpage to Plain Text » ToTheWeb]		[https://totheweb.com/learning_center/tools-convert-html-text-to-plain-text-for-content-review/ Free Tool: Convert Your Webpage to Plain Text » ToTheWeb]
Line 49:		Line 51:
	* License: GPL + Apache License Version 2.0 {{Gd}}		* License: GPL + Apache License Version 2.0 {{Gd}}
	* Container		* Container

			[https://github.com/crscheid/php-article-extractor crscheid/php-article-extractor: A PHP library to extract article text from web pages]
			* Pricing / Free Limit: free
			* Source code of client: Available on GitHub
			* License:

			[https://getpocket.com/developer/docs/v3/article-view Pocket Developer Program: Pocket API: Article View] not available {{exclaim}}
			* Pricing / Free Limit:
			* Source code of client: n/a
			* License:

			== References ==
			<references />

			[[Category:Tool]]

	== Related pages ==		== Related pages ==

Unknown user: /* Methods for article extraction */

2023-01-18T06:06:19Z

Methods for article extraction

← Older revision		Revision as of 14:06, 18 January 2023
Line 36:		Line 36:
	* Requirement:		* Requirement:
	* License:		* License:
			* Container

			''$'' [https://www.fivefilters.org/full-text-rss/ Full-Text RSS - FiveFilters.org]
			* Demo: [http://ftr.fivefilters.org/ Full-Text RSS Feeds \| from fivefilters.org]
			* Requirement: PHP
			* License:
	* Container		* Container

Unknown user at 06:04, 18 January 2023

2023-01-18T06:04:23Z

← Older revision		Revision as of 14:04, 18 January 2023
Line 1:		Line 1:
	How to extract article content from websites		How to extract article content from websites

	== Methods ==		== Methods for article extraction ==
	[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]		[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]
	* Demo:		* Demo:

Unknown user: /* Methods */

2023-01-18T02:28:24Z

Methods

← Older revision		Revision as of 10:28, 18 January 2023
Line 23:		Line 23:
	* Demo:		* Demo:
	* Requirement: Node.js		* Requirement: Node.js
	* License: Apache License 2.0		* License: Apache License 2.0 {{Gd}}
	* Container		* Container

Line 41:		Line 41:
	* Demo:		* Demo:
	* Requirement:		* Requirement:
	* License: GPL + Apache License Version 2.0		* License: GPL + Apache License Version 2.0 {{Gd}}
	* Container		* Container

Unknown user: /* Methods */

2023-01-18T02:27:46Z

Methods

← Older revision		Revision as of 10:27, 18 January 2023
Line 2:		Line 2:

	== Methods ==		== Methods ==
			[https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]
			* Demo:
			* Requirement: Node.js
			* License: Apache License, Version 2.0 {{Gd}}
			* Container: [https://hub.docker.com/r/phpdockerio/readability-js-server phpdockerio/readability-js-server - Docker Image \| Docker Hub]

	[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites]		[https://github.com/timothytylee/full-text-rss timothytylee/full-text-rss: Fork of Full-Text RSS to improve handling of non UTF-8 sites]
Line 7:		Line 12:
	* Requirement: PHP		* Requirement: PHP
	* License: GNU Affero General Public License v3.0 {{Gd}}		* License: GNU Affero General Public License v3.0 {{Gd}}
			* Container

	[https://github.com/postlight/mercury-parser postlight/mercury-parser: 📜 Extract meaningful content from the chaos of a web page]		[https://github.com/postlight/mercury-parser postlight/mercury-parser: 📜 Extract meaningful content from the chaos of a web page]
Line 13:		Line 18:
	* Requirement: Node.js		* Requirement: Node.js
	* License: Apache License, Version 2.0 or MIT license {{Gd}}		* License: Apache License, Version 2.0 or MIT license {{Gd}}
			* Container

	[https://github.com/luin/readability luin/readability: 📚 Turn any web page into a clean view]		[https://github.com/luin/readability luin/readability: 📚 Turn any web page into a clean view]
Line 19:		Line 24:
	* Requirement: Node.js		* Requirement: Node.js
	* License: Apache License 2.0		* License: Apache License 2.0
			* Container

	[https://github.com/adbar/trafilatura adbar/trafilatura: Web scraping library and command-line tool for text discovery and extraction (main content, metadata, comments)] + [https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]		[https://github.com/adbar/trafilatura adbar/trafilatura: Web scraping library and command-line tool for text discovery and extraction (main content, metadata, comments)] + [https://github.com/mozilla/readability mozilla/readability: A standalone version of the readability lib]
Line 24:		Line 30:
	* Requirement:		* Requirement:
	* License:		* License:
			* Container

	''$'' [https://www.diffbot.com/products/extract/ Diffbot \| Extract Content From Websites Automatically] two weeks free trial		''$'' [https://www.diffbot.com/products/extract/ Diffbot \| Extract Content From Websites Automatically] two weeks free trial
Line 29:		Line 36:
	* Requirement:		* Requirement:
	* License:		* License:
			* Container

	[https://totheweb.com/learning_center/tools-convert-html-text-to-plain-text-for-content-review/ Free Tool: Convert Your Webpage to Plain Text » ToTheWeb]		[https://totheweb.com/learning_center/tools-convert-html-text-to-plain-text-for-content-review/ Free Tool: Convert Your Webpage to Plain Text » ToTheWeb]
Line 35:		Line 42:
	* Requirement:		* Requirement:
	* License: GPL + Apache License Version 2.0		* License: GPL + Apache License Version 2.0
			* Container

	== Related pages ==		== Related pages ==