Named entity recognition tools: Difference between revisions

Jump to navigation Jump to search

Named entity recognition tools (edit)

Revision as of 20:23, 8 January 2025

3,857 bytes added ,  8 January 2025
m
Text replacement - ": Image:Owl icon.jpg " to "{{Tips}} "
No edit summary
m (Text replacement - ": Image:Owl icon.jpg " to "{{Tips}} ")
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:
Named entity recognition (NER) 或稱[https://zh.wikipedia.org/wiki/%E5%91%BD%E5%90%8D%E5%AE%9E%E4%BD%93%E8%AF%86%E5%88%AB 命名實體識別]、實體識別、專有名詞辨識
Named entity recognition (NER) 或稱[https://zh.wikipedia.org/wiki/%E5%91%BD%E5%90%8D%E5%AE%9E%E4%BD%93%E8%AF%86%E5%88%AB 命名實體識別]、實體識別、專有名詞辨識
== CKIP Neural Chinese Word Segmentation, POS Tagging, and NER ==
[https://github.com/ckiplab/ckiptagger ckiplab/ckiptagger: CKIP Neural Chinese Word Segmentation, POS Tagging, and NER]
* license: [https://github.com/ckiplab/ckiptagger/blob/master/LICENSE GNU General Public License v3.0] {{Gd}}
* language support: Traditional Chinese
* programming language: Python
* Score:
* classes of entity<ref>[https://iptt.sinica.edu.tw/uploads/datas/2019/4/a251a61991139dc023d3559e93cd8d65.pdf 中文專有名詞辨識系統  簡報]</ref>
<table border="1" class="wikitable sortable">
<tr><th>Class name in English</th><th>Class name in Traditional Chinese</th></tr>
<tr><td>person</td><td>人名</td></tr>
<tr><td>norp</td><td>團體</td></tr>
<tr><td>FAC</td><td>設施</td></tr>
<tr><td>facility</td><td>設施*</td></tr>
<tr><td>ORG</td><td>組織</td></tr>
<tr><td>organization</td><td>組織*</td></tr>
<tr><td>gpe</td><td>地理</td></tr>
<tr><td>LOC</td><td>地點</td></tr>
<tr><td>location</td><td>地點*</td></tr>
<tr><td>product</td><td>商品</td></tr>
<tr><td>event</td><td>事件</td></tr>
<tr><td>WORK</td><td>藝術品</td></tr>
<tr><td>work of art</td><td>藝術品*</td></tr>
<tr><td>law</td><td>法律</td></tr>
<tr><td>language</td><td>語言</td></tr>
<tr><td>date</td><td>日期</td></tr>
<tr><td>time</td><td>時間</td></tr>
<tr><td>percent</td><td>比例</td></tr>
<tr><td>money</td><td>錢</td></tr>
<tr><td>quantity</td><td>數量</td></tr>
<tr><td>ordinal</td><td>序數</td></tr>
<tr><td>cardinal</td><td>數詞</td></tr>
</table>
: [[Image:Owl icon.jpg]] Notes: Asterisk symbol means there are different class name in English but same class name in Chinese.
== Stanford CoreNLP ==
[https://stanfordnlp.github.io/CoreNLP/index.html Stanford CoreNLP – Natural language software | Stanford CoreNLP]
* license: GNU General Public License v3 {{Gd}}
* language support: English, Chinese ..
* programming language: Java
* Score: Available
* classes of entity: "For English, by default, this annotator recognizes named (PERSON, LOCATION, ORGANIZATION, MISC), numerical (MONEY, NUMBER, ORDINAL, PERCENT), and temporal (DATE, TIME, DURATION, SET) entities (12 classes). <ref>[https://stanfordnlp.github.io/CoreNLP/ner.html#description Named Entity Recognition – NERClassifierCombiner | Stanford CoreNLP]</ref>"
== spaCy ==
[https://spacy.io/ spaCy · Industrial-strength Natural Language Processing in Python]
* license: [https://github.com/explosion/spaCy/blob/master/LICENSE MIT License] {{Gd}}
* language support:
* programming language: Python
* Score:
* classes of entity: "PERSON, NORP, FAC, ORG, GPE, LOC, PRODUCT, EVENT, WORK_OF_ART, LAW, LANGUAGE, DATE, TIME, PERCENT, MONEY, QUANTITY, ORDINAL and CARDINAL <ref>[https://spacy.io/api/annotation#named-entities Annotation Specifications · spaCy API Documentation]</ref>"
== Google Cloud Natural Language ==
[https://cloud.google.com/natural-language/ Cloud Natural Language  |  Cloud Natural Language API  |  Google Cloud]
* license:
* language support: [https://cloud.google.com/natural-language/docs/languages 語言支援  |  Cloud Natural Language API  |  Google Cloud] included Traditional Chinese
* programming language: multiple
* Score: Available. '''salience score''' in the [0, 1.0] range. "The salience score for an entity provides information about the importance or centrality of that entity to the entire document text. Scores closer to 0 are less salient, while scores closer to 1.0 are highly salient.<ref>[https://cloud.google.com/natural-language/docs/reference/rest/v1/Entity Entity  |  Cloud Natural Language API  |  Google Cloud]</ref>"
* classes of entity: Details on [https://cloud.google.com/natural-language/docs/reference/rest/v1/Entity Entity  |  Cloud Natural Language API  |  Google Cloud] -> Type of the entity e.g. "UNKNOWN, PERSON, LOCATION, ORGANIZATION, EVENT, WORK_OF_ART, CONSUMER_GOOD, OTHER, PHONE_NUMBER, ADDRESS, DATE, NUMBER and PRICE"


== Amazon Comprehend ==
== Amazon Comprehend ==
Line 66: Line 6:
* language support:
* language support:
* programming language:
* programming language:
* Score: Available
* Score: Available. "Each entity also has a score that indicates the level of confidence that Amazon Comprehend has that it correctly detected the entity type. You can filter out the entities with lower scores to reduce the risk of using incorrect detections.<ref>[https://docs.aws.amazon.com/zh_tw/comprehend/latest/dg/how-entities.html Detect Entities - Amazon Comprehend]</ref>"
* classes of entity: "COMMERCIAL_ITEM, DATE, EVENT, LOCATION, ORGANIZATION, OTHER, PERSON, QUANTITY and TITLE"<ref>[https://docs.aws.amazon.com/comprehend/latest/dg/how-entities.html Detect Entities - Amazon Comprehend]</ref> as the following:
* classes of entity: "COMMERCIAL_ITEM, DATE, EVENT, LOCATION, ORGANIZATION, OTHER, PERSON, QUANTITY and TITLE"<ref>[https://docs.aws.amazon.com/comprehend/latest/dg/how-entities.html Detect Entities - Amazon Comprehend]</ref> as the following:


Line 121: Line 61:
   </tr>
   </tr>
</table>
</table>
== Apache OpenNLP ==
[https://opennlp.apache.org/ Apache OpenNLP]
* license: Apache License, Version 2.0
* language support: English, French, German, Italian and Dutch. Not support Chinese. [https://opennlp.apache.org/models.html Models Download - Apache OpenNLP]
* programming language: Java
* Score: Available at github: [https://github.com/apache/opennlp apache/opennlp: Mirror of Apache OpenNLP].
* classes of entity:
== Baidu 百度AI开放平台 ==
[https://ai.baidu.com/tech/nlp 语言处理基础技术-百度AI开放平台] "专名识别"<ref>[https://ai.baidu.com/docs#/NLP-Basic-API/63eec4cf 词法分析接口]</ref> / [https://github.com/baidu/lac baidu/lac: 百度NLP：分词，词性标注，命名实体识别]
* license:
* language support: simplified Chinese
* programming language: multiple
* Score:
* classes of entity:
<table border="1" class="wikitable sortable">
<tr><th>Class name in English (缩略词)</th><th>Class name in Simplified Chinese</th><th>Class name in Traditional Chinese</th></tr>
<tr><td>PER</td><td>人名</td><td>人名</td></tr>
<tr><td>LOC</td><td>地名</td><td>地名</td></tr>
<tr><td>ORG</td><td>机构名</td><td>機構名</td></tr>
<tr><td>TIME</td><td>时间</td><td>時間</td></tr>
</table>
== CKIP Neural Chinese Word Segmentation, POS Tagging, and NER ==
[https://github.com/ckiplab/ckiptagger ckiplab/ckiptagger: CKIP Neural Chinese Word Segmentation, POS Tagging, and NER]
* license: [https://github.com/ckiplab/ckiptagger/blob/master/LICENSE GNU General Public License v3.0] {{Gd}}
* language support: Traditional Chinese
* programming language: Python
* Score:
* classes of entity<ref>[https://iptt.sinica.edu.tw/uploads/website/16/datas/2019/4/a251a61991139dc023d3559e93cd8d65.pdf 中文專有名詞辨識系統  簡報] ([https://web.archive.org/web/20240811001758/https://iptt.sinica.edu.tw/uploads/website/16/datas/2019/4/a251a61991139dc023d3559e93cd8d65.pdf 網頁備份])</ref>
<table border="1" class="wikitable sortable">
<tr><th>Class name in English</th><th>Class name in Traditional Chinese</th></tr>
<tr><td>person</td><td>人名</td></tr>
<tr><td>norp</td><td>團體</td></tr>
<tr><td>FAC</td><td>設施</td></tr>
<tr><td>facility</td><td>設施*</td></tr>
<tr><td>ORG</td><td>組織</td></tr>
<tr><td>organization</td><td>組織*</td></tr>
<tr><td>gpe</td><td>地理</td></tr>
<tr><td>LOC</td><td>地點</td></tr>
<tr><td>location</td><td>地點*</td></tr>
<tr><td>product</td><td>商品</td></tr>
<tr><td>event</td><td>事件</td></tr>
<tr><td>WORK</td><td>藝術品</td></tr>
<tr><td>work of art</td><td>藝術品*</td></tr>
<tr><td>law</td><td>法律</td></tr>
<tr><td>language</td><td>語言</td></tr>
<tr><td>date</td><td>日期</td></tr>
<tr><td>time</td><td>時間</td></tr>
<tr><td>percent</td><td>比例</td></tr>
<tr><td>money</td><td>錢</td></tr>
<tr><td>quantity</td><td>數量</td></tr>
<tr><td>ordinal</td><td>序數</td></tr>
<tr><td>cardinal</td><td>數詞</td></tr>
</table>
{{Tips}} Notes: Asterisk symbol means there are different class name in English but same class name in Chinese.
== Google Cloud Natural Language ==
[https://cloud.google.com/natural-language/ Cloud Natural Language  |  Cloud Natural Language API  |  Google Cloud]
* license:
* language support: [https://cloud.google.com/natural-language/docs/languages 語言支援  |  Cloud Natural Language API  |  Google Cloud] included Traditional Chinese
* programming language: multiple
* Score: Available. '''salience score''' in the [0, 1.0] range. "The salience score for an entity provides information about the importance or centrality of that entity to the entire document text. Scores closer to 0 are less salient, while scores closer to 1.0 are highly salient.<ref>[https://cloud.google.com/natural-language/docs/reference/rest/v1/Entity Entity  |  Cloud Natural Language API  |  Google Cloud]</ref>"
* classes of entity: Details on [https://cloud.google.com/natural-language/docs/reference/rest/v1/Entity Entity  |  Cloud Natural Language API  |  Google Cloud] -> Type of the entity e.g. "UNKNOWN, PERSON, LOCATION, ORGANIZATION, EVENT, WORK_OF_ART, CONSUMER_GOOD, OTHER, PHONE_NUMBER, ADDRESS, DATE, NUMBER and PRICE"


== IBM Watson ==
== IBM Watson ==
Line 129: Line 139:
* Score:
* Score:
* classes of entity: "Date, Duration, EmailAddress, Facility, GeographicFeature, Hashtag, IPAddress, JobTitle, Location and more ..."<ref>[https://cloud.ibm.com/docs/services/natural-language-understanding?topic=natural-language-understanding-entity-types-version-2&locale=en Entity types (Version 2)]</ref>
* classes of entity: "Date, Duration, EmailAddress, Facility, GeographicFeature, Hashtag, IPAddress, JobTitle, Location and more ..."<ref>[https://cloud.ibm.com/docs/services/natural-language-understanding?topic=natural-language-understanding-entity-types-version-2&locale=en Entity types (Version 2)]</ref>
== Microsoft Azure Cognitive Services ==
[https://docs.microsoft.com/zh-tw/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-entity-linking 搭配文字分析 API 使用實體辨識 - Azure Cognitive Services | Microsoft Docs] / [https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-entity-linking?tabs=version-3 Use entity recognition with the Text Analytics API - Azure Cognitive Services | Microsoft Docs]
* license
* language support: English & Chinese. See details on [https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/language-support?tabs=named-entity-recognition Language support - Text Analytics API - Azure Cognitive Services | Microsoft Docs].
* programming language: The language if supports sending a REST API request. See details on [https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-entity-linking?tabs=version-3#sending-a-rest-api-request Use entity recognition with the Text Analytics API - Azure Cognitive Services | Microsoft Docs]
* Score: Available.
* classes of entity: Person, PersonType, Location, Organization, Event, Product and more. See details on [https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/named-entity-types?tabs=general Supported Categories for Named Entity Recognition - Azure Cognitive Services | Microsoft Docs].
== MONPA ==
[https://github.com/monpa-team/monpa monpa-team/monpa: MONPA 罔拍是一個提供正體中文斷詞、詞性標註以及命名實體辨識的多任務模型]
* License: CC-BY-NC-SA 4.0 License
* language support: Chinese
* programming language: Python
* Score: Available at [https://github.com/monpa-team/monpa monpa-team/monpa github]
* classes of entity: PER ...
== spaCy ==
[https://spacy.io/ spaCy · Industrial-strength Natural Language Processing in Python]
* license: [https://github.com/explosion/spaCy/blob/master/LICENSE MIT License] {{Gd}}
* language support:
* programming language: Python
* Score:
* classes of entity: "PERSON, NORP, FAC, ORG, GPE, LOC, PRODUCT, EVENT, WORK_OF_ART, LAW, LANGUAGE, DATE, TIME, PERCENT, MONEY, QUANTITY, ORDINAL and CARDINAL <ref>[https://spacy.io/api/annotation#named-entities Annotation Specifications · spaCy API Documentation]</ref>"
== Natural Language Toolkit (NLTK) ne_chunk classifier ==
[https://www.nltk.org/ NLTK :: Natural Language Toolkit]
NE Type Examples<ref>[https://www.nltk.org/book/ch07.html 7. Extracting Information from Text]: 5  Named Entity Recognition
</ref>
* ORGANIZATION Georgia-Pacific Corp., WHO
* PERSON Eddy Bonte, President Obama
* LOCATION Murray River, Mount Everest
* DATE June, 2008-06-29
* TIME two fifty a m, 1:30 p.m.
* MONEY 175 million Canadian Dollars, GBP 10.40
* PERCENT twenty pct, 18.75 %
* FACILITY Washington Monument, Stonehenge
* GPE South East Asia, Midlothian
== Stanford CoreNLP ==
[https://stanfordnlp.github.io/CoreNLP/index.html Stanford CoreNLP – Natural language software | Stanford CoreNLP]
* license: GNU General Public License v3 {{Gd}}
* language support: English, Chinese ..
* programming language: Java
* Score: Available
* classes of entity: "For English, by default, this annotator recognizes named (PERSON, LOCATION, ORGANIZATION, MISC), numerical (MONEY, NUMBER, ORDINAL, PERCENT), and temporal (DATE, TIME, DURATION, SET) entities (12 classes). <ref>[https://stanfordnlp.github.io/CoreNLP/ner.html#description Named Entity Recognition – NERClassifierCombiner | Stanford CoreNLP]</ref>"


== 卓騰語言科技中文斷詞 ==
== 卓騰語言科技中文斷詞 ==
Line 137: Line 198:
* Score:
* Score:
* classes of entity: "person, location, time, measurement and more ... <ref>[https://api.droidtown.co/document/ 卓騰語言科技中文斷詞 API]</ref>"
* classes of entity: "person, location, time, measurement and more ... <ref>[https://api.droidtown.co/document/ 卓騰語言科技中文斷詞 API]</ref>"
== 百度AI开放平台 ==
[https://ai.baidu.com/tech/nlp 语言处理基础技术-百度AI开放平台] "专名识别"<ref>[https://ai.baidu.com/docs#/NLP-Basic-API/63eec4cf 词法分析接口]</ref>
* license:
* language support: simplified Chinese
* programming language: multiple
* Score:
* classes of entity:
<table border="1" class="wikitable sortable">
<tr><th>Class name in English (缩略词)</th><th>Class name in Simplified Chinese</th><th>Class name in Traditional Chinese</th></tr>
<tr><td>PER</td><td>人名</td><td>人名</td></tr>
<tr><td>LOC</td><td>地名</td><td>地名</td></tr>
<tr><td>ORG</td><td>机构名</td><td>機構名</td></tr>
<tr><td>TIME</td><td>时间</td><td>時間</td></tr>
</table>




Line 173: Line 219:
</table>
</table>


== other NER tools ==
 
* [https://www.diffbot.com/dev/docs/article/ Article Extraction API Documentation - Diffbot] "Array of tags/entities, generated from analysis of the extracted text and cross-referenced with DBpedia and other data sources. Language-specific tags will be returned if the source text is in English, Chinese, French, German, Spanish or Russian."
 
* [https://docs.microsoft.com/zh-tw/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-entity-linking 搭配文字分析 API 使用實體辨識 - Azure Cognitive Services | Microsoft Docs]
== Other similar NER tools ==
* ''$'' [https://www.diffbot.com/ Diffbot]: [https://www.diffbot.com/dev/docs/article/ Article Extraction API Documentation - Diffbot] "Array of tags/entities, generated from analysis of the extracted text and cross-referenced with DBpedia and other data sources. Language-specific tags will be returned if the source text is in English, Chinese, French, German, Spanish or Russian."
 
* [https://products.wolframalpha.com/api/ Wolfram|Alpha APIs: Computational Knowledge Integration] "Wolfram|Alpha makes numerous assumptions when analyzing a query and deciding how to present its results. A simple example is a word that can refer to multiple things, like "pi", which is a well-known mathematical constant but is also the name of a movie." [https://products.wolframalpha.com/docs/WolframAlpha-API-Reference.pdf]
 
* [https://www.wikidata.org/wiki/Wikidata:Data_access Wikidata:Data access - Wikidata]
 
* [https://api.duckduckgo.com/api DuckDuckGo 即時答案 API]


== References ==
== References ==
Retrieved from "https://wiki.planetoid.info/index.php/Special:MobileDiff/22027...25337"

Navigation menu