LemonWiki共筆

Count number of characters: Difference between revisions

← Older edit
mNo edit summary
Tag: wikieditor
 
(13 intermediate revisions by the same user not shown)
Line 1: Line 1:
Count number of characters in different approaches
Counting number of characters (or bytes) in different approaches


== BASH ==
== Character Count vs. Byte Count Comparison ==
<table border="1" class="wikitable sortable">
<tr>
<th>String example</th>
<th>Number of characters</th>
<th>Number of bytes</th>
</tr>
<tr>
<td>fox</td>
<td>3</td>
<td>3</td>
</tr>
<tr>
<td>The quick brown fox jumps over the lazy dog</td>
<td>43</td>
<td>43</td>
</tr>
<tr>
<td>狐</td>
<td>1</td>
<td>3</td>
</tr>
<tr>
<td>象</td>
<td>1</td>
<td>3</td>
</tr>
<tr>
<td>🐘</td>
<td>1</td>
<td>4</td>
</tr>
<tr>
<td>敏捷的棕毛狐狸從懶狗身上躍過</td>
<td>14</td>
<td>42</td>
</tr>
</table>
 
: [[Image:Owl icon.jpg]] Each Chinese character is approximately 3 bytes in UTF-8
 
Common Chinese characters (CJK Unified Ideographs, U+4E00 ~ U+9FFF) are all 3 bytes — for example, "你", "好", "狐", and "象". Characters beyond U+FFFF (outside the Basic Multilingual Plane, BMP<ref>[https://en.wikipedia.org/wiki/Plane_(Unicode) Plane (Unicode) - Wikipedia]</ref>) take 4 bytes. These are mostly rare characters from CJK Extension B (U+20000 and up)<ref>[https://en.wikipedia.org/wiki/CJK_Unified_Ideographs_Extension_B CJK Unified Ideographs Extension B - Wikipedia]</ref>, such as "[https://www.cns11643.gov.tw/wordView.jsp?ID=993142 𤆬]" (U+241AC) and "[https://www.cns11643.gov.tw/wordView.jsp?ID=402472 𠮷]" (U+20BB7).
 
== How to count characters with PHP ==
* PHP: [https://www.php.net/manual/en/function.strlen.php strlen] & [http://php.net/mb_strlen PHP mb_strlen function]
 
Number of characters
<pre>
echo mb_strlen("狐", 'UTF-8') . PHP_EOL; // return 1
echo mb_strlen("《王大文 Dawen》", 'UTF-8') . PHP_EOL; // return 11
</pre>
 
String length (number of bytes)
<pre>
echo strlen("狐") . PHP_EOL; // return 3
echo strlen("《王大文 Dawen》") . PHP_EOL; // return 21
</pre>
 
Number of words {{exclaim}} [https://www.php.net/manual/en/function.str-word-count.php str_word_count] function not support Chinese characters
<pre>
echo str_word_count("The quick brown fox jumps over the lazy dog"); // return 9
echo str_word_count("敏捷的棕毛狐狸從懶狗身上躍過"); // return 0
</pre>
 
== How to count characters with MySQL ==
* MySQL: [http://www.w3resource.com/mysql/string-functions/mysql-char_length-function.php MySQL CHAR_LENGTH() function]
<PRE>
// number of characters
SELECT CHAR_LENGTH("狐"); /* return 1 */
SELECT CHAR_LENGTH("《王大文 Dawen》"); /* return 11 */
 
// number of bytes
SELECT LENGTH("狐"); /* return 3 */
SELECT LENGTH("《王大文 Dawen》"); /* return 21 */
</PRE>
* [https://dev.mysql.com/doc/refman/8.0/en/char.html MySQL :: MySQL 8.0 Reference Manual :: 11.4.1 The CHAR and VARCHAR Types] e.g. {{kbd | key=<nowiki>VARCHAR(5)</nowiki>}} or {{kbd | key=<nowiki>CHAR(5)</nowiki>}} means can hold up to 5 characters.
 
 
== How to count characters with SQLite ==
[https://www.sqlitetutorial.net/sqlite-functions/sqlite-length/#targetText=SQLite%20Length,returns%20the%20number%20of%20bytes. Length] function
<PRE>
SELECT LENGTH("狐"); /* return 1 */
SELECT LENGTH("《王大文 Dawen》"); /* return 11 */
</PRE>
 
== How to count characters with Excel ==
* Excel: [https://support.office.com/en-us/article/len-lenb-functions-29236f94-cedc-429d-affd-b5e33d2c67cb LEN, LENB functions] / [https://support.office.com/zh-tw/article/LEN%E3%80%81LENB-%E5%87%BD%E6%95%B8-29236f94-cedc-429d-affd-b5e33d2c67cb LEN、LENB 函數] {{exclaim}} Result of the function {{kbd | key=LENB}} is not the same with the result in other programming language.
<pre>
// number of characters
=LEN("狐") // return 1
=LEN("《王大文 Dawen》") // return 11
 
// number of bytes
=LENB("狐") // return 2
=LENB("《王大文 Dawen》") // return 16
</pre>
 
* [http://string-functions.com/length.aspx Calculate String Length Online]
* [https://stackoverflow.com/questions/5290182/how-many-bytes-does-one-unicode-character-take string - How many bytes does one Unicode character take? - Stack Overflow]
 
== How to count characters with BASH ==
Step1: Using [https://www.computerhope.com/unix/uwc.htm Linux wc command]
Step1: Using [https://www.computerhope.com/unix/uwc.htm Linux wc command]
<pre>
# Count the total number of characters in a file named "input.txt", while ignoring all whitespace characters (including spaces, tabs, newlines, etc.).
tr -d '\r\n[:space:]' < input.txt | wc -m
</pre>
<pre>
<pre>
# print the character counts of txt files (contains the count of return symbol)
# print the character counts of txt files (contains the count of return symbol)
Line 21: Line 127:
Number of characters (not contains the [[Return symbol | return symbol]]) = result of {{kbd | key=<nowiki>wc -m *.txt</nowiki>}} - result of {{kbd | key=<nowiki>wc -m *.txt</nowiki>}} * 2 - 1 (the last blank line costs 1 character) - number of the whitespaces
Number of characters (not contains the [[Return symbol | return symbol]]) = result of {{kbd | key=<nowiki>wc -m *.txt</nowiki>}} - result of {{kbd | key=<nowiki>wc -m *.txt</nowiki>}} * 2 - 1 (the last blank line costs 1 character) - number of the whitespaces


[[Category:Software]] [[Category:Programming]] [[Category:Data Science]] [[Category:Text file processing]] [[Category:Data transformation]]
== How to count characters with Python ==
[[Category:Regular expression]]
Using the [https://docs.python.org/3/library/functions.html#len len()] function<ref>[https://stackoverflow.com/questions/30686701/python-get-size-of-string-in-bytes Python : Get size of string in bytes - Stack Overflow]</ref>. Try it on [https://replit.com/@planetoid/lenth-of-string#main.py replit].
 
=== Get the number of characters in a string in Python  ===
 
<pre>
string = "狐"
print(len(string))
// returns 1
</pre>
 
=== Get the number of bytes in a string in Python  ===
 
<pre>
string = "狐"
print(len(string.encode('utf-8')))
// returns 3
 
print(len(string.encode('utf-16-le')))
// returns 2
</pre>
 
 
== How to count characters with JavaScript ==
Using the [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/length length()] function and [https://developer.mozilla.org/en-US/docs/Web/API/Blob Blob] object <ref> [https://stackoverflow.com/questions/2219526/how-many-bytes-in-a-javascript-string How many bytes in a JavaScript string? - Stack Overflow]</ref>.
 
=== Get the number of characters in a string in JavaScript  ===
 
<pre>
var string = "狐"
console.log(string.length);
</pre>
 
=== Get the number of bytes in a string in JavaScript  ===
 
<pre>
var string = "狐"
console.log(new Blob([string]).size);
// returns 3
</pre>
 
== References ==
<references />
 
== Further reading ==
 
* [https://www.ithome.com.tw/voice/131688 Unicode與JavaScript字串 | iThome]
 
[[Category: Software]]  
[[Category: Programming]]  
[[Category: Data Science]]  
[[Category: String manipulation]]
[[Category: PHP]]  
[[Category: MySQL]]
[[Category: Revised with LLMs]]
Retrieved from "https://wiki.planetoid.info/index.php/Count_number_of_characters"