Merge pull request #3220 from joaquinelio/patch-19

Unicode art, grammar suggestions
This commit is contained in:
Ilya Kantor 2022-10-10 17:08:28 +02:00 committed by GitHub
commit bf7d8bb1af
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -2,7 +2,7 @@
# Unicode, String internals
```warn header="Advanced knowledge"
The section goes deeper into string internals. This knowledge will be useful for you if you plan to deal with emoji, rare mathematical or hieroglyphic characters or other rare symbols.
The section goes deeper into string internals. This knowledge will be useful for you if you plan to deal with emoji, rare mathematical or hieroglyphic characters, or other rare symbols.
```
As we already know, JavaScript strings are based on [Unicode](https://en.wikipedia.org/wiki/Unicode): each character is represented by a byte sequence of 1-4 bytes.
@ -11,11 +11,11 @@ JavaScript allows us to insert a character into a string by specifying its hexad
- `\xXX`
`XX` must be two hexadecimal digits with value between `00` and `FF`, then it's character whose Unicode code is `XX`.
`XX` must be two hexadecimal digits with a value between `00` and `FF`, then `\xXX` is the character whose Unicode code is `XX`.
Because the `\xXX` notation supports only two digits, it can be used only for the first 256 Unicode characters.
Because the `\xXX` notation supports only two hexadecimal digits, it can be used only for the first 256 Unicode characters.
These first 256 characters include latin alphabet, most basic syntax characters and some others. For example, `"\x7A"` is the same as `"z"` (Unicode `U+007A`).
These first 256 characters include the Latin alphabet, most basic syntax characters, and some others. For example, `"\x7A"` is the same as `"z"` (Unicode `U+007A`).
```js run
alert( "\x7A" ); // z
@ -23,13 +23,13 @@ JavaScript allows us to insert a character into a string by specifying its hexad
```
- `\uXXXX`
`XXXX` must be exactly 4 hex digits with the value between `0000` and `FFFF`, then `\uXXXX` is a character whose Unicode code is `XXXX` .
`XXXX` must be exactly 4 hex digits with the value between `0000` and `FFFF`, then `\uXXXX` is the character whose Unicode code is `XXXX`.
Characters with Unicode value greater than `U+FFFF` can also be represented with this notation, but in this case we will need to use a so called surrogate pair (we will talk about surrogate pairs later in this chapter).
Characters with Unicode values greater than `U+FFFF` can also be represented with this notation, but in this case, we will need to use a so called surrogate pair (we will talk about surrogate pairs later in this chapter).
```js run
alert( "\u00A9" ); // ©, the same as \xA9, using the 4-digit hex notation
alert( "\u044F" ); // я, the cyrillic alphabet letter
alert( "\u044F" ); // я, the Cyrillic alphabet letter
alert( "\u2191" ); // ↑, the arrow up symbol
```
@ -38,13 +38,13 @@ JavaScript allows us to insert a character into a string by specifying its hexad
`X…XXXXXX` must be a hexadecimal value of 1 to 6 bytes between `0` and `10FFFF` (the highest code point defined by Unicode). This notation allows us to easily represent all existing Unicode characters.
```js run
alert( "\u{20331}" ); // 佫, a rare Chinese hieroglyph (long Unicode)
alert( "\u{20331}" ); // 佫, a rare Chinese character (long Unicode)
alert( "\u{1F60D}" ); // 😍, a smiling face symbol (another long Unicode)
```
## Surrogate pairs
All frequently used characters have 2-byte codes. Letters in most european languages, numbers, and even most hieroglyphs, have a 2-byte representation.
All frequently used characters have 2-byte codes (4 hex digits). Letters in most European languages, numbers, and the basic unified CJK ideographic sets (CJK -- from Chinese, Japanese, and Korean writing systems), have a 2-byte representation.
Initially, JavaScript was based on UTF-16 encoding that only allowed 2 bytes per character. But 2 bytes only allow 65536 combinations and that's not enough for every possible symbol of Unicode.
@ -55,7 +55,7 @@ As a side effect, the length of such symbols is `2`:
```js run
alert( '𝒳'.length ); // 2, MATHEMATICAL SCRIPT CAPITAL X
alert( '😂'.length ); // 2, FACE WITH TEARS OF JOY
alert( '𩷶'.length ); // 2, a rare Chinese hieroglyph
alert( '𩷶'.length ); // 2, a rare Chinese character
```
That's because surrogate pairs did not exist at the time when JavaScript was created, and thus are not correctly processed by the language!
@ -120,7 +120,7 @@ For instance, the letter `a` can be the base character for these characters: `à
Most common "composite" characters have their own code in the Unicode table. But not all of them, because there are too many possible combinations.
To support arbitrary compositions, Unicode standard allows us to use several Unicode characters: the base character followed by one or many "mark" characters that "decorate" it.
To support arbitrary compositions, the Unicode standard allows us to use several Unicode characters: the base character followed by one or many "mark" characters that "decorate" it.
For instance, if we have `S` followed by the special "dot above" character (code `\u0307`), it is shown as Ṡ.
@ -167,6 +167,6 @@ alert( "S\u0307\u0323".normalize().length ); // 1
alert( "S\u0307\u0323".normalize() == "\u1e68" ); // true
```
In reality, this is not always the case. The reason being that the symbol `Ṩ` is "common enough", so Unicode creators included it in the main table and gave it the code.
In reality, this is not always the case. The reason is that the symbol `Ṩ` is "common enough", so Unicode creators included it in the main table and gave it the code.
If you want to learn more about normalization rules and variants -- they are described in the appendix of the Unicode standard: [Unicode Normalization Forms](https://www.unicode.org/reports/tr15/), but for most practical purposes the information from this section is enough.