Add ASCII-compatible charset checking method.#87
Add ASCII-compatible charset checking method.#87mscdex wants to merge 1 commit intopillarjs:masterfrom
Conversation
It can be useful to check for ASCII-compatible charsets when decoding data that is known to only have bytes that fall within the ASCII range.
|
Interesting. Although I'm sure it can be done without additional generated file. |
|
ASCII-compatible means bytes 0x00-0x7F in an encoding are all ASCII characters/bytes and not some other characters. Many character sets are compatible in this way, but some are not. Checking whether a destination encoding is ASCII-compatible is useful if you are already traversing binary data and you can check whether each byte is <= 0x7F. If there are no bytes above 0x7F and the encoding is ASCII-compatible, you don't have to run the entire set of data through a decoder which will end up giving you back the same string anyway. The reason I use a pre-generated file for this is that it's significantly faster to use an object (with fast properties) than to do a lookup on the fly. |
|
Got it. Makes sense. |
84ee650 to
9aa082f
Compare
It can be useful to check for ASCII-compatible charsets when decoding data that is known to only have bytes that fall within the ASCII range. Doing such a check avoids having to do useless decoding.