Documentation

A code unit is a 16-bit integer from which UTF-16 encoded text is built up. The Text type is represented as a UTF-16 string.

An index into the raw UTF-16 data of a Text. This is not the code point index as conventionally accepted by Text, so we wrap it to avoid confusing the two. Incorrect index manipulation can lead to surrogate pairs being sliced, so manipulate indices with care. This type is also used for lengths.

Return the length of the text, in number of code units.

Lowercase each individual code unit of a text without changing their index. This is not a proper case folding, but it does ensure that indices into the lowercased string correspond to indices into the original string.

Differences from toLower include code points in the BMP that lowercase to multiple code points, and code points outside of the BMP.

For example, İ (U+0130), which toLower converts to "i" (U+0069, U+0307), is converted into U+0069 only by lowerUtf16. Also, 𑢢 (U+118A2), a code point from the Warang City writing system in the Supplementary Multilingual Plane, introduced in 2014 to Unicode 7. It would be lowercased to U+118C2 by toLower, but it is left untouched by lowerUtf16.

Convert CodeUnits that represent a character on their own (i.e. that are not part of a surrogate pair) to their lower case representation.

This function has a special code path for ASCII characters, because Char.toLower is **incredibly** slow. It's implemented there if you want to see for yourself: (https:/github.comghcghcblobghc-8.6.3-releaselibrariesbasecbits/WCsubst.c#L4732) (It does a binary search on 1276 casing rules)

Return whether text is the same lowercase as uppercase, such that this function will not return true when Aho–Corasick would differentiate when doing case-insensitive matching.

Return a Text as a list of UTF-16 code units.

The complement of unsafeSliceUtf16: removes the slice, and returns the part before and after. See unsafeSliceUtf16 for details.

Extract a substring from a text, at a code unit offset and length. This is similar to `Text.take length . Text.drop begin`, except that the begin and length are in code *units*, not code points, so we can slice the UTF-16 array, and we don't have to walk the entire text to take surrogate pairs into account. It is the responsibility of the user to not slice surrogate pairs, and to ensure that the length is within bounds, hence this function is unsafe.

Return the code unit (not character) with the given index. Note: The boudns are not checked.