| Portability | GHC | 
|---|---|
| Stability | experimental | 
| Maintainer | bos@serpentine.com, rtomharper@googlemail.com, duncan@haskell.org | 
| Safe Haskell | Trustworthy | 
Data.Text
Contents
Description
A time and space-efficient implementation of Unicode text. Suitable for performance critical use, both in terms of large data quantities and high speed.
Note: Read below the synopsis for important notes on the use of this module.
This module is intended to be imported qualified, to avoid name
 clashes with Prelude functions, e.g.
import qualified Data.Text as T
To use an extended and very rich family of functions for working
 with Unicode text (including normalization, regular expressions,
 non-standard encodings, text breaking, and locales), see the
 text-icu package: http://hackage.haskell.org/package/text-icu
- data Text
- pack :: String -> Text
- unpack :: Text -> String
- singleton :: Char -> Text
- empty :: Text
- cons :: Char -> Text -> Text
- snoc :: Text -> Char -> Text
- append :: Text -> Text -> Text
- uncons :: Text -> Maybe (Char, Text)
- head :: Text -> Char
- last :: Text -> Char
- tail :: Text -> Text
- init :: Text -> Text
- null :: Text -> Bool
- length :: Text -> Int
- compareLength :: Text -> Int -> Ordering
- map :: (Char -> Char) -> Text -> Text
- intercalate :: Text -> [Text] -> Text
- intersperse :: Char -> Text -> Text
- transpose :: [Text] -> [Text]
- reverse :: Text -> Text
- replace :: Text -> Text -> Text -> Text
- toCaseFold :: Text -> Text
- toLower :: Text -> Text
- toUpper :: Text -> Text
- justifyLeft :: Int -> Char -> Text -> Text
- justifyRight :: Int -> Char -> Text -> Text
- center :: Int -> Char -> Text -> Text
- foldl :: (a -> Char -> a) -> a -> Text -> a
- foldl' :: (a -> Char -> a) -> a -> Text -> a
- foldl1 :: (Char -> Char -> Char) -> Text -> Char
- foldl1' :: (Char -> Char -> Char) -> Text -> Char
- foldr :: (Char -> a -> a) -> a -> Text -> a
- foldr1 :: (Char -> Char -> Char) -> Text -> Char
- concat :: [Text] -> Text
- concatMap :: (Char -> Text) -> Text -> Text
- any :: (Char -> Bool) -> Text -> Bool
- all :: (Char -> Bool) -> Text -> Bool
- maximum :: Text -> Char
- minimum :: Text -> Char
- scanl :: (Char -> Char -> Char) -> Char -> Text -> Text
- scanl1 :: (Char -> Char -> Char) -> Text -> Text
- scanr :: (Char -> Char -> Char) -> Char -> Text -> Text
- scanr1 :: (Char -> Char -> Char) -> Text -> Text
- mapAccumL :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)
- mapAccumR :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)
- replicate :: Int -> Text -> Text
- unfoldr :: (a -> Maybe (Char, a)) -> a -> Text
- unfoldrN :: Int -> (a -> Maybe (Char, a)) -> a -> Text
- take :: Int -> Text -> Text
- drop :: Int -> Text -> Text
- takeWhile :: (Char -> Bool) -> Text -> Text
- dropWhile :: (Char -> Bool) -> Text -> Text
- dropWhileEnd :: (Char -> Bool) -> Text -> Text
- dropAround :: (Char -> Bool) -> Text -> Text
- strip :: Text -> Text
- stripStart :: Text -> Text
- stripEnd :: Text -> Text
- splitAt :: Int -> Text -> (Text, Text)
- breakOn :: Text -> Text -> (Text, Text)
- breakOnEnd :: Text -> Text -> (Text, Text)
- break :: (Char -> Bool) -> Text -> (Text, Text)
- span :: (Char -> Bool) -> Text -> (Text, Text)
- group :: Text -> [Text]
- groupBy :: (Char -> Char -> Bool) -> Text -> [Text]
- inits :: Text -> [Text]
- tails :: Text -> [Text]
- splitOn :: Text -> Text -> [Text]
- split :: (Char -> Bool) -> Text -> [Text]
- chunksOf :: Int -> Text -> [Text]
- lines :: Text -> [Text]
- words :: Text -> [Text]
- unlines :: [Text] -> Text
- unwords :: [Text] -> Text
- isPrefixOf :: Text -> Text -> Bool
- isSuffixOf :: Text -> Text -> Bool
- isInfixOf :: Text -> Text -> Bool
- stripPrefix :: Text -> Text -> Maybe Text
- stripSuffix :: Text -> Text -> Maybe Text
- commonPrefixes :: Text -> Text -> Maybe (Text, Text, Text)
- filter :: (Char -> Bool) -> Text -> Text
- breakOnAll :: Text -> Text -> [(Text, Text)]
- find :: (Char -> Bool) -> Text -> Maybe Char
- partition :: (Char -> Bool) -> Text -> (Text, Text)
- index :: Text -> Int -> Char
- findIndex :: (Char -> Bool) -> Text -> Maybe Int
- count :: Text -> Text -> Int
- zip :: Text -> Text -> [(Char, Char)]
- zipWith :: (Char -> Char -> Char) -> Text -> Text -> Text
- copy :: Text -> Text
Strict vs lazy types
This package provides both strict and lazy Text types.  The
 strict type is provided by the Text package, while the lazy
 type is provided by the Lazy package.  Internally, the
 lazy Text type consists of a list of strict chunks.
The strict Text type requires that an entire string fit into
 memory at once.  The lazy Text type is capable of streaming
 strings that are larger than memory using a small memory footprint.
 In many cases, the overhead of chunked streaming makes the lazy
 Text type slower than its strict counterpart, but this is not
 always the case.  Sometimes, the time complexity of a function in
 one module may be different from the other, due to their differing
 internal structures.
Each module provides an almost identical API, with the main
 difference being that the strict module uses Int values for
 lengths and counts, while the lazy module uses Int64 lengths.
Acceptable data
A Text value is a sequence of Unicode scalar values, as defined
 in §3.9, definition D76 of the Unicode 5.2 standard:
 http://www.unicode.org/versions/Unicode5.2.0/ch03.pdf#page=35. As
 such, a Text cannot contain values in the range U+D800 to U+DFFF
 inclusive. Haskell implementations admit all Unicode code points
 (§3.4, definition D10) as Char values, including code points
 from this invalid range.  This means that there are some Char
 values that are not valid Unicode scalar values, and the functions
 in this module must handle those cases.
Within this module, many functions construct a Text from one or
 more Char values. Those functions will substitute Char values
 that are not valid Unicode scalar values with the replacement
 character "�" (U+FFFD).  Functions that perform this
 inspection and replacement are documented with the phrase
 "Performs replacement on invalid scalar values".
(One reason for this policy of replacement is that internally, a
 Text value is represented as packed UTF-16 data. Values in the
 range U+D800 through U+DFFF are used by UTF-16 to denote surrogate
 code points, and so cannot be represented. The functions replace
 invalid scalar values, instead of dropping them, as a security
 measure. For details, see Unicode Technical Report 36, §3.5:
 http://unicode.org/reports/tr36/#Deletion_of_Noncharacters)
Fusion
Most of the functions in this module are subject to fusion,
 meaning that a pipeline of such functions will usually allocate at
 most one Text value.
As an example, consider the following pipeline:
import Data.Text as T import Data.Text.Encoding as E import Data.ByteString (ByteString) countChars :: ByteString -> Int countChars = T.length . T.toUpper . E.decodeUtf8
From the type signatures involved, this looks like it should
 allocate one ByteString value, and two Text values. However,
 when a module is compiled with optimisation enabled under GHC, the
 two intermediate Text values will be optimised away, and the
 function will be compiled down to a single loop over the source
 ByteString.
Functions that can be fused by the compiler are documented with the phrase "Subject to fusion".
Types
A space efficient, packed, unboxed Unicode text type.
Creation and elimination
singleton :: Char -> TextSource
O(1) Convert a character into a Text. Subject to fusion. Performs replacement on invalid scalar values.
Basic interface
cons :: Char -> Text -> TextSource
O(n) Adds a character to the front of a Text.  This function
 is more costly than its List counterpart because it requires
 copying a new array.  Subject to fusion.  Performs replacement on
 invalid scalar values.
snoc :: Text -> Char -> TextSource
O(n) Adds a character to the end of a Text.  This copies the
 entire array in the process, unless fused.  Subject to fusion.
 Performs replacement on invalid scalar values.
O(1) Returns the first character of a Text, which must be
 non-empty.  Subject to fusion.
O(1) Returns the last character of a Text, which must be
 non-empty.  Subject to fusion.
O(1) Returns all characters after the head of a Text, which
 must be non-empty.  Subject to fusion.
O(1) Returns all but the last character of a Text, which must
 be non-empty.  Subject to fusion.
compareLength :: Text -> Int -> OrderingSource
Transformations
intercalate :: Text -> [Text] -> TextSource
O(n) The intercalate function takes a Text and a list of
 Texts and concatenates the list after interspersing the first
 argument between each element of the list.
intersperse :: Char -> Text -> TextSource
O(n) The intersperse function takes a character and places it
 between the characters of a Text.  Subject to fusion.  Performs
 replacement on invalid scalar values.
O(m+n) Replace every occurrence of one substring with another.
In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).
Case conversion
When case converting Text values, do not use combinators like
 map toUpper to case convert each character of a string
 individually, as this gives incorrect results according to the
 rules of some writing systems.  The whole-string case conversion
 functions from this module, such as toUpper, obey the correct
 case conversion rules.  As a result, these functions may map one
 input character to two or three output characters. For examples,
 see the documentation of each function.
Note: In some languages, case conversion is a locale- and
 context-dependent operation. The case conversion functions in this
 module are not locale sensitive. Programs that require locale
 sensitivity should use appropriate versions of the case mapping
 functions from the text-icu package:
 http://hackage.haskell.org/package/text-icu
toCaseFold :: Text -> TextSource
O(n) Convert a string to folded case. This function is mainly useful for performing caseless (also known as case insensitive) string comparisons.
A string x is a caseless match for a string y if and only if:
toCaseFold x == toCaseFold y
The result string may be longer than the input string, and may
 differ from applying toLower to the input string.  For instance,
 the Armenian small ligature "ﬓ" (men now, U+FB13) is case
 folded to the sequence "մ" (men, U+0574) followed by
 "ն" (now, U+0576), while the Greek "µ" (micro sign,
 U+00B5) is case folded to "μ" (small letter mu, U+03BC)
 instead of itself.
O(n) Convert a string to lower case, using simple case conversion. The result string may be longer than the input string. For instance, "İ" (Latin capital letter I with dot above, U+0130) maps to the sequence "i" (Latin small letter i, U+0069) followed by " ̇" (combining dot above, U+0307).
O(n) Convert a string to upper case, using simple case conversion. The result string may be longer than the input string. For instance, the German "ß" (eszett, U+00DF) maps to the two-letter sequence "SS".
Justification
justifyLeft :: Int -> Char -> Text -> TextSource
O(n) Left-justify a string to the given length, using the specified fill character on the right. Subject to fusion. Performs replacement on invalid scalar values.
Examples:
justifyLeft 7 'x' "foo" == "fooxxxx" justifyLeft 3 'x' "foobar" == "foobar"
justifyRight :: Int -> Char -> Text -> TextSource
O(n) Right-justify a string to the given length, using the specified fill character on the left. Performs replacement on invalid scalar values.
Examples:
justifyRight 7 'x' "bar" == "xxxxbar" justifyRight 3 'x' "foobar" == "foobar"
center :: Int -> Char -> Text -> TextSource
O(n) Center a string to the given length, using the specified fill character on either side. Performs replacement on invalid scalar values.
Examples:
center 8 'x' "HS" = "xxxHSxxx"
Folds
foldl' :: (a -> Char -> a) -> a -> Text -> aSource
O(n) A strict version of foldl.  Subject to fusion.
foldl1' :: (Char -> Char -> Char) -> Text -> CharSource
O(n) A strict version of foldl1.  Subject to fusion.
Special folds
Construction
Scans
Accumulating maps
mapAccumR :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)Source
The mapAccumR function behaves like a combination of map and
 a strict foldr; it applies a function to each element of a
 Text, passing an accumulating parameter from right to left, and
 returning a final value of this accumulator together with the new
 Text.
 Performs replacement on invalid scalar values.
Generation and unfolding
unfoldr :: (a -> Maybe (Char, a)) -> a -> TextSource
O(n), where n is the length of the result. The unfoldr
 function is analogous to the List unfoldr. unfoldr builds a
 Text from a seed value. The function takes the element and
 returns Nothing if it is done producing the Text, otherwise
 Just (a,b).  In this case, a is the next Char in the
 string, and b is the seed value for further production. Subject
 to fusion.  Performs replacement on invalid scalar values.
unfoldrN :: Int -> (a -> Maybe (Char, a)) -> a -> TextSource
O(n) Like unfoldr, unfoldrN builds a Text from a seed
 value. However, the length of the result should be limited by the
 first argument to unfoldrN. This function is more efficient than
 unfoldr when the maximum length of the result is known and
 correct, otherwise its performance is similar to unfoldr. Subject
 to fusion.  Performs replacement on invalid scalar values.
Substrings
Breaking strings
dropWhileEnd :: (Char -> Bool) -> Text -> TextSource
O(n) dropWhileEnd p t returns the prefix remaining after
 dropping characters that fail the predicate p from the end of
 t.  Subject to fusion.
 Examples:
dropWhileEnd (=='.') "foo..." == "foo"
dropAround :: (Char -> Bool) -> Text -> TextSource
O(n) dropAround p t returns the substring remaining after
 dropping characters that fail the predicate p from both the
 beginning and end of t.  Subject to fusion.
O(n) Remove leading and trailing white space from a string. Equivalent to:
dropAround isSpace
stripStart :: Text -> TextSource
O(n) Remove leading white space from a string. Equivalent to:
dropWhile isSpace
stripEnd :: Text -> TextSource
O(n) Remove trailing white space from a string. Equivalent to:
dropWhileEnd isSpace
breakOn :: Text -> Text -> (Text, Text)Source
O(n+m) Find the first instance of needle (which must be
 non-null) in haystack.  The first element of the returned tuple
 is the prefix of haystack before needle is matched.  The second
 is the remainder of haystack, starting with the match.
Examples:
 breakOn "::" "a::b::c" ==> ("a", "::b::c")
 breakOn "/" "foobar"   ==> ("foobar", "")
Laws:
append prefix match == haystack where (prefix, match) = breakOn needle haystack
If you need to break a string by a substring repeatedly (e.g. you
 want to break on every instance of a substring), use breakOnAll
 instead, as it has lower startup overhead.
In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).
breakOnEnd :: Text -> Text -> (Text, Text)Source
O(n+m) Similar to breakOn, but searches from the end of the
 string.
The first element of the returned tuple is the prefix of haystack
 up to and including the last match of needle.  The second is the
 remainder of haystack, following the match.
 breakOnEnd "::" "a::b::c" ==> ("a::b::", "c")
span :: (Char -> Bool) -> Text -> (Text, Text)Source
O(n) span, applied to a predicate p and text t, returns
 a pair whose first element is the longest prefix (possibly empty)
 of t of elements that satisfy p, and whose second is the
 remainder of the list.
groupBy :: (Char -> Char -> Bool) -> Text -> [Text]Source
O(n) Group characters in a string according to a predicate.
Breaking into many substrings
Splitting functions in this library do not perform character-wise
 copies to create substrings; they just construct new Texts that
 are slices of the original.
splitOn :: Text -> Text -> [Text]Source
O(m+n) Break a Text into pieces separated by the first
 Text argument, consuming the delimiter. An empty delimiter is
 invalid, and will cause an error to be raised.
Examples:
splitOn "\r\n" "a\r\nb\r\nd\r\ne" == ["a","b","d","e"] splitOn "aaa" "aaaXaaaXaaaXaaa" == ["","X","X","X",""] splitOn "x" "x" == ["",""]
and
intercalate s . splitOn s == id splitOn (singleton c) == split (==c)
In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).
split :: (Char -> Bool) -> Text -> [Text]Source
O(n) Splits a Text into components delimited by separators,
 where the predicate returns True for a separator element.  The
 resulting components do not contain the separators.  Two adjacent
 separators result in an empty component in the output.  eg.
split (=='a') "aabbaca" == ["","","bb","c",""] split (=='a') "" == [""]
chunksOf :: Int -> Text -> [Text]Source
O(n) Splits a Text into components of length k.  The last
 element may be shorter than the other chunks, depending on the
 length of the input. Examples:
chunksOf 3 "foobarbaz" == ["foo","bar","baz"] chunksOf 4 "haskell.org" == ["hask","ell.","org"]
Breaking into lines and words
Predicates
isPrefixOf :: Text -> Text -> BoolSource
O(n) The isPrefixOf function takes two Texts and returns
 True iff the first is a prefix of the second.  Subject to fusion.
isSuffixOf :: Text -> Text -> BoolSource
O(n) The isSuffixOf function takes two Texts and returns
 True iff the first is a suffix of the second.
View patterns
stripPrefix :: Text -> Text -> Maybe TextSource
O(n) Return the suffix of the second string if its prefix matches the entire first string.
Examples:
stripPrefix "foo" "foobar" == Just "bar" stripPrefix "" "baz" == Just "baz" stripPrefix "foo" "quux" == Nothing
This is particularly useful with the ViewPatterns extension to
 GHC, as follows:
 {-# LANGUAGE ViewPatterns #-}
 import Data.Text as T
 fnordLength :: Text -> Int
 fnordLength (stripPrefix "fnord" -> Just suf) = T.length suf
 fnordLength _                                 = -1
stripSuffix :: Text -> Text -> Maybe TextSource
O(n) Return the prefix of the second string if its suffix matches the entire first string.
Examples:
stripSuffix "bar" "foobar" == Just "foo" stripSuffix "" "baz" == Just "baz" stripSuffix "foo" "quux" == Nothing
This is particularly useful with the ViewPatterns extension to
 GHC, as follows:
 {-# LANGUAGE ViewPatterns #-}
 import Data.Text as T
 quuxLength :: Text -> Int
 quuxLength (stripSuffix "quux" -> Just pre) = T.length pre
 quuxLength _                                = -1
commonPrefixes :: Text -> Text -> Maybe (Text, Text, Text)Source
O(n) Find the longest non-empty common prefix of two strings and return it, along with the suffixes of each string at which they no longer match.
If the strings do not have a common prefix or either one is empty,
 this function returns Nothing.
Examples:
 commonPrefixes "foobar" "fooquux" == Just ("foo","bar","quux")
 commonPrefixes "veeble" "fetzer"  == Nothing
 commonPrefixes "" "baz"           == Nothing
Searching
O(n+m) Find all non-overlapping instances of needle in
 haystack.  Each element of the returned list consists of a pair:
- The entire string prior to the kth match (i.e. the prefix)
- The kth match, followed by the remainder of the string
Examples:
 breakOnAll "::" ""
 ==> []
 breakOnAll "/" "a/b/c/"
 ==> [("a", "/b/c/"), ("a/b", "/c/"), ("a/b/c", "/")]
In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).
The needle parameter may not be empty.
Indexing
If you think of a Text value as an array of Char values (which
 it is not), you run the risk of writing inefficient code.
An idiom that is common in some languages is to find the numeric
 offset of a character or substring, then use that number to split
 or trim the searched string.  With a Text value, this approach
 would require two O(n) operations: one to perform the search, and
 one to operate from wherever the search ended.
For example, suppose you have a string that you want to split on
 the substring "::", such as "foo::bar::quux". Instead of
 searching for the index of "::" and taking the substrings
 before and after that index, you would instead use breakOnAll "::".
Zipping and unzipping
Low level operations
O(n) Make a distinct copy of the given string, sharing no storage with the original string.
As an example, suppose you read a large string, of which you need
 only a small portion.  If you do not use copy, the entire original
 array will be kept alive in memory by the smaller string. Making a
 copy "breaks the link" to the original array, allowing it to be
 garbage collected if there are no other live references to it.