HTML: DOCTYPE token

DocumentType, describing the language used in the document.

HTML: start tag token

Element, marking the start of a markup section, or a point of markup which (per the specification) doesn't contain any content.

HTML: end tag token

Element with a / character before the name, marking the end of a section opened by StartTag.

HTML: comment token

Comment, marking author's notes or other text about the document itself, rather than being part of the content.

HTML: character token

Character, usually containing (a small portion of) text which should rendered for the user or included in the header metadata, but occasionally subject to further processing (i.e. the content of <script> or <style> sections).

HTML: end-of-file token

Represents both an explicit mark of the end of the stream, when a simple [] doesn't suffice, and provides a seat to carry ParseErrors if no other token is emitted at the same time.

Note: the former role doesn't have any guarantees; a stream can end without an EndOfStream termination, and EndOfStream tokens can occur in places other than the end of the file.

HTML: data state

The core rules, providing the most common tokenization behaviour.

HTML: RCDATA state

Character-focused production while, unlike RawTextState, resolving character reference values.

HTML: RAWTEXT state

Character-focused production which, unlike RCDataState, passes character reference sequences unchanged.

HTML: PLAINTEXT state

Blind conversion of the entire document stream into Character tokens.

HTML: script data state

Character-focused production according to the (occasionally complex) rules governing the handling of <script> contents.

HTML: script data escaped state

Character-focused production for data within a <!-- / --> section within ScriptDataState.

HTML: script data double escaped state

Character-focused production for data within a <script> section within ScriptDataEscapedState.

HTML: CDATA section state

Character-focused production for data within a foreign <[CDATA[ / ]]> escape section.

The UTF-8 encoding for Unicode.

The UTF-16 encoding for Unicode, in big endian order.

No encoder is provided for this scheme.

The UTF-16 encoding for Unicode, in little endian order.

No encoder is provided for this scheme.

Big5, primarily covering traditional Chinese characters.

EUC-JP, primarily covering Japanese as the union of JIS-0208 and JIS-0212.

EUC-KR, primarily covering Hangul.

The GB18030-2005 extension to GBK, with one tweak for web compatibility, primarily covering both forms of Chinese characters.

Note that this encoding also includes a large number of four-byte sequences which aren't listed in the linked visualization.

GBK, primarily covering simplified Chinese characters.

In practice, this is just Gb18030 with a restricted set of encodable characters; the decoder is identical.

DOS and OS/2 code page for Cyrillic characters.

A Japanese-focused implementation of the ISO 2022 meta-encoding, including both JIS-0208 and halfwidth katakana.

Latin-2 (Central European).

Latin-3 (South European and Esperanto)

Latin-4 (North European).

Latin/Cyrillic.

Latin/Arabic.

Latin/Greek (modern monotonic).

Latin/Hebrew (visual order).

Latin/Hebrew (logical order).

Latin-6 (Nordic).

Latin-7 (Baltic Rim).

Latin-8 (Celtic).

Latin-9 (revision of ISO 8859-1 Latin-1, Western European).

Latin-10 (South-Eastern European).

KOI-8 specialized for Russian Cyrillic.

KOI-8 specialized for Ukrainian Cyrillic.

Mac OS Roman.

Mac OS Cyrillic (as of Mac OS 9.0)

The Windows variant (code page 932) of Shift JIS.

ISO 8859-11 Latin/Thai with Windows extensions in the C1 control character slots.

Note that this encoding is always used instead of pure Latin/Thai.

The Windows extension and rearrangement of ISO 8859-2 Latin-2.

Windows Cyrillic.

The Windows extension of ISO 8859-1 Latin-1, replacing most of the C1 control characters with printable glyphs.

Note that this encoding is always used instead of pure Latin-1.

Windows Greek (modern monotonic).

The Windows extension of ISO 8859-9 Latin-5 (Turkish), replacing most of the C1 control characters with printable glyphs.

Note that this encoding is always used instead of pure Latin-5.

The Windows extension and rearrangement of ISO 8859-8 Latin/Hebrew.

Windows Arabic.

Windows Baltic.

Windows Vietnamese.

The input is reduced to a single \xFFFD replacement character.

No encoder is provided for this scheme.

Non-ASCII bytes (\x80 through \xFF) are mapped to a portion of the Unicode Private Use Area (\xF780 through \xF7FF).