seonbi: SmartyPants for Korean language

[ lgpl, library, program, text ] [ Propose Tags ]

Please see the README.md on GitHub at https://github.com/dahlia/seonbi.


[Skip to Readme]

Modules

[Last Documentation]

  • Text
    • Seonbi
      • Text.Seonbi.ContentTypes
      • Text.Seonbi.Facade
      • Text.Seonbi.Hangul
      • Text.Seonbi.Hanja
      • Text.Seonbi.Html
        • Text.Seonbi.Html.Clipper
        • Text.Seonbi.Html.Entity
        • Text.Seonbi.Html.Lang
        • Text.Seonbi.Html.Preservation
        • Text.Seonbi.Html.Printer
        • Text.Seonbi.Html.Scanner
        • Text.Seonbi.Html.Tag
        • Text.Seonbi.Html.TagStack
        • Text.Seonbi.Html.TextNormalizer
        • Text.Seonbi.Html.Wrapper
      • Text.Seonbi.PairedTransformer
      • Text.Seonbi.Punctuation
      • Text.Seonbi.Trie
      • Unihan
        • Text.Seonbi.Unihan.KHangul

Flags

Manual Flags

NameDescriptionDefault
embed-dictionary

Embed dictionary rather than load from file

Disabled
iconv

Use iconv

Disabled
static

Static link

Disabled

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 0.1.0, 0.1.1, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4
Change log CHANGES.md
Dependencies aeson (>=1.3.1 && <1.6), attoparsec (>=0.12 && <1), base (>=4.12 && <5), bytestring, bytestring-trie (>=0.2.5 && <0.3), case-insensitive (>=1 && <2), cases (>=0.1.3.2 && <0.1.5), cassava (>=0.5 && <0.6), cmark (>=0.6 && <1), containers, data-default (>=0.2 && <1), file-embed (>=0.0.10 && <0.0.16), filepath (>=1 && <2), html-charset (>=0.1 && <0.2), html-entities (>=1 && <2), http-types (>=0.12 && <0.13), optparse-applicative (>=0.14 && <0.17), seonbi, text, wai (>=3.2 && <3.4), warp (>=3.2 && <3.4) [details]
License LGPL-2.1-only
Copyright © 2018–2021 Hong Minhee
Author Hong Minhee <hong.minhee@gmail.com>
Maintainer Hong Minhee <hong.minhee@gmail.com>
Category Text
Home page https://github.com/dahlia/seonbi
Bug tracker https://github.com/dahlia/seonbi/issues
Source repo head: git clone git://github.com/dahlia/seonbi.git
Uploaded by hongminhee at 2022-09-17T18:57:49Z
Distributions
Executables seonbi-api, seonbi
Downloads 1677 total (24 in the last 30 days)
Rating 2.0 (votes: 1) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2022-09-17 [all 2 reports]

Readme for seonbi-0.3.4

[back to package description]

Seonbi: SmartyPants for Korean language

(TL;DR: See the demo web app.)

Seonbi (선비) is an HTML preprocessor that makes typographic adjustments to an HTML so that the result uses accurate punctuations according to the modern Korean orthography. (It's similar to what SmartyPants does for text written in English.)

It also transforms ko-Kore text (國漢文混用; Korean mixed script) into ko-Hang text (한글전용; Hangul-only script).

Seonbi provides a Haskell library, a CLI, and an HTTP API; any of them can perform the following transformations:

  • All hanja words (e.g., 漢字) into corresponding hangul-only words (e.g., 한자)
  • Straight quotes and apostrophes (" & ') into curly quotes HTML entities (, , , & )
  • Three consecutive periods (... or 。。。) into an ellipsis entity ()
  • Classical (Chinese-style) stops ( & ) into modern (English-style) stops (. & ,)
  • Pairs of less-than and greater-than inequality symbols (< & >) into pairs of proper angle quotes ( & )
  • Pairs of two consecutive inequality symbols (<< & >>) into pairs of proper double angle quotes ( & )
  • A hyphen (-) or hangul vowel eu () surrounded by spaces, or two/three consecutive hyphens (-- or ---) into a proper em dash ()
  • A less-than inequality symbol followed by a hyphen or an equality symbol (<-, <=) into arrows to the left (, )
  • A hyphen or an equality symbol followed by a greater-than inequality symbol (->, =>) into arrows to the right (, )
  • A hyphen or an equality symbol wrapped by inequality symbols (<->, <=>) into bi-directional arrows (, )

Each transformations can be partially turned on and off, and some transformations have many options.

All transformations work with both plain texts and rich text tree. In a similar way to SmartyPants, it does not modify characters within several sensitive HTML elements like <pre>/<code>/<script>/<kbd>. Chinese/Japanese stops or hanzi/kanji characters inside elements with lang="zh"/lang="ja"1 are never transformed.

End-user apps

Technically, Seonbi is basically exposed as a software component, which is also known as API (application programming interface), to be used as a module of other softwares.

However, as these official interfaces are not for humans but machines, it's not easy to use for end-users whom haven't experienced software programming. For such end-users, here's the list of end-user apps:

Installation

Seonbi provides official executable binaries for Linux (x86_64), macOS, and Windows (64-bit). You can download them from the releases page.

It is also distributed as a Docker image:

$ echo '訓民正音' | docker run -i dahlia/seonbi:latest seonbi
훈민정음

If you want to use it as a Haskell library install the seonbi package using Stack or Cabal.

CLI

The seonbi command basically takes the input HTML as standard input, and then transforms it into the output HTML as standard output:

seonbi < input.html > output.html

You could pass a filename as an argument instead (and it is - by default):

seonbi input.html > output.html

There is -o/--output option as well:

seonbi -o output.html input.html

Although it automatically detects text encoding of the input file, you could explicitly specify -e/--encoding:

seonbi -e euc-kr -o output.html input.html

Although there are several style options, e.g., -q/--quote, -c/--cite, -r/--render-hanja, in most cases, giving -p/--preset is enough:

echo '平壤 冷麵' | seonbi -p ko-kr  # 평양 냉면
echo '平壤 冷麵' | seonbi -p ko-kp  # 평양 랭면

Read -h/--help for details:

seonbi --help

HTTP API

The seonbi-api command starts an HTTP server that takes POST requests with an HTML source with transformation options, and responds with a transformed result HTML. You can decide a hostname and a port number with -H/--host and -p/--port options:

seonbi-api -H 0.0.0.0 -p 3800

The following is an example request:

POST / HTTP/1.1
Content-Type: application/json
Host: localhost:3800

{
  "preset": "ko-kr",
  "contentType": "text/html",
  "sourceHtml": "<p>하늘과 바람과 별과 詩</p>"
}

The HTTP API server would respond like this:

HTTP/1.1 200 OK
Content-Type: application/json
Server: Seonbi/0.3.0

{
  "success": true,
  "contentType": "text/html",
  "resultHtml": "<p>하늘과 바람과 별과 시</p>"
}

If a web app needs to use the HTTP API server, CORS should be configured through --allow-origin/-o option:

seonbi-api -o https://example.com

To learn more about parameters interactively, try the demo web app.

Haskell API

All functions and types lie inside Text.Seonbi module and its submodules. The highest-level API is Text.Seonbi.Facade module.

See also the API docs or Hackage.

Deno API

There is a simple client library for Deno as well. See also the scripts/deno/ directory.

License

Distributed under LGPL 2.1 or later.

Etymology

Seonbi (선비) means a classical scholar during Joseon periods (14c–19c). Today there's a meme that calls a person who feels morally superior or has elitism seonbi in the Korean internet. So seonbi and smarty pants have some things in common.

1

Technically, only Korean contents and language-unspecified elements are transformed. Elements having lang attribute with language tags referring to any Korean language are treated as Korean contents, e.g., ko, ko-Hang, kor-KP, kor-Kore.