Hackage :: [Package]

eros-http: JSON HTTP interface to Eros.

[ bsd3, program, text ] [ Propose Tags ]

This is a small program that runs an HTTP server.

Changelog

0.6.0.1
Remove unneeded dependency on wai-responsible.
0.6
Update to be compatible with Eros v.0.6
[Skip to Readme]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 0.5.0.0, 0.6.0.0, 0.6.0.1
Dependencies aeson (>=0.8 && <0.9), base (>=4.7 && <4.8), blaze-html (>=0.7 && <0.8), bytestring (>=0.10 && <0.11), eros (>=0.6 && <0.7), http-types (>=0.8 && <0.9), markdown (>=0.1 && <0.2), text (>=1.1 && <1.2), wai (>=3.0 && <3.1), warp (>=3.0 && <3.1) [details]
License BSD-3-Clause
Copyright 2014, Peter Harpending
Author Peter Harpending
Maintainer Peter Harpending <pharpend2@gmail.com>
Category Text
Uploaded by pharpend at 2014-07-20T10:00:10Z
Distributions
Reverse Dependencies 1 direct, 0 indirect [details]
Executables eros-http
Downloads 2403 total (10 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
Last success reported on 2015-05-19 [all 6 reports]

Readme for eros-http-0.6.0.1

[back to package description]

eros-http

This is an HTTP front-end to the Eros library. Eros is a text censorship library, that I wrote.

Usage

If the server receives a GET request, it returns an HTML representation of this file.

It takes an input string via POST, and returns some data in JSON mapping each phraselist to the score for the input string. The JSON is compressed.

I fed the server the GPL, here are the results, which have been prettified, and alphabetized by key.

{
  "chat": 0,
  "conspiracy": 0,
  "drug-advocacy": 0,
  "forums": 0,
  "gambling": 0,
  "games": 0,
  "gore": 0,
  "id-theft": 0,
  "illegal-drugs": 0,
  "intolerance": 0,
  "legal-drugs": 0,
  "malware": 0,
  "music": 0,
  "news": 0,
  "nudism": 0,
  "peer2peer": 0,
  "personals": 0,
  "pornography": 20,
  "proxies": 0,
  "secret-societies": 0,
  "self-labeling": 0,
  "sport": 30,
  "translation": 0,
  "upstream-filter": 0,
  "violence": 0,
  "warez-hacking": 0,
  "weapons": 0,
  "webmail": 0
}

The home page of PornHub yields this result:

{
  "chat": 0,
  "conspiracy": 0,
  "drug-advocacy": 0,
  "forums": 0,
  "gambling": 0,
  "games": 0,
  "gore": 0,
  "id-theft": 0,
  "illegal-drugs": 0,
  "intolerance": 0,
  "legal-drugs": 0,
  "malware": 0,
  "music": 0,
  "news": 0,
  "nudism": 0,
  "peer2peer": 0,
  "personals": 0,
  "pornography": 8790,
  "proxies": 0,
  "secret-societies": 0,
  "self-labeling": 0,
  "sport": 0,
  "translation": 0,
  "upstream-filter": 0,
  "violence": 0,
  "warez-hacking": 0,
  "weapons": 0,
  "webmail": 0
}

This is the command I used

curl www.pornhub.com | curl -d @- localhost:8000 | prettify.rb

This uses my JSON prettifier script.

Overflow bug

Edit - This bug was fixed in version 0.6.

There's a fundamental flaw in the algorithm where if it gets a lot of data with a lot of flagged phrases, it takes a long time to calculate the result. The server times out after 30 seconds. I haven't figured out how to get multiprocessing, so this will remain a bug until I do.

This won't be a problem for typical usage, only if you try to send paragraphs of dirty data in one string.

Typical usage is for SMS messages, which are at most 160 characters long.