ollama-haskell

Modern Haskell client library for the Ollama local LLM engine.

Features

• Client-Centric Architecture: Thread-safe OllamaClient handle with connection pooling and resource management (newClient, defaultClient, clientFromEnv, withClient).
• First-Class Streaming: conduit-based response streaming (chatStream, generateStream, pullStream, pushStream, createModelStream).
• Model Context Protocol (MCP) Bridge: Bidirectional integration with mcp-server for converting between Ollama tools and MCP tools, running MCP servers via stdio or HTTP (Ollama.MCP).
• Generic JSON Schema Derivation: Automatically derive JSON schemas from Haskell data types via GHC.Generics with ToSchema and formatFor.
• Complete API Surface: Text generation, chat completions, vector embeddings, model management (list, show, copy, delete, pull, push, create), system endpoints, and System One decision evaluation.
• System One Decision Models: Native support for /v1/systemone fast classification, binary probability (noul), and scoring (choiceQuestion, noulQuestion, scoreQuestion).
• Structured Outputs DSL: Powerful SchemaBuilder DSL (|+, |++, |!, |!!) for type-safe JSON Schema structured responses.
• Function / Tool Calling: Full support for tool definitions (Tool), tool calls (ToolCall), and execution results (toolResultMessage).
• Thinking Models & Logprobs: Native support for reasoning models (Think / ThinkingLevel with custom model-defined levels) and token log probabilities (Logprob / TokenLogprob).
• Environment & Auth Integration: Robust URL normalization for OLLAMA_HOST and bearer token support for OLLAMA_API_KEY.
• Configurable Resilience: Flexible retry policies (NoRetry, ConstantRetry, ExponentialRetry), custom timeouts, lifecycle callbacks, and structured logging.
• Conversation Store: Transactional STM-backed InMemoryStore and ConversationStore typeclass for managing multi-turn chat sessions.
• SDK Comparison Matrix: Detailed feature comparison against Python, JS/TS, and Go SDKs in COMPARISON.md.

Installation

Add ollama-haskell to your .cabal file:

build-depends:
    base >= 4.17 && < 5
  , ollama-haskell >= 0.4.2.0

Or using Stack in package.yaml:

dependencies:
  - ollama-haskell >= 0.4.2.0

Quick Start (5 Lines)

import Data.List.NonEmpty (NonEmpty ((:|)))
import Data.Text.IO qualified as TIO
import Ollama

main :: IO ()
main = do
  client <- defaultClient
  res <- chat client $ chatRequest "qwen3.5:2b" (userMessage "Why is the sky blue?" :| [])
  case res of
    Left err   -> print err
    Right resp -> mapM_ (TIO.putStrLn . messageContent) (crMessage resp)

Streaming Responses with Conduit

Stream LLM responses token-by-token in real time:

import Conduit (mapM_C, runConduit, (.|))
import Control.Monad.IO.Class (liftIO)
import Data.List.NonEmpty (NonEmpty ((:|)))
import Data.Text.IO qualified as TIO
import Ollama
import System.IO (hFlush, stdout)

main :: IO ()
main = do
  client <- defaultClient
  let req = chatRequest "qwen3.5:2b" (userMessage "Count from 1 to 5." :| [])

  -- Stream tokens to stdout as they arrive
  runConduit $
    chatStream client req .| mapM_C (\chunk -> liftIO $ do
      mapM_ (TIO.putStr . messageContent) (crMessage chunk)
      hFlush stdout
    )
  putStrLn ""

You can also accumulate all chunks at once with collectStream, or fold text with foldStream:

-- Collect all chunks:
chunks <- collectStream (chatStream client req)

-- Or fold into a single Text value:
fullText <- foldStream (\acc c -> acc <> maybe "" messageContent (crMessage c)) "" (chatStream client req)

Function & Tool Calling

Define function signatures with zero boilerplate using toolDef, objectParams, and typed parameters:

import Data.List.NonEmpty (NonEmpty ((:|)))
import qualified Data.Map.Strict as Map
import Ollama

calculatorTool :: Tool
calculatorTool =
  toolDef "add" "Add two numbers" $
    objectParams
      [ ("a", numberParam (Just "First number"))
      , ("b", numberParam (Just "Second number"))
      ]
      ["a", "b"]

main :: IO ()
main = do
  client <- defaultClient
  let req = (chatRequest "llama3.2:latest" (userMessage "What is 25 + 17?" :| []))
        { chatTools = Just [calculatorTool] }
  res <- chat client req
  case res of
    Left err   -> print err
    Right resp -> case crMessage resp of
      Just msg -> case messageToolCalls msg of
        Just (call : _) -> do
          putStrLn $ "Tool called: " <> show (tcfName (tcFunction call))
          putStrLn $ "Arguments: " <> show (tcfArguments (tcFunction call))
        _ -> putStrLn "No tool called."
      Nothing -> putStrLn "No message returned."

Multimodal Vision

Load local images directly and send them to vision models (gemma3, qwen3.5:2b, etc.):

main :: IO ()
main = do
  client <- defaultClient
  img <- readImageBase64 "chart.png"
  let msg = imageMessage "Describe the chart in 2 sentences." [img]
  res <- chat client (chatRequest "gemma3:latest" (msg :| []))
  case res of
    Right resp -> mapM_ (putStrLn . messageFullText) (crMessage resp)
    Left err   -> print err

Thinking / Reasoning Models

Control chain-of-thought tokens on reasoning models (e.g. DeepSeek-R1, Qwen 3.5):

let req = (chatRequest "qwen3.5:2b" (userMessage "Which is larger: 9.11 or 9.9?" :| []))
      { chatThink = Just ThinkEnabled } -- or ThinkDisabled, ThinkLevel ThinkHigh, ThinkCustom "..."
res <- chat client req
case res of
  Right resp -> case crMessage resp of
    Just msg -> do
      case messageThinking msg of
        Just th -> putStrLn $ "Thinking trace:\n" <> th
        Nothing -> pure ()
      putStrLn $ "Answer:\n" <> messageContent msg
    Nothing -> pure ()
  Left err -> print err

System One Decision Models

Evaluate high-speed classification, probability (noul), and scoring with /v1/systemone:

let q1 = choiceQuestion "Customer eligibility" (Map.fromList [("eligible", Just "Standard"), ("ineligible", Nothing)])
    q2 = noulQuestion "Will this cause churn?"
    req = systemOneRequest "nimble" "Customer account history..." (Map.fromList [("q1", q1), ("q2", q2)])
res <- systemOne client req

Structured Outputs (JSON Schema DSL)

Enforce structured JSON output formats using SchemaBuilder (re-exported directly from Ollama):

import Data.Text.IO qualified as TIO
import Ollama

personSchema :: Schema
personSchema = buildSchema $ emptyObject
  |+ ("name", JString)
  |+ ("age", JInteger)
  |! "name"

main :: IO ()
main = do
  client <- defaultClient
  let req = (generateRequest "qwen3.5:2b" "Generate a person profile.")
        { genFormat = Just (SchemaFormat personSchema) }
  res <- generate client req
  case res of
    Left err   -> print err
    Right resp -> TIO.putStrLn (grResponse resp)

Environment Variables & Configuration

Construct a client using environment variables (OLLAMA_HOST, OLLAMA_API_KEY):

main :: IO ()
main = do
  client <- clientFromEnv
  -- Automatically connects to OLLAMA_HOST with optional Authorization: Bearer header
  ...

Or configure custom retry policies and loggers:

customConfig :: OllamaClientConfig
customConfig = defaultConfig
  { configBaseUrl = "http://my-ollama-server:11434"
  , configTimeout = 120
  , configRetry   = ExponentialRetry 3 1000000 -- 3 retries with exponential backoff
  , configLogger  = Just (\level msg -> putStrLn $ "[" <> show level <> "] " <> show msg)
  }

main :: IO ()
main = withClient customConfig $ \client -> do
  ...

Runnable Examples

The repository includes complete, tested runnable examples for every library feature:

Run any example with Stack:

stack run ollama-example-tool-calling

Feature Matrix

Documentation & References

• Comparison Deep Dive — Detailed architectural comparison across language ecosystems.
• CONTRIBUTING.md — Development setup, testing guidelines, and code style.
• CHANGELOG.md — Release notes and version changelog.
• Hackage Documentation — Full Haddock API reference.

License

MIT © 2024–2026 Tushar Adhatrao