llm-simple

An experimental Haskell LLM toolbox for trying out providers, tool loops, and workspace agents locally. It is meant to get you from zero to a working agent quickly — not to be a production agent platform.

The API stays small on purpose: a JSON model catalog, bundled filesystem tools, and a straight line from loadModelOrThrow to generateText. Under that surface you still get multi-provider gateways, fallbacks, streaming, structured output, and sandboxed workspace tools — enough to prototype and compare ideas without assembling the pieces yourself.

Status: early 0.1.x release. APIs may change.

Features

• Providers: OpenAI, Claude, Gemini, Ollama, DeepSeek
• Generate: single-request text, streaming, and structured object generation with model fallbacks, timeouts, retries, and throttling
• Agent: multi-round tool loops (generateText, streamText, generateObject)
• Tools: filesystem tool suite with workspace path sandboxing
• Load: JSON model catalog and API-key gateway initialization
• Observability: hooks, logging, and generation lifecycle events

Install

cabal build

Requires GHC 9.6+ with GHC2021 (see llm-simple.cabal).

Quick start

1. Create a .env file with API keys for the providers you use:

OPENAI_API_KEY=...
CLAUDE_API_KEY=...
GEMINI_API_KEY=...
DEEPSEEK_API_KEY=...

Ollama needs no API key. The library reads keys from the process environment. Load .env yourself in main (the example executable does this), or call loadGatewaysWithDotenv when building gateways manually.

2. Configure models in model-catalog.json (an example is included in the repository and source distribution).

3. Create a workspace directory for filesystem tools (the example uses ./user-workspace/).

4. Run the example executable:

cabal run llm-simple

The example loads .env, loads models from the catalog, wires up filesystem tools, and runs a small agent that asks about files in the workspace.

Configuration

Model catalog

Models are defined in a JSON array. Each entry maps a logical config name to a provider and model, plus runtime settings:

A provider is only available if its API key is set (except Ollama, which is always available).

Loading models

import Configuration.Dotenv (defaultConfig, loadFile)
import Control.Exception (SomeException, catch)
import LLM.Load (loadModelsOrThrow)

main = do
  loadFile defaultConfig `catch` \(_ :: SomeException) -> pure ()
  (gemini, deepseek) <-
    loadModelsOrThrow "./model-catalog.json" ("gemini_2_5_flash", "deepseek4flash")
  ...

loadModelsOrThrow takes a filepath and a tuple of config names, returning a tuple of ModelConfig values. Use loadModelOrThrow for a single model. On failure it throws a catchable LoadConfigError.

Usage

Agent with tools

An agent runs a loop: the model may call tools, results are fed back, and the loop continues until the model finishes or agMaxToolRounds is reached.

import Configuration.Dotenv (defaultConfig, loadFile)
import Control.Exception (SomeException, catch)
import Heptapod (generate)
import LLM.Agent (Agent (..), RuntimeArgs (..), generateText, noEventObserver)
import LLM.Core (Turn (UserTurn))
import LLM.Generate (ModelWithFallbacks (..), Hooks (..), llmHooks, noHooks)
import LLM.Load (fsTools, loadModelsOrThrow)

main :: IO ()
main = do
  loadFile defaultConfig `catch` \(_ :: SomeException) -> pure ()

  (gemini, deepseek) <-
    loadModelsOrThrow "./model-catalog.json" ("gemini_2_5_flash", "deepseek4flash")

  let models = ModelWithFallbacks { mwfModel = gemini, mwfFallbacks = [deepseek] }

  toolMap <- fsTools "./user-workspace/"
  genId <- generate

  let agent =
        Agent
          { agName = "friendly-assistant",
            agSystemPrompt = Nothing,
            agTools = ["directory_tree"],
            agMaxToolRounds = 10,
            agContextWindow = Nothing
          }
      rt =
        RuntimeArgs
          { rtGenerationId = genId,
            rtAbortSignal = Nothing,
            rtLLMHooks = llmHooks noHooks,
            rtHooks = noHooks,
            rtOnEvent = noEventObserver,
            rtReadonly = False
          }

  r <- generateText agent models toolMap rt [UserTurn "what files are here?"]
  print r

Set rtReadonly = True to omit mutating tools from the schema sent to the model and to block them at execution time.

For local debugging, defaultDebugHooks enables stderr logging and writes request/response JSON to ./dumps — avoid that in production or shared environments.

Single request (no agent loop)

For one-shot generation without a tool loop, use the Generate layer directly:

import LLM.Generate (GenRequest (..), generateTextWithFallbacks)

result <- generateTextWithFallbacks genRequest models

Also available: streamTextWithFallbacks, genObject, and genObjectUntyped.

Custom tools

Define a Tool with a ToolDef (sent to the model) and an execution function, then add it to a ToolMap. Use toTool to convert typed tool definitions. See LLM.Agent.ToolUtils and the modules under LLM.Tools.

Built-in filesystem tools

fsTools registers these tools, all scoped to a workspace root:

Use fsTools' to map tool results through a custom embedding function.

Filesystem sandbox

Filesystem tools enforce workspace-relative path containment: .. escapes, absolute paths outside the workspace, and symlinks pointing outside are rejected. Recursive tools skip symlinks rather than following them.

Limitations (0.1.x):

• The sandbox is path-scoped, not kernel-level isolation. Use a dedicated, disposable workspace directory — not your home directory or a production tree.
• A time-of-check/time-of-use (TOCTOU) window exists between path validation and file open; a concurrent process inside the workspace could theoretically plant a symlink between those steps.
• There is no denylist for sensitive in-workspace files (.env, .git/config, etc.). Anything inside the workspace is accessible to tools the model can call.
• Read tools enforce byte caps: whole-file and edit tools use a 1 MiB limit; read_file_paginated accepts source files up to 10 MiB.
• rtReadonly omits mutating tools from the model schema and blocks them at execution time if they are invoked anyway.

Module layout

Import LLM for the common surface, or import sub-modules directly for agent loops, providers, and advanced configuration. Sub-module APIs are exposed but considered experimental in 0.1.x.

Note: Workflow support was removed from this repo and moved elsewhere.

Testing

cabal test

Tests replay recorded provider responses from test/fixtures/ so they run without live API calls.

License

BSD-3-Clause. See LICENSE.