postgresql-replicant: PostgreSQL logical streaming replication library

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain] [Publish]

Please see the README on GitHub at https://github.com/agentultra/postgres-replicant#readme


[Skip to Readme]

Properties

Versions 0.1.0.0, 0.1.0.1, 0.2.0.0, 0.2.0.1
Change log ChangeLog.md
Dependencies aeson, async, attoparsec, base (>=4.7 && <5), bits, bytestring, cereal, containers, keep-alive, postgresql-libpq, postgresql-replicant, scientific, stm, text, time [details]
License BSD-3-Clause
Copyright 2020, 2021, James King
Author James King
Maintainer james@agentultra.com
Category Experimental, Database
Home page https://github.com/agentultra/postgresql-replicant#readme
Bug tracker https://github.com/agentultra/postgresql-replicant/issues
Source repo head: git clone https://github.com/agentultra/postgresql-replicant
Uploaded by agentultra at 2021-05-02T20:14:34Z

Modules

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


Readme for postgresql-replicant-0.1.0.0

[back to package description]

postgres-replicant

A PostgreSQL streaming logical replication client for Haskell.

This library is currently EXPERIMENTAL and is not ready for production yet. Developers are encouraged to test out the library, add issues, contribute and make it awesome.

React to database changes in real-time. Enrich data, send notifications, monitor suspicious activity, etc.

Setup

In order to use a logical replication client you need to set the wal_level in Postgres to logical. You can set that, for example, in your postgres.conf file:

wal_level = logical

There is a known issue in the experimental release where the wal_sender process, if it has a timeout set, will time out a replicant client. This will be fixed in a future release. Unless you're trying to fix it, you can turn it off by setting:

wal_sender_timeout = 0

Then restart your server, log in as postgres or another super user and check:

SHOW wal_level

It should show logical.

You will also need a user who is allowed to use replication features.

Then add a database and your user with the REPLICATION trait to the database, grant all on it if you need to, etc.

We haven't added authorization support yet to the library so make sure in pg_hba.conf you trust local connections:

host    all             all             127.0.0.1/32            trust

Caveat emptor.

You will also need to install the wal2json plugin for your PostgreSQL server. For example in Ubuntu-like distributions you can just:

$ sudo apt install postgresql-12-wal2json

Assuming you have installed PostgreSQL from the package repositories.

You can then setup a basic "hello world" program using this library:

module Main where

import Control.Exception

import Database.PostgreSQL.Replicant

main :: IO ()
main = do
  let settings = PgSettings "my-user" "my-database" "localhost" "5432" "testing"
  withLogicalStream settings $ \change -> do
    print change
      `catch` \err ->
      print err

Which should connect, create a testing replication slot, and start sending your callback the changes to your database as they arrive.

WAL Output Plugins

Presently we only support the wal2json WAL output plugin and only the v1 format. Support for the v2 format is planned as are more output plugins.

Example Program

Included is a simple program, replicant-example which should help you test your connection settings and make sure that you can connect and receive WAL changes.

You can change the connection settings through environment variables:

PG_USER=myuser PG_DATABASE=mydb replicant-example

The configuration settings are:

FAQ

Why not use LISTEN/NOTIFY

You don't have to set up triggers for each table with logical replication. You also don't have the message size limitation. That limitation often forces clients to perform a query for each message to fetch the data. We only use one connection.