co-log-concurrent: Asynchronous backend for co-log library

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain] [Publish]

Buiding block for writing asynchronous logger pipelines.


[Skip to Readme]

Properties

Versions 0.4.0.0, 0.5.0.0, 0.5.0.0, 0.5.1.0
Change log CHANGELOG.markdown
Dependencies base (>=4.10.1.0 && <4.16), co-log-core (>=0.2.1.0 && <0.3), stm (>=2.4 && <2.6) [details]
License MPL-2.0
Copyright 2018-2020 Kowainik, 2020 Alexander Vershilov
Author Alexander Vershilov
Maintainer alexander.vershilov@gmail.com
Category Logging, Contravariant, Comonad
Home page https://github.com/qnikst/co-log-concurrent/
Bug tracker https://github.com/qnikst/co-log-concurrent/issues
Source repo head: git clone https://github.com/qnikst/co-log-concurrent.git
Uploaded by AlexanderVershilov at 2020-04-25T16:27:59Z

Modules

[Index] [Quick Jump]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


Readme for co-log-concurrent-0.5.0.0

[back to package description]

Hackage Stackage LTS Stackage Nightly MPL-2.0 license Build status

co-log-concurrent

co-log-concurrent is an asynchronous backend for the co-log library. The core idea of co-log-concurrent is that you may easily make your logs asynchronous by adding a few lines of code to the program that is using co-log. But we design this library in a way that you can easily inject it into your logs pipeline, in case if you need some particular functionality, and the library does not prevent you from doing so.

When do you need co-log-concurrent?

In some applications storing logs may become a bottleneck, it happens because if logs are synchronous, then before writing a log thread must take a lock on the resource. It means that we serialize the processing of the messages and introduce additional contention. Asynchronous logging can improve the situation, a thread emits logs and knows that it logs are written at some future point of time.

There are a few kinds of applications that may benefit from asynchronous logs writing: CPU intensive applications Web services In general asynchronous logging has some downsides.

  1. Unbounded memory usage - if there is no backpressure mechanism the user threads, threads may generate more logs that can we can store at the same amount of time. In such cases messages are accumulated in memory. It extends GC times and memory usage.
  2. Persistence requirements - sometimes application may want to ensure that we persisted the logs before it moved to the next statement. It is not a case with concurrent log systems in general; some we lose logs even the thread moves forward. It may happen when the application exits before dumping all logs.
  3. Non-precise logging - sometimes there may be anomalies when storing logs, such as logs reordering or imprecise timestamps.

co-log-concurrent provides a framework that you can use to have precisely the properties you need. But you still need to carefully think if a violation of the properties may harm your application.

How to use?

For a general description of the co-log framework refer to the co-log documentation it's always up to date with the latest library version. In this tutorial, we concentrate on the co-log-concurrent alone.

Simple case

You should use simple API if you don't need anything special and want the library to work. Simple API provides the following defaults: There is a backpressure mechanism: a thread is blocked if there are too many pending messages. Messages event from the different threads is never reordered. Messages may be lost if the program is abnormally killed (e.g. using sigKILL), but the library does it's best effort to dump all logs in all other cases.

To use simple API, you should wrap your program with withBackgroundWorker function. It takes the following parameters:

withBackgroundLogger
   (defCapacity :: Capacity)
   (logDumper :: LoggerAction IO a)
   (program :: IO a)

where defCapacity is the size of the buffer for not yet stored messages, it acts as a backpressure mechanism protecting your program from memory overflows. logDumper an action to store logs, you can use simple Colog.IO.logStringStdout or any other Logger function program is your ordinary program. So the skeleton for your application may look like:

-- usual imports:
import Colog.Actions
import Colog.Monad

import Colog.Concurrent -- import of the library

main :: IO ()
main =
  withBackgroundLogger
    defCapacity
    (pure ())
    logByteStringStdout $ \log ->
      usingLoggerT log $ do
        logMsg @ByteString "Starting application..."
        ...
        logMsg @ByteString "Finishing application..."

This approach is enough for most of the cases, and we try to keep this API fast and safe.

Advanced usage

Sometime your application may need some advanced features, in such a case, you need to know how to use co-log-concurrent for constructing log pipelines. Please refer to the haddocks for that.

Other implementations.

Comparison with other libraries may be outdated, as other libraries are improving and may have solved the issued described below.