static: Type-safe and interoperable static values and closures
Serialise closures in a type-safe way that interoperates across binaries.
This package is inspired by distributed-static
and GHC's static pointers
in GHC.StaticPtr
, which came out of the same research. However, we make
some significantly-different design choices, described below.
GHC made the design choice to focus on guaranteeing that static values could
be passed between nodes if they were running the exact same binary, since
they are indexed by 64-bit integers automatically-generated by the compiler.
distributed-static
attempts to support the same source program compiled by
different versions of GHC. As part of this effort to preserve stability, one
must pass in a table (RemoteTable
) whose keys represent the stability, and
whose values are resolved potentially differently across compiler versions.
The need for the caller to pass in a RemoteTable
is seen as a liability, so
two subsequent packages distributed-closure
and static-closure
take the
opposite approach, ripping out the RemoteTable
but doubling down on GHC's
choice to guarantee compatibility only across different processes running the
exact same binary program. Their uses cases are focused around compute
clusters and other forms of centralised distributed computing, where this is
easy to achieve and not a problem.
Sometimes security is cited as a reason to have this restriction, but this is a bogus argument. "Guarantee compatibility only across same binary" means "same binary => compatibility" whereas the bogus security argument depends on "compatibility => same binary", which is not true - anyone who analyses your binary will know which numbers to spoof, to convince your program via this interface that they are "running the same code". Guaranteeing "same binary" in an adversial setting is in fact extremely hard and cannot be achieved perfectly; in the real-world it can only be approximated, and should be done so via mechanisms designed for it, not via numbers that are slightly hard to brute force at best and trivial to find out at worst.
This package makes the opposite choice, intended for less restricted and
more open distributed computing environments such as the internet and
decentralised protocols. In these contexts, the requirement of running the
exact same binary program is impossible to achieve in practise. Furthermore,
we see it as an advantage that code does not need to be exactly the same -
for example, one can serialise a closure and its inputs, upgrade your code,
then resume running the closure on the same deserialised input arguments but
with a bugfixed closure. The necessity to pass in an explicit RemoteTable
(here simply called staticTab
) is not a liability, but a useful tool to
represent high-level compatibility and interoperability. Two nodes with the
same keys in their staticTab
s, know that they can talk to each other
interoperably even if their implementations differ significantly. One node
that wishes to talk to different nodes running different minor versions of
the same protocol, could instantiate two different staticTab
s with the same
keys but different implementations, to handle behavioural nuances between the
minor versions. In general, it's a useful bit of metadata to keep around in
your program code, and can help you perform smooth upgrades of a
non-centralised networking protocol more easily.
There are also a few technical differences between this and
distributed-static
, some of which could be re-adopted there too:
We use dependent-types and type-level programming to guarantee type safety, rather than
Rank1Dynamic
. This enables us to store all possible closure types instead of just rank-1 polymorphic functions, including but not limited to: rank-n functions, functions with constraints, those using higher-kinded types, etc.Our serialisation typeclasses are designed to interoperate with many different serialisation frameworks. Instances for
Data.Binary
andCodec.Serialise
are provided here for convenience.We have additional Template Haskell splices that support creating static values from top-level definitions that must refer to other static values, whether they be recursive or mutually-recursive or neither. This is achieved using an implementation of
mfix
for theQ
monad.
We did not implement the ability to compose static references. The main reason is that, in our view, the purpose of static closures is to represent which top-level tasks to execute, and the inputs to execute it on. This is the interface or contract of this concept. How you run the task is an implementation detail, and as discussed above, this might be different across different machines or as time passes and we upgrade the code. Therefore it makes no sense to serialise a representation of "task A is the composition of closure B and closure C", because it is irrelevant to the interface.
If your interface is actually "run arbitrary user-defined code" (e.g. in a VM or EDSL evaluator) then it would indeed make sense to support composition, but then you should define your own AST, evaluator, and serialiser for this; and pass the ASTs around as regular runtime values, not static values. Supporting arbitrary ASTs like this is outside of the scope of this library.
Further, in Haskell there are many ways of applying values not just (_ :: a
-> b) (_ :: a) :: b
, e.g. with constraints, with a combination of static and
non-static arguments, with type applications, and so on. Only the simple form
(_ :: a -> b) (_ :: a) :: b
is likely to be interoperable across multiple
languages. Supporting AST statics would therefore unnecessarily restrict how
we can implement the behaviour of a static closure in our chosen language.
See unit tests for example usage, e.g. UnitTests
Modules
- Control
- Control.Static
- Control.Static.Closure
- Control.Static.Common
- Control.Static.Serialise
- Control.Static.Static
- Control.Static.TH
- Control.Static
Flags
Manual Flags
Name | Description | Default |
---|---|---|
dev | Set compile flags for development | Disabled |
Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info
Downloads
- static-0.1.0.1.tar.gz [browse] (Cabal source package)
- Package description (as included in the package)
Maintainer's Corner
For package maintainers and hackage trustees
Candidates
- No Candidates
Versions [RSS] | 0.1.0.0, 0.1.0.1 |
---|---|
Change log | CHANGELOG.md |
Dependencies | base (>=4 && <5), binary, bytestring, constraints, serialise, singletons, template-haskell, text [details] |
Tested with | ghc >=8.10.1 |
License | GPL-3.0-or-later |
Copyright | 2020 Ximin Luo |
Author | Ximin Luo |
Maintainer | infinity0@pwned.gg |
Category | Control, Static, Closure |
Home page | https://github.com/infinity0/hs-static |
Bug tracker | https://github.com/infinity0/hs-static/issues |
Source repo | head: git clone https://github.com/infinity0/hs-static |
Uploaded | by infinity0 at 2020-05-29T10:30:13Z |
Distributions | |
Downloads | 398 total (9 in the last 30 days) |
Rating | (no votes yet) [estimated by Bayesian average] |
Your Rating | |
Status | Docs not available [build log] All reported builds failed as of 2020-05-29 [all 3 reports] |