sync-mht: Fast incremental file transfer using Merkle-Hash-Trees

[ library, mit, program, utility ] [ Propose Tags ] [ Report a vulnerability ]

A command line tool that can be used to incrementally synchronize a directory hierarchy with a second one. It is using a Merkle-Hash-Tree to compare the folders, such that the synchronization time and communication (round) complexity grows only logarithmically with the size of the directories (assuming the actual difference of the directories is small).

The communication happens through standard streams between parent and child processes, which can easily be routed through remote command execution tools.


[Skip to Readme]

Modules

[Last Documentation]

  • Sync
    • MerkleTree
      • Sync.MerkleTree.Analyse
      • Sync.MerkleTree.Client
      • Sync.MerkleTree.CommTypes
      • Sync.MerkleTree.Run
      • Sync.MerkleTree.Server
      • Sync.MerkleTree.Sync
      • Sync.MerkleTree.Trie
      • Sync.MerkleTree.Types
      • Util
        • Sync.MerkleTree.Util.GetFromInputStream
        • Sync.MerkleTree.Util.RequestMonad

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

Versions [RSS] 0.2.0.0, 0.2.1.0, 0.2.1.1, 0.3.0.0, 0.3.1.0, 0.3.2.0, 0.3.3.0, 0.3.4.0, 0.3.5.0, 0.3.6.0, 0.3.7.0, 0.3.8.0, 0.3.8.1, 0.3.8.2, 0.3.8.3, 0.3.8.4, 0.3.8.5
Dependencies array (>=0.5), base (>=4.7 && <4.9), base16-bytestring (>=0.1), byteable (>=0.1), bytes (>=0.15), bytestring (>=0.10), cereal (>=0.4), containers (>=0.5), cryptohash (>=0.11), directory (>=1.2), exceptions (>=0.7), filepath (>=1.3), HUnit (>=1.2), io-streams (>=1.2), mtl (>=2.1), process (>=1.2), random (>=1.0), regex-compat (>=0.95), temporary (>=1.2), text (>=1.1), time (>=1.4), transformers (>=0.3), unix (>=2.7), zlib (>=0.5) [details]
License MIT
Author Emin Karayel <me@eminkarayel.de>
Maintainer Emin Karayel <me@eminkarayel.de>
Category Utility
Home page https://github.com/ekarayel/sync-mht
Bug tracker https://github.com/ekarayel/sync-mht/issues
Source repo head: git clone https://github.com/ekarayel/sync-mht
this: git clone https://github.com/ekarayel/sync-mht(tag 0.3.8.5)
Uploaded by emink at 2016-07-16T15:15:39Z
Distributions
Reverse Dependencies 1 direct, 0 indirect [details]
Executables sync-mht
Downloads 9375 total (19 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2016-11-22 [all 2 reports]

Readme for sync-mht-0.3.8.5

[back to package description]

sync-mht

Build Status codecov.io Hackage Release Stackage LTS 5 Stackage Nightly

Synopsis

Fast incremental file transfer using Merkle-Hash-Trees

Description

A command line tool that can be used to incrementally synchronize a directory hierarchy with a second one. It is using a Merkle-Hash-Tree to compare the folders, such that the synchronization time and communication (round) complexity grows only logarithmically with the size of the directories (assuming the actual difference of the directories is small).

The communication happens through standard streams between parent and child processes, which can easily be routed through remote command execution tools.

Examples

The following command

sync-mht -s foo/ -d bar

will synchronize the local folder bar/ with the local folder foo/, but

sync-mht -s foo/ -d remote:/bar -r "ssh fred@example.org sync-mht"

will synchronize the folder bar/ in the home directory of the user fred on the host machine example.org with the local folder foo/.

It is also possible to use it with docker, e.g.

sync-mht -s foo/ -d remote:/bar -r "docker run -i --volumes-from bar ekarayel/sync-mht sync-mht"

to synchronize the folder /bar (of the container named bar) with the local folder foo/.

By default sync-mht will only show a statistic about the difference between the source and destination directories. The options --add, --update and --delete respectively allow copying of files to the target directory, updating files that are already in the target directory - not matching the contents in the source directory and deleting files that are in the destination directory but not in the source directory.

Setup

Installing the latest release of sync-mht with stack:

stack install sync-mht

Contribution

If you want to contribute - cloning the latest commit and building can be done using the following steps:

git clone --recursive https://github.com/ekarayel/sync-mht.git
cd sync-mht
stack exec runhaskell configure.hs
stack install