nix-freeze-tree: Convert a tree of files into fixed-output derivations

[ agpl, application, library, nix, program ] [ Propose Tags ]

nix-freeze-tree walks a directory tree and generates a tree of Nix expressions that recreate the tree from fixed-output derivations, one derivation per file in the source tree. See README.md to get started.


[Skip to Readme]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

Versions [RSS] 0.1.0.0, 0.1.0.1, 0.1.1.0
Change log CHANGELOG.md
Dependencies base (>=4.9 && <4.18), bytestring (>=0.10.8.2 && <0.12), cryptonite (>=0.28 && <0.31), data-fix (>=0.2.0 && <0.4), directory (>=1.3.3.0 && <1.4), directory-tree (>=0.12.1 && <0.13), hnix (>=0.16.0 && <0.17), hnix-store-core (>=0.5.0.0 && <0.6), nix-freeze-tree, optparse-applicative (>=0.14.3.0 && <0.18), path (>=0.6.1 && <0.10), path-io (>=1.6 && <1.9), prettyprinter (>=1.7.0 && <1.8), raw-strings-qq (>=1.1 && <1.2), text (>=1.2.3.1 && <1.3 || >=2.0 && <2.1), transformers (>=0.5.6.2 && <0.7) [details]
License AGPL-3.0-or-later
Copyright (c) 2020, 2023 Jack Kelly
Author Jack Kelly
Maintainer jack@jackkelly.name
Category Application, Nix
Home page https://git.sr.ht/~jack/nix-freeze-tree
Bug tracker https://todo.sr.ht/~jack/nix-freeze-tree
Source repo head: git clone https://git.sr.ht/~jack/nix-freeze-tree
Uploaded by jack at 2023-12-05T11:34:50Z
Distributions
Executables nix-freeze-tree
Downloads 518 total (9 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2023-12-05 [all 2 reports]

Readme for nix-freeze-tree-0.1.1.0

[back to package description]

nix-freeze-tree

builds.sr.ht status AGPLv3

nix-freeze-tree is a utility that walks a directory tree, and writes out Nix expressions which rebuild that tree. The generated expressions have two important properties:

  1. Each file in the tree is built by a separate fixed-output derivation, and

  2. Directories are built by derivations that symlink their contents recursively.

If you are using nix copy to ship a derivation between nix stores, copying the derivation built by evaluating the output of nix-freeze-tree can reuse existing files in the destination store, as fixed-output derivations can be checked against a hash before copying.

I use this to speed up deploys to my personal website. Running nix-freeze-tree on the output from my site generator and importing the result means that nix copy only uploads new files, instead of copying a single giant derivation containing every image on the site.

Installation

Nix

If you use Nix, you can install the command into your local environment by running nix profile install sourcehut:~jack/nix-freeze-tree. If you're not using flakes, you can instead try nix-env -f https://git.sr.ht/~jack/nix-freeze-tree/archive/master.tar.gz -iA command.

Cabal

If you have GHC installed, you can also build and install the binary by running cabal v2-install nix-freeze-tree. You can also run the binary from the build directory with cabal v2-run nix-freeze-tree -- ARGS.

Usage

Usage: nix-freeze-tree [--version] [-f|--force] [-v|--verbose]
                       [-o|--out-root OUT_ROOT] [IN_DIR]
  Write a tree of nix expressions to OUT_ROOT that build a derivation,
  symlinking every file in IN_DIR as a separate fixed-output derivation.

Available options:
  --version                Display version information and exit.
  -h,--help                Show this help text
  -f,--force               If any default.nix exist in the output directory,
                           remove them and generate anyway
  -v,--verbose             Display messages while working.
  -o,--out-root OUT_ROOT   Where to write the nix files (default: IN_DIR)
  IN_DIR                   Directory to freeze (default: .)

Calling from Nix

This repository's flake defines a function lib.${system}.freeze, which can be used to freeze the output of an existing derivation. You can use it like this:

{
  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nix-freeze-tree = "git+https://git.sr.ht/~jack/nix-freeze-tree"
  };

  outputs = inputs@{ ... }:
    inputs.flake-utils.lib.eachDefaultSystem (system:
      {
        defaultPackage = (import nix-freeze-tree { }).lib.${system}.freeze
          some-derivation-to-be-frozen;
      }
    );
}

The freeze function corresponding to builtins.currentSystem is also exposed in default.nix, so you can use it with Nix code that looks a bit like:

let
  # Some nixpkgs from somewhere
  nixpkgs = ...;

  # Whether you get it from a channel, a local checkout,
  # builtins.fetchTarball or builtins.fetchgit is up to you
  nix-freeze-tree-repo = builtins.fetchTarball { ... };

  # Derivation to freeze
  drv = mkDerivation { ... };

  # Evaluate default.nix from nix-freeze-tree-repo and hold onto the
  # freeze function
  freeze = (import nix-freeze-tree-repo { inherit nixpkgs; }).freeze;
in
  freeze drv

FAQ

Q: Why didn't you use symlinkJoin to tie all the subtrees together?

A: I had the program mostly working before I remembered that symlinkJoin existed. I thought about making each file's derivation contain the directories leading up to it and then symlinkJoining them all together, but decided against it for three reasons:

  1. You can easily build subtrees using the Nix expressions generated by nix-freeze-tree;
  2. Building the output of nix-freeze-tree will correctly create empty directories present in the input; and
  3. Files are hashed based on their content alone, not their position in the tree-to-be-frozen (reducing duplication).

Other Resources