This page is for me to share notes of things I was confused by as I’ve used Nix.

Concepts

Nix is different enough from other programming tools that it’s necessary to have some discussion of concepts.

What is Nix

Nix is to managing packages what lisp is to adding numbers together.

More concretely, “Nix” refers to a package manager, the language those packages are written in; and usually referring to “Nix” also includes the main repository of packages (nixpkgs) and the OS (NixOS).

Nix is cool (both in terms of several practical benefits that are difficult to get elsewhere, and in terms of just being elegant), but can be hard to use. – It’s 95% wonderful, 5% pain in the arse.

Perhaps the quickest way to illustrate the benefits: Nix’s UX is like Docker, without dealing with containers.

If you think what you read about Nix (elsewhere) sounds interesting, try it out passively bit by bit over months.

Derivations

“Derivations” confused be for a long time.

If you go through the widely recommended Nix Pills, prob’ly the first surprising Nix Pill is pill 7, its intro to “derivations”.

“Derivation” is a foundational concept in Nix.
As you can see in the blogpost, in one way, it’s like a JSON object.

I met someone who explained that he had been confused for some time about how immutable structures in functional programming could be used in programs which would manipulate the list. (How can something be immutable, but also change?).

With Nix, I felt a similar confusion for a long time. (How can something be a hashmap, but also be a package installed on a filesystem?).

The above pill compares building a .nix file to building a .c program.
This is fair, but didn’t click with me.

I’d rather phrase it: that the derivation is the Nix’s “how to build a package” in its pure, consistent way.
The Nix programs in .nix files contain programmatic abstractions built on top of that.

Or a bit more vaguely: a “nix derivation” is more/less like nix’s flavour of a software package.

Other important words: “Instantiate” is “generate the .drv from the high-level .nix”, and “realise” is the package actually being built (or copied) to the store.

The Nix pills take a very bottom-up approach to understanding Nix. I find it hard to appreciate without some familiarity with Nix stuff. But, Nix is much harder to understand without an intuition for what a derivation is.

Realising Derivations

This also confused me for a long time: the relationship between Nix expressions and realised derivations.

What also confused me is a .nix file might contain many derivations.
Which doesn’t sound all that confusing.
But.
In Python, each Python file is a module, and runs with a main function.
In Haskell, each file is its own module, and runs with a main function.
In Nix, each file is a Nix expression, and files don’t have a specific format.

A derivation (“nix package”) is declared.
Which itself is maybe not confusing. Terraform code involves declaring code and letting Terraform determine what resources to create/delete/update.
And the Nix language is all pure expressions (since it doesn’t have statements).

– But some combination of the above defied my intuitions.
Packages get built (which you’d expect to be an impure side-effect) from these declarative Nix expressions in source files, but not necessarily as simply/explicitly as “package { src = ... }”.

Flakes

Flakes are a recent addition to Nix.
People who understand flakes say flakes are cool.
(And now that I understand flakes a bit better: yeah, they’re cool).

Flakes are kinda like “nix squared”.

With Nix, you pay an upfront cost; where complexity is managed explicitly.
The implicit, imperative mutation of the global filesystem that programmers are used to when driving their workstations is verboten.

The typical way of using nixpkgs was using channels. This still leaved some room for inconsistency when installing the same packages across different machines (e.g. due to differences in how up-to-date the channels are in each machine), and leaves some things implicit/mutable. (e.g. what channels are available on the machine).

Nix flakes do away with using channels, and provide a consistent way for a repository to declare its inputs, and what outputs it provides.

Serokell’s “Practical Nix Flakes” blogpost is a good introduction.

With Nix flakes, there’s more complexity upfront, but some pretty neat benefits from paying that cost. (The CLI UX with flakes is nicer; and it’s easier to make use of flakes than non-flakes Nix expressions).

The new Nix commands make use of flakes.
e.g. the old-style command to install a package might be:

nix-env --install --attr nixpkgs.neovim

(install neovim from the nixpkgs channel), but the new-style command is:

nix profile install nixpkgs#neovim

(install neovim package from the nixpkgs flake).

e.g. I wrote a flake for my hakyll blog. I can build & run the static site generator with just nix run github:rgoulter/my-hakyll-blog.

How To

nix-env

nix-env is easiest to start with, but also discouraged for being footgun-prone.

Using nix-env can almost be a drop-in replacement for how you might otherwise use a package manager.

Instead of:

sudo apt-update
sudo apt install vim

you’d run:

nix-channel --update
nix-env --install --attr nixpkgs.vim

(Perhaps unexpectedly: not using sudo).

There’s only a marginal benefit to Nix compared to the system package manager. (You get the same versions of software on each system, rather than e.g. CentOS’ version being much older than Ubuntu’s; and nixpkgs tend to have very up-to-date packages).

I’ve seen this described as an anti-pattern, and that its use is discouraged. – I’ll say it’s prone to problems of “I forgot I did that” or “I didn’t know I did that” and ending up with problems that are hard to understand.

e.g. when I first installed NixOS, I logged in as the root user and installed firefox using # nix-env --install --attr nixpkgs.firefox, so I could continue reading the manual etc.
Months later, on my normal user account, I was confused why my Firefox was stuck on such an old version. I was especially confused since Firefox wasn’t installed to my user profile anywhere.
“I forgot I’d done that. It ended up with a hard to understand problem.”

I think it’s discouraged due to being the cause of a disproportionate amount of requests for help in the community. It’s most likely to be used by beginners, who are likely to not know all the details of what they’re doing, or likely to not be able to figure out why an issue is occuring.

Finding Packages

1. https://search.nixos.org/packages

2. To search the nixpkgs:

nix search nixpkgs <search term>

Declarative Package Management

“Declarative Package Management” is the easiest non-trivial thing to do with Nix.

The Nixpkgs manual has a section describing “declarative package management”.

This at least should be appealing to yak-shavers.

It gives an example using the expression:

pkgs.buildEnv {
    name = "my-packages";
    paths = [
      bc
      coreutils
      jq
      silver-searcher
    ];
  };
}

(If you’re persuaded against using nix-env, nix.dev’s tutorial on ad hoc developer environments is at about the same level).

The first tricky taste of Nix is that there are several different ways to install this package.

Non-exhaustively:

Installing with packageOverrides

At the time of writing, the manual suggests the way to use this expression is to override the list of packages in nixpkgs and add myPackages, by using ~/.config/nixpkgs/config.nix with contents:

{
  packageOverrides = pkgs: with pkgs; {
    myPackages = pkgs.buildEnv {
      name = "my-packages";
      paths = [
        bc
        coreutils
        jq
        silver-searcher
      ];
    };
  };
}

And then installing this by running:

nix-env --install --attr nixpkgs.myPackages

Installing with an Overlay

The chapter immediately following the Declarative Package Management in the nixpkgs manual is about overlays.

Which seems like a nicer solution than the packageOverrides in config.nix.

I used this approach for a long time.

I did this by having ~/.config/nixpkgs/overlays/myOverlays.nix with:

self: super:

{
  myPackages = self.buildEnv {
    name = "my-packages";
    paths = with self; [
      bc
      coreutils
      jq
      silver-searcher
    ];
  };
}

And then installing this by running:

nix-env --install --attr nixpkgs.myPackages

And just re-running that each time myPackages is changed.

e.g. here’s the overlay I was using.

Installing with Vanilla flake.nix

For x86-64 Linux, with the following flake.nix:

{
  description = "A basic flake with declaratively managed packages";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  outputs = { self, nixpkgs, flake-utils }: {
    packages = let
      pkgs = nixpkgs.legacyPackages.x86_64-linux;
    in {
      x86_64-linux.myPackages = pkgs.buildEnv {
        name = "my-packages";
        paths = with pkgs; [
          bc
          coreutils
          jq
          silver-searcher
        ];
      };
    };
  };
}

(If you put this in a git repository, the file will need to be checked into the repository).

Then, you can install this by running (in the same directory as the flake.nix):

nix profile install .#myPackages

A more complex flake.nix, but with "x86_64-linux" factored out:

{
  description = "A basic flake with declaratively managed packages";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  outputs = { self, nixpkgs, flake-utils }: let
    systems = [
      "x86_64-linux"
      "x86_64-darwin"
    ];
    forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f system);
  in {
    packages = forAllSystems (system: let
      pkgs = nixpkgs.legacyPackages.${system};
    in {
      myPackages = pkgs.buildEnv {
        name = "my-packages";
        paths = with pkgs; [
          bc
          coreutils
          jq
          silver-searcher
        ];
      };
    });
  };
}

Installing with Flakes (Nix User Repository)

Probably the tidiest way in the long term may be to organise your Nix code into its own repository.

Making use of nix-community’s nix-user-repository template (i.e. cloning it and pushing to your own fork/repository):

Although maybe not the nicest way of doing it, I’d change default.nix to have:

{ pkgs ? import <nixpkgs> { } }:

{
  myPackages = import ./pkgs/myPackages { pkgs = pkgs; };
}

And I add a file pkgs/myPackages/default.nix with:

{ pkgs }:

pkgs.buildEnv {
    name = "my-packages";
    paths = with pkgs; [
      bc
      coreutils
      jq
      silver-searcher
    ];
  };
}

And then this can be installed with nix profile .#myPackages, or nix profile install github:<username>/<repo>#myPackages or whatever URL. (The files have to be checked into a git repository).

e.g. my nix user repository.

Installing with Home Manager

Home Manager is to a user’s dotfiles what NixOS is to a Linux system.

It’s widely used, and widely recommended. And presumably any scheme you come up with will be an ad-hoc, half-baked imitation of what home manager does.

Home Manager can be used to manage what packages you have installed in your user profile. (Since home manager’s configuration is plaintext and can list packages to install, you wouldn’t need buildEnv that the above examples have).

I’ve used it, but only for managing my dotfiles, and in my experience, I can’t recommend it to beginners. The NixOS wiki’s page about Home Manager has an example of using Home Manager for declarative management of nix-env.

Seems like it’s worth trying if you’re comfortable copy-pasting now and understanding later, or once you’ve gained more familiarity with Nix.

nix-shell

A nix shell is a shell which leverages Nix to make packages available when using that shell. (i.e. not installing the program to the user’s system).
In a way, this is like a cousin to the virtual environments which various programming environments provide.
It’s also comparable to how Docker images are sometimes used as an easy way to distribute a toolchain or some CLI program,
or to various “X version managers” like Ruby Version Manager or Node Version Manager or asdf.

Running the command nix-shell will make use of a shell.nix file in the current directory. A simple shell.nix file is:

{ pkgs ? import <nixpkgs> {} }:

with pkgs;
mkShell {
  packages = [
    neovim
  ];
}

Shells with Nix Flakes

“But nix-shell with a shell.nix like the above uses nix channels, not the fancy new flakes”.

The new flake-aware Nix approach to get an ephemeral development environment in the shell is to integrate with direnv. e.g. to use nix-direnv.

The equivalent of shell.nix, with flakes? Well…

The nix-direnv project links to the following template flake.nix:

{
  description = "A basic flake with a shell";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
      in {
        devShell = pkgs.mkShell {
          packages = [ pkgs.neovim ];
        };
      });
}

which can be generated (along with a .envrc which makes use of the flake’s devShell) by running:

nix flake new -t github:nix-community/nix-direnv .

With the .envrc generated from this template, the flake’s devshell is automatically built, and its packages made available to the current shell.

Discussion about Hello World Tutorial

The Nix manual has a tutorial for a simple expression; how to write a Nix package for gnu hello.

This guides us with a default.nix:

{ stdenv, fetchurl, perl }:

stdenv.mkDerivation {
  name = "hello-2.1.1";
  builder = ./builder.sh;
  src = fetchurl {
    url = "ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz";
    sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
  };
  inherit perl;
}

and a builder.sh:

source $stdenv/setup

PATH=$perl/bin:$PATH

tar xvfz $src
cd hello-*
./configure --prefix=$out
make
make install

The tutorial apparently suggests to this within the nixpkgs repository itself, which seems a little impractical, and quite unintuitive to me.

“How am I supposed to build this?” otherwise brought me to the GitHub issue NixOS/nix#2259, with people wondering the same thing.
Because you’ll try running nix-build and it won’t “just work” with the above default.nix.

The easiest “well, this works” default.nix to stumble across is suggested as:

let
  pkgs = import <nixpkgs> {};
in
pkgs.stdenv.mkDerivation {
  name = "hello-2.1.1";
  builder = ./builder.sh;
  src = pkgs.fetchurl {
    url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
    sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
  };
  perl = pkgs.perl;
}

though now I’d suggest a closer “well, this works” default.nix file would instead be:

{ pkgs ? import <nixpkgs> {} }:

with pkgs;
stdenv.mkDerivation {
  name = "hello-2.1.1";
  builder = ./builder.sh;
  src = fetchurl {
    url = "ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz";
    sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
  };
  inherit perl;
}

A helpful commenter in the GitHub issue above links to a StackOverflow answer which helps explain the above.
It explains that there’s no single required nor conventional format for default.nix. It explains, though, that packages in nixpkgs are typically in a format for a “callPackage derivation”.

It suggests the rather unwieldy command:

nix-build -E 'with import <nixpkgs> { }; callPackage ./path/to/default.nix { }'

Or perhaps another way of putting it: if we instead renamed the default.nix from the tutorial as hello.nix, a default.nix file which could be built with just nix-build would be:

{ pkgs ? import <nixpkgs> {} }:

pkgs.callPackage ./hello.nix {}

or in a more explicit way:

let
  pkgs = import <nixpkgs> {};
in
import ./hello.nix {
  stdenv   = pkgs.stdenv;
  fetchurl = pkgs.fetchurl;
  perl     = pkgs.perl;
}

This … is clearly a bit more involved than you’d expect for “hello world”.

But at this point, the nix pills explain the rationale and the mechanisms: pill 12 explains the input design pattern (in this case, why { stdenv, fetchurl, perl }: instead of let pkgs = import <nixpkgs> {}; in ...), and pill 13 the callPackage design pattern (which explains the magic of what callPackage is doing).

But it’s also at this point your knowledge of nix is dangerous enough that you can start to make use of the language-support section of the nixpkgs manual, or the NixOS Wiki.

Opinions

Risks of Nix on non-NixOS

Using Nix on operating systems that aren’t NixOS (macOS, or some other Linux) is the easiest way to get a taste of the Nix package manager.

But it’s also prone to problems you won’t face on NixOS.

e.g. On macOS, I faced confusing errors when mixing Haskell’s stack with nixpkgs. It ended up in such a way that it was trying to use the GHC which stack downloaded, and the C compiler and Haskell packages that Nix had downloaded, and these didn’t mix well for some reason. (The Nix solution was to get haskell stack to use GHC provided by nix, too).

Errors like this end up being quite confusing to work through. Running the compiler from a pure nix shell is one way to mitigate it.

I think most people who use Nix eventually find there way to using NixOS.

NixOS is Easier Than You’d Anticipate

You can expect NixOS is not brutally difficult to use for most things.

In terms of using NixOS on as the OS on your workstation:

Put in the most optimistic pessimistic way: the main difference between NixOS and a Linux like Arch Linux is that instead of modifying various files under the /etc/, the system configuration is all managed starting from a single configuration.nix file. – I mean, using firefox on NixOS is no different than using firefox on Arch Linux; but changing system configuration files is entirely different.

Rather, I was worried about using NixOS, I was under the impression that even its happy path was difficult to use.
These days, NixOS is what I drive daily.

I gotta be ambivalent, here:
When stuff goes wrong, in NixOS you may have to understand what’s going on in order to fix it. (understand what the program you’re running is doing, understand how its dependencies do what they do, understand how NixOS deals with those dependencies). NixOS is a layer of tooling to understand above Linux.

There are absolutely people who try NixOS, are able to use it for a bit, and then decide they’re better off with a different OS. I like to think that most people who try NixOS and don’t stick with it stopped using it before they gained a good understanding of Nix.

I think once you grok Nix, then NixOS is wonderful and well worth trying.
But I’d recommend trying nix on Linux or macOS first to get a taste of what the advantages and difficulties will be.

Tips

How to Retain nix-shell so it Doesn’t Get Garbage Collected

https://github.com/NixOS/nix/issues/2208

The sha256 Hash is Used to Identify the Source for fetchFromGitHub, etc.

e.g. in the Nix expression:

fetchFromGitHub {
  owner = "istio";
  repo = "istio";
  rev = "1.8.5";
  sha256 = "1mmhiasq2brzhrnklxzxykmannv84wgzpbnqwz40sb8c1n67qzb0";
}

The sha256 value is used by Nix to identify the source. Which means if you copy paste this, and change the owner and repo:

fetchFromGitHub {
  owner = "rgoulter";
  repo = "my-hakyll-blog";
  rev = "v1.0";
  sha256 = "1mmhiasq2brzhrnklxzxykmannv84wgzpbnqwz40sb8c1n67qzb0";
}

then nix will see the sha256 is already the same as istio/istio rev 1.8.5 and use that repository instead.

Instead, the laziest thing to do is just replace the sha256 with an incorrect value:

fetchFromGitHub {
  owner = "rgoulter";
  repo = "my-hakyll-blog";
  rev = "v1.0";
  sha256 = "0000000000000000000000000000000000000000000000000000";
}

And then replace the sha256 with whatever the value is expected to be. (lib.fakeSha256 can be used instead of however many 0s that is).

Another way to find the sha256 value is to use a program like nix-prefetch.

e.g. running:

nix-prefetch fetchFromGitHub --owner istio --repo istio --rev 1.8.5

will give the correct sha256 value to use.