Inspecting Nix lambda function named arguments

Some times I get asked how callPackage works and then I realize I have failed to teach this person how to better navigate around with Nix expressions and in nixpkgs so let’s open a nix repl '<nixpkgs>' session to explore:

nix-repl> f = import "${pkgs.path}/pkgs/servers/varnish"

nix-repl> f
«lambda @ /nix/store/8drcpsqry4n2xhai208brjfyhv0s8xzm-a0mjrw6mcpw37sp7yzwkc40kf3718yww-8l2kzla1qx0iksya6pnx5ixm7zc2z49w-nixpkgs-965c944/pkgs/servers/varnish/default.nix:1:1»

nix-repl> builtins.functionArgs f
{ fetchurl = false; groff = false; libedit = false; libxslt = false; makeWrapper = false; ncurses = false; pcre = false; pkgconfig = false; python = false; pythonPackages = false; readline = false; stdenv = false; }

callPackage uses builtins.functionArgs to see what to supply it with from it’s current namespace. This is the magic sauce and this is very valuable when exploring the nixpkgs Nix expressions.

To finish off understanding callPackage though it just intersects attrsets like so:

nix-repl> builtins.intersectAttrs (builtins.functionArgs f) pkgs
{ fetchurl = «lambda @ /nix/store/a0mjrw6mcpw37sp7yzwkc40kf3718yww-8l2kzla1qx0iksya6pnx5ixm7zc2z49w-nixpkgs-965c944/pkgs/build-support/fetchurl/default.nix:38:1»; groff = «derivation /nix/store/zmv1aadh26njgxr5jwzgzyqwch4vpaz9-groff-1.22.3.drv»; libedit = «derivation /nix/store/znngmjzm7cb3vcli6574kvfwv2v05qk4-libedit-20160903-3.1.drv»; libxslt = «derivation /nix/store/l7ydwp52j6rq75zr9bh4x69lc8f8w3i3-libxslt-1.1.29.drv»; makeWrapper = «derivation /nix/store/ml1arp76zl0p1khfn1d3bj9s2mbbfsnz-hook.drv»; ncurses = «derivation /nix/store/dis752dbllygrjb8ql4fwdzxm4l7mzy0-ncurses-6.0-20171125.drv»; pcre = «derivation /nix/store/56v16nr7llsphz4v9p86d6hc74f48gml-pcre-8.41.drv»; pkgconfig = «derivation /nix/store/0iv24kmnrf7x56jk03hz9qs0fhzkkl5w-pkg-config-0.29.2.drv»; python = «derivation /nix/store/c0la0fgiq55j801mrda90vhjjapjr8jh-python-2.7.14.drv»; pythonPackages = { ... }; readline = «derivation /nix/store/nk9kflnhpgxqzsdkyxmwcs4sg9ac44wf-readline-6.3p08.drv»; stdenv = «derivation /nix/store/i8nz0gpadq8khdcrimjagmragkdxld00-stdenv.drv»; }

So next time you are interested in what named arguments a Nix lambda takes then use builtins.functionArgs.

Aspell with custom dictionaries configured

This quick post describes how to configure your aspell dictionaries. This might be needed for your development environment or it could be required for production systems to operate properly. One development need I have is to configure dictionaries for the aspell package for flyspell to work in my new emacs/spacemacs configuration (formerly my local vim configuration).

We can accomplish this in several ways:

• configure this at the system level
• configure this at the user level
• have our shell rc scripts set the ASPELL_CONF environment variable to the appropriate path for the dict-dir attribute.

On my development NixOS I chose to configure this in my user profile. If a production server required this I would chose the first approach (system level configuration) in my Nix expression for the system.

The rationale for the user level of Nix customization for development environments is because I like to reduce the scope of the system configuration on my laptops/tablets. I will typically only have one user account on these devices so I will not gain much by providing modern conveniences at the system-level, plus I can scp my user Nix expression to remote development environments to make my development experience portable, distributable, and reproducible with minimal effort.

User-level Nix Setup of aspell dictionaries

Since I care about the reproducibility of my development environment a great deal (hey, shit happens, even if the Universe has not killed my hard drive recently, I expect over time that will change) I have a user-level Nix expression under the path ${HOME}/.config/nixpkgs/config.nix. Nix/NixOS allows you to override/overlay your own configuration on top of the default nixpkgs/nixos channels in a variety of ways:

• using package overrides (old fashioned at this point given we have overlays)
• using overlays (introduced in 17.09, I think; relatively new)

Note: the user Nix expression used to be located/expected at ${HOME}/.nixpkgs/config.nix.

I am not going to share the entirety of my own user Nix expression inline here but my user Nix expression looks something like the following:

{ pkgs }:
let
  inherit (pkgs) aspellWithDicts; # among other inherits

  # setup custom vim and emacs/spacemacs configuration using Nix helpers already in nixpkgs

  # Now the custom aspell with custom dictionary set configured
  myaspell = aspellWithDicts (d: [d.en]);
  # Yes, I am a grotesque monolinguist, but you can put whatever dictionaries aspell offers in the list returned in the
  # lambda provided to `aspellWithDicts`.
in {
  allowUnfree = true;

  # some firefox and chromium attribute settings here that you probably don't care about, etc.

  # This is the old method of package overriding which I haven't yet converted to overlays, I know CURMUDGEON!
  packagesOverrides = pkgs: rec {
    myDevenv = buildEnv {
      name = "my-devenv";
      paths = with pkgs; [
        mtr
        tcpdump
        inetutils

        # lots of other things and then
        myaspell
      ];
    };
  };
}

Now let’s test this in our user environment/profile:

$ nix-env -i myaspell
... redacted spurious output not relevant to our testing ...

$ env | grep ASPELL
ASPELL_CONF=dict-dir /home/myusername/.nix-profile/lib/aspell

$ ls "$(readlink -f ~/.nix-profile/lib/aspell)/en_US*"
/nix/store/abcdef0123456789...-aspell-env/lib/aspell/en_US.multi
/nix/store/abcdef0123456789...-aspell-env/lib/aspell/en_US-variant_0.multi
...other entried redacted for space considerations
/nix/store/abcdef0123456789...-aspell-env/lib/aspell/en_US-wo_accents-only.rws

$ aspell list <<<"some words one of which will be mispelledk"
mispelledk

Overlays are recommended in recent versions of nixpkgs and above.

Cheers, until next time.

Tips & tricks for systemd and journald on NixOS

This document contains a list of tips and tricks for working with systemd, journalctl, and related tools.

SysVinit vs Upstart vs Systemd

The simplest cheatsheet:

Systemd Unit Types

Systemd has the following unit types you might be concerned with:

• services: A service unit describes how to manage a typically long-running application process. This includes how to start, stop, reload, etc the service, under which circumstances it should be automatically started, timeout periods or events, and the dependency or ordering relative to other systemd units. In NixOS you can create a new systemd service like so:

  systemd.services.myservice = {
    description = "My service is responsible for ...";
    after = [ "multi-user.target" ];
    wantedBy = [ "multi-user.target" ];
    path = [ pkgs.bash ];
    environment = {
      MY_SERVICE_HOME = "/my/path/here";
      MY_SERVICE_MAX_CONNS = toString myVar;
    };
    serviceConfig = {
      User = "myuser";
      ExecStart = path;
      Restart = "always";
    };
  };
  

• paths: This type of unit defines a path to be used for path-based activation. For example, service units could be started, restarted, stopped, reloaded, etc when the file a path unit represents encounters a specific state. inotify is used to monitor the path for state changes.
• slices: Slice units map to Linux Control Groups. This allows resources to be restricted or assigned to processes associated with the slice. The root slice is named -.slice.
• sockets: A socket unit describes a network or IPC socket, or a FIFO buffer that systemd uses for socket-based activation. Socket units are associated to services to trigger their start.
• swaps: This unit describes swap space on the system.
• targets: A target unit is used to provide synchronization points for other units when booting up or changing states. The target of interest to most systemd service definers will likely be multi-user.target.
• timers: Timer units define a timer managed by systemd. It represents a periodic or event-based activation. A matching unit, typically a service, will be started when the timer or event requirements are met.

Systemd has other types of units but the above list is a good starting point. For more information please consult man systemctl.

The following commands can be used to query information about systemd units:

# List dependencies for a unit
$ systemctl list-dependencies UNITNAME

# List sockets
$ systemctl list-sockets

# List active systemd jobs
$ systemctl list-jobs

# List all units and their respective states
$ systemctl list-unit-files

# List all loaded or active units
$ systemctl list-units

Systemd Services

Most of the time we will be concerned with systemd services. Below are a list of useful commands for working with these:

# Need to have sudo privileges to stop/start/restart services
$ sudo systemctl stop SERVICE
$ sudo systemctl start SERVICE
$ sudo systemctl restart SERVICE

# Query commands anyone can run
$ systemctl status SERVICE
$ systemctl is-active SERVICE
$ systemctl show SERVICE

You can also run systemctl commands remotely like so:

$ systemctl -H hostname status SERVICE

This works for systemctl commands other than just systemd service specific commands.

Log Accessibility By journalctl

For all services that need logs accessed via journalctl you should log to the console from a systemd unit.

For example, Elasticsearch logging configuration can be set as so:

rootLogger: INFO, console
logger:
  action: INFO
  com.amazonaws: WARN
appender:
  console:
    type: console
    layout:
      type: consolePattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

Then you will be able to query logs from the elasticsearch service unit by using:

$ journalctl -f -u elasticsearch

Accessing Logs Via journalctl

# tail "follow" all log messages for elasticsearch unit/service
$ journalctl -f -u elasticsearch

# show last 1000 error messages for elasticsearch unit/service (command
# terminates without ^C)
$ journalctl -fen1000 -u elasticsearch

# only show kernel messages in tail "follow" mode:
$ journalctl -k -f

# only show log messages for service BLA since last "boot"
$ journalctl -b -u BLA

# show all error log messages from all log sources since last "boot"
$ journalctl -xab

Many more permutations of options are available on journalctl. Please consult man journalctl for more information.

User Access To journalctl Logs

All users that are in the systemd-journal group should be able to query logs via journalctl. Ensure your SSH user is in this group via groups USERNAME.

NixOS Configuration for journald

The NixOS expression for a node’s configuration contains the following settings that are worth tuning on servers with high frequency events being logged.

As of NixOS 16.03, the defaults for services.journald.rateLimitBurst and services.journald.rateLimitInterval are worth evaluating for your needs:

$ sudo nixos-option services.journald.rateLimitBurst
Value:
100

Default:
100

Description:

Configures the rate limiting burst limit (number of messages per
interval) that is applied to all messages generated on the system.
This rate limiting is applied per-service, so that two services
which log do not interfere with each other's limit.

...

And:

$ sudo nixos-option services.journald.rateLimitInterval
Value:
"10s"

Default:
"10s"

Description:

Configures the rate limiting interval that is applied to all
messages generated on the system. This rate limiting is applied
per-service, so that two services which log do not interfere with
each other's limit. The value may be specified in the following
units: s, min, h, ms, us. To turn off any kind of rate limiting,
set either value to 0.

This means on this system journald will rate limit events per service after 100 messages within 10s. For many servers this is low, and you will want to adjust it with values like the following:

  services.journald.rateLimitBurst = 1000;
  services.journald.rateLimitInterval = 1s;

The above will rate limit services to logging 1000 messages per second.

You can also turn off rate limiting in journald with the following:

  services.journald.rateLimitInterval = 0;

Nix if-then-else expressions

Nix if-then-else expressions

A coworker asked the following question:

I understand how to use an if/else, but I don’t understand how to get an if/elseif/else.

Below is my Slack response (slightly edited) to explain the dissonance in the question itself and then provide a path forward for him. I imagine this might be a common misunderstanding for those coming to Nix without prior understanding of expression languages vs languages that use statements.

Nix being an expression language just has expressions (where the result of each part evaluates to a value [eventually]). Most mainstream imperative languages like Perl, Bash, etc. that most people are familiar with use this notion of statements statements rather than expressions. Statements allow these languages to support if/elseif/…/else as an extension.

In Nix and other expression based languages, this is not the case and a limiting factor of expressions is that every expression must evaluate to a value. So you might write:

{
  key = if builtins.pathExists ./path then "woot" else "bummer";
}

In the case the ./path exists it will evaluate to the value "woot" otherwise it evaluates to the value "bummer".

So the result of the top level expression is:

{ key = "woot"; }

# OR

{ key = "bummer"; }

This does not translate to languages that model ifs as statements, for example:

$ declare bla=$(if true; then "bla"; else "foo"; fi)
$ echo "${bla}"

$ declare bla=$(if true; then echo "bla"; else echo "foo"; fi)
$ echo "${bla}"
bla

Note: that side effects are required inside each clause.

Looking at the equivalent if/else statement for the expression example above we have:

if [ -f ./path ]; then
  declare key="woot"
else
  declare key="bummer"
fi

Here you see that Bash uses side effects to do the assignment in each case, but say we had this:

if [ -d ./path ]; then
  declare key="woot"
elseif [ -x ./path ]; then
  echo "executable"
else
  declare key="bummer"
fi

Now we have a case (where the file is not a directory and also executable) that the variable key is not set. This wouldn’t happen in an expression based language.

In short if-then-else is the only way we can build an expression to always evaluate to a value where all logical paths are covered without the program needing to know about the underlying data or condition clauses inspected in the if portion.

You can think of if-then-else as a lambda that is defined as:

  ifThenElse = cond: t: f: if cond then t else f

So to solve the problem you take one of two approaches.

• If your use case is matching strings exactly in each if/ifelse condition, then you can use an attrset with the keys as the values you need to match:

{ envType, defaultCfg }:
let
  cases = { "dev" = devCfg; "test" = testCfg; "prod" = prodCfg; };
  lookup = attrs: key: default:
    if attrs ? key then attrs."${key}" else default;
in lookup environments envType defaultCfg

• Use nested if-then-else expressions when you cannot just lookup a key in an attrset.

What's the Nix cookbook?

This is a cookbook-style teaching tool to show how to write Nix expressions for the purposes of packaging, system configuration, provisioning, orchestrated deployment, CI jobs, etc.

Here is some introductory material to get you started:

• Nix By Example
• NixOS By Example
• NixOps By Example

More to come soon!