4 Commits

Author SHA1 Message Date
ahuston-0 b4233b8f1c luks migration, home migration fixes
Check flake.lock / Check health of `flake.lock` (pull_request) Successful in 9s
Check Nix flake / Perform Nix flake checks (pull_request) Successful in 3m49s
2026-05-03 17:33:56 -04:00
ahuston-0 43c026c451 add argiletum
Check flake.lock / Check health of `flake.lock` (pull_request) Successful in 9s
Check Nix flake / Perform Nix flake checks (pull_request) Failing after 2m17s
2026-05-03 14:09:35 -04:00
ahuston-0 9222be4052 add argiletum 2026-05-03 13:59:56 -04:00
ahuston-0 24d451f825 add tang and clevis to palatine-hill 2026-05-03 13:29:39 -04:00
15 changed files with 428 additions and 38 deletions
@@ -0,0 +1,32 @@
---
description: "Use when working with SOPS secrets files (secrets.yaml). Never modify secrets.yaml files directly — always prompt the user to make changes using sops edit."
applyTo: "**"
---
# SOPS Secrets Files — Read-Only
Never modify any `secrets.yaml` file in this repository. These files are SOPS-encrypted and editing them directly (without `sops edit`) will corrupt the encryption and make the secrets unrecoverable.
## Rules
- **Do NOT edit `secrets.yaml` files** using file editing tools, even for renaming keys, restructuring blocks, or adding new entries.
- **Do NOT suggest patches or diffs** that target `secrets.yaml` files.
- **Always prompt the user** to make the change themselves using:
```bash
sops edit <path-to-secrets.yaml>
```
- When a new secret key is needed (e.g., for a new SOPS reference in Nix code), tell the user the exact key name and value to add, and ask them to add it via `sops edit`.
- You may **read** `secrets.yaml` files (e.g., with grep to check key names) — reading is safe. Only writing is forbidden.
## Example
Instead of editing `systems/palatine-hill/secrets.yaml` directly, say:
> Please run `sops edit systems/palatine-hill/secrets.yaml` and add the following under the `kanidm:` block:
>
> ```yaml
> kanidm:
> gitea_oidc_client_secret: "<your-generated-secret>"
> ```
+10
View File
@@ -9,6 +9,10 @@ keys:
- &artemision-home age1t29a6z6cfy8m3cnc8uva0ey833vhcppue8psyumts7mtyf0zufcqvfshuc - &artemision-home age1t29a6z6cfy8m3cnc8uva0ey833vhcppue8psyumts7mtyf0zufcqvfshuc
- &palatine-hill age1qw5k8h72k3fjg5gmlxx8q8gwlc2k6n6u08d8hdzpm2pk9r0fnfxsmw33nh - &palatine-hill age1qw5k8h72k3fjg5gmlxx8q8gwlc2k6n6u08d8hdzpm2pk9r0fnfxsmw33nh
- &selinunte age1jd2dcpykagz20kpk2kkchte3augqncwfn6nywursx0dkfyze6feqdzxkq2 - &selinunte age1jd2dcpykagz20kpk2kkchte3augqncwfn6nywursx0dkfyze6feqdzxkq2
# argiletum: replace placeholder after first boot with:
# nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
# then run: sops updatekeys systems/argiletum/secrets.yaml
- &argiletum age1aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
# cspell:enable # cspell:enable
# add new users by executing: sops users/<user>/secrets.yaml # add new users by executing: sops users/<user>/secrets.yaml
# then have someone already in the repo run the below # then have someone already in the repo run the below
@@ -55,3 +59,9 @@ creation_rules:
- *admin_alice - *admin_alice
age: age:
- *palatine-hill - *palatine-hill
- path_regex: systems/argiletum/secrets.*\.yaml$
key_groups:
- pgp:
- *admin_alice
age:
- *argiletum
+206
View File
@@ -0,0 +1,206 @@
> Note: This document was AI-generated and reviewed by a maintainer.
# ADR 0001 — ZFS Native Encryption: Non-Interactive initrd Key Loading
| | |
|---|---|
| **Status** | Accepted |
| **Date** | 2026-05-03 |
| **Deciders** | Alice Huston |
| **Affects** | `systems/palatine-hill/hardware-changes.nix`, `systems/palatine-hill/zfs.nix` |
---
## Context
`palatine-hill` uses ZFS native encryption for the `/nix` dataset (`ZFS-primary/nix`). The ZFS encryption key was stored on a separate LVM volume (`/crypto/keys/zfs-nix-store-key`) inside the same LUKS container as root.
This created a forced ordering dependency: the `/nix` dataset could not be unlocked until root (`/`) and `/crypto` were both mounted, even though logically they are independent. Two custom initrd units worked around this:
- `zfs-import-zfs-primary` — polling import loop (duplicates NixOS-native logic)
- `zfs-load-nix-key` — reads key from `/sysroot/crypto/keys/zfs-nix-store-key` after `sysroot.mount`
Additionally, `boot.zfs.requestEncryptionCredentials` was forced off entirely, and a `postBootCommands` fallback ran
`zfs load-key -a` after stage 2 as a belt-and-suspenders measure. LUKS unlock was also interactive, requiring manual
passphrase entry at boot.
### Current initrd dependency graph (before this ADR)
```mermaid
flowchart TD
A([initrd start]) --> B[systemd-udev-settle]
A --> C["LUKS unlock nixos-pv\n⚠ interactive"]
C --> D[LVM activate]
D --> E["sysroot.mount\n/ on ext4"]
D --> F["sysroot-crypto.mount\n/crypto on LVM volume"]
B --> G["zfs-import-zfs-primary\n(custom polling loop, 60s timeout)"]
E --> H["zfs-load-nix-key\n(reads /sysroot/crypto/keys/zfs-nix-store-key)"]
F --> H
G --> H
H --> I["sysroot-nix.mount\nZFS-primary/nix"]
I --> J([initrd-fs.target])
E --> J
J --> K([stage 2])
K --> L["postBootCommands:\nzfs load-key -a"]
```
### Problems with the old approach
1. **Cross-filesystem key dependency**: `/nix` unlock depends on root mount, coupling two logically independent operations.
2. **Duplicated pool import logic**: the custom unit reimplements a polling loop that NixOS already generates natively; upstream fixes don't apply automatically.
3. **Native credential handling fully disabled**: `requestEncryptionCredentials = false` makes the configuration opaque to NixOS module evaluation.
4. **Double key load**: `postBootCommands` is a workaround indicating the initrd path is not reliable.
5. **Interactive LUKS unlock**: manual passphrase entry required at every boot — defeats unattended operation.
---
## Options Considered
### Option A — Key embedded in initrd (`boot.initrd.secrets`)
Store the ZFS key directly inside the initrd cpio archive. The key is available from the very start of stage 1 without mounting anything.
**Pro**: Eliminates the cross-mount dependency; re-enables native NixOS ZFS handling; zero new infrastructure.
**Con**: Key lives in the initrd on `/boot`, which is an unencrypted vfat partition. Anyone with physical or boot-partition read access has the key. Does not solve interactive LUKS unlock.
### Option B — Tang network key fetch (Clevis) ✅ Chosen
Encrypt both secrets (LUKS passphrase and ZFS key) as Clevis JWE blobs. At boot, the initrd reaches a Tang server
on the LAN to decrypt them. NixOS's `boot.initrd.clevis` module natively supports `luks`, `zfs`, and `bcachefs`
**no custom unit is needed for ZFS**.
**Pro**: Key never present on disk in plaintext; unified unlock surface for both LUKS and ZFS; no cross-mount dependency; JWE blobs on disk are useless without the Tang server.
**Con**: Adds Tang server as a boot dependency; server won't boot if Tang is unreachable.
---
## Decision
**Option B (Tang/Clevis) is adopted** for both the LUKS root device and the ZFS `/nix` dataset.
`boot.initrd.clevis.devices` handles both unlock targets natively. The custom `zfs-load-nix-key` unit is deleted
entirely. The `zfs-import-zfs-primary` unit is retained — the pool must still be imported before Clevis can load the
dataset key.
Static networking is configured in the initrd using systemd-networkd with a static IP (`192.168.76.2/24`). DNS
resolution (`192.168.76.1`, the OPNsense router running Unbound) allows the Tang URL to be `http://tang.lan`.
### New initrd dependency graph
```mermaid
flowchart TD
A([initrd start]) --> N["initrd-networkd\neno1: 192.168.76.2/24\nDNS: 192.168.76.1"]
A --> B[systemd-udev-settle]
N --> T["Tang server\ntang.lan"]
T -->|"boot.initrd.clevis\n.devices.nixos-pv"| C["LUKS unlock nixos-pv\n(Clevis/Tang — unattended)"]
T -->|"boot.initrd.clevis\n.devices.ZFS-primary/nix"| Z["ZFS-primary/nix key load\n(Clevis/Tang — unattended)"]
C --> D[LVM activate]
D --> E["sysroot.mount\n/ on ext4"]
B --> G["zfs-import-zfs-primary\n(custom polling loop — retained)"]
G --> Z
Z --> I["sysroot-nix.mount\nZFS-primary/nix"]
E --> J([initrd-fs.target])
I --> J
J --> L([stage 2 — fully unattended])
```
### Files changed
| File | Change |
|---|---|
| `systems/palatine-hill/hardware-changes.nix` | Removed `requestEncryptionCredentials = mkForce false`, removed `postBootCommands`, added `boot.initrd.clevis` block for both devices, added `boot.initrd.systemd.network` with static IP + DNS, removed `/crypto` from `/nix` depends |
| `systems/palatine-hill/zfs.nix` | Removed `zfs-load-nix-key` unit, added `boot.zfs.requestEncryptionCredentials = false` |
### Comparison
| | Before | After |
|---|---|---|
| Custom initrd units | 2 (import + key load) | 1 (import only; key load is native Clevis) |
| Key source | `/crypto` LVM volume (disk) | Tang server (network) |
| Disk-based key exposure | Key on LVM volume inside LUKS | `.jwe` blob only; useless without Tang |
| Cross-mount dependency | Yes | No |
| LUKS interactive unlock | Yes | No (Clevis/Tang) |
| Unattended boot | No | Yes (when Tang reachable) |
---
## Consequences
- Boot requires Tang server to be reachable on `tang.lan`. If Tang is down, boot stalls at the Clevis timeout. Maintain Tang server uptime accordingly.
- The `.jwe` files are safe to commit to the repository — they are encrypted blobs that are useless without the Tang server's private key.
- Rolling back to a generation without Clevis (pre-ADR) requires manual LUKS passphrase entry at the console; ensure prior generations remain in the bootloader during initial cutover.
---
## Implementation Notes
### Prerequisites
1. Deploy a Tang server on the LAN and create a DNS host override in OPNsense:
- Services → Unbound DNS → Host Overrides → `tang` / `lan` / `<tang IP>`
2. Verify DNS from palatine-hill before rebooting:
```bash
resolvectl query tang.lan
```
### Create the JWE files
Run from the repository root on a machine that has the LUKS passphrase and access to the running `/crypto` volume:
```bash
# LUKS passphrase JWE — substitute your actual passphrase
echo -n "your-luks-passphrase" | \
clevis encrypt tang '{"url":"http://tang.lan"}' \
> systems/palatine-hill/nixos-pv.jwe
# ZFS dataset key JWE — key file from the running system
clevis encrypt tang '{"url":"http://tang.lan"}' \
< /crypto/keys/zfs-nix-store-key \
> systems/palatine-hill/nix-store.jwe
```
### Commit and build
```bash
git add systems/palatine-hill/nixos-pv.jwe systems/palatine-hill/nix-store.jwe
git commit -m "feat(palatine-hill): add Clevis JWE files for Tang-based boot unlock"
nix build .#palatine-hill # verify build succeeds
```
### Deploy
```bash
nh os switch # keep previous generation in bootloader for rollback
```
### Verify after reboot
```bash
# Confirm ZFS dataset was unlocked automatically
zfs get keystatus ZFS-primary/nix
# Expected: keystatus = available
# Check Clevis log output
journalctl -b | grep -i clevis
# Confirm Tang was reached during initrd
journalctl -b | grep -i tang
```
### Rollback procedure (if needed)
Select the previous generation from the systemd-boot menu at boot. You will be prompted interactively for the LUKS passphrase — this is expected for the old generation.
Generated
+21
View File
@@ -68,6 +68,26 @@
"type": "github" "type": "github"
} }
}, },
"disko": {
"inputs": {
"nixpkgs": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1777713215,
"narHash": "sha256-8GzXDOXckDWwST8TY5DbwYFjdvQLlP7K9CLSVx6iTTo=",
"owner": "nix-community",
"repo": "disko",
"rev": "63b4e7e6cf75307c1d26ac3762b886b5b0247267",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "disko",
"type": "github"
}
},
"firefox-addons": { "firefox-addons": {
"inputs": { "inputs": {
"nixpkgs": [ "nixpkgs": [
@@ -565,6 +585,7 @@
}, },
"root": { "root": {
"inputs": { "inputs": {
"disko": "disko",
"firefox-addons": "firefox-addons", "firefox-addons": "firefox-addons",
"flake-compat": "flake-compat", "flake-compat": "flake-compat",
"flake-parts": "flake-parts", "flake-parts": "flake-parts",
+5
View File
@@ -38,6 +38,11 @@
systems.url = "github:nix-systems/default"; systems.url = "github:nix-systems/default";
# flake inputs with dependencies (in alphabetic order) # flake inputs with dependencies (in alphabetic order)
disko = {
url = "github:nix-community/disko";
inputs.nixpkgs.follows = "nixpkgs";
};
firefox-addons = { firefox-addons = {
url = "gitlab:rycee/nur-expressions?dir=pkgs/firefox-addons"; url = "gitlab:rycee/nur-expressions?dir=pkgs/firefox-addons";
inputs = { inputs = {
+1
View File
@@ -167,6 +167,7 @@ rec {
outputs outputs
server server
system system
home
; ;
}; };
modules = [ modules = [
+4 -1
View File
@@ -3,6 +3,7 @@
inputs, inputs,
outputs, outputs,
server, server,
home,
system, system,
... ...
}: }:
@@ -22,6 +23,9 @@
mutableUsers = lib.mkDefault false; mutableUsers = lib.mkDefault false;
}; };
networking.firewall.enable = lib.mkDefault true;
}
// lib.optionalAttrs home {
home-manager = { home-manager = {
useGlobalPkgs = true; useGlobalPkgs = true;
useUserPackages = true; useUserPackages = true;
@@ -34,5 +38,4 @@
}; };
}; };
networking.firewall.enable = lib.mkDefault true;
} }
+37
View File
@@ -0,0 +1,37 @@
{ lib, ... }:
{
imports = [ ./disk.nix ];
time.timeZone = "America/New_York";
networking = {
hostId = "c3798ccc";
firewall = {
enable = true;
allowedTCPPorts = [ 80 ];
};
useNetworkd = true;
};
# Raspberry Pi 4 uses U-Boot / extlinux — disable both GRUB and systemd-boot
# TPM 2.0 HAT: systemd initrd required for tpm2-device auto-unlock
# After first install, enroll with:
# systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=0+7 --recovery-key /dev/mmcblk0p3
boot = {
useSystemdBoot = lib.mkForce false;
loader.grub.enable = lib.mkOverride 0 false;
initrd = {
systemd.enable = true;
luks.devices."cryptroot".crypttabExtraOpts = [ "tpm2-device=auto" ];
};
};
sops = {
defaultSopsFile = ./secrets.yaml;
age.sshKeyPaths = [ "/etc/ssh/ssh_host_ed25519_key" ];
};
services.tang.enable = true;
system.stateVersion = "26.11";
}
+12
View File
@@ -0,0 +1,12 @@
{ inputs, ... }:
{
system = "aarch64-linux";
server = true;
home = false;
sops = true;
users = [ "alice" ];
modules = [
inputs.nixos-hardware.nixosModules.raspberry-pi-4
inputs.disko.nixosModules.disko
];
}
+56
View File
@@ -0,0 +1,56 @@
{
disko.devices = {
disk = {
# SD card — change device to /dev/sda if booting from USB instead
main = {
type = "disk";
device = "/dev/mmcblk0";
content = {
type = "gpt";
partitions = {
# Raspberry Pi firmware partition — must be vfat and first
firmware = {
size = "256MiB";
type = "EF00";
priority = 1;
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot/firmware";
mountOptions = [
"fmask=0077"
"dmask=0077"
];
};
};
# NixOS boot partition — holds kernels/initrds for each generation
boot = {
size = "1GiB";
priority = 2;
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/boot";
};
};
# Root filesystem — LUKS-encrypted, unlocked via TPM 2.0 HAT
root = {
size = "100%";
priority = 3;
content = {
type = "luks";
name = "cryptroot";
settings.allowDiscards = true;
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/";
};
};
};
};
};
};
};
};
}
+7
View File
@@ -0,0 +1,7 @@
# TODO: after first boot, regenerate with:
# sudo nixos-generate-config --no-filesystems
# (disko owns fileSystems; do not add them here)
{ ... }:
{
swapDevices = [ ];
}
+3
View File
@@ -40,6 +40,9 @@
dbus = { dbus = {
enable = true; enable = true;
implementation = "broker"; implementation = "broker";
packages = with pkgs; [
gcr
];
}; };
}; };
+24 -9
View File
@@ -1,11 +1,7 @@
{ lib, pkgs, ... }: { lib, ... }:
{ {
boot = { boot = {
zfs.requestEncryptionCredentials = lib.mkForce false;
postBootCommands = ''
${pkgs.zfs}/bin/zfs load-key -a
'';
initrd = { initrd = {
services.lvm.enable = true; services.lvm.enable = true;
luks.devices = { luks.devices = {
@@ -16,6 +12,28 @@
}; };
}; };
clevis = {
enable = true;
useTang = true;
devices = {
# Unlock LUKS root device via Tang
"nixos-pv".secretFile = ./nixos-pv.jwe;
# Unlock ZFS native-encrypted dataset via Tang
"ZFS-primary/nix".secretFile = ./nix-store.jwe;
};
};
# Static networking needed in initrd so Tang is reachable before any disk mounts
systemd.network = {
enable = true;
networks."10-initrd-eno1" = {
matchConfig.Name = "eno1";
address = [ "192.168.76.2/24" ];
routes = [ { Gateway = "192.168.76.1"; } ];
dns = [ "192.168.76.1" ];
linkConfig.RequiredForOnline = "routable";
};
};
}; };
}; };
@@ -37,10 +55,7 @@
"dmask=0077" "dmask=0077"
]; ];
"/nix".depends = [ "/nix".depends = [ "/" ];
"/"
"/crypto"
];
}; };
} }
+1 -27
View File
@@ -7,6 +7,7 @@
{ {
boot = { boot = {
zfs.extraPools = [ "ZFS-primary" ]; zfs.extraPools = [ "ZFS-primary" ];
zfs.requestEncryptionCredentials = false;
filesystem = "zfs"; filesystem = "zfs";
extraModprobeConfig = '' extraModprobeConfig = ''
options zfs zfs_arc_min=82463372083 options zfs zfs_arc_min=82463372083
@@ -85,33 +86,6 @@
fi fi
''; '';
}; };
zfs-load-nix-key = {
description = "Load ZFS key for ZFS-primary/nix in initrd";
wantedBy = [ "initrd-fs.target" ];
requires = [
"sysroot.mount"
"zfs-import-zfs-primary.service"
];
after = [
"sysroot.mount"
"zfs-import-zfs-primary.service"
];
before = [
"initrd-fs.target"
"sysroot-nix.mount"
];
unitConfig.DefaultDependencies = "no";
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
};
path = with pkgs; [ zfs ];
script = ''
key_file="/sysroot/crypto/keys/zfs-nix-store-key"
zfs load-key -L "file://$key_file" "ZFS-primary/nix"
'';
};
}; };
}; };
+9 -1
View File
@@ -207,6 +207,14 @@ in
}; };
}; };
}; };
rbw = {
enable = true;
settings = {
lockTimeout = 300;
pinentry = pkgs.pinentry-gnome3;
email = "snowinginwonderland@gmail.com";
};
};
}; };
services.gnome-keyring.enable = true; services.gnome-keyring.enable = true;
@@ -268,7 +276,7 @@ in
nextcloud-client nextcloud-client
bitwarden-cli bitwarden-cli
bitwarden-menu rofi-rbw-wayland
wtype wtype
obsidian obsidian
libreoffice-qt-fresh libreoffice-qt-fresh