No More NixOS

Published: May 27, 2024

Title. For the past 6 years or so I’ve been running NixOS as my personal and professional OS, but as of a couple weeks ago, my desktop and laptop no longer run NixOS and instead are running a form of Fedora Atomic Desktops. When I was looking for my next OS, I really only wanted reversible system upgrades. All the other benefits of a declarative system I could forego, but I’d prefer to avoid having my system unusable after an update. Admittedly it’s been a long time since I’ve used an Nvidia GPU 😋 I didn’t use fancy btrfs snapshots or whatever because I’m too reckless to be mucking about on a filesystem.

The Next Setup

Fedora Atomic Desktops are Fedora based systems that have a read-only filesystem for OS layer packages and allow for A-B system switching (OSTree is the backing tech). And when I say “OS layer”, I do mean “layers”. It uses the same layering technology as Docker containers. Overall, yes, what we have now is just a weaker version of NixOS’ generation system that only supports the current and previous generation.

I created a personal variant of Fedora with my preferred software defined declaratively. This was built from a BlueBuild template which is pretty neat. It allows me to define my system in yaml and has GitHub workflows builtin to build my OS daily. This pushes a container to GitHub’s container registry which can replace the OS layer of my system… Yes, this is just a weaker variant of nixpkgs and environment.systemPackages.

So far I’m still running home-manager to manage user configs though I might slowly transition to chezmoi which has better integrations with BlueBuild. I’m not sure about support for things like creating custom symlinks, which I now must use to add cli util names into PATH. Disgustingly, I have a signal-desktop symlink pointing to /var/lib/flatpak/exports/bin/org.signal.Signal.

I also have a shitty setup script that bridges the gap between fresh install and mostly-configured system. In short, it installs nix and home-manager. Reminds me of my old scripts when I used to use Arch btw. I contemplated Ansible. And then I decided against it because I’m lazy.

Anywho “grass is always greener on the other side”. I’m pretty sure I just replaced an old set of problems with another set. Did you know, flatpaks have limited filesystem view so you get an assortment of issues like:

  • Steam can’t load non-Steam games
  • Signal can’t save images into your home folder

But guess what, I have Krita with krita-ai-tools installed. And I didn’t even have to create a nix derivation and recompile Krita! Admittedly, my solution of using an AppImage and having a disgusting-disgusting shell script in my PATH that calls APPDIR=/squashfs-root APPIMAGE=1 ~/.Applications/Krita/squashfs-root/AppRun could have been done in NixOS. Which is a recurring theme now that I’m on a container-focused OS. Containerized tools were always available on NixOS, but I was always in the headspace of creating a nix derivation. Whatever, I’ll be back to NixOS when they have wayland as a top-level config!

NixOS Retrospective

I think if you’re someone that manages multiple semi-snowflake systems, NixOS could be a good option for you. I could imagine plenty of time-save setting up services with something like:

services.sshd.enable = true

Additionally, there’s some security benefits, as all the pre-defined NixOS options properly set up firewalls for you. I would NOT suggest declarative secrets via something like sops-nix. It’s too much of a hassle transferring keys to a new system, especially since I don’t have a hardware key like Yubikey.

For me with nearly only one system and as someone who wants to try out new software and the interesting workflows they might introduce, I need to position myself to best to meet the tools where they are. I need to not have to spend a day writing a nix derivation to run software (Cascadeur). I need to not have Electron apps missing hardware acceleration (Plasticity). I need to be able to compile an imperatively installed python package with hardware acceleration (llama-cpp-python). I need to know that Err(RPC request RequestContact failed: invalid notification kind "ContactRequest") is a Linux issue and not a NixOS issue (Zed).

I’ve spent days of my life diagnosing and mostly failing to solve NixOS specific issues. We’ve been through a lot!

  • Babby’s first problem of patchelf-ing your typical Linux executable (This was before your modern niceties of buildFHSEnv, steam-run, nix-alien).
  • The GLIBC update(s) that I swear broke half the packages upstream. This same update also broke any old cached nix-shell environments.
  • The breaking change announced in… the nixos discourse and NO WHERE ELSE. I hope that’s hyperbole. This commit shouldn’t be unplanned work. NixOS has an unprofessional deprecation game.
  • A disk space crash, that corrupted passwd and parts of /nix/store/ which prevented NixOS userspace initialization.
  • Numerous nix derivations. God debugging is the worst. Edits to the postInstall phase requires re-compiling the program even if you only modify the postInstall phase. Imagine having to recompile Godot - minimally a 15 minute affair - because you didn’t git add a file that you were mv-ing into the output directory after the compilation phase.

Despite all the rigor of NixOS, I’d say I had a 50/50 chance of an update of my system working. This isn’t badly updated software. This is the upstream nixos definition breaking for one package or another. I repeat, I couldn’t update my system if any upstream package definition broke. Admittedly, I use the unstable channel of NixOS, but isn’t the allure of declarative package definitions the creation of pure functions that won’t break when you update a dependency in nixpkgs?

I know technically why this isn’t possible, but shouldn’t all packages in nixpkgs always compile and work with the caveat that it may be grossly outdated. You know… like a regular distro? I had numerous updates that I either had to wait for an upstream fix or exclude a package from my system to move forward. Yes I could pull the package from an alternative nixpkgs, but nix lang is a hassle.

On the flip side of all the issues, it’s fun to be possibly one of one person to be experiencing a problem.

The Transition

I have major sunk-cost fallacy with this one… NixOS is currently gaining a lot of traction and popularity. It’s a strange time to be leaving it. What if I’m leaving at the cusp of a UX revolution?

The very strange straw that broke the camel’s back was trying to use a Blender plugin that pip installed librosa. NixOS Blender doesn’t have pip installed. Blender would use a system installed version of librosa, but I needed a specific outdated version. Blender doesn’t use virtualenv.

I do not want to figure out whatever nix expression I need to include python packages in Blender, especially not with a specific old version of package. It’s probably a PYTHON_PATH edit away but… I’m just tired of NixOS problems and being completely unable to resolve them.

And on my new setup this Blender plugin just works!

Things I left at home:

  • My entire Firefox history. I thought it was stored in Firefox Sync. It is not.
  • My Windows VFIO VM libvirt xml config. I now have to play my gamba game in shitty resolution through waydroid…
  • My Krita shortcuts. Who would’ve thunk that wouldn’t be in the krita config dir?
  • My folder of internet cat pictures!!! 😞

I know what you’re saying, “Have you considered… backing up your home folder before reinstalling your OS?“. Well I have this bit where I don’t have any hard drive space and don’t back anything up so that I look informationally incompetent. So I gotta’ keep that schtick going. When does the persona I’m wearing become me though?

Welp, I’m done ranting about the highlights of the past half-decade.