diff --git a/README.md b/README.md deleted file mode 100644 index 0471024..0000000 --- a/README.md +++ /dev/null @@ -1,107 +0,0 @@ -μVMs -=== - -Slowly comprehending [spectrum-os](https://spectrum-os.org) and [microvm.nix](https://github.com/microvm-nix/microvm.nix), by reproducing bits and pieces using [NixOS](https://nixos.org), [systemd](https://nixos.org/manual/nixos/stable/options#opt-systemd.services), and [cloud-hypervisor](https://github.com/cloud-hypervisor/cloud-hypervisor). - -Prior Art ---- - -### People - -Following the right people is one of the most effective ways to maintain bibliography, a form of "importance sampling". -The following are the people whose work, at the time of writing, I already know to watch out for. Whom am I missing? - -- [Alyssa Ross](https://github.com/alyssais) - - Via [spectrum.org]https://spectrum-os.org/bibliography.html) and [Nixpkgs](https://github.com/NixOS/Nixpkgs). -- [Demi Marie](https://demimarie.github.io): - - Via [spectrum-devel](https://spectrum-os.org/lists/archives/spectrum-devel/2ff17d00-6603-46ae-9eb0-d4fa179db86c@gmail.com/) and Qubes. -- [Thomas Leonard](https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kvm-and-wayland) - - Via [spectrum-discuss](https://spectrum-os.org/lists/archives/spectrum-discuss/CAG4opy8BZn2pXDRBHOjcENFBHJON1LoG7A8GPdP0Wt_3KLaHyw@mail.gmail.com/), [qubes-lite, and `wayland-proxy-virtwl`](https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kvm-and-wayland). -- [Astro](https://spaceboyz.net/~astro/): - - Via [microvm.nix](https://github.com/microvm-nix/microvm.nix) -- [Jean-Philippe](https://github.com/jpouellet): - - Via [Nixpkgs](https://github.com/NixOS/Nixpkgs), [Qubes](https://doc.qubes-os.org/en/latest/developer/general/devel-books.html), and [Genode](https://genode.org/about/challenges). -- [Joanna Rutkowska](https://blog.invisiblethings.org/papers): - - Via [Qubes](https://doc.qubes-os.org/en/latest/developer/general/devel-books.html) [OS](https://theinvisiblethings.blogspot.com/2012/09/how-is-qubes-os-different-from.html). -- ... - -### Projects - -- Obvious: Qubes, Spectrum, Genode, ... -- Google ChromeOS and [AWS Firecracker](https://firecracker-microvm.github.io): - - rust-vmm, [crosvm](https://crosvm.dev/book), and [sommelier](https://chromium.googlesource.com/chromiumos/platform2/+/master/vm_tools/sommelier); -- [Asahi muvm](https://github.com/AsahiLinux/muvm): microvms using [libkrun](https://github.com/containers/libkrun) and virtio-gpu "native context". -- [AppVM](https://github.com/jollheef/appvm): apparently available in Nixpkgs under `nixos/modules/virtualisation/appvm.nix`. Based on (NixOS,) qemu and libvirt. -- ... - - -### Timeline - -The following are the questions I'd like to eventually answer about how virtualization happened: - -- [ ] Was Qubes the first attempt at isolating e.g. peripheral and network devices using virtualization? -- [ ] Was Chromium OS the first and (or) the main driver for paravirtualized devices? -- ... - -### Choices - -The following are some of the current "am I holding this right?" questions: - -- [ ] Memory: it is said that Linux always needs some swap in order not to be weird; - what is the general scheme for allocating memory and swap to the hypervisor and the guests? - - - [x] It seems that hotplug memory is generaly preferred ot ballooning. - - [x] It seems that we want the hotplug "banks" to not be too small, - as to avoid fragmentation. - - [ ] Zswap or zram on the hypervisor? On the guest? - - [ ] How much swap to allocate on the hypervisor? On the guest? - - > Context: - > most of the day I'm using a laptop with 8GiB of RAM; this necessitates frequent OOMs, - > even when not running VMs. -- [ ] Startup time: [lightvm](https://dl.acm.org/doi/10.1145/3132747.3132763) claims boot time of 2.3ms; - can we ever achieve comparable numbers with NixOS and systemd? What is the first bottleneck? -- [ ] Guest-to-guest communication: in order to implement configurations similar to Whonix (e.g. Tor in a separate VM), - guests need to be able to talk to each other directly, without exposing the hypervisor to their traffic. - Generally, I've heard of three solutions to guest-to-guest communication: - - - NAT via hypervisor, - - [MACVTAP](https://microvm-nix.github.io/microvm.nix/interfaces.html#type--macvtap), - - and [vhost-device-vsock](https://github.com/rust-vmm/vhost-device/blob/main/vhost-device-vsock/README.md#sibling-vm-communication). - - I've only ever implemented the first. - The second is something of a bridge, but the packets (frames?) never enter the hypervisor's network stack. - I've only recently learned about the third: mentioned by Alyssa in the spectrum matrix chat. - I do not entirely understand how it fits into the bigger picture. -- [ ] Filesystems: - - Virtio-blk appears to be the way to allocate persistent storage for VMs that require it. - In practice this means allocating a zvol or a contiguous file on the hypervisor, - to be exposed to the guest as a block device. One suspicion I have is that allocating - a CoW filesystem (e.g. xfs, btrfs) on top of another CoW filesystem (e.g. zfs) may have non-trivial - implications for fragmentation, depending on parameters like the chunksizes. - On the other hand, use-cases like `/nix/store` offer many deduplication opportunities, - and I'm generally not using `snix-store` just yet. -- [ ] ... - -### Why Not? - -#### `spectrum-os` - -...is in active development and not advertised as user-ready yet. -Spectrum OS appears to be a balance-shifting project, building up towards a principled solution, which must require patience... -It does not, for example, reuse NixOS systemd modules, but uses s6 instead. - -#### `microvm.nix` - -Is inherently static. -A cynical spin on `microvm.nix` would be, and I mean it with utmost respect, that it's a glorified qemu flags generator, written in Nix. -When using `microvm.nix` you write, for example, each TAP's `hwaddr` by hand, and the rebuild the "runner script". -When using the "fully-declarative mode" you also engangle the guest's and the hypervisor's life cycles, and double the NixOS evaluation time. -Microvm-nix ships support for a wide selection of different hypervisors, but you may only care about e.g. `cloud-hypervisor`. -An instructive reference implementation and a convenient entry point, `microvm.nix` may not be a direct or complete answer to the question "what does a life cycle of a microvm-deployed service look like". - -#### appvm - -I only noticed the option in `man configuration.nix` a few days ago, -so I just never tried. Long-term I'd definitely prefer not to use qemu. diff --git a/examples/dummy.nix b/examples/dummy.nix index 4689af1..24b41e1 100644 --- a/examples/dummy.nix +++ b/examples/dummy.nix @@ -26,7 +26,7 @@ neededForBoot = true; }; boot.initrd.systemd.settings.Manager.DefaultTimeoutStartSec = 30; - systemd.settings.Manager.DefaultTimeoutStopSec = 10; + systemd.settings.Manager.DefaultTimeoutStopSec= 10; networking.useNetworkd = true; networking.nftables.enable = true; diff --git a/profiles/resources.nix b/profiles/resources.nix index 7e1cbd7..3c19727 100644 --- a/profiles/resources.nix +++ b/profiles/resources.nix @@ -1,9 +1,4 @@ -{ - config, - options, - lib, - ... -}: +{ config, options, lib, ... }: { config = lib.mkMerge [