this post was submitted on 04 Jul 2023
93 points (97.9% liked)

Selfhosted

39950 readers
638 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

I'm thinking about starting a self hosting setup, and my first thought was to install k8s (k3s probably) and containerise everything.

But I see most people on here seem to recommend virtualizing everything with proxmox.

What are the benefits of using VMs/proxmox over containers/k8s?

Or really I'm more interested in the reverse, are there reasons not to just run everything with k8s as the base layer? Since it's more relevant to my actual job, I'd lean towards ramping up on k8s unless there's a compelling reason not to.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 12 points 1 year ago (3 children)

What I did is install proxmox on the bare metal, setup a vm in which I put the containers.

Proxmox itself stays (almost) completely stock. The only changes I've made to it were to add the NUT client package so it could gracefully shut down if my NUT server indicates that the UPS is running out of power during an outage.

In your VMs you can do whatever. Setup OMV, or a stock Ubuntu or Debian vm and install your services on the VM or use Docker/Podman. Setup Fedora CoreOS or IoT vms and host all your services in Podman containers.

The great thing about Proxmox is you can do snapshot backups which take mere moments to complete. Then pass those off to a NAS where they can survive a irreparable loss of your Proxmox server.

You can also spin up new vms as needed to just try to fuck around with new techs or just a new way of setting up your home lab. It gives you a ton of flexibility and makes backing stuff up way easier.

Another great thing you can do is if 3 years down the line you are looking to replace your server hardware with some newer or more powerful stuff you can just add the new device as a node to the cluster. Then you can migrate all your existing VMs over to your new hardware and decommission your old one with very little to no downtime on anything.

[–] [email protected] 4 points 1 year ago

The great thing about Proxmox is you can do snapshot backups which take mere moments to complete. Then pass those off to a NAS where they can survive a irreparable loss of your Proxmox server.

Hopefully you put a giant asterix by this point. You need the snapshot AND the original backup. Snapshots are only diffs and can't survive without their base backup.

[–] [email protected] 3 points 1 year ago (1 children)

This is my exact setup as well. Proxmox with one beefy vm dedicated just to docker and then a few other vms for non docker workloads (eg, home assistant, pihole, jelltfin). I can probably run those in docket as well, but the to worked better as vms when I set them up

[–] [email protected] 1 points 1 year ago

Appreciate your take on this and specifically mentioning that you have a VM for Home Assistant. That was a lightbulb moment for me as I like how easy it is to manage updates as an OS install rather than in a Docker container. If I ever get around to rebuilding my server architecture I'm definitely going to do this!

[–] [email protected] 1 points 1 year ago

I have a similar setup, but 2 VMs on each of my 2 servers, then on server 1, I have VM A running one test K3s node and VM B running one live (Production) K3s node with the same on server 2, so I can take one server full down for maintenance, but keep my test and live sites running. It's way overkill, but allows me to learn about how to set up and maintain resilient systems. One day, I'll do the same for my network :-(