Howdy 👋

Recently, I started getting the dreaded emails from smartd on one of my Proxmox nodes. The drive was (slowly) failing and needed to be replaced. Unfortunately due to my nodes being micro form-factor(MFF), they only have a single drive. This means that I would also need to migrate the VM disks to the new drive too. As this is a homelab setup, I don’t have a ton of capacity or any fancy storage solutions, so I would need to get a little creative and replace the drive in-situ....

A little while back, I wrote a short piece on integrating Cilium with OPNsense using BGP. With more recent releases of Cilium, the team have introduced the Cilium BGP Control Plane (currently as a beta feature). This reworking of the BGP integration replaces the old MetalLB-based control plane and as such the older feature must first be disabled. To enable the new feature, you can either pass an argument to Cilium:...

Running immutable infra is the holy grail for many people, however there are times when you’ll need to get down in the weeds in order to troubleshoot issues. Let’s imagine a scenario; you need to verify that a pod is receiving traffic, but the image is built FROM scratch. As scratch containers are as minimal as possible, there’s no shell in the image, so there’s no way you can exec into it and hope to do anything remotely useful....

I’ve recently been experimenting with Hashicorp’s Consul in my home infrastructure because I want to use it to provide service discovery and automatic DNS provisioning when I create Proxmox instances with Terraform. Consul is a bit of a hefty beast to get to grips with and getting DNS lookups working when you have ACLs enabled can be a little tricky - it’s taken me a day or two of going round in circles to figure this one out....

Update: Cilium now has a new BGP integration so things have changed a little; see this post for more details. If (like me) you happen to follow the development of the Cilium CNI plugin for Kubernetes then you’ll have seen the recent 1.10 release which included many shiny features. One exciting addition is the ability to announce Service IPs via BGP. Running Kubernetes in a homelab environment quickly highlights that there are some aspects which are a little lacking when compared to the integration you get from the cloud provider offerings....

This blog is going to be about using Hashicorp’s Vault to issue short-lived certificates to use with SSH. Most guides have you using a username & password to authenticate with Vault, but I’ve chosen to delegate that to GitHub instead. I’m assuming you already have a Vault server running - I won’t be covering that in the course of this blog. You’ll also need a sufficiently-privileged Vault token, and jq installed on the machine you wish to SSH from....

Running Thanos without Kubernetes If you’ve been around the cloud-native world for a while, you’ll no doubt be familiar with (and quite likely already be using) Prometheus. You may however not have heard of Thanos. Put simply, Thanos takes Prometheus and makes it even more awesome. In their own words, the high-level description of Thanos is the following: Thanos is a set of components that can be composed into a highly available metric system with unlimited storage capacity, which can be added seamlessly on top of existing Prometheus deployments....

Storing Terraform’s remote state in Minio Whilst AWS’s free S3 tier is almost certainly sufficient to store Terraform’s remote state, it may be the case that you have a requirement to keep the data on-site, or alternatively if you’re using Terraform in an air-gapped environment then you have no choice but to self-host. Enter Minio. If you’ve not used it before, the TLDR is that Minio provides an S3-compatible API in a single binary....

Obtaining a wildcard LetsEncrypt cert with Ansible Earlier this year, LetsEncrypt made their wildcard x509 certificates available to the general public. Whilst this is a massive step forward over individual certificates for each domain, it does come with the overhead of having to distribute the wildcard certificate to the (possibly many) places you would use it. Ignoring that issue for now, I wrote a quick Ansible playbook which uses the dns-01 challenge method and my Memset DNS management modules (available in Ansible 2....

Lessons I learnt whilst developing modules Having now spent quite some time working on my initial Ansible modules for Memset, I’ve assembled some handy hints on areas which tripped me up at various times in my journey. It should be noted that this post is written from the point of view of someone who is not a developer and is therefore not as au fait with some of the processes mentioned as others may be....