The home lab grew up, so it needed backups

The home-lab plan now has enough real moving parts to be useful: the Raspberry Pi router, a Raspberry Pi access point, and a development server running containers such as Prometheus. That is also enough moving parts to make rebuilding from memory a bad recovery plan.

Configuration was scattered across the machines, so I built one backup routine with Ansible, scheduled it with a systemd timer, and sent the outcome to Discord. This follows naturally from the earlier lab build: once infrastructure survives beyond an experiment, recovery becomes part of the design.

Put every target in one inventory #

[routers]
router-pi ansible_host=192.168.1.1 ansible_user=admin
 
[access_points]
ap-pi ansible_host=192.168.1.10 ansible_user=admin
 
[dev_servers]
dev-server ansible_host=192.168.1.20 ansible_user=devuser
 
[all:vars]
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
backup_date="{{ ansible_date_time.date }}"
backup_base_dir="/home/{{ ansible_user }}/backups"

The router contributes dnsmasq, iptables, and WireGuard configuration. The access point contributes hostapd and dhcpcd. The development server contributes its Docker and monitoring configuration.

For a small set of files, Ansible’s copy module with remote_src: yes is easier to reason about than synchronize. The latter is excellent for large directory trees, but its rsync direction and SSH path resolution can become surprisingly theatrical.

Keep latest, snapshot changes #

The first version created a dated directory every run. That stored many identical copies, obscured when something had actually changed, and converted “backup history” into “folder archaeology.”

The revised policy is:

1. Always copy the current files into latest/.
2. Inspect Ansible’s registered results for a changed item.
3. Create a dated snapshot only when at least one file changed.

- name: Copy router configuration into latest
  ansible.builtin.copy:
    src: '{{ item }}'
    dest: '{{ backup_base_dir }}/latest/'
    remote_src: true
  register: router_backup_result
  loop: '{{ config_files }}'
  ignore_errors: true
 
- name: Record whether anything changed
  ansible.builtin.set_fact:
    router_changed: >-
      {{ router_backup_result.results
         | selectattr('changed', 'equalto', true)
         | list | length > 0 }}
 
- name: Save a dated snapshot after a change
  ansible.builtin.copy:
    src: '{{ backup_base_dir }}/latest/'
    dest: '{{ backup_base_dir }}/router-{{ backup_date }}/'
    remote_src: true
  when: router_changed

This is a good fit for small, infrequently changed configuration. Use synchronize when volume and file count make rsync worthwhile, and be explicit about which host initiates the transfer.

Schedule it like a service #

For a monthly job, systemd timers offer integrated journals, dependency ordering, persistent missed runs, randomized delays, and service-level failure handling.

# /etc/systemd/system/ansible-backup.timer
[Unit]
Description=Run Ansible Backup Monthly
Requires=ansible-backup.service
 
[Timer]
OnCalendar=monthly
Persistent=true
RandomizedDelaySec=600
 
[Install]
WantedBy=timers.target

Persistent=true runs a missed job after a machine comes back online. That matters in a home lab, where “scheduled downtime” often means somebody needed the power socket.

Discord receives a blue start message, a green success message with completion time and change count, or a red failure message with the relevant log excerpt. Notifications should make absence visible; they should not require a human to celebrate every successful copy.

Central orchestration, change-only snapshots, and observable scheduling turn a pile of configuration files into something recoverable. The next improvement is the one every backup article owes its readers: restore drills. A backup that has never been restored is merely an optimistic collection of bytes.