Backing Up My Own Git Forge

I ran a self-hosted GitLab instance until fairly recently for many years; alas, it’s time to move on. GitLab is built for a company with a platform team behind it, and I’m bun a humble single developer. Most of what it ships — the whole CI/CD runner fleet, Geo, the enterprise-tier nags — is stuff I don’t use; it just sits there gigabytes upon gigabytes on ram on the VPS. Forgejo to the rescue! It does everything I need — hosting git, pull requests, pipelines, and a container and package registry, all in a nice small go runtime. Forgejo Actions doesn’t have all the bells and whistles of GitLab CI, but it’s good enough for me.

To avoid the whole migration ending in possible tears, I wanted a safety net: a local, restorable copy of every repository on the GitLab instance, in case moving them broke something I couldn’t undo. That itch turned into backup-git-repos.

What it backs up

#

Three backends, one config file: Forgejo and self-hosted GitLab instances, and a GitHub.com account, each just another entry under forges:, mixed freely in the same run. For every repository it finds:

• A bare mirror clone — every branch and tag, not just the default one — is kept under the forge’s own namespace structure on disk.
• Filtering by archived, active, or both, with an incremental refresh instead of a full re-clone on every run.
• Optionally, a .tar.gz alongside the mirror — a gzipped copy of the bare repository itself, not a working-tree checkout, so a restore from it still gets every branch and tag.
• Git LFS content fetched alongside the mirror, for whichever repositories actually use it.
• On GitLab, a project’s wiki and snippets too, which are their own git repositories the projects API never returns on its own.
• --prune-removed, to delete a mirror once its repository is gone upstream, so a years-long backup tree doesn’t keep every deleted or renamed repository forever.
• --export-metadata issues,releases,pull-requests: everything the repo doesn’t contain — issue and pull/merge request
• threads with their comments, and releases with their uploaded assets — written out as JSON alongside the mirror.

Developing this (Well, tbh, Claude did most of the work. I just wrote prompts, reviewed prs, and cursed at my screen), some interesting lessons were learnt. I’ll save you the “AI acting like an autistic slave” rant.

Real usage, real bugs

#

Once I pointed it at my actual GitLab instance instead of a test fixture, a run surfaced a run of small, honest bugs — the kind you only find by using a thing, not by reading its diff:

• A ~ in --dest created a literal directory called ~ instead of expanding to the home dir.
• A config listing more than one forge only ever used the first forge’s destination directory.
• Archived repositories were meant to be written out as an archive only, but were also left behind as a live mirror.
• Concurrency defaulted to 1 instead of the number of CPUs available, so a backup of a few hundred repositories took far longer than it needed to.

None of these were subtle — they’re the kind of thing a week of dogfooding turns up — but they’re also exactly the bugs that make you trust a backup tool less, and I wanted to nip issues in the bud. Some strategies that formed organically:

• config init to write a starter config instead of hand-typing YAML from the README
• a progress bar so a long run isn’t just a silent terminal,
• --dry-run to see what a run would do before it touches the disk
• a default --config path under $XDG_CONFIG_HOME so a second run doesn’t need the flag repeated.
• v1.2.0 through v1.4.0 are that whole arc — bugfixes and the UX that came out of finding them, back to back, over about a day and a half.

A backup that outlives the forge

#

A .git directory on the same disk as everything else isn’t a backup, it’s a copy — so v1.3.0 added a multi-arch container image, hardened to run as non-root with every capability dropped, so the tool can sit on its own schedule.

What actually matters more is that a tool built to move other people’s histories around gets tested against real histories, not mocked HTTP responses — a nasty surprise here means a broken backup nobody notices until they need it. CI boots a real Forgejo container on every push, and a real GitLab CE container nightly — GitLab CE is too heavy to boot on every push, so it gets its own slower schedule instead. It’s caught real bugs, not just given false confidence (can you tell AI generates this part of the post): a real concurrent run against many repositories surfaced a race where sibling clones sharing a not existing parent directory could fail with could not create leading directories, and the nightly GitLab lane once started failing outright the same day a new GitLab CE point release rejected a discussion-seeding request the test suite had relied on. The one exception is three of the GitLab LFS tests, which are skipped in that nightly run rather than fixed outright: GitLab CE’s LFS backend isn’t reliably warm by the time the container reports ready, so a push racing right behind it flakes on an LFS client error before the code under test ever runs. Everything else in the GitLab suite still runs for real, every night, against a real GitLab.

The bigger gap was what a bare mirror clone actually captures. It’s every branch and tag, which is most of a repository — but not a GitLab project’s wiki or snippets, which are their own git repositories the projects API never returns; not Git LFS content, which lives outside the object store a mirror clone pulls down; and not a repository that’s been deleted or renamed upstream, which a mirror just silently keeps forever unless you tell it not to. Late August added all three: wiki and snippet mirroring for GitLab (fetched concurrently across projects once the naive serial version turned out slow enough to notice), LFS content fetched alongside the mirror wherever a repository actually uses it, and --prune-removed to delete a mirror once its repository is well and truly gone.

Then the same question one level up: a bare mirror is everything in git, but a GitLab or Forgejo project isn’t only git. Issues, releases, and merge requests live in the forge’s own database, not the repository history, and none of them survives a mirror clone on its own. --export-metadata issues,releases,pull-requests writes each of those out as JSON alongside the mirror — one file per issue or request, comments and review threads included, release assets downloaded rather than just linked. That’s v1.5.0 through v1.10.0, and it’s the point where the tool stopped being “a faster git clone --mirror in a loop” and started being an actual answer to “what does this forge know about my repository that git doesn’t.”

What it looks like

#

No subcommands to memorize before it does anything useful — --help covers the whole surface, and config init gets a starter config on disk without copying YAML out of a README. Both are captured by actually running the binary, not typed out from memory:

Back up git repositories from GitLab, Forgejo, and GitHub

Usage:
  backup-git-repos [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  config      Manage the backup-git-repos config file
  help        Help about any command
  list        Print what would be backed up, clone nothing
  run         Mirror repositories from the configured forges
  version     Print the version

Flags:
  -c, --config string   config file (default: $XDG_CONFIG_HOME/backup-git-repos/config.yaml)
  -h, --help            help for backup-git-repos

Use "backup-git-repos [command] --help" for more information about a command.

wrote config to /home/ryan/.config/backup-git-repos/config.yaml

Installing it

#

Pick whichever fits how you run things — README (linked below) has the full detail on configuration and flags.

Arch Linux, from the AUR:

git clone https://aur.archlinux.org/backup-git-repos-bin.git
cd backup-git-repos-bin
makepkg -si

Or with a helper: paru -S backup-git-repos-bin / yay -S backup-git-repos-bin.

Debian/Ubuntu and Fedora/RHEL, from the latest release:

# Debian/Ubuntu
sudo dpkg -i backup-git-repos_*_linux_amd64.deb

# Fedora/RHEL
sudo rpm -i backup-git-repos_*_linux_amd64.rpm

Docker, a natural fit for a scheduled job:

docker run --rm \
  --read-only --tmpfs /tmp \
  --cap-drop=ALL --security-opt=no-new-privileges \
  -v /srv/backups/git:/srv/backups/git -v ./config.yaml:/config.yaml:ro \
  ghcr.io/alrayyes/backup-git-repos:latest run --config /config.yaml

Nix and NixOS:

nix run github:alrayyes/backup-git-repos       # try it
nix profile install github:alrayyes/backup-git-repos  # keep it

go install, or anywhere none of the above fits:

go install github.com/alrayyes/backup-git-repos/cmd/backup-git-repos@latest

The repository has the full README, including token scopes per forge and the config file layout. If you use it and find — or don’t find — it useful, let me know what you think.