Continuous Deployment to Cloudflare Pages from a SourceHut repository

SourceHut Pages

In my opinion, publishing to SourceHut Pages is very easy.

tar zc index.html \
  | curl --oauth2-bearer $srhttoken \
         -Fcontent=@/dev/stdin \
         https://pages.sr.ht/publish/sqwishy.srht.site

I don’t have to install anything scrupulous with npm. It uses existing things I’m familiar with.

One irritation about SourceHut in general is that they have two different things called Personal Access Tokens. One at https://meta.sr.ht/oauth; another at https://meta.sr.ht/oauth2. The former is “legacy”. But the non-legacy API still lacks features that exist in the legacy API[2].

They use different credentials but both are called Personal Access Tokens. They aren’t interchangeable and if you use the wrong Personal Access Token, you don’t always get a clear error back.

Another important consideration is that this little webshit I made is super useful and everyone is going to love it for sure and it’s going to get loads of visits. Probably top ten websites on the internet.

But the data and images are about 500 kB with compression. So, should I put it on SourceHut Pages if the traffic is absolutely going to cause service degradation for other users? I asked on IRC and the only reply was:

If you have to ask about usage limits, that’s generally a good indicator that you might want something different.

Though, that was exactly the feeling I had that compelled me into asking the question to begin with. SourceHut Pages is free – and not free as in email. Isn’t asking about my obligations part of being a responsible user of the service? Or is it evidence that I don’t belong. ¯\_(ツ)_/¯

I didn’t say anything to them, but it was like I had just walked into a restaurant and been handed a menu with no prices. And, when asking about what the food costs, the server, almost looking off and discretely screening themselves from second-hand embarrassment, peers at me with judgement and explains that, if I have to ask, then I’m at the wrong restaurant. And after a moment of silence for my dignity, politely moves on as if, for my sake, I had never asked such a fucking stupid question.

Cloudflare Pages

To upload static assets to Cloudflare Pages, they advise using a JavaScript program called wrangler (or manually upload zip files to a web form). It’s one of those programs that vomits emojis in every line of output to keep you engaged. It seems very engineered and I couldn’t find a simpler way using curl so I conceded I’d run wrangler in a container and avoid letting it near anything with an emoji allergy.

SourceHut Webhooks

I’m hosting the source code for my webshit on git.sr.ht, which has webhooks. The documentation for this is a little bit wild[3]; there are two main resources:

• Information about webhooks across SourceHut services generally.

• Information about the webhooks for git.sr.ht specifically.

At each respective page linked above they describe these URL patterns:

• /api/.../webhooks

• /api/:username/repos/:name/...

Put them together and prefix your username with ~. For example, this creates a webhook for my repository named europan-materialist:

jo url=https://butt.froghat.ca/europan-materialist \
   events[]=repo:post-update \
   | curl --oauth2-bearer $srhttoken \
          --json @- \
          https://git.sr.ht/api/~sqwishy/repos/europan-materialist/webhooks

On success, the request’s response contains an object representing the webhook you created.

Or we can query the list of webhooks with:

$ curl --oauth2-bearer $srhttoken \
    https://git.sr.ht/api/~sqwishy/repos/europan-materialist/webhooks

… which replies …

{
  "next": null,
  "results": [
    {
      "id": 28961,
      "created": "2023-11-17T13:50:06+00:00",
      "events": [
        "repo:post-update"
      ],
      "url": "https://butt.froghat.ca/europan-materialist"
    }
  ],
  "total": 1,
  "results_per_page": 50
}

butt.froghat.ca

I’m using git.sr.ht to host my project but I want to mirror it to GitHub for the social credit. So the webhooks at SourceHut POST to a Python script in my “infrastructure” that does two things.

• Mirror the repository to GitHub.

• Build and upload the site to Cloudflare Pages.

This is broken down into three oneshot services in systemd that the script activates through a target in systemd.

europan-materialist-fetch.service updates the remote tracking branches in a bare/mirror git repository. It is RequiredBy and Before both the mirror and deploy steps.

europan-materialist-mirror.service pushes that bare/mirror repository to GitHub.

europan-materialist-deploy.service updates its own checkout of the local mirror and builds and runs a container in podman that deploys to Cloudflare Pages.

The europan-materialist-cd.target is activated over dbus when the Python script at butt.froghat.ca gets POSTed a webhook. This target Requires both mirror and deploy steps so they are started when the target is activated. And since the mirror and deploy steps have Requires and After to the fetch step, systemd’s ordering ensures the local mirror at /mirror is updated successfully before the steps that need it are run in parallel.

These are files.

# ~/.config/systemd/user/europan-materialist-cd.target
[Unit]
Requires=europan-materialist-deploy.service
Requires=europan-materialist-mirror.service

# ~/.config/systemd/user/europan-materialist-deploy.service
[Unit]
Requires=europan-materialist-fetch.service
After=europan-materialist-fetch.service

[Service]
Type=oneshot
WorkingDirectory=%h/europan-materialist
ExecStartPre=/usr/bin/git fetch --prune
ExecStartPre=/usr/bin/git checkout FETCH_HEAD
ExecStartPre=/usr/bin/podman build -f Containerfile \
        -v %h/Barotrauma/Content:/Content:ro \
        --tag europan-materialist
ExecStart=/usr/bin/podman run \
        --rm \
        -v %h/cfkeys:/run/secrets/wrangler:ro \
        europan-materialist \
        ash -c 'npm x -- vite build && npm install wrangler && env $$(cat /run/secrets/wrangler) npm x -- wrangler pages deploy --project-name materialist-next ./dist'

# ~/.config/systemd/user/europan-materialist-mirror.service
[Unit]
Requires=europan-materialist-fetch.service
After=europan-materialist-fetch.service

[Service]
Type=oneshot
ExecStart=/usr/bin/git -C /mnt/kaput/projects/europan-materialist.git push --mirror github

# ~/.config/systemd/user/europan-materialist-fetch.service
[Service]
Type=oneshot
ExecStart=/usr/bin/git -C /mnt/kaput/projects/europan-materialist.git fetch --prune origin

That’s it. It hasn’t fallen over yet. Last time it ran it finished in about thirty seconds. Nearly a third of that is the emoji program. Here’s even some logs from it as bonus.

Nov 20 13:04:36 banana podman[352347]: Uploading... (15/15)
Nov 20 13:04:36 banana podman[352347]: ✨ Success! Uploaded 3 files (12 already uploaded) (0.95 sec)
Nov 20 13:04:42 banana podman[352347]: ✨ Deployment complete! Take a peek over at https://3fccd3c7.materialist-next.pages.dev

The website is served at materialist.pages.dev. The code is on GitHub and SourceHut.