I’m a new trigger.dev user and I am ready to promote my development env to production. So I run this command
bunx trigger.dev@latest deploy --api-url https://trigger.sbtp.xyz
Unfortunately, I see an error.
(node:500706) Warning: `--localstorage-file` was provided without a valid path
(Use `node --trace-warnings ...` to show where the warning was created)
Trigger.dev (4.4.6)
------------------------------------------------------
┌ Deploying project
│
◇ Retrieved your account details for chris@grimtech.net
│
◇ Successfully built code
│
◇ Failed to deploy project
│
└ Error: Error building image. Full build logs have been saved to /tmp/trigger-qBS0ff/build-3ihrfcab.log
Last few lines of logs:
------
> exporting to image:
------
ERROR: failed to build: failed to solve: failed to fetch anonymous token: unexpected status from GET request to https://ghcr.io/token?scope=repository%3Atrigger%2Fproj_zlvybyxzqtafjwonlxid%3Apull%2Cpush&service=ghcr.io: 403 Forbidden
I’m stumped here, so I created a discussion on triggerdotdev github. https://github.com/triggerdotdev/trigger.dev/discussions/3645
A few days later, I think I figured out the issue. I was thrown off by the following mention in the docs.
Built-in container registry and object storage. You can now deploy and execute tasks without needing third party services for this.
I think this isn’t true for self-hosting. For self-hosting in production, you have to configure a docker registry using env vars.
https://trigger.dev/docs/self-hosting/docker#registry-setup
so I deployed a docker registry (https://distribution.github.io/distribution) on my Coolify.
Then I needed to ensure the following env vars were present in my docker-compose.yaml file, under the trigger service. I’m using coolify so these were already in the docker-compose.
- 'DEPLOY_REGISTRY_HOST=${DEPLOY_REGISTRY_HOST:-ghcr.io}'
- 'DOCKER_REGISTRY_URL=https://${DEPLOY_REGISTRY_HOST:-ghcr.io}'
- 'DOCKER_REGISTRY_USERNAME=${DEPLOY_REGISTRY_USERNAME}'
- 'DOCKER_REGISTRY_PASSWORD=${DEPLOY_REGISTRY_PASSWORD}'
Then I made sure that I had DEPLOY_REGISTRY_HOST, DOCKER_REGISTRY_USERNAME, and DOCKER_REGISTRY_PASSWORD set in my environment variables section.
Then I had to auth. I’m self-hosting gitea which has a container registry. I used my gitea user account
docker login -u <username> gitea.sbtp.xyz
At first, I was seeing a failure, and I was dismayed.
$ bunx trigger.dev@latest deploy --api-url https://trigger.sbtp.xyz
(node:2744271) Warning: `--localstorage-file` was provided without a valid path
(Use `node --trace-warnings ...` to show where the warning was created)
Trigger.dev (4.4.6)
------------------------------------------------------
┌ Deploying project
│
◇ Retrieved your account details for chris@grimtech.net
│
◇ Successfully built code
│
◇ Failed to deploy project
│
└ Error: Error building image. Full build logs have been saved to /tmp/trigger-0qnQ3R/build-lgeyf630.log
Last few lines of logs:
4.266 error: unexpected status from HEAD request to https://registry.sbtp.xyz/v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/sha256:cc3cbd41b1b229178fd92114af9f51857a04481895bb8dfcc13174fbecfbd1b1: 503 Service Unavailable
4.266 retrying in 4s
8.026 error: unexpected status from HEAD request to https://registry.sbtp.xyz/v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/sha256:5c288477bc58cad135aaff9b27580f617e25daa92f171dca3610c984d2af6104: 503 Service Unavailable
------
ERROR: failed to build: failed to solve: failed to push registry.sbtp.xyz/trigger/proj_zlvybyxzqtafjwonlxid:20260520.3.production.lgeyf630: unexpected status from HEAD request to https://registry.sbtp.xyz/v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/sha256:5c288477bc58cad135aaff9b27580f617e25daa92f171dca3610c984d2af6104: 503 Service Unavailable
error: script "deploy" exited with code 1
Important
This is a complex system. To get things working, it’s often easier to break down big problems into smaller parts
Before we attempt another trigger deploy, we first want to make sure we can push to the registry. To do this, we try pushing an arbitrary docker image.
I just happened to have redis cached locally, so I tagged it for our registry and pushed it.
docker tag redis:6-alpine registry.sbtp.xyz/redis:6-alpine
docker push registry.sbtp.xyz/redis:6-alpine
Failure!
➜ lufs2 git:(main) ✗ docker push registry.sbtp.xyz/redis:6-alpine
The push refers to repository [registry.sbtp.xyz/redis]
f546396032e0: Unavailable
902184b21654: Unavailable
bc07708ca400: Unavailable
f75c6e138866: Unavailable
c14c7509f545: Unavailable
897d797d2723: Unavailable
4f4fb700ef54: Unavailable
unknown: unexpected status from HEAD request to https://registry.sbtp.xyz/v2/redis/blobs/sha256:902184b21654d047c1a6a0fa8436a5b921edc64e85ca1440e20f9c981d7dd610: 503 Service Unavailable
Our gitea container registry isn’t accepting our image. We will need to fix this before the trigger.dev deployment will succeed.
@todo insert solution
omg I realized that I was using the wrong domain! it’s gitea.sbtp.xyz, not registry.sbtp.xyz. So that means I need to re-tag and push
docker tag redis:6-alpine gitea.sbtp.xyz/redis:6-alpine
docker push gitea.sbtp.xyz/redis:6-alpine
The push refers to repository [gitea.sbtp.xyz/redis]
902184b21654: Unavailable
f546396032e0: Unavailable
bc07708ca400: Unavailable
f75c6e138866: Unavailable
4f4fb700ef54: Unavailable
897d797d2723: Unavailable
c14c7509f545: Unavailable
error from registry: unauthorized
unauthorized
Different error. This is progress!
Note
Errors and failures are not bad things. They are information!
Digging around the docs, I realized I had the docker tag format wrong too! (gitea container registry docs say the format should be docker push gitea.example.com/{owner}/{image}:{tag} like the following.
docker tag redis:6-alpine gitea.sbtp.xyz/grimtechnet/redis:6-alpine
docker push gitea.sbtp.xyz/grimtechnet/redis:6-alpine
The push refers to repository [gitea.sbtp.xyz/grimtechnet/redis]
902184b21654: Pushed
c14c7509f545: Pushed
897d797d2723: Pushed
f546396032e0: Pushed
bc07708ca400: Pushed
4f4fb700ef54: Pushed
f75c6e138866: Pushed
6-alpine: digest: sha256:e8773aa976bf0e4ca3e7707b759110c893be504da1f25a61bb4000d1d56b7bd3 size: 2286
i Info → Not all multiplatform-content is present and only the available single-platform image was pushed
sha256:bc315e7e61ccee683c1760621a85463b3c5cbbf1ca0f5c01e063da8ed5f077ad -> sha256:e8773aa976bf0e4ca3e7707b759110c893be504da1f25a61bb4000d1d56b7bd3
Success!
Next I updated my env vars in my trigger.dev project in my Coolify.
DEPLOY_REGISTRY_HOST=gitea.sbtp.xyz
DEPLOY_REGISTRY_PASSWORD=xxx
DEPLOY_REGISTRY_USERNAME=xxx
These values I set to my gitea credentials. Since I use 2FA in my gitea, I had to use a gitea application token as the password.
I re-deployed the project on my Coolify.
ERROR: failed to build: failed to solve: failed to push gitea.sbtp.xyz/trigger/proj_zlvybyxzqtafjwonlxid:20260520.6.production.3q052w20: unexpected status from POST request to https://gitea.sbtp.xyz/v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/: 404 Not Found: Not found.
Another error, but the error is different this time. Exciting times!
Let’s try another trigger deploy.
bun run deploy
$ bunx trigger.dev@latest deploy --api-url https://trigger.sbtp.xyz
(node:2792995) Warning: `--localstorage-file` was provided without a valid path
(Use `node --trace-warnings ...` to show where the warning was created)
Trigger.dev (4.4.6)
------------------------------------------------------
┌ Deploying project
│
◇ Retrieved your account details for chris@grimtech.net
│
◇ Successfully built code
│
◑ Building version 20260520.7 (local) View deployment: #21 exporting to image..◒ Building version 20260520.7 (local) View deployment: #21 exporting to image..◐ Building version 20260520.7 (local) View deployment: #21 exporting to image..◓
<message repeated many times>
│
└ Error: Error building image. Full build logs have been saved to /tmp/trigger-9MihHY/build-ofqd2j2d.log
Last few lines of logs:
194.8 error: failed to copy: unexpected status from PUT request to https://gitea.sbtp.xyz/v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/1w4rw4hwavrl8as8eenm0lihm?digest=sha256%3A2704302743ed6f48f2647fafc9ddf87cef27f23baf700193e0f8a1147e72547e: 500 Internal Server Error
194.8 retrying in 4s
255.1 error: failed to copy: unexpected status from PUT request to https://gitea.sbtp.xyz/v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/4pmbxtsxg6pn7xowewxmylc8q?digest=sha256%3Aee5f447b414e35eb93aa9645b225df21e7a6b6a5b4f77b1fb4b24f650298efbf: 500 Internal Server Error
------
ERROR: failed to build: failed to solve: failed to push gitea.sbtp.xyz/trigger/proj_zlvybyxzqtafjwonlxid:20260520.7.production.ofqd2j2d: unexpected status from PUT request to https://gitea.sbtp.xyz/v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/4pmbxtsxg6pn7xowewxmylc8q?digest=sha256%3Aee5f447b414e35eb93aa9645b225df21e7a6b6a5b4f77b1fb4b24f650298efbf: 500 Internal Server Error
error: script "deploy" exited with code 1
Hmm.. Internal error on the gitea side! Let’s look at the gitea logs.
2026-May-20 12:36:01 2026/05/20 12:36:01 HTTPRequest [I] router: completed GET / for 127.0.0.1:40690, 200 OK in 4.8ms @ web/home.go:32(web.Home) 2026-May-20 12:36:03 2026/05/20 12:36:03 .../container/container.go:433:PutBlobsUpload() [E] Package registry API internal error: 500 unexpected EOF 2026-May-20 12:36:03 2026/05/20 12:36:03 HTTPRequest [I] router: completed PUT /v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/4pmbxtsxg6pn7xowewxmylc8q?digest=sha256%3Aee5f447b414e35eb93aa9645b225df21e7a6b6a5b4f77b1fb4b24f650298efbf for 98.97.37.176:0, 500 Internal Server Error in 60000.9ms @ container/container.go:411(container.PutBlobsUpload) 2026-May-20 12:36:03 2026/05/20 12:36:03 HTTPRequest [I] router: completed POST /v2/token for 98.97.37.176:0, 404 Not Found in 2.9ms @ container/container.go:201(container.AuthenticateNotImplemented) 2026-May-20 12:36:03 2026/05/20 12:36:03 HTTPRequest [I] router: completed GET / for 127.0.0.1:40706, 200 OK in 2.8ms @ web/home.go:32(web.Home) 2026-May-20 12:36:03 2026/05/20 12:36:03 HTTPRequest [I] router: completed GET /v2/token?scope=%2A%3A%3A&scope=repository%3Atrigger%2Fproj_zlvybyxzqtafjwonlxid%3Apull%2Cpush&service=container_registry for 98.97.37.176:0, 200 OK in 21.5ms @ container/container.go:165(container.Authenticate) 2026-May-20 12:36:03 2026/05/20 12:36:03 .../container/container.go:433:PutBlobsUpload() [E] Package registry API internal error: 500 unexpected EOF 2026-May-20 12:36:03 2026/05/20 12:36:03 HTTPRequest [I] router: completed PUT /v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/nsm4y0x433x8uecdty5jzf8n9?digest=sha256%3A5c171741ca48bbbd5ddec5ef1669908bd2d75db1f6d2383798d4de52dd0bbad0 for 98.97.37.176:0, 500 Internal Server Error in 59999.7ms @ container/container.go:411(container.PutBlobsUpload) 2026-May-20 12:36:03 2026/05/20 12:36:03 .../container/container.go:433:PutBlobsUpload() [E] Package registry API internal error: 500 offset mismatch between file and model 2026-May-20 12:36:03 2026/05/20 12:36:03 HTTPRequest [I] router: completed PUT /v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/4pmbxtsxg6pn7xowewxmylc8q?digest=sha256%3Aee5f447b414e35eb93aa9645b225df21e7a6b6a5b4f77b1fb4b24f650298efbf for 98.97.37.176:0, 500 Internal Server Error in 4.3ms @ container/container.go:411(container.PutBlobsUpload) 2026-May-20 12:36:03 2026/05/20 12:36:03 .../container/container.go:433:PutBlobsUpload() [E] Package registry API internal error: 500 offset mismatch between file and model 2026-May-20 12:36:03 2026/05/20 12:36:03 HTTPRequest [I] router: completed PUT /v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/nsm4y0x433x8uecdty5jzf8n9?digest=sha256%3A5c171741ca48bbbd5ddec5ef1669908bd2d75db1f6d2383798d4de52dd0bbad0 for 98.97.37.176:0, 500 Internal Server Error in 5.0ms @ container/container.go:411(container.PutBlobsUpload) 2026-May-20 12:36:03 2026/05/20 12:36:03 .../container/container.go:433:PutBlobsUpload() [E] Package registry API internal error: 500 offset mismatch between file and model 2026-May-20 12:36:03 2026/05/20 12:36:03 HTTPRequest [I] router: completed PUT /v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/4pmbxtsxg6pn7xowewxmylc8q?digest=sha256%3Aee5f447b414e35eb93aa9645b225df21e7a6b6a5b4f77b1fb4b24f650298efbf for 98.97.37.176:0, 500 Internal Server Error in 4.2ms @ container/container.go:411(container.PutBlobsUpload) 2026-May-20 12:36:05 2026/05/20 12:36:05 HTTPRequest [I] router: completed GET / for 127.0.0.1:40708, 200 OK in 3.8ms @ web/home.go:32(web.Home) 2026-May-20 12:36:06 2026/05/20 12:36:06 .../container/container.go:433:PutBlobsUpload() [E] Package registry API internal error: 500 unexpected EOF 2026-May-20 12:36:06 2026/05/20 12:36:06 HTTPRequest [I] router: completed PUT /v2/trigger/proj_zlvybyxzqtafjwonlxid/blobs/uploads/wksi5x3yqketsosuiewqgoofc?digest=sha256%3A2704302743ed6f48f2647fafc9ddf87cef27f23baf700193e0f8a1147e72547e for 98.97.37.176:0, 500 Internal Server Error in 58342.3ms @ container/container.go:411(container.PutBlobsUpload)
What the hecki? The thing that stick out here is,
offset mismatch between file and model
Ok, so, like, w-t-f?
Oh btw I had to create a trigger user in my gitea, because I think it might be hardcoded
@todo figure out how to proceed
, and I was seeing activity on the production env in my trigger instance.