Docker Deployment
Docker is the recommended way to deploy job-orchestrator in production.
Quick Start
git clone https://codeberg.org/rvhonorato/job-orchestrator.git
cd job-orchestrator
docker compose up --build
This starts:
- Server on port 5000 (exposed to the host)
- Example client on port 9000 (internal network only, not exposed to the host)
Docker Images
Official Image
docker pull codeberg.org/rvhonorato/job-orchestrator:latest
The image has no entrypoint: the binary lives at /job-orchestrator and is not on PATH, so always pass the full command (/job-orchestrator server or /job-orchestrator client).
The published image contains the binary plus bash and sqlite, without any application. It works as a server as-is. To use it as a client, build your own image on top of it with the tools your run.sh needs. example/Dockerfile shows how, adding the gdock demo application.
Build Locally
# Build the job-orchestrator image (server, and base for clients)
docker build -t job-orchestrator .
# Build the example client image (adds gdock on top of the local image)
docker build --build-arg BASE_IMAGE=job-orchestrator -t job-orchestrator-client example/
Docker Compose
Basic Setup
services:
server:
image: codeberg.org/rvhonorato/job-orchestrator:latest
command: ["/job-orchestrator", "server"]
ports:
- "5000:5000"
environment:
PORT: 5000
DB_PATH: /opt/data/db.sqlite
DATA_PATH: /opt/data
MAX_AGE: 864000
MAX_UPLOAD_SIZE: 419430400
SERVICE_EXAMPLE_UPLOAD_URL: http://client:9000/submit
SERVICE_EXAMPLE_DOWNLOAD_URL: http://client:9000/retrieve
SERVICE_EXAMPLE_TERMINATE_URL: http://client:9000/kill
SERVICE_EXAMPLE_RUNS_PER_USER: 5
SERVICE_EXAMPLE_MAX_RUNS: 10
volumes:
- server-data:/opt/data
client:
image: codeberg.org/rvhonorato/job-orchestrator:latest
command: ["/job-orchestrator", "client"]
environment:
PORT: 9000
DATA_PATH: /opt/data
DB_PATH: /opt/data/db.sqlite
volumes:
- client-data:/opt/data
depends_on:
- server
volumes:
server-data:
client-data:
Production Setup
See Production Deployment - Container Hardening for details on each security option.
Multiple Clients
Scaling Horizontally
A service is bound to exactly one client URL, so adding more clients for the same service does not spread its jobs. Extra clients only receive work if they are configured as separate services, as shown below. Do not place a load balancer in front of several clients: payload IDs are local to each client, so status and result requests must reach the client that got the job.
Multiple Services
services:
server:
environment:
# Light jobs
SERVICE_LIGHT_UPLOAD_URL: http://client-light:9000/submit
SERVICE_LIGHT_DOWNLOAD_URL: http://client-light:9000/retrieve
SERVICE_LIGHT_TERMINATE_URL: http://client-light:9000/kill
SERVICE_LIGHT_RUNS_PER_USER: 10
# Heavy jobs
SERVICE_HEAVY_UPLOAD_URL: http://client-heavy:9000/submit
SERVICE_HEAVY_DOWNLOAD_URL: http://client-heavy:9000/retrieve
SERVICE_HEAVY_TERMINATE_URL: http://client-heavy:9000/kill
SERVICE_HEAVY_RUNS_PER_USER: 2
client-light:
image: codeberg.org/rvhonorato/job-orchestrator:latest
command: ["/job-orchestrator", "client"]
environment:
PORT: 9000
deploy:
resources:
limits:
cpus: '2'
memory: 2G
client-heavy:
image: codeberg.org/rvhonorato/job-orchestrator:latest
command: ["/job-orchestrator", "client"]
environment:
PORT: 9000
deploy:
resources:
limits:
cpus: '8'
memory: 16G
Volume Management
Persistent Storage
Always use named volumes for production:
volumes:
server-data:
driver: local
driver_opts:
type: none
o: bind
device: /data/job-orchestrator/server
client-data:
driver: local
driver_opts:
type: none
o: bind
device: /data/job-orchestrator/client
Backup Strategy
# Stop services (optional, for consistent backup)
docker compose stop
# Backup server and client data (the client holds results of running jobs)
tar -czf backup-$(date +%Y%m%d).tar.gz /data/job-orchestrator/server /data/job-orchestrator/client
# Resume services
docker compose start
Networking
Internal Network
Keep client internal:
services:
server:
ports:
- "5000:5000" # Exposed to host
networks:
- internal
- external
client:
# No ports exposed to host
networks:
- internal
networks:
internal:
internal: true
external:
With Reverse Proxy
services:
nginx:
image: nginx:alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./certs:/etc/nginx/certs:ro
depends_on:
- server
server:
# No ports exposed, accessed via nginx
networks:
- internal
Logging
View Logs
# All services
docker compose logs -f
# Server only
docker compose logs -f server
# Last 100 lines
docker compose logs --tail 100 server
Log Rotation
services:
server:
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
Commands
# Start
docker compose up -d
# Stop
docker compose down
# Restart
docker compose restart
# Rebuild and start
docker compose up --build -d
# View status
docker compose ps
# Shell into container
docker compose exec server /bin/sh