Client Configuration
The client is configured through environment variables and runs as a job executor.
Environment Variables
| Variable | Default | Description |
|---|---|---|
PORT | 5000 | HTTP port the client listens on |
DB_PATH | ./db.sqlite | Path to the client’s SQLite payload database |
DATA_PATH | ./data | Working directory where job files are stored |
MAX_AGE | 864000 | Seconds before the client’s cleaner deletes a payload directory (10 days) |
The client shares the exact same config loader as the server, so these variables have the same names and same defaults in both roles. SERVICE_* variables are ignored by the client. There is no client-specific default for PORT: if you run a server and a client on the same host, pick a different port explicitly (9000 is a common convention, shown below) to avoid a clash.
Example Configuration
Basic Setup
export PORT=9000
export DB_PATH=/var/lib/job-orchestrator-client/db.sqlite
export DATA_PATH=/var/lib/job-orchestrator-client/data
job-orchestrator client
Docker Compose
services:
client:
image: codeberg.org/rvhonorato/job-orchestrator:latest
command: ["/job-orchestrator", "client"]
environment:
PORT: 9000
DATA_PATH: /opt/data
DB_PATH: /opt/data/db.sqlite
volumes:
- client-data:/opt/data
volumes:
client-data:
Note there’s no ports: mapping here — in the shipped docker-compose.yml the client is only reachable over the internal Docker network from the server, not from the host. See Client Security below.
How the Client Works
Database
The client uses the same DB_PATH-backed SQLite database mechanism as the server — by default it’s a real file on disk, not in-memory, so payload records survive a client restart.
If you actually want an ephemeral, in-memory database (e.g. for a disposable/test client), set DB_PATH=:memory: explicitly. That’s an opt-in choice, not the default behavior. Be aware that payload IDs then restart at 1 after every restart, while the server still holds the old IDs as dest_id, and old payload directories with the same numbers may still be on disk. Only use it for throwaway clients.
The client’s database keeps a record of every payload it received (cleaned ones stay with status Cleaned). The server is still the source of truth for job history.
Working Directory
The client stores job files under DATA_PATH. Each payload gets a subdirectory named after its numeric payload ID:
/opt/data/
├── 1/
│ ├── run.sh
│ ├── input.pdb
│ ├── output.txt (created by run.sh)
│ ├── .orchestrator.exit (written by the trap)
│ └── output.zip (created on the first completed /retrieve)
├── 2/
│ └── ...
Uploaded files are stored by base name only, so any directory structure is flattened.
Gotcha: unlike the server, the client does not create DATA_PATH at startup. It is created when the first payload arrives. Until then the client’s cleaner logs could not read directory every 60 seconds, which is harmless.
Execution Environment
When the Runner task executes a job:
- Validates
run.sh(size, UTF-8, required trap, blocked patterns) - Runs
bash run.shwith the payload directory as working directory - Stores the process ID; the exit code is read later from
.orchestrator.exit - All files in the directory are included in results
The Updater task then watches Running payloads:
- If a payload has the killed flag set, it marks it
Killed - If
.orchestrator.exitexists, it marks itCompleted(exit code0) orFailed - If the process is still alive, it leaves it
Running - If the process is gone and never wrote
.orchestrator.exit, it marks itLostand logs a warning (see Lost Processes)
Job Termination
The client supports on-demand job termination via the /kill/:id endpoint:
- Receives termination requests from the server
- Sends SIGTERM to the running process (only the
bashprocess; child processes it started may keep running unless the script forwards the signal) - Sets the payload’s killed flag; the Updater changes the status to
Killedif the payload isRunning - The server’s
/terminate/:idendpoint triggers this via HTTP
Resource Reporting
The client exposes a /load endpoint that reports CPU usage:
curl http://localhost:9000/load
Returns a float representing CPU usage percentage. This can be used by the server for load-aware scheduling (planned feature).
Multiple Clients
You can run multiple clients for:
- Scaling: Handle more concurrent jobs
- Isolation: Different services on different machines
- Redundancy: Failover capability
Same Service, Multiple Clients
Not supported. Each service maps to exactly one client URL, and the server only accepts one value per variable. Do not put a load balancer in front of several clients either: payload IDs are local to each client, so a /retrieve/{id} sent to a different client would hit the wrong payload or a 404.
To spread load today, split the work into several services, each with its own client.
Different Services
Run specialized clients for different workloads:
# Client for general jobs
PORT=9000 job-orchestrator client
# Client for heavy computation (different machine)
PORT=9001 job-orchestrator client
Server configuration:
SERVICE_LIGHT_UPLOAD_URL=http://client-1:9000/submit
SERVICE_LIGHT_DOWNLOAD_URL=http://client-1:9000/retrieve
SERVICE_LIGHT_TERMINATE_URL=http://client-1:9000/kill
SERVICE_HEAVY_UPLOAD_URL=http://client-2:9001/submit
SERVICE_HEAVY_DOWNLOAD_URL=http://client-2:9001/retrieve
SERVICE_HEAVY_TERMINATE_URL=http://client-2:9001/kill
Client Security
Network Access
The client should only be accessible by the orchestrator server:
- Use internal networks / VPCs
- Firewall rules to restrict access
- Never expose client ports to the internet
Execution Sandbox
The client executes arbitrary run.sh scripts. Consider:
- Running in containers with resource limits
- Using separate user accounts with minimal permissions
- Mounting only necessary directories
- Network isolation if jobs don’t need internet
Docker Resource Limits
services:
client:
image: codeberg.org/rvhonorato/job-orchestrator:latest
command: ["/job-orchestrator", "client"]
deploy:
resources:
limits:
cpus: '4'
memory: 8G
reservations:
cpus: '1'
memory: 1G
Monitoring
The examples below assume the client is reachable directly (e.g. running it outside Docker, or with a ports: mapping added). With the shipped docker-compose.yml, the client has no host-exposed port — reach it with docker compose exec client curl localhost:9000/health instead, or from the server container over the internal network.
Health Check
curl http://localhost:9000/health
Load Check
curl http://localhost:9000/load
Container Logs
docker logs -f client-container
Troubleshooting
Client Not Receiving Jobs
- Verify server can reach client URL
- Check firewall rules
- Verify service configuration on server
Jobs Stuck in Prepared
- Check the client logs for errors
- Check for permission issues in the working directory
High Memory Usage
The client holds each incoming upload fully in memory while handling /submit, and builds /retrieve_partial archives in memory. Large inputs or partial downloads of large directories cause memory spikes. The database is file-backed and is not the cause. If memory is high:
- Check the size of the payloads being sent and of the partial downloads being requested
- Check for stuck payloads
- Restarting the client frees memory, but jobs running at that moment become lost (see Lost Processes)