Server Configuration
The orchestrator server is configured primarily through environment variables.
Environment Variables
Core Settings
| Variable | Default | Description |
|---|---|---|
PORT | 5000 | HTTP port the server listens on |
DB_PATH | ./db.sqlite | Path to SQLite database file |
DATA_PATH | ./data | Directory for job file storage |
MAX_AGE | 864000 | Job retention time in seconds (default: 10 days) |
MAX_UPLOAD_SIZE | 419430400 | Maximum /upload request body in bytes (default: 400MB). Larger uploads get 413 |
Service Configuration
For each service you want to support, configure these variables:
| Variable Pattern | Description |
|---|---|
SERVICE_<NAME>_UPLOAD_URL | Client endpoint for submitting jobs |
SERVICE_<NAME>_DOWNLOAD_URL | Client endpoint for retrieving results |
SERVICE_<NAME>_TERMINATE_URL | Client endpoint for terminating jobs |
SERVICE_<NAME>_RUNS_PER_USER | Maximum concurrent jobs per user (default: 5) |
SERVICE_<NAME>_MAX_RUNS | Maximum concurrent jobs for the service across all users (default: 10) |
Note: <NAME> is converted to lowercase, and that lowercase name is what clients must send in the service field (which is case-sensitive). SERVICE_EXAMPLE_* defines the service example; uploading with service=EXAMPLE returns 422 Invalid service.
Gotcha: <NAME> cannot contain underscores. The loader takes the text between the first and second _ as the name, so SERVICE_MY_TOOL_UPLOAD_URL defines a service my with an unknown setting TOOL_UPLOAD_URL, which is silently ignored.
Gotcha: PORT, MAX_AGE, MAX_UPLOAD_SIZE, RUNS_PER_USER, and MAX_RUNS are parsed as numbers with no fallback on a bad value — if you set one to something non-numeric, the server panics on startup instead of falling back to its default. Make sure these are always valid integers.
Example Configuration
Minimal Setup
export PORT=5000
export DB_PATH=/var/lib/job-orchestrator/db.sqlite
export DATA_PATH=/var/lib/job-orchestrator/data
export SERVICE_EXAMPLE_UPLOAD_URL=http://localhost:9000/submit
export SERVICE_EXAMPLE_DOWNLOAD_URL=http://localhost:9000/retrieve
export SERVICE_EXAMPLE_TERMINATE_URL=http://localhost:9000/kill
Production Setup
# Core settings
export PORT=5000
export DB_PATH=/opt/orchestrator/db.sqlite
export DATA_PATH=/opt/orchestrator/data
export MAX_AGE=864000 # 10 days
# Example service (general purpose)
export SERVICE_EXAMPLE_UPLOAD_URL=http://compute-1:9000/submit
export SERVICE_EXAMPLE_DOWNLOAD_URL=http://compute-1:9000/retrieve
export SERVICE_EXAMPLE_TERMINATE_URL=http://compute-1:9000/kill
export SERVICE_EXAMPLE_RUNS_PER_USER=10
# Simulation service (specialized, long-running)
export SERVICE_SIMULATE_UPLOAD_URL=http://sim-cluster:9001/submit
export SERVICE_SIMULATE_DOWNLOAD_URL=http://sim-cluster:9001/retrieve
export SERVICE_SIMULATE_TERMINATE_URL=http://sim-cluster:9001/kill
export SERVICE_SIMULATE_RUNS_PER_USER=3
Docker Compose
services:
server:
image: codeberg.org/rvhonorato/job-orchestrator:latest
command: ["/job-orchestrator", "server"]
ports:
- "5000:5000"
environment:
PORT: 5000
DB_PATH: /opt/data/db.sqlite
DATA_PATH: /opt/data
MAX_AGE: 864000
SERVICE_EXAMPLE_UPLOAD_URL: http://client:9000/submit
SERVICE_EXAMPLE_DOWNLOAD_URL: http://client:9000/retrieve
SERVICE_EXAMPLE_TERMINATE_URL: http://client:9000/kill
SERVICE_EXAMPLE_RUNS_PER_USER: 5
SERVICE_EXAMPLE_MAX_RUNS: 10
volumes:
- server-data:/opt/data
volumes:
server-data:
Configuration Details
PORT
The HTTP port for the REST API. Users will connect to this port to submit jobs and download results.
PORT=5000
DB_PATH
Path to the SQLite database file. The directory must exist and be writable.
DB_PATH=/var/lib/job-orchestrator/db.sqlite
The database is created automatically on first run. It stores:
- Job metadata (ID, user, service, status)
- Job locations and timestamps
- Client payload references
DATA_PATH
Directory where job files are stored. Each job gets a unique subdirectory.
DATA_PATH=/var/lib/job-orchestrator/data
Structure:
/var/lib/job-orchestrator/data/
├── a1b2c3d4-e5f6-7890-abcd-ef1234567890/
│ ├── run.sh
│ ├── input.pdb
│ └── output.zip (after completion)
├── b2c3d4e5-f6a7-8901-bcde-f12345678901/
│ └── ...
MAX_AGE
How long to keep job directories before cleanup, in seconds. Age is measured from the directory’s last modification time and the rule applies to every job, whatever its status. A job that is still running (or queued) when its directory passes MAX_AGE is cleaned too, so set it well above your longest expected job.
| Value | Duration |
|---|---|
3600 | 1 hour |
86400 | 24 hours |
172800 | 48 hours |
604800 | 1 week |
864000 | 10 days (default) |
MAX_AGE=864000
Jobs older than this are removed by the Cleaner task.
Service URLs
Each service needs upload, download, and terminate URLs pointing to a client:
SERVICE_MYSERVICE_UPLOAD_URL=http://client-host:9000/submit
SERVICE_MYSERVICE_DOWNLOAD_URL=http://client-host:9000/retrieve
SERVICE_MYSERVICE_TERMINATE_URL=http://client-host:9000/kill
- UPLOAD_URL: Where to POST job files
- DOWNLOAD_URL: Where to GET results (
/{id}is appended automatically). The partial download URL is derived from it by replacing the last path segment withretrieve_partial - TERMINATE_URL: Where to POST termination requests (
/{id}is appended automatically)
Gotcha: if you leave UPLOAD_URL, DOWNLOAD_URL, or TERMINATE_URL unset for a service, the server does not fail at startup — the field is silently stored as an empty string, and requests routed to that service will fail at request time instead. Double-check all three are set for every service you configure.
RUNS_PER_USER
Controls how many jobs a single user can have running simultaneously for a service:
SERVICE_EXAMPLE_RUNS_PER_USER=5
- Jobs exceeding the quota remain in
Queuedstatus - They’re automatically dispatched when slots become available
- Set higher for quick jobs, lower for resource-intensive jobs
Fixed Settings
These are not configurable:
- Task intervals: Sender and Getter every 500ms, Cleaner every 60s
- Log level: always
INFO.RUST_LOGis ignored. Every HTTP request is logged with its headers, so keep secrets out of headers or filter logs - Outgoing HTTP timeouts: none. A client that accepts the connection but never answers can stall the request (and a termination stays
Lockedmeanwhile)
File Permissions
Ensure the server process has:
- Read/Write access to
DB_PATHparent directory - Read/Write access to
DATA_PATHdirectory - Network access to all configured client URLs
Validating Configuration
Start the server and check logs:
job-orchestrator server
You should see:
WARNlines for every core variable left at its default- A
Config { services: {...}, ... }line listing the parsed services. Check that every service appears with all three URLs set Using database: sqlite://...listening on 0.0.0.0:<port>
Test with a health check:
curl http://localhost:5000/health