Your First Job
This guide walks you through submitting and retrieving your first job.
Prerequisites
Make sure you have job-orchestrator running. See Quick Start if you haven’t set it up yet.
Understanding Jobs
A job in job-orchestrator consists of:
- Files: One or more files to be processed
- A
run.shscript: The entry point that gets executed - User ID: Identifies who submitted the job (for quota tracking)
- Service: Which service/backend should process this job
Creating a Simple Job
Create a simple run.sh script:
cat > run.sh << 'EOF'
#!/bin/bash
# Required: Capture exit code for the orchestrator client
trap 'echo $? > .orchestrator.exit' EXIT
echo "Hello from job-orchestrator!" > output.txt
echo "Processing complete at $(date)" >> output.txt
EOF
chmod +x run.sh
Submitting the Job
Submit the job using curl:
curl -X POST http://localhost:5000/upload \
-F "file=@run.sh" \
-F "user_id=1" \
-F "service=example" | jq
You’ll receive a response like:
{
"id": 1,
"status": "Queued",
"message": "Job successfully uploaded"
}
Note the id field - you’ll need this to check status and download results.
Checking Job Status
Check the job status via GET request:
curl http://localhost:5000/download/1
If the job is not yet completed, you’ll get a JSON response:
{
"id": 1,
"status": "Submitted",
"message": ""
}
The status field will be one of: Queued, Processing, Submitted, Prepared, Running, Completed, Failed, Invalid, Cleaned, Unknown, Locked, Killed, or Lost. See Job States for descriptions of each.
Downloading Results
Once the status is Completed, the same endpoint returns the ZIP file:
curl -o results.zip http://localhost:5000/download/1
Extract and view:
unzip results.zip
cat output.txt
You should see:
Hello from job-orchestrator!
Processing complete at <timestamp>
The archive contains the whole job directory: your input files, everything
run.sh wrote, and the .orchestrator.exit file.
Downloading Partial Results
/download/{id} only returns a ZIP for Completed jobs. For any other status,
including Failed, it returns the JSON status. To get the files of a failed,
stuck, or still-running job, use /download_partial/{id}:
# Get current state of job even if incomplete
curl -o partial_results.zip http://localhost:5000/download_partial/1
Canceling a Job
If you need to cancel a running job, use the terminate endpoint:
# Cancel job with ID 1
curl -X POST http://localhost:5000/terminate/1
Response on success:
{
"id": 1,
"status": "Unknown",
"message": "job terminated"
}
The
statusfield in the terminate response is not meaningful. UseGET /download/{id}to confirm the job’s actual status after termination.
Notes:
- The server sends a termination request to the client, which sends
SIGTERMto the process - Terminated jobs will have status
Killed Queuedjobs cannot be cancelled: they have not reached a client yet, so the request fails with500, the job staysQueued, and it will still run later- Terminating a job that already finished still “succeeds” and changes its status to
Killed. After that,/download/{id}returns JSON instead of the ZIP, so the results are only reachable through/download_partial/{id}(until the client cleans them up)
Security
Script Execution
The client executes user-submitted run.sh scripts with full privileges.
While a script validator rejects obviously dangerous patterns (destructive
commands, network exfiltration tools, reverse shells, privilege escalation),
it is NOT a sandbox and can be bypassed by determined actors.
Important: Always run the client inside a container with:
- Resource limits (CPU, memory, PIDs)
- Read-only root filesystem
- Drop all capabilities (
--cap-drop=ALL) - Internal network only (no external exposure)
- Non-root user
See Production Deployment for details.
Path Traversal Protection
Uploaded filenames are reduced to their base name, so ../../etc/passwd is
stored as passwd inside the job directory. When results are zipped, any entry
that resolves outside the job directory (such as a symlink to /etc) makes
the archive fail instead of being included. See
Path Traversal Protection.
Important Notes
The run.sh Script
- Must be named exactly
run.sh - Is run as
bash run.shinside the job directory, so it does not need the executable bit or a shebang - Exit code
0indicates success - Non-zero exit code indicates failure
- All output files in the working directory are included in results
Important: The script MUST include the following trap at the beginning. The client checks for it before running the script and marks the job Invalid if it is missing:
#!/bin/bash
trap 'echo $? > .orchestrator.exit' EXIT
# ... rest of your script
This ensures that when your run.sh exits, it writes the exit code to .orchestrator.exit, which the client’s Updater task reads to determine if the job completed successfully or failed.
File Size Limits
The server accepts request bodies up to 400MB by default (all files in one upload combined). Change it with the MAX_UPLOAD_SIZE environment variable, in bytes. Larger uploads are rejected with 413.
User ID Parameter
Jobs can be submitted with a user_id for quota management:
curl -X POST http://localhost:5000/upload \
-F "file=@run.sh" \
-F "user_id=1" \
-F "service=example"
user_id is required and must be an integer; the server returns 422 if it is missing or not a number, if service is unknown, or if no run.sh was uploaded. service must match a configured service name exactly, in lowercase (example, not EXAMPLE).
Job Retention
Job directories are deleted once they are older than MAX_AGE (default: 10 days), and the job is marked Cleaned. Age is measured from the directory’s last modification time, not from when the job finished, and the rule applies to jobs in any status. Make sure to download your results before they expire.
Next Steps
- Learn about the Job Lifecycle
- Configure Quotas for your users
- Set up Production Deployment