Partial Downloads and Debugging
This guide demonstrates how to use the /download_partial and /retrieve_partial endpoints to debug stuck or incomplete jobs.
Use Cases
Use partial download endpoints when:
- A job appears stuck in
SubmittedorRunningstatus - A job ended as
Failedand you want its output (/downloadonly returns ZIPs forCompletedjobs) - You need to inspect the current working directory
- Debugging why a job isn’t progressing
- Verifying intermediate outputs of long-running jobs
Server-Side: /download_partial/
Example: Checking Incomplete Job
# Download current state of job ID 1
curl -o partial_results.zip http://localhost:5000/download_partial/1
# Extract and inspect contents
unzip -l partial_results.zip
unzip partial_results.zip
# View specific files
cat intermediate_output.txt
Client-Side: /retrieve_partial/
This endpoint is called internally by the server’s /download_partial endpoint.
It provides the same functionality at the client level.
The server does not have a separate setting for this URL. It takes the
service’s DOWNLOAD_URL and replaces its last path segment with
retrieve_partial (http://client:9000/retrieve becomes
http://client:9000/retrieve_partial). If you put a proxy in front of the
client, make sure that path is routed too.
In the shipped Docker Compose setup the client has no ports: mapping — it’s
only reachable from the server over the internal network (e.g.
http://client:9000), not from the host. The example below assumes you’re
running the client directly (outside Docker Compose) or calling it from
another container on the same network.
Example (Client Developer)
# Get partial results directly from client
curl -o partial_results.zip http://localhost:9000/retrieve_partial/1
Comparison: Full vs Partial Download
| Endpoint | Returns | Use Case |
|---|---|---|
/download/{id} | Final results only | Normal completion |
/download_partial/{id} | Current state always | Debugging, monitoring |
Combined Example Script
#!/bin/bash
SERVER_URL="http://localhost:5000"
# Submit job and parse the returned ID
echo "Submitting job..."
RESPONSE=$(curl -s -X POST "${SERVER_URL}/upload" \
-F "file=@run.sh" \
-F "user_id=1" \
-F "service=example")
echo "$RESPONSE"
JOB_ID=$(echo "$RESPONSE" | jq -r '.id')
if [ -z "$JOB_ID" ] || [ "$JOB_ID" = "null" ]; then
echo "Failed to get job ID from response"
exit 1
fi
echo "Job ID: $JOB_ID"
# Poll until complete, saving partial snapshots for inspection
echo "Monitoring job..."
for i in $(seq 1 10); do
sleep 10
CT=$(curl -s -o response.tmp -w '%{content_type}' "${SERVER_URL}/download/${JOB_ID}")
if [[ "$CT" == application/zip* ]]; then
mv response.tmp final_results.zip
echo "Job completed, results saved to final_results.zip"
exit 0
fi
STATUS=$(jq -r '.status' response.tmp)
case "$STATUS" in
Failed|Invalid|Killed|Lost|Cleaned)
echo "Job ended with status $STATUS"
curl -s -o final_partial.zip "${SERVER_URL}/download_partial/${JOB_ID}"
exit 1
;;
esac
echo "Check $i: status=$STATUS, saving partial snapshot..."
curl -s -o "partial_${i}.zip" "${SERVER_URL}/download_partial/${JOB_ID}"
unzip -l "partial_${i}.zip" 2>/dev/null | head -20
done
echo "Job did not complete within the polling window"
exit 1
Limitations
Queuedjobs have not reached a client yet, so/download_partialreturns404(“Job payload not found on client”)- Until
run.shwrites something, the ZIP only contains the input files - After the client’s cleaner has removed the payload directory, the request fails
- Partial downloads of running jobs may be incomplete or inconsistent
- Use partial downloads for debugging, not for production result retrieval
Security Notes
- Building the ZIP fails if any entry in the payload directory resolves outside it (for example, a symlink created by
run.shthat points to/etc), so such files cannot be exfiltrated this way - See Path Traversal Protection for details