Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Quota System

The quota system ensures fair resource allocation by limiting concurrent jobs per user and per service.

Two Independent Limits

The sender enforces two limits before dispatching a queued job:

VariableScopeDefaultDescription
SERVICE_<NAME>_RUNS_PER_USERPer user, per service5Max jobs a single user can have active at once
SERVICE_<NAME>_MAX_RUNSPer service (all users)10Max jobs the service can have active at once

Both limits must have available slots for a job to be dispatched. A job that would violate either limit remains in Queued status until a slot opens.

Jobs in Processing, Submitted, or Running status all count toward both limits. A job whose process disappears (for example after a client restart) is marked Lost and releases its slot (see Lost Processes).

How It Works

SERVICE_EXAMPLE_RUNS_PER_USER=3
SERVICE_EXAMPLE_MAX_RUNS=5

User 1 submits 4 jobs → 3 dispatched (per-user limit reached), 1 queued
User 2 submits 4 jobs → 2 dispatched (MAX_RUNS=5 reached), 2 queued

When one of User 1's jobs completes, one service slot frees up.
Both users now have a queued job and a free per-user slot, so either
User 1's or User 2's job is dispatched next. Which user goes first in a
sender pass is not deterministic.

Quota Scope

Quotas are enforced per user, per service (RUNS_PER_USER) and per service (MAX_RUNS):

UserServiceRUNS_PER_USERMAX_RUNSResult
user_1example510Up to 5 concurrent
user_2example510Up to 5 concurrent (independent)
user_1simulate24Up to 2 concurrent

Users don’t compete with each other for per-user slots. They do share the MAX_RUNS pool for the service.

Scheduling: Round-Robin Between Users

When multiple users have queued jobs for the same service, the sender distributes slots in round-robin order across users. This prevents a single user with many queued jobs from consuming all available slots ahead of others.

The rotation restarts on every sender pass (every 500ms) and the order of users within a pass is arbitrary. Within one user, jobs are in practice taken oldest first (there is no explicit ordering or priority).

Quota States

┌──────────┐   Both limits OK   ┌────────────┐
│  Queued  │ ─────────────────▶ │ Processing │
└──────────┘                    └────────────┘
     │                                │
     │ Either limit exceeded          ▼
     │                         ┌────────────┐
     └──────────────────────── │ Submitted  │
       (waits for a free slot) │ (running)  │
                               └────────────┘

Choosing Values

Factors to Consider

  1. Client capacity: MAX_RUNS should not exceed what your client hardware can handle concurrently
  2. Job duration: Longer jobs need lower per-user quotas
  3. Resource usage: CPU/memory intensive jobs need lower MAX_RUNS
  4. User base: More users may need lower per-user quotas

Guidelines

Job TypeSuggested RUNS_PER_USERSuggested MAX_RUNS
Quick jobs (< 1 min)10-2020-50
Medium jobs (1-10 min)5-1010-20
Long jobs (10+ min)2-54-10
Resource-intensive1-32-4

Example Scenarios

Scientific Computing Platform

# Quick validation jobs
SERVICE_VALIDATE_RUNS_PER_USER=10
SERVICE_VALIDATE_MAX_RUNS=50

# Standard analysis
SERVICE_ANALYZE_RUNS_PER_USER=5
SERVICE_ANALYZE_MAX_RUNS=20

# Heavy simulation
SERVICE_SIMULATE_RUNS_PER_USER=2
SERVICE_SIMULATE_MAX_RUNS=4

Educational Platform

# Student exercises
SERVICE_EXERCISE_RUNS_PER_USER=3
SERVICE_EXERCISE_MAX_RUNS=30

# Final projects
SERVICE_PROJECT_RUNS_PER_USER=5
SERVICE_PROJECT_MAX_RUNS=20

Testing Quotas

Submit multiple jobs to observe throttling:

Start the server with low limits (the variables must be set in the server’s environment, for example in docker-compose.yml, not in the shell you submit from):

SERVICE_EXAMPLE_RUNS_PER_USER=2
SERVICE_EXAMPLE_MAX_RUNS=3

Use a run.sh that takes a while (for example sleep 60 after the trap), then submit 5 jobs and check them:

for i in {1..5}; do
  curl -s -X POST http://localhost:5000/upload \
    -F "file=@run.sh" \
    -F "user_id=1" \
    -F "service=example" | jq -r '.id'
done

sleep 2
for id in 1 2 3 4 5; do
  curl -s http://localhost:5000/download/$id | jq -r '"\(.id) \(.status)"'
done

Expected behaviour (the upload response always says Queued, so check /download):

  • Jobs 1-2: dispatched (Submitted/Running); the per-user limit of 2 is now reached
  • Jobs 3-5: stay Queued. MAX_RUNS=3 would allow one more, but not for this user
  • Submitting as user_id=2 at this point would dispatch one job and fill MAX_RUNS

Fair Scheduling

The quota system provides basic fairness:

  • Per-user isolation: One user can’t starve others
  • Per-service cap: Prevents overloading a single client
  • Round-robin dispatch: Slots are shared evenly across users
  • Automatic queuing: No jobs are rejected, just delayed

Limitations

Current limitations (improvements planned):

  • No priority queues (FIFO within quota constraints)
  • No time-based quotas (e.g., jobs per hour)
  • No burst allowances

See Also