All products

Distributed infrastructure

Headgate

A job queue that treats dequeue as an admission decision.

My role
Creator & engineer
Status
Active build
Platform
Go · Rust · embedded console
Source
Apache-2.0 license

The idea behind it.

Headgate is a distributed job queue for teams that need more than 'give me N jobs.' Its admission gate evaluates rate limits, tenant fairness, concurrency budgets, and quarantine state atomically inside the store before a worker can claim work. The same core behavior is shared across Go and Rust and is exercised against PostgreSQL, MySQL, and Redis.

System overviewThe store applies admission policy before workers claim jobs.
Queued work
  • Typed jobs
  • Durable workflows
Atomic admission
  • Rate limits, fairness & concurrency
  • PostgreSQL, MySQL or Redis
Workers
  • Go & Rust
  • Leases & fencing

What it supports.

  • Atomic admission with fleet-wide rate limits and concurrency ceilings
  • Fair multi-tenant scheduling that keeps spare capacity useful
  • Leases, fencing, retries, deadlines, panic recovery, and graceful shutdown
  • Typed jobs, resumable steps, progress, results, and durable workflows
  • Quarantine and redrive controls with an embedded operations console
  • A shared wire format and executable conformance suite across languages and stores

Engineering decisions.

Dequeue as a policy decision

Most queues ask a worker which jobs it wants. Headgate asks what the fleet's current policy permits it to run. That shift brings rate limits, fairness, concurrency, and poison-pill handling into one atomic admission path instead of scattering them across workers.

Correctness belongs at the store boundary

A claim and its lease are written together, with fencing on every per-job operation. Admission is evaluated atomically inside the store across the PostgreSQL, MySQL, and Redis backends. Policy-rejected jobs remain visible, while lease loss is explicit and actionable.

A system for operators too

The project includes a control API, CLI, embedded web console, OpenTelemetry hooks, payload redaction, and a conformance register that ties capabilities to named evidence. The goal is a queue that is inspectable when the interesting failures happen.

From the build 7 min read

Your job queue should decide what may run

Why I built headgate around admission inside the store, and what a flooding tenant, a fast clock, and a passing test suite taught me about the design.

Read the essay

Keep exploring

All products

App infrastructure

Mosaic

Monetization infrastructure that stays native to every app.

Say hello Back to contact