A worker disappears
Its lease runs out, reconciliation finds the abandoned job, and the job goes back to a healthy worker.
Built for the day something breaks
Machines fail, processes linger and releases get rolled out. AetherGrid is designed around those moments.
Two workers want the same job
Only the worker holding the lease may act on it, and a worker that loses its lease stops acting as the owner.
A job leaves processes behind
When a job ends, times out or is cancelled, its whole process tree is stopped, not only the first process.
The scheduler runs in two places
The cluster tick runs in one process at a time, so two schedulers never place the same work.
Customers share one database
Tenant isolation is enforced in the application and in the database, so a worker, a job or a lease cannot be tied to another tenant’s records.
You upgrade to a new release
Each release is rehearsed on a database from the one before it: migrate, compare every table and the schema, then start the new API and read the data back.