Design document

How qlane is designed to work

This is the public version of the design. It describes what is planned, written for people who have not seen anything else. All names are proposals and may change.

Status: design stage. There is no code, no alpha and nothing to install. Where a number depends on a measurement that has not been made, this page says so instead of inventing one.

On this page

What it is

qlane is a queue and runtime for AI work that is expensive to repeat and risky to run twice. You hand it tickets. It runs them on workers in parallel, remembers each completed step so a retry never pays again for finished work, keeps spending inside limits you set, picks providers in the order you choose, and records side effects so a crash does not send the same email or make the same charge twice.

A checkout line is a good picture. A ticket is a shopper in line, and workers are the cashiers. A step is one item scanned: once it is scanned and paid for, a crash at the register does not make you scan and pay again. The spend cap is the shopper's budget: the cashier stops at the limit and hands over what was already bought. The effect ledger is the receipt: a note before the money is taken and a note after, so if the lights go out in between, there is a record of the uncertain moment. The scheduler is the store manager deciding how many shoppers each lane can handle, so one lane does not hog the cashiers.

Two example jobs. A deck-making pipeline: one ticket is "build this deck", with steps such as outline, write each slide, make images, assemble. If a worker dies after the slides are written and paid for, the retry resumes at the images step.

A bulk-generation job: each item is a ticket, and per-user and per-plan caps stop one user's batch from using everyone's budget.

What qlane is not: it does not sell model access, it does not proxy your provider calls, and speed is not its pitch. Your workers call providers directly.

The pitch is the reliability and cost safety of AI work. It leads with spend caps, an effect ledger that marks an unknown outcome as unknown, and crash tests. Saved steps are the baseline: other tools offer durable steps too, so qlane does not claim to be first at them.

The plan is a TypeScript library, running on Bun first, in one npm package. Open source, Apache-2.0 planned.

How it works

  1. Submit. Your app submits a ticket for a named workflow, with an optional key (to avoid duplicates), labels (user, plan, app), a queue, spending limits, and optionally a time to run or a deadline.
  2. Claim. A worker claims the ticket from one of the queues it serves. The claim is a single database update, and the worker holds a short lease it keeps renewing while it makes progress. Before the fair scheduler arrives, the claim takes the highest priority first, then the oldest due time, then the lowest id, so the order is deterministic.
  3. Run steps. The workflow runs step by step. Each step has a name. When a step finishes, its result is written down before the next one starts.
  4. Stop or finish. The ticket ends done, failed, cancelled, out of budget, or dead (the system gave up). Finished steps are kept in every case.
  5. Recover. If a worker disappears, its lease runs out and another worker picks the ticket up at the next unsaved step. A late result from the old worker is rejected, using a token that changes on every claim.

Workflow code is plain code. qlane does not replay it from the start to rebuild state, so it does not require your step code to be deterministic. The step context offers an injected clock and random source for code that wants to be testable.

Tickets and states

Ticket states and what each one means
StateMeaning
queuedWaiting for a worker.
runningA worker is executing steps.
scheduledDelayed or recurring, not due yet.
pausedWaiting on a decision (an approval, an admin pause, a cost policy, or a provider out of quota when you choose pause over fail) or, for parents, waiting for child tickets, or, for a ticket that depends on other tickets, waiting for them. A reason is always attached. Expires after 7 days by default, except when waiting for children or dependencies.
unknown_outcomeAn effect step may or may not have happened, and nobody has said which yet.
doneFinished.
failedA non-retryable error: a bug, bad input, or a provider out of quota with no fallback left (the default; it can end paused instead).
budget_exceededStopped by a spend cap. Partial results are kept. Retry with a larger budget.
cancelledCancelled by a person, or a rejected approval.
deadThe system gave up: retries used up, lease lost too many times, deadline passed, or a wait expired. A reason is recorded.

failed means the work could not succeed; dead means qlane gave up on it. Retrying a cancelled, failed or dead ticket creates a new linked ticket and keeps the finished steps, so nothing is paid twice. A repeat submit with the same key within 7 days returns the existing ticket.

Waiting for children, and waiting for dependencies, reuse the paused state with their own reasons; there is no extra state for them. Mapping a dependency wait to the paused state is a reading, not stated in the decision.

Saved steps

A step is identified by its name plus a hash of its input. When a step finishes, its result is stored. A retry or a restart reuses it. If the same step name comes back with different input, qlane fails loudly instead of quietly reusing a wrong answer. If the code that ran a saved step differs from the code now running, qlane warns and carries on; it never blocks.

  • Maximum age. A step may set a maximum age for its saved result. An older result is not reused; the step runs again, which costs money and counts against the caps. It never applies to effect steps (emails, charges), which are governed by the ledger.
  • Retention. Content (inputs, outputs, step results) is purged after 30 days by default. Small metadata such as state, cost and timestamps is kept for 12 months. Both are configurable.
  • Optional encryption of fields you mark, with your own key.

Limit: a saved step avoids paying twice only if the step finished and its result was recorded. A crash in the middle of a step means that step runs again.

Retries and timeouts

Defaults, all configurable per project, queue, ticket or step: 3 attempts with exponential backoff and full jitter (2 s base, doubling, 60 s cap). Rate-limit and server errors retry. Content-policy errors and a provider's out-of-quota answers (its own spending cap included) do not.

When a provider says how long to wait, qlane waits the longer of that and its own backoff. These are starting values to be tuned, not measured results.

A dropped stream is never a free retry

If a streaming call breaks, the tokens that arrived count as spent against the cap before the retry runs. If the amount is unknown, the reserved maximum counts. The adapter records this on the try. Limit: the count is what reached the worker, not the provider's bill.

One layer owns retries

qlane's provider adapters turn off the retries built into model SDKs, so qlane's policy is the only one. Retries in several layers multiply: if the SDK retries 3 times, the step 3 times and the ticket 3 times, one failing call can become 27 upstream requests, each possibly billed. With one layer, 3 attempts is 3 requests, all recorded and capped. A step that calls an SDK directly bypasses this, and the docs would show how to turn the SDK's retries off.

Safeguards and their starting values
SafeguardStarting value
Per-step timeout120 seconds
No-progress watchdog (counts tokens and events)60 seconds
Whole-ticket maximum, running time only1 hour
Heartbeat / leaseevery 10 seconds / 30 seconds

The one-hour maximum counts only time spent running. Time queued, scheduled, in retry backoff, paused or in unknown outcome does not count. A ticket whose worker is lost three times (configurable) ends dead. A ticket that keeps killing its worker, for example by running it out of memory, ends dead after those three losses and is flagged as a suspected poison job instead of being requeued forever.

A graceful shutdown hands running tickets back without using an attempt. A crash counts as a lost lease, never as an attempt.

Spend caps and reservations

Caps can be set in tokens, dollars (from a small bundled, dated price table, or your own rows) or counts, at five scopes: ticket, user, plan, app and queue. Before a call, qlane reserves an estimate from the relevant caps; after the call it settles to the actual usage. Reservation is exclusive: if many workers race for the last of a budget, exactly one reservation succeeds.

  • A submit that would exceed a cap is rejected up front.
  • A ticket that hits a cap mid-way stops and keeps its partial results, ending in the budget_exceeded state. It does not resume on its own unless you configure that.
  • A step that has a budget attached must declare a maximum output size, so the estimate has a ceiling.
  • Budgets can be lifetime or calendar windows in the project's time zone.
  • A provider that is out of quota, including its own monthly spend cap, is treated as a stop, not retried. See the next sections.

Limit: caps are a cooperative bound. Workers call providers directly, so qlane cannot stop a call made outside it or one already in flight. The overshoot will be measured and published next to the claim. See what qlane cannot do.

Provider fallback

A step can list providers in order, with rules for when to move on (for example on a rate limit or a server error). Moving to the next provider does not use up an attempt. Every try is recorded with its cost and counts against the cap. If every provider fails in one pass, that pass uses one attempt. When a provider returns a request id, it is saved with the try so you can match an invoice. A policy can pause the ticket for approval before falling back to a more expensive provider.

Each provider also has a circuit breaker (planned for M2): when the share of failed retryable calls passes a set percent over a window, new calls to that provider wait in the queue or use the fallback list, a few trial calls test it after a cool-down, and one alert is raised on trip and one on recovery. Learned limits and fallback still apply, and every threshold is filled after the build.

Escalation (planned for M2)

The same ordered list can hold a stronger model for when a cheaper one gives an answer that fails the step's output check: an entry is used after a failure, or after a failed check. Like a fallback, an escalation does not use up an attempt, and it waits for no backoff. Every try is recorded and its cost counts against the cap, so a cap can stop the stronger try. A failed check is not a provider failure and does not count toward the circuit breaker, and an answer that failed its check is never cached. The most escalation steps per step is a setting; its value is filled after the build.

Prompt hash and prompt version (planned for M2)

Each model-call try stores a hash of what was sent (system prompt, messages and request parameters, secrets left out) and an optional prompt version you set, next to the model version. They show in the step history, the terminal view and the dashboard. They only record: they never block a ticket, and they are separate from workflow versions pinned per ticket. The hash is not the result-cache key.

Planned providers: Anthropic; OpenAI-compatible services (OpenAI, DeepSeek, OpenRouter, Groq, Together); Google Gemini; local models through Ollama and llama.cpp. None of these integrations has been tested yet because there is no code.

Out of quota, and the result cache

Out of quota is not a rate limit

A provider answer that means out of quota or budget, in any HTTP code (for example 402, 422, or 429 with a quota code), is one error class, provider_quota_exhausted (name is a proposal). It is never retried, so a quota error cannot start a retry storm. Each adapter ships a table of quota codes for its provider, and you can configure the rule per provider. A provider's own spend cap is one entry of that table.

  • If a fallback is configured, qlane tries it. With none left, the ticket ends failed by default, or paused if you set that per provider or project. A ticket paused this way can be resumed, and expires after 7 days like any paused ticket.
  • One alert per provider until its next successful call. Every affected ticket still records its own stop. An out-of-quota stop does not count toward the circuit breaker.

Result cache (opt-in, planned for M2)

A step can turn on a cache of its model answer that works across tickets. It is off by default. The key is a hash of the prompt, the parameters and the exact model version you pin in the step, not an alias. If the model is an alias, qlane refuses to start the step unless you pin an exact version, so an alias that moved cannot serve an old answer. A new pinned version is a miss. An answer is saved only if the provider returns a version string equal to the pinned one. The scope is per user by default and can be changed. A maximum age is required when the cache is on; a step that turns it on without one is refused.

  • A hit is marked "cache hit" in the step history, costs nothing against the cap and shows the saved cost.
  • It never serves an effect step, a truncated or failed answer, an answer that failed the output check, a dropped partial stream, an error or quota answer, or an answer older than the maximum age. Only complete successful answers are saved.

Limit: at a temperature above 0 a hit returns one earlier answer, so behavior differs from a fresh call. That is why the cache is opt-in. It stores the model's answer only, not tool results, and it is qlane's own cache, not the provider's prompt-caching discount.

Effect ledger and unknown outcome

A step marked as an effect (sending an email, charging a card, writing to another system) is handled differently. Before the call, qlane writes an intent row with the exact arguments and an idempotency key. After the call, it updates the same row with the outcome. Every change also writes an audit event.

If a crash lands between the two, the outcome is unknown. Then:

  1. If you gave a probe (a check that asks the other system "did that happen?"), qlane runs it. If it cannot tell, qlane retries it on a timer a limited number of times, then raises an alert and the ticket stays in unknown_outcome until a person resolves it.
  2. Otherwise the step pauses as unknown and the ticket becomes unknown_outcome. A person or your app resolves it: confirmed done (optionally attaching the output), confirmed not done, or check again. "Check again" is refused when there is no probe.
  3. You can instead choose per step to retry automatically with the same key (at-least-once) or to skip (at-most-once).

Built-in helpers are planned for HTTP calls (sending an Idempotency-Key header), email and messages (treated as not queryable), database writes (intent and outcome rows in a transaction) and agent tool calls (qlane supplies the key). An unresolved unknown outcome expires after 7 days by default and ends dead. After one hour unresolved (a proposed default, with no data behind it) qlane raises one alert for that ticket. The alert is a reminder, not a resolution.

Limit: the ledger cannot make a tool exactly-once. That needs an idempotency key honored by the receiving system. For email, qlane can only record and ask.

Async provider jobs

Some provider work takes longer than one request: video, audio, image generation. A special step type submits the job and then polls by the provider's job id.

  • qlane saves the job id the moment the provider returns it, in the same write as the submit result and before any polling.
  • Polling and downloading have their own retry policy and timeouts. A dropped connection while polling never submits the job again. A crash resumes by the saved id.
  • Between polls the ticket is parked. It holds no worker and no slot, and does not count as running or toward the one-hour maximum.
  • The submission is costed once, with its real cost. Polls are free by default, with an optional per-poll cost.
  • A maximum wait and a maximum poll count end a stuck job as dead with a reason (inside a parallel branch only that branch fails).
  • Provider callbacks are not in release 1: polling works everywhere, including embedded mode with no public address.
  • If the submit itself timed out before an id came back, qlane uses a lookup (if the provider supports finding a job by a reference) or marks the outcome unknown for a person to decide.

Limit: a provider that cannot look jobs up cannot be recovered after a lost id. A job the provider has forgotten fails and is not re-submitted automatically.

Scheduling with missing-run alerts and receipts

Tickets can be delayed, recurring (cron), given an expiry deadline, or held to a start window such as off-peak batch time. Cron runs in UTC unless the schedule names a time zone.

Missed runs are skipped unless you ask for a catch-up, which creates exactly one run however many were missed. A deadline that passes while a ticket runs stops it at the next step boundary, keeps partial results and ends it dead with the reason.

Missing-run alert

Every schedule is checked without you declaring anything. If an expected run never happened, ended failed or dead, or finished without any step saving a receipt, qlane raises one alert and flags the schedule.

A receipt is a short note a step saves about what it produced, such as "daily report sent to 12 people".

You may set a tighter deadline ("a finished run must exist by 06:30"), or turn the check off for a schedule. One bad run raises exactly one alert.

Limit: a scheduled workflow that never saves a receipt raises the alert even when it worked, and a receipt is the code's own word.

Schedules in code (design sketch)

Schedules are written in your code and registered at startup by name, so a restart makes no duplicate. They use standard 5-field cron or an every-N-minutes or every-N-hours shorthand, with no seconds field. The every-N shorthand counts real elapsed time from the schedule's creation time, so daylight saving never affects it.

The dashboard lists them and can disable one; a disabled schedule stays disabled after a restart.

If a schedule fires while its previous run has not finished, the new run is skipped and an event is written, unless you allow overlap, and a skip on purpose is not a missing run.

A run in a skipped daylight-saving hour happens once at the next valid time, and a run in a repeated hour happens once, at the first occurrence.

A unique record per schedule and time, written in the same transaction as the ticket, makes a schedule fire once even with two instances.

Queues, windows and the tick (design sketch)

A queue can be paused (no new claims, running tickets finish) or drained (also no new submits, with a reason), and can have an optional limit on queued tickets. A queue can have several start windows for off-peak work, each with its own concurrency limit; tickets wait as scheduled until a window opens. The scheduler promotes due tickets on a tick, never early, and late by at most a tick interval you configure.

Limits: a tick can be late by up to its interval. The backlog limit counts only queued tickets, not scheduled ones. Overlapping start windows are refused when you save them, unless you set a mode that allows overlap and applies the higher or the lower limit while both are open. A run that falls while a schedule is disabled is skipped and recorded, and raises one missing-run alert with the case disabled, one per skipped run, so a schedule disabled by mistake is flagged too.

Expected-output checks

Every ordinary step is checked to have produced something: an empty result (nothing, an empty string, empty list or empty object) fails the step. Nothing needs to be declared.

A step can replace the default with its own check, which sees the result after the step runs, or switch the check off. A failed check marks the step failed and never records it as done, raises an alert event and shows a flag. It is not retried as a transient error unless you configure that. Effect steps are exempt because the ledger judges them.

A step may also save a receipt (a short note of what it produced); an oversized receipt is truncated and marked, and the step is not failed. A scheduled run with no receipt raises the missing-run alert, unless that schedule switches the receipt requirement off.

Limit: the default only catches an empty result, not a wrong one.

Fair scheduler and concurrency caps

The scheduler shares limited supply, which means provider rate limits, worker capacity, dollar budgets and local model slots, so one user or queue does not starve the others. The default is weighted fair share: equal weights unless you set them per plan or user, idle share can be borrowed, and nobody starves. Priority lanes and first-in-first-out with aging are also planned. The fairness unit is tokens by default, or dollars or steps.

  • qlane estimates before a call, reserves, then corrects to actual. Input is counted with a tokenizer; output is predicted from similar past tickets, or from the maximum output size.
  • Provider limits are starting values you supply, then learned from rate-limit headers and 429 replies. A learned limit is halved on a rate-limit reply and recovers gradually.
  • Local model slots (Ollama, llama.cpp) are a shared supply with counts you configure.
  • Priority and named queues. Priority is a whole number, higher first, default 0, with a range you configure. A ticket is in one queue, and a worker serves only the queues it lists. In the default fair share, the share picks the group (the user or the app, as you configure it) and priority orders the tickets inside that group.
  • Concurrency caps. An optional cap on how many tickets one user or one queue may have running at once. A ticket from a user at the cap waits while others go ahead. The default is none.

Limits: output size of an unknown ticket is predicted, not known. A concurrency cap limits slots, not cost; use spend caps for money.

Tickets that start tickets, and dependencies

Plain ctx.submit (planned for M1)

A workflow step may create another ticket. The call returns the child's id at once, the child is linked to its parent with a "caused by" link, and the parent can check the child's state in a later step. By default the idempotency key is the origin ticket id (the first ticket of a retry chain), step name and call index (you may override it), so a retried parent never creates a second child: it finds the existing ones.

A child inherits the parent's user, plan, app and queue scopes and counts against the same caps, with its own per-ticket cap (a default, or set by the developer).

Plain children are independent: cancelling or failing the parent never cascades to them. This ships in M1 so the loop guard can be tested early.

Optional dependency (planned for M3)

Adding {dependency: true} to the call parks the parent until every dependency child is final, then resumes it with the children's results. In M3 the wait is for all children; wait-for-any is a separate milestone, M3b, described below. A list of children gives fan-out and fan-in.

  • A waiting parent is paused with the reason "waiting for children". It holds no lease, worker or slot, does not expire at 7 days, and ends dead with a reason if it hits a separate maximum wait. The value of that maximum is not set yet.
  • If a dependency child fails, the default (onChildFail: fail) is that the parent resumes only after all dependency children are final (done, failed, dead, cancelled or budget_exceeded), then ends failed with the first failed child's error. Waiting for all children has no fail-fast; wait-for-any (M3b) can fail fast with onFirstFail. A per-call option, continue, lets the parent resume and receive each child's final state.
  • Cancelling the parent cancels its unfinished dependency children. If a waiting parent ends dead at the maximum wait, its unfinished dependency children are cancelled too; finished ones stay. Plain children are never cascaded. A command can cancel a whole chain by its correlation id.
  • A configurable limit, dependencies.max_children, caps the children per parent step and per parent ticket (value set after the M3 build). A submit over the limit is refused with an error. Each child also counts against the inherited user, plan and app caps.
  • Creating the child counts as an effect for the ledger: the submit is a saved step with the default key.

Wait-for-any, dependent tickets and parallel steps (planned for M3b)

A new milestone right after M3 holds three features. They are the first thing cut if time runs out, as a whole, and the wait-for-all in M3 does not depend on them. All names are proposals.

  • Wait-for-any: {dependency: true, wait: "any"} resumes the parent when the first dependency child finishes successfully. By default the other unfinished children are cancelled (they keep their partial results, and children that already finished stay as they are); per call you can let them finish. If the first child to finish fails, the default is to keep waiting, resuming on the first success and failing only if every child fails. It reuses the same paused state and the same maximum wait and children limits.
  • A ticket that depends on other tickets: a client can submit a ticket that depends on any ticket in the same project. A missing id or a cycle is refused with a 422 and no ticket is created. While it waits it holds no worker slot. By default, if any dependency failed, died or was cancelled, it ends failed without running; an option lets it run anyway. A submit key can depend only on tickets it can read, an admin key can name any ticket of the project, and a missing, other-project or unreadable id gives the same 422.
  • Parallel steps inside one ticket: named branches run at the same time. Each branch is a saved step with its own name, input hash, memo and cost record, and the result is a list in branch order. By default, if a branch fails the others finish and the step then fails with the first error; finished branches stay saved.

    Effect steps are allowed in branches, each with its own ledger record, and the unknown-outcome rules apply per branch, so one ticket may list several unknown outcomes. A limit on branches and a cap on how many run at once are configurable (values set after the build).

    Each branch counts against the same spend caps and reserves budget like any step. There is still one lease and heartbeat per ticket, step timeouts and the no-progress watchdog apply per branch, and the one-hour ticket maximum is unchanged.

    A branch may also use an async provider step, a dependency wait or a nested parallel step (nesting is limited by a configurable depth, default 2 levels, which is a proposed default and not backed by data).

    While any branch is active the ticket stays running on its one lease; a parked branch holds no worker slot and no lease, and the ticket parks only when every unfinished branch is parked. It resumes when the first parked branch wakes, and that wake-up resumes only that branch. A budget reserved by a parked branch stays reserved until the branch ends, or until a maximum wait ends it. A branch waiting out a retry backoff counts as parked. If every unfinished branch is parked for different reasons, the ticket is paused with a list of reasons, one per parked branch, and the main reason is the one that wakes first.

    A parked branch that hits the maximum wait for a provider job or for child tickets fails on its own, with its own error, its budget is released and the failure option you chose decides about the other branches; the ticket does not end dead (a parent ticket outside a parallel step still does).

    Every parallel call has a name that becomes part of the step path (step, call, branch), nested calls included; a nested call that goes past the depth limit or breaks a name rule fails only its branch. A branch with an unknown outcome waits for parked siblings before the ticket enters unknown outcome.

    Two options are kept as opt-in with the defaults unchanged: abort the other branches on a failure, and fail fast on the first failed child of a wait-for-any.

Dependency graph view and editor (planned for M5)

The web dashboard gets a graph of tickets (parents, children, dependencies and the state of each) and an editor. The editor can add or remove dependencies on scheduled tickets and on tickets waiting for dependencies (operator role only, cycles refused, every change recorded; queued, running and finished tickets stay read-only), and can build a new ticket graph from a saved workflow template.

Templates are defined in your code and registered at startup; the server stores them in the project database, so the dashboard can list them and show old versions.

You set the version; without one a hash of the declaration is used, and a changed declaration under the same version raises a warning while the server keeps the stored declaration, so you need a new version string to change it.

The dashboard only starts runs, shows a form for the declared input, refuses bad input with the same checks as a normal submit, and records which template version each run used.

A stored version is kept while any ticket made from it exists and is deleted only when an admin asks; nothing is deleted automatically, and deleting a project removes them all. How the input shape is described is decided at build time.

M5 is already the largest milestone and templates had no earlier design, so the editor is the first thing cut from M5, and the read-only graph view is cut after it. This is not a no-code builder.

Every ticket carries a correlation id, shared by retries and child tickets, that appears in every event, log line, webhook and CLI view. A loop guard records how deep a chain is and refuses a submit past a maximum depth, and refuses a repeat of the same workflow and input within one chain. Workers also emit started, stopped and lost events.

Schedule risk: if milestone M3 slips, dependencies are the first thing cut. Plain ctx.submit stays, as fire-and-forget with polling, and waiting moves after release 1.

Crash-test harness

The harness runs qlane under a seeded simulator with an injected clock, random source and database. It breaks things on purpose: process crashes anywhere, provider failures (rate limits, slow replies, dropped streams, truncation, out-of-quota answers), duplicates and delays, database and clock faults. It checks these rules on every run:

  • No ticket is lost.
  • No effect happens twice without being marked unknown.
  • No ticket is claimed by two workers.
  • A saved step never pays again; an async step with a saved job id is never resubmitted.
  • A step with a failed output check is never recorded as done.
  • A ticket that always kills its worker ends dead and never starves others.
  • Reservations are exclusive. A reservation whose call had started stays charged after a crash until its step settles or a person releases it, so it neither leaks nor gets freed while the provider may have billed it.
  • A retried ticket sends the same key for the same effect, so a retry cannot repeat an effect silently.
  • A restart or a stalled writer never kills a healthy ticket.
  • The correlation id is carried through retries and children.
  • A dropped stream shows its partial spend on the cap, with no free retry. A quota answer gives no retry storm and one alert per provider. The cache never serves an effect step and never returns one user's answer to another.

There are two tiers: a deterministic simulator over an in-memory store for every push, and a smaller suite on the real SQLite file with real kill -9. A one-command crash demo is planned right after M1 and again after M3. The same harness is planned for your own tickets, with a fake model-provider server that is also planned as its own small package. A run replays exactly from its seed; failing seeds become regression tests. In release 1 it tests qlane only; checking other queues is after release 1.

Limit: the simulator can differ from the real disk and runtime. A real-process kill-and-truncate suite on a real disk is a required gate for milestone M1. Shrinking failing runs is not promised.

Storage

Each project has its own SQLite file in WAL mode, behind a thin storage interface so another store could be added later. Writes use a short transaction. The planned settings are full synchronous writes until tests justify less, a busy timeout of at least 5 seconds, and group commit with a hard delay limit (value set after a benchmark). The record of an effect's intent is flushed immediately.

On a disk error qlane stops acknowledging writes, reopens the database and fails loudly; it never retries the sync on the same connection.

Backups: qlane backup takes a consistent snapshot while running, and one is taken automatically before a schema upgrade. Migrations only go forward. A restore drill is a release gate.

Limits: one writer, one machine. Backups have a loss window. Durability is claimed for fsync on ext4 only until tested.

Embedded vs server

Embedded
A library inside your process. Workers (threads or processes) share one SQLite file. No server to run and no keys. A local dashboard is included.
Server
A standalone server plus a client library. Remote workers connect over HTTP or a WebSocket. Several projects, each with its own file, keys and limits. API keys with roles, an HTTP API and a command line.

Ticket and step rules are identical in both. Server mode speaks plain HTTP and is meant to sit behind a TLS proxy. Inline closures work for embedded workers; remote workers need registered, named workflows. A hosted version is a later possibility: your local instance would dial out to it, nothing would leave your machine unless you mark specific queues or steps as allowed, and no ports would open on your machine.

Security

  • The open-source core never stores your provider API keys. They stay in the worker's environment and are redacted from logs and traces.
  • Prompts, outputs and keys do not appear in logs, traces, metric labels or the dashboard unless you allow it.
  • API keys are hashed and revocable, with roles (submit, worker, read, admin). Browsers get short-lived read tokens only, and cancel and approve go over authenticated HTTP.
  • Webhooks are signed (HMAC-SHA256) and refused if they point at private network addresses.
  • Runaway or buggy step code is limited by caps, timeouts and the ledger. That limits damage; it does not prevent it.

Not done: no formal security review or outside audit has happened.

Final review changes

The design went through a last review round. It kept every planned feature and cut nothing, which raises the risk that release 1 never ships at this size; that risk is recorded in the plan. The review changed or added these, all still plans:

  • Correctness. A retried ticket sends the same effect key. A call that had started stays charged after a crash. A restart is not a crash. One synchronous writer does the claiming. A ticket pins the workflow version it started on, and one with a saved side effect pauses instead of running under a different version.
  • Security and operations. Dashboard content is shown only as escaped text under a strict content policy. Dashboard users are managed from the command line. Each project upgrades on its own, with the backup on a path you choose and a free-space check, and a full disk stops writes with a clear error and recovers by itself. An old events cursor gets a clear error. One server owns a project file. Settings have one fixed order: flag, environment, project settings, config file, default. A ticket's webhook events arrive in order.
  • Added. A bundled, dated price table; observe mode; avoided cost and a JSON run report; an early alert on a cap; guarded bulk retry; bulk submit with a cost preview; an external schedule for jobs started by cron or launchd; compensation of a single completed effect on request; usage events as CSV; qlane doctor; starter templates; a fourth example, an agent loop.
  • One open question. How a presented key finds its project when keys live in per-project files is not decided.

Limit: caps can count a little too much after a crash, compensation is effect-specific and can be wrong, and none of this exists as code yet.

Build order, M1, M2, M3, M3b, M4, M5, M6

The plan is one open-source release, with no alpha: nothing goes to outside users until the milestones are done. Each milestone gets a time box when it starts; if it runs out, the feature is cut and ships later.

  1. M1: core. Queue and workers, saved steps, retries, the crash-test harness, correlation id, loop guard, poison protection, plain ctx.submit (child tickets, no waiting), basic step history in the terminal.
  2. M2: caps and providers. Spend caps and reservations, provider adapters and fallback, timeouts, dropped-stream accounting, the quota class, the opt-in result cache, the escalation rule and the prompt hash.
  3. M3: ledger, async jobs, dependencies. The effect ledger and unknown outcome, async provider jobs, and wait-for-all dependencies ({dependency: true}) on top of M1's ctx.submit. Dependencies are the first thing cut if M3 slips.
  4. M3b: parallel steps, wait-for-any, dependent tickets. Parallel steps inside a ticket, wait-for-any, and tickets that depend on other tickets. Cut first, as a whole, if time runs out.
  5. M4: scheduler. Token-aware fair share, schedules and the missing-run alert, concurrency caps.
  6. M5: tools. Terminal view, qlane init, the BullMQ importer, the web dashboard (with a graph view and, last in line, a graph editor with workflow templates) and WebSocket links.
  7. M6: docs and packaging. Documentation, examples, Docker image and single binary.

Before building, a few short experiments are planned: a benchmark of SQLite commit behavior, a test of the simulator's repeatability, a check of sync behavior on the target host, and token-counting accuracy.

The crash demo is built right after M1 and again right after M3. The newest feature items are cut before the older ones if a milestone runs out of time; the correctness and security fixes are never cut.

Not in release 1: a hosted service, a Python SDK, React and framework helpers, Node server support, a remote backup endpoint, and checking other queues with the harness.