Architecture decision records
This page lists the design decisions of Tollgate.
Each record has a number, a status, and a date. To change a decision, write a new record that supersedes the old record. Do not edit the decision of an accepted record.
| ADR | Title | Status |
|---|---|---|
| 0001 | Use Go and one static binary | Accepted |
| 0002 | Pass request bodies through without changes | Accepted |
| 0003 | Estimate input tokens from the body size | Accepted |
| 0004 | Keep limit state behind one store interface | Accepted |
| 0005 | Reserve the estimate, then settle the real cost | Accepted |
| 0006 | Rate limits fail open, hard budgets fail closed | Accepted |
| 0007 | Fall back only before the first byte | Accepted |
| 0008 | Use sticky routing and an exact cache, not a semantic cache | Accepted |
| 0009 | Relay streams as a tee | Accepted |
| 0010 | Log metadata only | Accepted |
| 0011 | Store money as integer micro-USD | Accepted |
| 0012 | Protect keys and the build pipeline | Accepted |
| 0013 | Drain open streams before shutdown | Accepted |
| 0014 | Use encoding/json/v2 | Accepted |
| 0015 | Keep SDK tests in a separate Go module | Accepted |
| 0016 | Define the four timeouts of an upstream request | Accepted |
| 0017 | Edit request bodies in place, without a new encode | Accepted |
| 0018 | Reserve a capped output, and document the overspend bound | Accepted |
| 0019 | Rebuild the spend from the usage file at start | Accepted |
| 0020 | Keep /readyz ready when the limit store fails | Accepted |
| 0021 | Apply ordered retries before the first byte | Accepted |
| 0022 | Limit extra attempts with a local retry budget | Accepted |
| 0023 | Keep circuit breakers in each process | Accepted |
| 0024 | Use Vertex AI for the Claude cloud host | Accepted |
| 0025 | Serve metrics and profiles on a separate admin port | Accepted |
| 0026 | Export traces as OTLP JSON with an in-house exporter | Accepted |
| 0027 | Derive session IDs from the original prompt prefix | Accepted | | 0028 | Keep the last successful target for the provider cache lifetime | Accepted | | 0029 | Cancel streams when estimated cumulative cost exceeds their hard budget room | Accepted | | 0030 | Cache exact responses in bounded tenant-scoped memory | Accepted | | 0031 | Mask JSON strings with request-local placeholders | Accepted | | 0032 | Serve a read-only admin API from a bounded event index | Accepted | | 0033 | Embed the dashboard build in the Go binary | Accepted |
Template
Use this structure for a new record. Copy the next number.
# ADR NNNN: Title in the imperative
- Status: Proposed | Accepted | Superseded by NNNN
- Date: YYYY-MM-DD
- Supersedes: none | NNNN
## Context
The problem and the facts that the decision depends on.
## Decision
What Tollgate does.
## Alternatives
1. Each alternative and the reason not to use it.
## Consequences
- What changes because of the decision.