Your software factory.
It never clocks out
Bugs, forum posts, Slack, alerts or a prompt start work; it lands on your board as a tested change, ready to ship.
Set it up once. It keeps shipping
What a normal day looks like, three months after setup.
Gateway p99 back to normal after the 08:00 spike.
Rate limiter is still the old in-memory one...
1 reply Last reply 9:16 AMRate limiter is still the old in-memory one, so it resets on every deploy and customers on the Scale plan keep hitting 429s at 08:00 UTC. @ForgeCode rewrite it on Redis, keep the public headers, per-tenant overrides from the plans table, and load test it before you open a PR.
On it. Opened mission LRK-410 on larkspur-cloud-1 with 5 issues. I will report back in this thread when a PR is ready.
| Issue | Status | Priority | Runs | Updated |
|---|---|---|---|---|
| LRK-412Token bucket store on Redis with per-key TTL | In Progress | High | Running | 09:16 |
| LRK-413Middleware: 429 with Retry-After and RateLimit-* headers | In Progress | High | Running | 09:16 |
| LRK-414Per-tenant overrides read from the plans table | In Progress | Medium | Running | 09:16 |
| LRK-415Load test: 12k rps burst, p99 under 40 ms | To Do | Medium | Blocked by 412, 413 | 09:16 |
| LRK-416Docs: rate limits page and SDK retry examples | To Do | Low | Idle | 09:16 |
| LRK-412Token bucket store | Done |
| LRK-413Middleware: 429 + headers | In Progress |
| LRK-414Per-tenant overrides | In Progress |
| LRK-415Load test 12k rps | To Do |
| LRK-416Docs and SDK retries | To Do |
Gateway p99 back to normal after the 08:00 spike.
Rate limiter is still the old in-memory one...
2 replies Last reply 6:32 PMRate limiter is still the old in-memory one... rewrite it on Redis, keep the public headers, per-tenant overrides, load test before you open a PR.
PR #1187 is ready for review. 4 of 5 issues done; LRK-416 (docs) is open in review for you.
- Redis token bucket with per-key TTL, 0 allocations on the hot path
- 429 responses carry Retry-After (integer seconds) and RateLimit-Limit / Remaining / Reset
- Tenant overrides read from
plans.rate_limit_rps, cached 30s - Load test: 12,400 rps burst for 5 min, p99 31 ms, 0 dropped
| Title | Trigger | Last run | Next run |
|---|---|---|---|
| Production alert: open incident, find root cause, open PR | alert sentry prod | 02:10 | on alert |
| Nudge stalled issues: rerun from checkpoint after 6h idle | watch: issue idle 6h | 03:35 | on stall |
| Remind reviewer when a PR waits more than 8h | watch: PR waiting 8h | 07:00 | on wait |
| GitHub bug report: triage, reproduce, open issue | github issues | Mon 16:48 | on event |
| Forum post tagged bug: reproduce and reply | forum discourse | Mon 11:02 | on post |
| Slack message in #api-platform | slack @ForgeCode | 09:14 | on message |
| Nightly flaky test sweep, escalate if above 1% | schedule 02:00 | 02:00 | Thu 02:00 |
NullPointer in billing/invoice.rs:212, 38 events in 4 min. LRK-419 opened.02:33 RCA: refunded invoices have no customer since PR #1174 dropped the join. PR #1188 opened; merge policy
merge when gates pass for hotfix/*: merged 02:41.02:44 posted for Dev Okonkwo (owns billing): what changed, why, and how to verify.
| Issue | Status | Priority | Runs | Updated |
|---|---|---|---|---|
| LRK-412Token bucket store on Redis with per-key TTL | Done | High | Idle | 11:02 |
| LRK-413Middleware: 429 with Retry-After and RateLimit-* headers | Done | High | Idle | 12:48 |
| LRK-414Per-tenant overrides read from the plans table | Done | Medium | Idle | 15:20 |
| LRK-415Load test: 12k rps burst, p99 under 40 ms | Done | Medium | Idle | 18:19 |
| LRK-416Docs: rate limits page and SDK retry examples | In Review | Low | Idle | 18:31 |
| LRK-419Incident: NullPointer in billing/invoice.rs (alert). PR #1188 merged by factory | Done | High | Idle | 02:41 |
- Tests: 212 passed
- Clippy: 0 warnings
- Load test: p99 31 ms
- Security scan: 0 findings
- Staging smoke: 18 of 18
- Reviewer agent: 2 fixed
Done Public API rate limiter rebuilt on Redis. Load tested at 12,400 rps, p99 31 ms.
Fixed Billing NullPointer from PR #1174. Hotfix merged under policy in 31 min.
Decide PR #1187 needs Priya's review to ship the rate limiter to Scale customers.
Better Flaky tests down to 0.6% from 1.4%. Review nudges now fire after 4h.
Next Thursday 02:00: Postgres 16 upgrade mission starts, merge policy ask me.
A coding harness makes one developer faster.
A factory raises organisational velocity
The factory you'd otherwise build yourself
Each pillar is plumbing your engineers never write or maintain. Tick a box to clear it off your backlog.
Any trigger starts the work
The factory opens an issue for it and starts in seconds. Nobody assigns it.
your backlog if you build it6 to do
All 6 ship with ForgeCode
Crashes restart and resume
The factory restarts on its own and every run resumes from its last automatic checkpoint.
your backlog if you build it9 to do
All 9 ship with ForgeCode
LRK-413LRK-416LRK-419Runs for months, day and night
Automations watch, nudge, re-run and escalate, so missions keep landing through nights and weekends.
your backlog if you build it8 to do
All 8 ship with ForgeCode
Every run makes the factory better
Every run is measured: evals score it and benchmarks on your past work catch regressions, so skills, memories and automations get tuned.
your backlog if you build it8 to do
All 8 ship with ForgeCode
…and everything you haven't hit yetour backlog, not yours
Your factory, your terms
Run factories on ForgeCode's cloud or bring your own cloud (BYOC), and bring your own model keys.
What teams run on a factory
Whole engineering roles and long-running programs, not single tasks.