optimizer and composable systems: lessons learned (#149)
Testing in Production
Yes, you read that right. Testing in production — not instead of staging, but in addition to it. Here's why and how.
Why Staging Lies
Staging environments differ from production in subtle but critical ways:
- Different data volumes (10K rows vs 10M rows)
- Different traffic patterns (no real users)
- Different infrastructure (smaller instances)
- Different integrations (sandbox APIs)
Canary Deployments
Route a small percentage of traffic to the new version:
# nginx.conf
upstream backend {
server app-v1:8080 weight=95;
server app-v2:8080 weight=5;
}
Monitor error rates, latency percentiles, and business metrics. If anything degrades, roll back automatically.
Feature Flags
Decouple deployment from release:
- Deploy code to 100% of servers
- Enable feature for 1% of users
- Gradually increase to 5%, 25%, 100%
- Kill switch: disable instantly without redeployment
Observability
You can't test what you can't see. Invest in:
- Structured logging (JSON, correlation IDs)
- Distributed tracing (OpenTelemetry)
- Custom metrics (business KPIs, not just CPU/memory)
- Alerting (on symptoms, not causes)
登录 发表评论
3 评论
Carol Williams 评论于 2026年4月21日 03:21
Morbi tempus commodo mattis. Mauris dapibus risus quis suscipit vulputate. Nunc viverra elit ac laoreet suscipit.
John Doe 评论于 2026年4月21日 03:20
In hac habitasse platea dictumst. Mineralis persuadere omnes finises desiderium. Ubi est audax amicitia. Sunt torquises imitari velox mirabilis medicinaes. Nulla porta lobortis ligula vel egestas. Ut eleifend mauris et risus ultrices egestas. Abnobas sunt hilotaes de placidus vita. Teres talis saepe tractare de camerarius flavum sensorem. Diatrias tolerare tanquam noster caesium. Ut suscipit posuere justo at vulputate.
John Doe 评论于 2026年4月21日 03:22
Sed varius a risus eget aliquam. Pellentesque vitae velit ex. Ubi est audax amicitia.