No. A live app needs a repair path that runs without the feature backlog. Check what is actually running every day, patch from the release in production, carry the fix back to development, and refuse any later release that would undo it. We call this production-led security maintenance. Most of its parts already exist; what's usually missing is the agreement that joins them.
Picture a booking service that shipped in the spring. Customers reserve tables, it does its job, and the team has moved on to a new checkout flow on the development branch. One Tuesday an advisory lands for the HTTP parser the booking API bundles. The fix is a single line in a lockfile.
That lockfile lives on a branch halfway through a feature nobody wants to ship yet. So the team meets the choice every small team meets eventually: wait for the feature, or patch production directly and hope the next release doesn't bring the old parser back.
A security fix is finished when it survives the next release.
WORKED EXAMPLE / A BOOKING API
Will the repair survive the next release?
One parser fix meets the next feature release. Step through it, then change what the gate checks.
Carry fix to development pending
Refused: missing the required parser fix.Newer is not enough. The gate holds every fix production depends on, refuses the candidate and names an owner to carry the fix into B. Production stays protected.
Explanation and primary sources
A worked example with an illustrative booking API. Production runs release A while development has moved on to branch B; when the bundled HTTP parser needs a security fix, maintenance builds A + fix from production and ships it through the deployment gate, which records the parser fix as required. A later build from B is newer and passes its tests but still carries the old parser: a gate that checks required fixes refuses it and assigns an owner to carry the fix into B, while a gate that checks release order alone admits it and production runs the vulnerable parser again.
- Git: merging fixes upwards
- GitLab: deployment safety
- Red Hat: backporting security fixes
Why waiting and hot-fixing both fail
Waiting ties security to feature work. The app is only as safe as the next reason someone has to develop it, and a finished app may never get one. The wait is long by design, too: in most teams a production release passes review, staging sign-off and a change approval. DORA's research found that heavyweight external approval slows delivery without lowering the rate of failed changes.
Patching the live server by hand is fast and fragile. Most pipelines, ours included, rebuild an app from its main branch and clean the directory before each release. A fix that exists only on the running machine disappears with the next deploy, and nobody gets an alert when it does.
The working answer keeps the speed of the hot fix and the record of the normal release.
Production leads, development catches up
Production-led security maintenance is an operating model in which what is actually deployed drives its own bounded repair loop. Each repair is verified in service, carried back into development and enforced on every later deployment. Three rules carry it.
Repair from what runs. Start from the release in production, not from the development branch. Record exactly what is serving: source revision, image digest, runtime version. Make the smallest permitted change and commit it before rollout. Nothing is edited live and forgotten.
Carry the fix back as an obligation. "Production is ahead" means one named fix is live and still owed to development. That is a tracked debt with an owner, never an untracked difference. If the development branch has changed the same dependency, the conflict stays visible until someone resolves it.
Refuse the regression. Every candidate passes one gate before production: feature releases, maintenance repairs and rollbacks alike. The gate checks today's required fixes rather than commit dates. Version numbers can mislead too: Red Hat backports security fixes without changing the upstream version.
The third rule is the hinge. In the booking story, a feature build from the development branch finishes the day after the repair. Its commit is newer and its tests pass. It still carries the old parser. A gate that only checks which build is newer ships it, and production is exposed again without anyone touching the server. GitLab's deployment safety controls stop an older job overwriting a newer deployment. That tells you which build is newer, not whether it contains the fix.
Daily is the floor, not the speed limit
A daily sweep asks what runs, what can restart, what changed in vulnerability data and which carry-back obligations remain open. Dormant jobs, scheduled tasks and rollback images belong in that inventory, because the day they wake up they serve customers. An urgent advisory, or an entry in CISA's known exploited vulnerabilities catalogue, starts the same loop immediately.
Each layer has its own repair route. Host packages follow the operating system's update channel. A library bundled in a container needs a new image; patching the host does not rewrite it. An application dependency needs a lockfile change and a fresh build. A single updater cannot own all three, so the model names who owns each.
Spend the scanning where customers are
A typical pipeline scans development, every preview, staging and production. One advisory becomes four alerts about the same library, three of them on servers no customer touches, and each needs triage before the one fix that matters starts its trip through the gated release.
Deep runtime monitoring belongs on live, exposed services. Source, secret and dependency checks stay before release, because a build runner holding production credentials is a target in its own right. When the same image runs in several places, reuse its assessment while the image, the scanner rules and the advisory data are unchanged. Amazon Inspector already rescans images when new vulnerabilities appear and maps them to running workloads.
WORKED EXAMPLE / ONE ADVISORY, TWO FLOWS
Where does the effort go?
One advisory for the HTTP parser bundled in the booking API, through two flows. Each row is one step, and both flows take their steps at the same pace.
- Production exposed
- Production protected
- Hands-on step
- Automated step
AdvisoryHTTP parser bundled in the booking API
A · How most teams are set upThrough the dev loop
- Automated · 4 alertsScan every environmentThree on servers no customer touches
- Hands-on ×4Triage four alertsWhich is real, and who owns it?
- Hands-onFix on the development branchNext to unfinished feature work
- Hands-onReview the pull requestA second pair of eyes on one line
- AutomatedCI rebuilds and rescansAlerts repeat until each is redeployed
- Hands-onStaging sign-offA person checks the release candidate
- Hands-on + waitApproval, then the release windowDORA: slower, no fewer failed changes
- Hands-onProtected hereDeploy to production, watchedSomeone watches the rollout
B · Development catches upProduction-led
- Automated · 1 findingSweep what is liveOne owner · daily, or at once if urgent
- AutomatedRepair the running releaseBounded, within the agreed policy
- AutomatedRoll out through the same gateThe gate every release crosses
- AutomatedProtected hereVerified in serviceChecked on the live booking path
- AutomatedCarry the fix back to developmentOwned; a person steps in on a conflict
UnchangedCI keeps its source, secret and dependency checks before release.
| Dev loop | Production-led | |
|---|---|---|
| 4 | Alerts to triage | 1 |
| 6 | Hands-on steps before production is protected | 0 |
| Yes | Waits for a release window | No |
| 8 of 8 | Production protected at step | 4 of 5 |
Same advisory: protected at step four, or at step eight.Production-led raises one finding with one owner, repairs the release that is running and carries the fix home after. The dev loop takes the fix down the road built for features: four alerts and six hands-on steps, then the release window.
Explanation and primary sources
One advisory for the HTTP parser in the booking API travels through two flows at the same pace. Through the dev loop, scanners on four environments raise four alerts (three on servers no customer touches), and the fix passes six hands-on steps, including review, staging sign-off and a change approval that waits for the release window: production is protected at step 8 of 8. Production-led sweeps what is live, raises one finding with one owner, repairs the running release within agreed policy and verifies it in service: production is protected at step 4 of 5, with the fix carried back to development afterwards while CI keeps its pre-release checks.
- DORA: streamlining change approval
- Amazon Inspector: rescanning images for new vulnerabilities
Repeated read-only scans don't cause drift between environments. Competing patch writers do. Give each finding one owner and keep the evidence for every place it runs.
None of the parts is new
AWS Lambda and Cloud Run already patch managed runtimes without a feature release. Ubuntu applies security updates daily. Git's own workflow merges fixes upward from the oldest supported branch. Flux commits image updates back to Git.
What is usually missing is the agreement between them: who maintains the live service, what automation may change, how a repair is proven and what stops the next release undoing it. We are naming that combination, not claiming its parts.
Where we stand
We run our own fleet of apps on the platform we offer to clients, so we checked it against these rules. A daily dependency audit runs and an outside monitor checks every app every few minutes. Applying a patch is still a person's call, and the carry-back and release gate are what we are building next, on our own apps first. That work is part of running a client's app on Orchestra OS.
Start with one service
Choose one live service whose deploy and recovery paths you understand. Name its maintenance owner and the repairs automation may make alone. Then rehearse the booking story somewhere safe: a repair, a conflicting branch, a feature build that would regress it and a rollback to an affected older image. When the gate catches all four, widen the authority.
Measure the time from exposure to a verified repair, the carry-back obligations still open and the regressions refused against those that escaped. Scan counts and merged pull requests describe activity; repairs that survive releases describe the outcome.
Should a security fix wait for the next feature release? Not when people depend on the app. Give production its own repair path, give every fix a way home and give the gate the final word.
Which of your apps would lose its last fix on the next deploy?