Skip to main content
Start a conversation

Project rescue tool

Urgent software rescue triage checklist

Urgent recovery starts by separating what must be contained now from what must be understood, fixed and improved over time.

When to use this

Use this when a release is blocked, a workflow is unstable or a delivery has stalled. It gives the team a shared incident and recovery picture before an intervention is promised.

  1. 01

    Describe the affected workflow

    State who is affected, what fails, when it started, the business or safety impact and the temporary manual route, if one exists.

  2. 02

    Preserve evidence

    Collect reproducible steps, logs, environment details, recent changes, error messages and relevant release records. Avoid changing production before the known facts are captured where that is safe to do.

  3. 03

    Identify immediate containment

    Decide whether to pause a release, restrict a feature, use a safe fallback, roll back, add monitoring or communicate a known limitation. Containment is not the same as root-cause remediation.

  4. 04

    Map ownership and access

    Name the technical owner, business owner, release authority, system provider and person who can grant the access needed to investigate safely.

  5. 05

    Choose a verified change path

    Before a fix, agree the environment, test approach, deployment control, rollback route and evidence that will show whether the intervention worked without creating a wider failure.

  6. 06

    Separate the next horizons

    Record the immediate containment action, the urgent fix, the reliability improvement and any longer modernisation work as distinct decisions with distinct evidence.

Applied guidance

Use the tool as a decision record, not a box-ticking exercise.

01

Stabilise before explaining

The first urgent question is whether people, data, service delivery or a critical business process need protection now. A feature pause, release hold, controlled manual route, supplier escalation or communication to affected users may be more important than an immediate code change. The team should describe the observed impact and select the least disruptive safe containment available. This is not an admission of failure; it creates the conditions to investigate without repeatedly changing a live system. Root-cause work can then proceed with clearer evidence and a lower chance of amplifying the incident.

02

Build a shared factual timeline

A recovery effort moves faster when the team separates confirmed facts from theories. Record when the issue was first seen, what changed recently, which users or environments are affected, what logs or error references exist, what has already been attempted and what effect it had. Keep the timeline visible to the technical and business owners. It prevents circular investigation and protects against a common failure mode: several people making plausible but conflicting changes without knowing which version or condition is currently live. The timeline becomes part of the release evidence for the eventual fix.

03

Narrow the investigation safely

A blocked workflow may involve application code, configuration, identity, data, a deployment, an external API or a user process. Start by identifying the smallest reproducible failing path and the system boundary at which expected behaviour diverges. Use authorised environments and preserve relevant evidence before altering it where safe. If production investigation is necessary, use a named access and change path. The objective is to reduce uncertainty, not to create an impressive list of possible causes. A short, verified hypothesis is more valuable than a broad diagnosis based on incomplete signals.

04

Verify the fix and its rollback route

Before changing production, state what should improve, how the improvement will be checked, what other workflow could be affected and how the team will retreat if the change is unsuitable. For a rapid fix, the evidence may be a focused reproduction, test result, reviewed configuration difference and a post-deployment observation period. It should still be explicit. A code change that removes one visible error can introduce a duplicate, permission problem or downstream failure. Verification has to include the affected workflow and the relevant dependency, not only a successful build or deployment message.

05

Separate rescue from modernisation

An urgent defect often reveals deeper issues such as poor observability, fragile deployment, unclear ownership or an integration contract that has drifted. Those deserve a follow-on plan, but they should not silently expand the immediate rescue scope. Record the containment, the verified fix and the reliability or modernisation opportunities as separate horizons with their own owners and evidence. This lets the organisation restore service promptly while retaining the learning needed to avoid a repeat. It also keeps any date or recovery statement honest about the work that has actually been accepted.

06

Communicate the operational state

During an urgent recovery, different audiences need different facts: affected users need to know the current workaround and when to seek help; sponsors need the impact and next decision; technical responders need the evidence and change boundary. Use clear language about what is known, what is being investigated and when the next update will be given. Avoid announcing a root cause or recovery date before it is supported. Good communication does not replace technical investigation, but it reduces duplicate reports, protects trust and gives the team the space to make a controlled change rather than respond to pressure with speculation.

Working prompts to adapt

Triage statement

[Workflow] is failing for [affected users] since [time/change], causing [impact]. The current safe workaround is [workaround/none].

Evidence needed

Before a code or configuration change we need [reproduction], [logs], [environment access], [release authority] and [rollback approach].

Recovery decision

The immediate action is [containment]. The condition for an urgent fix is [verification]. Longer-term improvement is owned by [role].

Related routes

Continue the delivery decision

01Software project rescue02Cloud platform and DevSecOps capability03Quality engineering and release assurance

Primary guidance