CPS 230 Scenario Testing: A Severe-But-Plausible Disruption Library for APRA Entities

An annual tabletop with a facilitator and a slide deck is not scenario testing under CPS 230. It is a meeting about scenario testing. The standard wants something you can run again, and show the result of, and that is a different piece of engineering.

CPS 230 has been in force since 1 July 2025, and most APRA-regulated entities I speak to have done the visible work: they have named their critical operations, set tolerance levels, and held a workshop. What far fewer have is a scenario testing capability that survives the workshop being over. The recurring gap is not the absence of testing. It is the absence of anything durable left behind by the testing. When APRA asks how you know your retail payments operation can recover within its tolerance, the honest answer for many entities is “we discussed it once and felt reassured.” That is an assertion, not evidence, and CPS 230 is a standard that asks you to prove the operation, not describe the policy.

What the standard actually asks of scenario testing

The scenario testing obligation sits in the business continuity part of CPS 230. Paragraph 43 requires an entity to conduct an annual business continuity exercise that tests the effectiveness of its business continuity plan, including the ability to maintain critical operations within tolerance levels across a range of severe but plausible scenarios. Paragraph 44 is the part firms underweight: those scenarios must include disruptions to services provided by material service providers, and situations where contingency arrangements are actually invoked. APRA also reserves the ability to direct an entity, or a whole industry, to test against a specific scenario.

Read that as an engineer rather than a lawyer and three requirements fall out. The test must be tied to defined tolerance levels, so a scenario without a pass/fail threshold is not testing anything. It must be plural — a range, not one favoured scenario. And it must reach into your supply chain, because the disruptions most likely to breach a payments or claims operation now originate at a provider you do not control. “Severe but plausible” is doing quiet work here: severe enough to stress the operation past its comfortable state, plausible enough that the board cannot wave it away as science fiction.

Why the one-off tabletop fails the test

A tabletop exercise is a useful thing. It builds muscle memory, surfaces disagreement, and gets the right people in a room. What it does not do is produce a repeatable, comparable record. Run the same tabletop twice and you get two different conversations, shaped by who attended and how the day was going. There is no baseline, so there is no way to show whether last year’s remediation actually moved a recovery time, and no way to demonstrate a trend to a supervisor who asks the obvious follow-up: is this getting better or worse.

The deeper problem is evidential. CPS 230 is, in practice, an evidence standard. The distance between demonstrated resilience and asserted resilience is exactly the distance between a recorded outcome and a remembered conversation. A tabletop leaves you a set of minutes and an action list. It does not leave you a measured recovery time you can assert against a tolerance, which is the artefact that actually answers the regulator’s question. The fix is not to run more tabletops. It is to treat the scenario set as a maintained asset with recorded results.

Treat the scenario set as a catalogue, not an event

The move that changes the character of the programme is to hold scenarios as data. Each scenario is a definition — a critical operation, a trigger, the material service provider involved if any, the board-approved tolerance it is testing against, and the measured outcome of the last run. Held that way, the catalogue becomes something you version, review, and extend as the business changes, rather than something you rebuild from memory each July. It also makes the pass/fail judgement mechanical rather than rhetorical: a recovery time either sits inside the tolerance or it does not.

Here is the shape of a minimal scenario-as-code catalogue for a single critical operation. It defines the tolerance once, holds each scenario’s measured recovery figures, and asserts them against that tolerance so a breach is a recorded, unambiguous result rather than a matter of interpretation.

# cps230_scenarios.py
# Severe-but-plausible disruption library for one CPS 230 critical operation.
# Run this after each exercise; a FAIL is a recorded tolerance breach to be
# escalated, not an error to be silenced.

from dataclasses import dataclass

@dataclass(frozen=True)
class Tolerance:
    max_disruption_hours: float      # board-approved disruption tolerance
    max_data_loss_minutes: float     # recovery point objective

@dataclass
class Scenario:
    ref: str
    name: str
    service_provider: str            # material service provider, or "none"
    last_run: str                    # ISO date of the exercise
    measured_recovery_hours: float
    measured_data_loss_minutes: float

    def evaluate(self, t: Tolerance) -> str:
        breaches = []
        if self.measured_recovery_hours > t.max_disruption_hours:
            breaches.append("disruption")
        if self.measured_data_loss_minutes > t.max_data_loss_minutes:
            breaches.append("data-loss")
        return "PASS" if not breaches else "FAIL:" + ",".join(breaches)

# Tolerance is set and approved by the board, not by the engineer.
RETAIL_PAYMENTS = Tolerance(max_disruption_hours=4.0, max_data_loss_minutes=15.0)

CATALOGUE = [
    Scenario("SBP-2026-011", "Primary data centre loss", "none",
             "2026-05-14", 3.2, 9.0),
    Scenario("SBP-2026-012", "Card processor API outage", "acme-card-processing",
             "2026-06-02", 6.5, 4.0),
]

if __name__ == "__main__":
    for s in CATALOGUE:
        print(f"{s.ref} {s.last_run} {s.evaluate(RETAIL_PAYMENTS)}")
    # SBP-2026-011 2026-05-14 PASS
    # SBP-2026-012 2026-06-02 FAIL:disruption

The point is not the language or the tooling. It is that the catalogue makes three things true at once: the scenario is defined precisely enough to run again, the outcome is measured against a tolerance the board owns, and the FAIL is loud. The second scenario recovered the card processor in six and a half hours against a four-hour tolerance — that is a breach, recorded with a date and a reference, ready to become a remediation item you can later show you closed.

Constructing a scenario that genuinely stresses the operation

A catalogue is only as good as the scenarios in it, and the common failure is comfort. Firms write scenarios their existing arrangements already handle, run them, pass, and file the result. That is testing the plan against its own assumptions. A scenario earns its place when it targets a specific critical operation, names the tolerance it threatens, and describes a disruption severe enough to have a real chance of breaching that tolerance. Start from the operation and its tolerance — the work of defining critical operations and tolerance levels with evidence is the foundation the whole scenario library sits on — then ask what would credibly push recovery past the line.

The most productive scenarios in my experience are the supply-chain ones paragraph 44 insists on: the material service provider whose outage you have never actually simulated, the concentration you discover when two critical operations turn out to depend on the same provider, the contingency arrangement that exists on paper but has never been invoked under load. For firms that want to move beyond desk-based scenarios to actually injecting the fault and measuring the response, fault injection against defined impact tolerances is the same discipline pushed into the live estate — with the same non-negotiable requirement that the tolerance is fixed before the test runs.

Recording outcomes so the testing is evidenced

The recorded outcome is the deliverable, not a by-product. For each run you want the date, the scenario reference, the measured recovery time and data loss, the pass or fail against tolerance, and — where it failed — the remediation raised and its status. Held in the catalogue, that record answers the questions APRA actually asks: what have you tested, when, against which tolerances, what broke, and what did you do about it. It also lets you demonstrate a programme rather than an event, because you can show the same scenario improving across successive runs.

This is also what makes the annual obligation cheaper to meet, not more expensive. A maintained catalogue means next year’s exercise starts from last year’s definitions and last year’s numbers, so the marginal cost is running the scenarios and updating the results, not reconstructing the whole thing. The entities that treat scenario testing as an annual project keep paying the full price every year and keep having nothing durable to show for it.

The regulator will not be impressed that you held a workshop. It will ask to see what the workshop produced. If the answer is a maintained library of scenarios with dated, measured outcomes against board-approved tolerances, you are demonstrating resilience. If the answer is a memory of a good discussion, you are asserting it — and CPS 230 was written precisely to stop firms getting away with the second one.