Test a maintenance connector by interrupting it at the moments when a request could be lost, repeated or misunderstood. Check the request actually stored in the maintenance system and the connector's matching record. A successful banner, a timeout or an empty search result cannot alone establish what happened.
The useful acceptance question is whether one eligible condition leads to one traceable work request despite repeat delivery and uncertain responses. The answer depends on the target's documented behavior and observed test evidence, not simply on adding retry logic.
Define the stored outcome before testing failures
A CMMS is a computerized maintenance management system. Use a nonproduction instance with isolated test records, or a simulator whose limitations are documented. Confirm that notifications, dispatches and integrations cannot reach real crews or equipment. This article proposes acceptance tests; it does not report an executed connector certification.
Hypothetical worked scenario
A synthetic condition, C09-001, requests inspection of fictional sorting-line drive SD-09. W09-001 is a placeholder for the target-assigned request ID, not a caller-chosen CMMS number. All identifiers and events below are invented. The connector owner runs the exercise; a maintenance-system reviewer checks the target records and unresolved outcomes.
- Business condition
- C09-001, already eligible under the test rule.
- Target outcome
- One stored inspection request, W09-001, linked to that condition.
- Evidence
- Target record, audit history and connector reconciliation ledger.
- Authority boundary
- A request does not authorize repair or physical control.
The ledger is a durable record of submissions and reconciliation. Keep its state separate from the work request's operational status. “Submission confirmed” means the request exists, not that maintenance accepted the job or completed the work.
Keep condition identity stable across deliveries
Give the business condition an identity that survives delivery attempts, process restarts and concurrent workers. A new message ID can identify a new delivery without creating a new condition. For this exercise, the submission key is T09:C09-001, including the fictional tenant scope T09.
| Delivery | Condition and submission key | Expected target link |
|---|---|---|
| D09-A, first delivery | C09-001; T09:C09-001. | W09-001. |
| D09-B, repeated delivery | C09-001; same key. | Same request. |
| D09-C, concurrent retry | C09-001; same key. | Same request. |
Record the agreed create payload, the request fields being sent, with that key. A changed asset or request purpose must not silently reuse an old create identity. Test that incompatible reuse is rejected or held for review under the target contract. Treat later descriptive updates as a separate operation with its own rules.
Repeated readings do not automatically define separate maintenance conditions. The business rule must say when one instance begins and ends. Otherwise, a connector can appear technically consistent while creating a fresh request for every delivery.
In a second isolated test tenant, T09B, reuse condition ID C09-001 with key T09B:C09-001. Verify separate target records, tenant-scoped lookups and ledger links. Neither scope may retrieve or attach the other's request.
Verify the contract where creation actually happens
Idempotency means repeating the operation has the same intended target effect as one request. Ask what the target actually guarantees: accepted key scope, payload comparison, concurrent calls, result lookup and retention. A header named “idempotency key” is not evidence that the server implements those guarantees.
AWS's retries guidance describes caller-provided request identifiers and recording the token with the mutation as an atomic operation. It also discusses late requests, retention and rejecting changed parameters under the same identifier. Those are contract questions here, not assumed CMMS capabilities.
- Scope: confirm whose key identifies the operation.
- Commit: establish how one key can create only one request.
- Recover: establish how an uncertain result is resolved.
- Expire: define what happens after key knowledge is no longer retained.
Atomic means the relevant changes succeed together or not at all. Two workers can both see an empty search result and both create a request. A local lock or ledger reservation alone does not atomically cover an external target write; a crash can separate them.
If the target lacks an enforceable create identity, document that limitation. Do not claim duplicate prevention from search-before-create alone. Uncertain submissions may need to stop for reconciliation instead of automatically issuing another create.
Include retries hidden inside client libraries or intermediaries in that review. Pausing a visible worker is insufficient if another component can still submit the operation. The test needs to expose all paths that can produce a create.
Lose the response after the target accepts the request
The decisive test lets the target store W09-001, then discards its response before the connector receives it. The connector sees a timeout; the target already has the request. This is different from rejecting the request before any write occurred.
| Milestone | Target truth | Connector ledger |
|---|---|---|
| Submission starts | No request yet in this test. | T09:C09-001 reserved; attempt recorded. |
| Target commits; response lost | W09-001 stored with correlation. | Outcome unknown, awaiting reconciliation. |
| Authoritative lookup succeeds | Same W09-001 located. | Confirmed link to W09-001 and supporting evidence. |
RFC 9110, section 9.2.2 defines idempotent request effects and cautions against automatically retrying non-idempotent operations without a basis for doing so. HTTP method names alone do not establish this connector's business guarantees.
Resolve uncertainty through the supported key lookup or repeat the same operation only under a verified target idempotency contract. Inspect the stored correlation, the reference joining W09-001 to T09:C09-001, and payload, then link the ledger. If evidence is unavailable or conflicting, keep the outcome unknown and assign review; do not label it failed and start over with a new key.
Inject faults and inspect the target after each run
Use controlled test fixtures with known starting state. Preserve both sides' evidence before resetting the next run. A simulator tests behavior against its modeled contract; repeat relevant tests against the actual nonproduction target before asserting compatibility.
| Injected fault | Required behavior | Evidence to inspect |
|---|---|---|
| Repeat delivery; overlapping workers | One target request for C09-001. | Stored request count and matching key. |
| Same condition ID in another tenant | Separate scoped request and ledger link. | No cross-tenant retrieval or association. |
| Hold first create; deliver retry first | One request after both attempts settle. | Arrival order and target commit history. |
| Accept create; drop response | Reconcile to W09-001 without an extra create. | Target commit and ledger resolution. |
| Crash after target commit | Restart preserves identity and resolves uncertainty. | Durable attempt record and target lookup. |
| Delay or reorder status events | Older state cannot overwrite verified newer state. | Target history and applied version evidence. |
| Return stale empty search | No unprotected create based on absence. | Authoritative lookup or explicit unknown state. |
| Deliver after key retention expires | No assumed duplicate protection. | Documented expiry behavior and reconciliation hold. |
| Reuse key with conflicting asset | Reject or hold incompatible intent. | Conflict record; original target request unchanged. |
| Recovery overlaps automated retry | No competing manual create. | Controlled ownership and target audit history. |
A cached “not found” result may describe an older view. Test that case deliberately. If the target cannot provide a sufficiently authoritative result, uncertainty remains. Similarly, a key retained for less time than possible retries leaves late arrivals outside the contract; do not invent a universal retention period.
Count correlated requests rather than only successful responses. A request stored without its ledger link is unresolved. A ledger marked confirmed with no matching target request also needs investigation.
A held case can pass its safe uncertainty-handling test while request delivery remains unresolved. Report those outcomes separately. A hold is neither a lost request proven absent nor a successful delivery.
Separate an old repeat from a genuine recurrence
Closing W09-001 does not make a delayed delivery of C09-001 a new condition. Preserve the original association. An old response describing an earlier state must not reopen the request or replace a verified current state.
- Original instance: C09-001 links to W09-001.
- Closure verified: the target records W09-001 as closed.
- Old repeat: delayed C09-001 still links to W09-001.
- Qualifying recurrence: the business rule establishes C09-002.
- New intent: T09:C09-002 may create W09-002 under the same tested contract.
Test delayed and duplicate status updates as well as create messages. Use supported revisions or another documented ordering mechanism. If updates cannot be ordered reliably, reconcile current state rather than selecting whichever message arrived last.
Closure in the target and clearance of the underlying condition are separate facts. The test rule must define recurrence explicitly; the connector must not manufacture clearance from ticket closure. Keep ambiguous recurrence assigned to the maintenance reviewer.
Make manual recovery part of acceptance
Manual recovery can create the duplicate that automated retries avoided. Before a reviewer acts, prevent competing submissions for the affected identity and account for attempts already in flight. Clicking “retry” while another worker still owns a create is not reconciliation.
| Review step | Required record |
|---|---|
| Establish ownership | Who holds recovery and which attempts remain in flight. |
| Inspect target truth | Matching requests, correlation, payload and current state. |
| Resolve or retain unknown | Existing request linked, conflicting evidence held, or justified next action. |
| Release the hold | Reviewer decision, ledger update and permission to resume. |
Retain the test inputs, injected fault, target records and ledger transitions in the acceptance record. Conventional software enforces tested constraints; AI may summarize discrepancies with references. Neither a summary nor a green banner proves completion.
A reviewer may approve a new submission only after accounting for earlier attempts and establishing that creation is safe under the target contract and evidence. Otherwise retain the hold. Manual action must not bypass an unresolved commit by inventing a fresh key.
The intended benefit is dependable maintenance ownership without hidden lost requests or duplicate work. Passing these tests would support an acceptance decision for the tested contract, configuration and cases, not prove all behavior or end-to-end exactly-once delivery. Repair authorization, isolation and equipment commands remain outside the connector.