Failures & retries

Durable execution does not make every error retryable. Classify transport delivery, activity attempts, deterministic replay, and terminal workflow outcomes separately.

Know which layer failed

Make activity retries safe

Pass activity policy in the command options supported by your runtime, and make the downstream action idempotent:

$receipt = $context->activity(
    'payments.capture',
    [$orderId, $amount],
    [
        'start_to_close_timeout_seconds' => 30,
        'retry_policy' => [
            'maximum_attempts' => 5,
            'initial_interval_seconds' => 1,
            'maximum_interval_seconds' => 20,
        ],
    ],
);

Use the order or activity identity as the payment provider's idempotency key. A retry policy cannot repair a non-idempotent downstream API.

Heartbeat long attempts

foreach ($batches as $index => $batch) {
    importBatch($batch);
    $context->heartbeat(['completed_batches' => $index + 1]);
}

Heartbeat details make slow work observable and provide a cancellation checkpoint. Catch ActivityCancelled only to release local resources, then rethrow it so the runtime records cancellation correctly.

Preserve terminal distinctions at the client

try {
    $result = $handle->result(timeoutSeconds: 60);
} catch (WorkflowTimedOut $failure) {
    reportDeadline($failure);
} catch (WorkflowFailed $failure) {
    reportBusinessFailure($failure);
}

A client-side wait timeout is not proof that the workflow failed; describe the handle or wait again. A workflow terminal timeout is durable Server state.

Treat nondeterminism as a deployment incident

Do not paper over NonDeterministicWorkflow with process retries. Compare the deployed build with the history's worker routing, restore the old command sequence, and roll forward with a replay-compatible change. Keep a replay corpus for long-lived workflow histories in testing.